Desktop version

Home arrow Communication

  • Increase font
  • Decrease font

<<   CONTENTS   >>

Functional and structural neural systems

As mentioned in the previous chapter, among the different formalisms of bioinspired computing aimed at creating intelligent systems by imitating the brain, neural networks are one of the most adopted modelling paradigms. The first models of artificial neural networks, however, proved to be too simplistic and not similar to the brain (Forbes, 2004) (i.e., they were simplistic functional models).

Nevertheless, they inspired statisticians and computer scientists to develop very successful non-linear statistical models and learning algorithms that comprise an important part of machine learning[1] [2] techniques today. The recent resurgence of interest in neural networks, more commonly referred to as “deep neural networks” or “deep learning models”,11 share the same representational commitments and often even the same learning algorithms as the earlier PDP models. For example, most neural networks use some form of gradient-based learning rules (e.g., backpropagation). It has long been argued, however, that backpropagation is not biologically plausible. For example, as Francis Crick (1989) famously pointed out, backpropagation requires that information be transmitted backwards along the axon. However, this phenomenon has never been observed in natural neural architectures and, therefore, cannot be considered a realistic mechanism. This evidence has not prevented backpropagation from being put to good use in connectionist models of cognition (e.g., in the computational models of cognition proposed by Rumelhart, McClelland, and the PDP Research Group, 1986) or in building deep neural networks for AI systems. In all these cases, in fact - and in particular the case of the connectionist models of cognition - the realism of such models is based on the presence of additional constraints (if any) that come directly from the anatomo-physiological knowledge of real, biological, neural networks (a classic example confirming this fact is represented by the work of Churchland and Sejnowski [1992J, which showed how different neurophysiological hypotheses about the vestibule-eye circuit could have been modelled by different neural net models providing different results to evaluate). Such constraints, once identified and put in place, provide what we are prepared to consider plausible models for the sake of explanation.[3] We need to keep in mind, however, that every model, even the most realistic one, is so at a certain level of abstraction and simplification of reality. Therefore, in the class of “biologically inspired” modelling methods and techniques also, it is possible to individuate a continuum between functional and structural artificial nets.

As pointed out by Shayani (2013), the field of artificial neural networks (apart from the boom in deep learning technologies) is attracting new attention thanks to the development of different types of novel, bio-inspired neural models. The author, for example, points out how spiking neural neuron models and Hierarchical Temporal Memory (HTM) represent two of the most interesting novelties of the last few years in this respect. Spiking neural networks - characterized by the fact that the activation of individual neurons resembles the one in biological neurons, which communicate via discrete spikes of voltage - have shown some advantages in power consumption (since spikes can be routed like data packets) and have been proven to result in computationally more powerful networks than classical ANNs. In addition, spiking recurrent neural nets, a variant of classical spiking neural models, have been proven to be more robust to noise and more efficient than classical ANNs in processing spatiotemporal data. HTM networks, on the other hand, have received interest due to their biological inspiration, taken from the characteristics of the mammalian brain. The main idea of these networks, introduced by Hawkins, is that there is a single common structure and algorithm controlling many different functions of the neocortex (from vision to language and motor planning, etc.). Among the main biological assumptions of HTM there is the importance of the temporal persistence of the causes (objects) in such networks (Hawkins and Blakeslee, 2005), as well as the use of sparse coding and Recurrent Neural Nets for spatiotemporal pattern recognition.

An additional effort towards establishing the biological plausibility of ANNs is pursued with the so-called “evo-devo networks”, incorporating evolutionary computing algorithms[4] that allow the network to adapt to a changing environment and guarantee more robustness and optimization capabilities for specific problems. In particular, the inclusion of evolutionary heuristics in bio-plausible neurodevelopment-constrained networks leverages the emergence of features such as fault-tolerance, self-organization, regeneration, and self-repair, and meanwhile also improves the evolvability and scalability of the system (Shayani, 2013: 1). A side effect of including all such constraints in neural models, however, is that it makes it necessary to have very powerful hardware platforms able to execute the computation on such networks. This problem is nowadays well-handled thanks to the development of new technologies like Field Programmable Gate Arrays (FP- GAs) or via the exploitation of GPUs (Graphical Processing Units) that are able to alleviate the issue. Mentioning all this plethora of neural network solutions is important for the purposes of the book, since these architectures - and many others that are available on the “connectionist market” - have, as has been described, different degrees of plausibility. Therefore, the functional/structural continuum applies to this class of modelling techniques as well. In the next section, we will show how the same dichotomy also applies to the class of symbolic formalisms.

  • [1] Machine learning represents a subfield of AI focusing on the development of automatic techniques for allowing machines to learn and abstract regularities from raw data. Deep learningtechniques represent a subset of machine learning.
  • [2] Deep neural networks, used in deep learning technologies, present many intermediate layersbetween the input and output units (usually hundreds of layers), thus creating a “deep” hierarchy of connections. The key idea of deep learning is that at each layer it is possible to learnfeatures of increasing abstraction with respect to the previous layer (Goodfellow et ah, 2016).For example, in a vision setting, the lowest level learns lines and edges, the next layer maylearn corners and curves, the next layer may learn simple shapes, and so on up the hierarchy.Upper levels are supposed to then learn complex categories (cars, people, dogs) or even specificinstances (your dog, your cat, etc.).
  • [3] Michael A. Arbib has been one of the most influential figures in brain-inspired neural networkmodels. A classic reference in the field is the Handbook of Brain Theories and Neural Networks(Arbib, 2002). In a more recent memoir, Arbib illustrates the role of the classic contributionsof cybernetics in building constrained models in the context of computational neuroscience(Arbib, 2018).
  • [4] Evolutionary Computation is a subfield of AI that adopts iterative heuristic techniques and algorithms inspired by the evolutionary processes of “growth” and “selection”. Swarm intelligence,genetic programming, evolutionary programming, evolutionary strategies, and evolutionaryalgorithms represent different subsets of techniques, partially intersecting, belonging to the fieldof evolutionary computation. In particular, one of the most well-known algorithms adopted inthis field are Genetic Algorithms (GA) (Holland, 1975). Evolutionary algorithms, in general,are known to be good meta-heuristic optimization and search techniques, where there is littleto no knowledge about the search space.
<<   CONTENTS   >>

Related topics