Jeffrey L. McKinstry

Texture discrimination by an autonomous mobile brain-based device with whiskers

Whiskers are widely used by many animal species for navigation and texture discrimination. This paper describes Darwin IX, a mobile physical device equipped with artificial whiskers, the behavior of which is controlled by a neural simulation based on the rat somatosensory system. During its autonomous behavior, Darwin IX is able to discriminate among textures in its environment and learns to avoid textures that are paired with aversive events.

Active sensing of visual and tactile stimuli by brain-based devices

We describe the construction and performance of `brain-based devices? (BBDs), physical devices whose behaviour is controlled by simulated nervous systems modelled on vertebrate neuroanatomy and neurophysiology, that carry out perceptual categorization and selective conditioning to visual and textural stimuli. BBDs take input from the environment through on-board sensors including cameras, microphones and artificial whiskers, and take action based on experiential learning. BBDs have a large-scale neural simulation, a phenotype, a body plan, and the means to learn through autonomous exploration. Key neural mechanisms in the present BBDs include synaptic plasticity, reward or value systems, reentrant connectivity, the dynamic synchronization of neuronal activity, and neuronal units with spatiotemporal response properties. With our BBDs, as with animals, it is the interaction of these neural mechanisms with the sensorimotor correlations generated by active sensing and self motion that is responsible for adaptive behaviour. BBDs permit analysis of activity at all levels of the nervous system during behaviour, and as such they provide a rich source of heuristics for generating hypotheses regarding brain function. Moreover, by taking inspiration from systems neuroscience, BBDs provide a novel architecture for the design of neuromorphic systems.

Temporal sequence learning in winner-take-all networks of spiking neurons demonstrated in a brain-based device

Animal behavior often involves a temporally ordered sequence of actions learned from experience. Here we describe simulations of interconnected networks of spiking neurons that learn to generate patterns of activity in correct temporal order. The simulation consists of large-scale networks of thousands of excitatory and inhibitory neurons that exhibit short-term synaptic plasticity and spike-timing dependent synaptic plasticity. The neural architecture within each area is arranged to evoke winner-take-all (WTA) patterns of neural activity that persist for tens of milliseconds. In order to generate and switch between consecutive firing patterns in correct temporal order, a reentrant exchange of signals between these areas was necessary. To demonstrate the capacity of this arrangement, we used the simulation to train a brain-based device responding to visual input by autonomously generating temporal sequences of motor actions.

Charting out the octopus connectome at submicron resolution using the knife-edge scanning microscope

The common octopus, or Octopus vulgaris, has the largest nervous system of any invertebrate, and has been shown to possess learning and memory capabilities that in many ways rival those of some vertebrates [1]. Nevertheless, the neural architecture of this cephalopod mollusk differs markedly from that of any vertebrate. Investigating the differences and similarities between the neural architecture—or connectome—of the octopus and mammals, such as the mouse, may lead to deep insights into the computational principles underlying animal cognition. The octopus brain provides some unique advantages for anatomical research, since its axons are generally thick and unmyelinated, allowing traditional staining methods, such as Golgi, to be used effectively. With this in mind, we first imaged the brain using the Knife-Edge Scanning Microscope [2], a custom serial sectioning microscope that can image large blocks of tissue (1 cm3) at sub-micrometer resolution. We imaged large portions of the octopus subesophageal mass (SUB) and the optic lobe (OL) which were stained using Golgi. In order to extract the geometry of the neuronal morphology, we used our Maximum Intensity Projection (MIP)-based tracing algorithm [3]. The imaging results are shown in Figure 1(a-d), and tracing results are shown in 1(e). Although quite preliminary, to our knowledge this is the first time large volumes of the octopus brain have been imaged at sub-micrometer resolution, allowing us to resolve many of the processes that make up the neural network. We expect that this pilot study and the more detailed investigations to follow will allow fruitful comparisons of the neural circuitries of individual octopuses with different ecological life histories, as well as of animals that have been exposed to a variety of neurodegenerative insults. Moreover, such explorations will engender a greater understanding of how functional neural architecture is altered by learning in invertebrates such as the octopus and vertebrates such as the mouse. In sum, this approach should contribute greatly to our understanding of the computational architecture of invertebrates and ultimately provide insights into the differences between invertebrate and vertebrate cognitive capabilities. Figure 1 Octopus subesophageal mass (SUB) and optic lobe (OL) imaged with the KESM (a–d), and tracing results (e). Scale (block width): (a) 1.44 mm, (b) 0.72 mm, (c) 1.44 mm, (d-f) 76.8 μm. Voxel resolution: 0.6 μm x 0.7 μm x 1.0 ...

Neuromorphic and Brain-Based Robots: The case for using brain-based devices to study consciousness

Introduction Within the past few decades, the nature of consciousness has become a central issue in neuroscience, and it is increasingly the focus of both theoretical and empirical work. Studying consciousness is vital to developing an understanding of human perception and behavior, of our relationships with one another, and of our relationships with other potentially conscious animals. Although the study of consciousness through the construction of artificial models is a recent innovation, the advantages of such an approach are clear. First, models allow us to investigate consciousness in ways that are currently not feasible using human subjects or other animals. Second, an artifact that exhibits the necessary and sufficient properties of consciousness may conceivably be the forerunner of a new and very useful class of neuromorphic robots. A model of consciousness must take into account current theories of its biological bases. Although the field of artificial consciousness is a new one, it is striking how little attention has been given to modeling mechanisms. Instead, great – and perhaps undue – emphasis has been placed on purely phenomenological models. Many of these models are strongly reductionist in aim and fail to specify neural mechanisms.

Spontaneous emergence of fast attractor dynamics in a model of developing primary visual cortex

Recent evidence suggests that neurons in primary sensory cortex arrange into competitive groups, representing stimuli by their joint activity rather than as independent feature analysers. A possible explanation for these results is that sensory cortex implements attractor dynamics, although this proposal remains controversial. Here we report that fast attractor dynamics emerge naturally in a computational model of a patch of primary visual cortex endowed with realistic plasticity (at both feedforward and lateral synapses) and mutual inhibition. When exposed to natural images (but not random pixels), the model spontaneously arranges into competitive groups of reciprocally connected, similarly tuned neurons, while developing realistic, orientation-selective receptive fields. Importantly, the same groups are observed in both stimulus-evoked and spontaneous (stimulus-absent) activity. The resulting network is inhibition-stabilized and exhibits fast, non-persistent attractor dynamics. Our results suggest that realistic plasticity, mutual inhibition and natural stimuli are jointly necessary and sufficient to generate attractor dynamics in primary sensory cortex.

Imagery May Arise from Associations Formed through Sensory Experience: A Network of Spiking Neurons Controlling a Robot Learns Visual Sequences in Order to Perform a Mental Rotation Task

Mental imagery occurs “when a representation of the type created during the initial phases of perception is present but the stimulus is not actually being perceived.” How does the capability to perform mental imagery arise? Extending the idea that imagery arises from learned associations, we propose that mental rotation, a specific form of imagery, could arise through the mechanism of sequence learning–that is, by learning to regenerate the sequence of mental images perceived while passively observing a rotating object. To demonstrate the feasibility of this proposal, we constructed a simulated nervous system and embedded it within a behaving humanoid robot. By observing a rotating object, the system learns the sequence of neural activity patterns generated by the visual system in response to the object. After learning, it can internally regenerate a similar sequence of neural activations upon briefly viewing the static object. This system learns to perform a mental rotation task in which the subject must determine whether two objects are identical despite differences in orientation. As with human subjects, the time taken to respond is proportional to the angular difference between the two stimuli. Moreover, as reported in humans, the system fills in intermediate angles during the task, and this putative mental rotation activates the same pathways that are activated when the system views physical rotation. This work supports the proposal that mental rotation arises through sequence learning and the idea that mental imagery aids perception through learned associations, and suggests testable predictions for biological experiments.

Temporal sequence learning in reentrantly coupled winner-take-all networks of spiking neurons

Patterns of activity in brains are commonly composed of temporal sequences of periods with steady-state firing rates lasting several hundred milliseconds separated by sharp transitions during movement [1], perception [2], and remembering [3]. Although network models involving mean-firing-rate neurons have been used to generate sequential neural activity [1], spiking networks with such capability require further exploration. We describe how large-scale Winner-Take-All (WTA) spiking networks can be coupled together and trained to generate such sequential neural activity. These networks are composed of conductance-based excitatory and inhibitory spiking neurons [4]. Model synapses were subject to short-term synaptic plasticity and spike-timing dependent plasticity (STDP). Each network was a Center-Annular-Surround (CAS) network, a variant of center-surround networks that we have found to effectively generate WTA dynamics in large-scale networks of spiking neurons. Two CAS networks were coupled together reentrantly to form a network capable of sequence learning and recall. We found that networks of this sort can be trained to respond to a sequence of sensory cues by generating temporally ordered patterns of neuronal activity. The patterns consist of brief steady states separated by sharp transitions that resemble those observed experimentally. After training, synaptic changes resulting from STDP acting on connections between the coupled networks formed a link between temporally adjacent patterns of neural activity within the sequence. Figure ​Figure11 reflects an analysis of spiking data from this simulated network trained with a repeating sequence of eight cues. Each row in the figure plots the match score over time to one of the eight patterns of neural activity corresponding to one of the cues. White is a perfect match, while black indicates a complete mismatch. The last training repeat is from t = 24 to t = 32. Subsequently, external cues were removed and network activity continued, cycling through all eight patterns until another input cue was presented. Then the activity pattern changed to the activity pattern appropriate to the cue presented (see at t = 37), continuing the sequence when the cue was removed. The system was robust with respect to various initial conditions. Figure 1 After training, a large-scale network of approximately 4,000 spiking neurons transitions between activity patterns, in the absence of external cues, reflecting the learned sequence We also used the present model to control specific motor sequences in a robotic device. The population activity pattern in this modeled nervous system has similarities to that observed in primate frontal cortex during multi-segmented limb movements.