Jeffrey L. Krichmar

2009 Special Issue: A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors

Neural network simulators that take into account the spiking behavior of neurons are useful for studying brain mechanisms and for various neural engineering applications. Spiking Neural Network (SNN) simulators have been traditionally simulated on large-scale clusters, super-computers, or on dedicated hardware architectures. Alternatively, Compute Unified Device Architecture (CUDA) Graphics Processing Units (GPUs) can provide a low-cost, programmable, and high-performance computing platform for simulation of SNNs. In this paper we demonstrate an efficient, biologically realistic, large-scale SNN simulator that runs on a single GPU. The SNN model includes Izhikevich spiking neurons, detailed models of synaptic plasticity and variable axonal delay. We allow user-defined configuration of the GPU-SNN model by means of a high-level programming interface written in C++ but similar to the PyNN programming interface specification. PyNN is a common programming interface developed by the neuronal simulation community to allow a single script to run on various simulators. The GPU implementation (on NVIDIA GTX-280 with 1 GB of memory) is up to 26 times faster than a CPU version for the simulation of 100K neurons with 50 Million synaptic connections, firing at an average rate of 7 Hz. For simulation of 10 Million synaptic connections and 100K neurons, the GPU SNN model is only 1.5 times slower than real-time. Further, we present a collection of new techniques related to parallelism extraction, mapping of irregular communication, and network representation for effective simulation of SNNs on GPUs. The fidelity of the simulation results was validated on CPU simulations using firing rate, synaptic weight distribution, and inter-spike interval analysis. Our simulator is publicly available to the modeling community so that researchers will have easy access to large-scale SNN simulations.

Effects of dendritic morphology on CA3 pyramidal cell electrophysiology: a simulation study

We investigated the effect of morphological differences on neuronal firing behavior within the hippocampal CA3 pyramidal cell family by using three-dimensional reconstructions of dendritic morphology in computational simulations of electrophysiology. In this paper, we report for the first time that differences in dendritic structure within the same morphological class can have a dramatic influence on the firing rate and firing mode (spiking versus bursting and type of bursting). Our method consisted of converting morphological measurements from three-dimensional neuroanatomical data of CA3 pyramidal cells into a computational simulator format. In the simulation, active channels were distributed evenly across the cells so that the electrophysiological differences observed in the neurons would only be due to morphological differences. We found that differences in the size of the dendritic tree of CA3 pyramidal cells had a significant qualitative and quantitative effect on the electrophysiological response. Cells with larger dendritic trees: (1) had a lower burst rate, but a higher spike rate within a burst, (2) had higher thresholds for transitions from quiescent to bursting and from bursting to regular spiking and (3) tended to burst with a plateau. Dendritic tree size alone did not account for all the differences in electrophysiological responses. Differences in apical branching, such as the distribution of branch points and terminations per branch order, appear to effect the duration of a burst. These results highlight the importance of considering the contribution of morphology in electrophysiological and simulation studies.

The Neuromodulatory System: A Framework for Survival and Adaptive Behavior in a Challenging World

Biological organisms have the ability to respond quickly to an ever-changing world. Because this adaptability is so critical for survival, all vertebrates have sub-cortical structures, which comprise the neuromodulatory systems, to regulate fundamental behavior and drive decision making in response to environmental events. In the vertebrate, there are separate neuromodulators that respond to threats, reward anticipation, novelty, and attentional effort. However, each of these neuromodulatory systems has a similar effect, that is, to cause an organism to be decisive when environmental conditions call for such actions, and allow the organism to be more exploratory when there are no pressing events. In this article, it is proposed that principles of the neuromodulatory system could provide a framework for controlling artificial agents that may improve current artificial agent behavior. These agents would operate autonomously, effectively explore their environment, and be decisive when environmental conditions call for action.

Spatial navigation and causal analysis in a brain-based device modeling cortical-hippocampal interactions.

We describe Darwin X, a physical device that interacts with a real environment, whose behavior is guided by a simulated nervous system incorporating aspects of the detailed anatomy and physiology of the hippocampus and its surrounding regions. This brain-based device integrates cues from its environment and solves a spatial memory task. The responses of simulated neuronal units in the hippocampal areas during its exploratory behavior are comparable to place cells in the rodent hippocampus and emerged by associating sensory cues during exploration. To identify different functional hippocampal pathways and their influence on behavior, we employed a time series analysis that distinguishes causal interactions within and between simulated hippocampal and neocortical regions while the device is engaged in a spatial memory task. Our analysis identified different functional pathways within the neural simulation and prompts novel predictions about the influence of the perforant path, the trisynaptic loop and hippocampal-cortical interactions on place cell activity and behavior during navigation. Moreover, this causal time series analysis may be useful in analyzing networks in general.

Brain-Based Devices for the Study of Nervous Systems and the Development of Intelligent Machines

The simultaneous study of brain function at all levels of organization is difficult to undertake with current experimental tools. Present day electrophysiology only allows the recording of at most hundreds of neurons while an animal is performing a behavioral task. Because of this limitation and the sheer complexity of the nervous system, computational modeling has become essential in developing theories of brain function. Accordingly, our group has constructed a series of brain-based devices (BBDs), that is, physical devices with simulated nervous systems that guide behavior, to serve as a heuristic for testing theories of brain function. Unlike animal models, BBDs permit analysis of activity at all levels of the nervous system as the device behaves in its environment. Although the principal focus of developing BBDs has been to test theories of brain function, this type of modeling may also provide a basis for robotic design and practical applications.

Efficient simulation of large-scale Spiking Neural Networks using CUDA graphics processors

Neural network simulators that take into account the spiking behavior of neurons are useful for studying brain mechanisms and for engineering applications. Spiking Neural Network (SNN) simulators have been traditionally simulated on large-scale clusters, super-computers, or on dedicated hardware architectures. Alternatively, Graphics Processing Units (GPUs) can provide a low-cost, programmable, and high-performance computing platform for simulation of SNNs. In this paper we demonstrate an efficient, Izhikevich neuron based large-scale SNN simulator that runs on a single GPU. The GPU-SNN model (running on an NVIDIA GTX-280 with 1GB of memory), is up to 26 times faster than a CPU version for the simulation of 100K neurons with 50 Million synaptic connections, firing at an average rate of 7Hz. For simulation of 100K neurons with 10 Million synaptic connections, the GPU-SNN model is only 1.5 times slower than real-time. Further, we present a collection of new techniques related to parallelism extraction, mapping of irregular communication, and compact network representation for effective simulation of SNNs on GPUs. The fidelity of the simulation results were validated against CPU simulations using firing rate, synaptic weight distribution, and inter-spike interval analysis. We intend to make our simulator available to the modeling community so that researchers will have easy access to large-scale SNN simulations.

An Efficient Simulation Environment for Modeling Large-Scale Cortical Processing

We have developed a spiking neural network simulator, which is both easy to use and computationally efficient, for the generation of large-scale computational neuroscience models. The simulator implements current or conductance based Izhikevich neuron networks, having spike-timing dependent plasticity and short-term plasticity. It uses a standard network construction interface. The simulator allows for execution on either GPUs or CPUs. The simulator, which is written in C/C++, allows for both fine grain and coarse grain specificity of a host of parameters. We demonstrate the ease of use and computational efficiency of this model by implementing a large-scale model of cortical areas V1, V4, and area MT. The complete model, which has 138,240 neurons and approximately 30 million synapses, runs in real-time on an off-the-shelf GPU. The simulator source code, as well as the source code for the cortical model examples is publicly available.

Categorization and decision-making in a neurobiologically plausible spiking network using a STDP-like learning rule

Understanding how the human brain is able to efficiently perceive and understand a visual scene is still a field of ongoing research. Although many studies have focused on the design and optimization of neural networks to solve visual recognition tasks, most of them either lack neurobiologically plausible learning rules or decision-making processes. Here we present a large-scale model of a hierarchical spiking neural network (SNN) that integrates a low-level memory encoding mechanism with a higher-level decision process to perform a visual classification task in real-time. The model consists of Izhikevich neurons and conductance-based synapses for realistic approximation of neuronal dynamics, a spike-timing-dependent plasticity (STDP) synaptic learning rule with additional synaptic dynamics for memory encoding, and an accumulator model for memory retrieval and categorization. The full network, which comprised 71,026 neurons and approximately 133 million synapses, ran in real-time on a single off-the-shelf graphics processing unit (GPU). The network was constructed on a publicly available SNN simulator that supports general-purpose neuromorphic computer chips. The network achieved 92% correct classifications on MNIST in 100 rounds of random sub-sampling, which is comparable to other SNN approaches and provides a conservative and reliable performance metric. Additionally, the model correctly predicted reaction times from psychophysical experiments. Because of the scalability of the approach and its neurobiological fidelity, the current model can be extended to an efficient neuromorphic implementation that supports more generalized object recognition and decision-making architectures found in the brain.

Retrospective and prospective responses arising in a modeled hippocampus during maze navigation by a brain-based device

Recent recordings of place field activity in rodent hippocampus have revealed correlates of current, recent past, and imminent future events in spatial memory tasks. To analyze these properties, we used a brain-based device, Darwin XI, that incorporated a detailed model of medial temporal structures shaped by experience-dependent synaptic activity. Darwin XI was tested on a plus maze in which it approached a goal arm from different start arms. In the task, a journey corresponded to the route from a particular starting point to a particular goal. During maze navigation, the device developed place-dependent responses in its simulated hippocampus. Journey-dependent place fields, whose activity differed in different journeys through the same maze arm, were found in the recordings of simulated CA1 neuronal units. We also found an approximately equal number of journey-independent place fields. The journey-dependent responses were either retrospective, where activity was present in the goal arm, or prospective, where activity was present in the start arm. Detailed analysis of network dynamics of the neural simulation during behavior revealed that many different neural pathways could stimulate any single CA1 unit. That analysis also revealed that place activity was driven more by hippocampal and entorhinal cortical influences than by sensory cortical input. Moreover, journey-dependent activity was driven more strongly by hippocampal influence than journey-independent activity.

A neurorobotic platform to test the influence of neuromodulatory signaling on anxious and curious behavior

The vertebrate neuromodulatory systems are critical for appropriate value-laden responses to environmental challenges. Whereas changes in the overall level of dopamine (DA) have an effect on the organisms reward or curiosity-seeking behavior, changes in the level of serotonin (5-HT) can affect its level of anxiety or harm aversion. Moreover, top-down signals from frontal cortex can exert cognitive control on these neuromodulatory systems. The cholinergic (ACh) and noradrenergic (NE) systems affect the ability to filter out noise and irrelevant events. We introduce a neural network for action selection that is based on these principles of neuromodulatory systems. The algorithm tested the hypothesis that high levels of serotonin lead to withdrawn behavior by suppressing DA action and that high levels of DA or low levels of 5-HT lead to curious, exploratory behavior. Furthermore, the algorithm tested the idea that top-down signals from the frontal cortex to neuromodulatory areas are critical for an organism to cope with both stressful and novel events. The neural network was implemented on an autonomous robot and tested in an open-field paradigm. The open-field test is often used to test for models anxiety or exploratory behavior in the rodent and allows for qualitative comparisons with the neurorobots behavior. The present neurorobotic experiments can lead to a better understanding of how neuromodulatory signaling affects the balance between anxious and curious behavior. Therefore, this experimental paradigm may also be informative in exploring a wide range of neurological diseases such as anxiety, autism, attention deficit disorders, and obsessive-compulsive disorders.