Elisabetta Chicca

A VLSI recurrent network of integrate-and-fire neurons connected by plastic synapses with long-term memory

Electronic neuromorphic devices with on-chip, on-line learning should be able to modify quickly the synaptic couplings to acquire information about new patterns to be stored (synaptic plasticity) and, at the same time, preserve this information on very long time scales (synaptic stability). Here, we illustrate the electronic implementation of a simple solution to this stability-plasticity problem, recently proposed and studied in various contexts. It is based on the observation that reducing the analog depth of the synapses to the extreme (bistable synapses) does not necessarily disrupt the performance of the device as an associative memory, provided that 1) the number of neurons is large enough; 2) the transitions between stable synaptic states are stochastic; and 3) learning is slow. The drastic reduction of the analog depth of the synaptic variable also makes this solution appealing from the point of view of electronic implementation and offers a simple methodological alternative to the technological solution based on floating gates. We describe the full custom analog very large-scale integration (VLSI) realization of a small network of integrate-and-fire neurons connected by bistable deterministic plastic synapses which can implement the idea of stochastic learning. In the absence of stimuli, the memory is preserved indefinitely. During the stimulation the synapse undergoes quick temporary changes through the activities of the pre- and postsynaptic neurons; those changes stochastically result in a long-term modification of the synaptic efficacy. The intentionally disordered pattern of connectivity allows the system to generate a randomness suited to drive the stochastic selection mechanism. We check by a suitable stimulation protocol that the stochastic synaptic plasticity produces the expected pattern of potentiation and depression in the electronic network.

Neuromorphic electronic circuits for building autonomous cognitive systems

Several analog and digital brain-inspired electronic systems have been recently proposed as dedicated solutions for fast simulations of spiking neural networks. While these architectures are useful for exploring the computational properties of large-scale models of the nervous system, the challenge of building low-power compact physical artifacts that can behave intelligently in the real world and exhibit cognitive abilities still remains open. In this paper, we propose a set of neuromorphic engineering solutions to address this challenge. In particular, we review neuromorphic circuits for emulating neural and synaptic dynamics in real time and discuss the role of biophysically realistic temporal dynamics in hardware neural processing architectures; we review the challenges of realizing spike-based plasticity mechanisms in real physical systems and present examples of analog electronic circuits that implement them;we describe the computational properties of recurrent neural networks and show how neuromorphic winner-take-all circuits can implement working-memory and decision-making mechanisms. We validate the neuromorphic approach proposed with experimental results obtained from our own circuits and systems, and argue how the circuits and networks presented in this work represent a useful set of components for efficiently and elegantly implementing neuromorphic cognition.

A Multichip Pulse-Based Neuromorphic Infrastructure and Its Application to a Model of Orientation Selectivity

The growing interest in pulse-mode processing by neural networks is encouraging the development of hardware implementations of massively parallel networks of integrate-and-fire neurons distributed over multiple chips. Address-event representation (AER) has long been considered a convenient transmission protocol for spike based neuromorphic devices. One missing, long-needed feature of AER-based systems is the ability to acquire data from complex neuromorphic systems and to stimulate them using suitable data. We have implemented a general-purpose solution in the form of a peripheral component interconnect (PCI) board (the PCI-AER board) supported by software. We describe the main characteristics of the PCI-AER board, and of the related supporting software. To show the functionality of the PCI-AER infrastructure we demonstrate a reconfigurable multichip neuromorphic system for feature selectivity which models orientation tuning properties of cortical neurons

Artificial Cognitive Systems: From VLSI Networks of Spiking Neurons to Neuromorphic Cognition

Neuromorphic engineering (NE) is an emerging research field that has been attempting to identify neural types of computational principles, by implementing biophysically realistic models of neural systems in Very Large Scale Integration (VLSI) technology. Remarkable progress has been made recently, and complex artificial neural sensory-motor systems can be built using this technology. Today, however, NE stands before a large conceptual challenge that must be met before there will be significant progress toward an age of genuinely intelligent neuromorphic machines. The challenge is to bridge the gap from reactive systems to ones that are cognitive in quality. In this paper, we describe recent advancements in NE, and present examples of neuromorphic circuits that can be used as tools to address this challenge. Specifically, we show how VLSI networks of spiking neurons with spike-based plasticity mechanisms and soft winner-take-all architectures represent important building blocks useful for implementing artificial neural systems able to exhibit basic cognitive abilities.

Synthesizing cognition in neuromorphic electronic systems

Significance Neuromorphic emulations express the dynamics of neural systems in analogous electronic circuits, offering a distributed, low-power technology for constructing intelligent systems. However, neuromorphic circuits are inherently imprecise and noisy, and there has been no systematic method for configuring reliable behavioral dynamics on these substrates. We describe such a method, which is able to install simple cognitive behavior on the neuromorphic substrate. Our approach casts light on the general question of how the neuronal circuits of the brain, and also future neuromorphic technologies, could implement cognitive behavior in a principled manner. The quest to implement intelligent processing in electronic neuromorphic systems lacks methods for achieving reliable behavioral dynamics on substrates of inherently imprecise and noisy neurons. Here we report a solution to this problem that involves first mapping an unreliable hardware layer of spiking silicon neurons into an abstract computational layer composed of generic reliable subnetworks of model neurons and then composing the target behavioral dynamics as a “soft state machine” running on these reliable subnets. In the first step, the neural networks of the abstract layer are realized on the hardware substrate by mapping the neuron circuit bias voltages to the model parameters. This mapping is obtained by an automatic method in which the electronic circuit biases are calibrated against the model parameters by a series of population activity measurements. The abstract computational layer is formed by configuring neural networks as generic soft winner-take-all subnetworks that provide reliable processing by virtue of their active gain, signal restoration, and multistability. The necessary states and transitions of the desired high-level behavior are then easily embedded in the computational layer by introducing only sparse connections between some neurons of the various subnets. We demonstrate this synthesis method for a neuromorphic sensory agent that performs real-time context-dependent classification of motion patterns observed by a silicon retina.

An event-based VLSI network of integrate-and-fire neurons

The growing interest in pulse-based neural networks is encouraging the development of hardware implementations of massively parallel, distributed networks of integrate-and-fire (I&F) neurons. We have developed a mixed-mode (analog/digital) VLSI device that comprises a reconfigurable network of I&F neurons and adaptive synapses. The synapses receive input spikes and the neurons transmit output spikes (events) using an asynchronous address-event representation (AER). We describe the network architecture, present experimental data demonstrating the characteristics of the single elements on the chip, and show that a competitive network configuration has winner-take-all (WTA) behaviour and produces spike synchronization.

A systematic method for configuring vlsi networks of spiking neurons

An increasing number of research groups are developing custom hybrid analog/digital very large scale integration (VLSI) chips and systems that implement hundreds to thousands of spiking neurons with biophysically realistic dynamics, with the intention of emulating brainlike real-world behavior in hardware and robotic systems rather than simply simulating their performance on general-purpose digital computers. Although the electronic engineering aspects of these emulation systems is proceeding well, progress toward the actual emulation of brainlike tasks is restricted by the lack of suitable high-level configuration methods of the kind that have already been developed over many decades for simulations on general-purpose computers. The key difficulty is that the dynamics of the CMOS electronic analogs are determined by transistor biases that do not map simply to the parameter types and values used in typical abstract mathematical models of neurons and their networks. Here we provide a general method for resolving this difficulty. We describe a parameter mapping technique that permits an automatic configuration of VLSI neural networks so that their electronic emulation conforms to a higher-level neuronal simulation. We show that the neurons configured by our method exhibit spike timing statistics and temporal dynamics that are the same as those observed in the software simulated neurons and, in particular, that the key parameters of recurrent VLSI neural networks (e.g., implementing soft winner-take-all) can be precisely tuned. The proposed method permits a seamless integration between software simulations with hardware emulations and intertranslatability between the parameters of abstract neuronal models and their emulation counterparts. Most important, our method offers a route toward a high-level task configuration language for neuromorphic VLSI systems.

Spike-based learning with a generalized integrate and fire silicon neuron

Spike-based learning circuits have been typically used in conjunction with linear integrate-and-flre neurons. As a new class of current-mode conductance-based silicon neurons has been recently developed, it is important to evaluate how the spike-based learning circuits perform, when interfaced to these new types of neuron circuits. Here, we describe a VLSI implementation of a current-mode conductance-based neuron, connected to synaptic circuits with spike-based learning capabilities. The conductance-based silicon neuron has built-in spike-frequency adaptation, refractory period mechanisms, and plasticity eligibility control circuits. The synaptic circuits exhibits realistic dynamics in the post-synaptic currents and comprise local spike-based learning circuits, controlled by the global post-synaptic eligibility circuits. We present experimental results which characterize the conductance-based neuron circuit properties and the spike-based learning circuits connected to it.

An adaptive silicon synapse

We present an analog circuit for implementing models of synapses with short-term adaptation, derive analytical solutions for spiking input signals, and present experimental results measured from a chip fabricated using a standard 1.5 /spl mu/m CMOS technology. The circuit is suitable for integration in large arrays of integrate-and-fire neurons and consequently for evaluating computational roles of short-term adaptation at the network level.

Modeling orientation selectivity using a neuromorphic multi-chip system

The growing interest in pulse-mode processing by neural networks is encouraging the development of hardware implementations of massively parallel, distributed networks of integrate-and-fire (I&F) neurons. We have developed a reconfigurable multi-chip neuronal system for modeling feature selectivity and applied it to oriented visual stimuli. Our system comprises a temporally differentiating imager and a VLSI competitive network of neurons which use an asynchronous address event representation (AER) for communication. Here we describe the overall system, and present experimental data demonstrating the effect of recurrent connectivity on the pulse-based orientation selectivity