Tobi Delbruck

A 128

This paper describes a 128 times 128 pixel CMOS vision sensor. Each pixel independently and in continuous time quantizes local relative intensity changes to generate spike events. These events appear at the output of the sensor as an asynchronous stream of digital pixel addresses. These address-events signify scene reflectance change and have sub-millisecond timing precision. The output data rate depends on the dynamic content of the scene and is typically orders of magnitude lower than those of conventional frame-based imagers. By combining an active continuous-time front-end logarithmic photoreceptor with a self-timed switched-capacitor differencing circuit, the sensor achieves an array mismatch of 2.1% in relative intensity event threshold and a pixel bandwidth of 3 kHz under 1 klux scene illumination. Dynamic range is > 120 dB and chip power consumption is 23 mW. Event latency shows weak light dependency with a minimum of 15 mus at > 1 klux pixel illumination. The sensor is built in a 0.35 mum 4M2P process. It has 40times40 mum2 pixels with 9.4% fill factor. By providing high pixel bandwidth, wide dynamic range, and precisely timed sparse digital output, this silicon retina provides an attractive combination of characteristics for low-latency dynamic vision under uncontrolled illumination with low post-processing requirements.

\times

Hardware implementations of spiking neurons can be extremely useful for a large variety of applications, ranging from high-speed modeling of large-scale neural systems to real-time behaving systems, to bidirectional brain–machine interfaces. The specific circuit solutions used to implement silicon neurons depend on the application requirements. In this paper we describe the most common building blocks and techniques used to implement these circuits, and present an overview of a wide range of neuromorphic silicon neurons, which implement different computational models, ranging from biophysically realistic and conductance-based Hodgkin–Huxley models to bi-dimensional generalized adaptive integrate and fire models. We compare the different design methodologies used for each silicon neuron design described, and demonstrate their features with experimental results, measured from a wide range of fabricated VLSI chips.

128 120 dB 15

This paper describes CAVIAR, a massively parallel hardware implementation of a spike-based sensing-processing-learning-actuating system inspired by the physiology of the nervous system. CAVIAR uses the asynchronous address-event representation (AER) communication framework and was developed in the context of a European Union funded project. It has four custom mixed-signal AER chips, five custom digital AER interface components, 45 k neurons (spiking cells), up to 5 M synapses, performs 12 G synaptic operations per second, and achieves millisecond object recognition and tracking latencies.

\mu

The author describes two small analog circuits that compute generalised measures of the similarity of two voltage inputs. The similarity outputs from the circuits, given as currents, become large when the input voltages are close to each other. One of the circuits, the bump circuit, computes only this similarity output. The other circuit, the bump-anti-bump circuit, computes the similarity output as well as the dissimilarity measure. Each of its dissimilarity outputs becomes large only when the corresponding input is sufficiently larger than the other input. The dissimilarity outputs can be summed together or left separate; when left separate, they resemble generalised rectifier outputs. Test results are presented that illustrate these transfer characteristics.<<ETX>>

s Latency Asynchronous Temporal Contrast Vision Sensor

Biology provides examples of efficient machines which greatly outperform conventional technology. Designers in neuromorphic engineering aim to construct electronic systems with the same efficient style of computation. This task requires a melding of novel engineering principles with knowledge gleaned from neuroscience. We discuss recent progress in realizing neuromorphic sensory systems which mimic the biological retina and cochlea, and subsequent sensor processing. The main trends are the increasing number of sensors and sensory systems that communicate through asynchronous digital signals analogous to neural spikes; the improved performance and usability of these sensors; and novel sensory processing methods which capitalize on the timing of spikes from these sensors. Experiments using these sensors can impact how we think the brain processes sensory information.

Neuromorphic silicon neuron circuits

We describe a photoreceptor circuit that can be used in massively parallel analog VLSI silicon chips, in conjunction with other local circuits, to perform initial analog visual information processing. The receptor provides a continuous-time output that has low gain for static signals (including circuit mismatches), and high gain for transient signals that are centered around the adaptation point. The response is logarithmic, which makes the response to a fixed image contrast invariant to absolute light intensity. The 5-transistor receptor can be fabricated in an area of about 70 /spl mu/m by 70 /spl mu/m in a 2-/spl mu/m single-poly CMOS technology. It has a dynamic range of 1-2 decades at a single adaptation level, and a total dynamic range of more than 6 decades. Several technical improvements in the circuit yield an additional 1-2 decades dynamic range over previous designs without sacrificing signal quality. The lower limit of the dynamic range, defined arbitrarily as the illuminance at which the bandwidth of the receptor is 60 Hz, is at approximately 1 lux, which is the border between rod and cone vision and also the limit of current consumer video cameras. The receptor uses an adaptive element that is resistant to excess minority carrier diffusion. The continuous and logarithmic transduction process makes the bandwidth scale with intensity. As a result, the total AC RMS receptor noise is constant, independent of intensity. The spectral density of the noise is within a factor of two of pure photon shot noise and varies inversely with intensity. The connection between shot and thermal noise in a system governed by Boltzmann statistics is beautifully illustrated.<<ETX>>

CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking

Deep Belief Networks (DBNs) have recently shown impressive performance on a broad range of classification problems. Their generative properties allow better understanding of the performance, and provide a simpler solution for sensor fusion tasks. However, because of their inherent need for feedback and parallel update of large numbers of units, DBNs are expensive to implement on serial computers. This paper proposes a method based on the Siegert approximation for Integrate-and-Fire neurons to map an offline-trained DBN onto an efficient event-driven spiking neural network suitable for hardware implementation. The method is demonstrated in simulation and by a real-time implementation of a 3-layer network with 2694 neurons used for visual classification of MNIST handwritten digits with input from a 128 × 128 Dynamic Vision Sensor (DVS) silicon retina, and sensory-fusion using additional input from a 64-channel AER-EAR silicon cochlea. The system is implemented through the open-source software in the jAER project and runs in real-time on a laptop computer. It is demonstrated that the system can recognize digits in the presence of distractions, noise, scaling, translation and rotation, and that the degradation of recognition performance by using an event-based approach is less than 1%. Recognition is achieved in an average of 5.8 ms after the onset of the presentation of a digit. By cue integration from both silicon retina and cochlea outputs we show that the system can be biased to select the correct digit from otherwise ambiguous input.

'Bump' circuits for computing similarity and dissimilarity of analog voltages

Event-based dynamic vision sensors (DVSs) asynchronously report log intensity changes. Their high dynamic range, sub-ms latency and sparse output make them useful in applications such as robotics and real-time tracking. However they discard absolute intensity information which is useful for object recognition and classification. This paper presents a dynamic and active pixel vision sensor (DAVIS) which addresses this deficiency by outputting asynchronous DVS events and synchronous global shutter frames concurrently. The active pixel sensor (APS) circuits and the DVS circuits within a pixel share a single photodiode. Measurements from a 240×180 sensor array of 18.5 μm 2 pixels fabricated in a 0.18 μm 6M1P CMOS image sensor (CIS) technology show a dynamic range of 130 dB with 11% contrast detection threshold, minimum 3 μs latency, and 3.5% contrast matching for the DVS pathway; and a 51 dB dynamic range with 0.5% FPN for the APS readout.

Neuromorphic sensory systems

Fast sensory-motor processing is challenging when using traditional frame-based cameras and computers. Here the authors show how a hybrid neuromorphic-procedural system consisting of an address-event silicon retina, a computer, and a servo motor can be used to implement a fast sensory-motor reactive controller to track and block balls shot at a goal. The system consists of a 128times128 retina that asynchronously reports scene reflectance changes, a laptop PC, and a servo motor controller. Components are interconnected by USB. The retina looks down onto the field in front of the goal. Moving objects are tracked by an event-driven cluster tracker algorithm that detects the ball as the nearest object that is approaching the goal. The balls position and velocity are used to control the servo motor. Running under Windows XP, the reaction latency is 2.8plusmn0.5 ms at a CPU load of <10% with a minimum observed latency of 1.8 ms. A 2 GHz Pentium M laptop can process at >1 million events per second (Meps), although fast balls only create ~30 keps. This system demonstrates the advantages of hybrid event-based sensory motor processing

Adaptive photoreceptor with wide dynamic range

Mixed-signal or analog chips often require a wide range of biasing currents that are independent of process and supply voltage and that are proportional to absolute temperature. This paper describes CMOS circuits that we use to generate a set of fixed bias currents typically spanning six decades at room temperature down to a few times the transistor off-current. A bootstrapped current reference with a new startup and power-control mechanism generates a master current, which is successively divided by a current splitter to generate the desired reference currents. These references are nondestructively copied to form the chip’s biases. Measurements of behavior, including temperature effects from 1.6 and 0.35 μ implementations, are presented and nonidealities are investigated. Temperature dependence of the transistor off-current is investigated because it determines the lower limit for generated currents. Readers are directed to a design kit that allows easy generation of the complete layout for a bias generator with a set of desired currents for scalable MOSIS CMOS processes.