Eric K. C. Tsang

Neuromorphic implementation of active gaze and vergence control

We present an active stereo system with gaze and vergence control driven by model of the disparity selective neurons in the mammalian visual cortex. The hardware consists of a mobile stereo camera and three Multimap boards. The Multi- map boards compute multiple cortical maps responding to target locations, orientations and disparities, and generate movement commands to track target in space. Each board can compute more than 10 cortical maps at 320*240 pixel resolution and 25 frames per second, and consumes 3.5 W.

Expandable hardware for computing cortical feature maps

We describe expandable hardware architecture for the rapid simulation of feature maps inspired by the visual cortex. Feature maps are retinotopically organized arrays of neurons selective to different combinations of visual features. The responses of these maps are believed to be important for the brain to merge information from different visual cues. This architecture is based around a custom designed board containing DSP and FPGA chips. It is modular in the sense that additional boards can be integrated into the system to accommodate cortical models of increasing complexity

A Scalable FPGA Implementation of Cellular Neural Networks for Gabor-type Filtering

We describe an implementation of Gabor-type filters on field programmable gate arrays using the cellular neural network (CNN) architecture. The CNN template depends upon the parameters (e.g., orientation, bandwidth) of the Gabor-type filter and can be modified at runtime so that the functionality of Gabor-type filter can be changed dynamically. Our implementation uses the Euler method to solve the ordinary differential equation describing the CNN. The design is scalable to allow for different pixel array sizes, as well as simultaneous computation of multiple filter outputs tuned to different orientations and bandwidths. For 1024 pixel frames, an implementation on a Xilinx Virtex XC2V1000-4 device uses 1842 slices, operates at 120 MHz and achieves 23,000 Euler iterations over one frame per second.

A preference for phase-based disparity in a neuromorphic implementation of the binocular energy model

The relative depth of objects causes small shifts in the left and right retinal positions of these objects, called binocular disparity. This letter describes an electronic implementation of a single binocularly tuned complex cell based on the binocular energy model, which has been proposed to model disparity-tuned complex cells in the mammalian primary visual cortex. Our system consists of two silicon retinas representing the left and right eyes, two silicon chips containing retinotopic arrays of spiking neurons with monocular Gabor-type spatial receptive fields, and logic circuits that combine the spike outputs to compute a disparity-selective complex cell response. The tuned disparity can be adjusted electronically by introducing either position or phase shifts between the monocular receptive field profiles. Mismatch between the monocular receptive field profiles caused by transistor mismatch can degrade the relative responses of neurons tuned to different disparities. In our system, the relative responses between neurons tuned by phase encoding are better matched than neurons tuned by position encoding. Our numerical sensitivity analysis indicates that the relative responses of phase-encoded neurons that are least sensitive to the receptive field parameters vary the most in our system. We conjecture that this robustness may be one reason for the existence of phase-encoded disparity-tuned neurons in biological neural systems.

Normalization enables robust validation of disparity estimates from neural populations

Binocular fusion takes place over a limited region smaller than one degree of visual angle (Panums fusional area), which is on the order of the range of preferred disparities measured in populations of disparity-tuned neurons in the visual cortex. However, the actual range of binocular disparities encountered in natural scenes extends over tens of degrees. This discrepancy suggests that there must be a mechanism for detecting whether the stimulus disparity is inside or outside the range of the preferred disparities in the population. Here, we compare the efficacy of several features derived from the population responses of phase-tuned disparity energy neurons in differentiating between in-range and out-of-range disparities. Interestingly, some features that might be appealing at first glance, such as the average activation across the population and the difference between the peak and average responses, actually perform poorly. On the other hand, normalizing the difference between the peak and average responses results in a reliable indicator. Using a probabilistic model of the population responses, we improve classification accuracy by combining multiple features. A decision rule that combines the normalized peak to average difference and the peak location significantly improves performance over decision rules based on either measure in isolation. In addition, classifiers using normalized difference are also robust to mismatch between the image statistics assumed by the model and the actual image statistics.

An on-off temporal filter circuit for visual motion analysis

We describe a temporal filtering circuit, which when cascaded with a previously reported chip for spatial filtering, implements the spatio-temporal filters required to construct motion energy filters, which have been used to model the functional characteristics of direction and speed tuned neurons in the primary visual cortex. To facilitate the combination, the temporal filter circuit uses the same on-off signal representation used by the spatial filtering chip. We present measurements results from a filter fabricated in a 1.5/spl mu/m CMOS n-well process that demonstrates that both the center frequency of the filter, which determines the tuned velocity bandwidth, can be tuned by adjusting external bias voltages. The filter can be tuned to bandwidths and center frequencies on the order of tens of Hertz, comparable to those measured in cortical neurons.

Disparity Estimation by Pooling Evidence From Energy Neurons

In this paper, we propose an algorithm for disparity estimation from disparity energy neurons that seeks to maintain simplicity and biological plausibility, while also being based upon a formulation that enables us to interpret the model outputs probabilistically. We use the Bayes factor from statistical hypothesis testing to show that, in contradiction to the implicit assumption of many previously proposed biologically plausible models, a larger response from a disparity energy neuron does not imply more evidence for the hypothesis that the input disparity is close to the preferred disparity of the neuron. However, we find that the normalized response can be interpreted as evidence, and that information from different orientation channels can be combined by pooling the normalized responses. Based on this insight, we propose an algorithm for disparity estimation constructed out of biologically plausible operations. Our experimental results on real stereograms show that the algorithm outperforms a previously proposed coarse-to-fine model. In addition, because its outputs can be interpreted probabilistically, the model also enables us to identify occluded pixels or pixels with incorrect disparity estimates.

THE HKUST MULTIMAP SYSTEM FOR ACTIVE VISION

This paper describes the HKUST MultiMap system, a hardware system we have developed to support the computation and integration of large numbers of neurally inspired feature maps at frame rates. The computed maps could serve as the basis for a generic image representation to enable robots to perform a variety of different tasks. In order to support the computation of feature representations with increasing complexity, the system can split the processing among a scalable number of processors, each on a different printed circuit board. We describe the hardware design of this board, as well as the communication protocol between different boards. Our experimental results on a four board system validate the performance of the proposed communication protocol. As an example of the applications enabled by this system, we describe a robotic binocular tracking system built by using it.

An on-off Temporal Filter Circuit for Computing Motion Energy

Motion energy units are important components of biologically inspired algorithms for image motion analysis. We describe an analog VLSI temporal filter circuit that can be cascaded with a spatial filter to implement the spatio-temporal filters required to compute motion energy. This circuit adopts an on-off differential representation used by a previously developed chip for spatial filtering. It has two main advantages over previous continuous-time temporal filtering circuits used to compute motion energy. First, its outputs are in exact quadrature phase, rather than an approximation. Second, it halves the total temporal filter order required to compute motion energy. We describe the filter architecture, analyze its dynamics theoretically, and provide measurement results from a prototype fabricated in a 1.5-mum CMOS n-well process

Robotic gaze and vergence control via disparity energy neurons

We briefly describe a demonstration of an active stereo vision system with gaze and vergence control driven by models of the disparity selective neurons in the mammalian visual cortex. The hardware consists of a stereo camera mounted on a pan-tilt head and three multimap boards. The multimap boards compute multiple cortical maps responding to target locations, orientations and disparities, and generate movement commands to track target in space. Each board can compute more than 10 cortical maps at 320*240 pixel resolution and 25 frames per second, and consumes 3.5 W. A more complete description is available in a paper presented at the 2008 International Symposium on Circuits and Systems.