Francesco Diotalevi

Evaluation of gradient descent learning algorithms with an adaptive local rate technique for hierarchical feedforward architectures

Gradient descent learning algorithms (namely backpropagation and weight perturbation) can significantly increase their classification performances by adopting a local and adaptive learning rate management approach. We present the results of the comparison of the classification performance of the two algorithms in a tough application: quality control analysis in the steel industry. The feedforward network is hierarchically organized (i.e. tree of multilayer perceptrons). The comparison has been performed starting from the same operating conditions (i.e. network topology, stopping criterion, etc.): the results show that the probability of correct classification is significantly better for the weight perturbation algorithm.

An analog on-chip learning circuit architecture of the weight perturbation algorithm

In this paper we present the analog on-chip learning architecture of a gradient descent learning algorithm: the Weight Perturbation learning algorithm. From the circuit implementation point of view our approach is based on current mode and translinear operated circuits. The proposed architecture is very efficient in terms of speed, size, precision and power consumption; moreover it exhibits also high scalability and modularity.

Analog CMOS current mode neural primitives

The CMOS circuit implementation of the feedforward neural primitives of a generic Multi Layer Perceptron network is presented. Basically our approach is based on current mode computation and is aimed at a low power/low voltage circuit implementation; moreover, it is easily scalable to implement networks of any size. Experimental results are reported.

A Bidirectional Brain-Machine Interface Featuring a Neuromorphic Hardware Decoder

Bidirectional brain-machine interfaces (BMIs) establish a two-way direct communication link between the brain and the external world. A decoder translates recorded neural activity into motor commands and an encoder delivers sensory information collected from the environment directly to the brain creating a closed-loop system. These two modules are typically integrated in bulky external devices. However, the clinical support of patients with severe motor and sensory deficits requires compact, low-power, and fully implantable systems that can decode neural signals to control external devices. As a first step toward this goal, we developed a modular bidirectional BMI setup that uses a compact neuromorphic processor as a decoder. On this chip we implemented a network of spiking neurons built using its ultra-low-power mixed-signal analog/digital circuits. On-chip on-line spike-timing-dependent plasticity synapse circuits enabled the network to learn to decode neural signals recorded from the brain into motor outputs controlling the movements of an external device. The modularity of the BMI allowed us to tune the individual components of the setup without modifying the whole system. In this paper, we present the features of this modular BMI and describe how we configured the network of spiking neuron circuits to implement the decoder and to coordinate it with the encoder in an experimental BMI paradigm that connects bidirectionally the brain of an anesthetized rat with an external object. We show that the chip learned the decoding task correctly, allowing the interfaced brain to control the objects trajectories robustly. Based on our demonstration, we propose that neuromorphic technology is mature enough for the development of BMI modules that are sufficiently low-power and compact, while being highly computationally powerful and adaptive.

A dedicated very low power analog VLSI architecture for smart adaptive systems

Abstract This paper deals with analog VLSI architectures addressed to the implementation of smart adaptive systems on silicon. In particular, we addressed the implementation of artificial neural networks with on-chip learning algorithms with the goal of efficiency in terms of scalability, modularity, computational density, real time operation and power consumption. We present the analog circuit architecture of a feed-forward network with on-chip weight perturbation learning in CMOS technology. Novelty of the approach lies in the circuit implementation of the feed-forward neural primitives and on the overall analog circuit architecture. The proposed circuits feature very low power consumption and robustness with respect to noise effects. We extensively tested the analog architecture with simulations at transistor level by using the netlist extracted from the physical design. The results compare favourably with those reported in the open literature. In particular, the architecture exhibits very high power efficiency and computational density and remarkable modularity and scalability features. The proposed approach is aimed to the implementation of embedded intelligent systems for ubiquitous computing.

Highly parallel and fast implementation of stereo vision algorithms on MIMD many-core Tilera architecture

In this paper we present a fast, and for some cases faster than real-time, implementation of a class of dense stereo vision algorithms including the sum of squared differences (SSD), SSD with left-right check, and SSD with multiple windows, on a low-power MIMD many-core architecture, Tilera. Stereo vision - a method to extract spatial depth information of a scene from two pairs of stereo images - is performed as a primary task and first step in many computer vision applications, e.g. 3D modeling and obstacle detection/avoidance in autonomous vehicles. To reduce the scene conditions in real environment and achieve a robust error rejection, intensive computation for implementing a multiple window with left-right checking scheme is required. Therefore, real-time implementation of these algorithms is a challenging problem, particularly in an embedded application. To the best of our knowledge, our results present the first implementation of any stereo vision algorithm on new emerging MIMD many-core architectures. We have achieved a faster than real-time performance of 207, 118, and 30.45 frames per second for VGA (640×480) images with a disparity range of 16 for standard SSD, SSD with left-right checking, and SSD with 5 multiple window implementations, respectively. For HDTV (1280×720) images, we have achieved rather unique results of 71, and 35.75 frames per second for standard SSD and SSD with left-right checking implementations, respectively. Such excellent performance along with the low power consumption of the Tilera architecture (less than 23W) makes it an excellent candidate to achieve a supercomputing level capability for mobile computer vision applications. Experimental results also clearly demonstrate that the new many-core MIMD parallel architectures can indeed achieve excellent performance in low-level image processing computations while providing a high degree of flexibility and programmability.

A VLSI architecture for weight perturbation on chip learning implementation

In this paper we present the analog on-chip learning architecture of a gradient descent learning algorithm: the weight perturbation learning algorithm. From the circuit implementation point of view our approach is based on current mode and translinear operated circuits. The proposed architecture is very efficient in terms of speed, size, precision and power consumption; moreover it exhibits also high scalability and modularity.

Spatially selective binaural hearing aids

Traditional hearing aids are limited by the absence of spatial selectivity. Superdirective microphone array can recover such limit, performing a spatial filtering to achieve an augmented SNR. We present Glassense, a platform hosting a double microphone array connected to a processing unit and mounted on the frame of common glasses. The platform has the potential of delivering binaural spatially selective audio inputs, allowing ecological pointing of acoustic sources through head motion. The designed microphone arrays exhibit a gain suitable to improve the speech reception threshold of hearing-impaired subjects, obtained on a lightweight and scalable hardware setup.

A modular configurable system for closed-loop bidirectional brain-machine interfaces

Extending bidirectional brain-machine interfaces (BMI) tailored for specific experiments with additional software and hardware tools can be very onerous, if not impossible. To overcome this problem, we developed a modular configurable system by modifying the architecture of an existing bidirectional BMI. This modular system enables the seamless and efficient inclusion of new features and the integration of new protocols without changing the native systems overall structure. By introducing a platform for the implementation of BMI algorithms on neuromorphic chips, this method represents a step towards the development of low-power, compact and computationally powerful tools for clinical applications.

A cooperative search algorithm for highly parallel implementation of RANSAC for model estimation on Tilera MIMD architecture

In this paper, we present a novel and fast algorithm for highly parallel implementation of the RANSAC on a many-core MIMD architecture, the Tilera. RANSAC is widely used in image processing applications for homography model estimation. It also represents one of the most computation intensive image processing tasks since it requires evaluation of a large number of models from a given data set. Therefore, increasing the efficiency in its computation by exploiting a massive degree of parallelism is the key enabling factor for many of its applications. Emerging highly parallel architectures such as Tilera provide such an opportunity of exploiting parallelism in many computations. In addition to its low power consumption and excellent GOPs per Watt performance, radiation-hard version of Tilera has also been developed which makes it one of the best candidates for future aerospace applications. In this paper, we first present a novel variant of the RANSAC by incorporating the concept of backtracking. We then present this variant as a cooperative search algorithm with excellent features for highly parallel implementation. In fact, our parallel implementation results in an asynchronous algorithm with a very limited communication requirement. Any processor performs a global broadcasting if and when it finds a partial solution better than previous one. We present our results for an extensive set of data with varying degree of outliers. Our practical results clearly demonstrate that excellent speedup in the computation is achieved by using 57 cores of the Tilera. In fact, for certain cases, our Cooperative Search Algorithms even achieve super-linear speedup, i.e., a speedup greater than 57. We discuss that such a result could have been indeed expected and can be used for other applications.