Francisco Gomez-Rodriguez

CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking

This paper describes CAVIAR, a massively parallel hardware implementation of a spike-based sensing-processing-learning-actuating system inspired by the physiology of the nervous system. CAVIAR uses the asynchronous address-event representation (AER) communication framework and was developed in the context of a European Union funded project. It has four custom mixed-signal AER chips, five custom digital AER interface components, 45 k neurons (spiking cells), up to 5 M synapses, performs 12 G synaptic operations per second, and achieves millisecond object recognition and tracking latencies.

Computer vision techniques for forest fire perception

This paper presents computer vision techniques for forest fire perception involving measurement of forest fire properties (fire front, flame height, flame inclination angle, fire base width) required for the implementation of advanced forest fire-fighting strategies. The system computes a 3D perception model of the fire and could also be used for visualizing the fire evolution in remote computer systems. The presented system integrates the processing of images from visual and infrared cameras. It applies sensor fusion techniques involving also telemetry sensors, and GPS. The paper also includes some results of forest fire experiments.

AER tools for communications and debugging

Address-event-representation (AER) is a communications protocol for transferring spikes between bio-inspired chips. Such systems may consist of a hierarchical structure with several chips that transmit spikes among them in real time, while performing some processing. To develop and test AER based systems it is convenient to have a set of instruments that would allow to: generate AER streams, monitor the output produced by neural chips and modify the spike stream produced by an emitting chip to adapt it to the requirements of the receiving elements. In this paper we present a set of tools that implement these functions developed in the CAVIAR EU project

Using FPGA for visuo-motor control with a silicon retina and a humanoid robot

The address-event representation (AER) is a neuromorphic communication protocol for transferring asynchronous events between VLSI chips. The event information is transferred using a high speed digital parallel bus. This paper present an experiment based on AER for visual sensing, processing and finally actuating a robot. The AER output of a silicon retina is processed by an AER filter implemented into a FPGA to produce a mimicking behaviour in a humanoid robot (The RoboSapiens V2). We have implemented the visual filter into the Spartan II FPGA of the USB-AER platform and the central pattern generator (CPG) into the Spartan 3 FPGA of the AER-Robot platform, both developed by authors.

Test infrastructure for address-event-representation communications

Address-Event-Representation (AER) is a communication protocol for transferring spikes between bio-inspired chips. Such systems may consist of a hierarchical structure with several chips that transmit spikes among them in real time, while performing some processing. To develop and test AER based systems it is convenient to have a set of instruments that would allow to: generate AER streams, monitor the output produced by neural chips and modify the spike stream produced by an emitting chip to adapt it to the requirements of the receiving elements. In this paper we present a set of tools that implement these functions developed in the CAVIAR EU project.

Smoke monitoring and measurement using image processing: application to forest fires

Smoke detection and monitoring is required for the implementation of advanced forest fire fighting strategies and validation of smoke dispersion models. The latter involve the measurement of smoke column properties. The method proposed in this paper is based on the application of computer-based image processing techniques to visual images taken from fire-spread tests. The method presented involves the application of wavelets and optical flow for fire smoke detection and monitoring. A set of experimental results are reported in the paper, showing the interest of the presented system.

Neuro-Inspired Spike-Based Motion: From Dynamic Vision Sensor to Robot Motor Open-Loop Control through Spike-VITE

In this paper we present a complete spike-based architecture: from a Dynamic Vision Sensor (retina) to a stereo head robotic platform. The aim of this research is to reproduce intended movements performed by humans taking into account as many features as possible from the biological point of view. This paper fills the gap between current spike silicon sensors and robotic actuators by applying a spike processing strategy to the data flows in real time. The architecture is divided into layers: the retina, visual information processing, the trajectory generator layer which uses a neuroinspired algorithm (SVITE) that can be replicated into as many times as DoF the robot has; and finally the actuation layer to supply the spikes to the robot (using PFM). All the layers do their tasks in a spike-processing mode, and they communicate each other through the neuro-inspired AER protocol. The open-loop controller is implemented on FPGA using AER interfaces developed by RTC Lab. Experimental results reveal the viability of this spike-based controller. Two main advantages are: low hardware resources (2% of a Xilinx Spartan 6) and power requirements (3.4 W) to control a robot with a high number of DoF (up to 100 for a Xilinx Spartan 6). It also evidences the suitable use of AER as a communication protocol between processing and actuation.

Two hardware implementations of the exhaustive synthetic AER generation method

Address-Event-Representation (AER) is a communications protocol for transferring images between chips, originally developed for bio-inspired image processing systems. In [6], [5] various software methods for synthetic AER generation were presented. But in neuro-inspired research field, hardware methods are needed to generate AER from laptop computers. In this paper two real time implementations of the exhaustive method, proposed in [6], [5], are presented. These implementations can transmit, through AER bus, images stored in a computer using USB-AER board developed by our RTCAR group for the CAVIAR EU project.

AER Auditory Filtering and CPG for Robot Control

Address-event-representation (AER) is a communication protocol for transferring asynchronous events between VLSI chips, originally developed for bio-inspired processing systems (for example, image processing). The event information in an AER system is transferred using a high-speed digital parallel bus. This paper presents an experiment using AER for sensing, processing and finally actuating a robot. The AER output of a silicon cochlea is processed by an AER filter implemented on a FPGA to produce rhythmic walking in a humanoid robot (Redbot). We have implemented both the AER rhythm detector and the central pattern generator (CPG) on a Spartan II FPGA which is part of a USB-AER platform developed by some of the authors

On the AER convolution processors for FPGA

Image convolution operations in digital computer systems are usually very expensive operations in terms of resource consumption (processor resources and processing time) for an efficient Real-Time application. In these scenarios the visual information is divided into frames and each one has to be completely processed before the next frame arrives in order to warranty the real-time. A spike-based philosophy for computing convolutions based on the neuro-inspired Address-Event-Representation (AER) is achieving high performances. In this paper we present two FPGA implementations of AER-based convolution processors for relatively small Xilinx FPGAs (Spartan-II 200 and Spartan-3 400), which process 64×64 images with 11×11 convolution kernels. The maximum equivalent operation rate that can be reached is 163.51 MOPS for 11×11 kernels, in a Xilinx Spartan 3 400 FPGA with a 50MHz clock. Formulations, hardware architecture, operation examples and performance comparison with frame-based convolution processors are presented and discussed.