Fernando Perez-Peña

Neuro-Inspired Spike-Based Motion: From Dynamic Vision Sensor to Robot Motor Open-Loop Control through Spike-VITE

In this paper we present a complete spike-based architecture: from a Dynamic Vision Sensor (retina) to a stereo head robotic platform. The aim of this research is to reproduce intended movements performed by humans taking into account as many features as possible from the biological point of view. This paper fills the gap between current spike silicon sensors and robotic actuators by applying a spike processing strategy to the data flows in real time. The architecture is divided into layers: the retina, visual information processing, the trajectory generator layer which uses a neuroinspired algorithm (SVITE) that can be replicated into as many times as DoF the robot has; and finally the actuation layer to supply the spikes to the robot (using PFM). All the layers do their tasks in a spike-processing mode, and they communicate each other through the neuro-inspired AER protocol. The open-loop controller is implemented on FPGA using AER interfaces developed by RTC Lab. Experimental results reveal the viability of this spike-based controller. Two main advantages are: low hardware resources (2% of a Xilinx Spartan 6) and power requirements (3.4 W) to control a robot with a high number of DoF (up to 100 for a Xilinx Spartan 6). It also evidences the suitable use of AER as a communication protocol between processing and actuation.

Towards AER VITE: Building spike gate signal

Neuromorphic engineers aim to mimic the precise and efficient mechanisms of the nervous system to process information using spikes from sensors to actuators. There are many available works that sense and process information in a spike-based way. But there are still several gaps in the actuation and motor control field in a spike-based way. Spike-based Proportional-Integrative-Derivative controllers (PID) are present in the literature. On the other hand, neuro-inspired control models as VITE (Vector Integration To End point) and FLETE (Factorization of muscle Length and muscle Tension) are also present in the literature. This paper presents another step toward the spike implementation of those neuro-inspired models. We present a spike-based ramp multiplier. VITE algorithm generates the way to achieve a final position targeted by a mobile robotic arm. The block presented is used as a gate for the way involved and it also puts the incoming movement on speed with a variable slope profile. Only spikes for information representation were used and the process is in real time. The software simulation based on Simulink and Xilinx System Generator shows the accurate adjust to the traditional processing for short time periods and the hardware tests confirm and extend the previous simulated results for any time. We have implemented the spikes generator, the ramp multiplier and the low pass filter into the Virtex-5 FPGA and connected this with an USB-AER (Address Event Representation) board to monitor the spikes.

Inter-spikes-intervals exponential and gamma distributions study of neuron firing rate for SVITE motor control model on FPGA

This paper presents a statistical study on a neuro-inspired spike-based implementation of the Vector-Integration-To-End-Point motor controller (SVITE) and compares its deterministic neuron-model stream of spikes with a proposed modification that converts the model, and thus the controller, in a Poisson like spike stream distribution. A set of hardware pseudo-random numbers generators, based on a Linear Feedback Shift Register (LFSR), have been introduced in the neuron-model so that they reach a closer biological neuron behavior. To validate the new neuron-model behavior a comparison between the Inter-Spikes-Interval empirical data and the Exponential and Gamma distributions has been carried out using the Kolmogorov-Smirnoff test. An in-hardware validation of the controller has been performed in a Spartan6 FPGA to drive directly with spikes DC motors from robotics to study the behavior and viability of the modified controller with random components.The results show that the original deterministic spikes distribution of the controller blocks can be swapped with Poisson distributions using 30-bit LFSRs. The comparative between the usable controlling signals such as the trajectory and the speed profile using a deterministic and the new controller show a standard deviation of 11.53spikes/s and 3.86spikes/s respectively. These rates do not affect our system because, within Pulse Frequency Modulation, in order to drive the motors, time length can be fixed to spread the spikes. Tuning this value, the slow rates could be filtered by the motor. Therefore, this SVITE neuro-inspired controller can be integrated within complex neuromorphic architectures with Poisson-like neurons.

An Approach to Motor Control for Spike-based Neuromorphic Robotics

This paper presents an approach to open-loop motor control using Integrate and Fire (IF) neurons. The controller aims at mimicking motor control structures found in the brain and consists of three neuron populations implemented on different VLSI chips. The first population codes the distance to the target in a form of a firing rate (similarly to some class of cells found in Area 4 in the motor mammalian cortex). The second population mimics the behavior of neurons of the basal ganglia and control the gating and speed of the movement, by means of an NMDA synapse and an excitatory connection. The third population codes the supposed position reached by the robot. The multi-chip neuromorphic setup is interfaced with a Field-Programmable Gate Array (FPGA) board by the Address Event Representation (AER) communication protocol. The FPGA elongates the spike duration to make them suitable for driving the motors with Pulse Frequency Modulation (PFM). This approach aims to compete with classic controllers offering lower power, simplified control and smoother movements.

On the AER Stereo-Vision Processing: A Spike Approach to Epipolar Matching

Image processing in digital computer systems usually considers visual information as a sequence of frames. These frames are from cameras that capture reality for a short period of time. They are renewed and transmitted at a rate of 25-30 fps (typical real-time scenario). Digital video processing has to process each frame in order to detect a feature on the input. In stereo vision, existing algorithms use frames from two digital cameras and process them pixel by pixel until it finds a pattern match in a section of both stereo frames. To process stereo vision information, an image matching process is essential, but it needs very high computational cost. Moreover, as more information is processed, the more time spent by the matching algorithm, the more inefficient it is. Spike-based processing is a relatively new approach that implements processing by manipulating spikes one by one at the time they are transmitted, like a human brain. The mammal nervous system is able to solve much more complex problems, such as visual recognition by manipulating neuron’s spikes. The spike-based philosophy for visual information processing based on the neuro-inspired Address-Event- Representation (AER) is achieving nowadays very high performances. The aim of this work is to study the viability of a matching mechanism in a stereo-vision system, using AER codification. This kind of mechanism has not been done before to an AER system. To do that, epipolar geometry basis applied to AER system are studied, and several tests are run, using recorded data and a computer. The results and an average error are shown (error less than 2 pixels per point); and the viability is proved.

A FPGA spike-based robot controlled with neuro-inspired VITE

This paper presents a spike-based control system applied to a fixed robotic platform. Our aim is to take a step forward to a future complete spikes processing architecture, from vision to direct motor actuation. This paper covers the processing and actuation layer over an anthropomorphic robot. In this way, the processing layer uses the neuro-inspired VITE algorithm, for reaching a target, based on PFM taking advantage of spike system information: its frequency. Thus, all the blocks of the system are based on spikes. Each layer is implemented within a FPGA board and spikes communication is codified under the AER protocol. The results show an accurate behavior of the robotic platform with 6-bit resolution for a 130o range per joint, and an automatic speed control of the algorithm. Up to 96 motor controllers could be integrated in the same FPGA, allowing the positioning and object grasping by more complex anthropomorphic robots.

Pipeline AER arbitration with event aging

We present a simple circuit to handle communication between cells of neuromorphic arrays. It allows cells to operate continuously without waiting for acknowledgement signals back from the AER (Address Event Representation) arbitration circuitry. The module also implements aging of cell petitions i.e., old petitions to access to the AER bus are automatically discarded to give priority to the more recent ones and alleviate the bus congestion. The new arbitration scheme has been implemented and tested. A particular application scenario with an image sensor with spiking pixels that sense light continuously is explained. The sensing errors per event due to discontinued pixel operation can be minimized a factor 8.1. Experimental data obtained with real visual scenes are provided.

Towards bioinspired close-loop local motor control: A simulated approach supporting neuromorphic implementations

Despite being well established in robotics, classical motor controllers have several disadvantages: they pose a high computational load, therefore requiring powerful devices, they are not easy to tune and they are not suited for neuroprosthetics. In contrast, bio-inspired controller do not transform the output of the controller therefore no delays are introduced and a smooth response is achieved; they also have a high scalability. Finally, the most important feature of bio-inspired controllers is that they could integrate learning features to make them adaptable to new tasks within the same hardware robotic platform. We present the model and simulation of a spiking neural network for low-level motor control. The proposed neural network acts as a motor controller and produces pulsed signals which can be directly interfaced with commercial DC motors. The simulated network is compatible with neuromorphic VLSI implementation and paves the way to the implementation bio-inspired motor controller which are compact, low power, scalable and compatible with neuroprosthetic. The network presented is inspired by the current knowledge about biological motor control: it comprises alpha motoneuron for driving the motor and spindle populations to provide the feedback and close the loop. The spikes from the motoneuron population are time lengthen to a fixed amount of time and supplied to the simulated motor: Pulse Frequency Modulation (PFM) modulation is used. This paper presents the software simulations using the Brian simulator for a position controller. Our controller is a first step toward a novel bio-inspired motor control approach suitable for robotics as well as neuroprosthetic.

Spike-based VITE control with dynamic vision sensor applied to an arm robot

Spike-based motor control is very important in the field of robotics and also for the neuromorphic engineering community to bridge the gap between sensing / processing devices and motor control without losing the spike philosophy that enhances speed response and reduces power consumption. This paper shows an accurate neuro-inspired spike-based system composed of a DVS retina, a visual processing system that detects and tracks objects, and a SVITE motor control, where everything follows the spike-based philosophy. The control system is a spike version of the neuroinspired open loop VITE control algorithm implemented in a couple of FPGA boards: the first one runs the algorithm and the second one drives the motors with spikes. The robotic platform is a low cost arm with four degrees of freedom.

SVITE: A Spike-Based VITE Neuro-Inspired Robot Controller

This paper presents an implementation of a neuro-inspired algorithm called VITE (Vector Integration To End Point) in FPGA in the spikes domain. VITE aims to generate a non-planned trajectory for reaching tasks in robots. The algorithm has been adapted to work completely in the spike domain under Simulink simulations. The FPGA implementation consists in 4 VITE in parallel for controlling a 4-degree-of-freedom stereo-vision robot. This work represents the main layer of a complex spike-based architecture for robot neuro-inspired reaching tasks in FPGAs. It has been implemented in two Xilinx FPGA families: Virtex-5 and Spartan-6. Resources consumption comparative between both devices is presented. Results obtained for Spartan device could allow controlling complex robotic structures with up to 96 degrees of freedom per FPGA, providing, in parallel, high speed connectivity with other neuromorphic systems sending movement references. An exponential and gamma distribution test over the inter spike interval has been performed to proof the approach to the neural code proposed.