Pablo Martínez-Cañada

Real-time tone mapping on GPU and FPGA

Low-level computer vision algorithms have high computational requirements. In this study, we present two real-time architectures using resource constrained FPGA and GPU devices for the computation of a new algorithm which performs tone mapping, contrast enhancement, and glare mitigation. Our goal is to implement this operator in a portable and battery-operated device, in order to obtain a low vision aid specially aimed at visually impaired people who struggle to manage themselves in environments where illumination is not uniform or changes rapidly. This aid device processes in real-time, with minimum latency, the input of a camera and shows the enhanced image on a head mounted display (HMD). Therefore, the proposed operator has been implemented on battery-operated platforms, one based on the GPU NVIDIA ION2 and another on the FPGA Spartan III, which perform at rates of 30 and 60 frames per second, respectively, when working with VGA resolution images (640 × 480).

A Computational Framework for Realistic Retina Modeling

Computational simulations of the retina have led to valuable insights about the biophysics of its neuronal activity and processing principles. A great number of retina models have been proposed to reproduce the behavioral diversity of the different visual processing pathways. While many of these models share common computational stages, previous efforts have been more focused on fitting specific retina functions rather than generalizing them beyond a particular model. Here, we define a set of computational retinal microcircuits that can be used as basic building blocks for the modeling of different retina mechanisms. To validate the hypothesis that similar processing structures may be repeatedly found in different retina functions, we implemented a series of retina models simply by combining these computational retinal microcircuits. Accuracy of the retina models for capturing neural behavior was assessed by fitting published electrophysiological recordings that characterize some of the best-known phenomena observed in the retina: adaptation to the mean light intensity and temporal contrast, and differential motion sensitivity. The retinal microcircuits are part of a new software platform for efficient computational retina modeling from single-cell to large-scale levels. It includes an interface with spiking neural networks that allows simulation of the spiking response of ganglion cells and integration with models of higher visual areas.

Connecting Artificial Brains to Robots in a Comprehensive Simulation Framework: The Neurorobotics Platform

Combined efforts in the fields of neuroscience, computer science, and biology allowed to design biologically realistic models of the brain based on spiking neural networks. For a proper validation of these models, an embodiment in a dynamic and rich sensory environment, where the model is exposed to a realistic sensory-motor task, is needed. Due to the complexity of these brain models that, at the current stage, cannot deal with real-time constraints, it is not possible to embed them into a real-world task. Rather, the embodiment has to be simulated as well. While adequate tools exist to simulate either complex neural networks or robots and their environments, there is so far no tool that allows to easily establish a communication between brain and body models. The Neurorobotics Platform is a new web-based environment that aims to fill this gap by offering scientists and technology developers a software infrastructure allowing them to connect brain models to detailed simulations of robot bodies and environments and to use the resulting neurorobotic systems for in silico experimentation. In order to simplify the workflow and reduce the level of the required programming skills, the platform provides editors for the specification of experimental sequences and conditions, environments, robots, and brain–body connectors. In addition to that, a variety of existing robots and environments are provided. This work presents the architecture of the first release of the Neurorobotics Platform developed in subproject 10 “Neurorobotics” of the Human Brain Project (HBP).1 At the current state, the Neurorobotics Platform allows researchers to design and run basic experiments in neurorobotics using simulated robots and simulated environments linked to simplified versions of brain models. We illustrate the capabilities of the platform with three example experiments: a Braitenberg task implemented on a mobile robot, a sensory-motor learning task based on a robotic controller, and a visual tracking embedding a retina model on the iCub humanoid robot. These use-cases allow to assess the applicability of the Neurorobotics Platform for robotic tasks as well as in neuroscientific experiments.

First Stage of a Human Visual System Simulator: The Retina

We propose a configurable simulation platform that reproduces the analog neural behavior of different models of the Human Visual System at the early stages. Our software can simulate efficiently many of the biological mechanisms found in retina cells, such as chromatic opponency in the red-green and blue-yellow pathways, signal gathering through chemical synapses and gap junctions or variations in the neuron density and the receptive field size with eccentricity. Based on an image-processing approach, simulated neurons can perform spatiotemporal and color processing of the input visual stimuli generating the visual maps of every intermediate stage, which correspond to membrane potentials and synaptic currents. An interface with neural network simulators has been implemented, which allows to reproduce the spiking output of some specific cells, such as ganglion cells, and integrate the platform with models of higher brain areas. Simulations of different retina models related to the color opponent mechanisms, obtained from electro-physiological experiments, show the capability of the platform to reproduce their neural response.

Retina Color-Opponency Based Pursuit Implemented Through Spiking Neural Networks in the Neurorobotics Platform

The ‘red-green’ pathway of the retina is classically recognized as one of the retinal mechanisms allowing humans to gather color information from light, by combining information from L-cones and M-cones in an opponent way. The precise retinal circuitry that allows the opponency process to occur is still uncertain, but it is known that signals from L-cones and M-cones, having a widely overlapping spectral response, contribute with opposite signs. In this paper, we simulate the red-green opponency process using a retina model based on linear-nonlinear analysis to characterize context adaptation and exploiting an image-processing approach to simulate the neural responses in order to track a moving target. Moreover, we integrate this model within a visual pursuit controller implemented as a spiking neural network to guide eye movements in a humanoid robot. Tests conducted in the Neurorobotics Platform confirm the effectiveness of the whole model. This work is the first step towards a bio-inspired smooth pursuit model embedding a retina model using spiking neural networks.

Towards a Generic Simulation Tool of Retina Models

The retina is one of the most extensively studied neural circuits in the Visual System. Numerous models have been proposed to predict its neural behavior on the response to artificial and natural visual patterns. These models can be considered an important tool for understanding the underlying biophysical and anatomical mechanisms. This paper describes a general-purpose simulation environment that fits to different retina models and provides a set of elementary simulation modules at multiple abstraction levels. The platform can simulate many of the biological mechanisms found in retinal cells, such as signal gathering though chemical synapses and gap junctions, variations in the receptive field size with eccentricity, membrane integration by linear and single-compartment models and short-term synaptic plasticity. A built-in interface with neural network simulators reproduces the spiking output of some specific cells, such as ganglion cells, and allows integration of the platform with models of higher visual areas. We used this software to implement whole retina models, from photoreceptors up to ganglion cells, that reproduce contrast adaptation and color opponency mechanisms in the retina. These models were fitted to published electro-physiological data to show the potential of this tool to generalize and adapt itself to a wide range of retina models.

Biophysical network modeling of the dLGN circuit: Effects of cortical feedback on spatial response properties of relay cells

Despite half-a-century of research since the seminal work of Hubel and Wiesel, the role of the dorsal lateral geniculate nucleus (dLGN) in shaping the visual signals is not properly understood. Placed on route from retina to primary visual cortex in the early visual pathway, a striking feature of the dLGN circuit is that both the relay cells (RCs) and interneurons (INs) not only receive feedforward input from retinal ganglion cells, but also a prominent feedback from cells in layer 6 of visual cortex. This feedback has been proposed to affect synchronicity and other temporal properties of the RC firing. It has also been seen to affect spatial properties such as the center-surround antagonism of thalamic receptive fields, i.e., the suppression of the response to very large stimuli compared to smaller, more optimal stimuli. Here we explore the spatial effects of cortical feedback on the RC response by means of a a comprehensive network model with biophysically detailed, single-compartment and multicompartment neuron models of RCs, INs and a population of orientation-selective layer 6 simple cells, consisting of pyramidal cells (PY). We have considered two different arrangements of synaptic feedback from the ON and OFF zones in the visual cortex to the dLGN: phase-reversed (‘push-pull’) and phase-matched (‘push-push’), as well as different spatial extents of the corticothalamic projection pattern. Our simulation results support that a phase-reversed arrangement provides a more effective way for cortical feedback to provide the increased center-surround antagonism seen in experiments both for flashing spots and, even more prominently, for patch gratings. This implies that ON-center RCs receive direct excitation from OFF-dominated cortical cells and indirect inhibitory feedback from ON-dominated cortical cells. The increased center-surround antagonism in the model is accompanied by spatial focusing, i.e., the maximum RC response occurs for smaller stimuli when feedback is present.

Low-cost Remote Monitoring of Biomedical Signals

The great usefulness of remote recording of biomedical signals in most aspects of daily life has generated an increasing interest in this field. Traditionally, monitoring devices from clinical enviroments are bulky, intrusive, and expensive. Thus, the development of wearable, mobile, and low-cost applications is desirable. Nevertheless, recent improvements in open-hardware allow developing low cost devices and portable designs for biosignal monitoring in out-of-lab applications, such as sports, leisure, e-Health, etc. This paper presents a low-cost wearable system able to simultaneously record electrical brain and heart activity (i.e. electroencephalography and electrocardiography). The system is able to send biomedical data to a platform for remote analyses. Both software and hardware are open-source. We assessed the system for its validity and reliability in a real road environment.

Modeling Retina Adaptation with Multiobjective Parameter Fitting

The retina continually adapts its kinetics, average response and sensitivity to the conditions of the environment. Retinal neurons adapt essentially to the mean light intensity and its temporal fluctuations over the mean, also called temporal contrast. Contrast adaptation has two distinct temporal expressions with fast and slow components. Here, we present a configurable retina simulation environment that accurately reproduces both contrast components. A contrast increase in the visual input accelerates kinetics of the filter, reduces sensitivity and depolarizes the membrane potential. Slow adaptation does not affect the temporal response but produces a progressive hyperpolarization of membrane potential. The implemented model for contrast adaptation provides a neural basis of each retinal stage, from photoreceptors up to ganglion cells, to explain the observed retina behavior. Both forms of contrast adaptation, fast and slow, are captured by a combined model of shunting feedback of bipolar cells and short-term plasticity (STP) at the bipolar-to-ganglion synapse. Biological accuracy of the model is evaluated by comparison of the measured neural response with the simulated response fitted to published physiological data. One problem with the simulated model is finding its optimal parameter settings, since the model response is described by a complex system of different retina stages with linear, nonlinear and feedback connections. We propose to use a multiobjective genetic optimization to automatically search the parameter space and easily find a feasible configuration solution.

Firing-rate based network modeling of the dLGN circuit: Effects of cortical feedback on spatiotemporal response properties of relay cells

Visually evoked signals in the retina pass through the dorsal geniculate nucleus (dLGN) on the way to the visual cortex. This is however not a simple feedforward flow of information: there is a significant feedback from cortical cells back to both relay cells and interneurons in the dLGN. Despite four decades of experimental and theoretical studies, the functional role of this feedback is still debated. Here we use a firing-rate model, the extended difference-of-Gaussians (eDOG) model, to explore cortical feedback effects on visual responses of dLGN relay cells. For this model the responses are found by direct evaluation of two- or three-dimensional integrals allowing for fast and comprehensive studies of putative effects of different candidate organizations of the cortical feedback. Our analysis identifies a special mixed configuration of excitatory and inhibitory cortical feedback which seems to best account for available experimental data. This configuration consists of (i) a slow (long-delay) and spatially widespread inhibitory feedback, combined with (ii) a fast (short-delayed) and spatially narrow excitatory feedback, where (iii) the excitatory/inhibitory ON-ON connections are accompanied respectively by inhibitory/excitatory OFF-ON connections, i.e. following a phase-reversed arrangement. The recent development of optogenetic and pharmacogenetic methods has provided new tools for more precise manipulation and investigation of the thalamocortical circuit, in particular for mice. Such data will expectedly allow the eDOG model to be better constrained by data from specific animal model systems than has been possible until now for cat. We have therefore made the Python tool pyLGN which allows for easy adaptation of the eDOG model to new situations.