Samuel F. Romero

Toward the development of a cortically based visual neuroprosthesis

Motivated by the success of cochlear implants for deaf patients, we are now facing the goal of creating a visual neuroprosthesis designed to interface with the occipital cortex as a means through which a limited but useful sense of vision could be restored in profoundly blind patients. We review the most important challenges regarding this neuroprosthetic approach and emphasize the need for basic human psychophysical research on the best way of presenting complex stimulating patterns through multiple microelectrodes. Continued research will hopefully lead to the development of and design specifications for the first generation of a cortically based visual prosthesis system.

Development of a cortical visual neuroprosthesis for the blind: the relevance of neuroplasticity

Clinical applications such as artificial vision require extraordinary, diverse, lengthy and intimate collaborations among basic scientists, engineers and clinicians. In this review, we present the state of research on a visual neuroprosthesis designed to interface with the occipital visual cortex as a means through which a limited, but useful, visual sense could be restored in profoundly blind individuals. We review the most important physiological principles regarding this neuroprosthetic approach and emphasize the role of neural plasticity in order to achieve desired behavioral outcomes. While full restoration of fine detailed vision with current technology is unlikely in the immediate near future, the discrimination of shapes and the localization of objects should be possible allowing blind subjects to navigate in a unfamiliar environment and perhaps even to read enlarged text. Continued research and development in neuroprosthesis technology will likely result in a substantial improvement in the quality of life of blind and visually impaired individuals.

An auditory brain–computer interface evoked by natural speech

Brain–computer interfaces (BCIs) are mainly intended for people unable to perform any muscular movement, such as patients in a complete locked-in state. The majority of BCIs interact visually with the user, either in the form of stimulation or biofeedback. However, visual BCIs challenge their ultimate use because they require the subjects to gaze, explore and shift eye-gaze using their muscles, thus excluding patients in a complete locked-in state or under the condition of the unresponsive wakefulness syndrome. In this study, we present a novel fully auditory EEG-BCI based on a dichotic listening paradigm using human voice for stimulation. This interface has been evaluated with healthy volunteers, achieving an average information transmission rate of 1.5 bits min⁻¹ in full-length trials and 2.7 bits min⁻¹ using the optimal length of trials, recorded with only one channel and without formal training. This novel technique opens the door to a more natural communication with users unable to use visual BCIs, with promising results in terms of performance, usability, training and cognitive effort.

Translating image sequences into spike patterns for cortical neuro-stimulation

Abstract This paper describes a bioinspired preprocessing and coding system devised for producing optimal multi-electrode stimulation at the cortical level, starting from image sequences and working at video rates. A hybrid platform with software and reconfigurable hardware delivers a continuously varying stream of pulses or spike patterns. The main objective of this work is to build a portable system for a visual neuro-prosthesis to stimulate efficiently an array of intra-cortical implanted microelectrodes. A set of parameters can be adjusted in the processing and spike-coding modules to trade-off their technology constraints with the biological plausibility of their functional features.

Cortical visual neuro-prosthesis for the blind: retina-like software/hardware preprocessor

The paper describes the implementation of a software/hardware (Sw/Hw) retina-like preprocessor designed for the study of bio-inspired visual processing schemes, and for experimentation in the development of a cortical visual neuro-prosthesis. The goal is to obtain, in the near future, an integrated artificial retina for a visual neuro-prosthesis prototype able to stimulate efficiently an array of intra-cortical implanted microelectrodes. The Sw/Hw platform presented is highly programmable and reconfigurable as required for experimentation, providing real-time video processing and neuromorphic stimulation. But its implementation is also devised to be finally mapped into fully portable hardware, particularized for each implanted blind individual.

A neuroengineering suite of computational tools for visual prostheses

The cooperation between neuroscience and biomedical engineering gave rise to a recent, but growing research field, known as neuroengineering. We follow its principles to have a system providing basic descriptions of the visual world to the brains cortex. We describe a set of software and hardware tools to interface with neural tissue, in order to transmit visual information encoded into a bioinspired neural-like form. The set is composed of a retina-like encoder, and a platform to optimize electrical stimulation parameters for a multi-electrode implant. The main objective is to progress towards a functional visual neuroprosthesis for the blind.

Automatic generation of bio-inspired retina-like processing hardware

This paper describes a tool devised for automatic design of bioinspired visual processing models using reconfigurable digital hardware. The whole system is indicated for the analysis of vision models, especially those with real–time requirements. We achieve a synthesizable FPGA/ASIC design starting from a high level description of a retina, which is made and simulated through an ad-hoc program. Our tool allows a thorough simulation of the visual model at different abstraction levels, from functional simulation of the visual specifications up to hardware-oriented simulation of the developed FPGA model. The main objective of this work is to build a portable and flexible system for a visual neuro-prosthesis and to stimulate efficiently an array of intra–cortical implanted microelectrodes. A set of parameters can be adjusted in every step of the design flow in order to maximize the design flexibility of the model. Furthermore these parameters allow the different scientists who have to deal with the development to modify a well known characteristic.

A computational tool to test neuromorphic encoding schemes for visual neuroprostheses

Recent advances in arrays of microelectrodes open the door to both better understanding of the way the brain works and to the restoration of damaged perceptive and motor functions. In the case of sensorial inputs, direct multi-channel interfacing with the brain for neuro-stimulation requires a computational layer capable of handling the translation from external stimuli into appropriate trains of spikes. The work here presented aims to provide automated and reconfigurable transformation of visual inputs into addresses of microelectrodes in a cortical implant for the blind. The development of neuroprostheses such as this one will contribute to reveal the neural language of the brain for the representation of perceptions, and offers a hope to persons with deep visual impairments. Our system serves as a highly flexible test-bench for almost any kind of retina model, and allows the validation of these models against multi-electrode recordings from experiments with biological retinas. The current version is a PC-based platform, and a compact stand-alone device is under development for the autonomy and portability required in chronic implants. This tool is useful for psychologists, neurophysiologists, and neural engineers as it offers a way to deal with the complexity of multi-channel electrical interfaces for the brain.

VIS2SOUND on Reconfigurable Hardware

This work describes the implementation of a bio-inspired visual processing system on configurable logic, designed for the enhancement of relevant information on a real scene, for its use on a complete system to assist the visually-impaired. For this purpose, a sensorial transduction module has been developed, which transforms visual information into sound patterns with 3D spatial localization. The requirements for this application, as portability, constrained power consumption, real-time performance, and customization to fit the specific features of the patientpsilas affection, lead us to consider programmable logic as the best option for the design and development of the whole system, specifically FPGAs. A prototype of the system, including two video inputs with three spatial filters, two weighed sums, a disparity computation module, and the 3D sound generator, has been implemented on a Virtex-II XC2V6000, and performs at a maximum rate of 60 frames per second.

Monitoring driver fatigue using a single-channel electroencephalographic device: a validation study by gaze-based, driving performance, and subjective data

Driver fatigue can impair performance as much as alcohol does. It is the most important road safety concern, causing thousands of accidents and fatalities every year. Thanks to technological developments, wearable, single-channel EEG devices are now getting considerable attention as fatigue monitors, as they could help drivers to assess their own levels of fatigue and, therefore, prevent the deterioration of performance. However, the few studies that have used single-channel EEG devices to investigate the physiological effects of driver fatigue have had inconsistent results, and the question of whether we can monitor driver fatigue reliably with these EEG devices remains open. Here, we assessed the validity of a single-channel EEG device (TGAM-based chip) to monitor changes in mental state (from alertness to fatigue). Fifteen drivers performed a 2-h simulated driving task while we recorded, simultaneously, their prefrontal brain activity and saccadic velocity. We used saccadic velocity as the reference index of fatigue. We also collected subjective ratings of alertness and fatigue, as well as driving performance. We found that the power spectra of the delta EEG band showed an inverted U-shaped quadratic trend (EEG power spectra increased for the first hour and half, and decreased during the last thirty minutes), while the power spectra of the beta band linearly increased as the driving session progressed. Coherently, saccadic velocity linearly decreased and speeding time increased, suggesting a clear effect of fatigue. Subjective data corroborated these conclusions. Overall, our results suggest that the TGAM-based chip EEG device is able to detect changes in mental state while performing a complex and dynamic everyday task as driving.