J. del R. Millan

A Brain-Actuated Wheelchair: Asynchronous and Non-Invasive Brain-Computer Interfaces for Continuous Control of Robots

OBJECTIVE To assess the feasibility and robustness of an asynchronous and non-invasive EEG-based Brain-Computer Interface (BCI) for continuous mental control of a wheelchair. METHODS In experiment 1 two subjects were asked to mentally drive both a real and a simulated wheelchair from a starting point to a goal along a pre-specified path. Here we only report experiments with the simulated wheelchair for which we have extensive data in a complex environment that allows a sound analysis. Each subject participated in five experimental sessions, each consisting of 10 trials. The time elapsed between two consecutive experimental sessions was variable (from 1h to 2months) to assess the system robustness over time. The pre-specified path was divided into seven stretches to assess the system robustness in different contexts. To further assess the performance of the brain-actuated wheelchair, subject 1 participated in a second experiment consisting of 10 trials where he was asked to drive the simulated wheelchair following 10 different complex and random paths never tried before. RESULTS In experiment 1 the two subjects were able to reach 100% (subject 1) and 80% (subject 2) of the final goals along the pre-specified trajectory in their best sessions. Different performances were obtained over time and path stretches, what indicates that performance is time and context dependent. In experiment 2, subject 1 was able to reach the final goal in 80% of the trials. CONCLUSIONS The results show that subjects can rapidly master our asynchronous EEG-based BCI to control a wheelchair. Also, they can autonomously operate the BCI over long periods of time without the need for adaptive algorithms externally tuned by a human operator to minimize the impact of EEG non-stationarities. This is possible because of two key components: first, the inclusion of a shared control system between the BCI system and the intelligent simulated wheelchair; second, the selection of stable user-specific EEG features that maximize the separability between the mental tasks. SIGNIFICANCE These results show the feasibility of continuously controlling complex robotics devices using an asynchronous and non-invasive BCI.

Error-Related EEG Potentials Generated During Simulated Brain–Computer Interaction

Brain-computer interfaces (BCIs) are prone to errors in the recognition of subjects intent. An elegant approach to improve the accuracy of BCIs consists in a verification procedure directly based on the presence of error-related potentials (ErrP) in the electroencephalogram (EEG) recorded right after the occurrence of an error. Several studies show the presence of ErrP in typical choice reaction tasks. However, in the context of a BCI, the central question is: ldquoAre ErrP also elicited when the error is made by the interface during the recognition of the subjects intent?rdquo We have thus explored whether ErrP also follow a feedback indicating incorrect responses of the simulated BCI interface. Five healthy volunteer subjects participated in a new human-robot interaction experiment, which seem to confirm the previously reported presence of a new kind of ErrP. However, in order to exploit these ErrP, we need to detect them in each single trial using a short window following the feedback associated to the response of the BCI. We have achieved an average recognition rate of correct and erroneous single trials of 83.5% and 79.2%, respectively, using a classifier built with data recorded up to three months earlier.

Brain-Controlled Wheelchairs: A Robotic Architecture

Independent mobility is central to being able to perform activities of daily living by oneself. However, power wheelchairs are not an option for many people who, due to severe motor disabilities, are unable to use conventional controls. For some of these people, noninvasive brain-computer interfaces (BCIs) offer a promising solution to this interaction problem.

Efficient learning of variable-resolution cognitive maps for autonomous indoor navigation

This paper presents an adaptive method that allows mobile robots to learn cognitive maps of indoor environments incrementally and online. Our approach models the environment. By means of a variable-resolution partitioning that discretizes the world in perceptually homogeneous regions. The resulting model incorporates both a compact geometrical representation of the environment and a topological map of the spatial relationships between its obstacle-free areas. The efficiency of the learning process is based on the use of local memory-based techniques for partitioning and of active learning techniques for selecting the most appropriate region to be explored next. In addition, a feedforward neural network is used to interpret sensor readings. We present experimental results obtained with two different mobile robots, the Nomad 200 and Khepera. The current implementation of the method relies on the assumption that obstacles are parallel or perpendicular to each other. This results in variable-resolution partitioning consisting of simple rectangular partitions and reduces the complexity of treating the underlying geometrical properties.

EEG-based Brain-Computer Interface to support post-stroke motor rehabilitation of the upper limb

Brain-Computer Interfaces (BCIs) process brain activity in real time, and mediate non-muscular interaction between and individual and the environment. The subserving algorithms can be used to provide a quantitative measurement of physiological or pathological cognitive processes - such as Motor Imagery (MI) - and feed it back the user. In this paper we propose the clinical application of a BCI-based rehabilitation device, to promote motor recovery after stroke. The BCI-based device and the therapy exploiting its use follow the same principles that drive classical neuromotor rehabilitation, and (i) provides the physical therapist with a monitoring instrument, to assess the patients participation in the rehabilitative cognitive exercise; (ii) assists the patient in the practice of MI. The device was installed in the ward of a rehabilitation hospital and a group of 29 patients were involved in its testing. Among them, eight have already undergone a one month training with the device, as an add-on to the regular therapy. An improved system, which includes analysis of Electromyographic (EMG) patterns and Functional Electrical Stimulation (FES) of the arm muscles, is also under clinical evaluation. We found that the rehabilitation exercise based on BCI mediated neurofeedback mechanisms enables a better engagement of motor areas with respect to motor imagery alone and thus it can promote neuroplasticity in brain regions affected by a cerebrovascular accident. Preliminary results also suggest that the functional outcome of motor rehabilitation may be improved by the use of the proposed device.

Feature Extraction for Multi-class BCI using Canonical Variates Analysis

To propose a new feature extraction method with canonical solution for multi-class brain-computer interfaces (BCI). The proposed method should provide a reduced number of canonical discriminant spatial patterns (CDSP) and rank the channels sorted by power discriminability (DP) between classes. The feature extractor relays in canonical variates analysis (CVA) which provides the CDSP between the classes. The number of CDSP is equal to the number of classes minus one. We analyze EEG data recorded with 64 electrodes from 4 subjects recorded in 20 sessions. They were asked to execute twice in each session three different mental tasks (left hand imagination movement, rest, and words association) during 7 seconds. A ranking of electrodes sorted by power discriminability between classes and the CDSP were computed. After splitting data in training and test sets, we compared the classification accuracy achieved by linear discriminant analysis (LDA) in frequency and temporal domains. The average LDA classification accuracies over the four subjects using CVA on both domains are equivalent (57.89% in frequency domain and 59.43% in temporal domain). These results, in terms of classification accuracies, are also reflected in the similarity between the ranking of relevant channels in both domains. CVA is a simple feature extractor with canonical solution useful for multi-class BCI applications that can work on temporal or frequency domain.

Fast Recognition of Anticipation-Related Potentials

Anticipation increases the efficiency of daily tasks by partial advance activation of neural substrates involved in it. Here, we develop a method for the recognition of EEG correlates of this activation as early as possible on single trials, which is essential for brain-computer interaction. We explore various features from the EEG recorded in a contingent negative variation (CNV) paradigm. We also develop a novel technique called time aggregation of classification (TAC) for fast and reliable decisions that combines the posterior probabilities of several classifiers trained with features computed from temporal blocks of EEG until a certainty threshold is reached. Experiments with nine naive subjects performing the CNV experiment with GO (anticipation) and NOGO (control) conditions with an interstimulus interval of 4 s show that the performance of the TAC method is above 70% for four subjects, around 60% for two other subjects, and random for the remaining subjects. On average over all subjects, more than 50% of the correct decisions are made at 2 s, without needing to wait until 4 s.

Characterizing the EEG Correlates of Exploratory Behavior

This study aims to characterize the electroencephalography (EEG) correlates of exploratory behavior. Decision making in an uncertain environment raises a conflict between two opposing needs: gathering information about the environment and exploiting this knowledge in order to optimize the decision. Exploratory behavior has already been studied using functional magnetic resonance imaging (fMRI). Based on a usual paradigm in reinforcement learning, this study has shown bilateral activation in the frontal and parietal cortex. To our knowledge, no previous study has been done on it using EEG. The study of the exploratory behavior using EEG signals raises two difficulties. First, the labels of trial as exploitation or exploration cannot be directly derived from the subject action. In order to access this information, a model of how the subject makes his decision must be built. The exploration related information can be then derived from it. Second, because of the complexity of the task, its EEG correlates are not necessarily time locked with the action. So the EEG processing methods used should be designed in order to handle signals that shift in time across trials. Using the same experimental protocol as the fMRI study, results show that the bilateral frontal and parietal areas are also the most discriminant. This strongly suggests that the EEG signal also conveys information about the exploratory behavior.

A comparative psychophysical and EEG study of different feedback modalities for HRI

This paper presents a comparison between six different ways to convey navigational information provided by a robot to a human. Visual, auditory, and tactile feedback modalities were selected and designed to suggest a direction of travel to a human user, who can then decide if he agrees or not with the robots proposition. This work builds upon a previous research on a novel semi-autonomous navigation system in which the human supervises an autonomous system, providing corrective monitoring signals whenever necessary. We recorded both qualitative (user impressions based on selected criteria and ranking of their feelings) and quantitative (response time and accuracy) information regarding different types of feedback. In addition, a preliminary analysis of the influence of the different types of feedback on brain activity is also shown. The result of this study may provide guidelines for the design of such a human-robot interaction system, depending on both the task and the human user.

Single trial recognition of anticipatory slow cortical potentials: The role of spatio-spectral filtering

Single trial recognition of slow cortical potentials (SCPs) from full-band EEG (FbEEG) faces different challenges to classical EEG such as noisy, high magnitude (~ ±100 μV) infra slow oscillations (ISO) with f ≤ 0.1 Hz and high frequency spatial noise from a variety of artifacts. We analyze offline the anticipation related SCPs recorded from 11 subjects over two days in a variation of the Contingent Negative Variation (CNV) paradigm with Go and No-go conditions in an assistive technology framework. The results suggest that widely used spatial filters such as Common Average Referencing (CAR) and Laplacian are sub-optimal for the single trial analysis of SCPs. We show that a spatial smoothing filter (SSF), which in combination with CAR enhances the spatially distributed SCP while attenuating high frequency spatial noise. We report, first, that a narrow band filter in the range [0.1 1] Hz captures anticipation related SCP better and effectively reduces ISOs. Second, the SSF in combination with CAR outperforms CAR-alone and Laplacian spatial filters. Third, we compare linear and quadratic classifiers calculated using optimally filtered Cz electrode potentials and report that the best methods resulted in single trial classification accuracies of 83 ±4%, where classifiers were trained on day 1 and tested using data from day 2, to ensure generalization capabilities across days (1-7 days).