Pierre W. Ferrez

A Brain-Actuated Wheelchair: Asynchronous and Non-Invasive Brain-Computer Interfaces for Continuous Control of Robots

OBJECTIVE To assess the feasibility and robustness of an asynchronous and non-invasive EEG-based Brain-Computer Interface (BCI) for continuous mental control of a wheelchair. METHODS In experiment 1 two subjects were asked to mentally drive both a real and a simulated wheelchair from a starting point to a goal along a pre-specified path. Here we only report experiments with the simulated wheelchair for which we have extensive data in a complex environment that allows a sound analysis. Each subject participated in five experimental sessions, each consisting of 10 trials. The time elapsed between two consecutive experimental sessions was variable (from 1h to 2months) to assess the system robustness over time. The pre-specified path was divided into seven stretches to assess the system robustness in different contexts. To further assess the performance of the brain-actuated wheelchair, subject 1 participated in a second experiment consisting of 10 trials where he was asked to drive the simulated wheelchair following 10 different complex and random paths never tried before. RESULTS In experiment 1 the two subjects were able to reach 100% (subject 1) and 80% (subject 2) of the final goals along the pre-specified trajectory in their best sessions. Different performances were obtained over time and path stretches, what indicates that performance is time and context dependent. In experiment 2, subject 1 was able to reach the final goal in 80% of the trials. CONCLUSIONS The results show that subjects can rapidly master our asynchronous EEG-based BCI to control a wheelchair. Also, they can autonomously operate the BCI over long periods of time without the need for adaptive algorithms externally tuned by a human operator to minimize the impact of EEG non-stationarities. This is possible because of two key components: first, the inclusion of a shared control system between the BCI system and the intelligent simulated wheelchair; second, the selection of stable user-specific EEG features that maximize the separability between the mental tasks. SIGNIFICANCE These results show the feasibility of continuously controlling complex robotics devices using an asynchronous and non-invasive BCI.

Error-Related EEG Potentials Generated During Simulated Brain–Computer Interaction

Brain-computer interfaces (BCIs) are prone to errors in the recognition of subjects intent. An elegant approach to improve the accuracy of BCIs consists in a verification procedure directly based on the presence of error-related potentials (ErrP) in the electroencephalogram (EEG) recorded right after the occurrence of an error. Several studies show the presence of ErrP in typical choice reaction tasks. However, in the context of a BCI, the central question is: ldquoAre ErrP also elicited when the error is made by the interface during the recognition of the subjects intent?rdquo We have thus explored whether ErrP also follow a feedback indicating incorrect responses of the simulated BCI interface. Five healthy volunteer subjects participated in a new human-robot interaction experiment, which seem to confirm the previously reported presence of a new kind of ErrP. However, in order to exploit these ErrP, we need to detect them in each single trial using a short window following the feedback associated to the response of the BCI. We have achieved an average recognition rate of correct and erroneous single trials of 83.5% and 79.2%, respectively, using a classifier built with data recorded up to three months earlier.

Towards a robust BCI: error potentials and online learning

Recent advances in the field of brain-computer interfaces (BCIs) have shown that BCIs have the potential to provide a powerful new channel of communication, completely independent of muscular and nervous systems. However, while there have been successful laboratory demonstrations, there are still issues that need to be addressed before BCIs can be used by nonexperts outside the laboratory. At IDIAP Research Institute, we have been investigating several areas that we believe will allow us to improve the robustness, flexibility, and reliability of BCIs. One area is recognition of cognitive error states, that is, identifying errors through the brains reaction to mistakes. The production of these error potentials (ErrP) in reaction to an error made by the user is well established. We have extended this work by identifying a similar but distinct ErrP that is generated in response to an error made by the interface, (a misinterpretation of a command that the user has given). This ErrP can be satisfactorily identified in single trials and can be demonstrated to improve the theoretical performance of a BCI. A second area of research is online adaptation of the classifier. BCI signals change over time, both between sessions and within a single session, due to a number of factors. This means that a classifier trained on data from a previous session will probably not be optimal for a new session. In this paper, we present preliminary results from our investigations into supervised online learning that can be applied in the initial training phase. We also discuss the future direction of this research, including the combination of these two currently separate issues to create a potentially very powerful BCI.

Adaptive Shared Control of a Brain-Actuated Simulated Wheelchair

The use of shared control techniques has a profound impact on the performance of a robotic assistant controlled by human brain signals. However, this shared control usually provides assistance to the user in a constant and identical manner each time. Creating an adaptive level of assistance, thereby complementing the users capabilities at any moment, would be more appropriate. The better the user can do by himself, the less assistance he receives from the shared control system; and vice versa. In order to do this, we need to be able to detect when and in what way the user needs assistance. An appropriate assisting behaviour would then be activated for the time the user requires help, thereby adapting the level of assistance to the specific situation. This paper presents such a system, helping a brain-computer interface (BCI) subject perform goal-directed navigation of a simulated wheelchair in an adaptive manner. Whenever the subject has more difficulties in driving the wheelchair, more assistance will be given. Experimental results of two subjects show that this adaptive shared control increases the task performance. Also, it shows that a subject with a lower BCI performance has more need for extra assistance in difficult situations, such as manoeuvring in a narrow corridor.

Context-based filtering for assisted brain-actuated wheelchair driving

Controlling a robotic device by using human brain signals is an interesting and challenging task. The device may be complicated to control and the nonstationary nature of the brain signals provides for a rather unstable input. With the use of intelligent processing algorithms adapted to the task at hand, however, the performance can be increased. This paper introduces a shared control system that helps the subject in driving an intelligent wheelchair with a noninvasive brain interface. The subjects steering intentions are estimated from electroencephalogram (EEG) signals and passed through to the shared control system before being sent to the wheelchair motors. Experimental results show a possibility for significant improvement in the overall driving performance when using the shared control system compared to driving without it. These results have been obtained with 2 healthy subjects during their first day of training with the brain-actuated wheelchair.

Non-Invasive Brain-Machine Interaction

The promise of Brain-Computer Interfaces (BCI) technology is to augment human capabilities by enabling interaction with computers through a conscious and spontaneous modulation of the brainwaves after a short training period. Indeed, by analyzing brain electrical activity online, several groups have designed brain-actuated devices that provide alternative channels for communication, entertainment and control. Thus, a person can write messages using a virtual keyboard on a computer screen and also browse the internet. Alternatively, subjects can operate simple computer games, or brain games, and interact with educational software. Work with humans has shown that it is possible for them to move a cursor and even to drive a wheelchair. This paper briefly reviews the field of BCI, with a focus on non-invasive systems based on electroencephalogram (EEG) signals. It also describes three brain-actuated devices we have developed: a virtual keyboard, a brain game, and a wheelchair. Finally, it shortly discusses current research directions we are pursuing in order to improve the performance and robustness of our BCI system, especially for real-time control of brain actuated robots.

The use of brain-computer interfacing for ambient intelligence

This book constitutes the refereed proceedings of the workshops of the First European Conference on Ambient Intelligence, AmI 2007, held in Darmstadt, Germany, in November 2007. The papers are organized in topical sections on AI methods for ambient intelligence, evaluating ubiquitous systems with users, model driven software engineering for ambient intelligence applications, smart products, ambient assisted living, human aspects in ambient intelligence, Amigo, WASP as well as the cojoint PERSONA and SOPRANO workshops and the KDubiq workshop.

Feature Extraction for Multi-class BCI using Canonical Variates Analysis

To propose a new feature extraction method with canonical solution for multi-class brain-computer interfaces (BCI). The proposed method should provide a reduced number of canonical discriminant spatial patterns (CDSP) and rank the channels sorted by power discriminability (DP) between classes. The feature extractor relays in canonical variates analysis (CVA) which provides the CDSP between the classes. The number of CDSP is equal to the number of classes minus one. We analyze EEG data recorded with 64 electrodes from 4 subjects recorded in 20 sessions. They were asked to execute twice in each session three different mental tasks (left hand imagination movement, rest, and words association) during 7 seconds. A ranking of electrodes sorted by power discriminability between classes and the CDSP were computed. After splitting data in training and test sets, we compared the classification accuracy achieved by linear discriminant analysis (LDA) in frequency and temporal domains. The average LDA classification accuracies over the four subjects using CVA on both domains are equivalent (57.89% in frequency domain and 59.43% in temporal domain). These results, in terms of classification accuracies, are also reflected in the similarity between the ranking of relevant channels in both domains. CVA is a simple feature extractor with canonical solution useful for multi-class BCI applications that can work on temporal or frequency domain.

Non-Invasive Estimation of Local Field Potentials for Neuroprosthesis Control

Recent experiments have shown the possibility of using the brain electrical activity to directly control the movement of robots or prosthetic devices in real time. Such neuroprostheses can be invasive or non-invasive, depending on how the brain signals are recorded. In principle, invasive approaches will provide a more natural and flexible control of neuroprostheses, but their use in humans is debatable given the inherent medical risks. Non-invasive approaches mainly use scalp electroencephalogram (EEG) signals and their main disadvantage is that these signals represent the noisy spatiotemporal overlapping of activity arising from very diverse brain regions, i.e., a single scalp electrode picks up and mixes the temporal activity of myriads of neurons at very different brain areas. In order to combine the benefits of both approaches, we propose to rely on the non-invasive estimation of local field potentials (LFP) in the whole human brain from the scalp measured EEG data using a recently developed inverse solution (ELECTRA) to the EEG inverse problem. The goal of a linear inverse procedure is to de-convolve or un-mix the scalp signals attributing to each brain area its own temporal activity. To illustrate the advantage of this approach we compare, using an identical set of spectral features, classification of rapid voluntary finger self-tapping with left and right hands based on scalp EEG and non-invasively estimated LFP on two subjects using a different number of electrodes.

Validation of Brain-Machine Interfaces during Parabolic Flight

Here we report on a validation study on brain-machine interfaces (BMIs) performed during the December 2007 ESA parabolic flight campaign. We investigated the feasibility of using BMIs for space applications by performing tests in microgravity. Brain signals were recorded with noninvasive electroencephalography before (calibration sessions) and during the parabolic flights on two subjects with prior BMI experience. The results of our experiments show that an experienced BMI user can achieve stable performance in all gravity conditions examined and, hence, demonstrate the feasibility of operating noninvasive BMIs in space.