Ferran Galán

A Brain-Actuated Wheelchair: Asynchronous and Non-Invasive Brain-Computer Interfaces for Continuous Control of Robots

OBJECTIVE To assess the feasibility and robustness of an asynchronous and non-invasive EEG-based Brain-Computer Interface (BCI) for continuous mental control of a wheelchair. METHODS In experiment 1 two subjects were asked to mentally drive both a real and a simulated wheelchair from a starting point to a goal along a pre-specified path. Here we only report experiments with the simulated wheelchair for which we have extensive data in a complex environment that allows a sound analysis. Each subject participated in five experimental sessions, each consisting of 10 trials. The time elapsed between two consecutive experimental sessions was variable (from 1h to 2months) to assess the system robustness over time. The pre-specified path was divided into seven stretches to assess the system robustness in different contexts. To further assess the performance of the brain-actuated wheelchair, subject 1 participated in a second experiment consisting of 10 trials where he was asked to drive the simulated wheelchair following 10 different complex and random paths never tried before. RESULTS In experiment 1 the two subjects were able to reach 100% (subject 1) and 80% (subject 2) of the final goals along the pre-specified trajectory in their best sessions. Different performances were obtained over time and path stretches, what indicates that performance is time and context dependent. In experiment 2, subject 1 was able to reach the final goal in 80% of the trials. CONCLUSIONS The results show that subjects can rapidly master our asynchronous EEG-based BCI to control a wheelchair. Also, they can autonomously operate the BCI over long periods of time without the need for adaptive algorithms externally tuned by a human operator to minimize the impact of EEG non-stationarities. This is possible because of two key components: first, the inclusion of a shared control system between the BCI system and the intelligent simulated wheelchair; second, the selection of stable user-specific EEG features that maximize the separability between the mental tasks. SIGNIFICANCE These results show the feasibility of continuously controlling complex robotics devices using an asynchronous and non-invasive BCI.

Adaptive Shared Control of a Brain-Actuated Simulated Wheelchair

The use of shared control techniques has a profound impact on the performance of a robotic assistant controlled by human brain signals. However, this shared control usually provides assistance to the user in a constant and identical manner each time. Creating an adaptive level of assistance, thereby complementing the users capabilities at any moment, would be more appropriate. The better the user can do by himself, the less assistance he receives from the shared control system; and vice versa. In order to do this, we need to be able to detect when and in what way the user needs assistance. An appropriate assisting behaviour would then be activated for the time the user requires help, thereby adapting the level of assistance to the specific situation. This paper presents such a system, helping a brain-computer interface (BCI) subject perform goal-directed navigation of a simulated wheelchair in an adaptive manner. Whenever the subject has more difficulties in driving the wheelchair, more assistance will be given. Experimental results of two subjects show that this adaptive shared control increases the task performance. Also, it shows that a subject with a lower BCI performance has more need for extra assistance in difficult situations, such as manoeuvring in a narrow corridor.

Non-Invasive Brain-Machine Interaction

The promise of Brain-Computer Interfaces (BCI) technology is to augment human capabilities by enabling interaction with computers through a conscious and spontaneous modulation of the brainwaves after a short training period. Indeed, by analyzing brain electrical activity online, several groups have designed brain-actuated devices that provide alternative channels for communication, entertainment and control. Thus, a person can write messages using a virtual keyboard on a computer screen and also browse the internet. Alternatively, subjects can operate simple computer games, or brain games, and interact with educational software. Work with humans has shown that it is possible for them to move a cursor and even to drive a wheelchair. This paper briefly reviews the field of BCI, with a focus on non-invasive systems based on electroencephalogram (EEG) signals. It also describes three brain-actuated devices we have developed: a virtual keyboard, a brain game, and a wheelchair. Finally, it shortly discusses current research directions we are pursuing in order to improve the performance and robustness of our BCI system, especially for real-time control of brain actuated robots.

The use of brain-computer interfacing for ambient intelligence

This book constitutes the refereed proceedings of the workshops of the First European Conference on Ambient Intelligence, AmI 2007, held in Darmstadt, Germany, in November 2007. The papers are organized in topical sections on AI methods for ambient intelligence, evaluating ubiquitous systems with users, model driven software engineering for ambient intelligence applications, smart products, ambient assisted living, human aspects in ambient intelligence, Amigo, WASP as well as the cojoint PERSONA and SOPRANO workshops and the KDubiq workshop.

Feature Extraction for Multi-class BCI using Canonical Variates Analysis

To propose a new feature extraction method with canonical solution for multi-class brain-computer interfaces (BCI). The proposed method should provide a reduced number of canonical discriminant spatial patterns (CDSP) and rank the channels sorted by power discriminability (DP) between classes. The feature extractor relays in canonical variates analysis (CVA) which provides the CDSP between the classes. The number of CDSP is equal to the number of classes minus one. We analyze EEG data recorded with 64 electrodes from 4 subjects recorded in 20 sessions. They were asked to execute twice in each session three different mental tasks (left hand imagination movement, rest, and words association) during 7 seconds. A ranking of electrodes sorted by power discriminability between classes and the CDSP were computed. After splitting data in training and test sets, we compared the classification accuracy achieved by linear discriminant analysis (LDA) in frequency and temporal domains. The average LDA classification accuracies over the four subjects using CVA on both domains are equivalent (57.89% in frequency domain and 59.43% in temporal domain). These results, in terms of classification accuracies, are also reflected in the similarity between the ranking of relevant channels in both domains. CVA is a simple feature extractor with canonical solution useful for multi-class BCI applications that can work on temporal or frequency domain.

Using mental tasks transitions detection to improve spontaneous mental activity classification

This paper presents an algorithm based on canonical variates transformation (CVT) and distance based discriminant analysis (DBDA) combined with a mental tasks transitions detector (MTTD) to classify spontaneous mental activities in order to operate a brain-computer interface working under an asynchronous protocol. The algorithm won the BCI Competition III -Data Set V: Multiclass Problem, Continous EEG- achieving an averaged classification accuracy over three subjects of 68.65% (79.60, 70.31 and 56.02%, respectively) in a three-class problem.

Characterizing the EEG Correlates of Exploratory Behavior

This study aims to characterize the electroencephalography (EEG) correlates of exploratory behavior. Decision making in an uncertain environment raises a conflict between two opposing needs: gathering information about the environment and exploiting this knowledge in order to optimize the decision. Exploratory behavior has already been studied using functional magnetic resonance imaging (fMRI). Based on a usual paradigm in reinforcement learning, this study has shown bilateral activation in the frontal and parietal cortex. To our knowledge, no previous study has been done on it using EEG. The study of the exploratory behavior using EEG signals raises two difficulties. First, the labels of trial as exploitation or exploration cannot be directly derived from the subject action. In order to access this information, a model of how the subject makes his decision must be built. The exploration related information can be then derived from it. Second, because of the complexity of the task, its EEG correlates are not necessarily time locked with the action. So the EEG processing methods used should be designed in order to handle signals that shift in time across trials. Using the same experimental protocol as the fMRI study, results show that the bilateral frontal and parietal areas are also the most discriminant. This strongly suggests that the EEG signal also conveys information about the exploratory behavior.

Visuo-Spatial Attention Frame Recognition for Brain-Computer Interfaces

Objective: To assess the feasibility of recognizing visual spatial attention frames for Brain-computer interfaces (BCI) applications. Methods: EEG data was recorded with 64 electrodes from 2 subjects executing a visual spatial attention task indicating 2 target locations. Continuous Morlet wavelet coefficients were estimated on 18 frequency components and 16 preselected electrodes in trials of 600 ms. The spatial patterns of the 16 frequency components frames were simultaneously detected and classified (between the two targets). The classification accuracy was assessed using 20-fold crossvalidation. Results: The maximum frames average classification accuracies are 80.64% and 87.31% for subject 1 and 2 respectively, both utilizing coefficients estimated at frequencies located in gamma band.

Non-invasive brain-actuated interaction

The promise of Brain-Computer Interfaces (BCI) technology is to augment human capabilities by enabling interaction with computers through a conscious and spontaneous modulation of the brainwaves after a short training period. Indeed, by analyzing brain electrical activity online, several groups have designed brain-actuated devices that provide alternative channels for communication, entertainment and control. Thus, a person can write messages using a virtual keyboard on a computer screen and also browse the internet. Alternatively, subjects can operate simple computer games, or brain games, and interact with educational software. Work with humans has shown that it is possible for them to move a cursor and even to drive a wheelchair. This paper briefly reviews the field of BCI, with a focus on non-invasive systems based on electroencephalogram (EEG) signals. It also describes three brain-actuated devices we have developed: a virtual keyboard, a brain game, and a wheelchair. Finally, it shortly discusses current research directions we are pursuing in order to improve the performance and robustness of our BCI system, especially for real-time control of brain-actuated robots.