Claudia Sannelli

The Berlin Brain–Computer Interface: Non-Medical Uses of BCI Technology

Brain–computer interfacing (BCI) is a steadily growing area of research. While initially BCI research was focused on applications for paralyzed patients, increasingly more alternative applications in healthy human subjects are proposed and investigated. In particular, monitoring of mental states and decoding of covert user states have seen a strong rise of interest. Here, we present some examples of such novel applications which provide evidence for the promising potential of BCI technology for non-medical uses. Furthermore, we discuss distinct methodological improvements required to bring non-medical applications of BCI technology to a diversity of layperson target groups, e.g., ease of use, minimal training, general usability, short control latencies.

Machine-learning-based coadaptive calibration for brain-computer interfaces

Brain-computer interfaces (BCIs) allow users to control a computer application by brain activity as acquired (e.g., by EEG). In our classic machine learning approach to BCIs, the participants undertake a calibration measurement without feedback to acquire data to train the BCI system. After the training, the user can control a BCI and improve the operation through some type of feedback. However, not all BCI users are able to perform sufficiently well during feedback operation. In fact, a nonnegligible portion of participants (estimated 1530) cannot control the system (a BCI illiteracy problem, generic to all motor-imagery-based BCIs). We hypothesize that one main difficulty for a BCI user is the transition from offline calibration to online feedback. In this work, we investigate adaptive machine learning methods to eliminate offline calibration and analyze the performance of 11 volunteers in a BCI based on the modulation of sensorimotor rhythms. We present an adaptation scheme that individually guides the user. It starts with a subject-independent classifier that evolves to a subject-optimized state-of-the-art classifier within one session while the user interacts continuously. These initial runs use supervised techniques for robust coadaptive learning of user and machine. Subsequent runs use unsupervised adaptation to track the features drift during the session and provide an unbiased measure of BCI performance. Using this approach, without any offline calibration, six users, including one novice, obtained good performance after 3 to 6 minutes of adaptation. More important, this novel guided learning also allows participants with BCI illiteracy to gain significant control with the BCI in less than 60 minutes. In addition, one volunteer without sensorimotor idle rhythm peak at the beginning of the BCI experiment developed it during the course of the session and used voluntary modulation of its amplitude to control the feedback application.

Co-adaptive calibration to improve BCI efficiency

All brain-computer interface (BCI) groups that have published results of studies involving a large number of users performing BCI control based on the voluntary modulation of sensorimotor rhythms (SMR) report that BCI control could not be achieved by a non-negligible number of subjects (estimated 20% to 25%). This failure of the BCI system to read the intention of the user is one of the greatest problems and challenges in BCI research. There are two main causes for this problem in SMR-based BCI systems: either no idle SMR is observed over motor areas of the user, or this idle rhythm is not modulated during motor imagery, resulting in a classification performance lower than 70% (criterion level) that renders the control of a BCI application (like a speller) difficult or impossible. Previously, we introduced the concept of machine learning based co-adaptive calibration, which provided substantially improved performance for a variety of users. Here, we use a similar approach and investigate to what extent co-adaptive learning enables significant BCI control for completely novice users, as well as for those who could not achieve control with a conventional SMR-based BCI.

Large-scale EEG/MEG source localization with spatial flexibility.

We propose a novel approach to solving the electro-/magnetoencephalographic (EEG/MEG) inverse problem which is based upon a decomposition of the current density into a small number of spatial basis fields. It is designed to recover multiple sources of possibly different extent and depth, while being invariant with respect to phase angles and rotations of the coordinate system. We demonstrate the methods ability to reconstruct simulated sources of random shape and show that the accuracy of the recovered sources can be increased, when interrelated field patterns are co-localized. Technically, this leads to large-scale mathematical problems, which are solved using recent advances in convex optimization. We apply our method for localizing brain areas involved in different types of motor imagery using real data from Brain-Computer Interface (BCI) sessions. Our approach based on single-trial localization of complex Fourier coefficients yields class-specific focal sources in the sensorimotor cortices.

On Optimal Channel Configurations for SMR-based Brain–Computer Interfaces

One crucial question in the design of electroencephalogram (EEG)-based brain–computer interface (BCI) experiments is the selection of EEG channels. While a setup with few channels is more convenient and requires less preparation time, a dense placement of electrodes provides more detailed information and henceforth could lead to a better classification performance. Here, we investigate this question for a specific setting: a BCI that uses the popular CSP algorithm in order to classify voluntary modulations of sensorimotor rhythms (SMR). In a first approach 13 different fixed channel configurations are compared to the full one consisting of 119 channels. The configuration with 48 channels results to be the best one, while configurations with less channels, from 32 to 8, performed not significantly worse than the best configuration in cases where only few training trials are available. In a second approach an optimal channel configuration is obtained by an iterative procedure in the spirit of stepwise variable selection with nonparametric multiple comparisons. As a surprising result, in the second approach a setting with 22 channels centered over the motor areas was selected. Thanks to the acquisition of a large data set recorded from 80 novice participants using 119 EEG channels, the results of this study can be expected to have a high degree of generalizability.

Predicting BCI performance to study BCI illiteracy

a Berlin Institute of Technology, Machine Learning Laboratory, Berlin, Germany b Fraunhofer FIRST IDA, Berlin, Germany c Dept. of Neurology, Campus Benjamin Franklin, Charite University Medicine Berlin, Germany d Institute of Medical Psychology and Behavioral Neurobiology, Universitat Tubingen, Germany e Department of Biological Psychology, Clinical Psychology and Psychotherapy, University of Wurzburg, Germany

Pre-Stimulus Sensorimotor Rhythms Influence Brain–Computer Interface Classification Performance

The influence of pre-stimulus ongoing brain activity on post-stimulus task performance has recently been analyzed in several studies. While pre-stimulus activity in the parieto-occipital area has been exhaustively investigated with congruent results, less is known about the sensorimotor areas, for which studies reported inconsistent findings. In this work, the topic is addressed in a brain-computer interface (BCI) setting based on modulations of sensorimotor rhythms (SMR). The goal is to assess whether and how pre-stimulus SMR activity influences the successive task execution quality and consequently the classification performance. Grand average data of 23 participants performing right and left hand motor imagery were analyzed. Trials were separated into two groups depending on the SMR amplitude in the 1000 ms interval preceding the cue, and classification by common spatial patterns (CSPs) preprocessing and linear discriminant analysis (LDA) was carried out in the post-stimulus time interval, i.e., during the task execution. The correlation between trial group and classification performance was assessed by an analysis of variance. As a result of this analysis, trials with higher SMR amplitude in the 1000 ms interval preceding the cue yielded significantly better classification performance than trials with lower amplitude. A further investigation of brain activity patterns revealed that this increase in accuracy is mainly due to the persistence of a higher SMR amplitude over the ipsilateral hemisphere. Our findings support the idea that exploiting information about the ongoing SMR might be the key to boosting performance in future SMR-BCI experiments and motor related tasks in general.

CSP patches: an ensemble of optimized spatial filters. An evaluation study.

Laplacian filters are widely used in neuroscience. In the context of brain-computer interfacing, they might be preferred to data-driven approaches such as common spatial patterns (CSP) in a variety of scenarios such as, e.g., when no or few user data are available or a calibration session with a multi-channel recording is not possible, which is the case in various applications. In this paper we propose the use of an ensemble of local CSP patches (CSPP) which can be considered as a compromise between Laplacian filters and CSP. Our CSPP only needs a very small number of trials to be optimized and significantly outperforms Laplacian filters in all settings studied. Additionally, CSPP also outperforms multi-channel CSP and a regularized version of CSP even when only very few calibration data are available, acting as a CSP regularizer without the need of additional hyperparameters and at a very low cost: 2-5 min of data recording, i.e. ten times less than CSP.

Common spatial pattern patches - An optimized filter ensemble for adaptive brain-computer interfaces

Laplacian filters are commonly used in Brain Computer Interfacing (BCI). When only data from few channels are available, or when, like at the beginning of an experiment, no previous data from the same user is available complex features cannot be used. In this case band power features calculated from Laplacian filtered channels represents an easy, robust and general feature to control a BCI, since its calculation does not involve any class information. For the same reason, the performance obtained with Laplacian features is poor in comparison to subject-specific optimized spatial filters, such as Common Spatial Patterns (CSP) analysis, which, on the other hand, can be used just in a later phase of the experiment, since they require a considerable amount of training data in order to enroll a stable and good performance. This drawback is particularly evident in case of poor performing BCI users, whose data is highly non-stationary and contains little class relevant information. Therefore, Laplacian filtering is preferred to CSP, e.g., in the initial period of co-adaptive calibration, a novel BCI paradigm designed to alleviate the problem of BCI illiteracy. In fact, in the co-adaptive calibration design the experiment starts with a subject-independent classifier and simple features are needed in order to obtain a fast adaptation of the classifier to the newly acquired users data. Here, the use of an ensemble of local CSP patches (CSPP) is proposed, which can be considered as a compromise between Laplacians and CSP: CSPP needs less data and channels than CSP, while being superior to Laplacian filtering. This property is shown to be particularly useful for the co-adaptive calibration design and is demonstrated on off-line data from a previous co-adaptive BCI study.

Ensembles of adaptive spatial filters increase BCI performance: an online evaluation.

OBJECTIVE In electroencephalographic (EEG) data, signals from distinct sources within the brain are widely spread by volume conduction and superimposed such that sensors receive mixtures of a multitude of signals. This reduction of spatial information strongly hampers single-trial analysis of EEG data as, for example, required for brain-computer interfacing (BCI) when using features from spontaneous brain rhythms. Spatial filtering techniques are therefore greatly needed to extract meaningful information from EEG. Our goal is to show, in online operation, that common spatial pattern patches (CSPP) are valuable to counteract this problem. APPROACH Even though the effect of spatial mixing can be encountered by spatial filters, there is a trade-off between performance and the requirement of calibration data. Laplacian derivations do not require calibration data at all, but their performance for single-trial classification is limited. Conversely, data-driven spatial filters, such as common spatial patterns (CSP), can lead to highly distinctive features; however they require a considerable amount of training data. Recently, we showed in an offline analysis that CSPP can establish a valuable compromise. In this paper, we confirm these results in an online BCI study. In order to demonstrate the paramount feature that CSPP requires little training data, we used them in an adaptive setting with 20 participants and focused on users who did not have success with previous BCI approaches. MAIN RESULTS The results of the study show that CSPP adapts faster and thereby allows users to achieve better feedback within a shorter time than previous approaches performed with Laplacian derivations and CSP filters. The success of the experiment highlights that CSPP has the potential to further reduce BCI inefficiency. SIGNIFICANCE CSPP are a valuable compromise between CSP and Laplacian filters. They allow users to attain better feedback within a shorter time and thus reduce BCI inefficiency to one-fourth in comparison to previous non-adaptive paradigms.