Emmanuel Maby

Openvibe: An open-source software platform to design, test, and use brain--computer interfaces in real and virtual environments

This paper describes the OpenViBE software platform which enables researchers to design, test, and use braincomputer interfaces (BCIs). BCIs are communication systems that enable users to send commands to computers solely by means of brain activity. BCIs are gaining interest among the virtual reality (VR) community since they have appeared as promising interaction devices for virtual environments (VEs). The key features of the platform are (1) high modularity, (2) embedded tools for visualization and feedback based on VR and 3D displays, (3) BCI design made available to non-programmers thanks to visual programming, and (4) various tools offered to the different types of users. The platform features are illustrated in this paper with two entertaining VR applications based on a BCI. In the first one, users can move a virtual ball by imagining hand movements, while in the second one, they can control a virtual spaceship using real or imagined foot movements. Online experiments with these applications together with the evaluation of the platform computational performances showed its suitability for the design of VR applications controlled with a BCI. OpenViBE is a free software distributed under an open-source license.

Impact of spatial filters during sensor selection in a visual P300 brain-computer interface.

A challenge in designing a Brain-Computer Interface (BCI) is the choice of the channels, e.g. the most relevant sensors. Although a setup with many sensors can be more efficient for the detection of Event-Related Potential (ERP) like the P300, it is relevant to consider only a low number of sensors for a commercial or clinical BCI application. Indeed, a reduced number of sensors can naturally increase the user comfort by reducing the time required for the installation of the EEG (electroencephalogram) cap and can decrease the price of the device. In this study, the influence of spatial filtering during the process of sensor selection is addressed. Two of them maximize the Signal to Signal-plus-Noise Ratio (SSNR) for the different sensor subsets while the third one maximizes the differences between the averaged P300 waveform and the non P300 waveform. We show that the locations of the most relevant sensors subsets for the detection of the P300 are highly dependent on the use of spatial filtering. Applied on data from 20 healthy subjects, this study proves that subsets obtained where sensors are suppressed in relation to their individual SSNR are less efficient than when sensors are suppressed in relation to their contribution once the different selected sensors are combined for enhancing the signal. In other words, it highlights the difference between estimating the P300 projection on the scalp and evaluating the more efficient sensor subsets for a P300-BCI. Finally, this study explores the issue of channel commonality across subjects. The results support the conclusion that spatial filters during the sensor selection procedure allow selecting better sensors for a visual P300 Brain-Computer Interface.

EEG sensor selection by sparse spatial filtering in P300 speller brain-computer interface

A Brain-Computer Interface (BCI) is a specific type of human-machine interface that enables communication between a subject/patient and a computer by direct control from decoding of brain activity. This paper deals with the P300-speller application that enables to write a text based on the oddball paradigm. To improve the ergonomics and minimize the cost of such a BCI, reducing the number of electrodes is mandatory. We propose a new algorithm to select a relevant subset of electrodes by estimating sparse spatial filters. A l1-norm penalization term, as an approximation of the l0-norm, is introduced in the xDAWN algorithm, which maximizes the signal to signal-plus-noise ratio. Experimental results on 20 subjects show that the proposed method is efficient to select the most relevant sensors: from 32 down to 10 sensors, the loss in classification accuracy is less than 5%.

Adaptive training session for a P300 speller brain-computer interface.

With a brain-computer interface (BCI), it is nowadays possible to achieve a direct pathway between the brain and computers thanks to the analysis of some particular brain activities. The detection of even-related potentials, like the P300 in the oddball paradigm exploited in P300-speller, provides a way to create BCIs by assigning several detected ERP to a command. Due to the noise present in the electroencephalographic signal, the detection of an ERP and its different components requires efficient signal processing and machine learning techniques. As a consequence, a calibration session is needed for training the models, which can be a drawback if its duration is too long. Although the model depends on the subject, the goal is to provide a reliable model for the P300 detection over time. In this study, we propose a new method to evaluate the optimal number of symbols (i.e. the number of ERP that shall be detected given a determined target probability) that should be spelt during the calibration process. The goal is to provide a usable system with a minimum calibration duration and such that it can automatically switch between the training and online sessions. The method allows to adaptively adjust the number of training symbols to each subject. The evaluation has been tested on data recorded on 20 healthy subjects. This procedure lets drastically reduced the calibration session: height symbols during the training session reach an initialized system with an average accuracy of 80% after five epochs.

Topographical Dynamics of Brain Connections for the Design of Asynchronous Brain-Computer Interfaces

This article presents a new processing method to design brain-computer interfaces (BCIs). It shows how to use the perturbations of the communication between different cortical areas due to a cognitive task. For this, the network of the cerebral connections is built from correlations between cortical areas at specific frequencies and is analyzed using graph theory. This allows us to describe the topological organisation of the networks using quantitative measures. This method is applied to an auditive steady-state evoked potentials experiment (dichotic binaural listening) and compared to a more classical method based on spectral filtering.

Toward a New Application of Real-Time Electrophysiology: Online Optimization of Cognitive Neurosciences Hypothesis Testing

Brain-computer interfaces (BCIs) mostly rely on electrophysiological brain signals. Methodological and technical progress has largely solved the challenge of processing these signals online. The main issue that remains, however, is the identification of a reliable mapping between electrophysiological measures and relevant states of mind. This is why BCIs are highly dependent upon advances in cognitive neuroscience and neuroimaging research. Recently, psychological theories became more biologically plausible, leading to more realistic generative models of psychophysiological observations. Such complex interpretations of empirical data call for efficient and robust computational approaches that can deal with statistical model comparison, such as approximate Bayesian inference schemes. Importantly, the latter enable the optimization of a model selection error rate with respect to experimental control variables, yielding maximally powerful designs. In this paper, we use a Bayesian decision theoretic approach to cast model comparison in an online adaptive design optimization procedure. We show how to maximize design efficiency for individual healthy subjects or patients. Using simulated data, we demonstrate the face- and construct-validity of this approach and illustrate its extension to electrophysiology and multiple hypothesis testing based on recent psychophysiological models of perception. Finally, we discuss its implications for basic neuroscience and BCI itself.

Improving BCI performance through co-adaptation: Applications to the P300-speller

A well-known neurophysiological marker that can easily be captured with electroencephalography (EEG) is the so-called P300: a positive signal deflection occurring at about 300 ms after a relevant stimulus. This brain response is particularly salient when the target stimulus is rare among a series of distracting stimuli, whatever the type of sensory input. Therefore, it has been proposed and extensively studied as a possible feature for direct brain-computer communication. The most advanced non-invasive BCI application based on this principle is the P300-speller. However, it is still a matter of debate whether this application will prove relevant to any population of patients. In a series of recent theoretical and empirical studies, we have been using this P300-based paradigm to push forward the performance of non-invasive BCI. This paper summarizes the proposed improvements and obtained results. Importantly, those could be generalized to many kinds of BCI, beyond this particular application. Indeed, they relate to most of the key components of a closed-loop BCI, namely: improving the accuracy of the system by trying to detect and correct for errors automatically; optimizing the computers speed-accuracy trade-off by endowing it with adaptive behavior; but also simplifying the hardware and time for set-up in the aim of routine use in patients. Our results emphasize the importance of the closed-loop interaction and of the ensuing co-adaptation between the user and the machine whenever possible. Most of our evaluations have been conducted in healthy subjects. We conclude with perspectives for clinical applications.

Impact of the time segment analysis for P300 detection with spatial filtering

A Brain-Computer Interface (BCI) allows the direct communication between humans and computers by analyzing brain activity. The oddball paradigm allows detecting event-related potentials (ERPs), like the P300 wave, on targets selected by the user. While this paradigm provides the location of the P300 wave in the signal, its exact location remains a hypothesis and depends on the subject. This paper deals with the choice of the time segment for the signal analysis and its impact on the classification. A method for selecting the relevant part of the signal that contains the P300 wave is proposed. First, spatial filters are estimated for enhancing the signal. Second, a part of the enhanced P300 wave is selected based on its magnitude. This selection aims at providing an optimal start for the time window representing the P300 wave. Three window lengths are compared. We show that a window length of 500ms provides on average the best results, but the optimal window length should be set individually. The proposed technique has been validated on data recorded on 20 healthy subjects.

Active SAmpling Protocol (ASAP) to Optimize Individual Neurocognitive Hypothesis Testing: A BCI-Inspired Dynamic Experimental Design

The relatively young field of Brain-Computer Interfaces has promoted the use of electrophysiology and neuroimaging in real-time. In the meantime, cognitive neuroscience studies, which make extensive use of functional exploration techniques, have evolved toward model-based experiments and fine hypothesis testing protocols. Although these two developments are mostly unrelated, we argue that, brought together, they may trigger an important shift in the way experimental paradigms are being designed, which should prove fruitful to both endeavors. This change simply consists in using real-time neuroimaging in order to optimize advanced neurocognitive hypothesis testing. We refer to this new approach as the instantiation of an Active SAmpling Protocol (ASAP). As opposed to classical (static) experimental protocols, ASAP implements online model comparison, enabling the optimization of design parameters (e.g., stimuli) during the course of data acquisition. This follows the well-known principle of sequential hypothesis testing. What is radically new, however, is our ability to perform online processing of the huge amount of complex data that brain imaging techniques provide. This is all the more relevant at a time when physiological and psychological processes are beginning to be approached using more realistic, generative models which may be difficult to tease apart empirically. Based upon Bayesian inference, ASAP proposes a generic and principled way to optimize experimental design adaptively. In this perspective paper, we summarize the main steps in ASAP. Using synthetic data we illustrate its superiority in selecting the right perceptual model compared to a classical design. Finally, we briefly discuss its future potential for basic and clinical neuroscience as well as some remaining challenges.

Incorporation of recent waking-life experiences in dreams correlates with frontal theta activity in REM sleep

Abstract Rapid eye movement (REM) sleep and its main oscillatory feature, frontal theta, have been related to the processing of recent emotional memories. As memories constitute much of the source material for our dreams, we explored the link between REM frontal theta and the memory sources of dreaming, so as to elucidate the brain activities behind the formation of dream content. Twenty participants were woken for dream reports in REM and slow wave sleep (SWS) while monitored using electroencephalography. Eighteen participants reported at least one REM dream and 14 at least one SWS dream, and they, and independent judges, subsequently compared their dream reports with log records of their previous daily experiences. The number of references to recent waking-life experiences in REM dreams was positively correlated with frontal theta activity in the REM sleep period. No such correlation was observed for older memories, nor for SWS dreams. The emotional intensity of recent waking-life experiences incorporated into dreams was higher than the emotional intensity of experiences that were not incorporated. These results suggest that the formation of wakefulness-related dream content is associated with REM theta activity, and accords with theories that dreaming reflects emotional memory processing taking place in REM sleep.