Hubert Cecotti

Spelling with non-invasive Brain-Computer Interfaces--current and future trends.

Brain-Computer Interfaces (BCIs) have become a large research field that include challenges mainly in neuroscience, signal processing, machine learning and user interface. A non-invasive BCI can allow the direct communication between humans and computers by analyzing electrical brain activity, recorded at the surface of the scalp with electroencephalography. The main purpose for BCIs is to enable communication for people with severe disabilities. Spelling is one of the first BCI application, it corresponds to the main communication mean for people who are unable to speak. While spelling can be the most basic application it remains a benchmark for communication applications and one challenge in the BCI community for some patients. This paper proposes a review of the current main strategies, and their limitations, for spelling words. It includes recent BCIs based on P300, steady-state visual evoked potentials and motor imagery. By considering some challenges in BCI spellers and virtual keyboards, some pragmatic issues are pointed out to eliminate false hopes about BCI for both disabled and healthy people.

Single-Trial Classification of Event-Related Potentials in Rapid Serial Visual Presentation Tasks Using Supervised Spatial Filtering

Accurate detection of single-trial event-related potentials (ERPs) in the electroencephalogram (EEG) is a difficult problem that requires efficient signal processing and machine learning techniques. Supervised spatial filtering methods that enhance the discriminative information in EEG data are commonly used to improve single-trial ERP detection. We propose a convolutional neural network (CNN) with a layer dedicated to spatial filtering for the detection of ERPs and with training based on the maximization of the area under the receiver operating characteristic curve (AUC). The CNN is compared with three common classifiers: 1) Bayesian linear discriminant analysis; 2) multilayer perceptron (MLP); and 3) support vector machines. Prior to classification, the data were spatially filtered with xDAWN (for the maximization of the signal-to-signal-plus-noise ratio), common spatial pattern, or not spatially filtered. The 12 analytical techniques were tested on EEG data recorded in three rapid serial visual presentation experiments that required the observer to discriminate rare target stimuli from frequent nontarget stimuli. Classification performance discriminating targets from nontargets depended on both the spatial filtering method and the classifier. In addition, the nonlinear classifier MLP outperformed the linear methods. Finally, training based AUC maximization provided better performance than training based on the minimization of the mean square error. The results support the conclusion that the choice of the systems architecture is critical and both spatial filtering and classification must be considered together.

Multimodal target detection using single trial evoked EEG responses in single and dual-tasks

The detection of event-related potentials in the electroencephalogram signal is a common way for creating a brain-computer interface (BCI). Successful detection of evoked responses can be enhanced by the user selectively attending to specific stimuli presented in the BCI task. Because BCI users need a system that performs well in a variety of contexts, even ones that may impair selective attention, it is critical to understand how single trial detection is affected by attention. We tested 16 participants using a rapid serial visual/auditory presentation paradigm under three conditions, one in which they detected the presence of a visual target, one in which they detected the presence of an auditory target, and one in which they detected both visual and auditory targets. The behavioral performance indicates that the visual task was more difficult than the auditory task. Consistent with the higher behavioral difficulty of the visual task, single trial performance showed no difference between single and dual-task for the visual target detection (mean=0.76). However, the area under the curve for the auditory target detection was significantly lower than the dual-task (mean=0.81 for single task, 0.75 for dual-task). The results support the conclusion that single-trial target detection is impaired when attention is divided between multiple tasks.

Isolating the neural mechanisms of interference during continuous multisensory dual-task performance

The need to engage in multiple tasks simultaneously is often encountered in everyday experience, but coordinating between two or more tasks can lead to impaired performance. Typical investigations of multitasking impairments have focused on the performance of two tasks presented in close temporal proximity on discrete trials; however, such paradigms do not match well with the continuous performance situations more typically encountered outside the laboratory. As a result, the stages of information processing that are affected during multisensory continuous dual tasks and how these changes in processing relate to behavior remain unclear. To address these issues, participants were presented simultaneous rapid visual and auditory stimulus sequences under three conditions: attend visual only, attend auditory only, and dual attention (attend both visual and auditory). Performance, measured in terms of response time and perceptual sensitivity (d′), revealed dual-task impairments only in the auditory task. Neural activity, measured by the ERP technique, revealed that both early stage sensory processing and later cognitive processing of the auditory task were affected by dual-task performance, but similar stages of processing of the visual task were not. Critically, individual differences in neural activity at both early and late stages of information processing accurately rank-ordered individuals based on the observed difference in behavioral performance between the single and dual attention conditions. These results reveal relationships between behavioral performance and the neural correlates of both early and late stage information processing that provide key insights into the complex interplay between the brain and behavior when multiple tasks are performed continuously.

Multiple stages of information processing are modulated during acute bouts of exercise

Acute bouts of aerobic physical exercise can modulate subsequent cognitive task performance and oscillatory brain activity measured with electroencephalography (EEG). Here, we investigated the sequencing of these modulations of perceptual and cognitive processes using scalp recorded EEG acquired during exercise. Twelve participants viewed pseudo-random sequences of frequent non-target stimuli (cars), infrequent distractors (obliquely oriented faces) and infrequent targets that required a simple detection response (obliquely oriented faces, where the angle was different than the infrequent distractors). The sequences were presented while seated on a stationary bike under three conditions during which scalp recorded EEG was also acquired: rest, low-intensity exercise, and high-intensity exercise. Behavioral target detection was faster during high-intensity exercise compared to both rest and low-intensity exercise. An event-related potential (ERP) analysis of the EEG data revealed that the mean amplitude of the visual P1 component evoked by frequent non-targets measured at parietal-occipital electrodes was larger during low-intensity exercise compared to rest. The P1 component evoked by infrequent targets also peaked earlier during low-intensity exercise compared to rest and high-intensity exercise. The P3a ERP component evoked by infrequent distractors measured at parietal electrodes peaked significantly earlier during both low- and high-intensity exercise when compared to rest. The modulation of the visual P1 and the later P3a components is consistent with the conclusion that exercise modulates multiple stages of neural information processing, ranging from early stage sensory processing (P1) to post-perceptual target categorization (P3a).

Effects of performing two visual tasks on single-trial detection of event-related potentials

The detection of event-related potentials (ERPs) in brain-computer interface (BCI) depends on the ability of the subject to pay attention to specific stimuli presented during the BCI task. For healthy users, a BCI shall be used as a complement to other existing devices, which involve the response to other tasks. Those tasks may impair selective attention, particularly if the stimuli have the same modality e.g. visual. It is therefore critical to analyze how single-trial detection of brain evoked response is impaired by the addition of tasks concerning the same modality. We tested 10 healthy participants using an application that has two visual target detection tasks. The first one corresponds to a rapid serial visual presentation paradigm where target detection is achieved by brain-evoked single-trial detection in the recorded electroencephalogram (EEG) signal. The second task is the detection of a visual event on a tactical map by a behavioral response. These tasks were tested individually (single task) and in parallel (dual-task). Whereas the performance of single-trial detection was not impaired between single and dual-task conditions, the behavioral performance decreased during the dual-task condition. These results quantify the performance drop that can occur in a dual-task system using both brain-evoked responses and behavioral responses.

Impact of target probability on single-trial EEG target detection in a difficult rapid serial visual presentation task

In non-invasive brain-computer interface (BCI), the analysis of event-related potentials (ERP) has typically focused on averaged trials, a current trend is to analyze single-trial evoked response individually with new approaches in pattern recognition and signal processing. Such single trial detection requires a robust response that can be detected in a variety task conditions. Here, we investigated the influence of target probability, a key factor known to influence the amplitude of the evoked response, on single trial target classification in a difficult rapid serial visual presentation (RSVP) task. Our classification approach for detecting target vs. non target responses, considers spatial filters obtained through the maximization of the signal to signal-plus-noise ratio, and then uses the resulting information as inputs to a Bayesian discriminant analysis. The method is evaluated across eight healthy subjects, on four probability conditions (P=0.05, 0.10, 0.25, 0.50). We show that the target probability has a statistically significant effect on both the behavioral performance and the target detection. The best mean area under the ROC curve is achieved with P=0.10, AUC=0.82. These results suggest that optimal performance of ERP detection in RSVP tasks is critically dependent on target probability.

A Radial Neural Convolutional Layer for Multi-oriented Character Recognition

The recognition of fully multi-oriented handwritten characters is a challenging problem. Contrary to univariate signals where the shift invariance property in the Fourier transform can be used, multivariate signals like images require special care to extract rotation invariant features. Several strategies to solve such classification tasks are possible. The proposed method considers input features obtained by the Radon transform or Polar transform. A convolutional neural network is then used for extracting higher level features. This classifier includes in addition the Fast Fourier Transform for extracting shift invariant features at the neural network level. The Radon transform and the convolutional layers process the image at the pixel level while the Fourier transform and the upper layers of the neural network process rotation invariant features. The classifier is evaluated on multi-oriented handwritten digits based on the MNIST database (Arabic digits) and on the ISI database (Bangla digits). The average recognition rate for multi-oriented characters is 93.10% for the Arabic digits and 77.01% for the Bangla digits. This neural architecture highlights the interest of the radial convolutional layer for the recognition of multi-oriented shapes.

Multiclass classification of single-trial evoked EEG responses

The detection of event-related potentials (ERPs) in the electroencephalogram (EEG) signal has several real-world applications, from cognitive state monitoring to brain-computer interfaces. Current systems based on the detection of ERPs only consider a single type of response to detect. Hence, the classification methods that are considered for ERP detection are binary classifiers (target vs. non target). Here we investigated multiclass classification of single-trial evoked responses during a rapid serial visual presentation task in which short video clips were presented to fifteen observers. Each trial contained potential targets that were human or non-human, stationary or moving. The goal of the classification analysis was to discriminate between three classes: moving human targets, moving non-human targets, and non-moving human targets. The analysis revealed that the mean volume under the ROC surface of 0.878. These results suggest that it is possible to efficiently discriminate between more than two types of evoked responses using single-trial detection.

Rejection Schemes in Multi-class Classification -- Application to Handwritten Character Recognition

The recognition of handwritten characters is an almost solved problem thanks to efficient machine learning techniques. However, the evaluation and the choice of thresholds to meet a certain level of performance remains a challenge. In this paper, we compare different rejection techniques to determine if a character has been successfully detected or not. Whereas the evaluation of binary classifiers through ROC curves and cost curves has been largely exploited in the literature, multi-class problems can involve different issues like the computational cost or the choice of having an adaptive threshold for each class. The proposed methods for rejection criteria include the maximization of the distance to the optimal performance in the ROC space. We show that optimizing this criterion is a suitable approach on two databases: Lampung handwritten characters, and Arabic handwritten digits. The results support the conclusion that in database where the accuracy is high, pair wise ROC curves analysis in multi-class problems can lead to a finer evaluation of the performance and thresholds definition.