Thibault Verhoeven

Towards Improved Design and Evaluation of Epileptic Seizure Predictors

Objective: Key issues in the epilepsy seizure prediction research are (1) the reproducibility of results (2) the inability to compare multiple approaches directly. To overcome these problems, the seizure prediction challenge was organized on Kaggle.com. It aimed at establishing benchmarks on a dataset with predefined train, validation, and test sets. Our main objective is to analyze the competition format, and to propose improvements, which would facilitate a better comparison of algorithms. The second objective is to present a novel deep learning approach to seizure prediction and compare it to other commonly used methods using patient centered metrics. Methods: We used the competitions datasets to illustrate the effects of data contamination. Having better data partitions, we compared three types of models in terms of different objectives. Results: We found that correct selection of test samples is crucial when evaluating the performance of seizure forecasting models. Moreover, we showed that models, which achieve state-of-the-art performance with respect to commonly used AUC, sensitivity, and specificity metrics, may not yet be suitable for practical usage because of low precision scores. Conclusion: Correlation between validation and test datasets used in the competition limited its scientific value. Significance: Our findings provide guidelines which allow for a more objective evaluation of seizure prediction models.

Towards a symbiotic brain–computer interface: exploring the application–decoder interaction

OBJECTIVE State of the art brain-computer interface (BCI) research focuses on improving individual components such as the application or the decoder that converts the users brain activity to control signals. In this study, we investigate the interaction between these components in the P300 speller, a BCI for communication. We introduce a synergistic approach in which the stimulus presentation sequence is modified to enhance the machine learning decoding. In this way we aim for an improved overall BCI performance. APPROACH First, a new stimulus presentation paradigm is introduced which provides us flexibility in tuning the sequence of visual stimuli presented to the user. Next, an experimental setup in which this paradigm is compared to other paradigms uncovers the underlying mechanism of the interdependence between the application and the performance of the decoder. MAIN RESULTS Extensive analysis of the experimental results reveals the changing requirements of the decoder concerning the data recorded during the spelling session. When few data is recorded, the balance in the number of target and non-target stimuli shown to the user is more important than the signal-to-noise rate (SNR) of the recorded response signals. Only when more data has been collected, the SNR becomes the dominant factor. SIGNIFICANCE For BCIs in general, knowing the dominant factor that affects the decoder performance and being able to respond to it is of utmost importance to improve system performance. For the P300 speller, the proposed tunable paradigm offers the possibility to tune the application to the decoders needs at any time and, as such, fully exploit this application-decoder interaction.

Automated diagnosis of temporal lobe epilepsy in the absence of interictal spikes

Objective To diagnose and lateralise temporal lobe epilepsy (TLE) by building a classification system that uses directed functional connectivity patterns estimated during EEG periods without visible pathological activity. Methods Resting-state high-density EEG recording data from 20 left TLE patients, 20 right TLE patients and 35 healthy controls was used. Epochs without interictal spikes were selected. The cortical source activity was obtained for 82 regions of interest and whole-brain directed functional connectivity was estimated in the theta, alpha and beta frequency bands. These connectivity values were then used to build a classification system based on two two-class Random Forests classifiers: TLE vs healthy controls and left vs right TLE. Feature selection and classifier training were done in a leave-one-out procedure to compute the mean classification accuracy. Results The diagnosis and lateralization classifiers achieved a high accuracy (90.7% and 90.0% respectively), sensitivity (95.0% and 90.0% respectively) and specificity (85.7% and 90.0% respectively). The most important features for diagnosis were the outflows from left and right medial temporal lobe, and for lateralization the right anterior cingulate cortex. The interaction between features was important to achieve correct classification. Significance This is the first study to automatically diagnose and lateralise TLE based on EEG. The high accuracy achieved demonstrates the potential of directed functional connectivity estimated from EEG periods without visible pathological activity for helping in the diagnosis and lateralization of TLE.

Learning from label proportions in brain-computer interfaces: Online unsupervised learning with guarantees

Objective Using traditional approaches, a brain-computer interface (BCI) requires the collection of calibration data for new subjects prior to online use. Calibration time can be reduced or eliminated e.g., by subject-to-subject transfer of a pre-trained classifier or unsupervised adaptive classification methods which learn from scratch and adapt over time. While such heuristics work well in practice, none of them can provide theoretical guarantees. Our objective is to modify an event-related potential (ERP) paradigm to work in unison with the machine learning decoder, and thus to achieve a reliable unsupervised calibrationless decoding with a guarantee to recover the true class means. Method We introduce learning from label proportions (LLP) to the BCI community as a new unsupervised, and easy-to-implement classification approach for ERP-based BCIs. The LLP estimates the mean target and non-target responses based on known proportions of these two classes in different groups of the data. We present a visual ERP speller to meet the requirements of LLP. For evaluation, we ran simulations on artificially created data sets and conducted an online BCI study with 13 subjects performing a copy-spelling task. Results Theoretical considerations show that LLP is guaranteed to minimize the loss function similar to a corresponding supervised classifier. LLP performed well in simulations and in the online application, where 84.5% of characters were spelled correctly on average without prior calibration. Significance The continuously adapting LLP classifier is the first unsupervised decoder for ERP BCIs guaranteed to find the optimal decoder. This makes it an ideal solution to avoid tedious calibration sessions. Additionally, LLP works on complementary principles compared to existing unsupervised methods, opening the door for their further enhancement when combined with LLP.

Reducing BCI calibration time with transfer learning: a shrinkage approach

Introduction: A brain-computer interface system (BCI) allows subjects to make use of neural control signals to drive a computer application. Therefor a BCI is generally equipped with a decoder to differentiate between types of responses recorded in the brain. For example, an application giving feedback to the user can benefit from recognizing the presence or absence of a so-called error potential (Errp), elicited in the brain of the user when this feedback is perceived as being ‘wrong’, a mistake of the system. Due to the high inter- and intra- subject variability in these response signals, calibration data needs to be recorded to train the decoder. This calibration session is exhausting and demotivating for the subject. Transfer Learning is a general name for techniques in which data from previous subjects is used as additional information to train a decoder for a new subject, thereby reducing the amount of subject specific data that needs to be recorded during calibration. In this work we apply transfer learning to an Errp detection task by applying single-target shrinkage to Linear Discriminant Analysis (LDA), a method originally proposed by Hohne et. al. to improve accuracy by compensating for inter-stimuli differences in an ERP-speller [1]. Material, Methods and Results: For our study we used the error potential dataset recorded by Perrin et al. in [2]. For 26 subjects each, 340 Errp/nonErrp responses were recorded with a #Errp to #nonErrp ratio of 0.41 to 0.94. 272 responses were available for training the decoder and the remaining 68 responses were left out for testing. For every subject separately we built three different decoders. First, a subject specific LDA decoder was built solely making use of the subject’s own train data. Second, we added the train data of the other 25 subjects to train a global LDA decoder, naively ignoring the difference between subjects. Finally, the single-target-shrinkage method (STS) [1] is used to regularize the parameters of the subject specific decoder towards those of the global decoder. Making use of cross validation this method assigns an optimal weight to the subject specific data and data from previous subjects to be used for training. Figure 1 shows the performance of the three decoders on the test data in terms of AUC as a function of the amount of subject specific calibration data used. Discussion: The subject specific decoder in Figure 1 shows how sensitive the decoding performance is to the amount of calibration data provided. Using data from previously recorded subjects the amount of calibration data, and as such the calibration time, can be reduced as shown by the global decoder. A certain amount of quality is however sacrificed. Making an optimal compromise between the subject specific and global decoder, the single-target-shrinkage decoder allows the calibration time to be reduced by 20% without any change in decoder quality (confirmed by a paired sample t-test giving p=0.72). Significance: This work serves as a first proof of concept in the use of shrinkage LDA as a transfer learning method. More specific, the error potential decoder built with reduced calibration time boosts the opportunity for error correcting methods in BCI.

Unsupervised Learning for Brain-Computer Interfaces Based on Event-Related Potentials: Review and Online Comparison [Research Frontier]

One of the fundamental challenges in brain-computer interfaces (BCIs) is to tune a brain signal decoder to reliably detect a users intention. While information about the decoder can partially be transferred between subjects or sessions, optimal decoding performance can only be reached with novel data from the current session. Thus, it is preferable to learn from unlabeled data gained from the actual usage of the BCI application instead of conducting a calibration recording prior to BCI usage. We review such unsupervised machine learning methods for BCIs based on event-related potentials of the electroencephalogram. We present results of an online study with twelve healthy participants controlling a visual speller. Online performance is reported for three completely unsupervised learning methods: (1) learning from label proportions, (2) an expectation-maximization approach and (3) MIX, which combines the strengths of the two other methods. After a short ramp-up, we observed that the MIX method not only defeats its two unsupervised competitors but even performs on par with a state-of-the-art regularized linear discriminant analysis trained on the same number of data points and with full label access. With this online study, we deliver the best possible proof in BCI that an unsupervised decoding method can in practice render a supervised method unnecessary. This is possible despite skipping the calibration, without losing much performance and with the prospect of continuous improvement over a session. Thus, our findings pave the way for a transition from supervised to unsupervised learning methods in BCIs based on eventrelated potentials.

Improving zero-training brain-computer interfaces by mixing model estimators

OBJECTIVE Brain-computer interfaces (BCI) based on event-related potentials (ERP) incorporate a decoder to classify recorded brain signals and subsequently select a control signal that drives a computer application. Standard supervised BCI decoders require a tedious calibration procedure prior to every session. Several unsupervised classification methods have been proposed that tune the decoder during actual use and as such omit this calibration. Each of these methods has its own strengths and weaknesses. Our aim is to improve overall accuracy of ERP-based BCIs without calibration. APPROACH We consider two approaches for unsupervised classification of ERP signals. Learning from label proportions (LLP) was recently shown to be guaranteed to converge to a supervised decoder when enough data is available. In contrast, the formerly proposed expectation maximization (EM) based decoding for ERP-BCI does not have this guarantee. However, while this decoder has high variance due to random initialization of its parameters, it obtains a higher accuracy faster than LLP when the initialization is good. We introduce a method to optimally combine these two unsupervised decoding methods, letting one methods strengths compensate for the weaknesses of the other and vice versa. The new method is compared to the aforementioned methods in a resimulation of an experiment with a visual speller. MAIN RESULTS Analysis of the experimental results shows that the new method exceeds the performance of the previous unsupervised classification approaches in terms of ERP classification accuracy and symbol selection accuracy during the spelling experiment. Furthermore, the method shows less dependency on random initialization of model parameters and is consequently more reliable. SIGNIFICANCE Improving the accuracy and subsequent reliability of calibrationless BCIs makes these systems more appealing for frequent use.

P362 Automated diagnosis of temporal lobe epilepsy in the absence of interictal spikes

Objective To diagnose and lateralise Temporal Lobe Epilepsy (TLE) by building a classification system that uses directed functional connectivity patterns estimated during EEG periods without visible pathological activity. Methods Resting-state high-density EEG recording data from 20 left TLE patients, 20 right TLE patients and 35 healthy controls was used. Epochs without interictal spikes were selected. The cortical source activity was obtained for 82 regions of interest and whole-brain directed functional connectivity was estimated in the theta, alpha and beta frequency bands. These connectivity values were then used to build a classification system based on two two-class Random Forests classifiers: TLE vs healthy controls and left vs right TLE. Feature selection and classifier training were done in a leave-one-out procedure to compute the mean classification accuracy. Results The diagnosis and lateralization classifiers achieved a high accuracy (90.7% and 90.0% respectively), sensitivity (95.0% and 90.0% respectively) and specificity (85.7% and 90.0% respectively). The most important features for diagnosis were the outflows from left and right medial temporal lobe, and for lateralization the right anterior cingulate cortex. The interaction between features was important to achieve correct classification. Conclusions This is the first study to automatically diagnose and lateralise TLE based on EEG. The high accuracy achieved demonstrates the potential of directed functional connectivity estimated from EEG periods without visible pathological activity for helping in the diagnosis and lateralization of TLE.

Switching characters between stimuli improves P300 speller accuracy

In this paper, an alternative stimulus presentation paradigm for the P300 speller is introduced. Similar to the checkerboard paradigm it minimizes the occurrence of the two most common causes of spelling errors: adjacency distraction and double flashes. Moreover, in contrast to the checkerboard paradigm, this new stimulus sequence does not increase the time required per stimulus iteration. Our new paradigm is compared to the basic row-column paradigm and the results indicate that, on average, the accuracy is improved.