Motoaki Kawanabe

Optimizing Spatial filters for Robust EEG Single-Trial Analysis

Due to the volume conduction multichannel electroencephalogram (EEG) recordings give a rather blurred image of brain activity. Therefore spatial filters are extremely useful in single-trial analysis in order to improve the signal-to-noise ratio. There are powerful methods from machine learning and signal processing that permit the optimization of spatio-temporal filters for each subject in a data dependent fashion beyond the fixed filters based on the sensor geometry, e.g., Laplacians. Here we elucidate the theoretical background of the common spatial pattern (CSP) algorithm, a popular method in brain-computer interface (BCD research. Apart from reviewing several variants of the basic algorithm, we reveal tricks of the trade for achieving a powerful CSP performance, briefly elaborate on theoretical aspects of CSP, and demonstrate the application of CSP-type preprocessing in our studies of the Berlin BCI (BBCI) project.

Toward Unsupervised Adaptation of LDA for Brain–Computer Interfaces

There is a step of significant difficulty experienced by brain-computer interface (BCI) users when going from the calibration recording to the feedback application. This effect has been previously studied and a supervised adaptation solution has been proposed. In this paper, we suggest a simple unsupervised adaptation method of the linear discriminant analysis (LDA) classifier that effectively solves this problem by counteracting the harmful effect of nonclass-related nonstationarities in electroencephalography (EEG) during BCI sessions performed with motor imagery tasks. For this, we first introduce three types of adaptation procedures and investigate them in an offline study with 19 datasets. Then, we select one of the proposed methods and analyze it further. The chosen classifier is offline tested in data from 80 healthy users and four high spinal cord injury patients. Finally, for the first time in BCI literature, we apply this unsupervised classifier in online experiments. Additionally, we show that its performance is significantly better than the state-of-the-art supervised approach.

A New Discriminative Kernel From Probabilistic Models

Recently, Jaakkola and Haussler (1999) proposed a method for constructing kernel functions from probabilistic models. Their so-called Fisher kernel has been combined with discriminative classifiers such as support vector machines and applied successfully in, for example, DNA and protein analysis. Whereas the Fisher kernel is calculated from the marginal log-likelihood, we propose the TOP kernel derived from tangent vectors of posterior log-odds. Furthermore, we develop a theoretical framework on feature extractors from probabilistic models and use it for analyzing the TOP kernel. In experiments, our new discriminative TOP kernel compares favorably to the Fisher kernel.

Stationary common spatial patterns for brain-computer interfacing

Classifying motion intentions in brain-computer interfacing (BCI) is a demanding task as the recorded EEG signal is not only noisy and has limited spatial resolution but it is also intrinsically non-stationary. The non-stationarities in the signal may come from many different sources, for instance, electrode artefacts, muscular activity or changes of task involvement, and often deteriorate classification performance. This is mainly because features extracted by standard methods like common spatial patterns (CSP) are not invariant to variations of the signal properties, thus should also change over time. Although many extensions of CSP were proposed to, for example, reduce the sensitivity to noise or incorporate information from other subjects, none of them tackles the non-stationarity problem directly. In this paper, we propose a method which regularizes CSP towards stationary subspaces (sCSP) and show that this increases classification accuracy, especially for subjects who are hardly able to control a BCI. We compare our method with the state-of-the-art approaches on different datasets, show competitive results and analyse the reasons for the improvement.

Kernel-based nonlinear blind source separation

We propose kTDSEP, a kernel-based algorithm for nonlinear blind source separation (BSS). It combines complementary research fields: kernel feature spaces and BSS using temporal information. This yields an efficient algorithm for nonlinear BSS with invertible nonlinearity. Key assumptions are that the kernel feature space is chosen rich enough to approximate the nonlinearity and that signals of interest contain temporal information. Both assumptions are fulfilled for a wide set of real-world applications. The algorithm works as follows: First, the data are (implicitly) mapped to a high (possibly infinite)dimensional kernel feature space. In practice, however, the data form a smaller submanifold in feature spaceeven smaller than the number of training data pointsa fact that has already been used by, for example, reduced set techniques for support vector machines. We propose to adapt to this effective dimension as a preprocessing step and to construct an orthonormal basis of this submanifold. The latter dimension-reduction step is essential for making the subsequent application of BSS methods computationally and numerically tractable. In the reduced space, we use a BSS algorithm that is based on second-order temporal decorrelation. Finally, we propose a selection procedure to obtain the original sources from the extracted nonlinear components automatically. Experiments demonstrate the excellent performance and efficiency of our kTDSEP algorithm for several problems of nonlinear BSS and for more than two sources.

A resampling approach to estimate the stability of one-dimensional or multidimensional independent components

When applying unsupervised learning techniques in biomedical data analysis, a key question is whether the estimated parameters of the studied system are reliable. In other words, can we assess the quality of the result produced by our learning technique? We propose resampling methods to tackle this question and illustrate their usefulness for blind-source separation (BSS). We demonstrate that our proposed reliability estimation can be used to discover stable one-dimensional or multidimensional independent components, to choose the appropriate BSS-model, to enhance significantly the separation performance, and, most importantly, to flag components that carry physical meaning. Application to different biomedical testbed data sets (magnetoencephalography (MEG)/electrocardiography (ECG)-recordings) underline the usefulness of our approach.

Divergence-Based Framework for Common Spatial Patterns Algorithms

Controlling a device with a brain-computer interface requires extraction of relevant and robust features from high-dimensional electroencephalographic recordings. Spatial filtering is a crucial step in this feature extraction process. This paper reviews algorithms for spatial filter computation and introduces a general framework for this task based on divergence maximization. We show that the popular common spatial patterns (CSP) algorithm can be formulated as a divergence maximization problem and computed within our framework. Our approach easily permits enforcing different invariances and utilizing information from other subjects; thus, it unifies many of the recently proposed CSP variants in a principled manner. Furthermore, it allows to design novel spatial filtering algorithms by incorporating regularization schemes into the optimization process or applying other divergences. We evaluate the proposed approach using three regularization schemes, investigate the advantages of beta divergence, and show that subject-independent feature spaces can be extracted by jointly optimizing the divergence problems of multiple users. We discuss the relations to several CSP variants and investigate the advantages and limitations of our approach with simulations. Finally, we provide experimental results on a dataset containing recordings from 80 subjects and interpret the obtained patterns from a neurophysiological perspective.

Modeling Sparse Connectivity Between Underlying Brain Sources for EEG/MEG

We propose a novel technique to assess functional brain connectivity in electroencephalographic (EEG)/magnetoencephalographic (MEG) signals. Our method, called sparsely connected sources analysis (SCSA), can overcome the problem of volume conduction by modeling neural data innovatively with the following ingredients: 1) the EEG/MEG is assumed to be a linear mixture of correlated sources following a multivariate autoregressive (MVAR) model; 2) the demixing is estimated jointly with the source MVAR parameters; and 3) overfitting is avoided by using the group lasso penalty. This approach allows us to extract the appropriate level of crosstalk between the extracted sources and, in this manner, we obtain a sparse data-driven model of functional connectivity. We demonstrate the usefulness of SCSA with simulated data and compare it to a number of existing algorithms with excellent results.

On-line learning in changing environments with applications in supervised and unsupervised learning

An adaptive on-line algorithm extending the learning of learning idea is proposed and theoretically motivated. Relying only on gradient flow information it can be applied to learning continuous functions or distributions, even when no explicit loss function is given and the Hessian is not available. The framework is applied for unsupervised and supervised learning. Its efficiency is demonstrated for drifting and switching non-stationary blind separation tasks of acoustic signals. Furthermore applications to classification (US postal service data set) and time-series prediction in changing environments are presented.

Dimensionality reduction for density ratio estimation in high-dimensional spaces

The ratio of two probability density functions is becoming a quantity of interest these days in the machine learning and data mining communities since it can be used for various data processing tasks such as non-stationarity adaptation, outlier detection, and feature selection. Recently, several methods have been developed for directly estimating the density ratio without going through density estimation and were shown to work well in various practical problems. However, these methods still perform rather poorly when the dimensionality of the data domain is high. In this paper, we propose to incorporate a dimensionality reduction scheme into a density-ratio estimation procedure and experimentally show that the estimation accuracy in high-dimensional cases can be improved.