Seyed Mostafa Kia

MEG decoding across subjects

Brain decoding is a data analysis paradigm for neuroimaging experiments that is based on predicting the stimulus presented to the subject from the concurrent brain activity. In order to make inference at the group level, a straightforward but sometimes unsuccessful approach is to train a classifier on the trials of a group of subjects and then to test it on unseen trials from new subjects. The extreme difficulty is related to the structural and functional variability across the subjects. We call this approach decoding across subjects. In this work, we address the problem of decoding across subjects for magnetoen-cephalographic (MEG) experiments and we provide the following contributions: first, we formally describe the problem and show that it belongs to a machine learning sub-field called transductive transfer learning (TTL). Second, we propose to use a simple TTL technique that accounts for the differences between train data and test data. Third, we propose the use of ensemble learning, and specifically of stacked generalization, to address the variability across subjects within train data, with the aim of producing more stable classifiers. On a face vs. scramble task MEG dataset of 16 subjects, we compare the standard approach of not modelling the differences across subjects, to the proposed one of combining TTL and ensemble learning. We show that the proposed approach is consistently more accurate than the standard one.

User-centric Affective Video Tagging from MEG and Peripheral Physiological Responses

This paper presents a new multimodal database and the associated results for characterization of affect (valence, arousal and dominance) using the Magneto encephalogram (MEG) brain signals and peripheral physiological signals (horizontal EOG, ECG, trapezius EMG). We attempt single-trial classification of affect in movie and music video clips employing emotional responses extracted from eighteen participants. The main findings of this study are that: (i) the MEG signal effectively encodes affective viewer responses, (ii) clip arousal is better predicted by MEG, while peripheral physiological signals are more effective for predicting valence and (iii) prediction performance is better for movie clips as compared to music video clips.

Movie Genre Classification by Exploiting MEG Brain Signals

Genre classification is an essential part of multimedia content recommender systems. In this study, we provide experimental evidence for the possibility of performing genre classification based on brain recorded signals. The brain decoding paradigm is employed to classify magnetoencephalography (MEG) data presented in [1] to four genre classes: Comedy, Romantic, Drama, and Horror. Our results show that: 1) there is a significant correlation between audio-visual features of movies and corresponding brain signals specially in the visual and temporal lobes; 2) the genre of movie clips can be classified with an accuracy significantly over the chance level using the MEG signal. On top of that we show that the combination of multimedia features and MEG-based features achieves the best accuracy. Our study provides a primary step towards user-centric media content retrieval using brain signals.

Multi-Task Learning for Interpretation of Brain Decoding Models

Improving the interpretability of multivariate models is of primary interest for many neuroimaging studies. In this study, we present an application of multi-task learning (MTL) to enhance the interpretability of linear classifiers once applied to neuroimaging data. To attain our goal, we propose to divide the data into spatial fractions and define the temporal data of each spatial unit as a task in MTL paradigm. Our result on magnetoencephalography (MEG) data reveals preliminary evidence that, (1) dividing the brain recordings into spatial fractions based on spatial units of data and (2) considering each spatial fraction as a task, are two factors that provide more stability and consequently more interpretability for brain decoding models.

Interpretability of Multivariate Brain Maps in Linear Brain Decoding: Definition, and Heuristic Quantification in Multivariate Analysis of MEG Time-Locked Effects

Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. Linear classifiers are widely employed in the brain decoding paradigm to discriminate among experimental conditions. Then, the derived linear weights are visualized in the form of multivariate brain maps to further study spatio-temporal patterns of underlying neural activities. It is well known that the brain maps derived from weights of linear classifiers are hard to interpret because of high correlations between predictors, low signal to noise ratios, and the high dimensionality of neuroimaging data. Therefore, improving the interpretability of brain decoding approaches is of primary interest in many neuroimaging studies. Despite extensive studies of this type, at present, there is no formal definition for interpretability of multivariate brain maps. As a consequence, there is no quantitative measure for evaluating the interpretability of different brain decoding methods. In this paper, first, we present a theoretical definition of interpretability in brain decoding; we show that the interpretability of multivariate brain maps can be decomposed into their reproducibility and representativeness. Second, as an application of the proposed definition, we exemplify a heuristic for approximating the interpretability in multivariate analysis of evoked magnetoencephalography (MEG) responses. Third, we propose to combine the approximated interpretability and the generalization performance of the brain decoding into a new multi-objective criterion for model selection. Our results, for the simulated and real MEG data, show that optimizing the hyper-parameters of the regularized linear classifier based on the proposed criterion results in more informative multivariate brain maps. More importantly, the presented definition provides the theoretical background for quantitative evaluation of interpretability, and hence, facilitates the development of more effective brain decoding algorithms in the future.

Group-level spatio-temporal pattern recovery in MEG decoding using multi-task joint feature learning

BACKGROUND The use of machine learning models to discriminate between patterns of neural activity has become in recent years a standard analysis approach in neuroimaging studies. Whenever these models are linear, the estimated parameters can be visualized in the form of brain maps which can aid in understanding how brain activity in space and time underlies a cognitive function. However, the recovered brain maps often suffer from lack of interpretability, especially in group analysis of multi-subject data. NEW METHOD To facilitate the application of brain decoding in group-level analysis, we present an application of multi-task joint feature learning for group-level multivariate pattern recovery in single-trial magnetoencephalography (MEG) decoding. The proposed method allows for recovering sparse yet consistent patterns across different subjects, and therefore enhances the interpretability of the decoding model. RESULTS Our experimental results demonstrate that the mutli-task joint feature learning framework is capable of recovering more meaningful patterns of varying spatio-temporally distributed brain activity across individuals while still maintaining excellent generalization performance. COMPARISON WITH EXISTING METHODS We compare the performance of the multi-task joint feature learning in terms of generalization, reproducibility, and quality of pattern recovery against traditional single-subject and pooling approaches on both simulated and real MEG datasets. CONCLUSIONS These results can facilitate the usage of brain decoding for the characterization of fine-level distinctive patterns in group-level inference. Considering the importance of group-level analysis, the proposed approach can provide a methodological shift towards more interpretable brain decoding models.

Sensor-level maps with the kernel two-sample test

Traditional approaches to create sensor-level maps from magnetoencephalographic (MEG) data rely on mass-univariate methods. In order to overcome some limitations of univariate approaches, multivariate approaches have been widely investigated, mostly based on the paradigm of classification. Recently a multivariate two-sample test called kernel two-sample test (KTST) has been proposed as an alternative to classification-based methods. Unfortunately the KTST lacks methods for neuroscientific interpretation of its result, e.g. in terms of sensor-level maps. In this work, we address this issue and we propose a cluster-based permutation kernel two-sample test (CBPKTST) to create sensor-level maps. Moreover we propose a procedure that massively reduces the computation so that maps can be produced in minutes. We report preliminary experiments on MEG data in which we show that the proposed procedure has much greater sensitivity than the traditional cluster-based permutation t-test.

Discrete Cosine Transform for MEG Signal Decoding

In this study, we propose the discrete cosine transform coefficients as a new and effective set of features for recognizing patterns of brain activity in MEG recording. We claim that computing DCT coefficients on the time-frequency representation of MEG signals is an efficient technique to reduce the dimensionality of feature space without losing discriminative power in brain decoding tasks. Our classification results on single-trial MEG decoding suggest that DCT is a viable method comparing to standard methods and it improves decoding accuracy by preserving the dynamic patterns of signal in time, frequency and space domains.

Interpretability of Multivariate Brain Maps in Brain Decoding: Definition and Quantification.

Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. Linear classifiers are widely employed in the brain decoding paradigm to discriminate among experimental conditions. Then, the derived linear weights are visualized in the form of multivariate brain maps to further study the spatio-temporal patterns of underlying neural activities. It is well known that the brain maps derived from weights of linear classifiers are hard to interpret because of high correlations between predictors, low signal to noise ratios, and the high dimensionality of neuroimaging data. Therefore, improving the interpretability of brain decoding approaches is of primary interest in many neuroimaging studies. Despite extensive studies of this type, at present, there is no formal definition for interpretability of multivariate brain maps. As a consequence, there is no quantitative measure for evaluating the interpretability of different brain decoding methods. In this paper, first, we present a theoretical definition of interpretability in brain decoding; we show that the interpretability of multivariate brain maps can be decomposed into their reproducibility and representativeness. Second, as an application of the proposed definition, we formalize a heuristic method for approximating the interpretability of multivariate brain maps in a binary magnetoencephalography (MEG) decoding scenario. Third, we pro pose to combine the approximated interpretability and the performance of the brain decoding into a new multi-objective criterion for model selection. Our results for the MEG data show that optimizing the hyper-parameters of the regularized linear classifier based on the proposed criterion results in more informative multivariate brain maps. More importantly, the presented definition provides the theoretical background for quantitative evaluation of interpretability, and hence, facilitates the development of more effective brain decoding algorithms in the future.