A deep multi-source adaptation transfer network for cross-subject electroencephalogram emotion recognition

Abstract

In real-world application of affective brain–computer interface (aBCI), individual differences across subjects and non-stationary characteristics of electroencephalogram (EEG) signals can cause data bias. Moreover, for new specific subject, the size of sample data is very small compared to that of existing subjects, which easily leads to overfitting in deep neural network training and reduces generalization performance of the network. In this paper, the deep multi-source adaptation transfer network (DMATN) is proposed for the new subjects in aBCI. In DMATN, the multi-source selection is employed to obtain the portion of existing EEG data mostly correlated with new subject and to decrease by two-fifth source data. To explore domain-invariant structures, deep adaptation network is used to map correlated source domain and the target domain (new subject) into reproducing kernel Hilbert space (RKHS) optimized by the multiple kernel variant of maximum mean discrepancies (MK-MMD). To more precisely predict the emotional state of the new subject, domain discriminator is applied in DMATN to make the data distribution of the two domains more similar. Finally, across-subject experiments on SEED dataset are conducted to evaluate the proposed method. The experimental results show that DMATN model can achieve the state-of-the-art performance of 84.46%, 83.32% and 84.90% in three sessions, respectively. It also shows great time efficiency in applications of aBCI.