Yufeng Ke

An EEG-based mental workload estimator trained on working memory task can work well under simulated multi-attribute task

Mental workload (MW)-based adaptive system has been found to be an effective approach to enhance the performance of human-machine interaction and to avoid human error caused by overload. However, MW estimated from the spontaneously generated electroencephalogram (EEG) was found to be task-specific. In existing studies, EEG-based MW classifier can work well under the task used to train the classifier (within-task) but crash completely when used to classify MW of a task that is similar to but not included in the training data (cross-task). The possible causes have been considered to be the task-specific EEG patterns, the mismatched workload across tasks and the temporal effects. In this study, cross-task performance-based feature selection (FS) and regression model were tried to cope with these challenges, in order to make EEG-based MW estimator trained on working memory tasks work well under a complex simulated multi-attribute task (MAT). The results show that the performance of regression model trained on working memory task and tested on multi-attribute task with the feature subset picked-out were significantly improved (correlation coefficient (COR): 0.740 ± 0.147 and 0.598 ± 0.161 for FS data and validation data respectively) when compared to the performance in the same condition with all features (chance level). It can be inferred that there do exist some MW-related EEG features can be picked out and there are something in common between MW of a relatively simple task and a complex task. This study provides a promising approach to measure MW across tasks.

Towards an effective cross-task mental workload recognition model using electroencephalography based on feature selection and support vector machine regression.

Electroencephalographic (EEG) has been believed to be a potential psychophysiological measure of mental workload. There however remain a number of challenges in building a generalized mental workload recognition model, one of which includes the inability of an EEG-based workload classifier trained on a specific task to handle other tasks. The primary goal of the present study was to examine the possibility of addressing this challenge using feature selection and regression model. Support vector machine classifier and regression models were examined under within-task conditions (trained and tested on the same task) and cross-task conditions (trained on one task and tested on another task) for well-trained verbal and spatial n-back tasks. A specifically designed cross-task recursive feature elimination (RFE) based feature selection was used to handle the possible causes responsible for the deterioration of the performance of cross-task regression model. The within-task classification and regression performed fairly well. Cross-task classification and regression performance, however, deteriorated to unacceptable levels (around chance level). Trained and tested with the most robust feature subset selected by cross-task RFE, the performance of cross-task regression was significantly improved, and there were no significant changes in the performance of within-task regression. It can be inferred that workload-related features can be picked out from those which have been contaminated using RFE, and regression models rather than classifiers may be a wiser choice for cross-task conditions. These encouraging results suggest that the cross-task workload recognition model built in this study is much more generalizable across task when compared to the model built in traditional way.

Visual Attention Recognition Based on Nonlinear Dynamical Parameters of EEG

Varieties of neurophysiological measures have been utilized in visual attention studies. The linear parameters like power spectrum are the most commonly used features in the existing studies. In this paper, however, nonlinear parameters including approximate entropy, sample entropy and multiscale entropy were tested. All subjects were instructed to perform tasks with three different attention levels (i.e. attention, no attention and rest) in two experiments. Nonlinear features were extracted from the EEG signals. Then, statistical analyses and classification with support vector machine (SVM) were performed. A comparison between the classification results based on the linear feature / and the sample entropy was performed for further analysis. The results suggest that sample entropy stands out in the dynamical parameters with the accuracies of 76.19% and 85.24% in recognition of three levels of attention for the two experiments respectively. And the further comparison shows that the sample entropy performs even better than the / power ratio. It is suggested that nonlinear dynamical parameters may be indispensable for a robust attention recognition system.

Training and testing ERP-BCIs under different mental workload conditions

OBJECTIVE As one of the most popular and extensively studied paradigms of brain-computer interfaces (BCIs), event-related potential-based BCIs (ERP-BCIs) are usually built and tested in ideal laboratory settings in most existing studies, with subjects concentrating on stimuli and intentionally avoiding possible distractors. This study is aimed at examining the effect of simultaneous mental activities on ERP-BCIs by manipulating various levels of mental workload during the training and/or testing of an ERP-BCI. APPROACH Mental workload was manipulated during the training or testing of a row-column P300-speller to investigate how and to what extent the spelling performance and the ERPs evoked by the oddball stimuli are affected by simultaneous mental workload. MAIN RESULTS Responses of certain ERP components, temporal-occipital N200 and the late reorienting negativity evoked by the oddball stimuli and the classifiability of ERP features between targets and non-targets decreased with the increase of mental workload encountered by the subject. However, the effect of mental workload on the performance of ERP-BCI was not always negative but depended on the conditions where the ERP-BCI was built and applied. The performance of ERP-BCI built under an ideal lab setting without any irrelevant mental activities declined with the increasing mental workload of the testing data. However, the performance was significantly improved when an ERP-BCI was built under an appropriate mental workload level, compared to that built under speller-only conditions. SIGNIFICANCE The adverse effect of concurrent mental activities may present a challenge for ERP-BCIs trained in ideal lab settings but which are to be used in daily work, especially when users are performing demanding mental processing. On the other hand, the positive effects of the mental workload of the training data suggest that introducing appropriate mental workload during training ERP-BCIs is of potential benefit to the performance in practical applications.

Enhancing performance of P300-Speller under mental workload by incorporating dual-task data during classifier training

As one of the most important brain-computer interface (BCI) paradigms, P300-Speller was shown to be significantly impaired once applied in practical situations due to effects of mental workload. This study aims to provide a new method of building training models to enhance performance of P300-Speller under mental workload. Three experiment conditions based on row-column P300-Speller paradigm were performed including speller-only, 3-back-speller and mental-arithmetic-speller. Data under dual-task conditions were introduced to speller-only data respectively to build new training models. Then performance of classifiers with different models was compared under the same testing condition. The results showed that when tasks of imported training data and testing data were the same, character recognition accuracies and round accuracies of P300-Speller with mixed-data training models significantly improved (FDR, p < 0.005). When they were different, performance significantly improved when tested on mental-arithmetic-speller (FDR, p < 0.05) while the improvement was modest when tested on n-back-speller (FDR, p < 0.1). The analysis of ERPs revealed that ERP difference between training data and testing data was significantly diminished when the dual-task data was introduced to training data (FDR, p < 0.05). The new method of training classifier on mixed data proved to be effective in enhancing performance of P300-Speller under mental workload, confirmed the feasibility to build a universal training model and overcome the effects of mental workload in its practical applications.

Concurrent mental activities affect ERPs and impair performance of ERP-spellers

Being one of the most popular and extensively studied ERP-BCI paradigm, ERP-spellers are commonly built and tested in ideal lab settings. However, in practical applications, users may encounter complex situations and various mental processing that have been believed to affect ERP signals. This kind of effect will probably induce a deteriorated performance of ERP-spellers. In the current study, a working memory task was interleaved within an RC ERP-speller paradigm to examine the effect of concurrent mental processing both on ERPs evoked by the stimuli of speller and its performance, especially when a speller is built under pure lab setting but used under complex mental activities in real life. The results show that the amplitude of N200, P360 and N550 were significantly affected by the working memory task. Moreover, the performance of ERP-spellers was significantly deteriorated by concurrently performing a working memory task, not only when a speller is trained and used in different settings, but also when a speller is both built and used in the same complex setting. These findings introduce a challenge for ERP-spellers to be used outside lab-settings and in daily work, especially when users are undergoing complex mental processing and experiencing heavy mental workload.

STIMULUS ARTIFACT REMOVAL OF SEMG SIGNALS DETECTED DURING FUNCTIONAL ELECTRICAL STIMULATION

The recording and interpretation of surface elec- tromyography (SEMG) under functional electrical stimula- tion (FES) is a useful technique for diagnostic, prognostic and therapeutic purposes. However, the stimulus arti- fact(SA) is a particularly troublesome form of interfer- ence. The paper developed a software-based two-level peak detection technique for SA removal. The results showed by repeatedly changing the values of the peak threshold, the entire SA was removed as expectation leav- ing the uncontaminated SEMG remaining.

The timing of theta phase synchronization accords with vigilant attention

Vigilant attention plays an important role in some industries and everyday life. However, its mechanism relating to phase synchronization of cortical oscillations is still unknown, which hinders the development of predicting and preventing vigilant attentional deficit. This study utilized psychomotor vigilance test (PVT) to elicit vigilance decrement. High and low levels of vigilant attention were represented by short and long reaction time, respectively. Electroencephalogram (EEG) was collected and phase synchronization between prefrontal and parietal cortices was analyzed by using debiased weighted phase lag index (dWPLI). The result suggests that vigilant attention of high level has earlier timing of theta (4–8Hz) phase synchronization, compared with that of low level. We concluded that phase synchronization may relate closely with the variation of vigilant attention.Vigilant attention plays an important role in some industries and everyday life. However, its mechanism relating to phase synchronization of cortical oscillations is still unknown, which hinders the development of predicting and preventing vigilant attentional deficit. This study utilized psychomotor vigilance test (PVT) to elicit vigilance decrement. High and low levels of vigilant attention were represented by short and long reaction time, respectively. Electroencephalogram (EEG) was collected and phase synchronization between prefrontal and parietal cortices was analyzed by using debiased weighted phase lag index (dWPLI). The result suggests that vigilant attention of high level has earlier timing of theta (4-8Hz) phase synchronization, compared with that of low level. We concluded that phase synchronization may relate closely with the variation of vigilant attention.

The effects of semantic congruency: a research of audiovisual P300-speller

BackgroundOver the past few decades, there have been many studies of aspects of brain–computer interface (BCI). Of particular interests are event-related potential (ERP)-based BCI spellers that aim at helping mental typewriting. Nowadays, audiovisual unimodal stimuli based BCI systems have attracted much attention from researchers, and most of the existing studies of audiovisual BCIs were based on semantic incongruent stimuli paradigm. However, no related studies had reported that whether there is difference of system performance or participant comfort between BCI based on semantic congruent paradigm and that based on semantic incongruent paradigm.MethodsThe goal of this study was to investigate the effects of semantic congruency in system performance and participant comfort in audiovisual BCI. Two audiovisual paradigms (semantic congruent and incongruent) were adopted, and 11 healthy subjects participated in the experiment. High-density electrical mapping of ERPs and behavioral data were measured for the two stimuli paradigms.ResultsThe behavioral data indicated no significant difference between congruent and incongruent paradigms for offline classification accuracy. Nevertheless, eight of the 11 participants reported their priority to semantic congruent experiment, two reported no difference between the two conditions, and only one preferred the semantic incongruent paradigm. Besides, the result indicted that higher amplitude of ERP was found in incongruent stimuli based paradigm.ConclusionsIn a word, semantic congruent paradigm had a better participant comfort, and maintained the same recognition rate as incongruent paradigm. Furthermore, our study suggested that the paradigm design of spellers must take both system performance and user experience into consideration rather than merely pursuing a larger ERP response.

Research on multi-dimensional N-back task induced EEG variations

In order to test the effectiveness of multi-dimensional N-back task for inducing deeper brain fatigue, we conducted a series of N*L-back experiments: 1*1-back, 1*2-back, 2*1-back and 2*2-back tasks. We analyzed and compared the behavioral results, EEG variations and mutual information among these four different tasks. There was no significant difference in average EEG power and power spectrum entropy (PSE) among the tasks. However, the behavioral result of N*2-back task showed significant difference compared to traditional one dimensional N-back task. Connectivity changes were observed with the addition of one more matching task in N-back. We suggest that multi-dimensional N-back task consume more brain resources and activate different brain areas. These results provide a basis for multi-dimensional N-back tasks that can be used to induce deeper mental fatigue or exert more workload.