Yichuan Liu

Measuring speaker–listener neural coupling with functional near infrared spectroscopy

The present study investigates brain-to-brain coupling, defined as inter-subject correlations in the hemodynamic response, during natural verbal communication. We used functional near-infrared spectroscopy (fNIRS) to record brain activity of 3 speakers telling stories and 15 listeners comprehending audio recordings of these stories. Listeners’ brain activity was significantly correlated with speakers’ with a delay. This between-brain correlation disappeared when verbal communication failed. We further compared the fNIRS and functional Magnetic Resonance Imaging (fMRI) recordings of listeners comprehending the same story and found a significant relationship between the fNIRS oxygenated-hemoglobin concentration changes and the fMRI BOLD in brain areas associated with speech comprehension. This correlation between fNIRS and fMRI was only present when data from the same story were compared between the two modalities and vanished when data from different stories were compared; this cross-modality consistency further highlights the reliability of the spatiotemporal brain activation pattern as a measure of story comprehension. Our findings suggest that fNIRS can be used for investigating brain-to-brain coupling during verbal communication in natural settings.

Towards a Hybrid P300-Based BCI Using Simultaneous fNIR and EEG

Next generation brain computer interfaces (BCI) are expected to provide robust and continuous control mechanism. In this study, we assessed integration of optical brain imaging (fNIR: functional near infrared spectroscopy) to a P300-BCI for improving BCI usability by monitoring cognitive workload and performance. fNIR is a safe and wearable neuroimaging modality that tracks cortical hemodynamics in response to sensory, motor, or cognitive activation. Eight volunteers participated in the study where simultaneous EEG and 16 optode fNIR from anterior prefrontal cortex were recorded while participants engaged with the P300-BCI for spatial navigation. The results showed a significant response in fNIR signals during high, medium and low performance indicating a positive correlation between prefrontal oxygenation changes and BCI performance. This preliminary study provided evidence that the performance of P300-BCI can be monitored by fNIR which in turn can help improve the robustness of the BCI classification.

A P300-based EEG-BCI for spatial navigation control

In this study, a Brain Computer Interface (BCI) based on the P300 oddball paradigm has been developed for spatial navigation control in virtual environments. Functionality and efficacy of the system were analyzed with results from nine healthy volunteers. Each participant was asked to gaze at an individual target in a 3×3 P300 matrix containing different symbolic navigational icons while EEG signals were collected. Resulting ERPs were processed online and classification commands were executed to control spatial movements within the MazeSuite virtual environment and presented to the user online during an experiment. Subjects demonstrated on average, ~89% online accuracy for simple mazes and ~82% online accuracy in longer more complex mazes. Results suggest that this BCI setup enables guided free-form navigation in virtual 3D environments.

Basic psychological needs and neurophysiological responsiveness to decisional conflict: an event-related potential study of integrative self processes.

Fulfillment of the basic psychological needs for competence, relatedness, and autonomy is believed to facilitate people’s integrative tendencies to process psychological conflicts and develop a coherent sense of self. The present study therefore used event-related potentials (ERPs) to examine the relation between need fulfillment and the amplitude of conflict negativity (CN), a neurophysiological measure of conflict during personal decision making. Participants completed a decision-making task in which they made a series of forced choices according to their personal preferences. Three types of decision-making situations were created on the basis of participants’ unique preference ratings, which were obtained prior to ERP recording: low-conflict situations (choosing between an attractive and an unattractive option), high-conflict approach-approach situations (choosing between two similarly attractive options), and high-conflict avoidance-avoidance situations (choosing between two similarly unattractive options). As expected, CN amplitudes were larger in high- relative to low-conflict situations, and source localization analyses suggested that the anterior cingulate cortex was the generating structure of the CN. Most importantly, people reporting higher need fulfillment exhibited larger CN amplitudes in avoidance–avoidance situations relative to low-conflict situations; to a lesser extent, they also exhibited larger CN amplitudes in approach–approach situations relative to low-conflict situations. By contrast, people reporting lower need fulfillment exhibited CN amplitudes that poorly discriminated the three decision situations. These results suggest that need fulfillment may promote self-coherent functioning by increasing people’s receptivity to and processing of events that challenge their abilities to make efficient, self-congruent choices.

Mental workload classification with concurrent electroencephalography and functional near-infrared spectroscopy

AbstractA brain-computer interface that measures the mental workload level of operators has applications in human-computer interactions (HCI) for reducing human error and improving work efficiency. In this study, concurrently recorded electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were combined at the decision fusion stage for the classification of three mental workload levels induced by an n-back working-memory task. An average three-class classification accuracy of 42, 43, and 49% has been achieved across 13 participants for the fNIR-alone, EEG-alone, and EEG-fNIRS combined approach, respectively. The current study demonstrated a multimodality-based approach to decode human mental workload levels that may potentially be used for adaptive HCI applications.

Brain-in-the-Loop Learning Using fNIR and Simulated Virtual Reality Surgical Tasks: Hemodynamic and Behavioral Effects

Functional near infrared spectroscopy (fNIR) is a noninvasive, portable optical imaging tool to monitor changes in hemodynamic responses (i.e., oxygenated hemoglobin (HbO)) within the prefrontal cortex (PFC) in response to sensory, motor or cognitive activation. We used fNIR for monitoring PFC activation during learning of simulated laparoscopic surgical tasks throughout 4 days of training and testing. Blocked (BLK) and random (RND) practice orders were used to test the practice schedule effect on behavioral, hemodynamic responses and relative neural efficiency (EFFrel-neural) measures during transfer. Left and right PFC for both tasks showed significant differences with RND using less HbO than BLK. Cognitive workload showed RND exhibiting high EFFrel-neural across the PFC for the coordination task while the more difficult cholecystectomy task showed EFFrel-neural differences only in the left PFC. Use of brain activation, behavioral and EFFrel-neural measures can provide a more accurate depiction of the generalization or transfer of learning.

Detection of attention shift for asynchronous P300-based BCI

Brain-computer interface (BCI) provides patients suffering from severe neuromuscular disorders an alternative way of interacting with the outside world. The P300-based BCI is among the most popular paradigms in the field and most current versions operate in synchronous mode and assume participant engagement throughout operation. In this study, we demonstrate a new approach for assessment of user engagement through a hybrid classification of ERP and band power features of EEG signals that could allow building asynchronous BCIs. EEG signals from nine electrode locations were recorded from nine participants during controlled engagement conditions when subjects were either engaged with the P3speller task or not attending. Statistical analysis of band power showed that there were significant contrasts of attending only for the delta and beta bands as indicators of features for user attendance classification. A hybrid classifier using ERP scores and band power features yielded the best overall performance of 0.98 in terms of the area under the ROC curve (AUC). Results indicate that band powers can provide additional discriminant information to the ERP for user attention detection and this combined approach can be used to assess user engagement for each stimulus sequence during BCI use.

Multisubject “Learning” for Mental Workload Classification Using Concurrent EEG, fNIRS, and Physiological Measures

An accurate measure of mental workload level has diverse neuroergonomic applications ranging from brain computer interfacing to improving the efficiency of human operators. In this study, we integrated electroencephalogram (EEG), functional near-infrared spectroscopy (fNIRS), and physiological measures for the classification of three workload levels in an n-back working memory task. A significantly better than chance level classification was achieved by EEG-alone, fNIRS-alone, physiological alone, and EEG+fNIRS based approaches. The results confirmed our previous finding that integrating EEG and fNIRS significantly improved workload classification compared to using EEG-alone or fNIRS-alone. The inclusion of physiological measures, however, does not significantly improves EEG-based or fNIRS-based workload classification. A major limitation of currently available mental workload assessment approaches is the requirement to record lengthy calibration data from the target subject to train workload classifiers. We show that by learning from the data of other subjects, workload classification accuracy can be improved especially when the amount of data from the target subject is small.

Neural Adaptation to a Working Memory Task: A Concurrent EEG-fNIRS Study

Simultaneously recorded electroencephalography (EEG) and functional near infrared spectroscopy (fNIRS) measures from sixteen subjects were used to assess neural correlates of a letter based n-back working memory task. We found that EEG alpha power increased and prefrontal cortical oxygenation decreased with increased practice time for the high memory load condition (2-back), suggesting lower brain activation and a tendency toward the ‘idle’ state. The cortical oxygenation changes for the low memory load conditions (0-back and 1-back) changed very little throughout the training session which the behavioral scores showed high accuracy and a ceiling effect. No significant effect of practice time were found for theta power or the behavioral performance measures.

EEG band powers for characterizing user engagement in P300-BCI

An asynchronous P300-based brain computer interface (BCI) allows users to operate the BCI at their own pace by being able to detect a users engagement. In our previous work, band powers has been shown to be able to provide additional information for characterizing user engagement and yielded better performance compared to the use of only the amplitudes of event-related potentials. In this follow up study, 19 subjects participated in an experiment which was designed to further evaluate additional predictors of user engagement using band powers. In addition to the regular P300 attended condition, two not-engaged conditions were considered: one with the P300 stimulus matrix still shown (control 1) and the other with stimulus covered by a blank screen (control 2). Alpha and beta band activities decreased in the order of control 2, control 1 and attended. Furthermore, the attended condition had lower delta activity compared to the control conditions. Classification results indicated that band powers were better at differentiating attended and control 2 conditions. Using band powers as additional features resulted in a moderate to moderately large (dz= 0.52 to 0.74) improvement over the classification of the two conditions.