Günther Bauernfeind

The hybrid BCI

Nowadays, everybody knows what a hybrid car is. A hybrid car normally has two engines to enhance energy efficiency and reduce CO2 output. Similarly, a hybrid brain-computer interface (BCI) is composed of two BCIs, or at least one BCI and another system. A hybrid BCI, like any BCI, must fulfill the following four criteria: (i) the device must rely on signals recorded directly from the brain; (ii) there must be at least one recordable brain signal that the user can intentionally modulate to effect goal-directed behaviour; (iii) real time processing; and (iv) the user must obtain feedback. This paper introduces hybrid BCIs that have already been published or are in development. We also introduce concepts for future work. We describe BCIs that classify two EEG patterns: one is the event-related (de)synchronisation (ERD, ERS) of sensorimotor rhythms, and the other is the steady-state visual evoked potential (SSVEP). Hybrid BCIs can either process their inputs simultaneously, or operate two systems sequentially, where the first system can act as a “brain switch”. For example, we describe a hybrid BCI that simultaneously combines ERD and SSVEP BCIs. We also describe a sequential hybrid BCI, in which subjects could use a brain switch to control an SSVEP-based hand orthosis. Subjects who used this hybrid BCI exhibited about half the false positives encountered while using the SSVEP BCI alone. A brain switch can also rely on hemodynamic changes measured through near-infrared spectroscopy (NIRS). Hybrid BCIs can also use one brain signal and a different type of input. This additional input can be an electrophysiological signal such as the heart rate, or a signal from an external device such as an eye tracking system.

Development, set-up and first results for a one-channel near-infrared spectroscopy system Entwicklung, Aufbau und vorlaufige Ergebnisse eines Einkanal- Nahinfrarot-Spektroskopie-Systems

Abstract Near-infrared spectroscopy (NIRS) is a non-invasive optical technique that can be used to assess functional activity in the human brain. This work describes the set-up of a one-channel NIRS system designed for use as an optical brain-computer interface (BCI) and reports on first measurements of deoxyhemoglobin (Hb) and oxyhemoglobin (HbO2) changes during mental arithmetic tasks. We found relatively stable and reproducible hemodynamic responses in a group of 13 healthy subjects. Unexpected observations of a decrease in HbO2 and increase in Hb concentrations measured over the prefrontal cortex were in contrast to the typical hemodynamic responses (increase in HbO2, decrease in Hb) during cortical activation previously reported. Zusammenfassung Die Nahinfrarot-Spektroskopie (NIRS) ist eine nichtinvasive optische Technik, welche die Erfassung funktioneller Aktivitäten im menschlichen Gehirn ermöglicht. Diese Arbeit beschreibt den Aufbau eines Einkanal-NIRS-Systems, konzipiert für eine zukünftige Verwendung als optisches Brain-Computer-Interface. Weiters werden erste Ergebnisse der Konzentrationsänderungen von Oxyhämoglobin (HbO2) und Deoxyhämoglobin (Hb) während mentaler arithmetischer Aufgaben, gemessen an einer Gruppe von 13 gesunden Probanden, vorgestellt. Die Messposition wurde dabei über dem präfrontalen Kortex gewählt. Bei diesen Messungen wurden relativ stabil reproduzierbare hämodynamische Konzentrationsänderungen gefunden. Unerwartet war dabei die Messung einer Erhöhung der Hb- und einer Erniedrigung der HbO2-Konzentration bei Aktivierung, welche einem der Literatur gegenläufigen Verlauf entspricht.

Toward smarter BCIs: extending BCIs through hybridization and intelligent control

This paper summarizes two novel ways to extend brain-computer interface (BCI) systems. One way involves hybrid BCIs. A hybrid BCI is a system that combines a BCI with another device to help people send information. Different types of hybrid BCIs are discussed, along with challenges and issues. BCIs are also being extended through intelligent systems. Software that allows high-level control, incorporates context and the environment and/or uses virtual reality can substantially improve BCI systems. Throughout the paper, we critically address the real benefits of these improvements relative to existing technology and practices. We also present new challenges that are likely to emerge as these novel BCI directions become more widespread.

Cortical effects of user training in a motor imagery based brain-computer interface measured by fNIRS and EEG.

The present study aims to gain insights into the effects of training with a motor imagery (MI)-based brain-computer interface (BCI) on activation patterns of the sensorimotor cortex. We used functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) to investigate long-term training effects across 10 sessions using a 2-class (right hand and feet) MI-based BCI in fifteen subjects. In the course of the training a significant enhancement of activation pattern emerges, represented by an [oxy-Hb] increase in fNIRS and a stronger event-related desynchronization in the upper β-frequency band in the EEG. These effects were only visible in participants with relatively low BCI performance (mean accuracy ≤ 70%). We found that training with an MI-based BCI affects cortical activation patterns especially in users with low BCI performance. Our results may serve as a valuable contribution to the field of BCI research and provide information about the effects that training with an MI-based BCI has on cortical activation patterns. This might be useful for clinical applications of BCI which aim at promoting and guiding neuroplasticity.

Single-trial classification of antagonistic oxyhemoglobin responses during mental arithmetic

Near-infrared spectroscopy (NIRS) is a non-invasive optical technique that can be used for brain–computer interfaces (BCIs) systems. A common challenge for BCIs is a stable and reliable classification of single-trial data, especially for cognitive (mental) tasks. With antagonistic activation pattern, recently found for mental arithmetic (MA) tasks, an improved online classification for optical BCIs using MA should become possible. For this investigation, we used the data of a previous study where we found antagonistic activation patterns (focal bilateral increase of [oxy-Hb] in the dorsolateral prefrontal cortex in parallel with a [oxy-Hb] decrease in the medial area of the anterior prefrontal cortex) in eight subjects. We used the [oxy-Hb] responses to search for the best antagonistic feature combination and compared it to individual features from the same regions. In addition, we investigated the use of antagonistic [deoxy-Hb], total hemoglobin [Hbtot] and pairs of [oxy-Hb] and [deoxy-Hb] features as well as the existence of a group-related feature set. Our results indicate that the use of the antagonistic [oxy-Hb] features significantly increases the classification accuracy from 63.3 to 79.7%. These results support the hypothesis that antagonistic hemodynamic response patterns are a suitable control strategy for optical BCI, and that only two prefrontal NIRS channels are needed for good performance.

Coupling between Intrinsic Prefrontal HbO2 and Central EEG Beta Power Oscillations in the Resting Brain

There is increasing interest in the intrinsic activity in the resting brain, especially that of ultraslow and slow oscillations. Using near-infrared spectroscopy (NIRS), electroencephalography (EEG), blood pressure (BP), respiration and heart rate recordings during 5 minutes of rest, combined with cross spectral and sliding cross correlation calculations, we identified a short-lasting coupling (duration s) between prefrontal oxyhemoglobin (HbO2) in the frequency band between 0.07 and 0.13 Hz and central EEG alpha and/or beta power oscillations in 8 of the 9 subjects investigated. The HbO2 peaks preceded the EEG band power peaks by 3.7 s in 6 subjects, with moderate or no coupling between BP and HbO2 oscillations. HbO2 and EEG band power oscillations were approximately in phase with BP oscillations in the 2 subjects with an extremely high coupling (squared coherence ) between BP and HbO2 oscillation. No coupling was identified in one subject. These results indicate that slow precentral (de)oxyhemoglobin concentration oscillations during awake rest can be temporarily coupled with EEG fluctuations in sensorimotor areas and modulate the excitability level in the brains’ motor areas, respectively. Therefore, this provides support for the idea that resting state networks fluctuate with frequencies of between 0.01 and 0.1 Hz (Mantini et.al. PNAS 2007).

A haemodynamic brain-computer interface based on real-time classification of near infrared spectroscopy signals during motor imagery and mental arithmetic

Over the past decade, an increasing number of studies have investigated near infrared (NIR) spectroscopy for signal acquisition in brain–computer interface (BCI) systems. However, although a BCI relies on classifying brain signals in real-time, the majority of previous studies did not perform real-time NIR spectroscopy signal classification but derived knowledge about the feasibility of NIR spectroscopy for BCI purposes from offline analyses. The present study investigates whether NIR spectroscopy signals evoked by two different mental tasks (i.e. motor imagery and mental arithmetic) can be classified in real-time in order to control a NIR-BCI application. Furthermore, since this is the first study that attempts to distinguish between the haemodynamic responses to these two tasks, we aimed to investigate whether this task-combination is feasible for controlling a NIR-BCI. Twelve healthy participants were asked to control a moving ball on a computer screen by performing motor imagery and mental arithmetic tasks. The real-time classification of their task-specific NIR spectroscopy signals yielded accuracy rates ranging from 45% up to 93%. Offline analyses across all participants showed that both tasks evoked different haemodynamic responses in prefrontal and sensorimotor cortex areas. On the one hand, these results demonstrate the considerable potential of NIR spectroscopy for BCI signal acquisition and the feasibility of the applied mental tasks for NIR-BCI control. On the other hand, since the classification accuracy showed an unsatisfactory stability across measurement sessions, we conclude that further investigations and progress in methodological issues are needed and we discuss further steps that have to be taken until it is conceivable to implement a real-time capable NIR-BCI that works with sufficient accuracy across a large group of individuals.

Separating heart and brain: on the reduction of physiological noise from multichannel functional near-infrared spectroscopy (fNIRS) signals

OBJECTIVE Functional near-infrared spectroscopy (fNIRS) is an emerging technique for the in vivo assessment of functional activity of the cerebral cortex as well as in the field of brain-computer interface (BCI) research. A common challenge for the utilization of fNIRS in these areas is a stable and reliable investigation of the spatio-temporal hemodynamic patterns. However, the recorded patterns may be influenced and superimposed by signals generated from physiological processes, resulting in an inaccurate estimation of the cortical activity. Up to now only a few studies have investigated these influences, and still less has been attempted to remove/reduce these influences. The present study aims to gain insights into the reduction of physiological rhythms in hemodynamic signals (oxygenated hemoglobin (oxy-Hb), deoxygenated hemoglobin (deoxy-Hb)). APPROACH We introduce the use of three different signal processing approaches (spatial filtering, a common average reference (CAR) method; independent component analysis (ICA); and transfer function (TF) models) to reduce the influence of respiratory and blood pressure (BP) rhythms on the hemodynamic responses. MAIN RESULTS All approaches produce large reductions in BP and respiration influences on the oxy-Hb signals and, therefore, improve the contrast-to-noise ratio (CNR). In contrast, for deoxy-Hb signals CAR and ICA did not improve the CNR. However, for the TF approach, a CNR-improvement in deoxy-Hb can also be found. SIGNIFICANCE The present study investigates the application of different signal processing approaches to reduce the influences of physiological rhythms on the hemodynamic responses. In addition to the identification of the best signal processing method, we also show the importance of noise reduction in fNIRS data.

Does conscious intention to perform a motor act depend on slow prefrontal (de)oxyhemoglobin oscillations in the resting brain

Characteristically within the resting brain there are slow fluctuations (around 0.1Hz) of EEG and NIRS-(de)oxyhemoglobin ([deoxy-Hb], [oxy-Hb]) signals. An interesting question is whether such slow oscillations can be related to the intention to perform a motor act. To obtain an answer we analyzed continuous blood pressure (BP), heart rate (HR), prefrontal [oxy-Hb], [deoxy-Hb] and EEG signals over sensorimotor areas in 10 healthy subjects during 5min of rest and during 10min of voluntary finger movements. Analyses of prefrontal [oxy-Hb]/[deoxy-Hb] oscillations around 0.1Hz and central EEG band power changes in the beta (alpha) band revealed that the positive [oxy-Hb] peaks preceded the central EEG beta (alpha) power peak by 3.6±0.9s in the majority of subjects. A similar relationship between prefrontal [oxy-Hb] and central EEG beta power was found during voluntary movements whereby the post movement beta power increase (beta rebound) is known to coexist with a decreased excitability of cortico-spinal neurons. Therefore, we speculate that the beta power increase ∼3s after slow fluctuating [oxy-Hb] peaks during rest is indicative for a slow excitability change of central motor cortex neurons. This work provides the first evidence that initiation of finger movements at free will in relatively constant intervals around 10s could be temporally related to slow oscillations of prefrontal [oxy-Hb] and autonomic blood pressure in the resting brain.

Hemodynamic Signal Changes Accompanying Execution and Imagery of Swallowing in Patients with Dysphagia: A Multiple Single-Case Near-Infrared Spectroscopy Study.

In the present multiple case study, we examined hemodynamic changes in the brain in response to motor execution (ME) and motor imagery (MI) of swallowing in dysphagia patients compared to healthy matched controls using near-infrared spectroscopy (NIRS). Two stroke patients with cerebral lesions in the right hemisphere, two stroke patients with lesions in the brainstem, and two neurologically healthy control subjects actively swallowed saliva (ME) and mentally imagined to swallow saliva (MI) in a randomized order while changes in concentration of oxygenated hemoglobin (oxy-Hb) and deoxygenated hemoglobin (deoxy-Hb) were assessed. In line with recent findings in healthy young adults, MI and ME of swallowing led to the strongest NIRS signal change in the inferior frontal gyrus in stroke patients as well as in healthy elderly. We found differences in the topographical distribution and time course of the hemodynamic response in dependence on lesion location. Dysphagia patients with lesions in the brainstem showed bilateral hemodynamic signal changes in the inferior frontal gyrus during active swallowing comparable to healthy controls. In contrast, dysphagia patients with cerebral lesions in the right hemisphere showed more unilateral activation patterns during swallowing. Furthermore, patients with cerebral lesions showed a prolonged time course of the hemodynamic response during MI and ME of swallowing compared to healthy controls and patients with brainstem lesions. Brain activation patterns associated with ME and MI of swallowing were largely comparable, especially for changes in deoxy-Hb. Hence, the present results provide new evidence regarding timing and topographical distribution of the hemodynamic response during ME and MI of swallowing in dysphagia patients and may have practical impact on future dysphagia treatment.