Thorsten O. Zander

The hybrid BCI

Nowadays, everybody knows what a hybrid car is. A hybrid car normally has two engines to enhance energy efficiency and reduce CO2 output. Similarly, a hybrid brain-computer interface (BCI) is composed of two BCIs, or at least one BCI and another system. A hybrid BCI, like any BCI, must fulfill the following four criteria: (i) the device must rely on signals recorded directly from the brain; (ii) there must be at least one recordable brain signal that the user can intentionally modulate to effect goal-directed behaviour; (iii) real time processing; and (iv) the user must obtain feedback. This paper introduces hybrid BCIs that have already been published or are in development. We also introduce concepts for future work. We describe BCIs that classify two EEG patterns: one is the event-related (de)synchronisation (ERD, ERS) of sensorimotor rhythms, and the other is the steady-state visual evoked potential (SSVEP). Hybrid BCIs can either process their inputs simultaneously, or operate two systems sequentially, where the first system can act as a “brain switch”. For example, we describe a hybrid BCI that simultaneously combines ERD and SSVEP BCIs. We also describe a sequential hybrid BCI, in which subjects could use a brain switch to control an SSVEP-based hand orthosis. Subjects who used this hybrid BCI exhibited about half the false positives encountered while using the SSVEP BCI alone. A brain switch can also rely on hemodynamic changes measured through near-infrared spectroscopy (NIRS). Hybrid BCIs can also use one brain signal and a different type of input. This additional input can be an electrophysiological signal such as the heart rate, or a signal from an external device such as an eye tracking system.

A Dry EEG-System for Scientific Research and Brain–Computer Interfaces

Although it ranks among the oldest tools in neuroscientific research, electroencephalography (EEG) still forms the method of choice in a wide variety of clinical and research applications. In the context of brain–computer interfacing (BCI), EEG recently has become a tool to enhance human–machine interaction. EEG could be employed in a wider range of environments, especially for the use of BCI systems in a clinical context or at the homes of patients. However, the application of EEG in these contexts is impeded by the cumbersome preparation of the electrodes with conductive gel that is necessary to lower the impedance between electrodes and scalp. Dry electrodes could provide a solution to this barrier and allow for EEG applications outside the laboratory. In addition, dry electrodes may reduce the time needed for neurological exams in clinical practice. This study evaluates a prototype of a three-channel dry electrode EEG system, comparing it to state-of-the-art conventional EEG electrodes. Two experimental paradigms were used: first, event-related potentials (ERP) were investigated with a variant of the oddball paradigm. Second, features of the frequency domain were compared by a paradigm inducing occipital alpha. Furthermore, both paradigms were used to evaluate BCI classification accuracies of both EEG systems. Amplitude and temporal structure of ERPs as well as features in the frequency domain did not differ significantly between the EEG systems. BCI classification accuracies were equally high in both systems when the frequency domain was considered. With respect to the oddball classification accuracy, there were slight differences between the wet and dry electrode systems. We conclude that the tested dry electrodes were capable to detect EEG signals with good quality and that these signals can be used for research or BCI applications. Easy to handle electrodes may help to foster the use of EEG among a wider range of potential users.

Enhancing Human-Computer Interaction with Input from Active and Passive Brain-Computer Interfaces

This chapter introduces a formal categorization of BCIs, according to their key characteristics within HCI scenarios. This comprises classical approaches, which we group into active and reactive BCIs, and the new group of passive BCIs. Passive BCIs provide easily applicable and yet efficient interaction channels carrying information on covert aspects of user state, while adding little further usage cost. All of these systems can also be set up as hybrid BCIs, by incorporating information from outside the brain to make predictions, allowing for enhanced robustness over conventional approaches. With these properties, passive and hybrid BCIs are particularly useful in HCI. When any BCI is transferred from the laboratory to real-world situations, one faces new types of problems resulting from uncontrolled environmental factors—mostly leading to artifacts contaminating data and results. The handling of these situations is treated in a brief review of training and calibration strategies. The presented theory is then underpinned by two concrete examples. First, a combination of Event Related Desynchronization (ERD)-based active BCI with gaze control, defining a hybrid BCI as solution for the midas touch problem. And second, a passive BCI based on human error processing, leading to new forms of automated adaptation in HCI. This is in line with the results from other recent studies of passive BCI technology and shows the broad potential of this approach.

Combining Eye Gaze Input With a Brain–Computer Interface for Touchless Human–Computer Interaction

A Brain–Computer Interface (BCI) provides a new communication channel for severely disabled people who have completely or partially lost control over muscular activity. It is questionable whether a BCI is the best choice for controlling a device if partial muscular activity still is available. For example, gaze-based interfaces can be utilized for people who are still able to control their eye movements. Such interfaces suffer from the lack of a natural degree of freedom for the selection command (e.g., a mouse click). One workaround for this problem is based on so-called dwell times, which easily leads to errors if the users do not pay close attention to where they are looking. We developed a multimodal interface combining eye movements and a BCI to a hybrid BCI, resulting in a robust and intuitive device for touchless interaction. This system especially is capable of dealing with different stimulus complexities.

Context-aware brain-computer interfaces: exploring the information space of user, technical system and environment

Brain-computer interface (BCI) systems are usually applied in highly controlled environments such as research laboratories or clinical setups. However, many BCI-based applications are implemented in more complex environments. For example, patients might want to use a BCI system at home, and users without disabilities could benefit from BCI systems in special working environments. In these contexts, it might be more difficult to reliably infer information about brain activity, because many intervening factors add up and disturb the BCI feature space. One solution for this problem would be adding context awareness to the system. We propose to augment the available information space with additional channels carrying information about the user state, the environment and the technical system. In particular, passive BCI systems seem to be capable of adding highly relevant context information-otherwise covert aspects of user state. In this paper, we present a theoretical framework based on general human-machine system research for adding context awareness to a BCI system. Building on that, we present results from a study on a passive BCI, which allows access to the covert aspect of user state related to the perceived loss of control. This study is a proof of concept and demonstrates that context awareness could beneficially be implemented in and combined with a BCI system or a general human-machine system. The EEG data from this experiment are available for public download at www.phypa.org.

Detecting affective covert user states with passive brain-computer interfaces

Brain-Computer Interfaces (BCIs) provide insight into ongoing cognitive and affective processes and are commonly used for direct control of human-machine systems [16]. Recently, a different type of BCI has emerged [4, 17], which instead focuses solely on the non-intrusive recognition of mental state elicited by a given primary human-machine interaction. These so-called passive BCIs (pBCIs) do, by their nature, not disturb the primary interaction, and thus allow for enhancement of human-machine systems with relatively low usage cost [12,18], especially in conjunction with gel-free sensors. Here, we apply pBCIs to detect cognitive processes containing covert user states, which are difficult to access with conventional exogenous measures. We present two variants of a task inspired by an erroneously adapting human-machine system, a scenario important in automated adaptation. In this context, we derive two related, yet complementary, applications of pBCIs. First, we show that pBCIs are capable of detecting a covert user state related to the perception of loss of control over a system. The detection is realized by exploiting non-stationarities induced by the loss of control. Second, we show that pBCIs can be used to detect a covert user state directly correlated to the users interpretation of erroneous actions of the machine. We then demonstrate the use of this information to enhance the interaction between the user and the machine, in an experiment outside the laboratory.

Using neurophysiological signals that reflect cognitive or affective state: six recommendations to avoid common pitfalls

Estimating cognitive or affective state from neurophysiological signals and designing applications that make use of this information requires expertise in many disciplines such as neurophysiology, machine learning, experimental psychology, and human factors. This makes it difficult to perform research that is strong in all its aspects as well as to judge a study or application on its merits. On the occasion of the special topic “Using neurophysiological signals that reflect cognitive or affective state” we here summarize often occurring pitfalls and recommendations on how to avoid them, both for authors (researchers) and readers. They relate to defining the state of interest, the neurophysiological processes that are expected to be involved in the state of interest, confounding factors, inadvertently “cheating” with classification analyses, insight on what underlies successful state estimation, and finally, the added value of neurophysiological measures in the context of an application. We hope that this paper will support the community in producing high quality studies and well-validated, useful applications.

Utilizing Secondary Input from Passive Brain-Computer Interfaces for Enhancing Human-Machine Interaction

A Brain-Computer Interface (BCI) directly translates patterns of brain activity to input for controlling a machine. The introduction of methods from statistical machine learning [1] to the field of brain-computer interfacing (BCI) had a deep impact on classification accuracy. It also minimized the effort needed to build up the skill of controlling a BCI system [2]. This enabled other fields of research to adapt methods from BCI research for their own purposes [3, 4]. Team PhyPA, the research group for physiological parameters of the chair for Human-Machine Systems (HMS) of the Technical University of Berlin, focuses on enabling new communication channels for HMS. Especially the use of passive BCIs (pBCI) [3, 4], not dependent on any intended action of the user, show a high potential for enhancing the interaction in HMS [5]. Additionally, as actual classification rates are still below the threshold for efficient primary control [6, 7] in HMS, we focus on establishing a secondary, BCI-based communication channel. This kind of interaction does not necessarily disturb the primary mode of interaction, providing a low usage cost and hence an efficient way of enhancement. We have designed several applications following this approach. Here we are going to present briefly the results from two studies, which show the capabilities arising from the use of passive and secondary BCI interaction. First, we show that a pBCI can be utilized to gain valuable information about HMSs, which are hard to detect by exogeneous factors. By mimicking a typical BCI interaction, we have been able to identify and isolate a factor inducing non-stationarities with a deep impact on the feature dynamics. The retained information can be utilized for automatically triggered classifier adaptation. And second, we show that pBCIs are indeed capable to enhance common HMS interaction outside the laboratory. With this, we would like to feed back our experiences made with the use of BCIs for HMS. We hope to povide new and useful information about brain dynamics which might be helpful for ongoing research in augmented cognition.

MATLAB-Based Tools for BCI Research

We first discuss two MATLAB-centered solutions for real-time data streaming, the environments FieldTrip (Donders Institute, Nijmegen) and DataSuite (Data- River, Producer, MatRiver) (Swartz Center, La Jolla). We illustrate the relative simplicity of coding BCI feature extraction and classification under MATLAB (The Mathworks, Inc.) using a minimalist BCI example, and then describe BCILAB (Team PhyPa, Berlin), a new BCI package that uses the data structures and extends the capabilities of the widely used EEGLAB signal processing environment. We finally review the range of standalone and MATLAB-based software currently freely available to BCI researchers.

BC(eye): Combining Eye-Gaze Input with Brain-Computer Interaction

Gaze-based interfaces gained increasing importance in multimodal human-computer interaction research with the improvement of tracking technologies over the last few years. The activation of selected objects in most eye-controlled applications is based on dwell times. This interaction technique can easily lead to errors if the users do not pay very close attention to where they are looking. We developed a multimodal interface involving eye movements to determine the object of interest and a Brain-Computer Interface to simulate the mouse click. Experimental results show, that although a combined BCI/eye-gaze interface is somewhat slower it reliably leads to less errors in comparison to standard dwell time eye-gaze interfaces.