Christoph Pokorny

The auditory P300-based single-switch brain-computer interface

OBJECTIVE Within this work an auditory P300 brain-computer interface based on tone stream segregation, which allows for binary decisions, was developed and evaluated. METHODS AND MATERIALS Two tone streams consisting of short beep tones with infrequently appearing deviant tones at random positions were used as stimuli. This paradigm was evaluated in 10 healthy subjects and applied to 12 patients in a minimally conscious state (MCS) at clinics in Graz, Würzburg, Rome, and Liège. A stepwise linear discriminant analysis classifier with 10×10 cross-validation was used to detect the presence of any P300 and to investigate attentional modulation of the P300 amplitude. RESULTS The results for healthy subjects were promising and most classification results were better than random. In 8 of the 10 subjects, focused attention on at least one of the tone streams could be detected on a single-trial basis. By averaging 10 data segments, classification accuracies up to 90.6% could be reached. However, for MCS patients only a small number of classification results were above chance level and none of the results were sufficient for communication purposes. Nevertheless, signs of consciousness were detected in 9 of the 12 patients, not on a single-trial basis, but after averaging of all corresponding data segments and computing significant differences. These significant results, however, strongly varied across sessions and conditions. CONCLUSION This work shows the transition of a paradigm from healthy subjects to MCS patients. Promising results with healthy subjects are, however, no guarantee of good results with patients. Therefore, more investigations are required before any definite conclusions about the usability of this paradigm for MCS patients can be drawn. Nevertheless, this paradigm might offer an opportunity to support bedside clinical assessment of MCS patients and eventually, to provide them with a means of communication.

A SINGLE-SWITCH BCI BASED ON PASSIVE AND IMAGINED MOVEMENTS: TOWARD RESTORING COMMUNICATION IN MINIMALLY CONSCIOUS PATIENTS

We investigate whether an electroencephalography technique could be used for yes/no communication with auditory scanning. To be usable by the target group, i.e., minimally conscious individuals, such a brain-computer interface (BCI) has to be very simple and robust. This leads to the concept of a single-switch BCI (ssBCI). With an ssBCI it is possible to reliably detect one certain, individually trained, brain pattern of the individual, and use it to control all kinds of applications using yes/no responses. A total of 10 healthy volunteers (20-27 years) participated in an initial cue-based session with a motor imagery (MI) task after brisk passive feet/hand movement. Four of them reached MI classification accuracies above 70% and, thus, fulfilled the inclusion criterion for participation in the 2nd session. In the 2nd session, MI was used to communicate yes/no answers to a series of questions in an auditory scanning mode. Two of the three participants of the 2nd session were able to reliably communicate their intent with 90% or above correct and 0% false responses. This work showed, for the 1st time, the use of a ssBCI based on passive and imagined movements for communication in auditory scanning mode.

The auditory p300-based SSBCI: A door to minimally conscious patients?

In this study we report on the evaluation of a novel auditory single-switch BCI in nine patients diagnosed with MCS. The task included a simple and a complex oddball paradigm, the latter uses the tone stream segregation phenomenon. In all patients a significant difference between deviant and frequent tones could be observed in EEG. However, in some cases the deviant tones produce a significant negative peak and in some a very late positive peak. These preliminary findings are relevant in order to address future customization of this auditory ssBCI-based paradigm for unresponsive patients.

Detection of mental imagery and attempted movements in patients with disorders of consciousness using EEG

Further development of an EEG based communication device for patients with disorders of consciousness (DoC) could benefit from addressing the following gaps in knowledge—first, an evaluation of different types of motor imagery; second, an evaluation of passive feet movement as a mean of an initial classifier setup; and third, rapid delivery of biased feedback. To that end we investigated whether complex and/or familiar mental imagery, passive, and attempted feet movement can be reliably detected in patients with DoC using EEG recordings, aiming to provide them with a means of communication. Six patients in a minimally conscious state (MCS) took part in this study. The patients were verbally instructed to perform different mental imagery tasks (sport, navigation), as well as attempted feet movements, to induce distinctive event-related (de)synchronization (ERD/S) patterns in the EEG. Offline classification accuracies above chance level were reached in all three tasks (i.e., attempted feet, sport, and navigation), with motor tasks yielding significant (p < 0.05) results more often than navigation (sport: 10 out of 18 sessions; attempted feet: 7 out of 14 sessions; navigation: 4 out of 12 sessions). The passive feet movements, evaluated in one patient, yielded mixed results: whereas time-frequency analysis revealed task-related EEG changes over neurophysiological plausible cortical areas, the classification results were not significant enough (p < 0.05) to setup an initial classifier for the detection of attempted movements. Concluding, the results presented in this study are consistent with the current state of the art in similar studies, to which we contributed by comparing different types of mental tasks, notably complex motor imagery and attempted feet movements, within patients. Furthermore, we explored new venues, such as an evaluation of passive feet movement as a mean of an initial classifier setup, and rapid delivery of biased feedback.

Somatosensory evoked potentials elicited by stimulating two fingers from one hand — Usable for BCI?

Steady-state somatosensory evoked potentials (SSSEPs) have been elicited using vibro-tactile stimulation on two fingers of the right hand. Fourteen healthy subjects participated in this study. A screening session, stimulating each participants thumb, was conducted to determine individual optimal resonance-like frequencies. After this screening session, two stimulation frequencies per subject were selected. Stimulation was then applied simultaneously on the participants thumbs and middle finger. It was investigated whether it is possible to classify SSSEP changes based on an attention modulation task to determine possible BCI applications. A cue indicated the participants to shift their attention to either the thumb or the middle finger. Offline classification with a lock-in analyzer system (LAS) and a linear discriminant analysis (LDA) classifier was performed. One bipolar channel and no further optimization methods were used. All participants except one reached classification results above chance level classifying a reference period without focused attention against focused attention either to the thumb or the middle finger. Only two subjects reached accuracies above chance, classifying focused attention to the thumb vs. attention to the middle finger.

A hybrid three-class brain-computer interface system utilizing SSSEPs and transient ERPs.

OBJECTIVE This paper investigates the fusion of steady-state somatosensory evoked potentials (SSSEPs) and transient event-related potentials (tERPs), evoked through tactile simulation on the left and right-hand fingertips, in a three-class EEG based hybrid brain-computer interface. It was hypothesized, that fusing the input signals leads to higher classification rates than classifying tERP and SSSEP individually. APPROACH Fourteen subjects participated in the studies, consisting of a screening paradigm to determine person dependent resonance-like frequencies and a subsequent online paradigm. The whole setup of the BCI system was based on open interfaces, following suggestions for a common implementation platform. During the online experiment, subjects were instructed to focus their attention on the stimulated fingertips as indicated by a visual cue. The recorded data were classified during runtime using a multi-class shrinkage LDA classifier and the outputs were fused together applying a posterior probability based fusion. Data were further analyzed offline, involving a combined classification of SSSEP and tERP features as a second fusion principle. The final results were tested for statistical significance applying a repeated measures ANOVA. MAIN RESULTS A significant classification increase was achieved when fusing the results with a combined classification compared to performing an individual classification. Furthermore, the SSSEP classifier was significantly better in detecting a non-control state, whereas the tERP classifier was significantly better in detecting control states. Subjects who had a higher relative band power increase during the screening session also achieved significantly higher classification results than subjects with lower relative band power increase. SIGNIFICANCE It could be shown that utilizing SSSEP and tERP for hBCIs increases the classification accuracy and also that tERP and SSSEP are not classifying control- and non-control states with the same level of accuracy.

A Tactile Stimulation Device for EEG Measurements in Clinical Use

A tactile stimulation device for EEG measurements in clinical environments is proposed. The main purpose of the tactile stimulation device is to provide tactile stimulation to different parts of the body. To stimulate all four major types of mechanoreceptors, different stimulation patterns with frequencies in the range of 5-250 Hz have to be generated. The device provides two independent channels, delivers enough power to drive different types of electromagnetic transducers, is small and portable, and no expensive components are required to construct this device. The generated stimulation patterns are very stable, and deterministic control of the device is possible. To meet electrical safety requirements, the device was designed to be fully galvanically isolated. Leakage currents of the entire EEG measurement system including the tactile stimulation device were measured by the European Testing and Certifying Body for Medical Products Graz (Notified Body 0636). All measured currents were far below the maximum allowable currents defined in the safety standard EN 60601-1:2006 for medical electrical equipment. The successful operation of the tactile stimulation device was tested during an EEG experiment. The left and right wrist of one healthy subject were randomly stimulated with seven different frequencies. Steady-state somatosensory evoked potential (SSSEPs) could successfully be evoked and significant tuning curves at electrode positions contralateral to the stimulated wrist could be found. The device is ready to be used in clinical environment in a variety of applications to investigate the somatosensory system, in brain-computer interfaces (BCIs), or to provide tactile feedback.

Evaluation of Healthy EEG Responses for Spelling Through Listener-Assisted Scanning

We investigated whether listener-assisted scanning, an alternative communication method for persons with severe motor and visual impairments but preserved cognitive skills, could be used for spelling with EEG. To that end spoken letters were presented sequentially, and the participants made selections by performing motor execution/imagery or a cognitive task. The motor task was a brisk dorsiflexion of both feet, and the cognitive task was related to working memory and perception of human voice. The motor imagery task yielded the most promising results with respect to letter selection accuracy, albeit with a large variation in individual performance. The cognitive task yielded significant (p = 0.05) albeit moderate results. Closer inspection of grand average ERPs for the cognitive task revealed task-related modulation of a late negative component, which is novel in the auditory BCI literature. Guidelines for further development are presented.

Associations between memory traces emerge in a generic neural circuit model through STDP

Memory traces and associations between them are fundamental for cognitive brain function. Neuron recordings suggest that distributed assemblies of neurons in the brain serve as memory traces for spatial information, real-world items, and concepts. However, there is conflicting evidence regarding neural codes for associated memory traces. Some studies suggest the emergence of overlaps between assemblies during an association, while others suggest that the assemblies themselves remain largely unchanged and new assemblies emerge as neural codes for associated memory items. Here we study the emergence of neural codes for associated memory items in a generic computational model of recurrent networks of spiking neurons with a data-based rule for spike-timing-dependent plasticity (STDP). The model depends critically on two parameters, which control the excitability of neurons and the scale of initial synaptic weights. By modifying these two parameters, the model can reproduce both experimental data from the human brain on the fast formation of associations through emergent overlaps between assemblies, and rodent data where new neurons are recruited to encode the associated memories. Hence our findings suggest that the brain can use both of these two neural codes for associations, and dynamically switch between them during consolidation.The formation of associations between memory items, that enables recall of one memory component by activating another, is a fundamental operation of higher brain function. Recent neural recordings provide insight into the way how such associations are encoded on the level of neurons in the human medial temporal lobe (MTL). We show that important features of these experimental data can be reproduced by a generic neural circuit model consisting of excitatory and inhibitory spiking neurons with data-based short- and long-term synaptic plasticity. A key result of the experimental data and the model is that the association process causes the emergence of overlaps between the assemblies of neurons that encode the memory components. These overlaps appear in the experiments and the model at the same time when the association becomes computationally functional. Hence our model elucidates computational and plasticity processes that are likely to shape memory systems in the brain.