Carsten Mehring

Review of the BCI Competition IV

The BCI competition IV stands in the tradition of prior BCI competitions that aim to provide high quality neuroscientific data for open access to the scientific community. As experienced already in prior competitions not only scientists from the narrow field of BCI compete, but scholars with a broad variety of backgrounds and nationalities. They include high specialists as well as students. The goals of all BCI competitions have always been to challenge with respect to novel paradigms and complex data. We report on the following challenges: (1) asynchronous data, (2) synthetic, (3) multi-class continuous data, (4) session-to-session transfer, (5) directionally modulated MEG, (6) finger movements recorded by ECoG. As after past competitions, our hope is that winning entries may enhance the analysis methods of future BCIs.

Low-latency multi-threaded processing of neuronal signals for brain-computer interfaces.

Brain-computer interfaces (BCIs) require demanding numerical computations to transfer brain signals into control signals driving an external actuator. Increasing the computational performance of the BCI algorithms carrying out these calculations enables faster reaction to user inputs and allows using more demanding decoding algorithms. Here we introduce a modular and extensible software architecture with a multi-threaded signal processing pipeline suitable for BCI applications. The computational load and latency (the time that the system needs to react to user input) are measured for different pipeline implementations in typical BCI applications with realistic parameter settings. We show that BCIs can benefit substantially from the proposed parallelization: firstly, by reducing the latency and secondly, by increasing the amount of recording channels and signal features that can be used for decoding beyond the amount which can be handled by a single thread. The proposed software architecture provides a simple, yet flexible solution for BCI applications.

Online Adaptation and Over-Trial Learning in Macaque Visuomotor Control

When faced with unpredictable environments, the human motor system has been shown to develop optimized adaptation strategies that allow for online adaptation during the control process. Such online adaptation is to be contrasted to slower over-trial learning that corresponds to a trial-by-trial update of the movement plan. Here we investigate the interplay of both processes, i.e., online adaptation and over-trial learning, in a visuomotor experiment performed by macaques. We show that simple non-adaptive control schemes fail to perform in this task, but that a previously suggested adaptive optimal feedback control model can explain the observed behavior. We also show that over-trial learning as seen in learning and aftereffect curves can be explained by learning in a radial basis function network. Our results suggest that both the process of over-trial learning and the process of online adaptation are crucial to understand visuomotor learning.

Reaching Movement Onset- and End-Related Characteristics of EEG Spectral Power Modulations

The spectral power of intracranial field potentials shows movement-related modulations during reaching movements to different target positions that in frequencies up to the high-γ range (approximately 50 to above 200u2009Hz) can be reliably used for single-trial inference of movement parameters. However, identifying spectral power modulations suitable for single-trial analysis for non-invasive approaches remains a challenge. We recorded non-invasive electroencephalography (EEG) during a self-paced center-out and center-in arm movement task, resulting in eight reaching movement classes (four center-out, four center-in). We found distinct slow (≤5u2009Hz), μ (7.5–10u2009Hz), β (12.5–25u2009Hz), low-γ (approximately 27.5–50u2009Hz), and high-γ (above 50u2009Hz) movement onset- and end-related responses. Movement class-specific spectral power modulations were restricted to the β band at approximately 1u2009s after movement end and could be explained by the sensitivity of this response to different static, post-movement electromyography (EMG) levels. Based on the β band, significant single-trial inference of reaching movement endpoints was possible. The findings of the present study support the idea that single-trial decoding of different reaching movements from non-invasive EEG spectral power modulations is possible, but also suggest that the informative time window is after movement end and that the informative frequency range is restricted to the β band.

Grasp detection from human ECoG during natural reach-to-grasp movements.

Various movement parameters of grasping movements, like velocity or type of the grasp, have been successfully decoded from neural activity. However, the question of movement event detection from brain activity, that is, decoding the time at which an event occurred (e.g. movement onset), has been addressed less often. Yet, this may be a topic of key importance, as a brain-machine interface (BMI) that controls a grasping prosthesis could be realized by detecting the time of grasp, together with an optional decoding of which type of grasp to apply. We, therefore, studied the detection of time of grasps from human ECoG recordings during a sequence of natural and continuous reach-to-grasp movements. Using signals recorded from the motor cortex, a detector based on regularized linear discriminant analysis was able to retrieve the time-point of grasp with high reliability and only few false detections. Best performance was achieved using a combination of signal components from time and frequency domains. Sensitivity, measured by the amount of correct detections, and specificity, represented by the amount of false detections, depended strongly on the imposed restrictions on temporal precision of detection and on the delay between event detection and the time the event occurred. Including neural data from after the event into the decoding analysis, slightly increased accuracy, however, reasonable performance could also be obtained when grasping events were detected 125 ms in advance. In summary, our results provide a good basis for using detection of grasping movements from ECoG to control a grasping prosthesis.

Error-related electrocorticographic activity in humans during continuous movements

Brain-machine interface (BMI) devices make errors in decoding. Detecting these errors online from neuronal activity can improve BMI performance by modifying the decoding algorithm and by correcting the errors made. Here, we study the neuronal correlates of two different types of errors which can both be employed in BMI: (i) the execution error, due to inaccurate decoding of the subjects movement intention; (ii) the outcome error, due to not achieving the goal of the movement. We demonstrate that, in electrocorticographic (ECoG) recordings from the surface of the human brain, strong error-related neural responses (ERNRs) for both types of errors can be observed. ERNRs were present in the low and high frequency components of the ECoG signals, with both signal components carrying partially independent information. Moreover, the observed ERNRs can be used to discriminate between error types, with high accuracy (≥83%) obtained already from single electrode signals. We found ERNRs in multiple cortical areas, including motor and somatosensory cortex. As the motor cortex is the primary target area for recording control signals for a BMI, an adaptive motor BMI utilizing these error signals may not require additional electrode implants in other brain areas.

Unsupervised adaptation of brain machine interface decoders

The performance of neural decoders can degrade over time due to non-stationarities in the relationship between neuronal activity and behavior. In this case, brain-machine interfaces (BMI) require adaptation of their decoders to maintain high performance across time. One way to achieve this is by use of periodical calibration phases, during which the BMI system (or an external human demonstrator) instructs the user to perform certain movements or behaviors. This approach has two disadvantages: (i) calibration phases interrupt the autonomous operation of the BMI and (ii) between two calibration phases the BMI performance might not be stable but continuously decrease. A better alternative would be that the BMI decoder is able to continuously adapt in an unsupervised manner during autonomous BMI operation, i.e., without knowing the movement intentions of the user. In the present article, we present an efficient method for such unsupervised training of BMI systems for continuous movement control. The proposed method utilizes a cost function derived from neuronal recordings, which guides a learning algorithm to evaluate the decoding parameters. We verify the performance of our adaptive method by simulating a BMI user with an optimal feedback control model and its interaction with our adaptive BMI decoder. The simulation results show that the cost function and the algorithm yield fast and precise trajectories toward targets at random orientations on a 2-dimensional computer screen. For initially unknown and non-stationary tuning parameters, our unsupervised method is still able to generate precise trajectories and to keep its performance stable in the long term. The algorithm can optionally work also with neuronal error-signals instead or in conjunction with the proposed unsupervised adaptation.

fNIRS Exhibits Weak Tuning to Hand Movement Direction

Functional near-infrared spectroscopy (fNIRS) has become an established tool to investigate brain function and is, due to its portability and resistance to electromagnetic noise, an interesting modality for brain-machine interfaces (BMIs). BMIs have been successfully realized using the decoding of movement kinematics from intra-cortical recordings in monkey and human. Recently, it has been shown that hemodynamic brain responses as measured by fMRI are modulated by the direction of hand movements. However, quantitative data on the decoding of movement direction from hemodynamic responses is still lacking and it remains unclear whether this can be achieved with fNIRS, which records signals at a lower spatial resolution but with the advantage of being portable. Here, we recorded brain activity with fNIRS above different cortical areas while subjects performed hand movements in two different directions. We found that hemodynamic signals in contralateral sensorimotor areas vary with the direction of movements, though only weakly. Using these signals, movement direction could be inferred on a single-trial basis with an accuracy of ∼65% on average across subjects. The temporal evolution of decoding accuracy resembled that of typical hemodynamic responses observed in motor experiments. Simultaneous recordings with a head tracking system showed that head movements, at least up to some extent, do not influence the decoding of fNIRS signals. Due to the low accuracy, fNIRS is not a viable alternative for BMIs utilizing decoding of movement direction. However, due to its relative resistance to head movements, it is promising for studies investigating brain activity during motor experiments.

Variance Based Measure for Optimization of Parametric Realignment Algorithms.

Neuronal responses to sensory stimuli or neuronal responses related to behaviour are often extracted by averaging neuronal activity over large number of experimental trials. Such trial-averaging is carried out to reduce noise and to diminish the influence of other signals unrelated to the corresponding stimulus or behaviour. However, if the recorded neuronal responses are jittered in time with respect to the corresponding stimulus or behaviour, averaging over trials may distort the estimation of the underlying neuronal response. Temporal jitter between single trial neural responses can be partially or completely removed using realignment algorithms. Here, we present a measure, named difference of time-averaged variance (dTAV), which can be used to evaluate the performance of a realignment algorithm without knowing the internal triggers of neural responses. Using simulated data, we show that using dTAV to optimize the parameter values for an established parametric realignment algorithm improved its efficacy and, therefore, reduced the jitter of neuronal responses. By removing the jitter more effectively and, therefore, enabling more accurate estimation of neuronal responses, dTAV can improve analysis and interpretation of the neural responses.