Sergei L. Shishkin

Application of the change-point analysis to the investigation of the brain’s electrical activity

This chapter is devoted to one of the most interesting applications of non-parametric statistical diagnosis, namely, to the analysis of the human brain’s electrical activity (the electroencephalogram, or EEG). The meaning and the features of the EEG, as well as the problems arising from the high non-stationarity of the EEG signal, are reviewed. We present experimental results demonstrating the application of the statistical diagnosis methods described in this book to the EEG, and discuss the prospects for further development of the change-point detection methodology with the emphasis on the estimation of coupling between different signal channels.

Adapting the P300-Based Brain–Computer Interface for Gaming: A Review

The P300-based brain-computer interface (P300 BCI) is currently a very popular topic in assistive technology development. However, only a few simple P300 BCI-based games have been designed so far. Here, we analyze the shortcomings of this BCI in gaming applications and show that solutions for overcoming them already exist, although these techniques are dispersed over several different games. Additionally, new approaches to improve the P300 BCI accuracy and flexibility are currently being proposed in the more general P300 BCI research. The P300 BCI, even in its current form, not only exhibits relatively high speed and accuracy, but also can be used without user training, after a short calibration. Taking these facts together, the broader use of the P300 BCI in BCI-controlled video games is recommended.

Event-related potentials in a moving matrix modification of the P300 brain-computer interface paradigm

In the standard design of the brain-computer interfaces (BCI) based on the P300 component of the event-related potentials (ERP), target and non-target stimuli are presented at fixed positions in a motionless matrix. Can we let this matrix be moving (e.g., if attached to a robot) without loosing the efficiency of BCI? We assessed changes of the positive peak at Pz in the time interval 300-500 ms after the stimulus onset (P300) and the negative peak at the occipital electrodes in the range 140-240 ms (N1), both important for the operation of the P300 BCI, during fixating a target cell of a moving matrix in healthy participants (n=12). N1 amplitude in the difference (target-non-target) waveforms decreased with the velocity, although remained high (M=-4.3, SD=2.1) even at highest velocity (20°/s). In general, the amplitudes and latencies of these ERP components were remarkably stable in studied types of matrix movement and all velocities of horizontal movement (5, 10 and 20°/s) comparing to matrix in fixed position. These data suggest that, for the users controlling their gaze, the P300 BCI design can be extended to modifications requiring stimuli matrix motion.

A P300-based brain-computer interface with stimuli on moving objects: four-session single-trial and triple-trial tests with a game-like task design.

Brain-computer interfaces (BCIs) are tools for controlling computers and other devices without using muscular activity, employing user-controlled variations in signals recorded from the user’s brain. One of the most efficient noninvasive BCIs is based on the P300 wave of the brain’s response to stimuli and is therefore referred to as the P300 BCI. Many modifications of this BCI have been proposed to further improve the BCI’s characteristics or to better adapt the BCI to various applications. However, in the original P300 BCI and in all of its modifications, the spatial positions of stimuli were fixed relative to each other, which can impose constraints on designing applications controlled by this BCI. We designed and tested a P300 BCI with stimuli presented on objects that were freely moving on a screen at a speed of 5.4°/s. Healthy participants practiced a game-like task with this BCI in either single-trial or triple-trial mode within four sessions. At each step, the participants were required to select one of nine moving objects. The mean online accuracy of BCI-based selection was 81% in the triple-trial mode and 65% in the single-trial mode. A relatively high P300 amplitude was observed in response to targets in most participants. Self-rated interest in the task was high and stable over the four sessions (the medians in the 1st/4th sessions were 79/84% and 76/71% in the groups practicing in the single-trial and triple-trial modes, respectively). We conclude that the movement of stimulus positions relative to each other may not prevent the efficient use of the P300 BCI by people controlling their gaze, e.g., in robotic devices and in video games.

EEG Negativity in Fixations Used for Gaze-Based Control: Toward Converting Intentions into Actions with an Eye-Brain-Computer Interface

We usually look at an object when we are going to manipulate it. Thus, eye tracking can be used to communicate intended actions. An effective human-machine interface, however, should be able to differentiate intentional and spontaneous eye movements. We report an electroencephalogram (EEG) marker that differentiates gaze fixations used for control from spontaneous fixations involved in visual exploration. Eight healthy participants played a game with their eye movements only. Their gaze-synchronized EEG data (fixation-related potentials, FRPs) were collected during games control-on and control-off conditions. A slow negative wave with a maximum in the parietooccipital region was present in each participants averaged FRPs in the control-on conditions and was absent or had much lower amplitude in the control-off condition. This wave was similar but not identical to stimulus-preceding negativity, a slow negative wave that can be observed during feedback expectation. Classification of intentional vs. spontaneous fixations was based on amplitude features from 13 EEG channels using 300 ms length segments free from electrooculogram contamination (200–500 ms relative to the fixation onset). For the first fixations in the fixation triplets required to make moves in the game, classified against control-off data, a committee of greedy classifiers provided 0.90 ± 0.07 specificity and 0.38 ± 0.14 sensitivity. Similar (slightly lower) results were obtained for the shrinkage Linear Discriminate Analysis (LDA) classifier. The second and third fixations in the triplets were classified at lower rate. We expect that, with improved feature sets and classifiers, a hybrid dwell-based Eye-Brain-Computer Interface (EBCI) can be built using the FRP difference between the intended and spontaneous fixations. If this direction of BCI development will be successful, such a multimodal interface may improve the fluency of interaction and can possibly become the basis for a new input device for paralyzed and healthy users, the EBCI “Wish Mouse.”

P300-based brain-computer interface: The effect of the stimulus position in a stimulus train

The most popular type of brain-computer interfaces (BCIs) are based on the detection of the P300 wave of the evoked potentials appearing in response to a stimulus chosen by the subject. In order to increase the speed of operation of these BCIs, it is possible to decrease the number of repeated stimulus presentations. It is associated with an increase in the relative importance of the response to the first stimulus in a train for correct recognition of the stimulus chosen. Event-related potentials (ERPs) in response to the first stimulus presentations are known to have their own specificity. Particularly, in many cases, the amplitude of the response to the first presentations is enhanced, which makes it very suitable for recognition in a BCI. However, this effect has not been studied to date. In this study, the ERPs recorded in healthy subjects in a standard BCI paradigm (n = 14) with ten presentations of stimuli or during triple-trial (n = 6) and single-trial (n = 6) presentations of stimuli in a modified BCI paradigm with moving objects have been analyzed. In both cases, first presentations of the target stimuli or single-trial presentation of the target stimulus were associated with higher amplitudes of ERPs. The opportunity to use specific differences between the responses to the first or single-trial presentations and the responses to later stimuli during their repeated presentations for improving high-speed operations in the P300-based BCI is discussed.

On an Approach to the Estimation of the Complexity of Curves (Using as an Example an Electroencephalogram of a Human Being)

A new approach to the estimation of the “complexity” of a continuous curve on a section is suggested. The basic idea of the approach consists in that the complexity of the curve must be estimated by a relative share of information, which is necessary for its recovery with a specified accuracy from a set of values in the finite number of points by means of a prescribed aggregate of methods. The suggested approach is illustrated by an estimate of the complexity of various fragments of an electroencephalogram (EEG) of a human being and can be used, in particular, for the macrostructural and microstructural analysis of the EEG.

Gaze based robot control: The communicative approach

We propose a novel way of robotic device control with communicative eye movements that could possibly help to solve the problem of false activations during the gaze control, known as the Midas touch problem. The proposed approach can be considered as explicitly based on communication between a human operator and a robot. Specifically, we employed gaze patterns that are characteristic for “joint attention” type of communication between two persons. “Joint attention” gaze patterns are automatized and able to convey information about object location even under a high cognitive load. Therefore, we assumed that they may make robot control with gaze more stable. In a study with 28 healthy participants who were naive to this approach most of them easily acquired robot control with “joint attention” gaze patterns. The study did not reveal higher preference for communicative type of control, possibly because the participants did not practice before the tests. We discuss potential benefits of the new approach that can be tested in future studies.

The Expectation Based Eye-Brain-Computer Interface: An Attempt of Online Test

In this preliminary study we tested online a new Eye-Brain-Computer Interface (EBCI) for selection of positions on a screen with a combination of gaze based control and a passive brain-computer interface (BCI). This hybrid BCI was trained offline to recognize the electroencephalogram (EEG) patterns recorded during gaze dwells intentionally used to make moves in a computer game. The patterns were presumably related to expectation of the interface feedback. In the online test, 500 ms gaze dwells led to actions each time the BCI classified them as intentional. When the BCI made a miss, a participant could still communicate the intention by prolonging the dwell up to 1000 ms. Also playing the game was possible, it was found that defining the ground truth for such an online system is not trivial and that further efforts will be needed to evaluate the performance of the expectation based EBCI reliably.