Simon P. Kelly

Steady-state VEP-based brain-computer interface control in an immersive 3D gaming environment

This paper presents the application of an effective EEG-based brain-computer interface design for binary control in a visually elaborate immersive 3D game. The BCI uses the steady-state visual evoked potential (SSVEP) generated in response to phase-reversing checkerboard patterns. Two power-spectrum estimation methods were employed for feature extraction in a series of offline classification tests. Both methods were also implemented during real-time game play. The performance of the BCI was found to be robust to distracting visual stimulation in the game and relatively consistent across six subjects, with 41 of 48 games successfully completed. For the best performing feature extraction method, the average real-time control accuracy across subjects was 89%. The feasibility of obtaining reliable control in such a visually rich environment using SSVEPs is thus demonstrated and the impact of this result is discussed.

A supramodal accumulation-to-bound signal that determines perceptual decisions in humans

In theoretical accounts of perceptual decision-making, a decision variable integrates noisy sensory evidence and determines action through a boundary-crossing criterion. Signals bearing these very properties have been characterized in single neurons in monkeys, but have yet to be directly identified in humans. Using a gradual target detection task, we isolated a freely evolving decision variable signal in human subjects that exhibited every aspect of the dynamics observed in its single-neuron counterparts. This signal could be continuously tracked in parallel with fully dissociable sensory encoding and motor preparation signals, and could be systematically perturbed mid-flight during decision formation. Furthermore, we found that the signal was completely domain general: it exhibited the same decision-predictive dynamics regardless of sensory modality and stimulus features and tracked cumulative evidence even in the absence of overt action. These findings provide a uniquely clear view on the neural determinants of simple perceptual decisions in humans.

Dissociation in performance of children with ADHD and high-functioning autism on a task of sustained attention.

Attention deficit hyperactivity disorder (ADHD) and autism are two neurodevelopmental disorders associated with prominent executive dysfunction, which may be underpinned by disruption within fronto-striatal and fronto-parietal circuits. We probed executive function in these disorders using a sustained attention task with a validated brain-behaviour basis. Twenty-three children with ADHD, 21 children with high-functioning autism (HFA) and 18 control children were tested on the Sustained Attention to Response Task (SART). In a fixed sequence version of the task, children were required to withhold their response to a predictably occurring no-go target (3) in a 1–9 digit sequence; in the random version the sequence was unpredictable. The ADHD group showed clear deficits in response inhibition and sustained attention, through higher errors of commission and omission on both SART versions. The HFA group showed no sustained attention deficits, through a normal number of omission errors on both SART versions. The HFA group showed dissociation in response inhibition performance, as indexed by commission errors. On the Fixed SART, a normal number of errors was made, however when the stimuli were randomised, the HFA group made as many commission errors as the ADHD group. Greater slow-frequency variability in response time and a slowing in mean response time by the ADHD group suggested impaired arousal processes. The ADHD group showed greater fast-frequency variability in response time, indicative of impaired top-down control, relative to the HFA and control groups. These data imply involvement of fronto-parietal attentional networks and sub-cortical arousal systems in the pathology of ADHD and prefrontal cortex dysfunction in children with HFA.

Visual spatial attention tracking using high-density SSVEP data for independent brain-computer communication

The steady-state visual evoked potential (SSVEP) has been employed successfully in brain-computer interface (BCI) research, but its use in a design entirely independent of eye movement has until recently not been reported. This paper presents strong evidence suggesting that the SSVEP can be used as an electrophysiological correlate of visual spatial attention that may be harnessed on its own or in conjunction with other correlates to achieve control in an independent BCI. In this study, 64-channel electroencephalography data were recorded from subjects who covertly attended to one of two bilateral flicker stimuli with superimposed letter sequences. Offline classification of left/right spatial attention was attempted by extracting SSVEPs at optimal channels selected for each subject on the basis of the scalp distribution of SSVEP magnitudes. This yielded an average accuracy of approximately 71% across ten subjects (highest 86%) comparable across two separate cases in which flicker frequencies were set within and outside the alpha range respectively. Further, combining SSVEP features with attention-dependent parieto-occipital alpha band modulations resulted in an average accuracy of 79% (highest 87%).

Uncovering the Neural Signature of Lapsing Attention: Electrophysiological Signals Predict Errors up to 20 s before They Occur

The extent to which changes in brain activity can foreshadow human error is uncertain yet has important theoretical and practical implications. The present study examined the temporal dynamics of electrocortical signals preceding a lapse of sustained attention. Twenty-one participants performed a continuous temporal expectancy task, which involved continuously monitoring a stream of regularly alternating patterned stimuli to detect a rarely occurring target stimulus whose duration was 40% longer. The stimulus stream flickered at a rate of 25 Hz to elicit a steady-state visual-evoked potential (SSVEP), which served as a continuous measure of basic visual processing. Increasing activity in the α band (8–14 Hz) was found beginning ∼20 s before a missed target. This was followed by decreases in the amplitude of two event-related components over a short pretarget time frame: the frontal P3 (3–4 s) and contingent-negative variation (during the target interval). In contrast, SSVEP amplitude before hits and misses was closely matched, suggesting that the efficacy of ongoing basic visual processing was unaffected. Our results show that the specific neural signatures of attentional lapses are registered in the EEG up to 20 s before an error.

Visual spatial attention control in an independent brain-computer interface

This paper presents a novel brain computer interface (BCI) design employing visual evoked potential (VEP) modulations in a paradigm involving no dependency on peripheral muscles or nerves. The system utilizes electrophysiological correlates of visual spatial attention mechanisms, the self-regulation of which is naturally developed through continuous application in everyday life. An interface involving real-time biofeedback is described, demonstrating reduced training time in comparison to existing BCIs based on self-regulation paradigms. Subjects were cued to covertly attend to a sequence of letters superimposed on a flicker stimulus in one visual field while ignoring a similar stimulus of a different flicker frequency in the opposite visual field. Classification of left/right spatial attention is achieved by extracting steady-state visual evoked potentials (SSVEPs) elicited by the stimuli. Six out of eleven physically and neurologically healthy subjects demonstrate reliable control in binary decision-making, achieving at least 75% correct selections in at least one of only five sessions, each of approximately 12-min duration. The highest-performing subject achieved over 90% correct selections in each of four sessions. This independent BCI may provide a new method of real-time interaction for those with little or no peripheral control, with the added advantage of requiring only brief training.

Internal and External Influences on the Rate of Sensory Evidence Accumulation in the Human Brain

We frequently need to make timely decisions based on sensory evidence that is weak, ambiguous, or noisy resulting from conditions in the external environment (e.g., a cluttered visual scene) or within the brain itself (e.g., inattention, neural noise). Here we examine how externally and internally driven variations in the quality of sensory evidence affect the build-to-threshold dynamics of a supramodal “decision variable” signal and, hence, the timing and accuracy of decision reports in humans. Observers performed a continuous-monitoring version of the prototypical two-alternative dot-motion discrimination task, which is known to strongly benefit from sequential sampling and temporal accumulation of evidence. A centroparietal positive potential (CPP), which we previously established as a supramodal decision signal based on its invariance to motor or sensory parameters, exhibited two key identifying properties associated with the “decision variable” long described in sequential sampling models: (1) its buildup rate systematically scaled with sensory evidence strength across four levels of motion coherence, consistent with temporal integration; and (2) its amplitude reached a stereotyped level at the moment of perceptual report executions, consistent with a boundary-crossing stopping criterion. The buildup rate of the CPP also strongly predicted reaction time within coherence levels (i.e., independent of physical evidence strength), and this endogenous variation was linked with attentional fluctuations indexed by the level of parieto-occipital α-band activity preceding target onset. In tandem with the CPP, build-to-threshold dynamics were also observed in an effector-selective motor preparation signal; however, the buildup of this motor-specific process significantly lagged that of the supramodal process.

A parametric feature extraction and classification strategy for brain-computer interfacing

Parametric modeling strategies are explored in conjunction with linear discriminant analysis for use in an electroencephalogram (EEG)-based brain-computer interface (BCI). A left/right self-paced typing exercise is analyzed by extending the usual autoregressive (AR) model for EEG feature extraction with an AR with exogenous input (ARX) model for combined filtering and feature extraction. The ensemble averaged Bereitschaftspotential (an event related potential preceding the onset of movement) forms the exogenous signal input to the ARX model. Based on trials with six subjects, the ARX case of modeling both the signal and noise was found to be considerably more effective than modeling the noise alone (common in BCI systems) with the AR method yielding a classification accuracy of 52.8 /spl plusmn/ 4.8% and the ARX method an accuracy of 79.1 /spl plusmn/ 3.9% across subjects. The results suggest a role for ARX-based feature extraction in BCIs based on evoked and event-related potentials.

The strength of anticipatory spatial biasing predicts target discrimination at attended locations: a high-density EEG study

Cueing relevant spatial locations in advance of a visual target results in modulated processing of that target as a consequence of anticipatory attentional deployment, the neural signatures of which remain to be fully elucidated. A set of electrophysiological processes has been established as candidate markers of the invocation and maintenance of attentional bias in humans. These include spatially‐selective event‐related potential (ERP) components over the lateral parietal (around 200–300 ms post‐cue), frontal (300–500 ms) and ventral visual (> 500 ms) cortex, as well as oscillatory amplitude changes in the alpha band (8–14 Hz). Here, we interrogated the roles played by these anticipatory processes in attentional orienting by testing for links with subsequent behavioral performance. We found that both target discriminability (d’) and reaction times were significantly predicted on a trial‐by‐trial basis by lateralization of alpha‐band amplitude in the 500 ms preceding the target, with improved speed and accuracy resulting from a greater relative decrease in alpha over the contralateral visual cortex. Reaction time was also predicted by a late posterior contralateral positivity in the broad‐band ERP in the same time period, but this did not influence d’. In a further analysis we sought to identify the control signals involved in generating the anticipatory bias, by testing earlier broad‐band ERP amplitude for covariation with alpha lateralization. We found that stronger alpha biasing was associated with a greater bilateral frontal positivity at ∼390 ms but not with differential amplitude across hemispheres in any time period. Thus, during the establishment of an anticipatory spatial bias, while the expected target location is strongly encoded in lateralized activity in parietal and frontal areas, a distinct non‐spatial control process seems to regulate the strength of the bias.

Oscillatory Sensory Selection Mechanisms during Intersensory Attention to Rhythmic Auditory and Visual Inputs: A Human Electrocorticographic Investigation

Oscillatory entrainment mechanisms are invoked during attentional processing of rhythmically occurring stimuli, whereby their phase alignment regulates the excitability state of neurons coding for anticipated inputs. These mechanisms have been examined in the delta band (1–3 Hz), where entrainment frequency matches the stimulation rate. Here, we investigated entrainment for subdelta rhythmic stimulation, recording from intracranial electrodes over human auditory cortex during an intersensory audiovisual task. Audiovisual stimuli were presented at 0.67 Hz while participants detected targets within one sensory stream and ignored the other. It was found that entrainment operated at twice the stimulation rate (1.33 Hz), and this was reflected by higher amplitude values in the FFT spectrum, cyclic modulation of alpha-amplitude, and phase–amplitude coupling between delta phase and alpha power. In addition, we found that alpha-amplitude was relatively increased in auditory cortex coincident with to-be-ignored auditory stimuli during attention to vision. Thus, the data suggest that entrainment mechanisms operate within a delimited passband such that for subdelta task rhythms, oscillatory harmonics are invoked. The phase of these delta-entrained oscillations modulates alpha-band power. This may in turn increase or decrease responsiveness to relevant and irrelevant stimuli, respectively.