José P. Ossandón

Combining EEG and eye tracking: identification, characterization, and correction of eye movement artifacts in electroencephalographic data

Eye movements introduce large artifacts to electroencephalographic recordings (EEG) and thus render data analysis difficult or even impossible. Trials contaminated by eye movement and blink artifacts have to be discarded, hence in standard EEG-paradigms subjects are required to fixate on the screen. To overcome this restriction, several correction methods including regression and blind source separation have been proposed. Yet, there is no automated standard procedure established. By simultaneously recording eye movements and 64-channel-EEG during a guided eye movement paradigm, we investigate and review the properties of eye movement artifacts, including corneo-retinal dipole changes, saccadic spike potentials and eyelid artifacts, and study their interrelations during different types of eye- and eyelid movements. In concordance with earlier studies our results confirm that these artifacts arise from different independent sources and that depending on electrode site, gaze direction, and choice of reference these sources contribute differently to the measured signal. We assess the respective implications for artifact correction methods and therefore compare the performance of two prominent approaches, namely linear regression and independent component analysis (ICA). We show and discuss that due to the independence of eye artifact sources, regression-based correction methods inevitably over- or under-correct individual artifact components, while ICA is in principle suited to address such mixtures of different types of artifacts. Finally, we propose an algorithm, which uses eye tracker information to objectively identify eye-artifact related ICA-components (ICs) in an automated manner. In the data presented here, the algorithm performed very similar to human experts when those were given both, the topographies of the ICs and their respective activations in a large amount of trials. Moreover it performed more reliable and almost twice as effective than human experts when those had to base their decision on IC topographies only. Furthermore, a receiver operating characteristic (ROC) analysis demonstrated an optimal balance of false positive and false negative at an area under curve (AUC) of more than 0.99. Removing the automatically detected ICs from the data resulted in removal or substantial suppression of ocular artifacts including microsaccadic spike potentials, while the relevant neural signal remained unaffected. In conclusion the present work aims at a better understanding of individual eye movement artifacts, their interrelations and the respective implications for eye artifact correction. Additionally, the proposed ICA-procedure provides a tool for optimized detection and correction of eye movement-related artifact components.

Spatial biases in viewing behavior

Viewing behavior exhibits temporal and spatial structure that is independent of stimulus content and task goals. One example of such structure is horizontal biases, which are likely rooted in left-right asymmetries of the visual and attentional systems. Here, we studied the existence, extent, and mechanisms of this bias. Left- and right-handed subjects explored scenes from different image categories, presented in original and mirrored versions. We also varied the spatial spectral content of the images and the timing of stimulus onset. We found a marked leftward bias at the start of exploration that was independent of image category. This left bias was followed by a weak bias to the right that persisted for several seconds. This asymmetry was found in the majority of right-handers but not in left-handers. Neither low- nor high-pass filtering of the stimuli influenced the bias. This argues against mechanisms related to the hemispheric segregation of global versus local visual processing. Introducing a delay in stimulus onset after offset of a central fixation spot also had no influence. The bias was present even when stimuli were presented continuously and without any requirement to fixate, associated to both fixation- and saccade-contingent image changes. This suggests the bias is not caused by structural asymmetries in fixation control. Instead the pervasive horizontal bias is compatible with known asymmetries of higher-level attentional areas related to the detection of novel events.

Superposition Model Predicts EEG Occipital Activity during Free Viewing of Natural Scenes

Visual event-related potentials (ERPs) produced by a stimulus are thought to reflect either an increase of synchronized activity or a phase realignment of ongoing oscillatory activity, with both mechanisms sharing the assumption that ERPs are independent of the current state of the brain at the time of stimulation. In natural viewing, however, visual inputs occur one after another at specific subject-paced intervals through unconstrained eye movements. We conjecture that during natural viewing, ERPs generated after each fixation are better explained by a superposition of ongoing oscillatory activity related to the processing of previous fixations, with new activity elicited by the visual input at the current fixation. We examined the electroencephalography (EEG) signals that occur in humans at the onset of each visual fixation, both while subjects freely viewed natural scenes and while they viewed a black or gray background. We found that the fixation ERPs show visual components that are absent when subjects move their eyes on a homogeneous gray or black screen. Single-trial EEG signals that comprise the ERP are predicted more accurately by a model of superposition than by either phase resetting or the addition of evoked responses and stimulus-independent noise. The superposition of ongoing oscillatory activity and the visually evoked response results in a modification of the ongoing oscillation phase. The results presented suggest that the observed EEG signals reflect changes occurring in a common neuronal substrate rather than a simple summation at the scalp of signals from independent sources.

Unmasking the contribution of low-level features to the guidance of attention

The role of low-level stimulus-driven control in the guidance of overt visual attention has been difficult to establish because low- and high-level visual content are spatially correlated within natural visual stimuli. Here we show that impairment of parietal cortical areas, either permanently by a lesion or reversibly by repetitive transcranial magnetic stimulation (rTMS), leads to fixation of locations with higher values of low-level features as compared to control subjects or in a no-rTMS condition. Moreover, this unmasking of stimulus-driven control crucially depends on the intrahemispheric balance between top-down and bottom-up cortical areas. This result suggests that although in normal behavior high-level features might exert a strong influence, low-level features do contribute to guide visual selection during the exploration of complex natural stimuli.

Predictions of Visual Content across Eye Movements and Their Modulation by Inferred Information

The brain is proposed to operate through probabilistic inference, testing and refining predictions about the world. Here, we search for neural activity compatible with the violation of active predictions, learned from the contingencies between actions and the consequent changes in sensory input. We focused on vision, where eye movements produce stimuli shifts that could, in principle, be predicted. We compared, in humans, error signals to saccade-contingent changes of veridical and inferred inputs by contrasting the electroencephalographic activity after saccades to a stimulus presented inside or outside the blind spot. We observed early (<250 ms) and late (>250 ms) error signals after stimulus change, indicating the violation of sensory and associative predictions, respectively. Remarkably, the late response was diminished for blind-spot trials. These results indicate that predictive signals occur across multiple levels of the visual hierarchy, based on generative models that differentiate between signals that originate from the outside world and those that are inferred.

The dynamic effect of reading direction habit on spatial asymmetry of image perception.

Exploration of images after stimulus onset is initially biased to the left. Here, we studied the causes of such an asymmetry and investigated effects of reading habits, text primes, and priming by systematically biased eye movements on this spatial bias in visual exploration. Bilinguals first read text primes with right-to-left (RTL) or left-to-right (LTR) reading directions and subsequently explored natural images. In Experiment 1, native RTL speakers showed a leftward free-viewing shift after reading LTR primes but a weaker rightward bias after reading RTL primes. This demonstrates that reading direction dynamically influences the spatial bias. However, native LTR speakers who learned an RTL language late in life showed a leftward bias after reading either LTR or RTL primes, which suggests the role of habit formation in the production of the spatial bias. In Experiment 2, LTR bilinguals showed a slightly enhanced leftward bias after reading LTR text primes in their second language. This might contribute to the differences of native RTL and LTR speakers observed in Experiment 1. In Experiment 3, LTR bilinguals read normal (LTR, habitual reading) and mirrored left-to-right (mLTR, nonhabitual reading) texts. We observed a strong leftward bias in both cases, indicating that the bias direction is influenced by habitual reading direction and is not secondary to the actual reading direction. This is confirmed in Experiment 4, in which LTR participants were asked to follow RTL and LTR moving dots in prior image presentation and showed no change in the normal spatial bias. In conclusion, the horizontal bias is a dynamic property and is modulated by habitual reading direction.

Irrelevant tactile stimulation biases visual exploration in external coordinates

We evaluated the effect of irrelevant tactile stimulation on humans’ free-viewing behavior during the exploration of complex static scenes. Specifically, we address the questions of (1) whether task-irrelevant tactile stimulation presented to subjects’ hands can guide visual selection during free viewing; (2) whether tactile stimulation can modulate visual exploratory biases that are independent of image content and task goals; and (3) in which reference frame these effects occur. Tactile stimulation to uncrossed and crossed hands during the viewing of static images resulted in long-lasting modulation of visual orienting responses. Subjects showed a well-known leftward bias during the early exploration of images, and this bias was modulated by tactile stimulation presented at image onset. Tactile stimulation, both at image onset and later during the trials, biased visual orienting toward the space ipsilateral to the stimulated hand, both in uncrossed and crossed hand postures. The long-lasting temporal and global spatial profile of the modulation of free viewing exploration by touch indicates that cross-modal cues produce orienting responses, which are coded exclusively in an external reference frame.

An extensive dataset of eye movements during viewing of complex images.

We present a dataset of free-viewing eye-movement recordings that contains more than 2.7 million fixation locations from 949 observers on more than 1000 images from different categories. This dataset aggregates and harmonizes data from 23 different studies conducted at the Institute of Cognitive Science at Osnabrück University and the University Medical Center in Hamburg-Eppendorf. Trained personnel recorded all studies under standard conditions with homogeneous equipment and parameter settings. All studies allowed for free eye-movements, and differed in the age range of participants (~7–80 years), stimulus sizes, stimulus modifications (phase scrambled, spatial filtering, mirrored), and stimuli categories (natural and urban scenes, web sites, fractal, pink-noise, and ambiguous artistic figures). The size and variability of viewing behavior within this dataset presents a strong opportunity for evaluating and comparing computational models of overt attention, and furthermore, for thoroughly quantifying strategies of viewing behavior. This also makes the dataset a good starting point for investigating whether viewing strategies change in patient groups.

STN-DBS Reduces Saccadic Hypometria but Not Visuospatial Bias in Parkinson's Disease Patients.

In contrast to its well-established role in alleviating skeleto-motor symptoms in Parkinsons disease, little is known about the impact of deep brain stimulation (DBS) of the subthalamic nucleus (STN) on oculomotor control and attention. Eye-tracking data of 17 patients with left-hemibody symptom onset was compared with 17 age-matched control subjects. Free-viewing of natural images was assessed without stimulation as baseline and during bilateral DBS. To examine the involvement of ventral STN territories in oculomotion and spatial attention, we employed unilateral stimulation via the left and right ventralmost contacts respectively. When DBS was off, patients showed shorter saccades and a rightward viewing bias compared with controls. Bilateral stimulation in therapeutic settings improved saccadic hypometria but not the visuospatial bias. At a group level, unilateral ventral stimulation yielded no consistent effects. However, the evaluation of electrode position within normalized MNI coordinate space revealed that the extent of early exploration bias correlated with the precise stimulation site within the left subthalamic area. These results suggest that oculomotor impairments “but not higher-level exploration patterns” are effectively ameliorable by DBS in therapeutic settings. Our findings highlight the relevance of the STN topography in selecting contacts for chronic stimulation especially upon appearance of visuospatial attention deficits.

Combining EEG and eye tracking: Identification, characterization and correction of eye movement artifacts in electroencephalographic data

Sensory processing under natural conditions has recently moved into the focus of neuroscience studies. This step necessitates solving a set of complex problems before the approach can be used to its full potential. One important aspect is the development of mobile recording techniques and their combination. On one hand is EEG a prime candidate for mobile physiological recordings while behavioural measurements include tracking of eye movements. These, however, introduce large artifacts to EEG recordings and thus render data analysis difficult or even impossible. To overcome this restriction, several correction methods including regression and blind source separation have been proposed. Yet, there is no automated standard procedure established, usable for evoked as well as induced activity in all frequency bands. Here we present a procedure based on Independent Component Analysis (ICA) in combination with high temporal resolution eye tracking. This procedure uses eye tracker information to reliably and objectively identify eye-artifact related ICA-components in an automated manner. We demonstrate that this approach allows removing or substantially reducing ocular artifacts including microsaccades without affecting the signal originating from brain sources. In conclusion the proposed method does not only provide a tool for detecting and correcting eye artifacts in standard EEG-paradigms but it also permits to study EEG-activity during eye tracking experiments and thus to investigate neural mechanisms of eye movement control and visual attention under natural conditions. Biomed Tech 2012; 57 (Suppl. 1)