Olaf Dimigen

Coregistration of eye movements and EEG in natural reading analyses and review

Brain-electric correlates of reading have traditionally been studied with word-by-word presentation, a condition that eliminates important aspects of the normal reading process and precludes direct comparisons between neural activity and oculomotor behavior. In the present study, we investigated effects of word predictability on eye movements (EM) and fixation-related brain potentials (FRPs) during natural sentence reading. Electroencephalogram (EEG) and EM (via video-based eye tracking) were recorded simultaneously while subjects read heterogeneous German sentences, moving their eyes freely over the text. FRPs were time-locked to first-pass reading fixations and analyzed according to the cloze probability of the currently fixated word. We replicated robust effects of word predictability on EMs and the N400 component in FRPs. The data were then used to model the relation among fixation duration, gaze duration, and N400 amplitude, and to trace the time course of EEG effects relative to effects in EM behavior. In an extended Methodological Discussion section, we review 4 technical and data-analytical problems that need to be addressed when FRPs are recorded in free-viewing situations (such as reading, visual search, or scene perception) and propose solutions. Results suggest that EEG recordings during normal vision are feasible and useful to consolidate findings from EEG and eye-tracking studies.

Human Microsaccade-Related Visual Brain Responses

Microsaccades are very small, involuntary flicks in eye position that occur on average once or twice per second during attempted visual fixation. Microsaccades give rise to EMG eye muscle spikes that can distort the spectrum of the scalp EEG and mimic increases in gamma band power. Here we demonstrate that microsaccades are also accompanied by genuine and sizeable cortical activity, manifested in the EEG. In three experiments, high-resolution eye movements were corecorded with the EEG: during sustained fixation of checkerboard and face stimuli and in a standard visual oddball task that required the counting of target stimuli. Results show that microsaccades as small as 0.15° generate a field potential over occipital cortex and midcentral scalp sites 100–140 ms after movement onset, which resembles the visual lambda response evoked by larger voluntary saccades. This challenges the standard assumption of human brain imaging studies that saccade-related brain activity is precluded by fixation, even when fully complied with. Instead, additional cortical potentials from microsaccades were present in 86% of the oddball task trials and of similar amplitude as the visual response to stimulus onset. Furthermore, microsaccade probability varied systematically according to the proportion of target stimuli in the oddball task, causing modulations of late stimulus-locked event-related potential (ERP) components. Microsaccades present an unrecognized source of visual brain signal that is of interest for vision research and may have influenced the data of many ERP and neuroimaging studies.

Interaction of facial expressions and familiarity : ERP evidence

There is mounting evidence that under some conditions the processing of facial identity and facial emotional expressions may not be independent; however, the nature of this interaction remains to be established. By using event-related brain potentials (ERP) we attempted to localize these interactions within the information processing system. During an expression discrimination task (Experiment 1) categorization was faster for portraits of personally familiar vs. unfamiliar persons displaying happiness. The peak latency of the P300 (trend) and the onset of the stimulus-locked LRP were shorter for familiar than unfamiliar faces. This implies a late perceptual but pre-motoric locus of the facilitating effect of familiarity on expression categorization. In Experiment 2 participants performed familiarity decisions about portraits expressing different emotions. Results revealed an advantage of happiness over disgust specifically for familiar faces. The facilitation was localized in the response selection stage as suggested by a shorter onset of the LRP. Both experiments indicate that familiarity and facial expression may not be independent processes. However, depending on the kind of decision different processing stages may be facilitated for happy familiar faces.

Eye movements and brain electric potentials during reading

The development of theories and computational models of reading requires an understanding of processing constraints, in particular of timelines related to word recognition and oculomotor control. Timelines of word recognition are usually determined with event-related potentials (ERPs) recorded under conditions of serial visual presentation (SVP) of words; timelines of oculomotor control are derived from parameters of eye movements (EMs) during natural reading. We describe two strategies to integrate these approaches. One is to collect ERPs and EMs in separate SVP and natural reading experiments for the same experimental material (but different subjects). The other strategy is to co-register EMs and ERPs during natural reading from the same subjects. Both strategies yield data that allow us to determine how lexical properties influence ERPs (e.g., the N400 component) and EMs (e.g., fixation durations) across neighboring words. We review our recent research on the effects of frequency and predictability of words on both EM and ERP measures with reference to current models of eye-movement control during reading. Results are in support of the proposition that lexical access is distributed across several fixations and across brain-electric potentials measured on neighboring words.

Emotions in cognitive conflicts are not aversive but are task specific

It has been suggested that cognitive conflicts require effortful processing and, therefore, are aversive (Botvinick, 2007). In the present study, we compared conflicts emerging from the inhibition of a predominant response tendency in a go/no-go task with those between incompatible response activations in a Simon task in a within-subjects design, using the same type of stimuli. Whereas no-go trials elicited reduced skin conductance and pupillometric responses, but prolonged corrugator muscle activity, as compared with go trials, incompatible and compatible Simon trials were indistinguishable with respect to these parameters. Furthermore, the conflictsensitive N2 components of the event-related brain potential were similar in amplitude, but showed significantly different scalp distributions, indicating dissociable neural generator systems. The present findings suggest the involvement of different emotional and cognitive processes in both types of cognitive conflicts—none being aversive, however. In addition, the N2 findings call into question claims of common monitoring systems for all kinds of cognitive conflicts.

Stimulus onset asynchrony and the timeline of word recognition : event-related potentials during sentence reading

Three ERP experiments examined the effect of word presentation rate (i.e., stimulus onset asynchrony, SOA) on the time course of word frequency and predictability effects in sentence reading. In Experiments 1 and 2, sentences were presented word-by-word in the screen center at an SOA of 700 and 490 ms, respectively. While these rates are typical for psycholinguistic ERP research, natural reading happens at a considerably faster pace. Accordingly, Experiment 3 employed a near-normal SOA of 280 ms, which approximated the rate of normal reading. Main results can be summarized as follows: (1) The onset latency of early frequency effects decreases gradually with increasing presentation rates. (2) An early interaction between top-down and bottom-up processing is observed only under a near-normal SOA. (3) N400 predictability effects occur later and are smaller at a near-normal (i.e., high) presentation rate than at the lower rates commonly used in ERP experiments. (4) ERP morphology is different at the shortest compared to longer SOAs. Together, the results point to a special role of a near-normal presentation rate for visual word recognition and therefore suggest that SOA should be taken into account in research of natural reading.

Functional network analysis reveals differences in the semantic priming task.

The recent years have seen the emergence of graph theoretical analysis of complex, functional brain networks estimated from neurophysiological measurements. The research has mainly focused on the graph characterization of the resting-state/default network, and its potential for clinical application. Functional resting-state networks usually display the characteristics of small-world networks and their statistical properties have been observed to change due to pathological conditions or aging. In the present paper we move forward in the application of graph theoretical tools in functional connectivity by investigating high-level cognitive processing in healthy adults, in a manner similar to that used in psychological research in the framework of event-related potentials (ERPs). More specifically we aim at investigating how graph theoretical approaches can help to discover systematic and task-dependent differences in high-level cognitive processes such as language perception. We will show that such an approach is feasible and that the results coincide well with the findings from neuroimaging studies.

Microsaccadic inhibition and P300 enhancement in a visual oddball task.

It has recently been demonstrated that the presentation of visual oddballs induces a prolonged inhibition of microsaccades. The amplitude of the P300 component in event-related potentials (ERPs) has been shown to be sensitive to the category (target vs. nontarget) of the eliciting stimulus, its overall probability, and the preceding stimulus sequence. In the present study we further specify the functional underpinnings of the prolonged microsaccadic inhibition in the visual oddball task, showing that the stimulus category, the frequency of a stimulus, and the preceding stimulus sequence influence microsaccade rate. Furthermore, by co-recording ERPs and eye movements, we were able to demonstrate that, despite being largely sensitive to the same experimental manipulation, the amplitude of P300 and the microsaccadic inhibition predict each other only weakly.

Parafoveal processing in reading Chinese sentences: Evidence from event‐related brain potentials

Natural reading involves the preprocessing of upcoming words, resulting in shorter fixations on words visible in the parafovea during preceding fixations. While this preview benefit is established in behavior, its brain-electric correlates have only recently been investigated. Using fixation-related potentials, an attenuation of the occipitotemporal N1 component for words that were parafoveally visible during preceding fixations has been demonstrated. In contrast, another study, using an RSVP paradigm with parafoveal flanker words, observed no such general preview benefit in ERPs, but instead reported N400 effects triggered by semantically incongruous parafoveal words. To follow up on these discrepant findings and to extend them to a nonalphabetic writing system, we conducted two ERP experiments with Chinese readers using the RSVP-with-flankers paradigm and rigorous fixation control via eye tracking. We replicate robust parafoveal N400 semantic congruency effects in Chinese participants. Additionally, we found that, once a word was directly looked at, words after a valid preview elicited a smaller N1 and a weaker N400 than those after an invalid preview. Results underline the importance of considering parafoveal vision in ERP studies on reading.

Microsaccade-related brain potentials signal the focus of visuospatial attention

Covert shifts of visuospatial attention are traditionally assumed to occur in the absence of oculomotor behavior. In contrast, recent behavioral studies have linked attentional cueing effects to the occurrence of microsaccades, small eye movements executed involuntarily during attempted fixation. Here we used a new type of electrophysiological marker to explore the attention-microsaccade relationship, the visual brain activity evoked by the microsaccade itself. By shifting the retinal image, microsaccades frequently elicit neural responses throughout the visual pathway, scalp-recordable in the human EEG as a microsaccade-related potential (mSRP). Although mSRPs contain similar signal components (P1/N1) as traditional visually-evoked potentials (VEPs), it is unknown whether they are also influenced by cognition. Based on established findings that VEPs are amplified for visual inputs at currently attended locations, we expected a selective gain-modulation also for mSRPs. Eye movements and EEG were coregistered in a classic spatial cueing task with an endogenous cue. Replicating behavioral findings, the direction of early microsaccades 200-400ms after cue onset was biased towards the cued side. However, for microsaccades throughout the cue-target interval, mSRPs were systematically enhanced at occipital scalp sites contralateral to the cued hemifield. This attention effect resembled that in a control condition with VEPs and did not interact with the direction of the underlying microsaccade, suggesting that mSRPs reflect the focus of sustained visuospatial attention, which remains fixed at the cued location, despite microsaccades. Microsaccades are not merely an artifact source in the EEG; instead, they are followed by cognitively modulated brain potentials that can serve as non-intrusive electrophysiological probes of attention.