Antje Nuthmann

SWIFT: A dynamical model of saccade generation during reading

Mathematical models have become an important tool for understanding the control of eye movements during reading. Main goals of the development of the SWIFT model (R. Engbert, A. Longtin, & R. Kliegl, 2002) were to investigate the possibility of spatially distributed processing and to implement a general mechanism for all types of eye movements observed in reading experiments. The authors present an advanced version of SWIFT that integrates properties of the oculomotor system and effects of word recognition to explain many of the experimental phenomena faced in reading research. They propose new procedures for the estimation of model parameters and for the test of the models performance. They also present a mathematical analysis of the dynamics of the SWIFT model. Finally, within this framework, they present an analysis of the transition from parallel to serial processing.

Tracking the Mind during Reading: The Influence of Past, Present, and Future Words on Fixation Durations.

Reading requires the orchestration of visual, attentional, language-related, and oculomotor processing constraints. This study replicates previous effects of frequency, predictability, and length of fixated words on fixation durations in natural reading and demonstrates new effects of these variables related to 144 sentences. Such evidence for distributed processing of words across fixation durations challenges psycholinguistic immediacy-of-processing and eye-mind assumptions. Most of the time the mind processes several words in parallel at different perceptual and cognitive levels. Eye movements can help to unravel these processes.

Object-based attentional selection in scene viewing

Two contrasting views of visual attention in scenes are the visual salience and the cognitive relevance hypotheses. They fundamentally differ in their conceptualization of the visuospatial representation over which attention is directed. According to the saliency model, this representation is image-based, while the cognitive relevance framework advocates an object-based representation. Previous research has shown that (1) viewers prefer to look at objects over background and that (2) the saliency model predicts human fixation locations significantly better than chance. However, it could be that saliency mainly acts through objects. To test this hypothesis, we investigated where people fixate within real objects and saliency proto-objects. To this end, we recorded eye movements of human observers while they inspected photographs of natural scenes under different task instructions. We found a preferred viewing location (PVL) close to the center of objects within naturalistic scenes. Compared to the PVL for real objects, there was less evidence for a PVL for human fixations within saliency proto-objects. There was no evidence for a PVL when only saliency proto-objects that did not spatially overlap with annotated real objects were analyzed. The results suggest that saccade targeting and, by inference, attentional selection in scenes is object-based.

CRISP: A computational model of fixation durations in scene viewing.

Eye-movement control during scene viewing can be represented as a series of individual decisions about where and when to move the eyes. While substantial behavioral and computational research has been devoted to investigating the placement of fixations in scenes, relatively little is known about the mechanisms that control fixation durations. Here, we propose a computational model (CRISP) that accounts for saccade timing and programming and thus for variations in fixation durations in scene viewing. First, timing signals are modeled as continuous-time random walks. Second, difficulties at the level of visual and cognitive processing can inhibit and thus modulate saccade timing. Inhibition generates moment-by-moment changes in the random walks transition rate and processing-related saccade cancellation. Third, saccade programming is completed in 2 stages: an initial, labile stage that is subject to cancellation and a subsequent, nonlabile stage. Several simulation studies tested the models adequacy and generality. An initial simulation study explored the role of cognitive factors in scene viewing by examining how fixation durations differed under different viewing task instructions. Additional simulations investigated the degree to which fixation durations were under direct moment-to-moment control of the current visual scene. The present work further supports the conclusion that fixation durations, to a certain degree, reflect perceptual and cognitive activity in scene viewing. Computational model simulations contribute to an understanding of the underlying processes of gaze control.

Flexible saccade-target selection in Chinese reading

As Chinese is written without orthographical word boundaries (i.e., spaces), it is unclear whether saccade targets are selected on the basis of characters or words and whether saccades are aimed at the beginning or the centre of words. Here, we report an experiment where 30 Chinese readers read 150 sentences while their eye movements were monitored. They exhibited a strong tendency to fixate at the word centre in single-fixation cases and at the word beginning in multiple-fixation cases. Different from spaced alphabetic script, initial fixations falling at the end of words were no more likely to be followed by a refixation than initial fixations at word centre. Further, single fixations were shorter than first fixations in two-fixation cases, which is opposite to what is found in Roman script. We propose that Chinese readers dynamically select the beginning or centre of words as saccade targets depending on failure or success with segmentation of parafoveal word boundaries.

An examination of binocular reading fixations based on sentence corpus data

Binocular eye movements of normal adult readers were examined as they read single sentences. Analyses of horizontal and vertical fixation disparities indicated that the most prevalent type of disparate fixation is crossed (i.e., the left eye is located further to the right than the right eye) while the left eye frequently fixates somewhat above the right eye. The Gaussian distribution of the binocular fixation point peaked 2.6 cm in front of the plane of text, reflecting the prevalence of horizontally crossed fixations. Fixation disparity accumulates during the course of successive saccades and fixations within a line of text, but only to an extent that does not compromise single binocular vision. In reading, the version and vergence system interact in a way that is qualitatively similar to what has been observed in simple nonreading tasks. Finally, results presented here render it unlikely that vergence movements in reading aim at realigning the eyes at a given saccade target word.

Mindless reading revisited: An analysis based on the SWIFT model of eye-movement control

In this article, we revisit the mindless reading paradigm from the perspective of computational modeling. In the standard version of the paradigm, participants read sentences in both their normal version as well as the transformed (or mindless) version where each letter is replaced with a z. z-String scanning shares the oculomotor requirements with reading but none of the higher-level lexical and semantic processes. Here we use the z-string scanning task to validate the SWIFT model of saccade generation [Engbert, R., Nuthmann, A., Richter, E., & Kliegl, R. (2005). SWIFT: A dynamical model of saccade generation during reading. Psychological Review, 112(4), 777-813] as an example for an advanced theory of eye-movement control in reading. We test the central assumption of spatially distributed processing across an attentional gradient proposed by the SWIFT model. Key experimental results like prolonged average fixation durations in z-string scanning compared to normal reading and the existence of a string-length effect on fixation durations and probabilities were reproduced by the model, which lends support to the models assumptions on visual processing. Moreover, simulation results for patterns of regressive saccades in z-string scanning confirm SWIFTs concept of activation field dynamics for the selection of saccade targets.

The effect of word position on eye-movements in sentence and paragraph reading.

The present study explores the role of the word position-in-text in sentence and paragraph reading. Three eye-movement data sets based on the reading of Dutch and German unrelated sentences reveal a sizeable, replicable increase in reading times over several words at the beginning and the end of sentences. The data from the paragraph-based English-language Dundee corpus replicate the pattern and also indicate that the increase in inspection times is driven by the visual boundaries of the text organized in lines, rather than by syntactic sentence boundaries. We argue that this effect is independent of several established lexical, contextual, and oculomotor predictors of eye-movement behaviour. We also provide evidence that the effect of word position-in-text has two independent components: a start-up effect, arguably caused by a strategic oculomotor programme of saccade planning over the line of text, and a wrap-up effect, originating in cognitive processes of comprehension and semantic integration.

How do the regions of the visual field contribute to object search in real-world scenes? Evidence from eye movements

An important factor constraining visual search performance is the inhomogeneity of the visual system. Engaging participants in a scene search task, the present study explored how the different regions of the visual field contribute to search. Gaze-contingent Blindspots and Spotlights were implemented to determine the absolute and relative importance of the different visual regions for object-in-scene search. Three Blindspot/Spotlight radii (1.6°, 2.9°, and 4.1°) were used to differentiate between foveal, parafoveal, and peripheral vision. When searching the scene with artificially impaired foveal or central vision (Blindspots), search performance was surprisingly unimpaired. Foveal vision was not necessary to attain normal search performance. When high-resolution scene information was withheld in both foveal and parafoveal vision (4.1° Blindspot), target localization was unimpaired but it took longer to verify the identity of the target. Artificially impairing extrafoveal scene analysis (Spotlights) affected attentional selection and visual processing; shrinking the Spotlight of high resolution led to longer search times, shorter saccades, and more and longer fixations. The 4.1° radius was identified as the crossover point of equal search times in Blindspot and Spotlight conditions. However, a gaze-data based decomposition of search times into behaviorally defined epochs revealed differences in particular subprocesses of search.

Single-trial classification of EEG in a visual object task using ICA and machine learning

Presenting different visual object stimuli can elicit detectable changes in EEG recordings, but this is typically observed only after averaging together data from many trials and many participants. We report results from a simple visual object recognition experiment where independent component analysis (ICA) data processing and machine learning classification were able to correctly distinguish presence of visual stimuli at around 87% (0.70 AUC, p<0.0001) accuracy within single trials, using data from single ICs. Seven subjects observed a series of everyday visual object stimuli while EEG was recorded. The task was to indicate whether or not they recognised each object as familiar to them. EEG or IC data from a subset of initial object presentations was used to train support vector machine (SVM) classifiers, which then generated a label for subsequent data. Task-label classifier accuracy gives a proxy measure of task-related information present in the data used to train. This allows comparison of EEG data processing techniques - here, we found selected single ICs that give higher performance than when classifying from any single scalp EEG channel (0.70 AUC vs 0.65 AUC, p<0.0001). Most of these single selected ICs were found in occipital regions. Scoring a sliding analysis window moving through the time-points of the trial revealed that peak accuracy is when using data from +75 to +125 ms relative to the object appearing on screen. We discuss the use of such classification and potential cognitive implications of differential accuracy on IC activations.