Sebastiaan Mathôt

OpenSesame: An open-source, graphical experiment builder for the social sciences

In the present article, we introduce OpenSesame, a graphical experiment builder for the social sciences. OpenSesame is free, open-source, and cross-platform. It features a comprehensive and intuitive graphical user interface and supports Python scripting for complex tasks. Additional functionality, such as support for eyetrackers, input devices, and video playback, is available through plug-ins. OpenSesame can be used in combination with existing software for creating experiments.

ScanMatch: A novel method for comparing fixation sequences

We present a novel approach to comparing saccadic eye movement sequences based on the Needleman-Wunsch algorithm used in bioinformatics to compare DNA sequences. In the proposed method, the saccade sequence is spatially and temporally binned and then recoded to create a sequence of letters that retains fixation location, time, and order information. The comparison of two letter sequences is made by maximizing the similarity score computed from a substitution matrix that provides the score for all letter pair substitutions and a penalty gap. The substitution matrix provides a meaningful link between each location coded by the individual letters. This link could be distance but could also encode any useful dimension, including perceptual or semantic space. We show, by using synthetic and behavioral data, the benefits of this method over existing methods. The ScanMatch toolbox for MATLAB is freely available online (www.scanmatch.co.uk).

PyGaze : An open-source, cross-platform toolbox for minimal-effort programming of eyetracking experiments

The PyGaze toolbox is an open-source software package for Python, a high-level programming language. It is designed for creating eyetracking experiments in Python syntax with the least possible effort, and it offers programming ease and script readability without constraining functionality and flexibility. PyGaze can be used for visual and auditory stimulus presentation; for response collection via keyboard, mouse, joystick, and other external hardware; and for the online detection of eye movements using a custom algorithm. A wide range of eyetrackers of different brands (EyeLink, SMI, and Tobii systems) are supported. The novelty of PyGaze lies in providing an easy-to-use layer on top of the many different software libraries that are required for implementing eyetracking experiments. Essentially, PyGaze is a software bridge for eyetracking research.

Evidence for the predictive remapping of visual attention

When attending an object in visual space, perception of the object remains stable despite frequent eye movements. It is assumed that visual stability is due to the process of remapping, in which retinotopically organized maps are updated to compensate for the retinal shifts caused by eye movements. Remapping is predictive when it starts before the actual eye movement. Until now, most evidence for predictive remapping has been obtained in single cell studies involving monkeys. Here, we report that predictive remapping affects visual attention prior to an eye movement. Immediately following a saccade, we show that attention has partly shifted with the saccade (Experiment 1). Importantly, we show that remapping is predictive and affects the locus of attention prior to saccade execution (Experiments 2 and 3): before the saccade was executed, there was attentional facilitation at the location which, after the saccade, would retinotopically match the attended location.

Gradual Remapping Results in Early Retinotopic and Late Spatiotopic Inhibition of Return

Here we report that immediately following the execution of an eye movement, oculomotor inhibition of return resides in retinotopic (eye-centered) coordinates. At longer postsaccadic intervals, inhibition resides in spatiotopic (world-centered) coordinates. These results are explained in terms of perisaccadic remapping. In the interval surrounding an eye movement, information is remapped within retinotopic maps to compensate for the retinal displacement. Because remapping is not an instantaneous process, a fast, but gradual, transfer of inhibition of return from retinotopic to spatiotopic coordinates can be observed in the postsaccadic interval. The observation that visual stability is preserved in inhibition of return is consistent with its function as a “foraging facilitator,” which requires locations to be inhibited across multiple eye movements. The current results support the idea that the visual system is retinotopically organized and that the appearance of a spatiotopic organization is due to remapping of visual information to compensate for eye movements.

Visual Attention and Stability

In the present review, we address the relationship between attention and visual stability. Even though with each eye, head and body movement the retinal image changes dramatically, we perceive the world as stable and are able to perform visually guided actions. However, visual stability is not as complete as introspection would lead us to believe. We attend to only a few items at a time and stability is maintained only for those items. There appear to be two distinct mechanisms underlying visual stability. The first is a passive mechanism: the visual system assumes the world to be stable, unless there is a clear discrepancy between the pre- and post-saccadic image of the region surrounding the saccade target. This is related to the pre-saccadic shift of attention, which allows for an accurate preview of the saccade target. The second is an active mechanism: information about attended objects is remapped within retinotopic maps to compensate for eye movements. The locus of attention itself, which is also characterized by localized retinotopic activity, is remapped as well. We conclude that visual attention is crucial in our perception of a stable world.

The Pupillary Light Response Reveals the Focus of Covert Visual Attention

The pupillary light response is often assumed to be a reflex that is not susceptible to cognitive influences. In line with recent converging evidence, we show that this reflexive view is incomplete, and that the pupillary light response is modulated by covert visual attention: Covertly attending to a bright area causes a pupillary constriction, relative to attending to a dark area under identical visual input. This attention-related modulation of the pupillary light response predicts cuing effects in behavior, and can be used as an index of how strongly participants attend to a particular location. Therefore, we suggest that pupil size may offer a new way to continuously track the focus of covert visual attention, without requiring a manual response from the participant. The theoretical implication of this finding is that the pupillary light response is neither fully reflexive, nor under complete voluntary control, but is instead best characterized as a stereotyped response to a voluntarily selected target. In this sense, the pupillary light response is similar to saccadic and smooth pursuit eye movements. Together, eye movements and the pupillary light response maximize visual acuity, stabilize visual input, and selectively filter visual information as it enters the eye.

New Light on the Mind’s Eye: The Pupillary Light Response as Active Vision

The eye’s pupils constrict (shrink) in brightness and dilate (expand) in darkness. The pupillary light response was historically considered a low-level reflex without any cognitive component. Here, we review recent studies that have dramatically changed this view: The light response depends not only on a stimulus’s brightness but also on whether you are aware of the stimulus, whether you are paying attention to it, and even whether you are thinking about it. We highlight the link between the pupillary light response and eye-movement preparation: When you intend to look at a bright stimulus, a pupillary constriction is prepared along with the eye movement before the eyes set in motion. This preparation allows the pupil to rapidly change its size as your eyes move from bright to dark objects and back again. We discuss the implications of these recent advances for our understanding of the subtle yet important role that pupillary responses play in vision.

The pupillary light response reflects exogenous attention and inhibition of return

Here we show that the pupillary light response reflects exogenous (involuntary) shifts of attention and inhibition of return. Participants fixated in the center of a display that was divided into a bright and a dark half. An exogenous cue attracted attention to the bright or dark side of the display. Initially, the pupil constricted when the bright, as compared to the dark, side of the display was cued, reflecting a shift of attention toward the exogenous cue. Crucially, this pattern reversed about 1 s after cue presentation. This later-occurring, relative dilation (when the bright side was cued) reflected disengagement from the previously attended location, analogous to the behavioral phenomenon of inhibition of return. Indeed, we observed a reliable correlation between pupillary inhibition and behavioral inhibition of return. Our results support the view that inhibition of return results from habituation to (or short-term depression of) visual input. We conclude that the pupillary light response is a complex eye movement that reflects how we selectively parse and interpret visual input.

A reinvestigation of the reference frame of the tilt-adaptation aftereffect

The tilt-adaptation aftereffect (TAE) is the phenomenon that prolonged perception of a tilted ‘adapter’ stimulus affects the perceived tilt of a subsequent ‘tester’ stimulus. Although it is clear that TAE is strongest when adapter and tester are presented at the same location, the reference frame of the effect is debated. Some authors have reported that TAE is spatiotopic (world centred): It occurs when adapter and tester are presented at the same display location, even when this corresponds to different retinal locations. Others have reported that TAE is exclusively retinotopic (eye centred): It occurs only when adapter and tester are presented at the same retinal location, even when this corresponds to different display locations. Because this issue is crucial for models of transsaccadic perception, we reinvestigated the reference frame of TAE. We report that TAE is exclusively retinotopic, supporting the notion that there is no transsaccadic integration of low-level visual information.