Wieske van Zoest

The role of stimulus-driven and goal-driven control in saccadic visual selection

Four experiments were conducted to investigate the role of stimulus-driven control in saccadic eye movements. Participants were required to make a speeded saccade toward a predefined target presented concurrently with multiple nontargets and possibly 1 distractor. Target and distractor were either equally salient (Experiments 1 and 2) or not (Experiments 3 and 4). The results uniformly demonstrated that fast eye movements were completely stimulus driven, whereas slower eye movements were goal driven. These results are in line with neither a bottom-up account nor a top-down notion of visual selection. Instead, they indicate that visual selection is the outcome of 2 independent processes, one stimulus driven and the other goal driven, operating in different time windows.

Effects of Salience Are Short-Lived

A salient event in the visual field tends to attract attention and the eyes. To account for the effects of salience on visual selection, models generally assume that the human visual system continuously holds information concerning the relative salience of objects in the visual field. Here we show that salience in fact drives vision only during the short time interval immediately following the onset of a visual scene. In a saccadic target-selection task, human performance in making an eye movement to the most salient element in a display was accurate when response latencies were short, but was at chance when response latencies were long. In a manual discrimination task, performance in making a judgment of salience was more accurate with brief than with long display durations. These results suggest that salience is represented in the visual system only briefly after a visual image enters the brain.

Reward creates oculomotor salience

Summary Theories of animal approach behaviour suggest that reward can create low-level biases in perceptual and motor systems, potentiating the processing of reward-associated environmental stimuli and causing animals to instinctively orient the head and eyes toward these objects [1]. However, the idea that reward can have this kind of direct impact on subsequent oculomotor processing has never been robustly tested, and existing research has largely confounded low-level effects with those mediated by strategy and attentional-set [2]. Here we demonstrate in humans that saccade trajectories are disrupted by a reward-associated distractor even when participants expect this object, know where it may appear, and do their best to ignore it. The reward history of a visual object thus has a direct, low-level, and non-strategic influence on how we deploy our eyes.

Saccadic target selection as a function of time

Recent evidence indicates that stimulus-driven and goal-directed control of visual selection operate independently and in different time windows (van Zoest et al., 2004). The present study further investigates how eye movements are affected by stimulus-driven and goal-directed control. Observers were presented with search displays consisting of one target, multiple non-targets and one distractor element. The task of observers was to make a fast eye movement to a target immediately following the offset of a central fixation point, an event that either co-occurred with or soon followed the presentation of the search display. Distractor saliency and target-distractor similarity were independently manipulated. The results demonstrated that the effect of distractor saliency was transient and only present for the fastest eye movements, whereas the effect of target-distractor similarity was sustained and present in all but the fastest eye movements. The results support an independent timing account of visual selection.

The effects of salience on saccadic target selection.

Two experiments were conducted to investigate the effects of saliency on saccadic target selection as a function of time. Participants were required to make a speeded saccade towards a target defined by a unique orientation presented concurrently with multiple nontargets and one distractor. Target and distractor were equally salient within the orientation dimension but varied in saliency in the colour dimension. Within the colour dimension, the target presented could be more, equally, or less salient than the distractor. The results showed that saliency played a large role early during processing while no effects of saliency were found in later processing. Results are discussed in terms of models on visual selection.

Bottom-up and Top-down Control in Visual Search

Previous research suggests that the allocation of attention is largely controlled either in a stimulus-driven or in a goal-driven manner. To date, few studies have systematically manipulated variables affecting stimulus-driven and goal-driven selection independently in order to investigate how both manners of control interrelate and affect performance in visual search. In the present study observers were presented with search displays consisting of an array of line segments rotated at various orientations. The task of observers was to indicate the presence or absence of a vertical line segment (the target) presented amongst a series of nontargets and possibly one distractor. By varying the absolute differences in orientation between the target, nontargets, and distractors, relative target–distractor salience and target–distractor similarity were independently manipulated to investigate the contribution of stimulus-driven and goal-driven control. The major result was that relative target–distractor salience and target–distractor similarity affected search performance independently. Performance was better in cases where the irrelevant distractor was not a salient item in the search display and did not look similar to the target. The results are discussed in terms of models of attentional control.

The time course of exogenous and endogenous control of covert attention

Studies of eye-movements and manual response have established that rapid overt selection is largely exogenously driven toward salient stimuli, whereas slower selection is largely endogenously driven to relevant objects. We use the N2pc, an event-related potential index of covert attention, to demonstrate that this time course reflects an underlying pattern in the deployment of covert attention. We find that shifts of attention that occur soon after the onset of a visual search array are directed toward salient, task-irrelevant visual stimuli and are associated with slow responses to the target. In contrast, slower shifts are target-directed and are associated with fast responses. The time course of exogenous and endogenous control provides a framework in which some inconsistent results in the capture literature might be reconciled; capture may occur when attention is rapidly deployed.

Capture of the eyes by relevant and irrelevant onsets

During early visual processing the eyes can be captured by salient visual information in the environment. Whether a salient stimulus captures the eyes in a purely automatic, bottom-up fashion or whether capture is contingent on task demands is still under debate. In the first experiment, we manipulated the relevance of a salient onset distractor. The onset distractor could either be similar or dissimilar to the target. Error saccade latency distributions showed that early in time, oculomotor capture was driven purely bottom-up irrespective of distractor similarity. Later in time, top-down information became available resulting in contingent capture. In the second experiment, we manipulated the saliency information at the target location. A salient onset stimulus could be presented either at the target or at a non-target location. The latency distributions of error and correct saccades had a similar time-course as those observed in the first experiment. Initially, the distributions overlapped but later in time task-relevant information decelerated the oculomotor system. The present findings reveal the interaction between bottom-up and top-down processes in oculomotor behavior. We conclude that the task relevance of a salient event is not crucial for capture of the eyes to occur. Moreover, task-relevant information may integrate with saliency information to initiate saccades, but only later in time.

Goal-driven modulation as a function of time in saccadic target selection.

Four experiments were performed to investigate the contribution of goal-driven modulation in saccadic target selection as a function of time. Observers were required to make an eye movement to a prespecified target that was concurrently presented with multiple nontargets and possibly one distractor. Target and distractor were defined in different dimensions (orientation dimension and colour dimension in Experiment 1), or were both defined in the same dimension (i.e., both defined in the orientation dimension in Experiment 2, or both defined in the colour dimension in Experiments 3 and 4). The identities of target and distractor were switched over conditions. Speed–accuracy functions were computed to examine the full time course of selection in each condition. There were three major results. First, the ability to exert goal-driven control increased as a function of response latency. Second, this ability depended on the specific target–distractor combination, yet was not a function of whether target and distractor were defined within or across dimensions. Third, goal-driven control was available earlier when target and distractor were dissimilar than when they were similar. It was concluded that the influence of goal-driven control in visual selection is not all or none, but is of a continuous nature.

Reward-associated stimuli capture the eyes in spite of strategic attentional set

Theories of reinforcement learning have proposed that the association of reward to visual stimuli may cause these objects to become fundamentally salient and thus attention-drawing. A number of recent studies have investigated the oculomotor correlates of this reward-priming effect, but there is some ambiguity in this literature regarding the involvement of top-down attentional set. Existing paradigms tend to create a situation where participants are actively looking for a reward-associated stimulus before subsequently showing that this selective bias sustains when it no longer has strategic purpose. This perseveration of attentional set is potentially different in nature than the direct impact of reward proposed by theory. Here we investigate the effect of reward on saccadic selection in a paradigm where strategic attentional set is decoupled from the effect of reward. We find that during search for a uniquely oriented target, the receipt of reward following selection of a target characterized by an irrelevant unique color causes subsequent stimuli characterized by this color to be preferentially selected. Importantly, this occurs regardless of whether the color characterizes the target or distractor. Other analyses demonstrate that only features associated with correct selection of the target prime the target representation, and that the magnitude of this effect can be predicted by variability in saccadic indices of feedback processing. These results add to a growing literature demonstrating that reward guides visual selection, often in spite of our strategic efforts otherwise.