Michi Matsukura

Attention effects during visual short-term memory maintenance: Protection or prioritization?

Interactions between visual attention and visual short-term memory (VSTM) play a central role in cognitive processing. For example, attention can assist in selectively encoding items into visual memory. Attention appears to be able to influence items already stored in visual memory, as well; cues that appear long after the presentation of an array of objects can affect memory for those objects (Griffin & Nobre, 2003). In five experiments, we distinguished two possible mechanisms for the effects of cues on items currently stored in VSTM. Aprotection account proposes that attention protects the cued item from becoming degraded during the retention interval. By contrast, aprioritization account suggests that attention increases a cued item’s priority during the comparison process that occurs when memory is tested. The results of the experiments were consistent with the first of these possibilities, suggesting that attention can serve to protect VSTM representations while they are being maintained.

Top-down control over biased competition during covert spatial orienting.

Larger benefits of spatial attention are observed when distractor interference is prevalent, supporting the view that spatial selection facilitates visual processing by suppressing distractor interference. The present work shows that cuing effects with identical visual displays can grow substantially as the probability of distractor interference increases. The probability of interference had no impact on spatial cuing effects in the absence of distractors, suggesting that the enlarged cuing effects were not caused by changes in signal enhancement or in the spatial distribution of attention. These findings suggest that attentional control settings determine more than where spatial attention is directed; top-down settings also influence how attention affects visual processing, with increased levels of distractor exclusion when distractor interference is likely. Spatial attention allows an observer to select specific locations within the visual environment, enabling better processing of attended than unattended stimuli. Current models acknowledge two distinct paths by which locations are selected. In the case of top-down selection, the goals and intentions of the observer determine the selected locations. In the case of stimulus-driven selection, the attended locations are determined by some aspect of the visual display. For example, the abrupt onset of new objects in the visual field can cause a stimulus-driven orienting of attention toward the location of the onset (Enns, Austen, Di Lollo, Rauschenberger, & Yantis, 2001; Jonides, 1981; Yantis & Jonides, 1984), even when this attentional shift is counter to the attentive goals of the observer (Remington, Johnston, & Yantis, 1992). However, although the distinction between stimulus-driven and top-down orienting is well established, there are interesting interactions between these processes. Yantis and Jonides (1990) showed that when attention is highly focused at a cued location, abrupt onsets do not necessarily disrupt this attentional focus. Folk, Remington, and Johnston (1992) demonstrated that the observer’s attentional control settings can determine which visual features (e.g., abrupt onsets or color singletons) will elicit stimulus-driven orienting in a given context. When observers anticipate a target that is defined by color, color singletons cause stimulus-driven capture, whereas abrupt onsets do not. However, when observers expect a target that is defined by its status as an abrupt-onset object, then abrupt onsets capture attention and color singletons have little effect. Thus, changes in top-down settings can lead to dramatic differences in the profile of stimulus-driven effects.

Visual Working Memory Modulates Rapid Eye Movements to Simple Onset Targets

Representations in visual working memory (VWM) influence attention and gaze control in complex tasks, such as visual search, that require top-down selection to resolve stimulus competition. VWM and visual attention clearly interact, but the mechanism of that interaction is not well understood. In the research reported here, we demonstrated that in the absence of stimulus competition or goal-level biases, VWM representations of object features influence the spatiotemporal dynamics of extremely simple eye movements. The influence of VWM therefore extends into the most basic operations of the oculomotor system.

Evidence for Two Components of Object-Based Selection

A wealth of research has shown that observers can bias visual processing toward specific locations, but the role of object-based selection is less clear. In support of object-based selection, previous research has shown that when two objects are presented simultaneously, observers are better at reporting two attributes from one of the objects than one attribute from each object. However, there has been controversy over whether this effect is best explained by object-based selection or spatial selection. Our work suggests that there are two separate components of selection in this task: (a) a spatial component that is observed when the relevant targets are cued for observers before the onset of the stimulus display and (b) an object-based component that can still be observed when the first component has been eliminated. The latter effect replicates the initial evidence in favor of object-based selection, and can be demonstrated even when the relevant targets are cued after the offset of the target stimuli.

Does visual short-term memory have a high-capacity stage?

Visual short-term memory (VSTM) has long been considered a durable, limited-capacity system for the brief retention of visual information. However, a recent work by Sligte et al. (Plos One 3:e1699, 2008) reported that, relatively early after the removal of a memory array, a cue allowed participants to access a fragile, high-capacity stage of VSTM that is distinct from iconic memory. In the present study, we examined whether this stage division is warranted by attempting to corroborate the existence of an early, high-capacity form of VSTM. The results of four experiments did not support Sligte et al.’s claim, since we did not obtain evidence for VSTM retention that exceeded traditional estimates of capacity. However, performance approaching that observed in Sligte et al. can be achieved through extensive practice, providing a clear explanation for their findings. Our evidence favors the standard view of VSTM as a limited-capacity system that maintains a few object representations in a relatively durable form.

The return of object-based attention: Selection of multiple-region objects

Objects can control the focus of attention, allowing features on the same object to be selected more easily than features on different objects. In the present experiments, we investigated the perceptual processes that contribute to such object-based attentional effects. Previous research has demonstrated that object-based effects occur for single-region objects but not for multiple-region objects under some conditions (Experiment 1, Watson & Kramer, 1999). Such results are surprising, because most objects in natural scenes are composed of multiple regions. Previous findings could therefore limit the usefulness of an object-based selection mechanism. We explored the generality of these single-region selection results by manipulating the extent to which different (i.e., multiple) regions of a single object perceptually grouped together. Object-based attentional effects were attenuated when multiple regions did not group into a single perceptual object (Experiment 1). However, when multiple regions grouped together based on (1) edge continuation (Experiments 2 and 3) or (2) part and occlusion cues (Experiment 4), we observed object-based effects. Our results suggest that object-based attention is a robust process that can select multiple-region objects, provided the regions of such objects cohere on the basis of perceptual grouping cues. nt ]mis|This research was made possible by a Sigma Xi Grants-in-Aid of Research Award to M.M. and a grant from the National Science Foundation (BCS 99-10727) to S.P.V.

Overt attentional prioritization of new objects and feature changes during real-world scene viewing

The authors investigated the extent to which a change to an objects colour is overtly prioritized for fixation relative to the appearance of a new object during real-world scene viewing. Both types of scene change captured gaze (and attention) when introduced during a fixation, although colour changes captured attention less often than new objects. Neither of these scene changes captured attention when they occurred during a saccade, but slower and less reliable memory-based mechanisms were nevertheless able to prioritize new objects and colour changes relative to the other stable objects in the scene. These results indicate that online memory for object identity and at least some object features are functional in detecting changes to real-world scenes. Additionally, visual factors such as the salience of onsets and colour changes did not affect prioritization of these events. We discuss these results in terms of current theories of attention allocation within, and online memory representations of, real-world scenes.

Visual working memory modulates low-level saccade target selection: Evidence from rapidly generated saccades in the global effect paradigm

In three experiments, we examined the influence of visual working memory (VWM) on the metrics of saccade landing position in a global effect paradigm. Participants executed a saccade to the more eccentric object in an object pair appearing on the horizontal midline, to the left or right of central fixation. While completing the saccade task, participants maintained a color in VWM for an unrelated memory task. Either the color of the saccade target matched the memory color (target match), the color of the distractor matched the memory color (distractor match), or the colors of neither object matched the memory color (no match). In the no-match condition, saccades tended to land at the midpoint between the two objects: the global, or averaging, effect. However, when one of the two objects matched VWM, the distribution of landing position shifted toward the matching object, both for target match and for distractor match. VWM modulation of landing position was observed even for the fastest quartile of saccades, with a mean latency as low as 112 ms. Effects of VWM on such rapidly generated saccades, with latencies in the express-saccade range, indicate that VWM interacts with the initial sweep of visual sensory processing, modulating perceptual input to oculomotor systems and thereby biasing oculomotor selection. As a result, differences in memory match produce effects on landing position similar to the effects generated by differences in physical salience.

Interference between object-based attention and object-based memory.

Research has shown that there are at least two kinds of visual selective attention: location based and object based. In the present study, we sought to determine the locus of spatially invariant object-based selection using a dual-task paradigm. In four experiments, observers performed an attention task (object feature report or visual search) with a concurrent memory task (object memory or spatial memory). Object memory was interfered with more by a concurrent object-based attention task than by a concurrent location-based attention task. However, this interference pattern was reversed for spatial memory, with greater interference by a location-based attention task than by an object-based attention task. These findings suggest that object-based attention and locationbased attention are functionally dissociable and that some forms of object-based selection operate within visual short-term memory.

Selection of multiple cued items is possible during visual short-term memory maintenance

Recent neuroimaging studies suggest that maintenance of a selected object feature held in visual short-term/working memory (VSTM/VWM) is supported by the same neural mechanisms that encode the sensory information. If VSTM operates by retaining “reasonable copies” of scenes constructed during sensory processing (Serences, Ester, Vogel, & Awh, 2009, p. 207, the sensory recruitment hypothesis), then attention should be able to select multiple items represented in VSTM as long as the number of these attended items does not exceed the typical VSTM capacity. It is well known that attention can select at least two noncontiguous locations at the same time during sensory processing. However, empirical reports from the studies that examined this possibility are inconsistent. In the present study, we demonstrate that (1) attention can indeed select more than a single item during VSTM maintenance when observers are asked to recognize a set of items in the manner that these items were originally attended, and (2) attention can select multiple cued items regardless of whether these items are perceptually organized into a single group (contiguous locations) or not (noncontiguous locations). The results also replicate and extend the recent finding that selective attention that operates during VSTM maintenance is sensitive to the observers’ goal and motivation to use the cueing information.