Trafton Drew

Neural Measures of Individual Differences in Selecting and Tracking Multiple Moving Objects

Attention can be divided so that multiple objects can be tracked simultaneously as they move among distractors. Although attentional tracking is known to be highly limited, such that most individuals can track only approximately four objects simultaneously, the neurophysiological mechanisms that underlie this capacity limitation have not been established. Here, we provide electrophysiological measures in humans of the initial selection and sustained attention processes that facilitate attentional tracking. Each measure was modulated by the number of objects the subject was tracking and was highly sensitive to each individuals specific tracking capacity. Consequently, these measures provide strong neurophysiological predictors of an individuals attentional tracking capacity. Moreover, by manipulating the difficulty of these two phases of the task, we observe that the limiting factor underlying tracking capacity can flexibly shift between these two attentional mechanisms depending on the requirements of the task.

The Invisible Gorilla Strikes Again: Sustained Inattentional Blindness in Expert Observers

Researchers have shown that people often miss the occurrence of an unexpected yet salient event if they are engaged in a different task, a phenomenon known as inattentional blindness. However, demonstrations of inattentional blindness have typically involved naive observers engaged in an unfamiliar task. What about expert searchers who have spent years honing their ability to detect small abnormalities in specific types of images? We asked 24 radiologists to perform a familiar lung-nodule detection task. A gorilla, 48 times the size of the average nodule, was inserted in the last case that was presented. Eighty-three percent of the radiologists did not see the gorilla. Eye tracking revealed that the majority of those who missed the gorilla looked directly at its location. Thus, even expert searchers, operating in their domain of expertise, are vulnerable to inattentional blindness.

Attentional enhancement during multiple-object tracking

What is the role of attention in multiple-object tracking? Does attention enhance target representations, suppress distractor representations, or both? It is difficult to ask this question in a purely behavioral paradigm without altering the very attentional allocation one is trying to measure. In the present study, we used event-related potentials to examine the early visual evoked responses to task-irrelevant probes without requiring an additional detection task. Subjects tracked two targets among four moving distractors and four stationary distractors. Brief probes were flashed on targets, moving distractors, stationary distractors, or empty space. We obtained a significant enhancement of the visually evoked P1 and N1 components (∼100–150 msec) for probes on targets, relative to distractors. Furthermore, good trackers showed larger differences between target and distractor probes than did poor trackers. These results provide evidence of early attentional enhancement of tracked target items and also provide a novel approach to measuring attentional allocation during tracking.

Delineating the Neural Signatures of Tracking Spatial Position and Working Memory during Attentive Tracking

In the attentive tracking task, observers track multiple objects as they move independently and unpredictably among visually identical distractors. Although a number of models of attentive tracking implicate visual working memory as the mechanism responsible for representing target locations, no study has ever directly compared the neural mechanisms of the two tasks. In the current set of experiments, we used electrophysiological recordings to delineate similarities and differences between the neural processing involved in working memory and attentive tracking. We found that the contralateral electrophysiological response to the two tasks was similarly sensitive to the number of items attended in both tasks but that there was also a unique contralateral negativity related to the process of monitoring target position during tracking. This signal was absent for periods of time during tracking tasks when objects briefly stopped moving. These results provide evidence that, during attentive tracking, the process of tracking target locations elicits an electrophysiological response that is distinct and dissociable from neural measures of the number of items being attended.

Event-related potential measures of visual working memory.

Visual working memory is a limited capacity system that temporarily maintains information about objects in the immediate visual environment. Psychophysical experiments have shown that most people are able to actively maintain 3 or 4 items in visual working memory at any point in time To better understand how this process works and why our working memory capacity is so limited, a variety of neurophysiological approaches have been employed. In recent years, there has been a surge of interest in understanding how visual information is maintained in working memory at the neural level. Single-cell research with non-human primates has shown that neuronal firing during the retention period reflects the information that is currently held in working memory. In humans, event-related potentials (ERPs) have been used to examine the maintenance of information in working memory. An event-related potential component, known as the negative slow wave (NSW), has been used to measure the maintenance of information in working memory “online” during a given trial. More recently, another ERP component, the contralateral delay activity (CDA) has been shown to be a fairly specific correlate of the current contents of working memory. This component is sensitive to an individuals working memory capacity and may provide a window into the operations of this central cognitive construct.

Swapping or dropping? Electrophysiological measures of difficulty during multiple object tracking

In the multiple object tracking task, participants are asked to keep targets separate from identical distractors as all items move randomly. It is well known that simple manipulations such as object speed and number of distractors dramatically alter the number of targets that are successfully tracked, but very little is known about what causes this variation in performance. One possibility is that participants tend to lose track of objects (dropping) more frequently under these conditions. Another is that the tendency to confuse a target with a distractor increases (swapping). These two mechanisms have very different implications for the attentional architecture underlying tracking. However, behavioral data alone cannot differentiate between these possibilities. In the current study, we used an electrophysiological marker of the number of items being actively tracked to assess which type of errors tended to occur during speed and distractor load manipulations. Our neural measures suggest that increased distractor load led to an increased likelihood of confusing targets with distractors while increased speed led to an increased chance of a target item being dropped. Behavioral experiments designed to test this novel prediction support this assertion.

A bilateral advantage for storage in visual working memory

Various studies have demonstrated enhanced visual processing when information is presented across both visual hemifields rather than in a single hemifield (the bilateral advantage). For example, Alvarez and Cavanagh (2005) reported that observers were able to track twice as many moving visual stimuli when the tracked items were presented bilaterally rather than unilaterally, suggesting that independent resources enable tracking in the two visual fields. Motivated by similarities in the apparent capacity and neural substrates that mediate tracking and visual working memory (WM), the present work examined whether or not a bilateral advantage also arises during storage in visual WM. Using a recall procedure to assess working memory for orientation information, we found a reliable bilateral advantage; recall error was smaller with bilateral sample displays than with unilateral displays. To demonstrate that the bilateral advantage influenced storage per se rather than just encoding efficiency, we replicated the observed bilateral advantage using sequentially presented stimuli. Finally, to further characterize how bilateral presentations enhanced storage in working memory, we measured both the number and the resolution of the stored items and found that bilateral presentations lead to an increased probability of storage, rather than enhanced mnemonic resolution. Thus, the bilateral advantage extends beyond the initial selection and encoding of visual information to influence online maintenance in visual working memory.

Neural measures of dynamic changes in attentive tracking load

In everyday life, we often need to track several objects simultaneously, a task modeled in the laboratory using the multiple-object tracking (MOT) task [Pylyshyn, Z., & Storm, R. W. Tracking multiple independent targets: Evidence for a parallel tracking mechanism. Spatial Vision, 3, 179–197, 1988]. Unlike MOT, however, in life, the set of relevant targets tends to be fluid and change over time. Humans are quite adept at “juggling” targets in and out of the target set [Wolfe, J. M., Place, S. S., & Horowitz, T. S. Multiple object juggling: Changing what is tracked during extended MOT. Psychonomic Bulletin & Review, 14, 344–349, 2007]. Here, we measured the neural underpinnings of this process using electrophysiological methods. Vogel and colleagues [McCollough, A. W., Machizawa, M. G., & Vogel, E. K. Electrophysiological measures of maintaining representations in visual working memory. Cortex, 43, 77–94, 2007; Vogel, E. K., McCollough, A. W., & Machizawa, M. G. Neural measures reveal individual differences in controlling access to working memory. Nature, 438, 500–503, 2005; Vogel, E. K., & Machizawa, M. G. Neural activity predicts individual differences in visual working memory capacity. Nature, 428, 748–751, 2004] have shown that the amplitude of a sustained lateralized negativity, contralateral delay activity (CDA) indexes the number of items held in visual working memory. Drew and Vogel [Drew, T., & Vogel, E. K. Neural measures of individual differences in selecting and tracking multiple moving objects. Journal of Neuroscience, 28, 4183–4191, 2008] showed that the CDA also indexes the number of items being tracking a standard MOT task. In the current study, we set out to determine whether the CDA is a signal that merely represents the number of objects that are attended during a trial or a dynamic signal capable of reflecting on-line changes in tracking load during a single trial. By measuring the response to add or drop cues, we were able to observe dynamic changes in CDA amplitude. The CDA appears to rapidly represent the current number of objects being tracked. In addition, we were able to generate some initial estimates of the time course of this dynamic process.

Remapping attention in multiple object tracking

Which coordinate system do we use to track moving objects? In a previous study using smooth pursuit eye movements, we argued that targets are tracked in both retinal (retinotopic) and scene-centered (allocentric) coordinates (Howe, Pinto, & Horowitz, 2010). However, multiple object tracking typically also elicits saccadic eye movements, which may change how object locations are represented. Observers fixated a cross while tracking three targets out of six identical disks confined to move within an imaginary square. The fixation cross alternated between two locations, requiring observers to make repeated saccades. By moving (or not moving) the imaginary square in sync with the fixation cross, we could disrupt either (or both) coordinate systems. Surprisingly, tracking performance was much worse when the objects moved with the fixation cross, although this manipulation preserved the retinal image across saccades, thereby avoiding the visual disruptions normally associated with saccades. Instead, tracking performance was best when the allocentric coordinate system was preserved, suggesting that targets locations are maintained in that coordinate system across saccades. This is consistent with a theoretical framework in which the positions of a small set of attentional pointers are predictively updated in advance of a saccade.

Hybrid Search in the Temporal Domain: Evidence for Rapid, Serial Logarithmic Search through Memory

In hybrid search, observers memorize a number of possible targets and then search for any of these in visual arrays of items. Wolfe (2012) has previously shown that the response times in hybrid search increase with the log of the memory set size. What enables this logarithmic search of memory? One possibility is a series of steps in which subsets of the memory set are compared to all items in the visual set simultaneously. In the present experiments, we presented single visual items sequentially in a rapid serial visual presentation (RSVP) display, eliminating the possibility of simultaneous testing of all items. We used a staircasing procedure to estimate the time necessary to effectively detect the target in the RSVP stream. Processing time increased in a log–linear fashion with the number of potential targets. This finding eliminates the class of models that require simultaneous comparison of some memory items to all (or many) items in the visual display. Experiment 3 showed that, similar to visual search, memory search efficiency in this paradigm is influenced by the similarity between the target set and the distractors. These results indicate that observers perform separate memory searches on each eligible item in the visual display. Moreover, it appears that memory search for one item can proceed while other items are being categorized as “eligible” or “not eligible.”