Emma Wu Dowd

What Can 1 Billion Trials Tell Us About Visual Search

Mobile technology (e.g., smartphones and tablets) has provided psychologists with a wonderful opportunity: through careful design and implementation, mobile applications can be used to crowd source data collection. By garnering massive amounts of data from a wide variety of individuals, it is possible to explore psychological questions that have, to date, been out of reach. Here we discuss 2 examples of how data from the mobile game Airport Scanner (Kedlin Co., http://www.airportscannergame.com) can be used to address questions about the nature of visual search that pose intractable problems for laboratory-based research. Airport Scanner is a successful mobile game with millions of unique users and billions of individual trials, which allows for examining nuanced visual search questions. The goals of the current Observation Report were to highlight the growing opportunity that mobile technology affords psychological research and to provide an example roadmap of how to successfully collect usable data.

Examining perceptual and conceptual set biases in multiple-target visual search

Visual search is a common practice conducted countless times every day, and one important aspect of visual search is that multiple targets can appear in a single search array. For example, an X-ray image of airport luggage could contain both a water bottle and a gun. Searchers are more likely to miss additional targets after locating a first target in multiple-target searches, which presents a potential problem: If airport security officers were to find a water bottle, would they then be more likely to miss a gun? One hypothetical cause of multiple-target search errors is that searchers become biased to detect additional targets that are similar to a found target, and therefore become less likely to find additional targets that are dissimilar to the first target. This particular hypothesis has received theoretical, but little empirical, support. In the present study, we tested the bounds of this idea by utilizing “big data” obtained from the mobile application Airport Scanner. Multiple-target search errors were substantially reduced when the two targets were identical, suggesting that the first-found target did indeed create biases during subsequent search. Further analyses delineated the nature of the biases, revealing both a perceptual set bias (i.e., a bias to find additional targets with features similar to those of the first-found target) and a conceptual set bias (i.e., a bias to find additional targets with a conceptual relationship to the first-found target). These biases are discussed in terms of the implications for visual-search theories and applications for professional visual searchers.

Quality and accessibility of visual working memory during cognitive control of attentional guidance: A Bayesian model comparison approach

Working memory (WM) can guide visual attention toward memory-matching objects. This influence of WM on attention can be modulated by cognitive control, such that attentional guidance is strategically suppressed or enhanced depending on whether WM contents are reliably hurtful or helpful for the current task. Cognitive control over memory-based guidance has been hypothesized to operate via some modulation of the WM representation itself, but it is unclear whether this modulation affects representational quality (i.e., how precise is it?) or accessibility of WM content (i.e., how easily is it remembered or forgotten?). Using probabilistic model fitting and Bayesian model comparison techniques, we show that cognitive control over memory-based guidance impacts the probability of remembering, but not the precision of, items in WM. These findings suggest that the WM-attention interaction may depend on distinct functional states in WM, which are in turn characterized by how easily an item is remembered or forgotten.

Attentional guidance by working memory differs by paradigm: An individual-differences approach

The contents of working memory (WM) have been repeatedly found to guide the allocation of visual attention; in a dual-task paradigm that combines WM and visual search, actively holding an item in WM biases visual attention towards memory-matching items during search (e.g., Soto et al., Journal of Experimental Psychology: Human Perception and Performance, 31(2), 248-261, 2005). A key debate is whether such memory-based attentional guidance is automatic or under strategic control. Generally, two distinct task paradigms have been employed to assess memory-based guidance, one demonstrating that attention is involuntarily captured by memory-matching stimuli even at a cost to search performance (Soto et al., 2005), and one demonstrating that participants can strategically avoid memory-matching distractors to facilitate search performance (Woodman & Luck, Journal of Experimental Psychology: Human Perception and Performance, 33(2), 363-377, 2007). The current study utilized an individual-differences approach to examine why the different paradigms—which presumably tap into the same attentional construct—might support contrasting interpretations. Participants completed a battery of cognitive tasks, including two types of attentional guidance paradigms (see Soto et al., 2005; Woodman & Luck, 2007), a visual WM task, and an operation span task, as well as attention-related self-report assessments. Performance on the two attentional guidance paradigms did not correlate. Subsequent exploratory regression analyses revealed that memory-based guidance in each task was differentially predicted by visual WM capacity for one paradigm, and by attention-related assessment scores for the other paradigm. The current results suggest that these two paradigms—which have previously produced contrasting patterns of performance—may probe distinct aspects of attentional guidance.

Decoding working memory content from attentional biases

What we are currently thinking influences where we attend. The finding that active maintenance of visual items in working memory (WM) biases attention toward memory-matching objects—even when WM content is irrelevant for attentional goals—suggests a tight link between WM and attention. To test whether this link is reliable enough to infer specific WM content from measures of attentional bias, we applied multivariate pattern classification techniques to response times from an unrelated visual search task during a WM delay. Single-trial WM content was successfully decoded from incidental attentional bias within an individual, highlighting the specificity and reliability of the WM-attention link. Furthermore, classifiers trained on a group of individuals predicted WM content in another, completely independent individual—implying a shared cognitive mechanism of memory-driven attentional bias. The existence of such classifiers demonstrates that memory-based attentional bias is both a robust and generalizable probe of WM.

Neural Representation of Working Memory Content Is Modulated by Visual Attentional Demand

Recent theories assert that visual working memory (WM) relies on the same attentional resources and sensory substrates as visual attention to external stimuli. Behavioral studies have observed competitive tradeoffs between internal (i.e., WM) and external (i.e., visual) attentional demands, and neuroimaging studies have revealed representations of WM content as distributed patterns of activity within the same cortical regions engaged by perception of that content. Although a key function of WM is to protect memoranda from competing input, it remains unknown how neural representations of WM content are impacted by incoming sensory stimuli and concurrent attentional demands. Here, we investigated how neural evidence for WM information is affected when attention is occupied by visual search—at varying levels of difficulty—during the delay interval of a WM match-to-sample task. Behavioral and fMRI analyses suggested that WM maintenance was impacted by the difficulty of a concurrent visual task. Critically, multivariate classification analyses of category-specific ventral visual areas revealed a reduction in decodable WM-related information when attention was diverted to a visual search task, especially when the search was more difficult. This study suggests that the amount of available attention during WM maintenance influences the detection of sensory WM representations.

Fear generalization gradients in visuospatial attention.

Fear learning can be adaptively advantageous, but only if the learning is integrated with higher-order cognitive processes that impact goal-directed behaviors. Recent work has demonstrated generalization (i.e., transfer) of conditioned fear across perceptual dimensions and conceptual categories, but it is not clear how fear generalization influences other cognitive processes. The current study investigated how associative fear learning impacts higher-order visuospatial attention, specifically in terms of attentional bias toward generalized threats (i.e., the heightened assessment of potentially dangerous stimuli). We combined discriminative fear conditioning of color stimuli with a subsequent visual search task, in which targets and distractors were presented inside colored circles that varied in perceptual similarity to the fear-conditioned color. Skin conductance responses validated the fear-conditioning manipulation. Search response times indicated that attention was preferentially deployed not just to the specific fear-conditioned color, but also to similar colors that were never paired with the aversive shock. Furthermore, this attentional bias decreased continuously and symmetrically from the fear-conditioned value along the color spectrum, indicating a generalization gradient based on perceptual similarity. These results support functional accounts of fear learning that promote broad, defensive generalization of attentional bias toward threat. (PsycINFO Database Record

Mind-reading without the scanner: Behavioural decoding of working memory content

ABSTRACT Sophisticated machine learning algorithms have been successfully applied to functional neuroimaging data in order to characterize internal cognitive states. But is it possible to “mind-read” without the scanner? Capitalizing on the robust finding that the contents of working memory guide visual attention toward memory-matching objects, we trained a multivariate pattern classifier on behavioural indices of attentional guidance. Working memory representations were successfully decoded from behaviour alone, both within and between individuals. The current study provides a proof-of-concept for applying machine learning techniques to simple behavioural outputs (e.g., response times) in order to decode information about specific internal cognitive states.

Competitive tradeoffs between working memory and attention: An fMRI approach

Previous research has highlighted a close relationship between working memory (WM; i.e., the short-term maintenance and manipulation of internal information) and attention (i.e., the selective processing of a limited amount of external information). This reciprocity between WM and attention can be thought of as two sides of the same coin, such that WM (internal) and visual attention (external) could draw from, and thus compete for, a common cognitive resource. To examine the behavioral and neural characteristics of this interaction, we acquired fMRI data during a dual-task paradigm that simultaneously and systematically taxed both internal (WM) and external (visual) attentional load. Participants remembered one or two images (i.e., low vs. high internal load) for a delayed match-to-sample task. During the delay, participants performed a series of visual searches for target items that were either highly distinct from or similar to visual distractors (i.e., low vs. high external load). Behavioral data revealed that the impact of internal load on memory performance was inversely related to the impact of external load on search performance, suggesting that the effective processing of cognitive load in one domain increased susceptibility to load in the other domain. For example, for participants whose search performance was less affected by high external load, memory performance was more impaired by high internal load. Neural activity also revealed robust interactions between the two domains, such that parietal and temporal regions were differentially recruited to manage internal load in the face of low and high external load, respectively. Moreover, individual differences in independent measures of visual WM capacity and visual search efficiency modulated these behavioral and neural tradeoffs between internal and external attention. Collectively, these findings support the idea of WM and visual attention as competitive and interdependent constructs, whose interactions are processed by a network of fronto-temporo-parietal substrates. Meeting abstract presented at VSS 2015.

Visual search at the airport: Testing TSA officers

A significant challenge for laboratory-based research is to adequately replicate conditions found in the real world. Likewise, a challenge for field-based research is to appropriately maintain the precision and control found within the laboratory. These hurdles are easily noticed when studying visual search, the act of finding a target amongst distractors. Decades of laboratory-based research have revealed many factors affecting visual search (see Nakayama & Martini, 2011 for a recent review); yet, these ‘sterile’ tasks conducted with novice participants can at times bear little resemblance to the tasks of professional searchers such as baggage screeners, radiologists, lifeguards, and military personnel. Conversely, conducting research with expert searchers in their natural environment can be logistically complex, which limits the scope of questions that can be asked. We are bridging this gap by conducting laboratory-based research with professional, expert searchers: employed Transportation Security Administration (TSA) officers at Raleigh-Durham International Airport. We have established a cognitive psychology laboratory within the airport, and the TSA officers participate in our research studies during their normal work hours. We are assessing a variety of visual and attentional abilities, including several measures of visual search. For example, in one task we employed a simplified visual search experiment to directly compare novice searchers (Duke University undergraduates) to expert searchers (TSA officers). Participants looked for ‘T’s amongst ‘L’s with set sizes of 8, 16, 24, and 32. Compared to undergraduates, TSA officers were slower to respond, with search slopes approximately 1.5 times larger. Importantly, the TSA agents were also more accurate at each set size, suggesting a greater search diligence. Through tasks such as these, combined with measures of individual differences (e.g., personality and clinical assessments), the goal of this project is to inform both cognitive theories of visual search and the TSA’s standard operating procedures.