Sage E. P. Boettcher

Searching for the right word: Hybrid visual and memory search for words

In “hybrid search” (Wolfe Psychological Science, 23(7), 698-703, 2012), observers search through visual space for any of multiple targets held in memory. With photorealistic objects as the stimuli, response times (RTs) increase linearly with the visual set size and logarithmically with the memory set size, even when over 100 items are committed to memory. It is well-established that pictures of objects are particularly easy to memorize (Brady, Konkle, Alvarez, & Oliva Proceedings of the National Academy of Sciences, 105, 14325–14329, 2008). Would hybrid-search performance be similar if the targets were words or phrases, in which word order can be important, so that the processes of memorization might be different? In Experiment 1, observers memorized 2, 4, 8, or 16 words in four different blocks. After passing a memory test, confirming their memorization of the list, the observers searched for these words in visual displays containing two to 16 words. Replicating Wolfe (Psychological Science, 23(7), 698-703, 2012), the RTs increased linearly with the visual set size and logarithmically with the length of the word list. The word lists of Experiment 1 were random. In Experiment 2, words were drawn from phrases that observers reported knowing by heart (e.g., “London Bridge is falling down”). Observers were asked to provide four phrases, ranging in length from two words to no less than 20 words (range 21–86). All words longer than two characters from the phrase, constituted the target list. Distractor words were matched for length and frequency. Even with these strongly ordered lists, the results again replicated the curvilinear function of memory set size seen in hybrid search. One might expect to find serial position effects, perhaps reducing the RTs for the first (primacy) and/or the last (recency) members of a list (Atkinson & Shiffrin, 1968; Murdock Journal of Experimental Psychology, 64, 482–488, 1962). Surprisingly, we showed no reliable effects of word order. Thus, in “London Bridge is falling down,” “London” and “down” were found no faster than “falling.”

Hybrid search in context: How to search for vegetables in the produce section and cereal in the cereal aisle

Amano, S., & Kondo, T. (1999). Nihongo-no goi-tokusei [Lexical properties of Japanese], Vol. 1 (CD-ROM version). Tokyo, Japan: Sanseido. (in Japanese) Asano, M., & Yokosawa, K. (2011). Synesthetic colors are elicited by sound quality in Japanese synesthetes. Consciousness and Cognition, 20, 1816–1823. doi:10.1016/j.concog.2011.05.012 Asano, M., & Yokosawa, K. (2012). Synesthetic colors for Japanese late acquired graphemes. Consciousness and Cognition, 21, 983–993. doi:10.1016/j.concog.2012.02.005 Beeli, G., Esslen, M., & Jäncke, L. (2007). Frequency correlates in grapheme-color synaesthesia. Psychological Science, 18, 788–792. doi:10.1111/j.1467-9280.2007.01980.x Boles, D. B., & Clifford, J. E. (1989). An upperand lowercase alphabetic similarity matrix, with derived generation similarity values. Behavior Research Methods, Instruments, and Computers, 21, 579–586. doi:10.3758/BF03210580 Kawakami, M., & Tsuji, H. (2012). Hiragana-moji-no keitai-teki ruijisei hyoutei chousa [Survey on orthographical similarity of Japanese Hiragana script characters]. Science of Reading, 54, 80–87. (in Japanese) Rich, A. N., Bradshaw, J. L., & Mattingley, J. B. (2005). A systematic, large-scale study of synaesthesia: Implications for the role of early experience in lexical-colour associations. Cognition, 98, 53–84. doi:10.1016/j.cognition.2004.11.003 Wagner, K., & Dobkins, K. R. (2011). Synaesthetic associations decrease during infancy. Psychological Science, 22, 1067–1072. doi:10.1177/0956797611416250 Watson, M. R., Akins, K. A., & Enns, J. T. (2012). Second-order mappings in grapheme-color synesthesia. Psychonomic Bulletin and Review, 19, 211–217. doi:10.3758/s13423-011-0208-4 Witthoft, N., & Winawer, J. (2013). Learning, memory, and synesthesia. Psychological Science, 24, 258–265. doi:10.1177/0956797612452573

Searching while loaded: Visual working memory does not interfere with hybrid search efficiency but hybrid search uses working memory capacity

In “hybrid search” tasks, such as finding items on a grocery list, one must search the scene for targets while also searching the list in memory. How is the representation of a visual item compared with the representations of items in the memory set? Predominant theories would propose a role for visual working memory (VWM) either as the site of the comparison or as a conduit between visual and memory systems. In seven experiments, we loaded VWM in different ways and found little or no effect on hybrid search performance. However, the presence of a hybrid search task did reduce the measured capacity of VWM by a constant amount regardless of the size of the memory or visual sets. These data are broadly consistent with an account in which VWM must dedicate a fixed amount of its capacity to passing visual representations to long-term memory for comparison to the items in the memory set. The data cast doubt on models in which the search template resides in VWM or where memory set item representations are moved from LTM through VWM to earlier areas for comparison to visual items.

Lost in the supermarket: Quantifying the cost of partitioning memory sets in hybrid search

The items on a memorized grocery list are not relevant in every aisle; for example, it is useless to search for the cabbage in the cereal aisle. It might be beneficial if one could mentally partition the list so only the relevant subset was active, so that vegetables would be activated in the produce section. In four experiments, we explored observers’ abilities to partition memory searches. For example, if observers held 16 items in memory, but only eight of the items were relevant, would response times resemble a search through eight or 16 items? In Experiments 1a and 1b, observers were not faster for the partition set; however, they suffered relatively small deficits when “lures” (items from the irrelevant subset) were presented, indicating that they were aware of the partition. In Experiment 2 the partitions were based on semantic distinctions, and again, observers were unable to restrict search to the relevant items. In Experiments 3a and 3b, observers attempted to remove items from the list one trial at a time but did not speed up over the course of a block, indicating that they also could not limit their memory searches. Finally, Experiments 4a, 4b, 4c, and 4d showed that observers were able to limit their memory searches when a subset was relevant for a run of trials. Overall, observers appear to be unable or unwilling to partition memory sets from trial to trial, yet they are capable of restricting search to a memory subset that remains relevant for several trials. This pattern is consistent with a cost to switching between currently relevant memory items.

One visual search, many memory searches: An eye-tracking investigation of hybrid search

Suppose you go to the supermarket with a shopping list of 10 items held in memory. Your shopping expedition can be seen as a combination of visual search and memory search. This is known as “hybrid search.” There is a growing interest in understanding how hybrid search tasks are accomplished. We used eye tracking to examine how manipulating the number of possible targets (the memory set size [MSS]) changes how observers (Os) search. We found that dwell time on each distractor increased with MSS, suggesting a memory search was being executed each time a new distractor was fixated. Meanwhile, although the rate of refixation increased with MSS, it was not nearly enough to suggest a strategy that involves repeatedly searching visual space for subgroups of the target set. These data provide a clear demonstration that hybrid search tasks are carried out via a “one visual search, many memory searches” heuristic in which Os examine items in the visual array once with a very low rate of refixations. For each item selected, Os activate a memory search that produces logarithmic response time increases with increased MSS. Furthermore, the percentage of distractors fixated was strongly modulated by the MSS: More items in the MSS led to a higher percentage of fixated distractors. Searching for more potential targets appears to significantly alter how Os approach the task, ultimately resulting in more eye movements and longer response times.

Memory search for the first target modulates the magnitude of the attentional blink

The resolution of temporal attention is limited in a manner that makes it difficult to identify two targets in short succession. This limitation produces the phenomenon known as the attentional blink (AB), in which processing of a first target (T1) impairs identification of a second target (T2). In the AB literature, there is broad agreement that increasing the time it takes to process T1 leads to a larger AB. One might, therefore, predict that increasing the number of possible T1 identities, or target set, from 1 to 16 would lead to a larger AB. We were surprised to find that this manipulation of T1 difficulty had no influence on AB magnitude. In subsequent experiments, we found that AB magnitude interacts with T1 processing time only under certain circumstances. Specifically, when the T1 task was either well masked or had to be completed online, we found a reliable interaction between AB magnitude and the target set size. When neither of these conditions was fulfilled, there was no interaction between target set size and the AB. Previous research found that when the target set changes from trial to trial, trials with more possible targets elicited a larger AB. In the present study, the target set is held constant, reducing the demands on working memory. Nevertheless, AB magnitude still interacts with target set size, as long as the T1 task cannot be processed offline. Thus, the act of searching memory delays subsequent processing, even when the role of working memory has been minimized.

When Does the Aardvark Move to the Next Anthill? Foraging search with moving targets

Wolfe, J. M. (2013). When is it time to move to the next raspberry bush? Foraging rules in human visual search. Journal of Vision, 13(3), 1–17. • Moving item displays are an effective method for investigating visual foraging • Moving item displays reduce spatial search systamaticity • Searchers ignored the quality of the upcoming trial when deciding when to quit • In Hybrid Foraging, searchers make runs of one target type before switching to to another target type • Anecdotally, these experiments are way more fun than our usual static search experiments! Task: Get a high rate of point accumulation • Two patches of moving dots appear side-by-side • Left patch is the active search display, right patch is a preview of the next trial • Preview was absent on 25% of trials • Initial set size of 48 dots/patch • Dots worth 0–16 points; greener was always better • 36 different patches with different mean greenness • “Next” button advanced the trial, with the preview moving left to the active side Task: Achieve a set point goal to complete the experiment • Four target items: • Display sizes of 60, 80, 100, & 120 • 20–30% inital target prevalance on each trial • Choosing an item removed it from the display, +2 points for Hits, –1 point for FAs • “Next” button advanced the trial