Haokui Xu

Tracking object number or information load in visual working memory: Revisiting the cognitive implication of contralateral delay activity

Two accounts prevail for the ERP component contralateral delay activity (CDA). One is that CDA tracks the number of objects stored in visual working memory (VWM), the more objects the higher amplitude (object number account). The other is that CDA reflects the maintained information load (information load account), the higher load the higher amplitude. The two accounts were tested by manipulating the information load and the object number of stored objects. Two or four arrows with low- or high-resolution information were remembered in separate blocks. In two experiments we found that the CDA-amplitude was higher for 4 arrows than for 2 arrows in low-resolution condition, yet no difference in high-resolution condition. Critically, there was no difference on CDA-amplitude among 2 low- and high-resolution objects, as well as 4 high-resolution objects, yet all were significantly lower than 4 low-resolution arrows. These results supported the object number account of CDA.

Does high memory load kick task-irrelevant information out of visual working memory?

The limited capacity of visual working memory (VWM) requires the existence of an efficient information selection mechanism. While it has been shown that under low VWM load, an irrelevant simple feature can be processed, its fate under high load (e.g., six objects) remains unclear. We explored this issue by probing the “irrelevant-change distracting effect,” in which the change of a stored irrelevant feature affects performance. Simple colored shapes were used as stimuli, with color as the target. Using a whole-probe method (presenting six objects in both the memory and test arrays), in Experiment 1 we found that a change to one of the six shapes led to a significant distracting effect. Using a partial-probe method (presenting the probe either at the screen center or at a location selected from the memory array), in Experiment 2 we showed the distracting effect again. These results suggest that irrelevant simple features can be stored into VWM, regardless of memory load.

The working memory Ponzo illusion: Involuntary integration of visuospatial information stored in visual working memory

Visual working memory (VWM) has been traditionally viewed as a mental structure subsequent to visual perception that stores the final output of perceptual processing. However, VWM has recently been emphasized as a critical component of online perception, providing storage for the intermediate perceptual representations produced during visual processing. This interactive view holds the core assumption that VWM is not the terminus of perceptual processing; the stored visual information rather continues to undergo perceptual processing if necessary. The current study tests this assumption, demonstrating an example of involuntary integration of the VWM content, by creating the Ponzo illusion in VWM: when the Ponzo illusion figure was divided into its individual components and sequentially encoded into VWM, the temporally separated components were involuntarily integrated, leading to the distorted length perception of the two horizontal lines. This VWM Ponzo illusion was replicated when the figure components were presented in different combinations and presentation order. The magnitude of the illusion was significantly correlated between VWM and perceptual versions of the Ponzo illusion. These results suggest that the information integration underling the VWM Ponzo illusion is constrained by the laws of visual perception and similarly affected by the common individual factors that govern its perception. Thus, our findings provide compelling evidence that VWM functions as a buffer serving perceptual processes at early stages.

Social constraints from an observer’s perspective: Coordinated actions make an agent’s position more predictable

Action prediction, a crucial ability to support social activities, is sensitive to the individual goals of expected actions. This article reports a novel finding that the predictions of observed actions for a temporarily invisible agent are influenced, and even enhanced, when this agent has a joint/collective goal to implement coordinated actions with others (i.e., with coordination information). Specifically, we manipulated the coordination information by presenting two chasers and one common target to perform coordinated or individual chases, and subjects were required to predict the expected action (i.e., position) for one chaser after it became momentarily invisible. To control for possible low-level physical properties, we also established some intense paired controls for each type of chase, such as backward replay (Experiment 1), making the chasing target invisible (Experiment 2) and a direct manipulation of the goal-directedness of one chasers movements to disrupt coordination information (Experiment 3). The results show that the prediction error for invisible chasers depends on whether the second chaser is coordinated with the first, and this effect vanishes when the chasers behaves with exactly the same motions, but without coordination information between them; furthermore, this influence results in enhancing the performance of action prediction. These findings extend the influential factors of action prediction to the level of observed coordination information, implying that the functional characteristic of mutual constraints of coordinated actions can be utilized by vision.

Does visual working memory work as a few fixed slots

The present study investigated whether visual working memory (VWM) functions as a few (about 3 ∼ 4) fixed slots by examining how the distribution of VWM is adjusted. Adopting a change-detection paradigm, we required subjects to memorize four items, one of which was prioritized. If VWM functions as 3 ∼ 4 slots, allocating multiple slots to the prioritized item would leave no slot for some other items; consequently no information would be stored for them, leading to a substantial decrease in change-detection performance no matter whether small or large changes occurred. The result showed that small changes on the unfavoured items were detected less accurately, indicating that more VWM was allocated to the favoured item. Yet meanwhile large changes that occurred on those unfavoured items could still be detected as well as those on the favoured one, indicating that each of them was still stored to some extent rather than completely discarded. The results suggested that VWM may not work as 3 ∼ 4 fixed slots. Possible mechanisms were discussed based on the present results, including a modified slot model with more available slots, a continuous resource model, and a hierarchical model that assumes storage of ensemble information in addition to the information of individual items.

Seeing “what” through “why”: Evidence from probing the causal structure of hierarchical motion.

Although our world is hierarchically organized, the perception, attention, and memory of hierarchical structures remain largely unknown. The current study shows how a hierarchical motion representation enhances the inference of an object’s position in a dynamic display. The motion hierarchy is formed as an acyclic tree in which each node represents a distinctive motion component. Each individual object is instantiated as a node in the tree. In a position inference task, participants were asked to infer the position of a target object, given how it moved jointly with other objects. The results showed that the inference is supported by the context formed by nontarget objects. More importantly, this contextual effect is (a) structured, with stronger support from objects forming a hierarchical tree than from those moving independently; (b) degreed, with stronger support from objects closer to the target in the motion tree; and (c) directed, with stronger support from the target’s ancestor nodes than from its descendent nodes. Computational modeling results further indicated that the contextual effect cannot be explained by correlated and contingent movements without an explicit causal representation of the motion hierarchy. Together, these studies suggest that human vision is a type of intelligence, which sees what are in the dynamic displays by recovering why and how they are generated.

Object-Based Attention on Social Units: Visual Selection of Hands Performing a Social Interaction

Traditionally, objects of attention are characterized either as full-fledged entities or either as elements grouped by Gestalt principles. Because humans appear to use social groups as units to explain social activities, we proposed that a socially defined group, according to social interaction information, would also be a possible object of attentional selection. This hypothesis was examined using displays with and without handshaking interactions. Results demonstrated that object-based attention, which was measured by an object-specific attentional advantage (i.e., shorter response times to targets on a single object), was extended to two hands performing a handshake but not to hands that did not perform meaningful social interactions, even when they did perform handshake-like actions. This finding cannot be attributed to the familiarity of the frequent co-occurrence of two handshaking hands. Hence, object-based attention can select a grouped object whose parts are connected within a meaningful social interaction. This finding implies that object-based attention is constrained by top-down information.

Social Coordination Information in Dynamic Chase Modulates EEG Mu Rhythm

Understanding actions plays an impressive role in our social life. Such processing has been suggested to be reflected by EEG Mu rhythm (8–13 Hz in sensorimotor regions). However, it remains unclear whether Mu rhythm is modulated by the social nature of coordination information in interactive actions (i.e., inter-dependency). This study used a novel manipulation of social coordination information: in a computer-based task, participants viewed a replay of two chasers chasing a common target coordinately (coordinated chase) or independently (solo chase). Simultaneously, to distinguish the potential effect of social coordination information from that of object-directed goal information, a control version of each condition was created by randomizing one chaser’s movement. In a second experiment, we made the target invisible to participants to control for low-level properties. Watching replays of coordinated chases induced stronger Mu suppression than solo chases, although both involved a common target. These effects were not explained by attention mechanisms or low-level physical patterns (e.g., the degree of physical synchronization). Therefore, the current findings suggest that processing social coordination information can be reflected by Mu rhythm. This function of Mu rhythm may characterize the activity of human mirror neuron system.

Corrigendum to “Social constraints from an observer’s perspective: Coordinated actions make an agent’s position more predictable” [Cognition 151 (2016) 10–17]

http://dx.doi.org/10.1016/j.cognition.2016.05.023 0010-0277/ 2016 Elsevier B.V. All rights reserved. DOI of original article: http://dx.doi.org/10.1016/j.cognition.2016.02.009 ⇑ Corresponding author at: Department of Psychology and Behavioral Sciences, Xixi Campus, Zhejiang University, Hangzhou 310028, PR China. E-mail address: mwshen@zju.edu.cn (M. Shen). Jun Yin , Haokui Xu , Xiaowei Ding , Junying Liang , Rende Shui , Mowei Shen a,⇑

The working memory Ponzo illusion: Involuntary integration of visuospatial information stored in visual working memory.

Visual working memory (VWM) has been traditionally viewed as a mental structure subsequent to visual perception that stores the final output of perceptual processing. However, VWM has recently been emphasized as a critical component of online perception, providing storage for the intermediate perceptual representations produced during visual processing. This interactive view holds the core assumption that VWM is not the terminus of perceptual processing; the stored visual information rather continues to undergo perceptual processing if necessary. The current study tests this assumption, demonstrating an example of involuntary integration of the VWM content, by creating the Ponzo illusion in VWM: when the Ponzo illusion figure was divided into its individual components and sequentially encoded into VWM (so that the four lines in Ponzo figure components were never simultaneously presented), the temporally separated components were involuntarily integrated, leading to the distorted length perception of the two horizontal lines. This VWM Ponzo illusion was replicated when the figure components were presented in different combinations and presentation order. The magnitude of the illusion was significantly correlated between VWM and perceptual versions of the Ponzo illusion. These results suggest that the information integration underling the VWM Ponzo illusion is constrained by the laws of visual perception and similarly affected by the common individual factors that govern its perception. Thus, our findings provide compelling evidence that VWM functions as a buffer serving perceptual processes at early stages. Meeting abstract presented at VSS 2015.