Ruyuan Zhang

Action video game play facilitates the development of better perceptual templates

Significance Recent advances in the field of learning have identified improvement of perceptual templates as a key mechanism underlying training-induced performance enhancements. Here, using a combination of psychophysics and neural modeling, we demonstrate that this mechanism—improved learning of perceptual templates—is also engaged after action video game play. Habitual action gamers or individuals trained to play action games demonstrate perceptual templates better tuned to the task and stimulus at hand than control groups, a difference shown to emerge as learning proceeds. This work further illustrates the importance of the development of improved perceptual templates as a mechanism mediating training and transfer effects and provides a novel account for the surprisingly broad transfer of performance enhancements noted after action game play. The field of perceptual learning has identified changes in perceptual templates as a powerful mechanism mediating the learning of statistical regularities in our environment. By measuring threshold-vs.-contrast curves using an orientation identification task under varying levels of external noise, the perceptual template model (PTM) allows one to disentangle various sources of signal-to-noise changes that can alter performance. We use the PTM approach to elucidate the mechanism that underlies the wide range of improvements noted after action video game play. We show that action video game players make use of improved perceptual templates compared with nonvideo game players, and we confirm a causal role for action video game play in inducing such improvements through a 50-h training study. Then, by adapting a recent neural model to this task, we demonstrate how such improved perceptual templates can arise from reweighting the connectivity between visual areas. Finally, we establish that action gamers do not enter the perceptual task with improved perceptual templates. Instead, although performance in action gamers is initially indistinguishable from that of nongamers, action gamers more rapidly learn the proper template as they experience the task. Taken together, our results establish for the first time to our knowledge the development of enhanced perceptual templates following action game play. Because such an improvement can facilitate the inference of the proper generative model for the task at hand, unlike perceptual learning that is quite specific, it thus elucidates a general learning mechanism that can account for the various behavioral benefits noted after action game play.

Illusory Movement of Stationary Stimuli in the Visual Periphery: Evidence for a Strong Centrifugal Prior in Motion Processing

Visual input is remarkably diverse. Certain sensory inputs are more probable than others, mirroring statistical regularities of the visual environment. The visual system exploits many of these regularities, resulting, on average, in better inferences about visual stimuli. However, by incorporating prior knowledge into perceptual decisions, visual processing can also result in perceptions that do not match sensory inputs. Such perceptual biases can often reveal unique insights into underlying mechanisms and computations. For example, a prior assumption that objects move slowly can explain a wide range of motion phenomena. The prior on slow speed is usually rationalized by its match with visual input, which typically includes stationary or slow moving objects. However, this only holds for foveal and parafoveal stimulation. The visual periphery tends to be exposed to faster motions, which are biased toward centrifugal directions. Thus, if prior assumptions derive from experience, peripheral motion processing should be biased toward centrifugal speeds. Here, in experiments with human participants, we support this hypothesis and report a novel visual illusion where stationary objects in the visual periphery are perceived as moving centrifugally, while objects moving as fast as 7°/s toward fovea are perceived as stationary. These behavioral results were quantitatively explained by a Bayesian observer that has a strong centrifugal prior. This prior is consistent with both the prevalence of centrifugal motions in the visual periphery and a centrifugal bias of direction tuning in cortical area MT, supporting the notion that visual processing mirrors its input statistics.

Visual recovery in cortical blindness is limited by high internal noise

Damage to the primary visual cortex typically causes cortical blindness (CB) in the hemifield contralateral to the damaged hemisphere. Recent evidence indicates that visual training can partially reverse CB at trained locations. Whereas training induces near-complete recovery of coarse direction and orientation discriminations, deficits in fine motion processing remain. Here, we systematically disentangle components of the perceptual inefficiencies present in CB fields before and after coarse direction discrimination training. In seven human CB subjects, we measured threshold versus noise functions before and after coarse direction discrimination training in the blind field and at corresponding intact field locations. Threshold versus noise functions were analyzed within the framework of the linear amplifier model and the perceptual template model. Linear amplifier model analysis identified internal noise as a key factor differentiating motion processing across the tested areas, with visual training reducing internal noise in the blind field. Differences in internal noise also explained residual perceptual deficits at retrained locations. These findings were confirmed with perceptual template model analysis, which further revealed that the major residual deficits between retrained and intact field locations could be explained by differences in internal additive noise. There were no significant differences in multiplicative noise or the ability to process external noise. Together, these results highlight the critical role of altered internal noise processing in mediating training-induced visual recovery in CB fields, and may explain residual perceptual deficits relative to intact regions of the visual field.

Perceptual training yields rapid improvements in visually impaired youth

Visual function demands coordinated responses to information over a wide field of view, involving both central and peripheral vision. Visually impaired individuals often seem to underutilize peripheral vision, even in absence of obvious peripheral deficits. Motivated by perceptual training studies with typically sighted adults, we examined the effectiveness of perceptual training in improving peripheral perception of visually impaired youth. Here, we evaluated the effectiveness of three training regimens: (1) an action video game, (2) a psychophysical task that combined attentional tracking with a spatially and temporally unpredictable motion discrimination task, and (3) a control video game. Training with both the action video game and modified attentional tracking yielded improvements in visual performance. Training effects were generally larger in the far periphery and appear to be stable 12 months after training. These results indicate that peripheral perception might be under-utilized by visually impaired youth and that this underutilization can be improved with only ~8 hours of perceptual training. Moreover, the similarity of improvements following attentional tracking and action video-game training suggest that well-documented effects of action video-game training might be due to the sustained deployment of attention to multiple dynamic targets while concurrently requiring rapid attending and perception of unpredictable events.

High internal noise and poor external noise filtering characterize perception in autism spectrum disorder

An emerging hypothesis postulates that internal noise is a key factor influencing perceptual abilities in autism spectrum disorder (ASD). Given fundamental and inescapable effects of noise on nearly all aspects of neural processing, this could be a critical abnormality with broad implications for perception, behavior, and cognition. However, this proposal has been challenged by both theoretical and empirical studies. A crucial question is whether and how internal noise limits perception in ASD, independently from other sources of perceptual inefficiency, such as the ability to filter out external noise. Here, we separately estimated internal noise and external noise filtering in ASD. In children and adolescents with and without ASD, we computationally modeled individuals’ visual orientation discrimination in the presence of varying levels of external noise. The results revealed increased internal noise and worse external noise filtering in individuals with ASD. For both factors, we also observed high inter-individual variability in ASD, with only the internal noise estimates significantly correlating with severity of ASD symptoms. We provide evidence for reduced perceptual efficiency in ASD that is due to both increased internal noise and worse external noise filtering, while highlighting internal noise as a possible contributing factor to variability in ASD symptoms.

Disentangling locus of perceptual learning in the visual hierarchy of motion processing

Visual perceptual learning (VPL) can lead to long-lasting perceptual improvements. While the efficacy of VPL is well established, there is still a considerable debate about what mechanisms underlie the effects of VPL. Much of this debate concentrates on where along the visual processing hierarchy behaviorally relevant plasticity takes place. Here, we aimed to tackle this question in context of motion processing, a domain where links between behavior and processing hierarchy are well established. Specifically, we took advantage of an established transition from component-dependent representations at the earliest level to pattern-dependent representations at the middle-level of cortical motion processing. We trained two groups of participants on the same motion direction identification task using either grating or plaid stimuli. A set of pre- and post-training tests was used to determine the degree of learning specificity and generalizability. This approach allowed us to disentangle contributions from both low- and mid-level motion processing, as well as high-level cognitive changes. We observed a complete bi-directional transfer of learning between component and pattern stimuli as long as they shared the same apparent motion direction. This result indicates learning-induced plasticity at intermediate levels of motion processing. Moreover, we found that motion VPL is specific to the trained stimulus direction, speed, size, and contrast, highlighting the pivotal role of basic visual features in VPL, and diminishing the possibility of non-sensory decision-level enhancements. Taken together, our study psychophysically examined a variety of factors mediating motion VPL, and demonstrated that motion VPL most likely alters visual computation in the middle stage of motion processing.

Flexible top-down modulation in human ventral temporal cortex

Visual neuroscientists have long characterized attention as inducing a scaling or additive effect on fixed parametric functions describing neural responses (e.g., contrast response functions). Here, we instead propose that top-down effects are more complex and manifest in ways that depend not only on attention but also other cognitive processes involved in executing a task. To substantiate this theory, we analyze fMRI responses in human ventral temporal cortex (VTC) in a study where stimulus eccentricity and cognitive task are varied. We find that as stimuli are presented farther into the periphery, bottom-up stimulus-driven responses decline but top-down attentional enhancement increases substantially. This disproportionate enhancement of weak responses cannot be easily explained by conventional models of attention. Furthermore, we find that attentional effects depend on the specific cognitive task performed by the subject, indicating the influence of additional cognitive processes other than attention (e.g., decision-making). The effects we observe replicate in an independent experiment from the same study, and also generalize to a separate study involving different stimulus manipulations (contrast and phase coherence). Our results suggest that a quantitative understanding of top-down modulation requires more nuanced and more precise characterization of multiple cognitive factors involved in completing a perceptual task.

Atypically larger variability of resource allocation accounts for visual working memory deficits in schizophrenia

Schizophrenia patients are known to have profound deficits in visual working memory (VWM), and almost all previous studies attribute the deficits to decreased memory capacity. This account, however, ignores the potential contributions of other VWM components (e.g., memory precision). Here, we measure the VWM performance of schizophrenia patients and healthy control subjects on two classical delay-estimation tasks. Moreover, we thoroughly evaluate several established computational models of VWM to compare the performance of the two groups. We find that the model assuming variable precision across items and trials is the best model to explain the performance of both groups. According to the variable-precision model, schizophrenia subjects exhibit abnormally larger variability of allocating memory resources rather than resources per se. These results invite a rethink of the widely accepted decreased-capacity theory and propose a new perspective on the diagnosis and rehabilitation of schizophrenia.

A critical assessment of data quality and venous effects in ultra-high-resolution fMRI

Advances in hardware, pulse sequences, and reconstruction techniques have made it possible to perform functional magnetic resonance imaging (fMRI) at sub-millimeter resolution while maintaining high spatial coverage and acceptable signal-to-noise ratio. Here, we examine whether ultra-high-resolution fMRI can be exploited for routine use in neuroscience research. We conducted fMRI in human visual cortex during a simple event-related visual experiment (7T, gradient-echo EPI, 0.8-mm isotropic voxels, 2.2-s sampling rate, 84 slices), and developed analysis and visualization tools to assess the quality of the data. We make three main observations. First, we find that the acquired fMRI images, combined with appropriate surface-based processing, provide reliable and accurate measurements of fine-scale blood oxygenation level dependent (BOLD) activity patterns. Second, we show that the highly folded structure of cortex causes substantial biases on spatial resolution and data visualization. Third, we examine the well-recognized issue of venous contributions to fMRI signals. In a systematic assessment of large sections of cortex measured at a fine scale, we show that time-averaged T2*-weighted EPI intensity is a simple, robust marker of venous effects. These venous effects are unevenly distributed across cortex, are more pronounced in gyri and outer cortical depths, and are, to a certain degree, in consistent locations across subjects relative to cortical folding. Furthermore, we show that these venous effects are strongly correlated with BOLD responses evoked by the experiment. We conclude that ultra-high-resolution fMRI can provide robust information about fine-scale BOLD activity patterns, but special care must be exercised in visualizing and interpreting these patterns, especially with regards to the confounding influence of the brain’s vasculature. To help translate these methodological findings to neuroscience research, we provide practical suggestions for both high-resolution and standard-resolution fMRI studies.

Binocular Rivalry: A Window into Cortical Competition and Suppression

When the two eyes view very dissimilar images, the visual system often fails to combine the images and one experiences stochastically alternating percepts. This phenomenon, called binocular rivalry, has fascinated researchers for centuries since it provides insights into two critical aspects of visual perception: visual consciousness and cortical suppression. Here, we review the mechanisms of binocular rivalry from a cognitive neuroscience perspective, focusing on empirical findings from two widely used methods—functional magnetic resonance imaging (fMRI) and electroencephalography (EEG). With these techniques, researchers have been able to identify the cortical sites of suppression in binocular rivalry, probe neural responses evoked by unconscious (invisible) visual stimuli, and examine the role of top-down attentional signals in rivalry. We conclude by proposing some future directions for the study of binocular rivalry.