Joel Pearson

Sensory memory for ambiguous vision

In recent years the overlap between visual perception and memory has shed light on our understanding of both. When ambiguous images that normally cause perception to waver unpredictably are presented briefly with intervening blank periods, perception tends to freeze, locking into one interpretation. This indicates that there is a form of memory storage across the blank interval. This memory trace codes low-level characteristics of the stored stimulus. Although a trace is evident after a single perceptual instance, the trace builds over many separate stimulus presentations, indicating a flexible, variable-length time-course. This memory shares important characteristics with priming by non-ambiguous stimuli. Computational models now provide a framework to interpret many empirical observations.

The Functional Impact of Mental Imagery on Conscious Perception

Mental imagery has been proposed to contribute to a variety of high-level cognitive functions, including memory encoding and retrieval, navigation, spatial planning, and even social communication and language comprehension. However, it is debated whether mental imagery relies on the same sensory representations as perception, and if so, what functional consequences such an overlap might have on perception itself. We report novel evidence that single instances of imagery can have a pronounced facilitatory influence on subsequent conscious perception. Either seeing or imagining a specific pattern could strongly bias which of two competing stimuli reach awareness during binocular rivalry. Effects of imagery and perception were location and orientation specific, accumulated in strength over time, and survived an intervening visual task lasting several seconds prior to presentation of the rivalry display. Interestingly, effects of imagery differed from those of feature-based attention. The results demonstrate that imagery, in the absence of any incoming visual signals, leads to the formation of a short-term sensory trace that can bias future perception, suggesting a means by which high-level processes that support imagination and memory retrieval may shape low-level sensory representations.

Mental Imagery: Functional Mechanisms and Clinical Applications

Mental imagery research has weathered both disbelief of the phenomenon and inherent methodological limitations. Here we review recent behavioral, brain imaging, and clinical research that has reshaped our understanding of mental imagery. Research supports the claim that visual mental imagery is a depictive internal representation that functions like a weak form of perception. Brain imaging work has demonstrated that neural representations of mental and perceptual images resemble one another as early as the primary visual cortex (V1). Activity patterns in V1 encode mental images and perceptual images via a common set of low-level depictive visual features. Recent translational and clinical research reveals the pivotal role that imagery plays in many mental disorders and suggests how clinicians can utilize imagery in treatment.

Evaluating the Mind’s Eye The Metacognition of Visual Imagery

Can people evaluate phenomenal qualities of internally generated experiences, such as whether a mental image is vivid or detailed? This question exemplifies a problem of metacognition: How well do people know their own thoughts? In the study reported here, participants were instructed to imagine a specific visual pattern and rate its vividness, after which they were presented with an ambiguous rivalry display that consisted of the previously imagined pattern plus an orthogonal pattern. On individual trials, higher ratings of vividness predicted a greater likelihood that the imagined pattern would appear dominant when the participant was subsequently presented with the binocular rivalry display. Off-line self-report questionnaires measuring imagery vividness also predicted individual differences in the strength of imagery bias over the entire study. Perceptual bias due to mental imagery could not be attributed to demand characteristics, as no bias was observed on catch-trial presentations of mock rivalry displays. Our findings provide novel evidence that people have a good metacognitive understanding of their own mental imagery and can reliably evaluate the vividness of single episodes of imagination.

Intermittent ambiguous stimuli: Implicit memory causes periodic perceptual alternations

When viewing a stimulus that has multiple plausible real-world interpretations, perception alternates between these interpretations every few seconds. Alternations can be halted by intermittently removing the stimulus from view. The same interpretation dominates over many successive presentations, and perception stabilizes. Here we study perception during long sessions of such intermittent presentation. We demonstrate that, rather than causing truly stable perception, intermittent presentation gives rise to a perceptual alternation cycle with its own characteristics and dependencies, different from those during continuous presentation. Alternations during intermittent viewing typically occur once every few minutes--much less frequently than the seconds-scale alternations during continuous viewing. Strikingly, alternations during intermittent viewing occur at fairly regular intervals, making for a surprisingly periodic alternation cycle. The duration of this cycle becomes longer as the blank duration between presentations is increased, reaching dozens of minutes in some cases. We interpret our findings in terms of a mathematical model that describes a neural network with competition between alternative interpretations. Network sensitivities depend on prior dominance, thus providing a memory for past perception. Slow changes in sensitivity produce both perceptual stabilization and the regular but infrequent alternations, meaning that the same memory traces are responsible for both. This model provides a good description of psychophysical findings, and offers several indications regarding their neural basis.

Determinants of visual awareness following interruptions during rivalry.

The inability of the human visual system to fuse dissimilar patterns in corresponding regions of the two eyes results in stochastic alternation of perceptual dominance between the two patterns: rivalry. When rivalrous stimuli are presented intermittently their perception is stabilized (Leopold, Wilke, Maier, & Logothetis, 2002). This stability indicates the operation of some kind of perceptual memory across interruptions in stimulation. Here we examined the contents of this perceptual memory to quantify the relative contributions of different sources of information: eye-of-origin, orientation, and color. Stimuli were intermittently presented and, during each blank interruption, we swapped either the color, orientation, or eye of presentation of the gratings. Comparing the percepts reported before and after each interruption allowed us to establish what aspects of perception remained stable. During conventional binocular rivalry, the eye in which the stimulus was presented remained stable across 74% of interruptions. Stimulus color and orientation also had weaker significant effects. When eye-of-origin information was eliminated by alternating the patterns rapidly between the two eyes, stimulus color remained stable across 86% of interruptions. Stimulus orientation again had a weaker but significant effect. These results demonstrate that the mechanisms mediating perceptual stability across interruptions in rivalry can operate at both monocular and binocular levels, much like the mechanisms operating during continuous viewing of rivalrous stimuli. On the basis of this similarity, we speculate that perceptual memory across interruptions in rivalry may involve the same neural representations as visual competition during rivalry. If this is the case, the use of intermittent stimulation in rivalry might permit the investigation of aspects of the mechanisms underlying visual competition that remain hidden during continuous presentation.

Mental Imagery and Visual Working Memory

Visual working memory provides an essential link between past and future events. Despite recent efforts, capacity limits, their genesis and the underlying neural structures of visual working memory remain unclear. Here we show that performance in visual working memory - but not iconic visual memory - can be predicted by the strength of mental imagery as assessed with binocular rivalry in a given individual. In addition, for individuals with strong imagery, modulating the background luminance diminished performance on visual working memory and imagery tasks, but not working memory for number strings. This suggests that luminance signals were disrupting sensory-based imagery mechanisms and not a general working memory system. Individuals with poor imagery still performed above chance in the visual working memory task, but their performance was not affected by the background luminance, suggesting a dichotomy in strategies for visual working memory: individuals with strong mental imagery rely on sensory-based imagery to support mnemonic performance, while those with poor imagery rely on different strategies. These findings could help reconcile current controversy regarding the mechanism and location of visual mnemonic storage.

Perceptual Grouping of Biological Motion Promotes Binocular Rivalry

Investigation of perceptual rivalry between conflicting stimuli presented one to each eye can further understanding of the neural underpinnings of conscious visual perception. During rivalry, visual awareness fluctuates between perceptions of the two stimuli. Here, we demonstrate that high-level perceptual grouping can promote rivalry between stimulus pairs that would otherwise be perceived as nonrivalrous. Perceptual grouping was generated with point-light walker stimuli that simulate human motion, visible only as lights placed on the joints. Although such walking figures are unrecognizable when stationary, recognition judgments as complex as gender and identity can accurately be made from animated displays, demonstrating the efficiency with which our visual system can group dynamic local signals into a globally coherent walking figure. We find that point-light walker stimuli presented one to each eye and in different colors and configurations results in strong rivalry. However, rivalry is minimal when the two walkers are split between the eyes or both presented to one eye. This pattern of results suggests that processing animated walker figures promotes rivalry between signals from the two eyes rather than between higher-level representations of the walkers. This leads us to hypothesize that awareness during binocular rivalry involves the integrated activity of high-level perceptual mechanisms in conjunction with lower-level ocular suppression modulated via cortical feedback.

The effects of transcranial magnetic stimulation on visual rivalry

One extensively investigated form of perceptual bistability is binocular rivalry--When dissimilar patterns are presented one to each eye, these patterns compete for perceptual dominance. Here, we report that transcranial magnetic stimulation (TMS) over early visual areas induces alternations during binocular rivalry. The effect of TMS on binocular rivalry was retinotopic, suggesting that rivalry mechanisms are localized in the cortical representation of visual space. The timing of perturbations was highly dependent on individual differences in rivalry alternation frequencies, with more delayed effects found in slower alternators. This finding suggests that both binocular rivalry and TMS dynamics might be contingent on individual differences among observers. We performed an analogous set of experiments by replacing TMS with transient visual stimulation. The results, however, qualitatively and quantitatively differed from those reported with TMS. Finally, we found that TMS over early visual areas does not produce any time-locked effects on another dynamical process--eye-swapping stimulus rivalry. These findings constitute the first causative evidence that binocular rivalry is contingent on neural activity in early visual areas and suggest that binocular rivalry and stimulus rivalry have different neural correlates, supporting multilevel theories of visual rivalry.

The heterogeneity of mental representation: Ending the imagery debate

The possible ways that information can be represented mentally have been discussed often over the past thousand years. However, this issue could not be addressed rigorously until late in the 20th century. Initial empirical findings spurred a debate about the heterogeneity of mental representation: Is all information stored in propositional, language-like, symbolic internal representations, or can humans use at least two different types of representations (and possibly many more)? Here, in historical context, we describe recent evidence that humans do not always rely on propositional internal representations but, instead, can also rely on at least one other format: depictive representation. We propose that the debate should now move on to characterizing all of the different forms of human mental representation.