Janneke Jehee

Attention Improves Encoding of Task-Relevant Features in the Human Visual Cortex

When spatial attention is directed toward a particular stimulus, increased activity is commonly observed in corresponding locations of the visual cortex. Does this attentional increase in activity indicate improved processing of all features contained within the attended stimulus, or might spatial attention selectively enhance the features relevant to the observers task? We used fMRI decoding methods to measure the strength of orientation-selective activity patterns in the human visual cortex while subjects performed either an orientation or contrast discrimination task, involving one of two laterally presented gratings. Greater overall BOLD activation with spatial attention was observed in visual cortical areas V1–V4 for both tasks. However, multivariate pattern analysis revealed that orientation-selective responses were enhanced by attention only when orientation was the task-relevant feature and not when the contrast of the grating had to be attended. In a second experiment, observers discriminated the orientation or color of a specific lateral grating. Here, orientation-selective responses were enhanced in both tasks, but color-selective responses were enhanced only when color was task relevant. In both experiments, task-specific enhancement of feature-selective activity was not confined to the attended stimulus location but instead spread to other locations in the visual field, suggesting the concurrent involvement of a global feature-based attentional mechanism. These results suggest that attention can be remarkably selective in its ability to enhance particular task-relevant features and further reveal that increases in overall BOLD amplitude are not necessarily accompanied by improved processing of stimulus information.

Perceptual Learning Selectively Refines Orientation Representations in Early Visual Cortex

Although practice has long been known to improve perceptual performance, the neural basis of this improvement in humans remains unclear. Using fMRI in conjunction with a novel signal detection-based analysis, we show that extensive practice selectively enhances the neural representation of trained orientations in the human visual cortex. Twelve observers practiced discriminating small changes in the orientation of a laterally presented grating over 20 or more daily 1 h training sessions. Training on average led to a twofold improvement in discrimination sensitivity, specific to the trained orientation and the trained location, with minimal improvement found for untrained orthogonal orientations or for orientations presented in the untrained hemifield. We measured the strength of orientation-selective responses in individual voxels in early visual areas (V1–V4) using signal detection measures, both before and after training. Although the overall amplitude of the BOLD response was no greater after training, practice nonetheless specifically enhanced the neural representation of the trained orientation at the trained location. This training-specific enhancement of orientation-selective responses was observed in the primary visual cortex (V1) as well as higher extrastriate visual areas V2–V4, and moreover, reliably predicted individual differences in the behavioral effects of perceptual learning. These results demonstrate that extensive training can lead to targeted functional reorganization of the human visual cortex, refining the cortical representation of behaviorally relevant information.

Reinstatement of Associative Memories in Early Visual Cortex Is Signaled by the Hippocampus

The cortical reinstatement hypothesis of memory retrieval posits that content-specific cortical activity at encoding is reinstated at retrieval. Evidence for cortical reinstatement was found in higher-order sensory regions, reflecting reactivation of complex object-based information. However, it remains unclear whether the same detailed sensory, feature-based information perceived during encoding is subsequently reinstated in early sensory cortex and what the role of the hippocampus is in this process. In this study, we used a combination of visual psychophysics, functional neuroimaging, multivoxel pattern analysis, and a well controlled cued recall paradigm to address this issue. We found that the visual information human participants were retrieving could be predicted by the activation patterns in early visual cortex. Importantly, this reinstatement resembled the neural pattern elicited when participants viewed the visual stimuli passively, indicating shared representations between stimulus-driven activity and memory. Furthermore, hippocampal activity covaried with the strength of stimulus-specific cortical reinstatement on a trial-by-trial level during cued recall. These findings provide evidence for reinstatement of unique associative memories in early visual cortex and suggest that the hippocampus modulates the mnemonic strength of this reinstatement.

Predictive Feedback Can Account for Biphasic Responses in the Lateral Geniculate Nucleus

Biphasic neural response properties, where the optimal stimulus for driving a neural response changes from one stimulus pattern to the opposite stimulus pattern over short periods of time, have been described in several visual areas, including lateral geniculate nucleus (LGN), primary visual cortex (V1), and middle temporal area (MT). We describe a hierarchical model of predictive coding and simulations that capture these temporal variations in neuronal response properties. We focus on the LGN-V1 circuit and find that after training on natural images the model exhibits the brains LGN-V1 connectivity structure, in which the structure of V1 receptive fields is linked to the spatial alignment and properties of center-surround cells in the LGN. In addition, the spatio-temporal response profile of LGN model neurons is biphasic in structure, resembling the biphasic response structure of neurons in cat LGN. Moreover, the model displays a specific pattern of influence of feedback, where LGN receptive fields that are aligned over a simple cell receptive field zone of the same polarity decrease their responses while neurons of opposite polarity increase their responses with feedback. This phase-reversed pattern of influence was recently observed in neurophysiology. These results corroborate the idea that predictive feedback is a general coding strategy in the brain.

Sensory uncertainty decoded from visual cortex predicts behavior

Bayesian theories of neural coding propose that sensory uncertainty is represented by a probability distribution encoded in neural population activity, but direct neural evidence supporting this hypothesis is currently lacking. Using fMRI in combination with a generative model-based analysis, we found that probability distributions reflecting sensory uncertainty could reliably be estimated from human visual cortex and, moreover, that observers appeared to use knowledge of this uncertainty in their perceptual decisions.

A review of the mechanisms by which attentional feedback shapes visual selectivity

The glut of information available for the brain to process at any given moment necessitates an efficient attentional system that can ‘pick and choose’ what information receives prioritized processing. A growing body of work, spanning numerous methodologies and species, reveals that one powerful way in which attending to an item separates the wheat from the chaff is by altering a basic response property in the brain: neuronal selectivity. Selectivity is a cornerstone response property, largely dictating our ability to represent and interact with the environment. Although it is likely that selectivity is altered throughout many brain areas, here we focus on how directing attention to an item affects selectivity in the visual system, where this response property is generally more well characterized. First, we review the neural architecture supporting selectivity, and then discuss the various changes that could occur in selectivity for an attended item. In a survey of the literature, spanning neurophysiology, neuroimaging and psychophysics, we reveal that there is general convergence regarding the manner with which selectivity is shaped by attentional feedback. In a nutshell, the literature suggests that the type of changes in selectivity that manifest appears to depend on the type of attention being deployed: whereas directing spatial attention towards an item only alters spatial selectivity, directing feature-based attention can alter the selectivity of attended features.

Dynamic Coding of Signed Quantities in Cortical Feedback Circuits

In the early sensory and motor areas of the cortex, individual neurons transmit information about specific sensory features via a peaked response. This concept has been crystallized as “labeled lines,” to denote that axons communicate the specific properties of their sensory or motor parent cell. Such cells also can be characterized as being polarized, that is, as representing a signed quantity that is either positive or negative. We show in a model simulation that there are two important consequences when learning receptive fields using such signed codings in circuits that subtract different inputs. The first is that, in feedback circuits using labeled lines, such arithmetic operations need to be distributed across multiple distinct pathways. The second consequence is that such pathways must be necessarily dynamic, i.e., that synapses can grow and retract when forming receptive fields. The model monitors the breaking and growing of new circuit connections when their synapses need to change polarities and predicts that the rate of such changes should be inversely correlated with the progress of receptive field formation.

Dual Roles for Spike Signaling in Cortical Neural Populations

A prominent feature of signaling in cortical neurons is that of randomness in the action potential. The output of a typical pyramidal cell can be well fit with a Poisson model, and variations in the Poisson rate repeatedly have been shown to be correlated with stimuli. However while the rate provides a very useful characterization of neural spike data, it may not be the most fundamental description of the signaling code. Recent data showing γ frequency range multi-cell action potential correlations, together with spike timing dependent plasticity, are spurring a re-examination of the classical model, since precise timing codes imply that the generation of spikes is essentially deterministic. Could the observed Poisson randomness and timing determinism reflect two separate modes of communication, or do they somehow derive from a single process? We investigate in a timing-based model whether the apparent incompatibility between these probabilistic and deterministic observations may be resolved by examining how spikes could be used in the underlying neural circuits. The crucial component of this model draws on dual roles for spike signaling. In learning receptive fields from ensembles of inputs, spikes need to behave probabilistically, whereas for fast signaling of individual stimuli, the spikes need to behave deterministically. Our simulations show that this combination is possible if deterministic signals using γ latency coding are probabilistically routed through different members of a cortical cell population at different times. This model exhibits standard features characteristic of Poisson models such as orientation tuning and exponential interval histograms. In addition, it makes testable predictions that follow from the γ latency coding.

Modeling correlated noise is necessary to decode uncertainty

ABSTRACT Brain decoding algorithms form an important part of the arsenal of analysis tools available to neuroscientists, allowing for a more detailed study of the kind of information represented in patterns of cortical activity. While most current decoding algorithms focus on estimating a single, most likely stimulus from the pattern of noisy fMRI responses, the presence of noise causes this estimate to be uncertain. This uncertainty in stimulus estimates is a potentially highly relevant aspect of cortical stimulus processing, and features prominently in Bayesian or probabilistic models of neural coding. Here, we focus on sensory uncertainty and how best to extract this information with fMRI. We first demonstrate in simulations that decoding algorithms that take into account correlated noise between fMRI voxels better recover the amount of uncertainty (quantified as the width of a probability distribution over possible stimuli) associated with the decoded estimate. Furthermore, we show that not all correlated variability should be treated equally, as modeling tuning‐dependent correlations has the greatest impact on decoding performance. Next, we examine actual noise correlations in human visual cortex, and find that shared variability in areas V1‐V3 depends on the tuning properties of fMRI voxels. In line with our simulations, accounting for this shared noise between similarly tuned voxels produces important benefits in decoding. Our findings underscore the importance of accurate noise models in fMRI decoding approaches, and suggest a statistically feasible method to incorporate the most relevant forms of shared noise. HIGHLIGHTSInformation in a cortical response is best characterized by a probability distribution.Accurate decoding of this distribution requires accounting for correlated noise.Correlated noise is most relevant when it mimics a stimulus‐driven response.Including this noise in fMRI forward models enables the estimation of cortical uncertainty.

Perceptual learning increases orientation sampling efficiency

Visual orientation discrimination is known to improve with extensive training, but the mechanisms underlying this behavioral benefit remain poorly understood. Here, we examine the possibility that more reliable task performance could arise in part because observers learn to sample information from a larger portion of the stimulus. We used a variant of the classification image method in combination with a global orientation discrimination task to test whether a change in information sampling underlies training-based benefits in behavioral performance. The results revealed that decreases in orientation thresholds with perceptual learning were accompanied by increases in stimulus sampling. In particular, while stimulus sampling was restricted to the parafoveal, inner portion of the stimulus before training, we observed an outward spread of sampling after training. These results demonstrate that the benefits of perceptual learning may arise, in part, from a strategic increase in the efficiency with which the observer samples information from a visual stimulus.