David J. Heeger

Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1

The linear transform model of functional magnetic resonance imaging (fMRI) hypothesizes that fMRI responses are proportional to local average neural activity averaged over a period of time. This work reports results from three empirical tests that support this hypothesis. First, fMRI responses in human primary visual cortex (V1) depend separably on stimulus timing and stimulus contrast. Second, responses to long-duration stimuli can be predicted from responses to shorter duration stimuli. Third, the noise in the fMRI data is independent of stimulus contrast and temporal period. Although these tests can not prove the correctness of the linear transform model, they might have been used to reject the model. Because the linear transform model is consistent with our data, we proceeded to estimate the temporal fMRI impulse–response function and the underlying (presumably neural) contrast–response function of human V1.

Normalization of cell responses in cat striate cortex.

Simple cells in the striate cortex have been depicted as half-wave-rectified linear operators. Complex cells have been depicted as energy mechanisms, constructed from the squared sum of the outputs of quadrature pairs of linear operators. However, the linear/energy model falls short of a complete explanation of striate cell responses. In this paper, a modified version of the linear/energy model is presented in which striate cells mutually inhibit one another, effectively normalizing their responses with respect to stimulus contrast. This paper reviews experimental measurements of striate cell responses, and shows that the new model explains a significantly larger body of physiological data.

Shiftable multiscale transforms

One of the major drawbacks of orthogonal wavelet transforms is their lack of translation invariance: the content of wavelet subbands is unstable under translations of the input signal. Wavelet transforms are also unstable with respect to dilations of the input signal and, in two dimensions, rotations of the input signal. The authors formalize these problems by defining a type of translation invariance called shiftability. In the spatial domain, shiftability corresponds to a lack of aliasing; thus, the conditions under which the property holds are specified by the sampling theorem. Shiftability may also be applied in the context of other domains, particularly orientation and scale. Jointly shiftable transforms that are simultaneously shiftable in more than one domain are explored. Two examples of jointly shiftable transforms are designed and implemented: a 1-D transform that is jointly shiftable in position and scale, and a 2-D transform that is jointly shiftable in position and orientation. The usefulness of these image representations for scale-space analysis, stereo disparity measurement, and image enhancement is demonstrated. >

A model of neuronal responses in visual area MT

Electrophysiological studies indicate that neurons in the middle temporal (MT) area of the primate brain are selective for the velocity of visual stimuli. This paper describes a computational model of MT physiology, in which local image velocities are represented via the distribution of MT neuronal responses. The computation is performed in two stages, corresponding to neurons in cortical areas V1 and MT. Each stage computes a weighted linear sum of inputs, followed by rectification and divisive normalization. V1 receptive field weights are designed for orientation and direction selectivity. MT receptive field weights are designed for velocity (both speed and direction) selectivity. The paper includes computational simulations accounting for a wide range of physiological data, and describes experiments that could be used to further test and refine the model.

The Normalization Model of Attention

Attention has been found to have a wide variety of effects on the responses of neurons in visual cortex. We describe a model of attention that exhibits each of these different forms of attentional modulation, depending on the stimulus conditions and the spread (or selectivity) of the attention field in the model. The model helps reconcile proposals that have been taken to represent alternative theories of attention. We argue that the variety and complexity of the results reported in the literature emerge from the variety of empirical protocols that were used, such that the results observed in any one experiment depended on the stimulus conditions and the subjects attentional strategy, a notion that we define precisely in terms of the attention field in the model, but that has not typically been completely under experimental control.

Normalization as a canonical neural computation

There is increasing evidence that the brain relies on a set of canonical neural computations, repeating them across brain regions and modalities to apply similar operations to different problems. A promising candidate for such a computation is normalization, in which the responses of neurons are divided by a common factor that typically includes the summed activity of a pool of neurons. Normalization was developed to explain responses in the primary visual cortex and is now thought to operate throughout the visual system, and in many other sensory modalities and brain regions. Normalization may underlie operations such as the representation of odours, the modulatory effects of visual attention, the encoding of value and the integration of multisensory information. Its presence in such a diversity of neural systems in multiple species, from invertebrates to mammals, suggests that it serves as a canonical neural computation.

Optical flow using spatiotemporal filters

A model is presented, consonant with current views regarding the neurophysiology and psychophysics of motion perception, that combines the outputs of a set of spatiotemporal motion-energy filters to estimate image velocity. A parallel implementation computes a distributed representation of image velocity. A measure of image-flow uncertainty is formulated; preliminary results indicate that this uncertainty measure may be used to recognize ambiguity due to the aperture problem. The model appears to deal with the aperture problem as well as the human visual system since it extracts the correct velocity for some patterns that have large differences in contrast at different spatial orientations.

Activity in primary visual cortex predicts performance in a visual detection task

Visual attention can affect both neural activity and behavior in humans. To quantify possible links between the two, we measured activity in early visual cortex (V1, V2 and V3) during a challenging pattern-detection task. Activity was dominated by a large response that was independent of the presence or absence of the stimulus pattern. The measured activity quantitatively predicted the subjects pattern-detection performance: when activity was greater, the subject was more likely to correctly discern the presence or absence of the pattern. This stimulus-independent activity had several characteristics of visual attention, suggesting that attentional mechanisms modulate activity in early visual cortex, and that this attention-related activity strongly influences performance.

Perceptual image distortion

In this paper, we present a perceptual distortion measure that predicts image integrity far better than mean-squared error. This perceptual distortion measure is based on a model of human visual processing that fits empirical measurements of: (1) the response properties of neurons in the primary visual cortex, and (2) the psychophysics of spatial pattern detection. We also illustrate the usefulness of the model in measuring perceptual distortion in real images.

Two Retinotopic Visual Areas in Human Lateral Occipital Cortex

We describe two visual field maps, lateral occipital areas 1 (LO1) and 2 (LO2), in the human lateral occipital cortex between the dorsal part of visual area V3 and visual area V5/MT+. Each map contained a topographic representation of the contralateral visual hemifield. The eccentricity representations were shared with V1/V2/V3. The polar angle representation in LO1 extended from the lower vertical meridian (at the boundary with dorsal V3) through the horizontal to the upper vertical meridian (at the boundary with LO2). The polar angle representation in LO2 was the mirror-reversal of that in LO1. LO1 and LO2 overlapped with the posterior part of the object-selective lateral occipital complex and the kinetic occipital region (KO). The retinotopy and functional properties of LO1 and LO2 suggest that they correspond to two new human visual areas, which lack exact homologues in macaque visual cortex. The topography, stimulus selectivity, and anatomical location of LO1 and LO2 indicate that they integrate shape information from multiple visual submodalities in retinotopic coordinates.