Maria Concetta Morrone

A cortical area that responds specifically to optic flow, revealed by fMRI

The continuously changing optic flow on the retina provides information about direction of heading and about the three-dimensional structure of the environment. Here we use functional magnetic resonance imaging (fMRI) to demonstrate that an area in human cortex responds selectively to components of optic flow, such as circular and radial motion. This area is within the region commonly referrred to as V5/MT complex, but is distinct from the part of this region that responds to translation. The functional properties of these two areas of the V5/MT complex are also different; the response to optic flow was obtained only with changing flow stimuli, whereas response to translation occurred during exposure to continuous motion.

Spatiotopic selectivity of BOLD responses to visual motion in human area MT

Many neurons in the monkey visual extrastriate cortex have receptive fields that are affected by gaze direction. In humans, psychophysical studies suggest that motion signals may be encoded in a spatiotopic fashion. Here we use functional magnetic resonance imaging to study spatial selectivity in the human middle temporal cortex (area MT or V5), an area that is clearly implicated in motion perception. The results show that the response of MT is modulated by gaze direction, generating a spatial selectivity based on screen rather than retinal coordinates. This area could be the neurophysiological substrate of the spatiotopic representation of motion signals.

Auditory dominance over vision in the perception of interval duration.

The “ventriloquist effect” refers to the fact that vision usually dominates hearing in spatial localization, and this has been shown to be consistent with optimal integration of visual and auditory signals (Alais and Burr in Curr Biol 14(3):257–262, 2004). For temporal localization, however, auditory stimuli often “capture” visual stimuli, in what has become known as “temporal ventriloquism”. We examined this quantitatively using a bisection task, confirming that sound does tend to dominate the perceived timing of audio-visual stimuli. The dominance was predicted qualitatively by considering the better temporal localization of audition, but the quantitative fit was less than perfect, with more weight being given to audition than predicted from thresholds. As predicted by optimal cue combination, the temporal localization of audio-visual stimuli was better than for either sense alone.

Spatiotopic coding and remapping in humans

How our perceptual experience of the world remains stable and continuous in the face of continuous rapid eye movements still remains a mystery. This review discusses some recent progress towards understanding the neural and psychophysical processes that accompany these eye movements. We firstly report recent evidence from imaging studies in humans showing that many brain regions are tuned in spatiotopic coordinates, but only for items that are actively attended. We then describe a series of experiments measuring the spatial and temporal phenomena that occur around the time of saccades, and discuss how these could be related to visual stability. Finally, we introduce the concept of the spatio-temporal receptive field to describe the local spatiotopicity exhibited by many neurons when the eyes move.

Color and Luminance Contrasts Attract Independent Attention

Paying attention can improve vision in many ways, including some very basic functions such as contrast discrimination, a task that probably reflects very early levels of visual processing. Electrophysiological, psychophysical, and imaging studies on humans as well as single recordings in monkey show that attention can modulate the neuronal response at an early stage of visual processing, probably by acting on the response gain. Here, we measure incremental contrast thresholds for luminance and color stimuli to derive the contrast response of early neural mechanisms and their modulation by attention. We show that, for both cases, attention improves contrast discrimination, probably by multiplicatively increasing the gain of the neuronal response to contrast. However, the effects of attention are highly specific to the visual modality: concurrent attention to a competing luminance, but not chromatic pattern, greatly impedes luminance contrast discrimination; and attending to a competing chromatic, but not luminance, task impedes color contrast discrimination. Thus, the effects of attention are highly modality specific, implying separate attentional resources for different fundamental visual attributes at early stages of visual processing.

Spatiotopic Coding of BOLD Signal in Human Visual Cortex Depends on Spatial Attention

The neural substrate of the phenomenological experience of a stable visual world remains obscure. One possible mechanism would be to construct spatiotopic neural maps where the response is selective to the position of the stimulus in external space, rather than to retinal eccentricities, but evidence for these maps has been inconsistent. Here we show, with fMRI, that when human subjects perform concomitantly a demanding attentive task on stimuli displayed at the fovea, BOLD responses evoked by moving stimuli irrelevant to the task were mostly tuned in retinotopic coordinates. However, under more unconstrained conditions, where subjects could attend easily to the motion stimuli, BOLD responses were tuned not in retinal but in external coordinates (spatiotopic selectivity) in many visual areas, including MT, MST, LO and V6, agreeing with our previous fMRI study. These results indicate that spatial attention may play an important role in mediating spatiotopic selectivity.

Different attentional resources modulate the gain mechanisms for color and luminance contrast

We used an interference paradigm to investigate whether attention is attribute-specific at early levels of visual processing. We show that the peripheral increment thresholds for luminance contrast deteriorate when the observer is currently performing another luminance (form or contrast) discrimination task in central view, but not when he or she is performing a color discrimination task. Similar results were obtained for color increment thresholds, indicating that the interference is specific to contrast modality. The effects are strong and robust over primary task difficulties and perceptual learning levels. Modeling suggests that attention improves contrast sensitivity by modulating the gain of the neuronal response to contrast. These results suggest that attention is parceled in independent resources for luminance and color contrast.

Spatiotopic neural representations develop slowly across saccades

One of the long-standing unsolved mysteries of visual neuroscience is how the world remains apparently stable in the face of continuous movements of eyes, head and body. Many factors seem to contribute to this stability, including rapid updating mechanisms that temporarily remap the visual input to compensate for the impending saccade. However, there is also a growing body of evidence pointing to more long-lasting spatiotopic neural representations, which remain solid in external rather than retinal coordinates. In this study, we show that these spatiotopic representations take hundreds of milliseconds to build up robustly.

Fusion of Visual and Auditory Stimuli during Saccades: A Bayesian Explanation for Perisaccadic Distortions

Brief stimuli presented near the onset of saccades are grossly mislocalized in space. In this study, we investigated whether the Bayesian hypothesis of optimal sensory fusion could account for the mislocalization. We required subjects to localize visual, auditory, and audiovisual stimuli at the time of saccades (compared with an earlier presented target). During fixation, vision dominates and spatially “captures” the auditory stimulus (the ventriloquist effect). But for perisaccadic presentations, auditory localization becomes more important, so the mislocalized visual stimulus is seen closer to its veridical position. The precision of the bimodal localization (as measured by localization thresholds or just-noticeable difference) was better than either the visual or acoustic stimulus presented in isolation. Both the perceived position of the bimodal stimuli and the improved precision were well predicted by assuming statistically optimal Bayesian-like combination of visual and auditory signals. Furthermore, the time course of localization was well predicted by the Bayesian approach. We present a detailed model that simulates the time-course data, assuming that perceived position is given by the sum of retinal position and a sluggish noisy eye-position signal, obtained by integrating optimally the output of two populations of neural activity: one centered at the current point of gaze, the other centered at the future point of gaze.

Temporal mechanisms of multimodal binding

The simultaneity of signals from different senses—such as vision and audition—is a useful cue for determining whether those signals arose from one environmental source or from more than one. To understand better the sensory mechanisms for assessing simultaneity, we measured the discrimination thresholds for time intervals marked by auditory, visual or auditory–visual stimuli, as a function of the base interval. For all conditions, both unimodal and cross-modal, the thresholds followed a characteristic ‘dipper function’ in which the lowest thresholds occurred when discriminating against a non-zero interval. The base interval yielding the lowest threshold was roughly equal to the threshold for discriminating asynchronous from synchronous presentations. Those lowest thresholds occurred at approximately 5, 15 and 75 ms for auditory, visual and auditory–visual stimuli, respectively. Thus, the mechanisms mediating performance with cross-modal stimuli are considerably slower than the mechanisms mediating performance within a particular sense. We developed a simple model with temporal filters of different time constants and showed that the model produces discrimination functions similar to the ones we observed in humans. Both for processing within a single sense, and for processing across senses, temporal perception is affected by the properties of temporal filters, the outputs of which are used to estimate time offsets, correlations between signals, and more.