Lars Muckli

The constructive nature of vision: direct evidence from functional magnetic resonance imaging studies of apparent motion and motion imagery

Echoplanar functional magnetic resonance imaging was used to monitor activation changes of brain areas while subjects viewed apparent motion stimuli and while they were engaged in motion imagery. Human cortical areas MT (V5) and MST were the first areas of the ‘dorsal’ processing stream which responded with a clear increase in signal intensity to apparent motion stimuli as compared with flickering control conditions. Apparent motion of figures defined by illusory contours evoked greater activation in V2 and MT/MST than appropriate control conditions. Several areas of the dorsal pathway (V3A, MT/MST, areas in the inferior and superior parietal lobule) as well as prefrontal areas including FEF and BA 9/46 responded strongly when subjects merely imagined moving stimuli which they had seen several seconds before. The activation during motion imagery increased with the synaptic distance of an area from V1 along the dorsal processing stream. Area MT/MST was selectively activated during motion imagery but not during a static imagery control condition. The comparison between the results obtained with objective motion, apparent motion and imagined motion provides further insights into a complex cortical network of motion‐sensitive areas driven by bottom‐up and top‐down neural processes.

Cortical capacity constraints for visual working memory: dissociation of fMRI load effects in a fronto-parietal network

Working memory (WM) capacity limitations and their neurophysiological correlates are of special relevance for the understanding of higher cognitive functions. Evidence from behavioral studies suggests that restricted attentional resources contribute to these capacity limitations. In an event-related functional magnetic resonance imaging (fMRI) study, we probed the capacity of the human visual WM system for up to four complex nonnatural objects using a delayed discrimination task. A number of prefrontal and parietal areas bilaterally showed increased blood oxygen level-dependent activity, relative to baseline, throughout the task when more than one object had to be held in memory. Monotonic increases in response to memory load were observed bilaterally in the dorsolateral prefrontal cortex (DLPFC) and the presupplementary motor area (pre-SMA). Conversely, activity in the frontal eye fields (FEFs) and in areas along the intraparietal sulcus (IPS) peaked when subjects had to maintain only two or three objects and decreased in the highest load condition. This dissociation of memory load effects on cortical activity suggests that the cognitive operations subserved by the IPS and FEF, which are most likely related to attention, fail to support visual WM when the capacity limit is approached. The correlation of brain activity with performance implies that only the operations performed by the DLPFC and pre-SMA, which support an integrated representation of visual information, helped subjects to maintain a reasonable level of performance in the highest load condition. These results indicate that at least two distinct cortical subsystems are recruited for visual WM, and that their interplay changes when the capacity limit is reached.

Stimulus Predictability Reduces Responses in Primary Visual Cortex

In this functional magnetic resonance imaging study we tested whether the predictability of stimuli affects responses in primary visual cortex (V1). The results of this study indicate that visual stimuli evoke smaller responses in V1 when their onset or motion direction can be predicted from the dynamics of surrounding illusory motion. We conclude from this finding that the human brain anticipates forthcoming sensory input that allows predictable visual stimuli to be processed with less neural activation at early stages of cortical processing.

Primary Visual Cortex Activity along the Apparent-Motion Trace Reflects Illusory Perception

The illusion of apparent motion can be induced when visual stimuli are successively presented at different locations. It has been shown in previous studies that motion-sensitive regions in extrastriate cortex are relevant for the processing of apparent motion, but it is unclear whether primary visual cortex (V1) is also involved in the representation of the illusory motion path. We investigated, in human subjects, apparent-motion-related activity in patches of V1 representing locations along the path of illusory stimulus motion using functional magnetic resonance imaging. Here we show that apparent motion caused a blood-oxygenation-level-dependent response along the V1 representations of the apparent-motion path, including regions that were not directly activated by the apparent-motion-inducing stimuli. This response was unaltered when participants had to perform an attention-demanding task that diverted their attention away from the stimulus. With a bistable motion quartet, we confirmed that the activity was related to the conscious perception of movement. Our data suggest that V1 is part of the network that represents the illusory path of apparent motion. The activation in V1 can be explained either by lateral interactions within V1 or by feedback mechanisms from higher visual areas, especially the motion-sensitive human MT/V5 complex.

Object Familiarity and Semantic Congruency Modulate Responses in Cortical Audiovisual Integration Areas

The cortical integration of auditory and visual features is crucial for efficient object recognition. Previous studies have shown that audiovisual (AV) integration is affected by where and when auditory and visual features occur. However, because relatively little is known about the impact of what is integrated, we here investigated the impact of semantic congruency and object familiarity on the neural correlates of AV integration. We used functional magnetic resonance imaging to identify regions involved in the integration of both (congruent and incongruent) familiar animal sounds and images and of arbitrary combinations of unfamiliar artificial sounds and object images. Unfamiliar object images and sounds were integrated in the inferior frontal cortex (IFC), possibly reflecting learning of novel AV associations. Integration of familiar, but semantically incongruent combinations also correlated with IFC activation and additionally involved the posterior superior temporal sulcus (pSTS). For highly familiar semantically congruent AV pairings, we again found AV integration effects in pSTS and additionally in superior temporal gyrus. These findings demonstrate that the neural correlates of object-related AV integration reflect both semantic congruency and familiarity of the integrated sounds and images.

The temporal characteristics of motion processing in hMT/V5+: combining fMRI and neuronavigated TMS

Functional imaging has demonstrated the specific involvement of the human middle-temporal complex (hMT/V5+) during processing of moving stimuli. Some studies applied transcranial magnetic stimulation (TMS) to investigate the causal relevance of hMT/V5+ for motion perception. Although the studies used similar visual stimuli and TMS parameters, the critical time point of functionally relevant hMT/V5+ activity differed by 100 ms and more. The present study aimed to elucidate further the temporal characteristics of motion processing in hMT/V5+ by investigating all critical time windows currently debated in the literature. In contrast to previous studies, we used TMS neuronavigation based on individual fMRI results of five participants to target hMT/V5+, applying single-pulse TMS at 24 different time windows (-50 till +200 ms relative to stimulus onset). We revealed that TMS significantly impaired motion perception when applied over hMT/V5+ at 40 to 30 ms before as well as 130 to 150 ms after onset of the moving stimuli. While the late effective time window conforms to results from previous experiments, we did not find evidence for an early time window around 0 ms that has been reported in other studies. Our neuronavigation approach enabled us to quantify the interindividual variance in the exact location of hMT/V5+ and the respective TMS target position on the skull of the participants. Considering that shifting the TMS coil position only by a few millimeters can already lead to a complete loss of TMS effects, our study clearly demonstrates the utility of neuronavigated TMS when investigating specific neuronal effects as in the case of motion processing.

Contextual Feedback to Superficial Layers of V1.

Summary Neuronal cortical circuitry comprises feedforward, lateral, and feedback projections, each of which terminates in distinct cortical layers [1–3]. In sensory systems, feedforward processing transmits signals from the external world into the cortex, whereas feedback pathways signal the brain’s inference of the world [4–11]. However, the integration of feedforward, lateral, and feedback inputs within each cortical area impedes the investigation of feedback, and to date, no technique has isolated the feedback of visual scene information in distinct layers of healthy human cortex. We masked feedforward input to a region of V1 cortex and studied the remaining internal processing. Using high-resolution functional brain imaging (0.8 mm3) and multivoxel pattern information techniques, we demonstrate that during normal visual stimulation scene information peaks in mid-layers. Conversely, we found that contextual feedback information peaks in outer, superficial layers. Further, we found that shifting the position of the visual scene surrounding the mask parametrically modulates feedback in superficial layers of V1. Our results reveal the layered cortical organization of external versus internal visual processing streams during perception in healthy human subjects. We provide empirical support for theoretical feedback models such as predictive coding [10, 12] and coherent infomax [13] and reveal the potential of high-resolution fMRI to access internal processing in sub-millimeter human cortex.

Cortical responses to self and others

The extrastriate body area (EBA) is one among the multiple, functionally specialized regions of the human visual cortex exhibiting modulation by body‐related stimuli. Here we investigate whether activation patterns differ for the perception of ones own body and the bodies of others. We used functional magnetic resonance imaging to identify body‐related brain areas and to see how these areas differentiate between images of ones own body and those of others in the absence of facial or motion cues. Whole brain explorative group‐level analysis identified body‐related blood oxygen level dependent (BOLD) signal enhancement in five regions of the right and in one region of the left hemisphere (right: in the extrastriate visual and parietal cortex and in the precentral gyrus, left: in the extrastriate visual cortex). General linear model group‐level random effects analysis of the self–other contrast revealed self‐related responses in the extrastriate and parietal regions in the right hemisphere but also in the right middle frontal gyrus. These results suggest the existence of a cortical network for the extraction of body‐related information and another cortical network for the extraction of self‐related body information. The two networks partially overlap in the right superior and inferior parietal cortices, but are clearly segregated in the extrastriate visual cortex and in the middle frontal gyrus. In addition, we report that the classical EBA is only involved in the analysis of body‐related information but not in the assignment of body identity. The latter appears to be accomplished by a network in right hemisphere comprising the fusiform body area, regions of the superior parietal lobe, the inferior parietal cortex, and the middle frontal gyrus. Hum Brain Mapp, 2009.

Cerebral correlates of impaired grating perception in individual, psychophysically assessed human amblyopes

We investigated neuronal correlates of amblyopic deficits resulting from early onset strabismus or anisometropia by monitoring individual responses in retinotopically mapped cortical visual areas with functional magnetic resonance imaging (fMRI) in eight psychophysically assessed adult amblyopes. In lower visual areas (V1/V2), grating stimuli presented to the normal and the amblyopic eye evoked strong cortical responses, while responses to the amblyopic eye were progressively reduced in higher areas on the central visual pathway (V3a/VP; V4/V8; lateral occipital complex, LOC). Selective reduction for high spatial frequency gratings was especially obvious in LOC. This suggests that transmission of activity from the amblyopic eye is increasingly impaired while it is relayed towards higher processing levels. Elevated responses in parts of areas V1 and V2 to monocular stimulation of the amblyopic eye might be related to the spatial and temporal distortions experienced by some amblyopic subjects.

Decoding Sound and Imagery Content in Early Visual Cortex

Summary Human early visual cortex was traditionally thought to process simple visual features such as orientation, contrast, and spatial frequency via feedforward input from the lateral geniculate nucleus (e.g., [1]). However, the role of nonretinal influence on early visual cortex is so far insufficiently investigated despite much evidence that feedback connections greatly outnumber feedforward connections [2–5]. Here, we explored in five fMRI experiments how information originating from audition and imagery affects the brain activity patterns in early visual cortex in the absence of any feedforward visual stimulation. We show that category-specific information from both complex natural sounds and imagery can be read out from early visual cortex activity in blindfolded participants. The coding of nonretinal information in the activity patterns of early visual cortex is common across actual auditory perception and imagery and may be mediated by higher-level multisensory areas. Furthermore, this coding is robust to mild manipulations of attention and working memory but affected by orthogonal, cognitively demanding visuospatial processing. Crucially, the information fed down to early visual cortex is category specific and generalizes to sound exemplars of the same category, providing evidence for abstract information feedback rather than precise pictorial feedback. Our results suggest that early visual cortex receives nonretinal input from other brain areas when it is generated by auditory perception and/or imagery, and this input carries common abstract information. Our findings are compatible with feedback of predictive information to the earliest visual input level (e.g., [6]), in line with predictive coding models [7–10].