Elliot Freeman

Concurrent TMS-fMRI and Psychophysics Reveal Frontal Influences on Human Retinotopic Visual Cortex

BACKGROUND Regions in human frontal cortex may have modulatory top-down influences on retinotopic visual cortex, but to date neuroimaging methods have only been able to provide indirect evidence for such functional interactions between remote but interconnected brain regions. Here we combined transcranial magnetic stimulation (TMS) with concurrent functional magnetic resonance imaging (fMRI), plus psychophysics, to show that stimulation of the right human frontal eye-field (FEF) produced a characteristic topographic pattern of activity changes in retinotopic visual areas V1-V4, with functional consequences for visual perception. RESULTS FEF TMS led to activity increases for retinotopic representations of the peripheral visual field, but to activity decreases for the central field, in areas V1-V4. These frontal influences on visual cortex occurred in a top-down manner, independently of visual input. TMS of a control site (vertex) did not elicit such visual modulations, and saccades, blinks, or pupil dilation could not account for our results. Finally, the effects of FEF TMS on activity in retinotopic visual cortex led to a behavioral prediction that we confirmed psychophysically by showing that TMS of the frontal site (again compared with vertex) enhanced perceived contrast for peripheral relative to central visual stimuli. CONCLUSIONS Our results provide causal evidence that circuits originating in the human FEF can modulate activity in retinotopic visual cortex, in a manner that differentiates the central and peripheral visual field, with functional consequences for perception. More generally, our study illustrates how the new approach of concurrent TMS-fMRI can now reveal causal interactions between remote but interconnected areas of the human brain.

Lateral interactions between targets and flankers in low-level vision depend on attention to the flankers

Detection of an oriented visual target can be facilitated by collinear visual flankers. Such lateral interactions are thought to reflect integrative processes in low-level vision. In past studies, the flankers were task-irrelevant, and were typically assumed to be unattended. Here we manipulated attention to the flankers directly, by requiring observers to judge the relative alignment of two flankers while ignoring a second flanker-pair. Under identical stimulus conditions, attended flankers produced typical lateral interactions, but ignored flankers did not. These data show that lateral interactions can depend on attention to the flanking context, revealing the functional consequences of attentional modulation in low-level vision.

Direction of Visual Apparent Motion Driven Solely by Timing of a Static Sound

In temporal ventriloquism, auditory events can illusorily attract perceived timing of a visual onset [1-3]. We investigated whether timing of a static sound can also influence spatio-temporal processing of visual apparent motion, induced here by visual bars alternating between opposite hemifields. Perceived direction typically depends on the relative interval in timing between visual left-right and right-left flashes (e.g., rightwards motion dominating when left-to-right interflash intervals are shortest [4]). In our new multisensory condition, interflash intervals were equal, but auditory beeps could slightly lag the right flash, yet slightly lead the left flash, or vice versa. This auditory timing strongly influenced perceived visual motion direction, despite providing no spatial auditory motion signal whatsoever. Moreover, prolonged adaptation to such auditorily driven apparent motion produced a robust visual motion aftereffect in the opposite direction, when measured in subsequent silence. Control experiments argued against accounts in terms of possible auditory grouping, or possible attention capture. We suggest that the motion arises because the sounds change perceived visual timing, as we separately confirmed. Our results provide a new demonstration of multisensory influences on sensory-specific perception [5], with timing of a static sound influencing spatio-temporal processing of visual motion direction.

Dissociations and Interactions between Time, Numerosity and Space Processing.

This study investigated time, numerosity and space processing in a patient (CB) with a right hemisphere lesion. We tested whether these magnitude dimensions share a common magnitude system or whether they are processed by dimension-specific magnitude systems. Five experimental tasks were used: Tasks 1–3 assessed time and numerosity independently and time and numerosity jointly. Tasks 4 and 5 investigated space processing independently and space and numbers jointly. Patient CB was impaired at estimating time and at discriminating between temporal intervals, his errors being underestimations. In contrast, his ability to process numbers and space was normal. A unidirectional interaction between numbers and time was found in both the patient and the control subjects. Strikingly, small numbers were perceived as lasting shorter and large numbers as lasting longer. In contrast, number processing was not affected by time, i.e. short durations did not result in perceiving fewer numbers and long durations in perceiving more numbers. Numbers and space also interacted, with small numbers answered faster when presented on the left side of space, and the reverse for large numbers. Our results demonstrate that time processing can be selectively impaired. This suggests that mechanisms specific for time processing may be partially independent from those involved in processing numbers and space. However, the interaction between numbers and time and between numbers and space also suggests that although independent, there maybe some overlap between time, numbers and space. These data suggest a partly shared mechanism between time, numbers and space which may be involved in magnitude processing or may be recruited to perform cognitive operations on magnitude dimensions.

Top-Down Modulation of Lateral Interactions in Early Vision: Does Attention Affect Integration of the Whole or Just Perception of the Parts?

Attention can modulate sensitivity to local stimuli in early vision. But, can attention also modulate integration of local stimuli into global visual patterns? We recently measured effects of attention on the phenomenon of lateral interactions between collinear elements, commonly thought to reflect long-range mechanisms in early visual cortex underlying contour integration. We showed improved detection of low-contrast central Gabor targets in the context of collinear flankers, but only when the collinear flankers were attended for a secondary task rather than ignored in favor of an orthogonal flanker pair. Here, we contrast two hypotheses for how attention might modulate flanker influences on the target: by changing just local sensitivity to the flankers themselves (flanker-modulation-only hypothesis), or by weighting integrative connections between flanker and target (connection-weighting hypothesis). Modeled on the known nonlinear dependence of target visibility on collinear flanker contrast, the first hypothesis predicts that an increase in physical flanker contrast should readily offset any reduction in their effective contrast when ignored, thus eliminating attentional modulation. Conversely, the second hypothesis predicts that attentional modulation should persist even for the highest flanker contrasts. Our results showed the latter outcome and indicated that attention modulates flanker-target integration, rather than just processing of local flanker elements.

Numbers and time doubly dissociate

The magnitude dimensions of number, time and space have been suggested to share some common magnitude processing, which may imply symmetric interaction among dimensions. Here we challenge these suggestions by presenting a double dissociation between two neuropsychological patients with left (JT) and right (CB) parietal lesions and selective impairment of number and time processing respectively. Both patients showed an influence of task-irrelevant number stimuli on time but not space processing. In JT otherwise preserved time processing was severely impaired in the mere presence of task-irrelevant numbers, which themselves could not be processed accurately. In CB, impaired temporal estimation was influenced by preserved number processing: small numbers made (already grossly underestimated) time intervals appear even shorter relative to large numbers. However, numerical estimation was not influenced by time in healthy controls and in both patients. This new double dissociation between number and time processing and the asymmetric interaction of number on time: (1) provides further support to the hypothesis of a partly shared magnitude system among dimensions, instead of the proposal of a single, fully shared system or of independent magnitude systems which would not explain dissociations or interactions among dimensions; (2) may be explained in terms of a stable hierarchy of dimensions, with numbers being the strongest.

The middle house or the middle floor: Bisecting horizontal and vertical mental number lines in neglect

This study explores the processing of mental number lines and physical lines in five patients with left unilateral neglect. Three tasks were used: mental number bisection (‘report the middle number between two numbers’), physical line bisection (‘mark the middle of a line’), and a landmark task (‘is the mark on the line to the left/right or higher/lower than the middle of the line?’). We manipulated the number line orientation purely by task instruction: neglect patients were told that the number-pairs represented either houses on a street (horizontal condition) or floors in a building (vertical condition). We also manipulated physical line orientation for comparison. All five neglect patients showed a rightward bias for horizontally oriented physical and number lines (e.g. saying ‘five’ is the middle house number between ‘two’ and ‘six’). Only three of these patients also showed an upward bias for vertically oriented number lines. The remaining two patients did not show any bias in processing vertical lines. Our results suggest that: (1) horizontal and vertical neglect can associate or dissociate among different patients; (2) bisecting number lines operates on internal horizontal and vertical representations possibly analogous to horizontal and vertical physical lines; (3) at least partially independent mechanisms may be involved in processing horizontal and vertical number lines.

Looming sounds enhance orientation sensitivity for visual stimuli on the same side as such sounds

Several recent multisensory studies show that sounds can influence visual processing. Some visual judgments can be enhanced for visual stimuli near a sound occurring around the same time. A recent TMS study (Romei et al. 2009) indicates looming sounds might influence visual cortex particularly strongly. But unlike most previous behavioral studies of possible audio–visual exogenous effects, TMS phosphene thresholds rather than judgments of external visual stimuli were measured. Moreover, the visual hemifield assessed relative to the hemifield of the sound was not varied. Here, we compared the impact of looming sounds to receding or “static” sounds, using auditory stimuli adapted from Romei et al. (2009), but now assessing any influence on visual orientation discrimination for Gabor patches (well-known to involve early visual cortex) when appearing in the same hemifield as the sound or on the opposite side. The looming sounds that were effective in Romei et al. (2009) enhanced visual orientation sensitivity (d′) here on the side of the sound, but not for the opposite hemifield. This crossmodal, spatially specific effect was stronger for looming than receding or static sounds. Similarly to Romei et al. (2009), the differential effect for looming sounds was eliminated when using white noise rather than structured sounds. Our new results show that looming structured sounds can specifically benefit visual orientation sensitivity in the hemifield of the sound, even when the sound provides no information about visual orientation itself.

The role of right and left parietal lobes in the conceptual processing of numbers

Neuropsychological and functional imaging studies have associated the conceptual processing of numbers with bilateral parietal regions (including intraparietal sulcus). However, the processes driving these effects remain unclear because both left and right posterior parietal regions are activated by many other conceptual, perceptual, attention, and response-selection processes. To dissociate parietal activation that is number-selective from parietal activation related to other stimulus or response-selection processes, we used fMRI to compare numbers and object names during exactly the same conceptual and perceptual tasks while factoring out activations correlating with response times. We found that right parietal activation was higher for conceptual decisions on numbers relative to the same tasks on object names, even when response time effects were fully factored out. In contrast, left parietal activation for numbers was equally involved in conceptual processing of object names. We suggest that left parietal activation for numbers reflects a range of processes, including the retrieval of learnt facts that are also involved in conceptual decisions on object names. In contrast, number selectivity in right parietal cortex reflects processes that are more involved in conceptual decisions on numbers than object names. Our results generate a new set of hypotheses that have implications for the design of future behavioral and functional imaging studies of patients with left and right parietal damage.

Interactions between attention and perceptual grouping in human visual cortex

Freeman et al. demonstrated that detection sensitivity for a low contrast Gabor stimulus improved in the presence of flanking, collinearly oriented grating stimuli, but only when observers attended to them. By recording visual event-related potentials (ERPs) elicited by a Gabor stimulus, we investigated whether this contextual cueing effect involves changes in the short-latency afferent visual signal from V1 that have a stimulus onset latency between 60 and 80 ms and/or longer-latency changes from visual cortex. Under dual-task conditions, the subjects performed contrast discrimination for a central Gabor and an orientation judgment for a pre-specified subset of the flanking Gabors. On random trials, the central Gabor could be collinearly or orthogonally oriented with respect to the attended flankers. Subjects showed improvements in discriminating the contrast of the central grating when it was oriented collinearly with the attended flankers. The ERP difference between attending to collinear versus orthogonal flankers manifested as a positive polarity response at occipital electrodes with a latency of 180-250 ms after stimulus onset. No shorter-latency contextual cueing differences were observed in the ERPs. The ERP latency profile of the contextual cueing effect argues against the hypothesis that short-latency afferent activity from V1 is the stage of processing at which attention can influence neuronal lateral interactions. However, the scalp voltage distribution of the longer-latency contextual cueing effect is similar to the one generated by the early phasic stimulus onset activity from V1. These findings leave open the possibility that V1 is involved in the attentional modulation of lateral interactions but that this has a longer time course, likely being mediated by re-afferent inputs from later stages of the visual pathway.