John-Dylan Haynes

Concurrent TMS-fMRI and Psychophysics Reveal Frontal Influences on Human Retinotopic Visual Cortex

BACKGROUND Regions in human frontal cortex may have modulatory top-down influences on retinotopic visual cortex, but to date neuroimaging methods have only been able to provide indirect evidence for such functional interactions between remote but interconnected brain regions. Here we combined transcranial magnetic stimulation (TMS) with concurrent functional magnetic resonance imaging (fMRI), plus psychophysics, to show that stimulation of the right human frontal eye-field (FEF) produced a characteristic topographic pattern of activity changes in retinotopic visual areas V1-V4, with functional consequences for visual perception. RESULTS FEF TMS led to activity increases for retinotopic representations of the peripheral visual field, but to activity decreases for the central field, in areas V1-V4. These frontal influences on visual cortex occurred in a top-down manner, independently of visual input. TMS of a control site (vertex) did not elicit such visual modulations, and saccades, blinks, or pupil dilation could not account for our results. Finally, the effects of FEF TMS on activity in retinotopic visual cortex led to a behavioral prediction that we confirmed psychophysically by showing that TMS of the frontal site (again compared with vertex) enhanced perceived contrast for peripheral relative to central visual stimuli. CONCLUSIONS Our results provide causal evidence that circuits originating in the human FEF can modulate activity in retinotopic visual cortex, in a manner that differentiates the central and peripheral visual field, with functional consequences for perception. More generally, our study illustrates how the new approach of concurrent TMS-fMRI can now reveal causal interactions between remote but interconnected areas of the human brain.

Sound alters activity in human V1 in association with illusory visual perception

When a single brief visual flash is accompanied by two auditory bleeps, it is frequently perceived incorrectly as two flashes. Here, we used high field functional MRI in humans to examine the neural basis of this multisensory perceptual illusion. We show that activity in retinotopic visual cortex is increased by the presence of concurrent auditory stimulation, irrespective of any illusory perception. However, when concurrent auditory stimulation gave rise to illusory visual perception, activity in V1 was enhanced, despite auditory and visual stimulation being unchanged. These findings confirm that responses in human V1 can be altered by sound and show that they reflect subjective perception rather than the physically present visual stimulus. Moreover, as the right superior temporal sulcus and superior colliculus were also activated by illusory visual perception, together with V1, they provide a potential neural substrate for the generation of this multisensory illusion.

Odor quality coding and categorization in human posterior piriform cortex

Efficient recognition of odorous objects universally shapes animal behavior and is crucial for survival. To distinguish kin from nonkin, mate from nonmate and food from nonfood, organisms must be able to create meaningful perceptual representations of odor qualities and categories. It is currently unknown where and in what form the brain encodes information about odor quality. By combining functional magnetic resonance imaging (fMRI) with multivariate (pattern-based) techniques, we found that spatially distributed ensemble activity in human posterior piriform cortex (PPC) coincides with perceptual ratings of odor quality, such that odorants with more (or less) similar fMRI patterns were perceived as more (or less) alike. We did not observe these effects in anterior piriform cortex, amygdala or orbitofrontal cortex, indicating that ensemble coding of odor categorical perception is regionally specific for PPC. These findings substantiate theoretical models emphasizing the importance of distributed piriform templates for the perceptual reconstruction of odor object quality.

Neural Responses to Unattended Products Predict Later Consumer Choices

Imagine you are standing at a street with heavy traffic watching someone on the other side of the road. Do you think your brain is implicitly registering your willingness to buy any of the cars passing by outside your focus of attention? To address this question, we measured brain responses to consumer products (cars) in two experimental groups using functional magnetic resonance imaging. Participants in the first group (high attention) were instructed to closely attend to the products and to rate their attractiveness. Participants in the second group (low attention) were distracted from products and their attention was directed elsewhere. After scanning, participants were asked to state their willingness to buy each product. During the acquisition of neural data, participants were not aware that consumer choices regarding these cars would subsequently be required. Multivariate decoding was then applied to assess the choice-related predictive information encoded in the brain during product exposure in both conditions. Distributed activation patterns in the insula and the medial prefrontal cortex were found to reliably encode subsequent choices in both the high and the low attention group. Importantly, consumer choices could be predicted equally well in the low attention as in the high attention group. This suggests that neural evaluation of products and associated choice-related processing does not necessarily depend on attentional processing of available items. Overall, the present findings emphasize the potential of implicit, automatic processes in guiding even important and complex decisions.

Flow of affective information between communicating brains.

When people interact, affective information is transmitted between their brains. Modern imaging techniques permit to investigate the dynamics of this brain-to-brain transfer of information. Here, we used information-based functional magnetic resonance imaging (fMRI) to investigate the flow of affective information between the brains of senders and perceivers engaged in ongoing facial communication of affect. We found that the level of neural activity within a distributed network of the perceivers brain can be successfully predicted from the neural activity in the same network in the senders brain, depending on the affect that is currently being communicated. Furthermore, there was a temporal succession in the flow of affective information from the senders brain to the perceivers brain, with information in the perceivers brain being significantly delayed relative to information in the senders brain. This delay decreased over time, possibly reflecting some ‘tuning in’ of the perceiver with the sender. Our data support current theories of intersubjectivity by providing direct evidence that during ongoing facial communication a ‘shared space’ of affect is successively built up between senders and perceivers of affective facial signals.

Primary visual cortex activation on the path of apparent motion is mediated by feedback from hMT+/V5

Apparent motion (AM) is the illusory perception of real motion created when two spatially distinct stationary visual objects are presented in alternating sequence. In common with many other illusory percepts, activation during AM can be identified in unstimulated regions of V1 representing the illusory motion path. However, little is known about the mechanisms underlying such activation and its relationship with motion-sensitive area hMT+/V5. Using fMRI and a novel AM stimulus, we replicated previous findings showing a correlate of the perceived AM path in V1. To more closely characterize the mechanisms underlying these activations, we performed analyses of effective connectivity and found that the AM-induced activations on the illusory AM path were associated with enhanced feedback (but not feedforward) connectivity from hMT+/V5. These findings provide for the first time evidence for the involvement of cortico-cortical coupling in generating an illusory percept of AM. They therefore emphasize the role of recurrent processing between visual cortical areas in human perceptual awareness.

Decoding different roles for vmPFC and dlPFC in multi-attribute decision making

In everyday life, successful decision making requires precise representations of expected values. However, for most behavioral options more than one attribute can be relevant in order to predict the expected reward. Thus, to make good or even optimal choices the reward predictions of multiple attributes need to be integrated into a combined expected value. Importantly, the individual attributes of such multi-attribute objects can agree or disagree in their reward prediction. Here we address where the brain encodes the combined reward prediction (averaged across attributes) and where it encodes the variability of the value predictions of the individual attributes. We acquired fMRI data while subjects performed a task in which they had to integrate reward predictions from multiple attributes into a combined value. Using time-resolved pattern recognition techniques (support vector regression) we find that (1) the combined value is encoded in distributed fMRI patterns in the ventromedial prefrontal cortex (vmPFC) and that (2) the variability of value predictions of the individual attributes is encoded in the dorsolateral prefrontal cortex (dlPFC). The combined value could be used to guide choices, whereas the variability of the value predictions of individual attributes indicates an ambiguity that results in an increased difficulty of the value-integration. These results demonstrate that the different features defining multi-attribute objects are encoded in non-overlapping brain regions and therefore suggest different roles for vmPFC and dlPFC in multi-attribute decision making.

Encoding the identity and location of objects in human LOC

We are able to recognize objects independent of their location in the visual field. At the same time, we also keep track of the location of objects to orient ourselves and to interact with the environment. The lateral occipital complex (LOC) has been suggested as the prime cortical region for representation of object identity. However, the extent to which LOC also represents object location has remained debated. In this study we used high-resolution fMRI in combination with multivoxel pattern classification to investigate the cortical encoding of three object exemplars from four different categories presented in two different locations. This approach allowed us to study location-tolerant object information and object-tolerant location information in LOC, both at the level of categories and exemplars. We found evidence for both location-tolerant object information and object-tolerant location information in LOC at the level of categories and exemplars. Our results further highlight the mixing of identity and location information in the ventral visual pathway.

Blinking suppresses the neural response to unchanging retinal stimulation

Blinks profoundly interrupt visual input but are rarely noticed, perhaps because of blink suppression, a visual-sensitivity loss that begins immediately prior to blink onset. Blink suppression is thought to result from an extra-retinal signal that is associated with the blink motor command and may act to attenuate the sensory consequences of the motor action. However, the neural mechanisms underlying this phenomenon remain unclear. They are challenging to study because any brain-activity changes resulting from an extra-retinal signal associated with the blink motor command are potentially masked by profound neural-activity changes caused by the retinal-illumination reduction that results from occlusion of the pupil by the eyelid. Here, we distinguished direct top-down effects of blink-associated motor signals on cortical activity from purely mechanical or optical effects of blinking on visual input by combining pupil-independent retinal stimulation with functional MRI (fMRI) in humans. Even though retinal illumination was kept constant during blinks, we found that blinking nevertheless suppressed activity in visual cortex and in areas of parietal and prefrontal cortex previously associated with awareness of environmental change. Our findings demonstrate active top-down modulation of visual processing during blinking, suggesting a possible mechanism by which blinks go unnoticed.

Compositionality of Rule Representations in Human Prefrontal Cortex

Rules are widely used in everyday life to organize actions and thoughts in accordance with our internal goals. At the simplest level, single rules can be used to link individual sensory stimuli to their appropriate responses. However, most tasks are more complex and require the concurrent application of multiple rules. Experiments on humans and monkeys have shown the involvement of a frontoparietal network in rule representation. Yet, a fundamental issue still needs to be clarified: Is the neural representation of multiple rules compositional, that is, built on the neural representation of their simple constituent rules? Subjects were asked to remember and apply either simple or compound rules. Multivariate decoding analyses were applied to functional magnetic resonance imaging data. Both ventrolateral frontal and lateral parietal cortex were involved in compound representation. Most importantly, we were able to decode the compound rules by training classifiers only on the simple rules they were composed of. This shows that the code used to store rule information in prefrontal cortex is compositional. Compositional coding in rule representation suggests that it might be possible to decode other complex action plans by learning the neural patterns of the known composing elements.