Patrik Vuilleumier

Effects of attention and emotion on face processing in the human brain: an event-related fMRI study

We used event-related fMRI to assess whether brain responses to fearful versus neutral faces are modulated by spatial attention. Subjects performed a demanding matching task for pairs of stimuli at prespecified locations, in the presence of task-irrelevant stimuli at other locations. Faces or houses unpredictably appeared at the relevant or irrelevant locations, while the faces had either fearful or neutral expressions. Activation of fusiform gyri by faces was strongly affected by attentional condition, but the left amygdala response to fearful faces was not. Right fusiform activity was greater for fearful than neutral faces, independently of the attention effect on this region. These results reveal differential influences on face processing from attention and emotion, with the amygdala response to threat-related expressions unaffected by a manipulation of attention that strongly modulates the fusiform response to faces.

Distinct spatial frequency sensitivities for processing faces and emotional expressions

High and low spatial frequency information in visual images is processed by distinct neural channels. Using event-related functional magnetic resonance imaging (fMRI) in humans, we show dissociable roles of such visual channels for processing faces and emotional fearful expressions. Neural responses in fusiform cortex, and effects of repeating the same face identity upon fusiform activity, were greater with intact or high-spatial-frequency face stimuli than with low-frequency faces, regardless of emotional expression. In contrast, amygdala responses to fearful expressions were greater for intact or low-frequency faces than for high-frequency faces. An activation of pulvinar and superior colliculus by fearful expressions occurred specifically with low-frequency faces, suggesting that these subcortical pathways may provide coarse fear-related inputs to the amygdala.

Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging

Brain imaging studies in humans have shown that face processing in several areas is modulated by the affective significance of faces, particularly with fearful expressions, but also with other social signals such gaze direction. Here we review haemodynamic and electrical neuroimaging results indicating that activity in the face-selective fusiform cortex may be enhanced by emotional (fearful) expressions, without explicit voluntary control, and presumably through direct feedback connections from the amygdala. fMRI studies show that these increased responses in fusiform cortex to fearful faces are abolished by amygdala damage in the ipsilateral hemisphere, despite preserved effects of voluntary attention on fusiform; whereas emotional increases can still arise despite deficits in attention or awareness following parietal damage, and appear relatively unaffected by pharmacological increases in cholinergic stimulation. Fear-related modulations of face processing driven by amygdala signals may implicate not only fusiform cortex, but also earlier visual areas in occipital cortex (e.g., V1) and other distant regions involved in social, cognitive, or somatic responses (e.g., superior temporal sulcus, cingulate, or parietal areas). In the temporal domain, evoked-potentials show a widespread time-course of emotional face perception, with some increases in the amplitude of responses recorded over both occipital and frontal regions for fearful relative to neutral faces (as well as in the amygdala and orbitofrontal cortex, when using intracranial recordings), but with different latencies post-stimulus onset. Early emotional responses may arise around 120ms, prior to a full visual categorization stage indexed by the face-selective N170 component, possibly reflecting rapid emotion processing based on crude visual cues in faces. Other electrical components arise at later latencies and involve more sustained activities, probably generated in associative or supramodal brain areas, and resulting in part from the modulatory signals received from amygdala. Altogether, these fMRI and ERP results demonstrate that emotion face perception is a complex process that cannot be related to a single neural event taking place in a single brain regions, but rather implicates an interactive network with distributed activity in time and space. Moreover, although traditional models in cognitive neuropsychology have often considered that facial expression and facial identity are processed along two separate pathways, evidence from fMRI and ERPs suggests instead that emotional processing can strongly affect brain systems responsible for face recognition and memory. The functional implications of these interactions remain to be fully explored, but might play an important role in the normal development of face processing skills and in some neuropsychiatric disorders.

Distant influences of amygdala lesion on visual cortical activation during emotional face processing

Emotional visual stimuli evoke enhanced responses in the visual cortex. To test whether this reflects modulatory influences from the amygdala on sensory processing, we used event-related functional magnetic resonance imaging (fMRI) in human patients with medial temporal lobe sclerosis. Twenty-six patients with lesions in the amygdala, the hippocampus or both, plus 13 matched healthy controls, were shown pictures of fearful or neutral faces in task-releant or task-irrelevant positions on the display. All subjects showed increased fusiform cortex activation when the faces were in task-relevant positions. Both healthy individuals and those with hippocampal damage showed increased activation in the fusiform and occipital cortex when they were shown fearful faces, but this was not the case for individuals with damage to the amygdala, even though visual areas were structurally intact. The distant influence of the amygdala was also evidenced by the parametric relationship between amygdala damage and the level of emotional activation in the fusiform cortex. Our data show that combining the fMRI and lesion approaches can help reveal the source of functional modulatory influences between distant but interconnected brain regions.

Multiple levels of visual object constancy revealed by event-related fMRI of repetition priming

We conducted two event-related functional magnetic resonance imaging (fMRI) experiments to investigate the neural substrates of visual object recognition in humans. We used a repetition-priming method with visual stimuli recurring at unpredictable intervals, either with the same appearance or with changes in size, viewpoint or exemplar. Lateral occipital and posterior inferior temporal cortex showed lower activity for repetitions of both real and non-sense objects; fusiform and left inferior frontal regions showed decreases for repetitions of only real objects. Repetition of different exemplars with the same name affected only the left inferior frontal cortex. Crucially, priming-induced decreases in activity of the right fusiform cortex depended on whether the three-dimensional objects were repeated with the same viewpoint, regardless of whether retinal image size changed; left fusiform decreases were independent of both viewpoint and size. These data show that dissociable subsystems in ventral visual cortex maintain distinct view-dependent and view-invariant object representations.

The processing of emotional facial expression is gated by spatial attention: evidence from event-related brain potentials.

To investigate whether the processing of faces and emotional facial expression can be modulated by spatial attention, ERPs were recorded in response to stimulus arrays containing two faces and two non-face stimuli (houses). In separate trials, attention was focused on the face pair or on the house pair, and facial expression was either fearful or neutral. When faces were attended, a greater frontal positivity in response to arrays containing fearful faces was obtained, starting about 100 ms after stimulus onset. In contrast, with faces located outside the attentional focus, this emotional expression effect was completely eliminated. This differential result demonstrates for the first time a strong attentional gating of brain processes involved in the analysis of emotional facial expression. It is argued that while an initial detection of emotionally relevant events mediated by the amygdala may occur pre-attentively, subsequent stages of emotional processing require focal spatial attention. The face-sensitive N170 component was unaffected by emotional facial expression, but N170 amplitudes were enhanced when faces were attended, suggesting that spatial attention can modulate the structural encoding of faces.

The voices of wrath: brain responses to angry prosody in meaningless speech

We report two functional magnetic resonance imaging experiments showing enhanced responses in human middle superior temporal sulcus for angry relative to neutral prosody. This emotional enhancement was voice specific, unrelated to isolated acoustic amplitude or frequency cues in angry prosody, and distinct from any concomitant task-related attentional modulation. Attention and emotion seem to have separate effects on stimulus processing, reflecting a fundamental principle of human brain organization shared by voice and face perception.

Neuroanatomy of hemispatial neglect and its functional components: a study using voxel-based lesion-symptom mapping

Spatial neglect is a perplexing neuropsychological syndrome, in which patients fail to detect (and/or respond to) stimuli located contralaterally to their (most often right) hemispheric lesion. Neglect is characterized by a wide heterogeneity, and a role for multiple components has been suggested, but the exact nature of the critical components remains unclear. Moreover, many different lesion sites have been reported, leading to enduring controversies about the relative contribution of different cortical and/or subcortical brain regions. Here we report a systematic anatomo-functional study of 80 patients with a focal right hemisphere stroke, who were examined by a series of neuropsychological tests assessing different clinical manifestations of neglect. We first performed a statistical factorial analysis of their behavioural performance across all tests, in order to break down neglect symptoms into coherent profiles of co-varying deficits. We then examined the neural correlates of these distinct neglect profiles using a statistical voxel-based lesion-symptom mapping method that correlated the anatomical extent of brain damage with the relative severity of deficits along the different profiles in each patient. Our factorial analysis revealed three main factors explaining 82% of the total variance across all neglect tests, which suggested distinct components related to perceptive/visuo-spatial, exploratory/visuo-motor, and allocentric/object-centred aspects of spatial neglect. Our anatomical voxel-based lesion-symptom mapping analysis pointed to specific neural correlates for each of these components, including the right inferior parietal lobule for the perceptive/visuo-spatial component, the right dorsolateral prefrontal cortex for the exploratory/visuo-motor component, and deep temporal lobe regions for the allocentric/object-centred component. By contrast, standard anatomical overlap analysis indicated that subcortical damage to paraventricular white matter tracts was associated with severe neglect encompassing several tests. Taken together, our results provide new support to the view that the clinical manifestations of hemispatial neglect might reflect a combination of distinct components affecting different domains of spatial cognition, and that intra-hemispheric disconnection due to white matter lesions might produce severe neglect by impacting on more than one functional domain.

Modulation of visual processing by attention and emotion: windows on causal interactions between human brain regions

Visual processing is not determined solely by retinal inputs. Attentional modulation can arise when the internal attentional state (current task) of the observer alters visual processing of the same stimuli. This can influence visual cortex, boosting neural responses to an attended stimulus. Emotional modulation can also arise, when affective properties (emotional significance) of stimuli, rather than their strictly visual properties, influence processing. This too can boost responses in visual cortex, as for fear-associated stimuli. Both attentional and emotional modulation of visual processing may reflect distant influences upon visual cortex, exerted by brain structures outside the visual system per se. Hence, these modulations may provide windows onto causal interactions between distant but interconnected brain regions. We review recent evidence, noting both similarities and differences between attentional and emotional modulation. Both can affect visual cortex, but can reflect influences from different regions, such as fronto-parietal circuits versus the amygdala. Recent work on this has developed new approaches for studying causal influences between human brain regions that may be useful in other cognitive domains. The new methods include application of functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) measures in brain-damaged patients to study distant functional impacts of their focal lesions, and use of transcranial magnetic stimulation concurrently with fMRI or EEG in the normal brain. Cognitive neuroscience is now moving beyond considering the putative functions of particular brain regions, as if each operated in isolation, to consider, instead, how distinct brain regions (such as visual cortex, parietal or frontal regions, or amygdala) may mutually influence each other in a causal manner.

Emotional facial expressions capture attention

Objective: To determine whether the emotional significance of stimuli can influence spatial attention. Background: Motivational and emotional factors may affect attention toward stimuli. However, this has never been examined in brain-damaged patients who present with unilateral inattention due to left spatial neglect. Methods: The authors studied three patients with chronic left neglect and visual extinction after right parietal stroke. Shapes or faces with neutral, happy, or angry expressions were briefly presented in the right, left, or both visual fields. On unilateral trials, the patients detected all stimuli equally on both sides. On bilateral trials, they extinguished faces in the contralesional field much less often than shapes, and faces with happy or angry facial expressions much less than faces with a neutral expression. Conclusion: Facial features and emotional expressions can be analyzed despite lying on the unattended side, and may influence the spatial distribution of attention. These findings support the view that attention is controlled by neural mechanisms involving not only frontoparietal areas but also limbic components in cingulate cortex and amygdala, which may interact with ventral visual areas in the temporal lobe to detect affective value and prioritize attention to salient stimuli.