James V. Haxby

The distributed human neural system for face perception

Face perception, perhaps the most highly developed visual skill in humans, is mediated by a distributed neural system in humans that is comprised of multiple, bilateral regions. We propose a model for the organization of this system that emphasizes a distinction between the representation of invariant and changeable aspects of faces. The representation of invariant aspects of faces underlies the recognition of individuals, whereas the representation of changeable aspects of faces, such as eye gaze, expression, and lip movement, underlies the perception of information that facilitates social communication. The model is also hierarchical insofar as it is divided into a core system and an extended system. The core system is comprised of occipitotemporal regions in extrastriate visual cortex that mediate the visual analysis of faces. In the core system, the representation of invariant aspects is mediated more by the face-responsive region in the fusiform gyrus, whereas the representation of changeable aspects is mediated more by the face-responsive region in the superior temporal sulcus. The extended system is comprised of regions from neural systems for other cognitive functions that can be recruited to act in concert with the regions in the core system to extract meaning from faces.

Beyond mind-reading: multi-voxel pattern analysis of fMRI data.

A key challenge for cognitive neuroscience is determining how mental representations map onto patterns of neural activity. Recently, researchers have started to address this question by applying sophisticated pattern-classification algorithms to distributed (multi-voxel) patterns of functional MRI data, with the goal of decoding the information that is represented in the subjects brain at a particular point in time. This multi-voxel pattern analysis (MVPA) approach has led to several impressive feats of mind reading. More importantly, MVPA methods constitute a useful new tool for advancing our understanding of neural information processing. We review how researchers are using MVPA methods to characterize neural coding and information processing in domains ranging from visual perception to memory search.

The functional organization of human extrastriate cortex: a PET-rCBF study of selective attention to faces and locations

The functional dissociation of human extrastriate cortical processing streams for the perception of face identity and location was investigated in healthy men by measuring visual task-related changes in regional cerebral blood flow (rCBF) with positron emission tomography (PET) and H2(15)O. Separate scans were obtained while subjects performed face matching, location matching, or sensorimotor control tasks. The matching tasks used identical stimuli for some scans and stimuli of equivalent visual complexity for others. Face matching was associated with selective rCBF increases in the fusiform gyrus in occipital and occipitotemporal cortex bilaterally and in a right prefrontal area in the inferior frontal gyrus. Location matching was associated with selective rCBF increases in dorsal occipital, superior parietal, and intraparietal sulcus cortex bilaterally and in dorsal right premotor cortex. Decreases in rCBF, relative to the sensorimotor control task, were observed for both matching tasks in auditory, auditory association, somatosensory, and midcingulate cortex. These results suggest that, within a sensory modality, selective attention is associated with increased activity in those cortical areas that process the attended information but is not associated with decreased activity in areas that process unattended visual information. Selective attention to one sensory modality, on the other hand, is associated with decreased activity in cortical areas dedicated to processing input from other sensory modalities. Direct comparison of our results with those from other PET-rCBF studies of extrastriate cortex demonstrates agreement in the localization of cortical areas mediating face and location perception and dissociations between these areas and those mediating the perception of color and motion.

Distinct representations of eye gaze and identity in the distributed human neural system for face perception

Face perception requires representation of invariant aspects that underlie identity recognition as well as representation of changeable aspects, such as eye gaze and expression, that facilitate social communication. Using functional magnetic resonance imaging (fMRI), we investigated the perception of face identity and eye gaze in the human brain. Perception of face identity was mediated more by regions in the inferior occipital and fusiform gyri, and perception of eye gaze was mediated more by regions in the superior temporal sulci. Eye-gaze perception also seemed to recruit the spatial cognition system in the intraparietal sulcus to encode the direction of anothers gaze and to focus attention in that direction.

Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects

The cognitive and neural mechanisms underlying category-specific knowledge remain controversial. Here we report that, across multiple tasks (viewing, delayed match to sample, naming), pictures of animals and tools were associated with highly consistent, category-related patterns of activation in ventral (fusiform gyrus) and lateral (superior and middle temporal gyri) regions of the posterior temporal lobes. In addition, similar patterns of category-related activity occurred when subjects read the names of, and answered questions about, animals and tools. These findings suggest that semantic object information is represented in distributed networks that include sites for storing information about specific object attributes such as form (ventral temporal cortex) and motion (lateral temporal cortex).

Discrete Cortical Regions Associated with Knowledge of Color and Knowledge of Action

The areas of the brain that mediate knowledge about objects were investigated by measuring changes in regional cerebral blood flow (rCBF) using positron emission tomography (PET). Subjects generated words denoting colors and actions associated with static, achromatic line drawings of objects in one experiment, and with the written names of objects in a second experiment. In both studies, generation of color words selectively activated a region in the ventral temporal lobe just anterior to the area involved in the perception of color, whereas generation of action words activated a region in the middle temporal gyrus just anterior to the area involved in the perception of motion. These data suggest that object knowledge is organized as a distributed system in which the attributes of an object are stored close to the regions of the cortex that mediate perception of those attributes.

Human neural systems for face recognition and social communication

Face perception is mediated by a distributed neural system in humans that consists of multiple, bilateral regions. The functional organization of this system embodies a distinction between the representation of invariant aspects of faces, which is the basis for recognizing individuals, and the representation of changeable aspects, such as eye gaze, expression, and lip movement, which underlies the perception of information that facilitates social communication. The system also has a hierarchical organization. A core system, consisting of occipitotemporal regions in extrastriate visual cortex, mediates the visual analysis of faces. An extended system consists of regions from neural systems for other cognitive functions that can act in concert with the core system to extract meaning from faces. Of regions in the extended system for face perception, the amygdala plays a central role in processing the social relevance of information gleaned from faces, particularly when that information may signal a potential threat.

Spatial Pattern Analysis of Functional Brain Images Using Partial Least Squares

This paper introduces a new tool for functional neuroimage analysis: partial least squares (PLS). It is unique as a multivariate method in its choice of emphasis for analysis, that being the covariance between brain images and exogenous blocks representing either the experiment design or some behavioral measure. What emerges are spatial patterns of brain activity that represent the optimal association between the images and either of the blocks. This process differs substantially from other multivariate methods in that rather than attempting to predict the individual values of the image pixels, PLS attempts to explain the relation between image pixels and task or behavior. Data from a face encoding and recognition PET rCBF study are used to illustrate two types of PLS analysis: an activation analysis of task with images and a brain-behavior analysis. The commonalities across the two analyses are suggestive of a general face memory network differentially engaged during encoding and recognition. PLS thus serves as an important extension by extracting new information from imaging data that is not accessible through other currently used univariate and multivariate image analysis tools.

The Effect of Face Inversion on Activity in Human Neural Systems for Face and Object Perception

The differential effect of stimulus inversion on face and object recognition suggests that inverted faces are processed by mechanisms for the perception of other objects rather than by face perception mechanisms. We investigated the face inversion using functional magnetic resonance imaging (fMRI). The principal effect of face inversion on was an increased response in ventral extrastriate regions that respond preferentially to another class of objects (houses). In contrast, house inversion did not produce a similar change in face-selective regions. Moreover, stimulus inversion had equivalent, minimal effects for faces in in face-selective regions and for houses in house-selective regions. The results suggest that the failure of face perception systems with inverted faces leads to the recruitment of processing resources in object perception systems, but this failure is not reflected by altered activity in face perception systems.

Positron emission tomography in Alzheimer's disease

Twenty-one patients with a clinical diagnosis of dementia of the Alzheimers type (DAT) and 29 healthy, age-matched controls were studied using positron emission tomography (PET) and [18F]2-fluoro-2-deoxy-D-glucose to measure regional cerebral glucose consumption in the resting state. Reductions in ratio measures of relative metabolism in some parietal, temporal, and frontal regions were found in mild, moderate, and severe DAT groups. A significant increase in right/left metabolic asymmetry, particularly in parietal regions, also was seen in mild and moderate groups. Only in the severely demented patients was the absolute cerebral metabolic rate reduced significantly from control values. Fourteen patients had repeated PET studies, but only those patients with moderate to severe dementia showed a decline in IQ over 6 to 15 months. There were no significant changes in metabolic measures over time. PET is useful in quantifying regional cerebral dysfunction in DAT, even in the early stages of the disease.