Assaf Harel

Deconstructing Visual Scenes in Cortex: Gradients of Object and Spatial Layout Information

Real-world visual scenes are complex cluttered, and heterogeneous stimuli engaging scene- and object-selective cortical regions including parahippocampal place area (PPA), retrosplenial complex (RSC), and lateral occipital complex (LOC). To understand the unique contribution of each region to distributed scene representations, we generated predictions based on a neuroanatomical framework adapted from monkey and tested them using minimal scenes in which we independently manipulated both spatial layout (open, closed, and gradient) and object content (furniture, e.g., bed, dresser). Commensurate with its strong connectivity with posterior parietal cortex, RSC evidenced strong spatial layout information but no object information, and its response was not even modulated by object presence. In contrast, LOC, which lies within the ventral visual pathway, contained strong object information but no background information. Finally, PPA, which is connected with both the dorsal and the ventral visual pathway, showed information about both objects and spatial backgrounds and was sensitive to the presence or absence of either. These results suggest that 1) LOC, PPA, and RSC have distinct representations, emphasizing different aspects of scenes, 2) the specific representations in each region are predictable from their patterns of connectivity, and 3) PPA combines both spatial layout and object information as predicted by connectivity.

Task context impacts visual object processing differentially across the cortex

Significance Visual recognition is often thought to depend on neural representations that primarily reflect the physical properties of the environment. However, in this study we demonstrate that the intent of the observer fundamentally perturbs cortical representations of visual objects. Using functional MRI we measured the patterns of response to identical objects under six different tasks. In any given task, these patterns could be used to distinguish which object was being viewed. However, this ability was disrupted when the task changed, indicating that object representations reflect not only the physical properties of the stimulus, but also the internal state of the observer. Perception reflects an integration of “bottom-up” (sensory-driven) and “top-down” (internally generated) signals. Although models of visual processing often emphasize the central role of feed-forward hierarchical processing, less is known about the impact of top-down signals on complex visual representations. Here, we investigated whether and how the observer’s goals modulate object processing across the cortex. We examined responses elicited by a diverse set of objects under six distinct tasks, focusing on either physical (e.g., color) or conceptual properties (e.g., man-made). Critically, the same stimuli were presented in all tasks, allowing us to investigate how task impacts the neural representations of identical visual input. We found that task has an extensive and differential impact on object processing across the cortex. First, we found task-dependent representations in the ventral temporal and prefrontal cortex. In particular, although object identity could be decoded from the multivoxel response within task, there was a significant reduction in decoding across tasks. In contrast, the early visual cortex evidenced equivalent decoding within and across tasks, indicating task-independent representations. Second, task information was pervasive and present from the earliest stages of object processing. However, although the responses of the ventral temporal, prefrontal, and parietal cortex enabled decoding of both the type of task (physical/conceptual) and the specific task (e.g., color), the early visual cortex was not sensitive to type of task and could only be used to decode individual physical tasks. Thus, object processing is highly influenced by the behavioral goal of the observer, highlighting how top-down signals constrain and inform the formation of visual representations.

Top-Down Engagement Modulates the Neural Expressions of Visual Expertise

Perceptual expertise is traditionally associated with enhanced brain activity in response to objects of expertise in category-selective visual cortex, primarily face-selective regions. We reevaluated this view by investigating whether the brain activity associated with expertise in object recognition is limited to category-selective cortex and specifically whether the extent of expertise-related activity manifests automatically or whether it can be top-down modulated. We conducted 2 functional magnetic resonance imaging studies comparing changes in hemodynamic activity associated with car expertise in a conventional 1-back task (Experiment 1) and when the task relevance of cars was explicitly manipulated (Experiment 2). Whole-brain analysis unveiled extensive expertise-related activity throughout the visual cortex, starting as early as V1 and extending into nonvisual areas. However, when the cars were task irrelevant, the expertise-related activity drastically diminished, indeed, becoming similar to the activity elicited by cars in novices. We suggest that expertise entails voluntary top-down engagement of multiple neural networks in addition to stimulus-driven activation associated with perceptual mechanisms.

Aging effects on early-stage face perception: An ERP study

We compared early stages of face processing in young and older participants as indexed by ERPs elicited by faces and non-face stimuli presented in upright and inverted orientations. The P1 and N170 components were larger in older than in young participants. However, the early distinction between stimulus categories as reflected by N170 face was similar across groups. Face inversion increased and delayed the N170 peak in the younger group while in older participants inversion delayed the N170 peak but had no effect on amplitude. The N170 amplitude was right-lateralized in the young, but not in the older group. Yet, the difference between the N170 elicited by faces and non-face stimuli was similarly right-lateralized in both groups. These data suggest that detection of faces and their streaming to face-characteristic structural encoding is not altered by age. In contrast, the absence of face-inversion effects on N170 amplitudes in the elderly suggest that face individuation, which is probably the default strategy in younger people, might not be attempted by default in older people, at least when they look at young faces.

Mutual information of image fragments predicts categorization in humans: electrophysiological and behavioral evidence.

Computational models suggest that features of intermediate complexity (IC) play a central role in object categorization [Ullman, S., Vidal-Naquet, M., & Sali, E. (2002). Visual features of intermediate complexity and their use in classification. Nature Neuroscience, 5, 682-687.]. The critical aspect of these features is the amount of mutual information (MI) they deliver. We examined the relation between MI, human categorization and an electrophysiological response to IC features. Categorization performance correlated with MI level as well as with the amplitude of a posterior temporal potential, peaking around 270 ms. Hence, an objective MI measure predicts human object categorization performance and its underlying neural activity. These results demonstrate that informative IC features serve as categorization features in human vision.

Stimulus Type, Level of Categorization, and Spatial-Frequencies Utilization: Implications for Perceptual Categorization Hierarchies

The type of visual information needed for categorizing faces and nonface objects was investigated by manipulating spatial frequency scales available in the image during a category verification task addressing basic and subordinate levels. Spatial filtering had opposite effects on faces and airplanes that were modulated by categorization level. The absence of low frequencies impaired the categorization of faces similarly at both levels, whereas the absence of high frequencies was inconsequential throughout. In contrast, basic-level categorization of airplanes was equally impaired by the absence of either low or high frequencies, whereas at the subordinate level, the absence of high frequencies had more deleterious effects. These data suggest that categorization of faces either at the basic level or by race is based primarily on their global shape but also on the configuration of details. By contrast, basic-level categorization of objects is based on their global shape, whereas category-specific diagnostic details determine the information needed for their subordinate categorization. The authors conclude that the entry point in visual recognition is flexible and determined conjointly by the stimulus category and the level of categorization, which reflects the observers recognition goal.

Neuroanatomical Correlates of Visual Car Expertise

Expertise in non-visual domains such as musical performance is associated with differences in gray matter volume of particular regions of the human brain. Whether this is also the case for expertise in visual object recognition is unknown. Here we tested whether individual variability in the ability to recognize car models, from novice performance to high level of expertise, is associated with specific structural changes in gray matter volume. We found that inter-individual variability in expertise with cars was significantly and selectively correlated with gray matter volume in prefrontal cortex. Inter-individual differences in the recognition of airplanes, that none of the participants had expertise with, were correlated with structural variability of regions bordering the visual cortex. These results highlight the role of prefrontal regions outside the visual cortex in accessing and processing visual knowledge about objects from the domain of expertise and suggest that expertise in visual object recognition may entail structural changes in regions associated with semantic knowledge.

Association and dissociation between detection and discrimination of objects of expertise: Evidence from visual search

Expertise in face recognition is characterized by high proficiency in distinguishing between individual faces. However, faces also enjoy an advantage at the early stage of basic-level detection, as demonstrated by efficient visual search for faces among nonface objects. In the present study, we asked (1) whether the face advantage in detection is a unique signature of face expertise, or whether it generalizes to other objects of expertise, and (2) whether expertise in face detection is intrinsically linked to expertise in face individuation. We compared how groups with varying degrees of object and face expertise (typical adults, developmental prosopagnosics [DP], and car experts) search for objects within and outside their domains of expertise (faces, cars, airplanes, and butterflies) among a variable set of object distractors. Across all three groups, search efficiency (indexed by reaction time slopes) was higher for faces and airplanes than for cars and butterflies. Notably, the search slope for car targets was considerably shallower in the car experts than in nonexperts. Although the mean face slope was slightly steeper among the DPs than in the other two groups, most of the DPs’ search slopes were well within the normative range. This pattern of results suggests that expertise in object detection is indeed associated with expertise at the subordinate level, that it is not specific to faces, and that the two types of expertise are distinct facilities. We discuss the potential role of experience in bridging between low-level discriminative features and high-level naturalistic categories.

Are all types of expertise created equal? Car experts use different spatial frequency scales for subordinate categorization of cars and faces.

A much-debated question in object recognition is whether expertise for faces and expertise for non-face objects utilize common perceptual information. We investigated this issue by assessing the diagnostic information required for different types of expertise. Specifically, we asked whether face categorization and expert car categorization at the subordinate level relies on the same spatial frequency (SF) scales. Fifteen car experts and fifteen novices performed a category verification task with spatially filtered images of faces, cars, and airplanes. Images were categorized based on their basic (e.g. “car”) and subordinate level (e.g. “Japanese car”) identity. The effect of expertise was not evident when objects were categorized at the basic level. However, when the car experts categorized faces and cars at the subordinate level, the two types of expertise required different kinds of SF information. Subordinate categorization of faces relied on low SFs more than on high SFs, whereas subordinate expert car categorization relied on high SFs more than on low SFs. These findings suggest that expertise in the recognition of objects and faces do not utilize the same type of information. Rather, different types of expertise require different types of diagnostic visual information.

The representational dynamics of task and object processing in humans

Despite the importance of an observer’s goals in determining how a visual object is categorized, surprisingly little is known about how humans process the task context in which objects occur and how it may interact with the processing of objects. Using magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI) and multivariate techniques, we studied the spatial and temporal dynamics of task and object processing. Our results reveal a sequence of separate but overlapping task-related processes spread across frontoparietal and occipitotemporal cortex. Task exhibited late effects on object processing by selectively enhancing task-relevant object features, with limited impact on the overall pattern of object representations. Combining MEG and fMRI data, we reveal a parallel rise in task-related signals throughout the cerebral cortex, with an increasing dominance of task over object representations from early to higher visual areas. Collectively, our results reveal the complex dynamics underlying task and object representations throughout human cortex.