Stefania Bracci

Closely overlapping responses to tools and hands in left lateral occipitotemporal cortex

The perception of object-directed actions performed by either hands or tools recruits regions in left fronto-parietal cortex. Here, using functional MRI (fMRI), we tested whether the common role of hands and tools in object manipulation is also reflected in the distribution of response patterns to these categories in visual cortex. In two experiments we found that static pictures of hands and tools activated closely overlapping regions in left lateral occipitotemporal cortex (LOTC). Left LOTC responses to tools selectively overlapped with responses to hands but not with responses to whole bodies, nonhand body parts, other objects, or visual motion. Multivoxel pattern analysis in left LOTC indicated a high degree of similarity between response patterns to hands and tools but not between hands or tools and other body parts. Finally, functional connectivity analysis showed that the left LOTC hand/tool region was selectively connected, relative to neighboring body-, motion-, and object-responsive regions, with regions in left intraparietal sulcus and left premotor cortex that have previously been implicated in hand/tool action-related processing. Taken together, these results suggest that action-related object properties shared by hands and tools are reflected in the organization of high-order visual cortex. We propose that the functional organization of high-order visual cortex partly reflects the organization of downstream functional networks, such as the fronto-parietal action network, due to differences within visual cortex in the connectivity to these networks.

Dissociable Neural Responses to Hands and Non-Hand Body Parts in Human Left Extrastriate Visual Cortex

Accumulating evidence points to a map of visual regions encoding specific categories of objects. For example, a region in the human extrastriate visual cortex, the extrastriate body area (EBA), has been implicated in the visual processing of bodies and body parts. Although in the monkey, neurons selective for hands have been reported, in humans it is unclear whether areas selective for individual body parts such as the hand exist. Here, we conducted two functional MRI experiments to test for hand-preferring responses in the human extrastriate visual cortex. We found evidence for a hand-preferring region in left lateral occipitotemporal cortex in all 14 participants. This region, located in the lateral occipital sulcus, partially overlapped with left EBA, but could be functionally and anatomically dissociated from it. In experiment 2, we further investigated the functional profile of hand- and body-preferring regions by measuring responses to hands, fingers, feet, assorted body parts (arms, legs, torsos), and non-biological handlike stimuli such as robotic hands. The hand-preferring region responded most strongly to hands, followed by robotic hands, fingers, and feet, whereas its response to assorted body parts did not significantly differ from baseline. By contrast, EBA responded most strongly to body parts, followed by hands and feet, and did not significantly respond to robotic hands or fingers. Together, these results provide evidence for a representation of the hand in extrastriate visual cortex that is distinct from the representation of other body parts.

Body and object effectors: the organization of object representations in high-level visual cortex reflects body-object interactions.

The principles driving the functional organization of object representations in high-level visual cortex are not yet fully understood. In four human fMRI experiments, we provide evidence that the organization of high-level visual cortex partly reflects the degree to which objects are typically controlled by the body to interact with the world, thereby extending the bodys boundaries. Univariate whole-brain analysis showed an overlap between responses to body effectors (e.g., hands, feet, and limbs) and object effectors (e.g., hammers, combs, and tennis rackets) in lateral occipitotemporal cortex (LOTC) and parietal cortex. Region of interest analyses showed that a hand-selective region in left LOTC responded selectively to object effectors relative to a range of noneffector object control conditions (e.g., graspable objects, “act-on” objects, musical instruments). Object ratings showed that the strong response to object effectors in hand-selective LOTC was not due to general action-related object properties shared with these control conditions, such as hand priming, hand grasping, and hand-action centrality. Finally, whole-brain representational similarity analysis revealed that the similarity of multivoxel object response patterns in left lateral occipitotemporal cortex selectively predicted the degree to which objects were rated as being controlled by and extending the body. Together, these results reveal a clustering of body and object effector representations, indicating that the organization of object representations in high-level visual cortex partly reflects how objects relate to the body.

Tool selectivity in left occipitotemporal cortex develops without vision

Previous studies have provided evidence for a tool-selective region in left lateral occipitotemporal cortex (LOTC). This region responds selectively to pictures of tools and to characteristic visual tool motion. The present human fMRI study tested whether visual experience is required for the development of tool-selective responses in left LOTC. Words referring to tools, animals, and nonmanipulable objects were presented auditorily to 14 congenitally blind and 16 sighted participants. Sighted participants additionally viewed pictures of these objects. In whole-brain group analyses, sighted participants showed tool-selective activity in left LOTC in both visual and auditory tasks. Importantly, virtually identical tool-selective LOTC activity was found in the congenitally blind group performing the auditory task. Furthermore, both groups showed equally strong tool-selective activity for auditory stimuli in a tool-selective LOTC region defined by the picture-viewing task in the sighted group. Detailed analyses in individual participants showed significant tool-selective LOTC activity in 13 of 14 blind participants and 14 of 16 sighted participants. The strength and anatomical location of this activity were indistinguishable across groups. Finally, both blind and sighted groups showed significant resting state functional connectivity between left LOTC and a bilateral frontoparietal network. Together, these results indicate that tool-selective activity in left LOTC develops without ever having seen a tool or its motion. This finding puts constraints on the possible role that this region could have in tool processing and, more generally, provides new insights into the principles shaping the functional organization of OTC.

Task Context Overrules Object- and Category-Related Representational Content in the Human Parietal Cortex

Abstract The dorsal, parietal visual stream is activated when seeing objects, but the exact nature of parietal object representations is still under discussion. Here we test 2 specific hypotheses. First, parietal cortex is biased to host some representations more than others, with a different bias compared with ventral areas. A prime example would be object action representations. Second, parietal cortex forms a general multiple‐demand network with frontal areas, showing similar task effects and representational content compared with frontal areas. To differentiate between these hypotheses, we implemented a human neuroimaging study with a stimulus set that dissociates associated object action from object category while manipulating task context to be either action‐ or category‐related. Representations in parietal as well as prefrontal areas represented task‐relevant object properties (action representations in the action task), with no sign of the irrelevant object property (category representations in the action task). In contrast, irrelevant object properties were represented in ventral areas. These findings emphasize that human parietal cortex does not preferentially represent particular object properties irrespective of task, but together with frontal areas is part of a multiple‐demand and content‐rich cortical network representing task‐relevant object properties.

Representational Similarity of Body Parts in Human Occipitotemporal Cortex

Regions in human lateral and ventral occipitotemporal cortices (OTC) respond selectively to pictures of the human body and its parts. What are the organizational principles underlying body part responses in these regions? Here we used representational similarity analysis (RSA) of fMRI data to test multiple possible organizational principles: shape similarity, physical proximity, cortical homunculus proximity, and semantic similarity. Participants viewed pictures of whole persons, chairs, and eight body parts (hands, arms, legs, feet, chests, waists, upper faces, and lower faces). The similarity of multivoxel activity patterns for all body part pairs was established in whole person-selective OTC regions. The resulting neural similarity matrices were then compared with similarity matrices capturing the hypothesized organizational principles. Results showed that the semantic similarity model best captured the neural similarity of body parts in lateral and ventral OTC, which followed an organization in three clusters: (1) body parts used as action effectors (hands, feet, arms, and legs), (2) noneffector body parts (chests and waists), and (3) face parts (upper and lower faces). Whole-brain RSA revealed, in addition to OTC, regions in parietal and frontal cortex in which neural similarity was related to semantic similarity. In contrast, neural similarity in occipital cortex was best predicted by shape similarity models. We suggest that the semantic organization of body parts in high-level visual cortex relates to the different functions associated with the three body part clusters, reflecting the unique processing and connectivity demands associated with the different types of information (e.g., action, social) different body parts (e.g., limbs, faces) convey. SIGNIFICANCE STATEMENT While the organization of body part representations in motor and somatosensory cortices has been well characterized, the principles underlying body part representations in visual cortex have not yet been explored. In the present fMRI study we used multivoxel pattern analysis and representational similarity analysis to characterize the organization of body maps in human occipitotemporal cortex (OTC). Results indicate that visual and shape dimensions do not fully account for the organization of body part representations in OTC. Instead, the representational structure of body maps in OTC appears strongly related to functional-semantic properties of body parts. We suggest that this organization reflects the unique processing and connectivity demands associated with the different types of information different body parts convey.

Avoiding illusory effects in representational similarity analysis: What (not) to do with the diagonal

Representational similarity analysis (RSA) is an important part of the methodological toolkit in neuroimaging research. The focus of the approach is the construction of representational dissimilarity matrices (RDMs), which provide a single format for making comparisons between different neural data types, computational models, and behavior. We highlight two issues for the construction and comparison of RDMs. First, the diagonal values of RDMs, which should reflect within condition reliability of neural patterns, are typically not estimated in RSA. However, without such an estimate, one lacks a measure of the reliability of an RDM as a whole. Thus, when carrying out RSA, one should calculate the diagonal values of RDMs and not take them for granted. Second, although diagonal values of a correlation matrix can be used to estimate the reliability of neural patterns, these values must nonetheless be excluded when comparing RDMs. Via a simple simulation we show that inclusion of these values can generate convincing looking, but entirely illusory, correlations between independent and entirely unrelated data sets. Both of these points are further illustrated by a critical discussion of Coggan et al. (2016), who investigated the extent to which category-selectivity in the ventral temporal cortex can be accounted for by low-level image properties of visual object stimuli. We observe that their results may depend on the improper inclusion of diagonal values in their analysis.

On the partnership between neural representations of object categories and visual features in the ventral visual pathway

A dominant view in the cognitive neuroscience of object vision is that regions of the ventral visual pathway exhibit some degree of category selectivity. However, recent findings obtained with multivariate pattern analyses (MVPA) suggest that apparent category selectivity in these regions is dependent on more basic visual features of stimuli. In which case a rethinking of the function and organization of the ventral pathway may be in order. We suggest that addressing this issue of functional specificity requires clear coding hypotheses, about object category and visual features, which make contrasting predictions about neuroimaging results in ventral pathway regions. One way to differentiate between categorical and featural coding hypotheses is to test for residual categorical effects: effects of category selectivity that cannot be accounted for by visual features of stimuli. A strong method for testing these effects, we argue, is to make object category and target visual features orthogonal in stimulus design. Recent studies that adopt this approach support a feature-based categorical coding hypothesis according to which regions of the ventral stream do indeed code for object category, but in a format at least partially based on the visual features of stimuli.

View-invariant representation of hand postures in the human lateral occipitotemporal cortex

ABSTRACT Understanding other peoples actions and mental states includes the interpretation of body postures and movements. In particular, hand postures are an important channel to signal both action and communicative intentions. Recognizing hand postures is computationally challenging because hand postures often differ only in the subtle configuration of relative finger positions and because visual characteristics of hand postures change across viewpoints. To allow for accurate interpretation, the brain needs to represent hand postures in a view‐invariant but posture‐specific manner. Here we test for such representations in hand‐, body‐, and object‐selective regions of the lateral occipitotemporal cortex (LOTC). We used multivariate pattern analysis of fMRI data to test for view‐specific and view‐invariant representations of individual hand postures, separately for two domains: action‐related postures (e.g., a precision grasp) and communicative postures (e.g., thumbs up). Results showed that hand‐selective LOTC, but not nearby body‐ and object‐selective LOTC, represented hand postures in a view‐invariant manner, with relatively similar activity patterns to the same hand posture seen from different viewpoints. View invariance was equally strong for action and communicative postures. By contrast, object‐selective cortex represented hand postures in a view‐specific manner. These results indicate a role for hand‐selective LOTC in solving the view‐invariance problem for individual hand postures. View‐invariant representations of hand postures in this region may then be accessed and further interpreted by multiple downstream systems to inform high‐level judgments related to action understanding, emotion recognition, and non‐verbal communication. HIGHLIGHTSThe occipitotemporal hand‐selective region represented view‐invariant hand postures.View invariance was equally strong for action and communicative postures.No view‐invariant hand posture information in the nearby body‐selective region.Object‐selective cortex represented hand postures in a view‐specific manner.