Quanjing Chen

The Representation of Object-Directed Action and Function Knowledge in the Human Brain

The appropriate use of everyday objects requires the integration of action and function knowledge. Previous research suggests that action knowledge is represented in frontoparietal areas while function knowledge is represented in temporal lobe regions. Here we used multivoxel pattern analysis to investigate the representation of object-directed action and function knowledge while participants executed pantomimes of familiar tool actions. A novel approach for decoding object knowledge was used in which classifiers were trained on one pair of objects and then tested on a distinct pair; this permitted a measurement of classification accuracy over and above object-specific information. Region of interest (ROI) analyses showed that object-directed actions could be decoded in tool-preferring regions of both parietal and temporal cortex, while no independently defined tool-preferring ROI showed successful decoding of object function. However, a whole-brain searchlight analysis revealed that while frontoparietal motor and peri-motor regions are engaged in the representation of object-directed actions, medial temporal lobe areas in the left hemisphere are involved in the representation of function knowledge. These results indicate that both action and function knowledge are represented in a topographically coherent manner that is amenable to study with multivariate approaches, and that the left medial temporal cortex represents knowledge of object function.

Distinct Regions of Right Temporal Cortex Are Associated with Biological and Human–Agent Motion: Functional Magnetic Resonance Imaging and Neuropsychological Evidence

In human lateral temporal cortex, some regions show specific sensitivity to human motion. Here we examine whether such effects reflect a general biological–nonbiological organizational principle or a process specific to human–agent processing by comparing processing of human, animal, and tool motion in a functional magnetic resonance imaging (fMRI) experiment with healthy participants and a voxel-based lesion-symptom mapping (VLSM) study of patients with brain damage (77 stroke patients). The fMRI experiment revealed that in the lateral temporal cortex, the posterior superior temporal sulcus shows a preference for human and animal motion, whereas the middle part of the right superior temporal sulcus/gyrus (mSTS/STG) shows a preference for human and functional tool motion. VLSM analyses also revealed that damage to this right mSTS/STG region led to more severe impairment in the recognition of human and functional tool motion relative to animal motion, indicating the causal role of this brain area in human–agent motion processing. The findings for the right mSTS/STG cannot be reduced to a preference for articulated motion or processing of social variables since neither factor is involved in functional tool motion recognition. We conclude that a unidimensional biological–nonbiological distinction cannot fully explain the visual motion effects in lateral temporal cortex. Instead, the results suggest the existence of distinct components in right posterior temporal cortex and mSTS/STG that are associated, respectively, with biological motion and human–agent motion processing.

Connectivity-based constraints on category-specificity in the ventral object processing pathway

ABSTRACT Recent efforts to characterize visual object representations in the ventral object processing pathway in the human brain have led to contrasting proposals about the causes of neural specificity for different categories. Here we use multivariate techniques in a novel way to relate patterns of functional connectivity to patterns of stimulus preferences. Stimulus preferences were measured throughout the ventral stream to tools, animals, faces and places; separately, we measured the strength of functional connectivity of each voxel in the ventral stream to category‐preferring regions outside the ventral stream. Multivariate analyses were then performed over ventral stream voxels, relating ‘category‐preferences’ to ‘functional connectivity preferences’. We show that the relation of those two measures doubly dissociates ‘tools’ and ‘places’, within what is ostensibly ‘place’ selective cortex (parahippocampal gyrus). Specifically, in the parahippocampal gyrus, functional connectivity to the left inferior parietal lobule is selectively related to stimulus preferences for tools (and not places), while functional connectivity to retrosplenial cortex is selectively related to place preferences (and not tools preferences). These findings indicate that functional connectivity can be used to index representational content rather than just provide an understanding of ‘which regions are talking to which regions’. We suggest that the connectivity of the brain is what drives category‐specificity in the ventral stream, and that if this is correct, then understanding the connectivity of the ventral stream will be key to understanding the causes and function of category‐specific neural organization. HighlightsA multivariate relation between stimulus preferences and functional connectivity.Place preferences are selectively related to connectivity to retrosplenial cortex.Tool preferences are selectively related to connectivity with the inferior parietal lobule.Connectivity shapes the organization of the ventral stream.Non‐spatial representational boundaries within ventral stream between categories.

Topographical functional connectivity patterns exist in the congenitally, prelingually deaf

Congenital deafness causes large changes in the auditory cortex structure and function, such that without early childhood cochlear-implant, profoundly deaf children do not develop intact, high-level, auditory functions. But how is auditory cortex organization affected by congenital, prelingual, and long standing deafness? Does the large-scale topographical organization of the auditory cortex develop in people deaf from birth? And is it retained despite cross-modal plasticity? We identified, using fMRI, topographic tonotopy-based functional connectivity (FC) structure in humans in the core auditory cortex, its extending tonotopic gradients in the belt and even beyond that. These regions show similar FC structure in the congenitally deaf throughout the auditory cortex, including in the language areas. The topographic FC pattern can be identified reliably in the vast majority of the deaf, at the single subject level, despite the absence of hearing-aid use and poor oral language skills. These findings suggest that large-scale tonotopic-based FC does not require sensory experience to develop, and is retained despite life-long auditory deprivation and cross-modal plasticity. Furthermore, as the topographic FC is retained to varying degrees among the deaf subjects, it may serve to predict the potential for auditory rehabilitation using cochlear implants in individual subjects.

Decoding Visual Location From Neural Patterns in the Auditory Cortex of the Congenitally Deaf.

Sensory cortices of individuals who are congenitally deprived of a sense can exhibit considerable plasticity and be recruited to process information from the senses that remain intact. Here, we explored whether the auditory cortex of congenitally deaf individuals represents visual field location of a stimulus—a dimension that is represented in early visual areas. We used functional MRI to measure neural activity in auditory and visual cortices of congenitally deaf and hearing humans while they observed stimuli typically used for mapping visual field preferences in visual cortex. We found that the location of a visual stimulus can be successfully decoded from the patterns of neural activity in auditory cortex of congenitally deaf but not hearing individuals. This is particularly true for locations within the horizontal plane and within peripheral vision. These data show that the representations stored within neuroplastically changed auditory cortex can align with dimensions that are typically represented in visual cortex.

Multisensory part-based representations of objects in human lateral occipital cortex

The format of high-level object representations in temporal-occipital cortex is a fundamental and as yet unresolved issue. Here we use fMRI to show that human lateral occipital cortex (LOC) encodes novel 3-D objects in a multisensory and part-based format. We show that visual and haptic exploration of objects leads to similar patterns of neural activity in human LOC and that the shared variance between visually and haptically induced patterns of BOLD contrast in LOC reflects the part structure of the objects. We also show that linear classifiers trained on neural data from LOC on a subset of the objects successfully predict a novel object based on its component part structure. These data demonstrate a multisensory code for object representations in LOC that specifies the part structure of objects.

Abstract Representations of Object-Directed Action in the Left Inferior Parietal Lobule

Prior neuroimaging and neuropsychological research indicates that the left inferior parietal lobule in the human brain is a critical substrate for representing object manipulation knowledge. In the present functional MRI study we used multivoxel pattern analyses to test whether action similarity among objects can be decoded in the inferior parietal lobule independent of the task applied to objects (identification or pantomime) and stimulus format in which stimuli are presented (pictures or printed words). Participants pantomimed the use of objects, cued by printed words, or identified pictures of objects. Classifiers were trained and tested across task (e.g., training data: pantomime; testing data: identification), stimulus format (e.g., training data: word format; testing format: picture) and specific objects (e.g., training data: scissors vs. corkscrew; testing data: pliers vs. screwdriver). The only brain region in which action relations among objects could be decoded across task, stimulus format and objects was the inferior parietal lobule. By contrast, medial aspects of the ventral surface of the left temporal lobe represented object function, albeit not at the same level of abstractness as actions in the inferior parietal lobule. These results suggest compulsory access to abstract action information in the inferior parietal lobe even when simply identifying objects.

Decoding intransitive actions in primary motor cortex using fMRI: toward a componential theory of ‘action primitives’ in motor cortex

ABSTRACT Multivoxel pattern analysis (MVPA) over functional MRI data can distinguish neural representational states that do not differ in their overall amplitude of BOLD contrast. Here we used MVPA to test whether simple intransitive actions can be distinguished in primary motor cortex. Participants rotated and flexed each of their extremities (hands and feet) during fMRI scanning. The primary motor cortex for the hand/wrist was functionally defined in each hemisphere in each subject. Within those subject-specific ROIs, we found that the average amplitude of BOLD contrast for two different movements of the contralateral hand (rotation, flexion) were higher than for the ipsilateral hand, as well as movements by both feet; however, there was no difference in amplitude between the two different types of movements for the contralateral hand. Using multivoxel pattern analysis (linear correlation), we were able to distinguish the two movements for the contralateral hand. These findings demonstrate that simple intransitive actions can be distinguished in primary motor areas using multivoxel pattern analysis.

Areas Recruited during Action Understanding Are Not Modulated by Auditory or Sign Language Experience

The observation of other people’s actions recruits a network of areas including the inferior frontal gyrus (IFG), the inferior parietal lobule (IPL), and posterior middle temporal gyrus (pMTG). These regions have been shown to be activated through both visual and auditory inputs. Intriguingly, previous studies found no engagement of IFG and IPL for deaf participants during non-linguistic action observation, leading to the proposal that auditory experience or sign language usage might shape the functionality of these areas. To understand which variables induce plastic changes in areas recruited during the processing of other people’s actions, we examined the effects of tasks (action understanding and passive viewing) and effectors (arm actions vs. leg actions), as well as sign language experience in a group of 12 congenitally deaf signers and 13 hearing participants. In Experiment 1, we found a stronger activation during an action recognition task in comparison to a low-level visual control task in IFG, IPL and pMTG in both deaf signers and hearing individuals, but no effect of auditory or sign language experience. In Experiment 2, we replicated the results of the first experiment using a passive viewing task. Together, our results provide robust evidence demonstrating that the response obtained in IFG, IPL, and pMTG during action recognition and passive viewing is not affected by auditory or sign language experience, adding further support for the supra-modal nature of these regions.

Hemispheric asymmetries in subcortical visual and auditory relay structures in congenital deafness

Neuroplasticity – the capacity of the brain to change as a response to internal and external pressures – has been studied from a number of different perspectives. Perhaps one of the most powerful models is the study of populations that have been congenitally deprived of a sense. It has been shown that the right Auditory Cortex (AC) of congenitally deaf humans is neuroplastically modified in order to represent visual properties of a stimulus. One unresolved question is how this visual information is routed to the AC of congenitally deaf individuals. Here, we performed volumetric analysis of subcortical auditory and visual brains regions – namely the thalamus (along with three thalamic nuclei: the pulvinar, the lateral geniculate nucleus and the medial geniculate nucleus), and the inferior and superior colliculi – in deaf and hearing participants in order to identify which structures may be responsible for relaying visual information toward the altered AC. Because there is a hemispheric asymmetry in the neuroplastic changes observed in the AC of the congenitally deaf, we reasoned that subcortical structures that also showed a similar asymmetry in their total volume could have been enlisted in the effort of relaying visual information to the neuroplastically altered right AC. We show that for deaf, but not for hearing individuals, the right thalamus, right lateral geniculate nucleus and right inferior colliculus are larger than their left counterparts. These results suggest that these subcortical structures may be responsible for rerouting visual information to the AC in congenital deafness.