Rutvik H. Desai

Where Is the Semantic System? A Critical Review and Meta-Analysis of 120 Functional Neuroimaging Studies

Semantic memory refers to knowledge about people, objects, actions, relations, self, and culture acquired through experience. The neural systems that store and retrieve this information have been studied for many years, but a consensus regarding their identity has not been reached. Using strict inclusion criteria, we analyzed 120 functional neuroimaging studies focusing on semantic processing. Reliable areas of activation in these studies were identified using the activation likelihood estimate (ALE) technique. These activations formed a distinct, left-lateralized network comprised of 7 regions: posterior inferior parietal lobe, middle temporal gyrus, fusiform and parahippocampal gyri, dorsomedial prefrontal cortex, inferior frontal gyrus, ventromedial prefrontal cortex, and posterior cingulate gyrus. Secondary analyses showed specific subregions of this network associated with knowledge of actions, manipulable artifacts, abstract concepts, and concrete concepts. The cortical regions involved in semantic processing can be grouped into 3 broad categories: posterior multimodal and heteromodal association cortex, heteromodal prefrontal cortex, and medial limbic regions. The expansion of these regions in the human relative to the nonhuman primate brain may explain uniquely human capacities to use language productively, plan, solve problems, and create cultural and technological artifacts, all of which depend on the fluid and efficient retrieval and manipulation of semantic knowledge.

The neurobiology of semantic memory

Semantic memory includes all acquired knowledge about the world and is the basis for nearly all human activity, yet its neurobiological foundation is only now becoming clear. Recent neuroimaging studies demonstrate two striking results: the participation of modality-specific sensory, motor, and emotion systems in language comprehension, and the existence of large brain regions that participate in comprehension tasks but are not modality-specific. These latter regions, which include the inferior parietal lobe and much of the temporal lobe, lie at convergences of multiple perceptual processing streams. These convergences enable increasingly abstract, supramodal representations of perceptual experience that support a variety of conceptual functions including object recognition, social cognition, language, and the remarkable human capacity to remember the past and imagine the future.

A new method for improving functional-to-structural MRI alignment using local Pearson correlation

Accurate registration of Functional Magnetic Resonance Imaging (FMRI) T2-weighted volumes to same-subject high-resolution T1-weighted structural volumes is important for Blood Oxygenation Level Dependent (BOLD) FMRI and crucial for applications such as cortical surface-based analyses and pre-surgical planning. Such registration is generally implemented by minimizing a cost functional, which measures the mismatch between two image volumes over the group of proper affine transformations. Widely used cost functionals, such as mutual information (MI) and correlation ratio (CR), appear to yield decent alignments when visually judged by matching outer brain contours. However, close inspection reveals that internal brain structures are often significantly misaligned. Poor registration is most evident in the ventricles and sulcal folds, where CSF is concentrated. This observation motivated our development of an improved modality-specific cost functional which uses a weighted local Pearson coefficient (LPC) to align T2- and T1-weighted images. In the absence of an alignment gold standard, we used three human observers blinded to registration method to provide an independent assessment of the quality of the registration for each cost functional. We found that LPC performed significantly better (p<0.001) than generic cost functionals including MI and CR. Generic cost functionals were very often not minimal near the best alignment, thereby suggesting that optimization is not the cause of their failure. Lastly, we emphasize the importance of precise visual inspection of alignment quality and present an automated method for generating composite images that help capture errors of misalignment.

Neural Systems for Reading Aloud: A Multiparametric Approach

Reading aloud involves computing the sound of a word from its visual form. This may be accomplished 1) by direct associations between spellings and phonology and 2) by computation from orthography to meaning to phonology. These components have been studied in behavioral experiments examining lexical properties such as word frequency; length in letters or phonemes; spelling–sound consistency; semantic factors such as imageability, measures of orthographic, or phonological complexity; and others. Effects of these lexical properties on specific neural systems, however, are poorly understood, partially because high intercorrelations among lexical factors make it difficult to determine if they have independent effects. We addressed this problem by decorrelating several important lexical properties through careful stimulus selection. Functional magnetic resonance imaging data revealed distributed neural systems for mapping orthography directly to phonology, involving left supramarginal, posterior middle temporal, and fusiform gyri. Distinct from these were areas reflecting semantic processing, including left middle temporal gyrus/inferior-temporal sulcus, bilateral angular gyrus, and precuneus/posterior cingulate. Left inferior frontal regions generally showed increased activation with greater task load, suggesting a more general role in attention, working memory, and executive processes. These data offer the first clear evidence, in a single study, for the separate neural correlates of orthography–phonology mapping and semantic access during reading aloud.

The neural career of sensory-motor metaphors

The role of sensory-motor systems in conceptual understanding has been controversial. It has been proposed that many abstract concepts are understood metaphorically through concrete sensory-motor domains such as actions. Using fMRI, we compared neural responses with literal action (Lit; The daughter grasped the flowers), metaphoric action (Met; The public grasped the idea), and abstract (Abs; The public understood the idea) sentences of varying familiarity. Both Lit and Met sentences activated the left anterior inferior parietal lobule, an area involved in action planning, with Met sentences also activating a homologous area in the right hemisphere, relative to Abs sentences. Both Met and Abs sentences activated the left superior temporal regions associated with abstract language. Importantly, activation in primary motor and biological motion perception regions was inversely correlated with Lit and Met familiarity. These results support the view that the understanding of metaphoric action retains a link to sensory-motor systems involved in action performance. However, the involvement of sensory-motor systems in metaphor understanding changes through a gradual abstraction process whereby relatively detailed simulations are used for understanding unfamiliar metaphors, and these simulations become less detailed and involve only secondary motor regions as familiarity increases. Consistent with these data, we propose that anterior inferior parietal lobule serves as an interface between sensory-motor and conceptual systems and plays an important role in both domains. The similarity of abstract and metaphoric sentences in the activation of left superior temporal regions suggests that action metaphor understanding is not completely based on sensory-motor simulations but relies also on abstract lexical-semantic codes.

Activation of Sensory–Motor Areas in Sentence Comprehension

The sensory-motor account of conceptual processing suggests that modality-specific attributes play a central role in the organization of object and action knowledge in the brain. An opposing view emphasizes the abstract, amodal, and symbolic character of concepts, which are thought to be represented outside the brains sensory-motor systems. We conducted a functional magnetic resonance imaging study in which the participants listened to sentences describing hand/arm action events, visual events, or abstract behaviors. In comparison to visual and abstract sentences, areas associated with planning and control of hand movements, motion perception, and vision were activated when understanding sentences describing actions. Sensory-motor areas were activated to a greater extent also for sentences with actions that relied mostly on hands, as opposed to arms. Visual sentences activated a small area in the secondary visual cortex, whereas abstract sentences activated superior temporal and inferior frontal regions. The results support the view that linguistic understanding of actions partly involves imagery or simulation of actions, and relies on some of the same neural substrate used for planning, performing, and perceiving actions.

Volumetric vs. surface-based alignment for localization of auditory cortex activation.

The high degree of intersubject structural variability in the human brain is an obstacle in combining data across subjects in functional neuroimaging experiments. A common method for aligning individual data is normalization into standard 3D stereotaxic space. Since the inherent geometry of the cortex is that of a 2D sheet, higher precision can potentially be achieved if the intersubject alignment is based on landmarks in this 2D space. To examine the potential advantage of surface-based alignment for localization of auditory cortex activation, and to obtain high-resolution maps of areas activated by speech sounds, fMRI data were analyzed from the left hemisphere of subjects tested with phoneme and tone discrimination tasks. We compared Talairach stereotaxic normalization with two surface-based methods: Landmark Based Warping, in which landmarks in the auditory cortex were chosen manually, and Automated Spherical Warping, in which hemispheres were aligned automatically based on spherical representations of individual and average brains. Examination of group maps generated with these alignment methods revealed superiority of the surface-based alignment in providing precise localization of functional foci and in avoiding mis-registration due to intersubject anatomical variability. Human left hemisphere cortical areas engaged in complex auditory perception appear to lie on the superior temporal gyrus, the dorsal bank of the superior temporal sulcus, and the lateral third of Heschls gyrus.

Specialization along the Left Superior Temporal Sulcus for Auditory Categorization

The affinity and temporal course of functional fields in middle and posterior superior temporal cortex for the categorization of complex sounds was examined using functional magnetic resonance imaging (fMRI) and event-related potentials (ERPs) recorded simultaneously. Data were compared before and after subjects were trained to categorize a continuum of unfamiliar nonphonemic auditory patterns with speech-like properties (NP) and a continuum of familiar phonemic patterns (P). fMRI activation for NP increased after training in left posterior superior temporal sulcus (pSTS). The ERP P2 response to NP also increased with training, and its scalp topography was consistent with left posterior superior temporal generators. In contrast, the left middle superior temporal sulcus (mSTS) showed fMRI activation only for P, and this response was not affected by training. The P2 response to P was also independent of training, and its estimated source was more anterior in left superior temporal cortex. Results are consistent with a role for left pSTS in short-term representation of relevant sound features that provide the basis for identifying newly acquired sound categories. Categorization of highly familiar phonemic patterns is mediated by long-term representations in left mSTS. Results provide new insight regarding the function of ventral and dorsal auditory streams.

Left posterior temporal regions are sensitive to auditory categorization

Recent studies suggest that the left superior temporal gyrus and sulcus (LSTG/S) play a role in speech perception, although the precise function of these areas remains unclear. Here, we test the hypothesis that regions in the LSTG/S play a role in the categorization of speech phonemes, irrespective of the acoustic properties of the sounds and prior experience of the listener with them. We examined changes in functional magnetic resonance imaging brain activation related to a perceptual shift from nonphonetic to phonetic analysis of sine-wave speech analogs. Subjects performed an identification task before scanning and a discrimination task during scanning with phonetic (P) and nonphonetic (N) sine-wave sounds, both before (Pre) and after (Post) being exposed to the phonetic properties of the P sounds. Behaviorally, experience with the P sounds induced categorical identification of these sounds. In the PostP > PreP and PostP > PostN contrasts, an area in the posterior LSTG/S was activated. For both P and N sounds, the activation in this region was correlated with the degree of categorical identification in individual subjects. The results suggest that these areas in the posterior LSTG/S are sensitive neither to the acoustic properties of speech nor merely to the presence of phonetic information, but rather to the listeners awareness of category representations for auditory inputs.

Attentional and linguistic interactions in speech perception.

The role of attention in speech comprehension is not well understood. We used fMRI to study the neural correlates of auditory word, pseudoword, and nonspeech (spectrally rotated speech) perception during a bimodal (auditory, visual) selective attention task. In three conditions, Attend Auditory (ignore visual), Ignore Auditory (attend visual), and Visual (no auditory stimulation), 28 subjects performed a one-back matching task in the assigned attended modality. The visual task, attending to rapidly presented Japanese characters, was designed to be highly demanding in order to prevent attention to the simultaneously presented auditory stimuli. Regardless of stimulus type, attention to the auditory channel enhanced activation by the auditory stimuli (Attend Auditory>Ignore Auditory) in bilateral posterior superior temporal regions and left inferior frontal cortex. Across attentional conditions, there were main effects of speech processing (word+pseudoword>rotated speech) in left orbitofrontal cortex and several posterior right hemisphere regions, though these areas also showed strong interactions with attention (larger speech effects in the Attend Auditory than in the Ignore Auditory condition) and no significant speech effects in the Ignore Auditory condition. Several other regions, including the postcentral gyri, left supramarginal gyrus, and temporal lobes bilaterally, showed similar interactions due to the presence of speech effects only in the Attend Auditory condition. Main effects of lexicality (word>pseudoword) were isolated to a small region of the left lateral prefrontal cortex. Examination of this region showed significant word>pseudoword activation only in the Attend Auditory condition. Several other brain regions, including left ventromedial frontal lobe, left dorsal prefrontal cortex, and left middle temporal gyrus, showed Attention x Lexicality interactions due to the presence of lexical activation only in the Attend Auditory condition. These results support a model in which neutral speech presented in an unattended sensory channel undergoes relatively little processing beyond the early perceptual level. Specifically, processing of phonetic and lexical-semantic information appears to be very limited in such circumstances, consistent with prior behavioral studies.