Robert A. Mason

Predicting human brain activity associated with the meanings of nouns.

The question of how the human brain represents conceptual knowledge has been debated in many scientific fields. Brain imaging studies have shown that different spatial patterns of neural activation are associated with thinking about different semantic categories of pictures and words (for example, tools, buildings, and animals). We present a computational model that predicts the functional magnetic resonance imaging (fMRI) neural activation associated with words for which fMRI data are not yet available. This model is trained with a combination of data from a trillion-word text corpus and observed fMRI data associated with viewing several dozen concrete nouns. Once trained, the model predicts fMRI activation for thousands of other concrete nouns in the text corpus, with highly significant accuracies over the 60 nouns for which we currently have fMRI data.

Theory of Mind Disruption and Recruitment of the Right Hemisphere during Narrative Comprehension in Autism.

The intersection of Theory of Mind (ToM) processing and complex narrative comprehension in high functioning autism was examined by comparing cortical activation during the reading of passages that required inferences based on either intentions, emotional states, or physical causality. Right hemisphere activation was substantially greater for all sentences in the autism group than in a matched control group suggesting decreased LH capacity in autism resulting in a spillover of processing to RH homologs. Moreover, the ToM network was disrupted. The autism group showed similar activation for all inference types in the right temporo-parietal component of the ToM network whereas the control participants selectively activated this network only when appropriate. The autism group had lower functional connectivity within the ToM network and also between the ToM and a left hemisphere language network. Furthermore, the within-network functional connectivity in autism was correlated with the size of the anterior portion of the corpus callosum.

How the Brain Processes Causal Inferences in Text A Theoretical Account of Generation and Integration Component Processes Utilizing Both Cerebral Hemispheres

Theoretical models of text processing, such as the construction-integration framework, pose fundamental questions about causal inference making that are not easily addressed by behavioral studies. In particular, a common result is that causal relatedness has a different effect on text reading times than on memory for the text: Whereas reading times increase linearly as causal relatedness decreases, memory for the text is best for events that are related by a moderate degree of causal relatedness and is poorer for events with low and high relatedness. Our functional magnetic resonance imaging study of the processing of two-sentence passages that varied in their degree of causal relatedness suggests that the inference process can be analyzed into two components, generation and integration, that are subserved by two large-scale cortical networks (a reasoning system in dorsolateral prefrontal cortex and the right-hemisphere language areas). These two cortical networks, which are distinguishable from the classical left-hemisphere language areas, approximately correspond to the two functional relations observed in the behavioral results.

Lexical ambiguity in sentence comprehension

An event-related fMRI paradigm was used to investigate brain activity during the reading of sentences containing either a lexically ambiguous word or an unambiguous control word. Higher levels of activation occurred during the reading of sentences containing a lexical ambiguity. Furthermore, the activated cortical network differed, depending on: (1) whether the sentence contained a balanced (i.e., both meanings equally likely) or a biased (i.e., one meaning more likely than other meanings) ambiguous word; and, (2) the working memory capacity of the individual as assessed by reading span. The findings suggest that encountering a lexical ambiguity is dealt with by activating multiple meanings utilizing processes involving both hemispheres. When an early interpretation of a biased ambiguous word is later disambiguated to the subordinate meaning, the superior frontal cortex activates in response to the coherence break and the right inferior frontal gyrus and the insula activate, possibly to suppress the incorrect interpretation. Negative correlations between reading span scores and activation in the right hemisphere for both types of ambiguous words suggest that readers with lower spans are more likely to involve show right hemisphere involvement in the processing of the ambiguity. A positive correlation between reading span scores and insula activation appearing only for biased sentences disambiguated to the subordinate meaning indicates that individuals with higher spans were more likely to initially maintain both meanings and as a result had to suppress the unintended dominant meaning.

Commonality of neural representations of words and pictures

In this work we explore whether the patterns of brain activity associated with thinking about concrete objects are dependent on stimulus presentation format, whether an object is referred to by a written or pictorial form. Multi-voxel pattern analysis methods were applied to brain imaging (fMRI) data to identify the item category associated with brief viewings of each of 10 words (naming 5 tools and 5 dwellings) and, separately, with brief viewings of each of 10 pictures (line drawings) of the objects named by the words. These methods were able to identify the category of the picture the participant was viewing, based on neural activation patterns observed during word-viewing, and identify the category of the word the participant was viewing, based on neural activation patterns observed during picture-viewing, using data from only that participant or only from other participants. These results provide an empirical demonstration of object category identification across stimulus formats and across participants. In addition, we were able to identify the category of the word that the participant was viewing based on the patterns of neural activation generated during word-viewing by that participant or by all other participants. Similarly, we were able to identify with even higher accuracy the category of the picture the participant was viewing, based on the patterns of neural activation demonstrated during picture-viewing by that participant or by all other participants. The brain locations that were important for category identification were similar across participants and were distributed throughout the cortex where various object properties might be neurally represented. These findings indicate consistent triggering of semantic representations using different stimulus formats and suggest the presence of stable, distributed, and identifiable neural states that are common to pictorial and verbal input referring to object categories.

Ambiguity in the Brain: What Brain Imaging Reveals About the Processing of Syntactically Ambiguous Sentences

Two fMRI studies investigated the time course and amplitude of brain activity in language-related areas during the processing of syntactically ambiguous sentences. In Experiment 1, higher levels of activation were found during the reading of unpreferred syntactic structures as well as more complex structures. In Experiments 2A and 2B higher levels of brain activation were found for ambiguous sentences compared with unambiguous sentences matched for syntactic complexity, even when the ambiguities were resolved in favor of the preferred syntactic construction (despite the absence of this difference in previous reading time results). Although results can be reconciled with either serial or parallel models of sentence parsing, they arguably fit better into the parallel framework. Serial models can admittedly be made consistent but only by including a parallel component. The fMRI data indicate the involvement of a parallel component in syntactic parsing that might be either a selection mechanism or a construction of multiple parses.

Chapter 19 – Neuroimaging Contributions to the Understanding of Discourse Processes

Publisher Summary This chapter highlights some key neuroimaging studies of discourse processing. Some of the components of discourse processing revealed by neuroimaging research, like protagonist perspective monitoring, are relatively new to the discourse processing theory. At the same time, there is uncertainty about the reality of these networks and about their anatomical location. Moreover, these networks must function in interaction with somewhat lower level comprehension processes that operate at the lexical and sentence level. Dating back to Brocas and Wernickes findings on brain-damaged patients with specific language deficits in the late 1800s, psychologists have had some idea of the brains functioning as a language processing mechanism. Discourse theories become critical in developing the understanding of the cortical discourse processing network. In addition, neuroimaging research has led to the development of several new discourse theories such as the coarse coding theory of right hemisphere processing, the dynamic recruitment of networks in response to text constraints, a Theory of Mind system responsible for awareness of different perspectives, and the spillover of processing to other differential specialized networks in response to capacity utilization.

Identifying bilingual semantic neural representations across languages

The goal of the study was to identify the neural representation of a nouns meaning in one language based on the neural representation of that same noun in another language. Machine learning methods were used to train classifiers to identify which individual noun bilingual participants were thinking about in one language based solely on their brain activation in the other language. The study shows reliable (p<.05) pattern-based classification accuracies for the classification of brain activity for nouns across languages. It also shows that the stable voxels used to classify the brain activation were located in areas associated with encoding information about semantic dimensions of the words in the study. The identification of the semantic trace of individual nouns from the pattern of cortical activity demonstrates the existence of a multi-voxel pattern of activation across the cortex for a single noun common to both languages in bilinguals.

Semantic and Episodic Effects on Bridging Inferences

In 3 experiments, participants read pairs of sentences requiring a bridging inference between a category in 1 sentence and a typical or atypical exemplar of that category in the following sentence. In Experiments 2 and 3, the critical pair of sentences was preceded by several lines of text in which the exemplar was introduced and then backgrounded by a shift in topic. Accessibility of the earlier mentioned exemplar was manipulated by varying the distance between the first and the subsequent mention of the exemplar, or by varying the elaboration of the exemplar when first introduced into the text. Eye fixations in Experiment 1 revealed that the bridging inference was affected by typicality, that the process began at the earliest opportunity but was not completed when the eye left the exemplar word. In Experiments 2 and 3, both typicality and episodic accessibility influenced reading time. These effects are discussed in terms of several models of the underlying process.

Brain function differences in language processing in children and adults with autism.

Comparison of brain function between children and adults with autism provides an understanding of the effects of the disorder and associated maturational differences on language processing. Functional imaging (functional magnetic resonance imaging) was used to examine brain activation and cortical synchronization during the processing of literal and ironic texts in 15 children with autism, 14 children with typical development, 13 adults with autism, and 12 adult controls. Both the children and adults with autism had lower functional connectivity (synchronization of brain activity among activated areas) than their age and ability comparison group in the left hemisphere language network during irony processing, and neither autism group had an increase in functional connectivity in response to increased task demands. Activation differences for the literal and irony conditions occurred in key language‐processing regions (left middle temporal, left pars triangularis, left pars opercularis, left medial frontal, and right middle temporal). The children and adults with autism differed from each other in the use of some brain regions during the irony task, with the adults with autism having activation levels similar to those of the control groups. Overall, the children and adults with autism differed from the adult and child controls in (a) the degree of network coordination, (b) the distribution of the workload among member nodes, and (3) the dynamic recruitment of regions in response to text content. Moreover, the differences between the two autism age groups may be indicative of positive changes in the neural function related to language processing associated with maturation and/or educational experience. Autism Res 2013, ●●: ●●–●●.