Gina R. Kuperberg

Neural mechanisms of language comprehension: Challenges to syntax

In 1980, the N400 event-related potential was described in association with semantic anomalies within sentences. When, in 1992, a second waveform, the P600, was reported in association with syntactic anomalies and ambiguities, the story appeared to be complete: the brain respected a distinction between semantic and syntactic representation and processes. Subsequent studies showed that the P600 to syntactic anomalies and ambiguities was modulated by lexical and discourse factors. Most surprisingly, more than a decade after the P600 was first described, a series of studies reported that semantic verb-argument violations, in the absence of any violations or ambiguities of syntax can evoke robust P600 effects and no N400 effects. These observations have raised fundamental questions about the relationship between semantic and syntactic processing in the brain. This paper provides a comprehensive review of the recent studies that have demonstrated P600s to semantic violations in light of several proposed triggers: semantic-thematic attraction, semantic associative relationships, animacy and semantic-thematic violations, plausibility, task, and context. I then discuss these findings in relation to a unifying theory that attempts to bring some of these factors together and to link the P600 produced by semantic verb-argument violations with the P600 evoked by unambiguous syntactic violations and syntactic ambiguities. I suggest that normal language comprehension proceeds along at least two competing neural processing streams: a semantic memory-based mechanism, and a combinatorial mechanism (or mechanisms) that assigns structure to a sentence primarily on the basis of morphosyntactic rules, but also on the basis of certain semantic-thematic constraints. I suggest that conflicts between the different representations that are output by these distinct but interactive streams lead to a continued combinatorial analysis that is reflected by the P600 effect. I discuss some of the implications of this non-syntactocentric, dynamic model of language processing for understanding individual differences, language processing disorders and the neuroanatomical circuitry engaged during language comprehension. Finally, I suggest that that these two processing streams may generalize beyond the language system to real-world visual event comprehension.

Common and Distinct Neural Substrates for Pragmatic, Semantic, and Syntactic Processing of Spoken Sentences: An fMRI Study

Extracting meaning from speech requires the use of pragmatic, semantic, and syntactic information. A central question is: Does the processing of these different types of linguistic information have common or distinct neuroanatomical substrates? We addressed this issue using functional magnetic resonance imaging (fMRI) to measure neural activity when subjects listened to spoken normal sentences contrasted with sentences that had either (A) pragmatical, (B) semantic (selection restriction), or (C) syntactic (subcategorical) violations sentences. All three contrasts revealed robust activation of the left-inferior-temporal/fusiform gyrus. Activity in this area was also observed in a combined analysis of all three experiments, suggesting that it was modulated by all three types of linguistic violation. Planned statistical comparisons between the three experiments revealed (1) a greater difference between conditions in activation of the left-superior-temporal gyrus for the pragmatic experiment than the semantic/syntactic experiments; (2) a greater difference between conditions in activation of the right-superior and middletemporal gyrus in the semantic experiment than in the syntactic experiment; and (3) no regions activated to a greater degree in the syntactic experiment than in the semantic experiment. These data show that, while left- and right-superior-temporal regions may be differentially involved in processing pragmatic and lexico-semantic information within sentences, the left-inferior-temporal/fusiform gyrus is involved in processing all three types of linguistic information. We suggest that this region may play a key role in using pragmatic, semantic (selection restriction), and subcategorical information to construct a higher representation of meaning of sentences.

Schizophrenia and cognitive function

Schizophrenia is often associated with cognitive deficits, particularly within the domains of memory and language. Specific cognitive deficits have recently been linked to psychotic phenomena, including verbal hallucinations and disorganized speech. Impairments of working and semantic memory are primarily due to dysfunction of the frontal cortex, temporal cortex, and hippocampus. Cognitive skills in schizophrenia predict social functioning and may serve as outcome measures in the development of effective treatment strategies.

Distinct Patterns of Neural Modulation during the Processing of Conceptual and Syntactic Anomalies

The aim of this study was to gain further insights into how the brain distinguishes between meaning and syntax during language comprehension. Participants read and made plausibility judgments on sentences that were plausible, morpho-syntactically anomalous, or pragmatically anomalous. In an event-related potential (ERP) experiment, morphosyntactic and pragmatic violations elicited significant P600 and N400 effects, respectively, replicating previous ERP studies that have established qualitative differences in processing conceptually and syntactic anomalies. Our main focus was a functional magnetic resonance imaging (fMRI) study in which the same subjects read the same sentences presented in the same pseudorandomized sequence while performing the same task as in the ERP experiment. Rapid-presentation event-related fMRI methods allowed us to estimate the hemodynamic response at successive temporal windows as the sentences unfolded word by word, without assumptions about the shape of the underlying response function. Relative to nonviolated sentences, the pragmatic anomalies were associated with an increased hemodynamic response in left temporal and inferior frontal regions and a decreased response in the right medial parietal cortex. Relative to nonviolated sentences, the morphosyntactic anomalies were associated with an increased response in bilateral medial and lateral parietal regions and a decreased response in left temporal and inferior frontal regions. Thus, overlapping neural networks were modulated in opposite directions to the two types of anomaly. These fMRI findings document both qualitative and quantitative differences in how the brain distinguishes between these two types of anomalies. This suggests that morphosyntactic and pragmatic information can be processed in different ways but by the same neural systems.

Semantic integration in videos of real-world events: An electrophysiological investigation

Event-related potentials (ERPs) discriminated between contextually appropriate and inappropriate objects appearing in video film clips of common activities. Incongruent objects elicited a larger negative-going deflection, which was similar to the N400 component described previously in association with words and static pictures and which has been argued to reflect the integration of semantic information into a mental representation of the preceding context. The onset of this potential occurred shortly after object presentation, indicating that semantic integration is a rapid online component of real-world perception. In addition, the anomalies in movies evoked a large late positive potential at posterior regions, suggesting that in event perception, semantic incongruity may trigger cognitive processes other than those mediating pure semantic integration.

Two neurocognitive mechanisms of semantic integration during the comprehension of visual real-world events

How do comprehenders build up overall meaning representations of visual real-world events? This question was examined by recording event-related potentials (ERPs) while participants viewed short, silent movie clips depicting everyday events. In two experiments, it was demonstrated that presentation of the contextually inappropriate information in the movie endings evoked an anterior negativity. This effect was similar to the N400 component whose amplitude has been previously reported to inversely correlate with the strength of semantic relationship between the context and the eliciting stimulus in word and static picture paradigms. However, a second, somewhat later, ERP componenta posterior late positivitywas evoked specifically when target objects presented in the movie endings violated goal-related requirements of the action constrained by the scenario context (e.g., an electric iron that does not have a sharp-enough edge was used in place of a knife in a cutting bread scenario context). These findings suggest that comprehension of the visual real world might be mediated by two neurophysiologically distinct semantic integration mechanisms. The first mechanism, reflected by the anterior N400-like negativity, maps the incoming information onto the connections of various strengths between concepts in semantic memory. The second mechanism, reflected by the posterior late positivity, evaluates the incoming information against the discrete requirements of real-world actions. We suggest that there may be a tradeoff between these mechanisms in their utility for integrating across people, objects, and actions during event comprehension, in which the first mechanism is better suited for familiar situations, and the second mechanism is better suited for novel situations.

Vascular responses to syntactic processing: event-related fMRI study of relative clauses.

Event‐related functional magnetic resonance imaging was used to investigate the localization of syntactic processing in sentence comprehension. Matched pairs of sentences containing identical lexical items were compared. One member of the pair consisted of a syntactically simpler sentence, containing a subject relativized clause. The second member of the pair consisted of a syntactically more complex sentence, containing an object relativized clause. Ten subjects made plausibility judgments about the sentences, which were presented one word at a time on a computer screen. There was an increase in BOLD hemodynamic signal in response to the presentation of all sentences compared to fixation in both right and left occipital cortex, the left perisylvian cortex, and the left premotor and motor areas. BOLD signal increased in the left angular gyrus when subjects processed the complex portion of syntactically more complex sentences. This study shows that a hemodynamic response associated with processing the syntactically complex portions of a sentence can be localized to one part of the dominant perisylvian association cortex. Hum. Brain Mapping 15:26–38, 2001.

What do we mean by prediction in language comprehension

ABSTRACT We consider several key aspects of prediction in language comprehension: its computational nature, the representational level(s) at which we predict, whether we use higher-level representations to predictively pre-activate lower level representations, and whether we “commit” in any way to our predictions, beyond pre-activation. We argue that the bulk of behavioural and neural evidence suggests that we predict probabilistically and at multiple levels and grains of representation. We also argue that we can, in principle, use higher-level inferences to predictively pre-activate information at multiple lower representational levels. We suggest that the degree and level of predictive pre-activation might be a function of its expected utility, which, in turn, may depend on comprehenders’ goals and their estimates of the relative reliability of their prior knowledge and the bottom-up input. Finally, we argue that all these properties of language understanding can be naturally explained and productively explored within a multi-representational hierarchical actively generative architecture whose goal is to infer the message intended by the producer, and in which predictions play a crucial role in explaining the bottom-up input.

Making sense of discourse: An fMRI study of causal inferencing across sentences

To build up coherence between sentences (comprehend discourse), we must draw inferences, i.e. activate and integrate information that is not actually stated. We used event-related fMRI to determine the localization and extent of brain activity mediating causal inferencing across short, three-sentence scenarios. Participants read and made causal coherence judgments to sentences that were highly causally related, intermediately related or unrelated to their preceding two-sentence contexts. The highly related and intermediately related scenarios were matched in terms of semantic similarities between their individual component words. A pre-rating study established that causal inferences were generated to the intermediately related but not to the highly related or unrelated scenarios. In the scanner, sentences that were intermediately related (relative to highly related or unrelated) to their preceding contexts were associated with longer judgment reaction times and sustained increases in hemodynamic activity within left lateral temporal/inferior parietal/prefrontal cortices, the right inferior prefrontal gyrus and bilateral superior medial prefrontal cortices. In contrast, sentences that were unrelated (relative to highly related) to their preceding contexts were associated with only transient increases in activity (at, but not after, the peak of the hemodynamic response) within the right lateral temporal cortex and the right inferior prefrontal gyrus. These data suggest that, to make sense of discourse, we activate a large bilateral cortical network in response to what is not explicitly stated. We suggest that this network reflects the activation, retrieval and integration of information from long-term semantic memory into incoming discourse structure during causal inferencing.

Neural correlates of processing syntactic, semantic, and thematic relationships in sentences

Event-related potentials were measured as subjects read sentences presented word by word. A small N400 and a robust P600 effect were elicited by verbs that assigned the thematic role of Agent to their preceding noun-phrase argument when this argument was inanimate in nature. The amplitude of the P600, but not the N400, was modulated by the transitivity of the critical verbs and by plausibility ratings of passivised versions of these sentences (reflecting the fit between the critical verb and the inanimate noun-phrase as the verbs Theme). The P600 was similar in scalp distribution although smaller in amplitude, than that elicited by verbs with morphosyntactic violations. Pragmatically unlikely verbs that did not violate thematic constraints elicited a larger N400 but no P600 effect. These findings support the theory that the cost of syntactic processing on a verb is influenced by the precise thematic relationships between that verb and its preceding arguments.