Ellen F. Lau

A cortical network for semantics: (de)constructing the N400.

Measuring event-related potentials (ERPs) has been fundamental to our understanding of how language is encoded in the brain. One particular ERP response, the N400 response, has been especially influential as an index of lexical and semantic processing. However, there remains a lack of consensus on the interpretation of this component. Resolving this issue has important consequences for neural models of language comprehension. Here we show that evidence bearing on where the N400 response is generated provides key insights into what it reflects. A neuroanatomical model of semantic processing is used as a guide to interpret the pattern of activated regions in functional MRI, magnetoencephalography and intracranial recordings that are associated with contextual semantic manipulations that lead to N400 effects.

The role of structural prediction in rapid syntactic analysis

A number of recent electrophysiological studies of sentence processing have shown that a subclass of syntactic violations elicits very rapid ERP responses, occurring within around 200 ms of the onset of the violation. Such findings raise the question of how it is possible to diagnose violations so quickly. This paper suggests that very rapid diagnosis of errors is possible specifically in situations where the diagnosis problem is tightly constrained by specific expectations generated before the critical word is presented. In an event-related potentials (ERP) study of visual sentence reading participants encountered violations of a word order constraint (...Maxs of...) that has elicited early ERP responses in previous studies. Across conditions the illicit sequence was held constant, while sentence context was used to manipulate the expectation for a noun following the possessor Maxs, by manipulating the possibility of ellipsis of the head noun. Results showed that the anterior negativity elicited by the word category violation was attenuated when the availability of ellipsis reduced the expectation for a noun in the position of the offending preposition of, with divergence between conditions starting around 200 ms after the onset of the violation. This suggests a role for structural expectations in accounting for very fast syntactic diagnosis processes.

Dissociating n400 effects of prediction from association in single-word contexts

When a word is preceded by a supportive context such as a semantically associated word or a strongly constraining sentence frame, the N400 component of the ERP is reduced in amplitude. An ongoing debate is the degree to which this reduction reflects a passive spread of activation across long-term semantic memory representations as opposed to specific predictions about upcoming input. We addressed this question by embedding semantically associated prime–target pairs within an experimental context that encouraged prediction to a greater or lesser degree. The proportion of related items was used to manipulate the predictive validity of the prime for the target while holding semantic association constant. A semantic category probe detection task was used to encourage semantic processing and to preclude the need for a motor response on the trials of interest. A larger N400 reduction to associated targets was observed in the high than the low relatedness proportion condition, consistent with the hypothesis that predictions about upcoming stimuli make a substantial contribution to the N400 effect. We also observed an earlier priming effect (205–240 msec) in the high-proportion condition, which may reflect facilitation because of form-based prediction. In summary, the results suggest that predictability modulates N400 amplitude to a greater degree than the semantic content of the context.

5: Grammatical Illusions and Selective Fallibility in Real-Time Language Comprehension

Grammatical constraints impose diverse requirements on the relations between words and phrases in a sentence. This chapter presents a preliminary profile of selective fallibility to grammatical illusions in language comprehension. It summarizes different mechanisms that comprehenders might use to access linguistic material in memory. These mechanisms present a trade-off between speed and structure-sensitivity of processing. The chapter surveys grammatical phenomena where comprehenders appear to show impressive online sensitivity (island constraints on unbounded dependencies, backwards anaphora and Principle C, and constraints on reflexives). It describes cases where comprehenders are susceptible to grammatical illusion (subject-verb agreement, case licensing, negative polarity item licensing, and comparatives). The antilocality constraint on pronouns (Principle B) is included, although its status remains uncertain. The chapter attempts a preliminary synthesis of findings on the effects of grammatical constraints on real-time language processing. Keywords: anaphora; antilocality constraint; case licensing; grammatical illusions; real-time language comprehension; selective fallibility; subject-verb agreement

Automatic Semantic Facilitation in Anterior Temporal Cortex Revealed through Multimodal Neuroimaging

A core property of human semantic processing is the rapid, facilitatory influence of prior input on extracting the meaning of what comes next, even under conditions of minimal awareness. Previous work has shown a number of neurophysiological indices of this facilitation, but the mapping between time course and localization—critical for separating automatic semantic facilitation from other mechanisms—has thus far been unclear. In the current study, we used a multimodal imaging approach to isolate early, bottom-up effects of context on semantic memory, acquiring a combination of electroencephalography (EEG), magnetoencephalography (MEG), and functional magnetic resonance imaging (fMRI) measurements in the same individuals with a masked semantic priming paradigm. Across techniques, the results provide a strikingly convergent picture of early automatic semantic facilitation. Event-related potentials demonstrated early sensitivity to semantic association between 300 and 500 ms; MEG localized the differential neural response within this time window to the left anterior temporal cortex, and fMRI localized the effect more precisely to the left anterior superior temporal gyrus, a region previously implicated in semantic associative processing. However, fMRI diverged from early EEG/MEG measures in revealing semantic enhancement effects within frontal and parietal regions, perhaps reflecting downstream attempts to consciously access the semantic features of the masked prime. Together, these results provide strong evidence that automatic associative semantic facilitation is realized as reduced activity within the left anterior superior temporal cortex between 300 and 500 ms after a word is presented, and emphasize the importance of multimodal neuroimaging approaches in distinguishing the contributions of multiple regions to semantic processing.

Spatiotemporal Signatures of Lexical–Semantic Prediction

Although there is broad agreement that top-down expectations can facilitate lexical-semantic processing, the mechanisms driving these effects are still unclear. In particular, while previous electroencephalography (EEG) research has demonstrated a reduction in the N400 response to words in a supportive context, it is often challenging to dissociate facilitation due to bottom-up spreading activation from facilitation due to top-down expectations. The goal of the current study was to specifically determine the cortical areas associated with facilitation due to top-down prediction, using magnetoencephalography (MEG) recordings supplemented by EEG and functional magnetic resonance imaging (fMRI) in a semantic priming paradigm. In order to modulate expectation processes while holding context constant, we manipulated the proportion of related pairs across 2 blocks (10 and 50% related). Event-related potential results demonstrated a larger N400 reduction when a related word was predicted, and MEG source localization of activity in this time-window (350-450 ms) localized the differential responses to left anterior temporal cortex. fMRI data from the same participants support the MEG localization, showing contextual facilitation in left anterior superior temporal gyrus for the high expectation block only. Together, these results provide strong evidence that facilitatory effects of lexical-semantic prediction on the electrophysiological response 350-450 ms postonset reflect modulation of activity in left anterior temporal cortex.

Hyper-active gap filling

Much work has demonstrated that speakers of verb-final languages are able to construct rich syntactic representations in advance of verb information. This may reflect general architectural properties of the language processor, or it may only reflect a language-specific adaptation to the demands of verb-finality. The present study addresses this issue by examining whether speakers of a verb-medial language (English) wait to consult verb transitivity information before constructing filler-gap dependencies, where internal arguments are fronted and hence precede the verb. This configuration makes it possible to investigate whether the parser actively makes representational commitments on the gap position before verb transitivity information becomes available. A key prediction of the view that rich pre-verbal structure building is a general architectural property is that speakers of verb-medial languages should predictively construct dependencies in advance of verb transitivity information, and therefore that disruption should be observed when the verb has intransitive subcategorization frames that are incompatible with the predicted structure. In three reading experiments (self-paced and eye-tracking) that manipulated verb transitivity, we found evidence for reading disruption when the verb was intransitive, although no such reading difficulty was observed when the critical verb was embedded inside a syntactic island structure, which blocks filler-gap dependency completion. These results are consistent with the hypothesis that in English, as in verb-final languages, information from preverbal noun phrases is sufficient to trigger active dependency completion without having access to verb transitivity information.

A “bag-of-arguments” mechanism for initial verb predictions

ABSTRACT Previous studies have shown that comprehenders use rich contextual information to anticipate upcoming input on the fly, but less is known about how comprehenders integrate different sources of information to generate predictions in real time. The current study examines the time course with which the lexical meaning and structural roles of preverbal arguments impact comprehenders’ lexical semantic predictions about an upcoming verb in two event-related potential (ERP) experiments that use the N400 amplitude as a measure of online predictability. Experiment 1 showed that the N400 was sensitive to predictability when the verbs cloze probability was reduced by substituting one of the arguments (e.g. “The superintendent overheard which tenant/realtor the landlord had evicted … ”), but not when the verbs cloze probability was reduced by simply swapping the roles of the arguments (e.g. “The restaurant owner forgot which customer/waitress the waitress/customer had served … ”). Experiment 2 showed that argument substitution elicited an N400 effect even when the substituted argument appeared elsewhere in the sentence, indicating that verb predictions are specifically driven by the arguments in the same clause as the verb, rather than by a simple “bag-of-words” mechanism. We propose that verb predictions initially rely on a “bag-of-arguments” mechanism, which specifically relies on the lexical meaning, but not the structural roles, of the arguments in a clause.

The role of the IFG and pSTS in syntactic prediction: Evidence from a parametric study of hierarchical structure in fMRI.

Sentences encode hierarchical structural relations among words. Several neuroimaging experiments aiming to localize combinatory operations responsible for creating this structure during sentence comprehension have contrasted short, simple phrases and sentences to unstructured controls. Some of these experiments have revealed activation in the left inferior frontal gyrus (IFG) and posterior superior temporal sulcus (pSTS), associating these regions with basic syntactic combination. However, the wide variability of these effects across studies raises questions about this interpretation. In an fMRI experiment, we provide support for an alternative hypothesis: these regions underlie top-down syntactic predictions that facilitate sentence processing but are not necessary for building syntactic structure. We presented stimuli with three levels of structure: unstructured lists, two-word phrases, and simple, short sentences; and two levels of content: natural stimuli with real words and stimuli with open-class items replaced with pseudowords (jabberwocky). While both the phrase and sentence conditions engaged syntactic combination, our experiment only encouraged syntactic prediction in the sentence condition. We found increased activity for both natural and jabberwocky sentences in the left IFG (pars triangularis and pars orbitalis) and pSTS relative to unstructured word lists and two-word phrases, but we did not find any such effects for two-word phrases relative to unstructured word lists in these areas. Our results are most consistent with the hypothesis that increased activity in IFG and pSTS for basic contrasts of structure reflects syntactic prediction. The pars opercularis of the IFG showed a response profile consistent with verbal working memory. We found incremental effects of structure in the anterior temporal lobe (ATL), and increased activation only for sentences in the angular gyrus (AG)/temporal-parietal junction (TPJ) - both regions showed these effects for stimuli with all real words. These findings support a role for the ATL in semantic combination and the AG/TPJ in thematic processing.

The Yin and the Yang of Prediction: An fMRI Study of Semantic Predictive Processing

Probabilistic prediction plays a crucial role in language comprehension. When predictions are fulfilled, the resulting facilitation allows for fast, efficient processing of ambiguous, rapidly-unfolding input; when predictions are not fulfilled, the resulting error signal allows us to adapt to broader statistical changes in this input. We used functional Magnetic Resonance Imaging to examine the neuroanatomical networks engaged in semantic predictive processing and adaptation. We used a relatedness proportion semantic priming paradigm, in which we manipulated the probability of predictions while holding local semantic context constant. Under conditions of higher (versus lower) predictive validity, we replicate previous observations of reduced activity to semantically predictable words in the left anterior superior/middle temporal cortex, reflecting facilitated processing of targets that are consistent with prior semantic predictions. In addition, under conditions of higher (versus lower) predictive validity we observed significant differences in the effects of semantic relatedness within the left inferior frontal gyrus and the posterior portion of the left superior/middle temporal gyrus. We suggest that together these two regions mediated the suppression of unfulfilled semantic predictions and lexico-semantic processing of unrelated targets that were inconsistent with these predictions. Moreover, under conditions of higher (versus lower) predictive validity, a functional connectivity analysis showed that the left inferior frontal and left posterior superior/middle temporal gyrus were more tightly interconnected with one another, as well as with the left anterior cingulate cortex. The left anterior cingulate cortex was, in turn, more tightly connected to superior lateral frontal cortices and subcortical regions—a network that mediates rapid learning and adaptation and that may have played a role in switching to a more predictive mode of processing in response to the statistical structure of the wider environmental context. Together, these findings highlight close links between the networks mediating semantic prediction, executive function and learning, giving new insights into how our brains are able to flexibly adapt to our environment.