Leon Bergen

Rational integration of noisy evidence and prior semantic expectations in sentence interpretation.

Sentence processing theories typically assume that the input to our language processing mechanisms is an error-free sequence of words. However, this assumption is an oversimplification because noise is present in typical language use (for instance, due to a noisy environment, producer errors, or perceiver errors). A complete theory of human sentence comprehension therefore needs to explain how humans understand language given imperfect input. Indeed, like many cognitive systems, language processing mechanisms may even be “well designed”–in this case for the task of recovering intended meaning from noisy utterances. In particular, comprehension mechanisms may be sensitive to the types of information that an idealized statistical comprehender would be sensitive to. Here, we evaluate four predictions about such a rational (Bayesian) noisy-channel language comprehender in a sentence comprehension task: (i) semantic cues should pull sentence interpretation towards plausible meanings, especially if the wording of the more plausible meaning is close to the observed utterance in terms of the number of edits; (ii) this process should asymmetrically treat insertions and deletions due to the Bayesian “size principle”; such nonliteral interpretation of sentences should (iii) increase with the perceived noise rate of the communicative situation and (iv) decrease if semantically anomalous meanings are more likely to be communicated. These predictions are borne out, strongly suggesting that human language relies on rational statistical inference over a noisy channel.

A Noisy-Channel Account of Crosslinguistic Word-Order Variation

The distribution of word orders across languages is highly nonuniform, with subject-verb-object (SVO) and subject-object-verb (SOV) orders being prevalent. Recent work suggests that the SOV order may be the default in human language. Why, then, is SVO order so common? We hypothesize that SOV/SVO variation can be explained by language users’ sensitivity to the possibility of noise corrupting the linguistic signal. In particular, the noisy-channel hypothesis predicts a shift from the default SOV order to SVO order for semantically reversible events, for which potential ambiguity arises in SOV order because two plausible agents appear on the same side of the verb. We found support for this prediction in three languages (English, Japanese, and Korean) by using a gesture-production task, which reflects word-order preferences largely independent of native language. Other patterns of crosslinguistic variation (e.g., the prevalence of case marking in SOV languages and its relative absence in SVO languages) also straightforwardly follow from the noisy-channel hypothesis.

Nonliteral understanding of number words

Significance Human communication is rife with nonliteral language, ranging from metaphor to irony to hyperbole. How do people go so far beyond the literal meaning of an utterance to infer the speaker’s intended meaning? We present a computational model that understands hyperbolic and other nonliteral uses of number words (e.g., “That watch costs 10,000 dollars”). Our model integrates empirically measured background knowledge, principles of communication, and reasoning about communicative goals to explain the computational basis of nonliteral language understanding. This framework sheds light on the nature of communication, marking a significant advancement in the flexibility and richness of formal models of language understanding. One of the most puzzling and important facts about communication is that people do not always mean what they say; speakers often use imprecise, exaggerated, or otherwise literally false descriptions to communicate experiences and attitudes. Here, we focus on the nonliteral interpretation of number words, in particular hyperbole (interpreting unlikely numbers as exaggerated and conveying affect) and pragmatic halo (interpreting round numbers imprecisely). We provide a computational model of number interpretation as social inference regarding the communicative goal, meaning, and affective subtext of an utterance. We show that our model predicts humans’ interpretation of number words with high accuracy. Our model is the first to our knowledge to incorporate principles of communication and empirically measured background knowledge to quantitatively predict hyperbolic and pragmatic halo effects in number interpretation. This modeling framework provides a unified approach to nonliteral language understanding more generally.

Speaker Knowledge Influences The Comprehension Of Pragmatic Inferences

Inferring what speakers mean from what they say requires consideration of what they know. For instance, depending on the speakers level of expertise, uttering Some squirrels hibernate can imply that not all squirrels hibernate, or it might imply the weaker proposition that the speaker does not know whether all squirrels hibernate. The present study examines the extent to which speaker knowledge influences implied meanings as well as the timing of any such influence. Using a self-paced presentation, participants read sentences containing some in contexts where a speaker should know whether all was true, or where the speaker merely might know whether all was true. This knowledge manipulation was found to have immediate and reliable effects on the type of inference that was drawn. In contrast, knowledge played no role when the same meanings were conveyed literally. This work thus demonstrates that perceivers consider the speakers knowledge state incrementally to establish the speakers communicative goals.

A rational inference approach to aphasic language comprehension

ABSTRACT Background: It has long been observed that, when confronted with an implausible sentence like The ball kicked the girl, individuals with aphasia rely more on plausibility information from world knowledge (such that a girl is likely to kick a ball, but not vice versa) than control non-impaired populations do. We here offer a novel hypothesis to explain this greater reliance on plausibility information for individuals with aphasia. The hypothesis is couched with the rational inference approach to language processing. A key idea in this approach is that to derive an interpretation for an input string, individuals combine their priors (about messages that are likely to be communicated) with their knowledge about how messages can get corrupted by noise (due to production or perception errors). Aims: We hypothesise that language comprehension in aphasia works in the same way, except with a greater amount of noise, which leads to stronger reliance on syntactic and semantic priors. Methods & Procedures: We evaluated this hypothesis in an act-out task in three groups of participants (8 individuals with aphasia, 7 older controls, 11 younger controls) on two sets of materials: (a) implausible double-object (DO)/prepositional-phrase object (PO) materials, where a single added or deleted word could lead to a plausible meaning; and (b) implausible active-passive materials, where at least two added or deleted words are needed to arrive at a plausible meaning. Outcomes & Results: We observed that, similar to controls, individuals with aphasia rely on plausibility to a greater extent in the DO/PO than in the active/passive alternation. Critically, however, as predicted, individuals with aphasia rely less on the literal syntax overall than either of the control groups, and use their world knowledge prior (plausibility) in both the active/passive and DO/PO alternations, whereas controls rely on plausibility only in the DO/PO alternation. In addition, older persons and persons with aphasia made more errors on the DO structures (which are less frequent than PO structures) independent of plausibility, thus providing evidence for reliance on a syntactic prior, the more frequent structure. Conclusions: These results are as predicted by the rational inference approach to language processing in individuals with aphasia.

The Strategic Use of Noise in Pragmatic Reasoning

We combine two recent probabilistic approaches to natural language understanding, exploring the formal pragmatics of communication on a noisy channel. We first extend a model of rational communication between a speaker and listener, to allow for the possibility that messages are corrupted by noise. In this model, common knowledge of a noisy channel leads to the use and correct understanding of sentence fragments. A further extension of the model, which allows the speaker to intentionally reduce the noise rate on a word, is used to model prosodic emphasis. We show that the model derives several well-known changes in meaning associated with prosodic emphasis. Our results show that nominal amounts of actual noise can be leveraged for communicative purposes.

Processing temporal presuppositions: an event-related potential study

ABSTRACT The ability to efficiently process presuppositions, which contain information that the speaker believes to be in the background to the conversation, is essential for effective communication. To get a deeper understanding of the nature and the time-course of temporal presupposition processing, we examined event-related potential evoked by the word again in two types of sentence contexts. The word again was presented in contexts that supported a presupposition (e.g. Jake had tipped a maid at the hotel once before. Today he tipped a maid at the hotel again … ) or violated it (e.g. Jake had never tipped a maid at the hotel before. Today he tipped a maid at the hotel again … ). The presupposition violation was associated with increased amplitudes of the P3b/P600 but not the N400 component. We argue for the centrality of the P3b/P600 component for presupposition processing. These findings demonstrate rapid integration of lexical presuppositions with contextual knowledge.