Delphine Dahan

Prosody in the comprehension of spoken language: A literature review

Research on the exploitation of prosodic information in the comprehension of spoken language is reviewed. The research falls into three main areas: the use of prosody in the recognition of spoken words, in which most attention has been paid to the question of whether the prosodic structure of a word plays a role in initial activation of stored lexical representations; the use of prosody in the computation of syntactic structure, in which the resolution of global and local ambiguities has formed the central focus; and the role of prosody in the processing of discourse structure, in which there has been a preponderance of work on the contribution of accentuation and deaccentuation to integration of concepts with an existing discourse model. The review reveals that in each area progress has been made towards new conceptions of prosodys role in processing, and in particular this has involved abandonment of previously held deterministic views of the relationship between prosodic structure and other aspects of linguistic structure.

The role of prosodic boundaries in the resolution of lexical embedding in speech comprehension

Participants eye movements were monitored as they heard sentences and saw four pictured objects on a computer screen. Participants were instructed to click on the object mentioned in the sentence. There were more transitory fixations to pictures representing monosyllabic words (e.g. ham) when the first syllable of the target word (e.g. hamster) had been replaced by a recording of the monosyllabic word than when it came from a different recording of the target word. This demonstrates that a phonemically identical sequence can contain cues that modulate its lexical interpretation. This effect was governed by the duration of the sequence, rather than by its origin (i.e. which type of word it came from). The longer the sequence, the more monosyllabic-word interpretations it generated. We argue that cues to lexical-embedding disambiguation, such as segmental lengthening, result from the realization of a prosodic boundary that often but not always follows monosyllabic words, and that lexical candidates whose word boundaries are aligned with prosodic boundaries are favored in the word-recognition process.

Time course of frequency effects in spoken-word recognition: evidence from eye movements.

In two experiments, eye movements were monitored as participants followed spoken instructions to click on and move pictures with a computer mouse. In Experiment 1, a referent picture (e.g., the picture of a bench) was presented along with three pictures, two of which had names that shared the same initial phonemes as the name of the referent (e.g., bed and bell). Participants were more likely to fixate the picture with the higher frequency name (bed) than the picture with the lower frequency name (bell). In Experiment 2, referent pictures were presented with three unrelated distractors. Fixation latencies to referents with high-frequency names were shorter than those to referents with low-frequency names. The proportion of fixations to the referents and distractors were analyzed in 33-ms time slices to provide fine-grained information about the time course of frequency effects. These analyses established that frequency affects the earliest moments of lexical access and rule out a late-acting, decision-bias locus for frequency. Simulations using models in which frequency operates on resting-activation levels, on connection strengths, and as a postactivation decision bias provided further constraints on the locus of frequency effects.

Accent and reference resolution in spoken-language comprehension

Abstract The role of accent in reference resolution was investigated by monitoring eye fixations to lexical competitors (e.g., candy and candle ) as participants followed prerecorded instructions to move objects above or below fixed geometric shapes using a computer mouse. In Experiment 1, the first utterance instructed participants to move one object above or below a shape (e.g., “Put the candle/candy below the triangle”) and the second utterance contained an accented or deaccented definite noun phrase which referred to the same object or introduced a new entity (e.g., “Now put the CANDLE above the square” vs. “Now put the candle ABOVE THE SQUARE”). Fixations to the competitor (e.g., candy ) demonstrated a bias to interpret deaccented nouns as anaphoric and accented nouns as nonanaphoric. Experiment 2 used only accented nouns in the second instruction, varying whether the referent of this second instruction was the Theme of the first instruction (e.g., “Put the candle below the triangle”) or the Goal of the first instruction (e.g., “Put the necklace below the candle”). Participants preferred to interpret accented noun phrases as referring to a previously mentioned nonfocused entity (the Goal) rather than as introducing a new unmentioned entity.

Subcategorical mismatches and the time course of lexical access: Evidence for lexical competition

Participants eye movements were monitored as they followed spoken instructions to click on a pictured object with a computer mouse (e.g., click on the net). Participants were slower to fixate the target picture when the onset of the target word came from a competitor word (e.g., ne(ck)t) than from a nonword (e.g., ne(p)t), as predicted by models of spoken-word recognition that incorporate lexical competition. This was found whether the picture of the competitor word (e.g., the picture of a neck) was present on the display or not. Simulations with the TRACE model captured the major trends of fixations to the target and its competitor over time. We argue that eye movements provide a fine-grained measure of lexical activation over time, and thus reveal effects of lexical competition that are masked by response measures such as lexical decisions.

Eye movements and lexical access in spoken-language comprehension: evaluating a linking hypothesis between fixations and linguistic processing.

A growing number of researchers in the sentence processing community are using eye movements to address issues in spoken language comprehension. Experiments using this paradigm have shown that visually presented referential information, including properties of referents relevant to specific actions, influences even the earliest moments of syntactic processing. Methodological concerns about task-specific strategies and the linking hypothesis between eye movements and linguistic processing are identified and discussed. These concerns are addressed in a review of recent studies of spoken word recognition which introduce and evaluate a detailed linking hypothesis between eye movements and lexical access. The results provide evidence about the time course of lexical activation that resolves some important theoretical issues in spoken-word recognition. They also demonstrate that fixations are sensitive to properties of the normal language-processing system that cannot be attributed to task-specific strategies.

Looking at the rope when looking for the snake: conceptually mediated eye movements during spoken-word recognition.

Participants’ eye movements to four objects displayed on a computer screen were monitored as the participants clicked on the object named in a spoken instruction. The display contained pictures of the referent (e.g., a snake), a competitor that shared features with the visual representation associated with the referent’s concept (e.g., a rope), and two distractor objects (e.g., a couch and an umbrella). As the first sounds of the referent’s name were heard, the participants were more likely to fixate the visual competitor than to fixate either of the distractor objects. Moreover, this effect was not modulated by the visual similarity between the referent and competitor pictures, independently estimated in a visual similarity rating task. Because the name of the visual competitor did not overlap with the phonetic input, eye movements reflected word-object matching at the level of lexically activated perceptual features and not merely at the level of preactivated sound forms.

The Time Course of Spoken Word Learning and Recognition: Studies With Artificial Lexicons

The time course of spoken word recognition depends largely on the frequencies of a word and its competitors, or neighbors (similar-sounding words). However, variability in natural lexicons makes systematic analysis of frequency and neighbor similarity difficult. Artificial lexicons were used to achieve precise control over word frequency and phonological similarity. Eye tracking provided time course measures of lexical activation and competition (during spoken instructions to perform visually guided tasks) both during and after word learning, as a function of word frequency, neighbor type, and neighbor frequency. Apparent shifts from holistic to incremental competitor effects were observed in adults and neural network simulations, suggesting such shifts reflect general properties of learning rather than changes in the nature of lexical representations.

Continuous Mapping From Sound to Meaning in Spoken-Language Comprehension: Immediate Effects of Verb-Based Thematic Constraints.

The authors used 2 visual-world eye-tracking experiments to examine lexical access using Dutch constructions in which the verb did or did not place semantic constraints on its subsequent subject noun phrase. In Experiment 1, fixations to the picture of a cohort competitor (overlapping with the onset of the referents name, the subject) did not differ from fixations to a distractor in the constraining-verb condition. In Experiment 2, cross-splicing introduced phonetic information that temporarily biased the input toward the cohort competitor. Fixations to the cohort competitor temporarily increased in both the neutral and constraining conditions. These results favor models in which mapping from the input onto meaning is continuous over models in which contextual effects follow access of an initial form-based competitor set.

On the discovery of novel wordlike units from utterances: an artificial-language study with implications for native-language acquisition

In 4 experiments, adults were familiarized with utterances from an artificial language. Short utterances occurred both in isolation and as part of a longer utterance, either at the edge or in the middle of the longer utterance. After familiarization, participants recognition memory for fragments of the long utterance was tested. Recognition was greatest for the remainder of the longer utterance after extraction of the short utterance, but only when the short utterance was located at the edge of the long utterance. These results support the incremental distributional regularity optimization (INCDROP) model of speech segmentation and word discovery, which asserts that people segment utterances into familiar and new wordlike units in such a way as to minimize the burden of processing new units. INCDROP suggests that segmentation and word discovery during native-language acquisition may be driven by recognition of familiar units from the start, with no need for transient bootstrapping mechanisms.