Julie C. Sedivy

Eye movements as a window into real-time spoken language comprehension in natural contexts.

When listeners follow spoken instructions to manipulate real objects, their eye movements to the objects are closely time locked to the referring words. We review five experiments showing that this time-locked characteristic of eye movements provides a detailed profile of the processes that underlie real-time spoken language comprehension. Together, the first four experiments showed that listerners immediately integrated lexical, sublexical, and prosodic information in the spoken input with information from the visual context to reduce the set of referents to the intended one. The fifth experiment demonstrated that a visual referential context affected the initial structuring of the linguistic input, eliminating even strong syntactic preferences that result in clear garden paths when the referential context is introduced linguistically. We argue that context affected the earliest moments of language processing because it was highly accessible and relevant to the behavioral goals of the listener.

Eye Movements to Pictures Reveal Transient Semantic Activation during Spoken Word Recognition.

Two experiments explore the activation of semantic information during spoken word recognition. Experiment 1 shows that as the name of an object unfolds (e.g., lock), eye movements are drawn to pictorial representations of both the named object and semantically related objects (e.g., key). Experiment 2 shows that objects semantically related to an uttered words onset competitors become active enough to draw visual attention (e.g., if the uttered word is logs, participants fixate on key because of partial activation of lock), despite that the onset competitor itself is not present in the visual display. Together, these experiments provide detailed information about the activation of semantic information associated with a spoken word and its phonological competitors and demonstrate that transient semantic activation is sufficient to impact visual attention.

Pragmatic versus Form-based Accounts of Referential Contrast: Evidence for Effects of Informativity Expectations

Characterizing the relationship between form-based linguistic knowledge and representation of context has long been of importance in the study of on-line language processing. Recent experimental research has shown evidence of very rapid effects of referential context in resolving local indeterminacies on-line. However, there has been no consensus regarding the nature of these context effects. The current paper summarizes recent work covering a range of phenomena for which referential contrast has been shown to influence on-line processing, including prenominal and postnominal modification, focus operators, and intonational focus. The results of the body of work suggest that referential context effects are not limited to situations in which the linguistic form of the utterance directly specifies the point of contact with context. Rather, context effects of a pragmatic, Gricean nature appear to be possible, suggesting the relationship between linguistic form and context in rapid on-line processing can be of a very indirect nature.

Lexical-semantic activation in broca's and wernicke's aphasia: Evidence from eye movements

Lexical processing requires both activating stored representations and selecting among active candidates. The current work uses an eye-tracking paradigm to conduct a detailed temporal investigation of lexical processing. Patients with Brocas and Wernickes aphasia are studied to shed light on the roles of anterior and posterior brain regions in lexical processing as well as the effects of lexical competition on such processing. Experiment 1 investigates whether objects semantically related to an uttered word are preferentially fixated, for example, given the auditory target hammer, do participants fixate a picture of a nail? Results show that, like normal controls, both groups of patients are more likely to fixate on an object semantically related to the target than an unrelated object. Experiment 2 explores whether Brocas and Wernickes aphasics show competition effects when words share onsets with the uttered word, for instance, given the auditory target hammer, do participants fixate a picture of a hammock? Experiment 3 investigates whether these patients activate words semantically related to onset competitors of the uttered word, for example, given the auditory target hammock, do participants fixate a nail due to partial activation of the onset competitor hammer? Results of Experiments 2 and 3 show pathological patterns of performance for both Brocas and Wernickes aphasics under conditions of lexical onset competition. However, the patterns of deficit differed, suggesting different functional and computational roles for anterior and posterior areas in lexical processing. Implications of the findings for the functional architecture of the lexical processing system and its potential neural substrates are considered.

The time course of lexical activation in Broca's and Wernicke's aphasia: Evidence from eye-movements

Studies of spoken word recognition in Broca’s and Wernicke’s aphasics suggest that the lexical processing deficits displayed by these patients are in part due to abnormal levels of lexical activation (Milberg, Blumstein, & Dworetzky, 1988). Specifically, it has been proposed that Broca’s aphasics have abnormally low levels of lexical activation, and that Wernicke’s aphasics have abnormally high levels of lexical activation. A number of studies have measured the time course of lexical activation in these patients at various points after word offset by varying the ISI in a semantic priming paradigm. However, a more direct and continuous measure of lexical activation could provide further insight into the lexical processing deficits of these patients. A simple eye-tracking paradigm has recently been used to uncover fine-grained information about the time course of spoken word recognition in normal subjects (e.g., Allopenna, Magnuson, & Tanenhaus, 1998). In this paradigm, participants’ eye movements are monitored as they hear a word and select the matching picture from a four picture display. When normal subjects hear a word (e.g., hammock), it has been found that initially, they are more likely to fixate on a picture of a distractor object whose name is an onset competitor (e.g., hammer) of the uttered word, than on unrelated distractors (Allopenna et al., 1998). Furthermore, it has also been found that normal subjects show onset mediated semantic competitor effects in that they preferentially fixate on a distractor object that is semantically related to an onset competitor of the uttered word, despite the fact that the onset competitor itself is absent from the display (e.g., when subjects hear hammock, they preferentially fixate on nail, even though there is no hammer in the display, Yee & Sedivy, 2001). The eye-tracking paradigm described above was used to investigate the time course of spoken word recognition in 5 Broca’s and 4 Wernicke’s aphasics, and in 12 age-matched controls. If Broca’s aphasics indeed have reduced levels of lexical activation, a simple prediction would be that the (limited) amount of acoustic input that matches the onset competitor might only weakly activate the onset competitor’s lexical representation. If true, then Broca’s aphasics should show both a smaller onset competitor effect and a smaller onset mediated semantic competitor effect than normal subjects (i.e., when Broca’s aphasics hear hammock, they should be less likely to fixate on hammer, and also less likely to fixate on nail compared to normal subjects). On the other hand, if Wernicke’s aphasics have abnormally high levels of lexical activation, even the (limited) amount of acoustic input that matches the onset competitor might be enough to strongly activate the onset competitor’s lexical representation. If true, then Wernicke’s aphasics might show both a larger onset competitor effect, and a larger onset mediated semantic competitor effect than normal subjects.