Julie E. Boland

Quantitative and Qualitative Evaluation of Darpa Communicator Spoken Dialogue Systems

This paper describes the application of the PARADISE evaluation framework to the corpus of 662 human-computer dialogues collected in the June 2000 Darpa Communicator data collection. We describe results based on the standard logfile metrics as well as results based on additional qualitative metrics derived using the DATE dialogue act tagging scheme. We show that performance models derived via using the standard metrics can account for 37% of the variance in user satisfaction, and that the addition of DATE metrics improved the models by an absolute 5%.

Evidence for the immediate use of verb control information in sentence processing

Abstract When a verb is followed by an infinitival complement, the particular verb determines whether its subject or object is the understood subject of the infinitive. Thus, the verb “controls” the interpretation of the infinitive (e.g., John promised/persuaded Mary to wash ). Frazier and colleagues have argued that verb control information is not immediately accessed and used in sentence processing based on whole-sentence comprehension times. The studies reported here examined the use of verb control using an on-line plausibility monitoring task. Subjects immediately detected incongruities that depended upon their having correctly used control information, indicating that verb control information is rapidly accessed and used. It is argued that the results support an approach to language comprehension that emphasizes the importance of lexical representations in rapidly integrating many of the different sources of linguistic and nonlinguistic knowledge that need to be coordinated during language comprehension.

Priming in pronunciation: Beyond pattern recognition and onset latency

Abstract Three experiments that address the impact of associative relatedness on both onset latencies and production durations in pronunciation performance are reported. In Experiment 1, a related response cue, presented after a to-be-pronounced target word, decreased the target words production duration, compared to an unrelated response cue, but did not influence its onset latency. In Experiment 2, two related or two unrelated words were simultaneously presented. The response cue was presented 400, 900, 1400, or 1900 ms after the stimuli were presented and indicated whether to pronounce the stimuli in a prepared sequence or in an unprepared sequence. The results indicated that the production durations were shorter when the two words were related, compared to unrelated, independent of cue delay. Also, the onset latencies were faster when the words were related compared to unrelated at each delay except the 1900-ms delay. In Experiment 3, three word sequences were presented to distinguish between associative-cooccurrence accounts and meaning-level accounts of the results obtained in Experiments 1 and 2. The results of Experiment 3 yielded a significant impact of the primes on both onset latencies and production durations. The pattern of priming effects supported a meaning-level account of the present production duration effects. The results from these experiments are interpreted within both an interactive activation model of speech production and a cooperative-based model of language processing.

The relationship between syntactic and semantic processes in sentence comprehension

Four experiments investigated the relationship between syntactic and semantic processing. The first two experiments, which used a word-by-word reading paradigm with a makes-sense judgem ent, demonstrated that verb argument structure is used to construct provisional interpretations at points in a sentence where the syntactic structure is ambiguous, and that resolution of syntactic ambiguity occurs even when it is not necessary for interpretation of the input. The last two experiments used a cross-modal integration paradigm and found evidence that multiple syntactic representations are accessed or constructed at points of syntactic ambiguity just as multiple meanings are accessed at points of lexical ambiguity. The experim ental results are evaluated with regard to serial autonomous models, strongly and weakly interactive models, and a hybrid model proposed here.

Lexical structure in Parsing long-distance dependencies.

We review a series of experiments investigating lexical influences in parsing sentences with long-distance dependencies. We report three primary results. First, gaps are posited and filled immediately following verbs that are typically used transitively, even when the filler is an implausible object of the verb. However, gaps are not posited after verbs that are typically used intransitively. Second, plausibility determines whether or not a filler is treated as the object of a verb when the verb is typically used with both a direct object and an infinitive complement. Finally, verb control information is used immediately in determining which noun phrase will be interpreted as the “understood” subject of an infinitive complement.

Linking Eye Movements to Sentence Comprehension in Reading and Listening

Eye tracking paradigms in both written and spoken modalities are the state of the art for online behavioral investigations of language comprehension. But it is almost a misnomer to refer to the two types of paradigms by the same ―eye-tracking‖ label, because they are quite different. Reading paradigms gauge local processing difficulty by measuring the participant’s gaze on the very material that he or she is trying to comprehend. The critical sentence regions are determined spatially, and gaze is measured in terms of the time spent looking within a region of interest, the likelihood of a regressive eye movement out of the region, and so forth. In contrast, listening paradigms gauge how rapidly successful comprehension occurs by measuring how quickly people look, or how likely people are to look, at objects referenced by the linguistic material.

The role of verb argument structure in sentence processing: Distinguishing between syntactic and semantic effects

This paper describes an ongoing research program designed to investigate how syntactic and semantic aspects of lexical information become available to the sentence processing system. The two experiments described here distinguished between syntactic and semantic representations by using cross-modal naming and lexical decision in a new way. The relationship between the main verb and the probe word was varied such that the probe word met either the syntactic criteria to be an argument, the semantic criteria, neither, or both the syntactic and semantic criteria. Lexical decision times were sensitive to both syntactic and semantic congruity, while naming times were sensitive only to syntactic congruity. The two tasks were then used to investigate syntactic and semantic representations when verb argument structure was ambiguous. Subcategorized structures were constructed without regard for biasing context, but the contextually inappropriate thematic frame was ruled out while the inappropriate syntactic frame was still available.

Interaction with autonomy: Multiple Output models and the inadequacy of the Great Divide

There are currently a number of psycholinguistic models in which processing at a particular level of representation is characterized by the generation of multiple outputs, with resolution--but not generation--involving the use of information from higher levels of processing. Surprisingly, models with this architecture have been characterized as autonomous within the domain of word recognition but as interactive within the domain of sentence processing. We suggest that the apparent confusion is not, as might be assumed, due to fundamental differences between lexical and syntactic processing. Rather, we believe that the labels in each domain were chosen in order to obtain maximal contrast between a new model and the model or models that were currently dominating the field. The contradiction serves to highlight the inadequacy of a simple autonomy/interaction dichotomy for characterizing the architectures of current processing models.

Children's use of language context in lexical ambiguity resolution

Lexical ambiguity resolution was examined in children aged 7 to 10 years and adults. In Experiment 1, participants heard sentences supporting one (or neither) meaning of a balanced ambiguous word in a cross-modal naming paradigm. Naming latencies for context-congruent versus context-incongruent targets and judgements of the relatedness of targets to the sentence served as indices of appropriate context use. While younger children were faster to respond to related targets regardless of the sentence context, older children and adults showed priming only for context-appropriate targets. In Experiment 2, only a single-word context preceded the homophone, and in contrast to Experiment 1, all groups showed contextual sensitivity. Individual working-memory span and inhibition ability were also measured in Experiment 2, and more mature executive function abilities were associated with greater contextual sensitivity. These findings support a developmental model whereby sentential context use for lexical ambiguity resolution increases with age, cognitive processing capacity, and reading skill.

Semantics Does Not Need a Processing License from Syntax in Reading Chinese.

Two event-related brain potential experiments were conducted to investigate whether there is a functional primacy of syntactic structure building over semantic processes during Chinese sentence reading. In both experiments, we found that semantic interpretation proceeded despite the impossibility of a well-formed syntactic analysis. In Experiment 1, we found an N400 difference between combined syntactic category and semantic violations and single syntactic violations. This finding is inconsistent with earlier German and French studies (e.g., Friederici, Gunter, Hahne, & Mauth, 2004; Friederici, Steinhauer, & Frisch, 1999; Hahne & Friederici, 2002) showing that semantic integration does not proceed for words of the wrong syntactic category. In Experiment 2, we used a design that was very similar to that used in earlier German and French studies, but semantic violations still evoked an N400, irrespective of a simultaneous syntactic category violation. We argue against processing models that do not allow for semantic integration of a word unless it can be grammatically attached to the developing phrase structure tree. Rather, language experience may modulate the mode of interplay between syntax and semantics.