Gregory Aist

Targeted help for spoken dialogue systems: intelligent feedback improves naive users' performance

We present experimental evidence that providing naive users of a spoken dialogue system with immediate help messages related to their out-of-coverage utterances improves their success in using the system. A grammar-based recognizer and a Statistical Language Model (SLM) recognizer are run simultaneously. If the grammar-based recognizer suceeds, the less accurate SLM recognizer hypothesis is not used. When the grammar-based recognizer fails and the SLM recognizer produces a recognition hypothesis, this result is used by the Targeted Help agent to give the user feedback on what was recognized, a diagnosis of what was problematic about the utterance, and a related in-coverage example. The in-coverage example is intended to encourage alignment between user inputs and the language model of the system. We report on controlled experiments on a spoken dialogue system for command and control of a simulated robotic helicopter.

Talking through procedures: an intelligent space station procedure assistant

We present a prototype system aimed at providing spoken dialogue support for complex procedures aboard the International Space Station. The system allows navigation one line at a time or in larger steps. Other user functions include issuing spoken corrections, requesting images and diagrams, recording voice notes and spoken alarms, and controlling audio volume.

Adding Human-Provided Emotional Scaffolding to an Automated Reading Tutor That Listens Increases Student Persistence

Everyone agrees emotions are important, and some have even built supportive language into their ITSs, such as praise. But what is the effect of such emotional scaffolding, and is it worth including in a system that is already providing cognitive scaffolding? This poster presents the first statistically reliable empirical evidence from a controlled study for the effect of human-provided emotional scaffolding on student persistence in an intelligent tutoring system. We conducted an experiment that added human-provided emotional scaffolding to an automated Reading Tutor that listens.

High Performance Segmentation of Spontaneous Speech Using Part of Speech and Trigger Word Information

We describe and experimentally evaluate an efficient method for automatically determining small clause boundaries in spontaneous speech. Our method applies an artificial neural network to information about part of speech and trigger words.We find that with a limited amount of data (less than 2500 words for the training set), a small sliding context window (+/-3 tokens) and only two hidden units, the neural net performs extremely well on this task: less than 5% error rate and F-score (combined precision and recall) of over .85 on unseen data.These results prove to be better than those reported earlier using different approaches.

Improving Story Choice in a Reading Tutor that Listens

This abstract summarizes how we improved task choice - picking a story to read - in successive versions of a Reading Tutor that listens to elementary students read aloud. We wanted to motivate children to spend time on the Reading Tutor by giving them some choice in what to read, without spending too much time picking stories. We also wanted them to read plenty of new text, so as to build vocabulary and decoding skills.

Fruit Carts: A Domain and Corpus for Research in Dialogue Systems and Psycholinguistics

We describe a novel domain, Fruit Carts, aimed at eliciting human language production for the twin purposes of (a) dialogue system research and development and (b) psycholinguistic research. Fruit Carts contains five tasks: choosing a cart, placing it on a map, painting the cart, rotating the cart, and filling the cart with fruit. Fruit Carts has been used for research in psycholinguistics and in dialogue systems. Based on these experiences, we discuss how well the Fruit Carts domain meets four desired features: unscripted, context-constrained, controllable difficulty, and separability into semi-independent subdialogues. We describe the domain in sufficient detail to allow others to replicate it; researchers interested in using the corpora themselves are encouraged to contact the authors directly.

Exploiting predictable response training to improve automatic recognition of children's spoken responses

The unpredictability of spoken responses by young children (6-7 years old) makes them problematic for automatic speech recognizers. Aist and Mostow proposed predictable response training to improve automatic recognition of childrens free-form spoken responses. We apply this approach in the context of Project LISTENs Reading Tutor to the task of teaching children an important reading comprehension strategy, namely to make up their own questions about text while reading it. We show how to use knowledge about strategy instruction and the story text to generate a language model that predicts questions spoken by children during comprehension instruction. We evaluated this model on a previously unseen test set of 18 utterances totaling 137 words spoken by 11 second grade children in response to prompts the Reading Tutor inserted as they read. Compared to using a baseline trigram language model that does not incorporate this knowledge, speech recognition using the generated language model achieved concept recall 5 times higher – so much that the difference was statistically significant despite small sample size.

A better reading tutor that listens

Project LISTENs Reading Tutor listens to children read aloud, and helps them learn to read, as illustrated on the Videos page of our website This Interactive Event encompasses both this basic interaction and new extensions we are developing.