Daniel Bolaños

FLORA: Fluent oral reading assessment of children's speech

We present initial results of FLORA, an accessible computer program that uses speech recognition to provide an accurate measure of childrens oral reading ability. FLORA presents grade-level text passages to children, who read the passages out loud, and computes the number of words correct per minute (WCPM), a standard measure of oral reading fluency. We describe the main components of the FLORA program, including the system architecture and the speech recognition subsystems. We compare results of FLORA to human scoring on 783 recordings of grade level text passages read aloud by first through fourth grade students in classroom settings. On average, FLORA WCPM scores were within 3 to 4 words of human scorers across students in different grade levels and schools.

Automatic assessment of expressive oral reading

We investigated the automatic assessment of expressive childrens oral reading of grade level text passages using a standardized rubric. After a careful review of the reading literature and a close examination of the rubric, we designed a novel set of prosodic and lexical features to characterize fluent expressive reading. A number of complementary sources of information were used to design the features, each of them motivated by research on different components of reading fluency. Features are connected to the childs reading rate, to the presence and number of pauses, filled-pauses and word-repetitions, the correlation between punctuation marks and pauses, the length of word groupings, syllable stress and duration and the location of pitch peaks and contours. The proposed features were evaluated on a corpus of 783 one-minute reading sessions from 313 students reading grade-leveled passages without assistance (cold unassisted reading). Experimental results show that the proposed lexical and prosodic features provide complementary information and are able to capture the characteristics of expressive reading. The results showed that on both the 2-point and the 4-point expressiveness scales, computer-generated ratings of expressiveness agreed with human raters better than the human raters agreed with each other. The results of the study suggest that automatic assessment of expressive oral reading can be combined with automatic measures of word accuracy and reading rate to produce an accurate multidimensional estimate of childrens oral reading ability.

Multi-scale Personalization for Voice Search Applications

Voice Search applications provide a very convenient and direct access to a broad variety of services and information. However, due to the vast amount of information available and the open nature of the spoken queries, these applications still suffer from recognition errors. This paper explores the utilization of personalization features for the post-processing of recognition results in the form of n-best lists. Personalization is carried out from three different angles: short-term, long-term and Web-based, and a large variety of features are proposed for use in a log-linear classification framework. Experimental results on data obtained from a commercially deployed Voice Search system show that the combination of the proposed features leads to a substantial sentence error rate reduction. In addition, it is shown that personalization features which are very different in nature can successfully complement each other.

A Reference Verification Framework and its Application to a Children's Speech Reading Tracker

In this article we present a novel approach to reference verification, the problem of determining if a speakers utterance matches a specific reference (text) string. We will then discuss its application to a reading tracker system for childrens speech. Unlike other reading tracker systems proposed in the literature, that are built over conventional speech recognizers with ad-hoc language models, the reading tracker described here is designed specifically for the task of estimating whether a child has read an expected sequence of words out loud. The tracker is designed to handle in a natural and flexible way the disfluencies that frequently appear in childrens speech while reading out loud, (e.g., partial-words, repetitions, self-corrections, sentence-restarts, etc), and to overcome problems caused by using language models within the reference verification task. Three mechanisms have been introduced for this purpose: the utilization of filler models and the inclusion of forward and backward inter-word transitions in the static decoding network. While this article focuses on the approach used to overcome errors observed in previous systems, the performance of this system will be evaluated on a corpus of childrens speech while reading out loud and compared to the performance of a traditional reading tracker system that is built on top of a speech recognition system. The results of this comparison will be presented at WOCCI 2009.

Spoken Dialogs with Children for Science Learning and Literacy

Advances in human language and character animation technologies have enabled a new generation of intelligent tutoring systems that support conversational interaction between young learners and a lifelike computer character that was designed to behave like a sensitive and effective human tutor My Science Tutor is a spoken dialog system in which children learn to construct science explanations through conversations with Marni, the virtual science tutor, in multimedia environments. MyST displays illustrations, silent animations or interactive simulations to the student, while Marni asks open-ended questions like “Whats going here?”. Based on MySTs analysis of the students spoken response, the system decides what the student understands about the science and what the student has not yet explained (or doesnt know), and generates a follow-on question a new prompt, and possibly a new animation, that is designed to scaffold learning and challenge the student to reason about the science. Two large scale evaluations were conducted in which third, fourth and fifth grade students received over 5 hours of tutoring during sixteen 20-minute sessions in four different areas of science. The results revealed that, relative to students who did not receive tutoring, students who used My Science Tutor achieved significant learning gains in standardized tests of science achievement, equivalent to gains achieved by students who received tutoring by expert human tutors. In recent research, we have extended the technologies used in MyST to a develop a new generation of interactive books that use text, speech and dialog technologies to help children learn to read science texts fluently, expressively, and with good comprehension. We will demonstrate these MindStars Books and present initial results of classroom testing.