Brian Langner

Spoken Dialog Challenge 2010

This paper describes the 2010 Spoken Dialog Challenge, a multi-site challenge to help investigate different spoken dialog system and evaluation techniques. The aim of the Challenge is to bring together multiple implementations of the same dialog task and deploy them in uncontrolled real user conditions and then make the results available for common evaluation techniques. This paper gives an overview of teh Challenge itself and the task, and presents the results for the “controlled task” part of the evaluation. The paper also discusses the infrastructure and organizational issues encountered and the solutions that made this challenge possible.

ONLINE SUPERVISED LEARNING OF NON-UNDERSTANDING RECOVERY POLICIES

Spoken dialog systems typically use a limited number of non- understanding recovery strategies and simple heuristic policies to engage them (e.g. first ask user to repeat, then give help, then transfer to an operator). We propose a supervised, online method for learning a non-understanding recovery policy over a large set of recovery strategies. The approach consists of two steps: first, we construct runtime estimates for the likelihood of success of each recovery strategy, and then we use these estimates to construct a policy. An experiment with a publicly available spoken dialog system shows that the learned policy produced a 12.5% relative improvement in the non-understanding recovery rate.

Using speech in noise to improve understandability for elderly listeners

This paper describes work designed to improve understandability of spoken output, specifically for the elderly, by using a speaking style employed by people to improve their understandability when speaking in poor channel conditions. We describe an experiment that shows the understandability gains that are possible using naturally-produced examples of this style. Additionally, we describe how to model this style, and evaluate the differences in understandability for speech synthesis produced using those models

Building Practical Spoken Dialog Systems

This tutorial will give a practical description of the free software Carnegie Mellon Olympus 2 Spoken Dialog Architecture. Building real working dialog systems that are robust enough for the general public to use is difficult. Most frequently, the functionality of the conversations is severely limited - down to simple question-answer pairs. While off-the-shelf toolkits help the development of such simple systems, they do not support more advanced, natural dialogs nor do they offer the transparency and flexibility required by computational linguistic researchers. However, Olympus 2 offers a complete dialog system with automatic speech recognition (Sphinx) and synthesis (SAPI, Festival) and has been used, along with previous versions of Olympus, for teaching and research at Carnegie Mellon and elsewhere for some 5 years. Overall, a dozen dialog systems have been built using various versions of Olympus, handling tasks ranging from providing bus schedule information to guidance through maintenance procedures for complex machinery, to personal calendar management. In addition to simplifying the development of dialog systems, Olympus provides a transparent platform for teaching and conducting research on all aspects of dialog systems, including speech recognition and synthesis, natural language understanding and generation, and dialog and interaction management. The tutorial will give a brief introduction to spoken dialog systems before going into detail about how to create your own dialog system within Olympus 2, using the Lets Go bus information system as an example. Further, we will provide guidelines on how to use an actual deployed spoken dialog system such as Lets Go to validate research results in the real world. As a possible testbed for such research, we will describe Lets Go Lab, which provides access to both the Lets Go system and its genuine user population for research experiments.