Harald Aust

A spoken language inquiry system for automatic train timetable information

Abstract This article describes the Philips automatic train timetable information system which enables the user to call up accurate information about train connections between 1200 German cities over the telephone. In contrast to most of the inquiry systems available so far, the caller can talk to our system in unrestricted, natural and fluent speech, very much like talking to a human operator. No instructions are given beforehand. The system consists of four main components: speech recognition, speech understanding, dialogue control, and speech output. They are separated into independent modules and executed sequentially. The speech recogniser creates a word graph from the spoken input. This word graph is then passed to the understanding component which computes the meaning, using an attributed stochastic context-free grammar. A dialogue manager analyses the results and either accesses the database or comes up with another question if necessary. The system has been made available to the general public in an ongoing field test, both to gather speech data and to evaluate its performance.

Evaluating dialog systems used in the real world

An important aspect of creating high performance natural language dialog systems is the question of how they are evaluated. While a universally accepted method for doing so for pure speech recognition exists, this is not clear for speech understanding or dialog systems. We describe the methods we typically use for our systems and argue that it is not sufficient to evaluate their constituents separately. Instead, a measure for a system in its entity is needed.