David Goodine

Multilingual spoken-language understanding in the MIT Voyager system

Abstract This paper describes our recent work in developing multilingual spoken language systems that support human-computer interactions. Our approach is based on the premise that a common semantic representation can be extracted from the input for all languages, at least within the context of restricted domains. In our design of such systems, language dependent information is separated from the system kernel as much as possible, and encoded in external data structures. The internal system manager, discourse and dialogue component, and database are all maintained in a language transparent form. Our description will focus on the development of the multilingual MIT Voyager spoken language system, which can engage in verbal dialogues with users about a geographical region within Cambridge, MA in the USA. The system can provide information about distances, travel times or directions between objects located within this area (e.g., restaurants, hotels, banks, libraries), as well as information such as the addresses, telephone numbers or location of the objects themselves. Voyager has been fully ported to Japanese and Italian, and we are in the process of porting to French and German as well. Evaluations for the English, Japanese and Italian systems are reported. Other related multilingual research activities are also briefly mentioned.

PEGASUS: a spoken dialogue interface for on-line air travel planning

Abstract This paper describes PEGASUS, a spoken dialogue interface for on-line air travel planning that we have recently developed. PEGASUS leverages off our spoken language technology development in the ATIS domain, and enables users to book flights using the American Airlines EAASY SABRE system. The input query is transformed by the speech understanding system to a frame representation that captures its meaning. The tasks of the System Manager include transforming the semantic representation into an EAASY SABRE command, transmitting it to the application backend, formatting and interpreting the resulting information, and managing the dialogue. Preliminary evaluation results suggest that users can learn to make productive use of PEGASUS for travel planning, although much work remains to be done.

The VOYAGER speech understanding system: preliminary development and evaluation

Early experience with the development of the MIT VOYAGER spoken language system is described, and its current performance is documented. The three components of VOYAGER, the speech recognition component, the natural language component, and the application back-end, are described.<<ETX>>

Integration of speech recognition and natural language processing in the MIT VOYAGER system

The MIT VOYAGER speech understanding system is an urban exploration and navigation system that interacts with the user through spoken dialogue, text, and graphics. The authors describe recent attempts at improving the integration between the speech recognition and natural language components. They used the generation capability of the natural language component to produce a word-pair language model to constrain the recognizers search space, thus improving the coverage of the overall system. They also implemented a strategy in which the recognizer generates the top N word strings and passes them along to the natural language component for filtering. Results on performance evaluation are presented.<<ETX>>

The SUMMIT speech recognition system: phonological modelling and lexical access

The phonological modeling and lexical access components of the SUMMIT speech recognition system are described in detail. SUMMIT makes explicit use of acoustic-phonetic knowledge, embedded in a segmental framework that can be trained automatically. Performance results for the complete system on the DARPA 1000-word Naval Resource Management task are presented.<<ETX>>

A bilingual VOYAGER system

This paper describes our initial efforts at porting the VOYAGER spoken language system to Japanese. In the process we have reorganized the structure of the system so that language dependent information is separated from the core engine as much as possible. For example, this information is encoded in tabular or rule-based form for the natural language understanding and generation components. The internal system manager, discourse and dialogue component, and database are all maintained in language transparent form. Once the generation component was ported, data were collected from 40 native speakers of Japanese using a wizard collection paradigm. A portion of these data was used to train the natural language and segment-based speech recognition components. The system obtained an overall understanding accuracy of 52% on the test data, which is similar to our earlier reported results for English [1].

The collection and preliminary analysis of a spontaneous speech database

As part of our effort in developing a spoken language system for interactive problem solving, we recently collected a sizeable amount of speech data. This database is composed of spontaneous sentences which were collected during a simulated human/machine dialogue. Since a computer log of the spoken dialogue was maintained, we were able to ask the subjects to provide read versions of the sentences as well. This paper documents the data collection process, and provides some preliminary analyses of the collected data.

Recent progress on the VOYAGER system

The VOYAGER speech recognition system, which was described in some detail at the last DARPA meeting [9], is an urban exploration system which provides the user with help in locating various sites in the area of Cambridge, Massachusetts. The system has a limited database of objects such as banks, restaurants, and post offices and can provide information about these objects (e.g., phone numbers, type of cuisine served) as well as providing navigational assistance between them. VOYAGER accepts both spoken and typed input and responds in the form of text, graphics, and synthesized speech. Since the last meeting, we have made developments to VOYAGER that have had an impact on the usability of the system.

The VOYAGER speech understanding system: a progress report

As part of the DARPA Spoken Language System program, we recently initiated an effort in spoken language understanding. A spoken language system addresses applications in which speech is used for interactive problem solving between a person and a computer. In these applications, not only must the system convert the speech signal into text, it must also understand the linguistic structure of a sentence in order to generate the correct response. This paper describes our early experience with the development of the MIT VOYAGER spoken language system.

Development and preliminary evaluation of the MIT ATIS system

This paper represents a status report on the MIT ATIS system. The most significant new achievement is that we now have a speech-input mode. It is based on the MIT SUMMIT system using context independent phone models, and includes a word-pair grammar with perplexity 92 (on the June-90 test set). In addition, we have completely redesigned the back-end component, in order to emphasize portability and extensibility. The parser now produces an intermediate semantic frame representation, which serves as the focal point for all back-end operations, such as history management, text generation, and SQL query generation. Most of those aspects of the system that are tied to a particular domain are now entered through a set of tables associated with a small artificial language for decoding them. We have also improved the display of the database table, making it considerably easier for a subject to comprehend the information given. We report here on the results of the official DARPA February-91 evaluation, as well as on results of an evaluation on data collected at MIT, for both speech input and text input.