Sunil Issar

Recent improvements in the CMU spoken language understanding system

We have been developing a spoken language system to recognize and understand spontaneous speech. It is difficult for such systems to achieve good coverage of the lexicon and grammar that subjects might use because spontaneous speech often contains disfluencies and ungrammatical constructions. Our goal is to respond appropriately to input, even though coverage is not complete. The natural language component of our system is oriented toward the extraction of information relevant to a task, and seeks to directly optimize the correctness of the extracted information (and therefore the system response). We use a flexible frame-based parser, which parses as much of the input as possible. This approach leads both to high accuracy and robustness. We have implemented a version of this system for the Air Travel Information Service (ATIS) task, which is being used by several ARPA-funded sites to develop and evaluate speech understanding systems. Users are asked to perform a task that requires getting information from an Air Travel database. In this paper, we describe recent improvements in our system resulting from our efforts to improve the coverage given a limited amount of training data. These improvements address a number of problems including generating an adequate lexicon and grammar for the recognizer, generating and generalizing an appropriate grammar for the parser, and dealing with ambiguous parses.

A class based language model for speech recognition

Class based language models are often used when there is insufficient data to generate a word based language model directly from the training data. In this approach, similar items are clustered into classes, an n-gram language model for the class tokens is generated, and then the probabilities for words in a class are distributed according to the smoothed relative unigram frequencies of the words. Classes expand to lists of single word tokens, that is, a class cannot represent a sequence of lexical tokens. We propose a more general mechanism for defining a language model class. In it, classes are expanded to word sequences through finite-state networks. This allows expansion to word sequences without requiring compound words in the lexicon. Where finite-state models are too brittle to represent sentence-level strings, they can represent class-level strings (dates, names and titles for example). We compared the perplexity on the ARPA Dec93 ATIS Test set and found that the new model reduced the perplexity by approximately 17 percent (relative).

TPS: A theorem-proving system for classical type theory

This is description of TPS, a theorem-proving system for classical type theory (Churchs typed λ-calculus). TPS has been designed to be a general research tool for manipulating wffs of first- and higher-order logic, and searching for proofs of such wffs interactively or automatically, or in a combination of these modes. An important feature of TPS is the ability to translate between expansion proofs and natural deduction proofs. Examples of theorems that TPS can prove completely automatically are given to illustrate certain aspects of TPSs behavior and problems of theorem proving in higher-order logic.

The TPS Theorem Proving System

When one is seeking an expansion proof for a theorem of higher-order logic, not all necessary substitution terms can be generated by unification of formulas already present, so certain expansion options [5] are applied, and then a search for a p-acceptable mating [2] is made, using Huets higher-order unification algorithm [8] to generate all remaining substitution terms. The expansion options consist of quantifier duplications and projective and primitive substitutions (such as

ETPS: A System to Help Students Write Formal Proofs

ETPS (Educational Theorem Proving System) is a program that logic students can use to write formal proofs in first-order logic or higher-order logic. It enables students to concentrate on the essential logical problems involved in proving theorems, and it automatically checks the proofs.

Integrating semantic constraints into the Sphinx-II recognition search

This paper describes a novel recursive transition network (RTN) speech decoder designed for robust processing of spontaneous spoken input. The system uses both stochastic and rule-based methods to constrain the recognition search. It is based on the multi-pass search used by the Sphinx-II speech decoder developed at Carnegie Mellon. We describe the basic decoder and system architectures and compare the system to our current loosely coupled system on spontaneous spoken dialogs from the DARPA air travel (ATIS) task.<<ETX>>

Speech understanding in open tasks

The Air Traffic Information Service task is currently used by DARPA as a common evaluation task for Spoken Language Systems. This task is an example of open type tasks. Subjects are given a task and allowed to interact spontaneously with the system by voice. There is no fixed lexicon or grammar, and subjects are likely to exceed those used by any given system. In order to evaluate system performance on such tasks, a common corpus of training data has been gathered and annotated. An independent test corpus was also created in a similar fashion. This paper explains the techniques used in our system and the performance results on the standard set of tests used to evaluate systems.

TPS: An Interactive and Automatic Tool for Proving Theorems of Type Theory

This is a demonstration of TPS, a theorem proving system for classical type theory (Churchs typed λ-calculus). TPS can be used interactively or automatically, or in a combination of these modes. An important feature of TPS is the ability to translate between expansion proofs and natural deduction proofs.

Unanswerable queries in a spontaneous speech task

This paper outlines the general strategies followed in developing the CMU (Carnegie Mellon University) speech understanding system. Our system is oriented toward the extraction of information relevant to a task. It uses a flexible frame-based parser. Our system handles phenomena that are natural in spontaneous speech, for example, restarts, repeats and grammatically ill-formed utterances. It maintains a history of the key features of the dialogue. It can resolve elliptical, anaphoric and other indirect references. In general, the system may not be able to generate a database response to a query for a number of reasons, for example, the user may inquire about information not in the database. These unanswerable (Class X) queries constituted about 30% of the data in the November 92 ARPA Spoken Language Systems benchmark evaluation. We will present some results which indicate that the perplexity of the unanswerable queries is significantly higher than the perplexity of the remaining data. In this paper, we focus on our attempts to improve the performance of CMUs Spoken Language System on the unanswerable queries. The system has been used to model an air travel information service (ATIS) task. In the December 93 DARPA Spoken Language Systems benchmark evaluation, the CMU ATIS system correctly answered 90.7% transcript inputs and 86.8% speech inputs. These were the best numbers reported for the evaluation.<<ETX>>