Arthur E. McNair

Locating and correcting erroneously recognized portions of utterances by rescoring based on two n-best lists

A method of repairing machine-recognized speech is comprised of the steps of receiving from a recognition engine a first n-best list of hypotheses and scores for each hypothesis generated in response to a primary utterance to be recognized. An error within the hypothesis having the highest score is located. Control signals are generated from the first n-best list which are input to the recognition engine to constrain the generation of a second n-best list of hypotheses, and scores for each hypothesis, in response to an event independent of the primary utterance. The scores for the hypotheses in the first n-best list are combined with the scores for the hypotheses in the second n-best list. The hypothesis having the highest combined score is selected as the replacement for the located error.

JANUS: a speech-to-speech translation system using connectionist and symbolic processing strategies

The authors present JANUS, a speech-to-speech translation system that utilizes diverse processing strategies including dynamic programming, stochastic techniques, connectionist learning, and traditional AI knowledge representation approaches. JANUS translates continuously spoken English utterances into Japanese and German speech utterances. The overall system performance on a corpus of conference registration conversations is 87%. Two versions of JANUS are compared: one using a LR parser (JANUS 1) and one using a connectionist parser (JANUS 2). Performance results were mixed, with JANUS 1 deriving benefit from a tighter language model and JANUS 2 benefitting from greater flexibility.<<ETX>>

Recent advances in Janus: a speech translation system

We present recent advances from our efforts in increasing coverage, robustness, generality and speed of JANUS, CMUs speech-to-speech translation system. JANUS is a speaker-independent system translating spoken utterances in English and also in German into one of German, English or Japanese. The system has been designed around the task of conference registration (CR). It has initially been built based on a speech database of 12 read dialogs, encompassing a vocabulary of around 500 words. We have since been expanding the system along several dimensions to improve speed, robustness and coverage and to move toward spontaneous input.

JANUS 93: towards spontaneous speech translation

We present first results from our efforts toward translation of spontaneously spoken speech. Improvements include increasing coverage, robustness, generality and speed of JANUS, the speech-to-speech translation system of Carnegie Mellon and Karlsruhe University. The recognition and machine translation engine have been upgraded to deal with requirements introduced by spontaneous human to human dialogs. To allow for development and evaluation of our system on adequate data, a large database with spontaneous scheduling dialogs is being gathered for English, German and Spanish.<<ETX>>

JANUS-II-translation of spontaneous conversational speech

JANUS-II is a research system to design and test components of speech-to-speech translation systems as well as a research prototype for such a system. We focus on two aspects of the system: (1) the new features of the speech recognition component JANUS-SR, and (2) the end-to-end performance of JANUS-II, including a comparison of two machine translation strategies used for JANUS-MT (PHOENIX and GLR*).

Testing generality in JANUS: a multi-lingual speech translation system

For speech translation to be practical and useful, speech translation systems should be portable to multiple languages without substantial modification. The authors present results of expanding the English-based JANUS speech translation system to translate from spoken German sentences to English and Japanese utterances. The authors also report the results of implementing part of the linked predictive neural network (LPNN) speech recognition module on a massively parallel machine. The JANUS approach generalizes well, with overall system performance of 97%. This surpasses English-based JANUS performance.<<ETX>>

Integrating different learning approaches into a multilingual spoken language translation system

Building multilingual spoken language translation systems requires knowledge about both acoustic models and language models of each language to be translated. Our multilingual translation system JANUS-2 is able to translate English and German spoken input into either English, German, Spanish, Japanese or Korean output. Getting optimal acoustic and language models as well as developing adequate dictionaries for all these languages requires a lot of hand-tuning and is time-consuming and labor intensive. In this paper we will present learning techniques that improve acoustic models by automatically adapting codebook sizes, a learning algorithm that increases and adapts phonetic dictionaries for the recognition process and also a statistically based language model with some linguistic knowledge that increases recognition performance. To ensure a robust translation system, semantic rather than syntactic analysis is done. Concept based speech translation and a connectionist parser that learns to parse into feature structures are introduced. Furthermore, different repair mechanisms to recover from recognition errors will be described.