Allen L. Gorin

Grammar fragment acquisition using syntactic and semantic clustering

A method and apparatus are provided for automatically acquiring grammar fragments for recognizing and understanding fluently spoken language. Grammar fragments representing a set of syntactically and semantically similar phrases may be generated using three probability distributions: of succeeding words, of preceding words, and of associated call-types. The similarity between phrases may be measured by applying Kullback-Leibler distance to these three probability distributions. Phrases being close in all three distances may be clustered into a grammar fragment.

Automated natural spoken dialog

The next generation of voice-based interface technology will enable easy-to-use automation of new and existing communication services, making human-machine interaction more natural.

Active learning for automatic speech recognition

State-of-the-art speech recognition systems are trained using transcribed utterances, preparation of which is labor intensive and time-consuming. In this paper, we describe a new method for reducing the transcription effort for training in automatic speech recognition (ASR). Active learning aims at reducing the number of training examples to be labeled by automatically processing the unlabeled examples, and then selecting the most informative ones with respect to a given cost function for a human to label. We automatically estimate a confidence score for each word of the utterance, exploiting the lattice output of a speech recognizer, which was trained on a small set of transcribed data. We compute utterance confidence scores based on these word confidence scores, then selectively sample the utterances to be transcribed using the utterance confidence scores. In our experiments, we show that we reduce the amount of labeled data needed for a given word accuracy by 27%.

Automatically training a problematic dialogue predictor for a spoken dialogue system

Spoken dialogue systems promise efficient and natural access to a large variety of information sources and services from any phone. However, current spoken dialogue systems are deficient in their strategies for preventing, identifying and repairing problems that arise in the conversation. This paper reports results on automatically training a Problematic Dialogue Predictor to predict problematic human-computer dialogues using a corpus of 4692 dialogues collected with the How May I Help YouSM spoken dialogue system. The Problematic Dialogue Predictor can be immediately applied to the systems decision of whether to transfer the call to a human customer care agent, or be used as a cue to the systems dialogue manager to modify its behavior to repair problems, and even perhaps, to prevent them. We show that a Problematic Dialogue Predictor using automatically-obtainable features from the first two exchanges in the dialogue can predict problematic dialogues 13.2% more accurately than the baseline.

Adaptive acquisition of language

Abstract At present, automatic speech recognition technology is based upon constructing models of the various levels of linguistic structure assumed to compose spoken language. These models are either constructed manually or automatically trained by example. A major impediment is the cost, or even the feasibility, of producing models of sufficient fidelity to enable the desired level of performance. The proposed alternative is to build a device capable of acquiring the necessary linguistic skills in the course of performing its task . We call this learning by doing , and contrast it with learning by example . The purpose of this paper is to describe some basic principles and mechanisms upon which such a device might be based, and to recount a rudimentary experiment evaluating their utility. Spoken language, the original natural language, evolved in order for humans to convey importnat messages to each other. A first principle, then, is that the primary function of language is to communicate. A consequence of this principle is that language acquisition involves gaining the capability of decoding the message. This is in contrast to much of the research on automated language acquisition, which focuses on discovering syntactic structure, often specifically to the exclusion of meaning. Our first principle leads us to investigate a language acquisition mechanism based on connectionist methods, in which the network builds associations between messages and meaningful responses to them. People learn while performing a task by receiving feedback as to the appropriateness of their actions. A second principle, then, is that the actual construction of the mapping from messages to meaning should be governed by a feedback control system where the error signal is at the level of meaning. This is in contrast to some learning research, which is governed by providing input/output pairs, and where the error signal is a parameter-space distortion measure. Our second principle leads us to investigate a mechanism for human-machine interaction based on control-theory methods, where the system input is the message and the error signal is a measure of appropriateness of the machines response. The utility of these principles is demonstrated and evaluated by applying them to an elementary inward-call-management task, the object of which is to connect a caller to the department of a large organization appropriate to his inquiry. Initially, the system knows nothing about the language for its task, that is no vocabulary, no grammer, and no semantic associations. In the course of directing incoming calls, the system acquires a vocabulary, learns the meaning of words and some rudimentary grammatical relationships relevant to its task. The mechanism used is a particular connectionist network embedded in a feedback control system which adjusts the connection weights of the network based on the success or failure of the machines behavior, as evaluated by the callers reaction to it. This mechanism has several intriguing mathematical properties. An experimental evaluation of the system has been conducted using typed rather than spoken input. The system was tested by 12 subjects over a 2-month period. Over 1000 conversations were held, during which the machine acquired a vocabulary of over 1500 words. Subsequent tests showed that the learning was stable, in that it retained 99% of the knowledge it had acquired in the interactions. Although the experiments conducted thus far are of a rudimentary nature, we consider them to be the early stages in a long-term study of automatic acquisition of intelligence by machines through interaction with a complex environment.

Construct Algebra: Analytical Dialog Management

In this paper we describe a systematic approach for creating a dialog management system based on a Construct Algebra, a collection of relations and operations on a task representation. These relations and operations are analytical components for building higher level abstractions called dialog motivators. The dialog manager, consisting of a collection of dialog motivators, is entirely built using the Construct Algebra.

Detecting and extracting named entities from spontaneous speech in a mixed-initiative spoken dialogue context: How May I Help You?sm,tm

The understanding module of a spoken dialogue system must extract, from the speech recognizer output, the kind of request expressed by the caller (the call type) and its parameters (numerical expressions, time expressions or propernames). Such expressions are called Named Entities and their definitions can be either generic or linked to the dialogue application domain. Detecting and extracting such Named Entities within a mixed-initiative dialogue context like How May I Help You? sm;tm (HMIHY) is the subject of this study. After reviewing standard methods based on hand-written grammars and statistical tagging, we propose a new approach, combining the advantages of both in a 2-step process. We also propose a novel architecture which exploits understanding to improve recognition accuracy: the output of the Automatic Speech Recognition module is now a word lattice and the understanding module is responsible for transcribing the word strings which are useful to the Dialogue Manager. All the methods proposed are trained and evaluated on a corpus comprising utterances from live customer traffic. � 2003 Elsevier B.V. All rights reserved.

Method and system for predicting understanding errors in automated dialog systems

This invention concerns a method and system for monitoring an automated dialog system for the automatic recognition of language understanding errors based on a users input communications. The method includes determining whether a probability of understanding the users input communication exceeds a first thresholds, where if the first threshold is exceeded, further dialog is conducted with the user. Otherwise, the user may be directed to a human for assistance. In another possible embodiment, the method operates as above except that if the probability also exceeds a second threshold, the second threshold being higher than the first, then further dialog is conducted with the user using the current dialog strategy. However, if the probability falls between a first threshold and a second threshold, the dialog strategy may be adapted in order to improve the chances of conducting a successful dialog with the user.

Social correlates of turn-taking behavior

The goal of this research is to infer traits about groups of people from their turn-taking behavior in natural conversation. These traits are latent attributes in a social network, whose relative frequencies we estimate from content-derived metadata. Our approach is to train statistical models of turn-taking behavior using automatic labels of speech activity, and measure the association of these models with socially correlated traits. We experimentally evaluate these ideas using the Switchboard-1 speech corpus, which provides speech content and metadata associated with each speaker, such as gender, age and education, as well as inferred social correlates such as willingness to participate and initiate. We show that population proportions of these socially correlated externals can be predicted with a root meansquared error of approximately 0.1 across all mixture proportions.

Adaptive acquisition of spoken language

The problem of building a device that acquires language during the course of performing its task, called learning by doing, is considered. Some basic principles and mechanisms upon which such a device might be constructed are described. In particular, a language acquisition mechanism that is based upon the intuition of building associations between messages and appropriate responses to them and a mechanism for human-machine interaction based on control theory methods are investigated. A conversational-mode system that demonstrates and evaluates the proposed principles and mechanisms is described. Experimental results that validate the approach are reported.<<ETX>>