Encarna Segarra

A statistical approach to spoken dialog systems design and evaluation

In this paper, we present a statistical approach for the development of a dialog manager and for learning optimal dialog strategies. This methodology is based on a classification procedure that considers all of the previous history of the dialog to select the next system answer. To evaluate the performance of the dialog system, the statistical approach for dialog management has been extended to model the user behavior. The statistical user simulator has been used for the evaluation and improvement of the dialog strategy. Both the user model and the system model are automatically learned from a training corpus that is labeled in terms of dialog acts. New measures have been defined to evaluate the performance of the dialog system. Using these measures, we evaluate both the quality of the simulated dialogs and the improvement of the new dialog strategy that is obtained with the interaction of the two modules. This methodology has been applied to develop a dialog manager within the framework of the DIHANA project, whose goal is the design and development of a dialog system to access a railway information system using spontaneous speech in Spanish. We propose the use of corpus-based methodologies to develop the main modules in the dialog system.

EXTRACTING SEMANTIC INFORMATION THROUGH AUTOMATIC LEARNING TECHNIQUES

We present an approach for the development of Language Understanding systems from a Transduction point of view. We describe the use of two types of automatically inferred transducers as the appropriate models for the understanding phase in dialog systems.

Error handling in a stochastic dialog system through confidence measures

In this work, we present an approach to take advantage of confidence measures obtained during the recognition and understanding processes of a dialog system, in order to guide the behavior of the dialog manager. Our approach allows the system to ask the user for confirmation about the data which have low confidence values associated to them, after the recognition or understanding processes. This technique could help to protect the system from recognition or understanding errors. Although the number of confirmation turns could increase, it would be less probable for the system to consider data with a low confidence value as correct. The understanding module and the dialog manager that we have used are modelled by stochastic automata, and some confidence measures are proposed for the understanding module. An evaluation of the behavior of the dialog system is also presented.

A stochastic approach to dialog management

In this article, we present an approach to the development of a stochastic dialog manager. The model used by this dialog manager to generate its turns takes into account both the last turns of the user and system, and the information supplied by the user throughout the dialog. As the space of situations that can be presented in the dialogs is too large, some techniques for reducing this space have been proposed. This system has been developed in the DIHANA project, whose goal is the design and development of a dialog system to access a railway information system using spontaneous speech in Spanish. A training corpus of 900 dialogs, that was acquired through the Wizard of Oz, was used to learn the models. An evaluation of the dialog manager is also presented

ON THE USE OF THE MORPHIC GENERATOR GRAMMATICAL INFERENCE (MGGI) METHODOLOGY IN AUTOMATIC SPEECH RECOGNITION

Recently, a new methodology, referred to as “Morphic Generator Grammatical Inference” (MGGI), has been introduced as a step towards a general methodology for the inference of regular languages. In this paper we consider the application of this methodology to a real problem of automatic speech recognition, thus allowing (and also requiring) the proposed problem to be properly formulated within the canonical framework of syntactic pattern recognition. The results show both the viability and appropriateness of the application of MGGI to the problem considered.

User simulation in a stochastic dialog system

We present a new methodology of user simulation applied to the evaluation and refinement of stochastic dialog systems. Common weaknesses of these systems are the scarceness of the training corpus and the cost of an evaluation made by real users. We have considered the user simulation technique as an alternative way of testing and improving our dialog system. We have developed a new dialog manager that plays the role of the user. This user dialog manager incorporates several knowledge sources, combining statistical and heuristic information in order to define its dialog strategy. Once the user simulator is integrated into the dialog system, it is possible to enhance the dialog models by an automatic strategy learning. We have performed an extensive evaluation, achieving a slight but clear improvement of the dialog system.

The Incorporation of Confidence Measures to Language Understanding

We present an approach to the definition and application of confidence measures to the speech understanding module in a spoken dialog system, which answers queries about railway timetables and prices by telephone in Spanish. Some experiments have been carried out, and the results in terms of understanding accuracy depending on the threshold of confidence considered, are presented.

Language Understanding Using Two-Level Stochastic Models with POS and Semantic Units

Over the last few years, stochastic models have been widely used in the natural language understanding modeling. Almost all of these works are based on the definition of segments of words as basic semantic units for the stochastic semantic models. In this work, we present a two-level stochastic model approach to the construction of the natural language understanding component of a dialog system in the domain of database queries. This approach will treat this problem in a way similar to the stochastic approach for the detection of syntactic structures (Shallow Parsing or Chunking) in natural language sentences; however, in this case, stochastic semantic language models are based on the detection of some semantic units from the user turns of the dialog. We give the results of the application of this approach to the construction of the understanding component of a dialog system, which answers queries about a railway timetable in Spanish.

Recognition and understanding simulation for a spoken dialog corpus acquisition

Since the design and acquisition of a new dialog corpus is a complex task, new methods to facilitate this task are necessary. In this paper, we present a methodology to make use of our previous work within the framework of dialog systems in order to acquire a dialog corpus for a new domain. The main idea is the simulation of recognition and understanding errors in the acquisition of the new dialog corpus. This simulation is based on the analysis of such errors in a previously acquired corpus and the definition of a correspondence table among the concepts and attributes of both tasks. This correspondence table is based on the similarity of semantic meaning and frequencies. Finally, the application of this methodology is illustrated in some examples.

On the Voice-Activated Question Answering

Question answering (QA) is probably one of the most challenging tasks in the field of natural language processing. It requires search engines that are capable of extracting concise, precise fragments of text that contain an answer to a question posed by the user. The incorporation of voice interfaces to the QA systems adds a more natural and very appealing perspective for these systems. This paper provides a comprehensive description of current state-of-the-art voice-activated QA systems. Finally, the scenarios that will emerge from the introduction of speech recognition in QA will be discussed.