Gonzalo Espejo

A multimodal dialogue system for an ambient intelligent application in home environments

This paper presents a multimodal dialogue system called Mayordomo which aims at easing the interaction with home appliances using speech and a graphical interface within an Ambient Intelligence environment. We present the methods employed for implementing the system describing the design of the user-system interactions as well as additional features such as the management of user profiles to restrict the access to domestic appliances and customize the recognition grammars and the generated responses.

A methodology for learning optimal dialog strategies

In this paper, we present a technique for learning new dialog strategies by using a statistical dialog manager that is trained from a dialog corpus. A dialog simulation technique has been developed to acquire data required to train the dialog model and then explore new dialog strategies. A set of measures has also been defined to evaluate the dialog strategy that is automatically learned. We have applied this technique to explore the space of possible dialog strategies for a dialog system that collects monitored data from patients suffering from diabetes.

Towards Fine-Grain User-Simulation for Spoken Dialogue Systems

Continuous advances in the field of spoken dialogue systems make the processe of design, implementation and evaluation of these systems more and more com-s plex. To solve problems emerging from this complexity, a technique which has attracted increasing interest during the last decades is based on the automatic generation of dialogues between the system and a user simulator, which is another system that represents human interactions with the dialogue system. This chapter describes the main methodologies and techniques developed to create user simulators, and presents a discussion of their main characteristics and the benefits that they provide for the development, improvement and assessment of this kind of systems. Additionally, we propose a user simulation technique to test the performance of spoken dialogue systems. The technique is based on a novel approach to simulating different levels of user cooperativeness, which allows carrying out a more detailed system assessment. In the experiments we have evaluated a spoken dialogue system designed for the fast food domain. Theevaluation has focused on the performance of the speech recogniser, semantic analyser and dialogue manager of this system. The results show that the technique provides relevant information to obtain a solid evaluation of the system, enabling us to find problems in these modules which cannot be observed taking into account just one cooperativeness level.

A System to Learn Frequent Behavioural Patterns

Intelligent Environments (IEs) are expected to support people in their daily lives. To achieve this goal, the environment should learn how to react to the actions and needs of the users, and this should be achieved in an unobtrusive and transparent way. In order to provide personalized and adapted services, it is necessary to know the preferences and habits of users. Thus, the ability to learn patterns of behaviour becomes an essential aspect for the successful implementation of IEs. This paper presents a system, Learning Frequent Patterns of User Behaviour System (LFPUBS), that discovers users frequent behaviours. LFPUBS was validated using data collected from real environments.

Multimodal Dialogue System for Interaction in AmI Environment by Means of File-Based Services

This paper presents our ongoing work on the development of a multimodal dialogue system to enable user control of home appliances in an ambient intelligence environment. The physical interaction with the appliances is carried out by means of Octopus, a system developed in a previous study to ease communication with hardware devices by abstracting them as network files. To operate the appliances and get information about their state, the dialogue system writes and reads files using WebDAV. This architecture presents an important advantage since the appliances are considered as abstract objects, which notably simplifies dialogue system’s interaction with them.

Mayordomo and Octopus : a case study in ambient intelligence environments

Mayordomo is a multimodal dialogue system developed to ease user interaction with home appliances using speech and a GUI interface. Octopus is an operating system designed to provide ubiquitous access to computing resources using the Internet as the interconnection means. In this paper we describe the main characteristics of both systems as well as our ongoing work to use both systems in conjunction for operation in an Ambient Intelligence environment.

Design and Implementation of a Toolkit for Evaluation of Spoken Dialogue Systems Designed for AmI Environments

This paper presents a new toolkit for the evaluation of spoken dialogue systems in Ambient Intelligence domains. It describes the design and implementation of the toolkit, which consists in several tools that can be employed to process corpora of recorded spoken interaction and automatically obtain statistical measures in order to assess and improve these systems. The toolkit has been applied to evaluate the Mayordomo multimodal dialogue system developed in our laboratory, which enables automatic control of home appliances. This evaluation was useful to detect problems in the dialogue system suggesting which modules must be improved.

Using knowledge about misunderstandings to increase the robustness of spoken dialogue systems

This paper proposes a new technique to enhance the performance of spoken dialogue systems employing a method that automatically corrects semantic frames which are incorrectly generated by the semantic analyser of these systems. Experiments have been carried out using two spoken dialogue systems previously developed in our lab: Saplen and Viajero, which employ prompt-dependent and prompt-independent language models for speech recognition. The results obtained from 10,000 simulated dialogues show that the technique improves the performance of the two systems for both kinds of language modelling, especially for the prompt-independent language model. Using this type of model the Saplen system increased sentence understanding by 19.54%, task completion by 26.25%, word accuracy by 7.53%, and implicit recovery of speech recognition errors by 20.30%, whereas for the Viajero system these figures increased by 14.93%, 18.06%, 6.98% and 15.63%, respectively.