Ian M. O'Neill

Handling errors and determining confirmation strategies - an object-based approach

Abstract A number of different approaches have been applied to the treatment of errors in spoken dialogue systems, including careful design to prevent potential errors, methods for on-line error detection, and error recovery when errors have occurred and have been detected. The approach to error handling presented here is premised on the theory of grounding, in which it is assumed that errors cannot be avoided in spoken dialogue and that it is more useful to focus on methods for determining what information needs to be grounded within a dialogue and how this grounding should be achieved. An object-based architecture is presented that incorporates generic confirmation strategies in combination with domain-specific heuristics that together contribute to determining the system’s confirmation strategies when attempting to complete a transaction. The system makes use of a representation of the system’s information state as it conducts a transaction along with discourse pegs that are used to determine whether values have been sufficiently confirmed for a transaction to be concluded. An empirical evaluation of the system is presented along with a discussion of the advantages of the object-based approach for error handling.

Implementing advanced spoken dialogue management in Java

In this article we describe how Java can be used to implement an object-based, cross-domain, mixed initiative spoken dialogue manager (DM). We describe how dialogue that crosses between several business domains can be modelled as an inheriting and collaborating suite of objects suitable for implementation in Java. We describe the main features of the Java implementation and how the Java dialogue manager can be interfaced via the Galaxy software hub, as used in the DARPA-sponsored Communicator projects in the United States, with the various off-the-shelf components that are needed in a complete end-to-end spoken dialogue system. We describe the interplay of the Java components in the course of typical dialogue turns and present an example of the sort of dialogue that the Java DM can support.

Promoting extension and reuse in a spoken dialog manager: An evaluation of the queen's communicator

This article describes how an object-oriented approach can be applied to the architectural design of a spoken language dialog system with the aim of facilitating the modification, extension, and reuse of discourse-related expertise. The architecture of the developed system is described and a functionally similar VoiceXML system is used to provide a comparative baseline across a range of modification and reuse scenarios. It is shown that the use of an object-oriented dialog manager can provide a capable means of reusing existing discourse expertise in a manner that limits the degree of structural decay associated with system change.

Adaptive dialogue strategy selection through imprecise probabilistic query answering

In a human-computer dialogue system, the dialogue strategy can range from very restrictive to highly flexible. Each specific dialogue style has its pros and cons and a dialogue system needs to select the most appropriate style for a given user. During the course of interaction, the dialogue style can change based on a users response and the system observation of the user. This allows a dialogue system to understand a user better and provide a more suitable way of communication. Since measures of the quality of the users interaction with the system can be incomplete and uncertain, frameworks for reasoning with uncertain and incomplete information can help the system make better decisions when it chooses a dialogue strategy. In this paper, we investigate how to select a dialogue strategy based on aggregating the factors detected during the interaction with the user. For this purpose, we use probabilistic logic programming (PLP) to model probabilistic knowledge about how these factors will affect the degree of freedom of a dialogue. When a dialogue system needs to know which strategy is more suitable, an appropriate query can be executed against the PLP and a probabilistic solution with a degree of satisfaction is returned. The degree of satisfaction reveals how much the system can trust the probability attached to the solution.

Using assessment software to create a dialogue-based tutorial

A tutorials enable students to prepare and revise topics whenever they wish. Very large and growing class sizes on key pathways mean that busy tutors and students are glad of software programs that automate some of the more routine student-tutor interactions. I present a worked example of a short automated adaptive tutorial, inspired by human-to-human dialogue and created using the kind of widely deployed assessment software that is used for authoring multiple-response (check-box) exam questions. Combining their teaching experience with some basic control logic, tutors can use to good educational effect the distinctive answer combinations that students provide.

Using probabilistic logic for dialogue strategy selection

Automated dialogue strategies range from very restrictive (limiting a person to yes/no responses) to highly flexible (the person can say what they like): dialogue strategies offer different ‘degrees of freedom’. Strategies may change midtransaction as a system responds to a user’s verbal and non-verbal input. Our prototype system aggregates a selection of strategy-influencing factors and uses a probabilistic logic program (PLP) to exploit developers’ understanding of how typical factors affect the degree of freedom in a dialogue. Not only is it possible to compute a ‘crisp’ probability that a ‘free’ dialogue strategy is appropriate in particular circumstances, but it is also possible to calculate degrees of satisfaction indicating how reliable probabilities within particular ranges are as the answer to a PLP query.

Development of a Java-based unified and flexible natural language discourse system

This paper outlines the design and development of a Java-based, unified and flexible natural language dialogue system that enables users to interact using natural language, e.g. speech. A number of software development issues are considered with the aim of designing an architecture that enables different discourse components to be readily and flexibly combined in a manner that permits information to be easily shared. Use of XML schemas assists this component interaction. The paper describes how a range of Java language features were employed to support the development of the architecture, providing an illustration of how a modern programming language makes tractable the development of a complex dialogue system.

Flexible Natural Language Generation in Multiple Contexts

We present a practical approach to Natural Language Generation (NLG) for spoken dialogue systems. The approach is based on small template fragments (mini-templates). The systems object architecture facilitates generation of phrases across pre-defined business domains and registers, as well as into different languages. The architecture simplifies NLG in well-understood application contexts, while providing the flexibility for a developer and for the system, to vary linguistic output according to dialogue context, including any intended affective impact. Mini-templates are used with a suite of domain term objects, resulting in an NLG system (MINTGEN --- MINi-Template GENerator) whose extensibility and ease of maintenance is enhanced by the sparsity of information devoted to individual domains. The system also avoids the need for specialist linguistic competence on the part of the system maintainer.

A Data Model for Content Modelling of Temporal Media

This paper describes a data model for content representation of temporal media in an IP based sensor network. The model is formed by introducing the idea of semantic-role from linguistics into the underlying concepts of formal event representation with the aim of developing a common event model. The architecture of a prototype system for a multi camera surveillance system, based on the proposed model is described. The important aspects of the proposed model are its expressiveness, its ability to model content of temporal media, and its suitability for use with a natural language interface. It also provides a platform for temporal information fusion, as well as organizing sensor annotations by help of ontologies.

A Speech Based Approach to Surveillance Video Retrieval

This paper describes the anatomy of a pilot surveillance system with a speech-based interface for content-based retrieval of video data. The proposed system relies on an ontology-based information sharing architecture and lets components of the system communicate among each other through TCP/IP communication channels. The aim of developing the pilot system was to explore dependencies between image analysis, event detection, video annotation, and speech- based retrieval of the video content in the context of a broader spoken dialogue system.