Cameron G. Smith

Interaction strategies for an affective conversational agent

The development of embodied conversational agents (ECA) as companions brings several challenges for both affective and conversational dialogue. These include challenges in generating appropriate affective responses, selecting the overall shape of the dialogue, providing prompt system response times, and handling interruptions. We present an implementation of such a companion showing the development of individual modules that attempt to address these challenges. Further, to resolve resulting conflicts, we present encompassing interaction strategies that attempt to balance the competing requirements along with dialogues from our working prototype to illustrate these interaction strategies in operation. Finally, we provide the results of an evaluation of the companion using an evaluation methodology created for conversational dialogue and including analysis using appropriateness annotation.

Multimodal and mobile conversational Health and Fitness Companions

Multimodal conversational spoken dialogues using physical and virtual agents provide a potential interface to motivate and support users in the domain of health and fitness. This paper describes how such multimodal conversational Companions can be implemented to support their owners in various pervasive and mobile settings. We present concrete system architectures, virtual, physical and mobile multimodal interfaces, and interaction management techniques for such Companions. In particular how knowledge representation and separation of low-level interaction modelling from high-level reasoning at the domain level makes it possible to implement distributed, but still coherent, interaction with Companions. The distribution is enabled by using a dialogue plan to communicate information from domain level planner to dialogue management and from there to a separate mobile interface. The model enables each part of the system to handle the same information from its own perspective without containing overlapping logic, and makes it possible to separate task-specific and conversational dialogue management from each other. In addition to technical descriptions, results from the first evaluations of the Companions interfaces are presented.

Persuasive dialogue based on a narrative theory: an ECA implementation

Embodied Conversational Agents (ECA) are poised to constitute a specific category within persuasive systems, in particular through their ability to support affective dialogue. One possible approach consists in using ECA as virtual coaches or personal assistants and to make persuasion part of a dialogue game implementing specific argumentation or negotiation features. In this paper, we explore an alternative framework, which emerges from the long-term development of ECA as “Companions” supporting free conversation with the user, rather than task-oriented dialogue. Our system aims at influencing user attitudes as part of free conversation, albeit on a limited set of topics. We describe the implementation of a Companion ECA to which the user reports on his working day, and which can assess the user’s emotional attitude towards daily events in the office, trying to influence such attitude using affective strategies derived from a narrative model. This discussion is illustrated through examples from a first fully-implemented prototype.

Integrating Planning and Dialogue in a Lifestyle Agent

In this paper, we describe an Embodied Conversational Agent advising users to promote a healthier lifestyle. This embodied agent provides advice on everyday user activities, in order to promote a healthy lifestyle. It operates by generating user activity models (similar to decompositional task models), using a Hierarchical Task Network (HTN) planner. These activity models are refined through various cycles of planning and dialogue, during which the agent suggests possible activities to the user, and the user expresses her preferences in return. A first prototype has been fully implemented (as a spoken dialogue system) and tested with 20 subjects. Early results show a high level of task completion despite the word error rate, and further potential for improvement.

Generating context-sensitive ECA responses to user barge-in interruptions

We present an Embodied Conversational Agent (ECA) that incorporates a context-sensitive mechanism for handling user barge-in. The affective ECA engages the user in social conversation, and is fully implemented. We will use actual examples of system behaviour to illustrate. The ECA is designed to recognise and be empathetic to the emotional state of the user. It is able to detect, react quickly to, and then follow up with considered responses to different kinds of user interruptions. The design of the rules which enable the ECA to respond intelligently to different types of interruptions was informed by manually analysed real data from human–human dialogue. The rules represent recoveries from interruptions as two-part structures: an address followed by a resumption. The system is robust enough to manage long, multi-utterance turns by both user and system, which creates good opportunities for the user to interrupt while the ECA is speaking.

Instantiating Interactive Narratives from Patient Education Documents

In this paper, we present a proof-of-concept demonstrator of an Interactive Narrative for patient education. Traditionally, patient education documents are produced by health agencies, yet these documents can be challenging to understand for a large fraction of the population. In contrast, an Interactive Narrative supports a game-like exploration of the situations described in patient education documents, which should facilitate understanding, whilst also familiarising patients with real-world situations. A specific feature of our prototype is that its plan-based narrative representations can be instantiated in part from the original patient education document, using NLP techniques. In the paper we introduce our interactive narrative techniques and follow this with a discussion of specific issues in text interpretation related to the occurrence of clinical actions. We then suggest mechanisms to generate direct or indirect representations of such actions in the virtual world as part of Interactive Narrative generation.

Learning to extract action descriptions from narrative text

This paper focuses on the mapping of natural language sentences in written stories to a structured knowledge representation. This process yields an exponential explosion of instance combinations since each sentence may contain a set of ambiguous terms, each one giving place to a set of instance candidates. The selection of the best combination of instances is a structured classification problem that yields a high-demanding combinatorial optimization problem which, in this paper, is approached by a novel and efficient formulation of a genetic algorithm, which is able to exploit the conditional independence among variables, while improving the parallel scalability. The automatic rating of the resulting set of instance combinations, i.e., possible text interpretations, demands an exhaustive exploitation of the state-of-the-art resources in natural language processing to feed the system with pieces of evidence to be fused by the proposed framework. In this sense, a mapping framework able to reason with uncertainty, to integrate supervision and evidence from external sources, was adopted. To improve the generalization capacity while learning from a limited amount of annotated data, a new constrained learning algorithm for Bayesian networks is introduced. This algorithm bounds the search space through a set of constraints which encode information on mutually exclusive values. The mapping of natural language utterances to a structured knowledge representation is important in the context of game construction, e.g., in an RPG setting, as it alleviates the manual knowledge acquisition bottleneck. The effectiveness of the proposed algorithm is evaluated on a set of three stories, yielding nine experiments. Our mapping framework yields performance gains in predicting the most likely structured representations of sentences when compared with a baseline algorithm.

P309 Serious Games As A New Medium For Patients Guidelines Dissemination

Background Patient education documents are faced with a number of issues that limit their dissemination, from intrinsic readability to how they appeal to patients. Previous studies have demonstrated a positive impact of visual media across the range of patients’ literacy skills, both health-specific and generic. Objectives To devise a production mechanism for serious games in patient education aligned with the current development of patient guidelines, keeping costs manageable in the long term. Methods The MUSE FP7 Project, funded by the European Commission, brings together computer scientists, cognitive psychologists and one GIN member. The project investigates the long-term potential of automatic generation of serious games from patient guidelines documents. It uses state-of-the art commercial gaming technology as well as developing new text analysis software. Results A first prototype has been developed on the topic of bariatric surgery education. It features male and female patient avatars in a hospital environment and supports the interactive exploration and rehearsal of the various stages of the process. All the game actions can be related to specific portions of the patient education document. Discussion The popularity of new media such as computer games improves dissemination prospects for patient education information. In addition, as suggested by recent research, the interactive nature of serious games makes the information more accessible, facilitates learning and addresses issues not covered by textual dissemination such as patient anxiety. Implications for Guideline Developers/Users Serious gaming is poised to become a major health-related medium, hence the need for specific development processes.