Valeria Carofiglio

Embodied contextual agent in information delivering application

We aim at building a new human-computer interface for Information Delivering applications: the conversational agent that we have developed is a multimodal believable agent able to converse with the User by exhibiting a synchronized and coherent verbal and nonverbal behavior. The agent is provided with a personality and a social role, that allows her to show her emotion or to refrain from showing it, depending on the context in which the conversation takes place. The agent is provided with a face and a mind. The mind is designed according to a BDI structure that depends on the agents personality; it evolves dynamically during the conversation, according to the Users dialog moves and to emotions triggered as a consequence of the Interlocutors move; such cognitive features are then translated into facial behaviors. In this paper, we describe the overall architecture of our system and its various components; in particular, we present our dynamic model of emotions. We illustrate our results with an example of dialog all along the paper. We pay particular attention to the generation of verbal and nonverbal behaviors and to the way they are synchronized and combined with each other. We also discuss how these acts are translated into facial expressions.

Greta. A Believable Embodied Conversational Agent

1. INTELLIGENT BELIEVABLE EMBODIED CONVERSATIONAL AGENTS A wide area of research on Autonomous Agents is presently devoted to the construction of ECAs, Embodied Conversational Agents (Cassell et al. 2000; Pelachaud & Poggi, 2001). An ECA is a virtual Agent that interacts with a User or another Agent through multimodal communicative behavior. It has a realistic or cartoon-like body and it can produce spoken discourse and dialogue, use voice with appropriate prosody and intonation, exhibit the visemes corresponding to the words uttered, make gestures, assume postures, produce facial expression and communicative gaze behavior. An ECA is generally a Believable Agent, that is, one able to express emotion (Bates, 1994) and to exhibit a given personality (Loyall & Bates, 1997). But, according to recent literature (Trappl & Payr, in press; de Rosis et al., in press a), an Agent is even more believable if it can behave in ways typical of given cultures, and if it has a personal communicative style (Canamero & Aylett, in press; Ruttkay et al., in press). This is, in fact, what makes a human a human. More, an ECA must be interactive, that is, take User and context into account, so as to tailor interaction onto the particular User and context at hand. In an ECA that fulfils these constraints the communicative output, that is, the particular combination of multimodal communicative signals displayed (words, prosody, gesture, face, gaze, body posture and movements) is determined by different aspects: a. contents to communicate, b. emotions, c. personality, d. culture, e. style, f. context and User sensitivity. At each moment of a communicative interaction, all of these aspects combine with each other to determine what the Agent will say, and how. In this paper we show how these aspects of an ECA can be modeled in terms of a belief and goal view of human communicative behavior. We then illustrate Greta, an ECA following these principles which is being implemented in the context of the EU project MagiCster

Can Computers Deliberately Deceive? A Simulation Tool and Its Application to Turing's Imitation Game

In this paper, we describe how agents can deceive within a probabilistic framework for representing their mental state: in doing so, we challenge the so‐called sincerity assumption in Human–Computer interaction (HCI) and multi‐agent systems (MAS). We distinguish deception from its special case of lie and characterize different forms of deception, by identifying several criteria for distinguishing among them. In particular, we propose a model of information impact on the Receivers mind. As the message Sender must plan its strategy by considering the Receivers criteria for believing, we also discuss some of these criteria, like content plausibility, source informativity, and information safety. We apply this model to a simplified version of Turings Imitation Game and describe how we implemented a Simulator of deceptive strategies that we called Mouth of Truth. We conclude the paper by describing an evaluation study that enabled us to verify the validity of our method and to revise it in part.

Affective Advice Giving Dialogs

In affective dialog simulation, recognition and interpretation of the affective state of the user should be integrated with display of empathy by the system and with dialog planning and execution. Cognitive models dealing with the inherent uncertainty of this interpretation are the method we propose to adopt. We describe how we integrated these models in an information-state approach to dialog modeling by illustrating, in particular, their application to a decision support system which is tailored to the ‘state of change’ of the user.

Shallow and Inner Forms of Emotional Intelligence in Advisory Dialog Simulation

A conversational agent aspiring to be believable in an emotional domain should be able to combine its rational and emotional intelligence in a consistent way. We claim that a cognitive model of emotion activation may contribute to this aim by providing knowledge to be employed in modeling emotion regulation and its influence on the dialog dynamics. We show how an XML markup language contributes to insuring independence between the agent’s body and mind, so as to favor adaptation of the dialog to the user characteristics. An example dialog in the eating disorder domain is employed throughout the chapter to illustrate the methods developed and the implemented prototype.

Exploiting Uncertainty and Incomplete Knowledge in Deceptive Argumentation

Argumentation is not always sincere. This is evident in competitive domains like politics or trading but occurs, as well, in domains in which debates are commonly considered to be governed by ‘purely rational’, non malicious goals and forms of reasoning, like science [3,4,5,7]. When people argue for or against a claim, the differences of arguments they employ are due, in part, to differences in the data they know or in the importance they attach to each argument. However, these differences may originate, as well, from the use of arguments that are not fully sincere: it is therefore worth reflecting on whether and how the various forms of deception that may be employed in argumentation might be formalised.

Cognitive Evaluations and Intuitive Appraisals: Can Emotion Models Handle Them Both?

This chapter deals with the complex relationships between cognitive representations and processes (not reduced to “epistemic” representations but including the motivational ones: goals) and emotions. It adopts a belief–desire–intention paradigm (the explicit account of mental representations and of their “reading” in interaction), but psychologically and computationally sophisticated: for example, by a “dual-process” theory, distinguishing the “intuitive thinking” from the “deliberative thinking,” or by a probabilistic approach to the beliefs–goals network. This representation of the mental background of the emotion is also necessary for accounting for emotional interaction, which is based on mind reading, not just on emotional expressions.

Emotional dialogs with an embodied agent

We discuss how simulating emotional dialogs with an Embodied Agent requires endowing it with ability to manifest appropriately emotions but also to exploit them in controlling behavior. We then describe a domain-independent testbed to simulate dialogs in affective domains and verify how they change when the context in which interaction occurs is varied. Emotion activation is simulated by dynamic belief networks while dialog simulation is implemented within a logical framework.

Dynamic user modeling in health promotion dialogs

We describe our experience with the design, implementation and revision of a dynamic user model for adapting health promotion dialogs with ECAs to the ‘stage of change’ of the users and to their ‘social’ attitude toward the agent. The user model was built by learning a bayesian network from a corpus of data collected with a Wizard of Oz study. We discuss how uncertainty in the recognition of the user’s mental state may be reduced by integrating a simple linguistic parser with knowledge about the interaction context represented in the model.

User brain-driven evaluation of an educational 3D virtual environment

In evaluating interactive systems, several attempts have been made to broaden the traditional focus on the efficient achievement of goals and incorporate an understanding of additional aspects linked to the User eXperience (UX). In this direction, usability tests also based on the users emotional level should be performed. This requires the introduction of suitable methods which can capture and convert subjective feelings into concrete design parameters. In this paper we propose a user-centered approach to the evaluation of the UX in a 3D Educational Virtual Environment, in which the user is required to explore reconstructions of cultural places and to solve games for educational purposes by using innovative controllers. We employ a commercial Brain Computer Interface to collect information on the users feelings. The preliminary results of this research show that our method allows the identification of elements that do not result in high subject agreement in terms of elicited emotions. Moreover, it provides the opportunity to redesign the environment. The domain-independent nature of the presented approach allows designers to apply it in a wide range of 3D interactive environments, such as games and serious games, training simulators, etc.