Etienne de Sevin

A Model of Personality and Emotional Traits

How do we construct credible personalities? The current SAL (Sensitive Artificial Listeners) characters were constructed intuitively and can be unconvincing. In addressing these issues, this paper considers a theory of personality and associated emotional traits, and discusses how behaviours associated with personality types in people may be adapted to develop characteristics of virtual agents. Our objective is to ensure that behavioural perceptions of a virtual agent credibly reflect the agents `actual personality as prescribed.

A listener model: introducing personality traits

We present a computational model that generates listening behaviour for a virtual agent. It triggers backchannel signals according to the user’s visual and acoustic behaviour. The appropriateness of the backchannel algorithm in a user-agent situation of storytelling, has been evaluated by naïve participants, who judged the algorithm-ruled timing of backchannels more positively than a random timing. The system can generate different types of backchannels. The choice of the type and the frequency of the backchannels to be displayed is performed considering the agent’s personality traits. The personality of the agent is defined in terms of two dimensions, extroversion and neuroticism. We link agents with a higher level of extroversion to a higher tendency to perform more backchannels than introverted ones, and we link neuroticism to less mimicry production and more response and reactive signals sent. We run a perception study to test these relations in agent-user interactions, as evaluated by third parties. We find that the selection of the frequency of backchannels performed by our algorithm contributes to the correct interpretation of the agent’s behaviour in terms of personality traits.

A demonstration of audiovisual sensitive artificial listeners

Sensitive artificial listeners (SAL) are virtual dialogue partners who, despite their very limited verbal understanding, intend to engage the user in a conversation by paying attention to the users emotions and non-verbal expressions. The SAL characters have their own emotionally defined personality, and attempt to drag the user towards their dominant emotion, through a combination of verbal and non-verbal expression. The demonstrator shows an early version of the fully autonomous SAL system based on audiovisual analysis and synthesis.

Influence of personality traits on backchannel selection

Our aim is to build a real-time Embodied Conversational Agent able to act as an interlocutor in interaction, generating automatically verbal and non verbal signals. These signals, called backchannels, provide information about the listeners mental state towards the perceived speech. The ECA reacts differently to users behavior depending on its predefined personality. Personality influences the generation and the selection of backchannels. In this paper, we propose a listeners action selection algorithm working in real-time to choose the type and the frequency of backchannels to be displayed by the ECA in accordance with its personality. The algorithm is based on the extroversion and neuroticism dimensions of personality. We present an evaluation on how backchanels managed by this algorithm are congruent with intuitive expectations of participants in terms of behavior specific to different personalities.

Greta : Une Plateforme d'Agent Conversationnel Expressif et Interactif

This paper presents a generic ,modular and interactive architecture for embodied conversational agent called Greta. It is 3D agent able to communicate with users using verbal and non verbal channels like gaze, head and torso movements, facial expressions and gestures. Our architecture follows the SAIBA framework that defines modular structure, functionalities and communication protocols for ECA systems. In this paper, we present our architecture, performance tests as well as several interactive applications.

Come and have an emotional workout with sensitive artificial listeners

This demonstration aims to showcase the recently completed SEMAINE system. The SEMAINE system is a publicly available, fully autonomous Sensitive Artificial Listeners (SAL) system that consists of virtual dialog partners based on audiovisual analysis and synthesis (see http://semaine.opendfki.de/wiki). The system runs in real-time, and combines incremental analysis of user behavior, dialog management, and synthesis of speaker and listener behavior of a SAL character, displayed as a virtual agent. The SAL characters intend to engage the user in a conversation by paying attention to the users emotions and nonverbal expressions. The characters have their own emotionally defined personality. During an interaction, the characters attempt to create an emotional workout for the user by drawing her/him towards their dominant emotion, through a combination of verbal and nonverbal expressions.

A cognitive module in a decision-making architecture for agents in urban simulations

This paper addresses the issue of hybridization between reactive and cognitive approaches within a single decision-making architecture for virtual agent in an urban simulation. We use a reactive module in order to manage reactive behaviors and agent autonomy, and a cognitive module for anticipation, learning and complex behaviors management. The purpose of the cognitive module is to increase the agents behavior credibility. The agents reactive and proactive behaviors are sent to a decision module which is able to integrate, decompose, combine and select an action.

Interacting with Emotional Virtual Agents

Sensitive Artificial Listener (SAL) is a multimodal dialogue system which allows users to interact with virtual agents. Four characters with different emotional traits engage users is emotionally coloured interactions. They not only encourage the users into talking but also try to drag them towards specific emotional states. Despite the agents very limited verbal understanding, they are able to react appropriately to the user’s non-verbal behaviour. The demonstrator shows an final version of the fully autonomous SAL system.

Real-Time Backchannel Selection for ECAs According to User's Level of Interest

A great challenge that is to be faced in the design of virtual agents is the issue of credibility, not only in the agents aspect but also in its behavior [1]. To be believable, the agent has to decide what to do next according to the internal and external variables of the agent. Besides others, we have to deal with the problem of action selection which can be resumed to choose the most appropriate action among all possible (conflicting) ones [2]. In our case, actions are backchannels. This work is part of the STREP EU SEMAINE project in which a real-time Embodied Conversational Agent (ECA) will be a Sensitive Artificial Listener (SAL) [3]. This project aims to build an autonomous talking agent able to exhibit autonomously appropriate verbal and non verbal behaviors in real-time when it plays the role of the listener in a conversation with a user.

Simulating affective behaviours: an approach based on the COR theory

The expression of emotion is usually considered an important step towards the believability of a virtual agent. However, current models based on emotion categories face important challenges in their attempts to model the influence of emotions on agents behaviour. To adress this problem, we propose an architecture based on the COnservation of Resources theory (COR) which aims at producing affective behaviours in various scenarios. In this paper we explain the principle of such a model, how it is implemented and can be evaluated.