Florian Pecune

Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface.

Affective brain-computer interfaces (BCI) harness Neuroscience knowledge to develop affective interaction from first principles. In this article, we explore affective engagement with a virtual agent through Neurofeedback (NF). We report an experiment where subjects engage with a virtual agent by expressing positive attitudes towards her under a NF paradigm. We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance. The magnitude of left-asymmetric DL-PFC activity, measured using functional near infrared spectroscopy (fNIRS) and treated as a proxy for approach, is mapped onto a control mechanism for the virtual agent’s facial expressions, in which action units (AUs) are activated through a neural network. We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent’s responses as realistic and consistent with their projected mental disposition. This interaction paradigm is particularly relevant in the case of affective BCI as it facilitates the volitional activation of specific areas normally not under conscious control. Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

Implementing and Evaluating a Laughing Virtual Character

Laughter is a social signal capable of facilitating interaction in groups of people: it communicates interest, helps to improve creativity, and facilitates sociability. This article focuses on: endowing virtual characters with computational models of laughter synthesis, based on an expressivity-copying paradigm; evaluating how the physically co-presence of the laughing character impacts on the user’s perception of an audio stimulus and mood. We adopt music as a means to stimulate laughter. Results show that the character presence influences the user’s perception of music and mood. Expressivity-copying has an influence on the user’s perception of music, but does not have any significant impact on mood.

Emotion and Attitude Modeling for Non-player Characters

Within this chapter, we are presenting how game developers could take inspiration from the research in Embodied Conversational Agent to develop non-player characters capable of expressing believable emotional and social reactions. Inspired by the social theories about human emotional and social reactions, the researchers working with Embodied Conversational Agents developed different computational models to reproduce these human mechanisms within virtual characters. We are listing some of these works, comparing the different approaches and theories considered.

Evaluating Social Attitudes of a Virtual Tutor

In this paper we evaluate a model of social decision-making for virtual agents. The model computes the social attitude of a virtual agent given its social role during the interaction and its social relation toward the interactant. The resulting attitude influences the agent’s social goals and therefore determines the decisions made by the agent in terms of actions and communicative intentions to accomplish. We conducted an empirical study in the context of virtual tutor-child interaction where participants evaluated the tutor’s perceived social attitude towards the child while the tutor’s social role and relation were manipulated by our model. Results showed that both role and social relation have an influence on the agent’s perceived social attitude.

A Cognitive Model of Social Relations for Artificial Companions

Artificial companions are made to establish and maintain long-term relationships with users. In order to model the companion’s social relations and to capture its dynamics, we propose a neural network model based on a formal representation of social relations. Based on psychological theories, we characterize social relations over two dimensions, namely liking and dominance. These two dimensions are formally described as a combination of beliefs and goals of the agent’s mental state. Such a model allows us to automatically compute the social relation of a virtual agent towards its interlocutor depending on its beliefs and goals.

Toward a Computational Model of Social Relations for Artificial Companions

The objective of my Ph.D is to develop a computational model of social relations for artificial companions. Such a model should consider the representation of the social relation, its initialization and its dynamics over time. Moreover, the influence of this relation on the agents decision-making should be modeled. In the proposed computational model described in this paper, the representation, initialization, and dynamics of the social relation are based on the research in Human and Social Sciences, considering the influence of individual and social parameters such as the personality of the agent or its social role.