Carlos Martinho

Affect recognition for interactive companions: challenges and design in real world scenarios

Affect sensitivity is an important requirement for artificial companions to be capable of engaging in social interaction with human users. This paper provides a general overview of some of the issues arising from the design of an affect recognition framework for artificial companions. Limitations and challenges are discussed with respect to other capabilities of companions and a real world scenario where an iCat robot plays chess with children is presented. In this scenario, affective states that a robot companion should be able to recognise are identified and the non-verbal behaviours that are affected by the occurrence of these states in the children are investigated. The experimental results aim to provide the foundation for the design of an affect recognition system for a game companion: in this interaction scenario children tend to look at the iCat and smile more when they experience a positive feeling and they are engaged with the iCat.

The influence of empathy in human-robot relations

The idea of robotic companions capable of establishing meaningful relationships with humans remains far from being accomplished. To achieve this, robots must interact with people in natural ways, employing social mechanisms that people use while interacting with each other. One such mechanism is empathy, often seen as the basis of social cooperation and prosocial behaviour. We argue that artificial companions capable of behaving in an empathic manner, which involves the capacity to recognise anothers affect and respond appropriately, are more successful at establishing and maintaining a positive relationship with users. This paper presents a study where an autonomous robot with empathic capabilities acts as a social companion to two players in a chess game. The robot reacts to the moves played on the chessboard by displaying several facial expressions and verbal utterances, showing empathic behaviours towards one player and behaving neutrally towards the other. Quantitative and qualitative results of 31 participants indicate that users towards whom the robot behaved empathically perceived the robot as friendlier, which supports our hypothesis that empathy plays a key role in human-robot interaction.

It's all in the game: Towards an affect sensitive and context aware game companion

Robot companions must be able to display social, affective behaviour. As a prerequisite for companionship, the ability to sustain long-term interactions with users requires companions to be endowed with affect recognition abilities. This paper explores application-dependent user states in a naturalistic scenario where an iCat robot plays chess with children. In this scenario, the role of context is investigated for the modelling of user states related both to the task and the social interaction with the robot. Results show that contextual features related to the game and the iCats behaviour are successful in helping to discriminate among the identified states. In particular, state and evolution of the game and display of facial expressions by the iCat proved to be the most significant: when the user is winning and improving in the game her feeling is more likely to be positive and when the iCat displays a facial expression during the game the users level of engagement with the iCat is higher. These findings will provide the foundation for a rigorous design of an affect recognition system for a game companion.

Why can't we be friends? an empathic game companion for long-term interaction

The ability of artificial companions (virtual agents or robots) to establish meaningful relationships with users is still limited. In humans, a key aspect of such ability is empathy, often seen as the basis of social cooperation and pro-social behaviour. In this paper, we present a study where a social robot with empathic capabilities interacts with two users playing a chess game against each other. During the game, the agent behaves in an empathic manner towards one of the players and in a neutral way towards the other. In an experiment conducted with 40 participants, results showed that users to whom the robot was empathic provided higher ratings in terms of companionship.

Closing the loop: from affect recognition to empathic interaction

Empathy is a very important capability in human social relationships. If we aim to build artificial companions (agents or robots) capable of establishing long-term relationships with users, they should be able to understand the users affective state and react accordingly, that is, behave in an empathic manner. Recent advances in affect recognition research show that it is possible to automatically analyse and interpret affective expressions displayed by humans. However, affect recognition in naturalistic environments is still a challenging issue and there are many unanswered questions related to how a virtual agent or a social robot should react to those states, and how that improves the interaction. We have developed a scenario in which a social robot recognises the users affective state and displays empathic behaviours. In this paper, we present part of the results of a study assessing the influence of the robots empathic behaviour on the users understanding of the interaction.

Inter-ACT: an affective and contextually rich multimodal video corpus for studying interaction with robots

The Inter-ACT (INTEracting with Robots - Affect Context Task) corpus is an affective and contextually rich multimodal video corpus containing affective expressions of children playing chess with an iCat robot. It contains videos that capture the interaction from different perspectives and includes synchronised contextual information about the game and the behaviour displayed by the robot. The Inter-ACT corpus is mainly intended to be a comprehensive repository of naturalistic and contextualised, task-dependent data for the training and evaluation of an affect recognition system in an educational game scenario. The richness of contextual data that captures the whole human-robot interaction cycle, together with the fact that the corpus was collected in the same interaction scenario of the target application, make the Inter-ACT corpus unique in its genre.

Designing a game companion for long-term social interaction

The ability to understand the users affective states and expressions is an important requirement for a robot companion. In this paper we focus on an interaction scenario where an iCat robot plays chess with children. We discuss the challenges to address in the design of an affect recognition system for a game companion and we present the initial steps carried out to build a framework for the dynamic modeling of the users affect. We conclude by providing examples of how we intend to exploit information about the users affect to generate an empathic behavior in our game companion.

Wasp-like agents for scheduling production in real-time strategy games

In this paper, we propose an algorithm inspired in the social intelligence of wasps for scheduling production in real-time strategy games, and evaluate its performance in a scenario developed as a modification of the game Warcraft III The Frozen Throne. Results show that such an approach is well suited for the highly dynamic nature of the environment in this game genre. We also believe such an approach may allow the exploration of new paradigms of gameplay, and provide some examples in the explored scenarios.

It's All in the Anticipation

Since the beginnings of character animation, anticipationhas been an effective part of the repertoire of tricks used to create believable animated characters. However, anticipation has had but a secondary role in the creation of synthetic virtual life forms. In this paper, we describe how a simple anticipatory mechanism that generates an affective signal resulting from the mismatch between sensed and predicted values -- the emotivector-- can help in the creation of consistent believable behaviour for intelligent virtual characters.