Marco Vala

Fearnot!: an experiment in emergent narrative

We discuss the experience of constructing the application FearNot! (Fun with Empathic Agents Reaching Novel Outcomes in Teaching), an application of virtual drama to anti-bullying education inspired by Forum Theatre. An appraisal-driven agent architecture is presented as a mechanism for generating an emergent, that is, unscripted, narrative. A small-scale evaluation is discussed and the lessons learned are described.

Towards tangibility in gameplay: building a tangible affective interface for a computer game

In this paper we describe a way of controlling the emotional states of a synthetic character in a game (FantasyA) through a tangible interface named SenToy. SenToy is a doll with sensors in the arms, legs and body, allowing the user to influence the emotions of her character in the game. The user performs gestures and movements with SenToy, which are picked up by the sensors and interpreted according to a scheme found through an initial Wizard of Oz study. Different gestures are used to express each of the following emotions: anger, fear, happiness, surprise, sadness and gloating. Depending upon the expressed emotion, the synthetic character in FantasyA will, in turn, perform different actions. The evaluation of SenToy acting as the interface to the computer game FantasyA has shown that users were able to express most of the desired emotions to influence the synthetic characters, and that overall, players, especially children, really liked the doll as an interface.

ION Framework --- A Simulation Environment for Worlds with Virtual Agents

Agents cannot be decoupled from their environment. An agent perceives and acts in a world and the model of the world influences how the agent makes decisions. Most systems with virtual embodied agents simulate the environment within a specific realization engine such as the graphics engine. As a consequence, these agents are bound to a particular kind of environment which compromises their reusability across different applications. We propose the ION Framework, a framework for simulating virtual environments which separates the simulation environment from the realization engine. In doing so, it facilitates the integration and reuse of the several components of the system. The ION Framework was used to create several 3D virtual worlds populated with autonomous embodied agents that were tested with hundreds of users.

FearNot’s appearance: reflecting children’s expectations and perspectives

This paper discusses FearNot, a virtual learning environment populated by synthetic characters aimed at the 8-12 year old age group for the exploration of bullying and coping strategies. Currently, FearNot is being redesigned from a lab-based prototype into a classroom tool. In this paper we focus on informing the design of the characters and of the virtual learning environment through our interpretation of qualitative data gathered about interaction with FearNot by 345 children. The paper focuses on qualitative data collected using the Classroom Discussion Forum technique and discusses its implications for the redesign of the media used for FearNot. The interpretation of the data identifies that the use of fairly naive synthetic characters for achieving empathic engagement appears to be an appropriate approach. Results do indicate a focus for redesign, with a clear need for improved transitions for animations; identification and repair of inconsistent graphical elements; and for a greater cast of characters and range of sets to achieve optimal engagement levels.

Robots meet IVAs: a mind-body interface for migrating artificial intelligent agents

We describe CMION, an open source architecture for coordinating the various sensors and effectors of an artificial intelligent agent with its mind, i.e. the high level decision making processes. The architecture was designed to work for virtual graphical agents, including those on mobile devices, as well as robots. Its built-in migration feature allows a character to move between these differing embodiments, inhabiting them in turn. We emphasize the importance of modularity for an architecture supporting migration and highlight design decisions promoting modularity in CMION. An applied example of the architectures use in a migration situation is given.

FantasyA and SenToy

FantasyA is a role-playing game where emotions are part of the game logic. SenToy is a tangible interface device [2], used to influence emotional behaviour in FantasyA. Players in the game FantasyA have to master SenToy and exhibit a particular set of emotions and perform a set of actions in order to evolve in the game [3]. A study was undertaken to gauge the success of the overall gaming experience, as well as the individual components, the FantasyA game with its emotional content and the SenToy control device with its gestural input.

What can i do with this?: finding possible interactions between characters and objects

Virtual environments are often populated by autonomous synthetic agents capable of acting and interacting with other agents as well as with humans. These virtual worlds also include objects that may have different uses and types of interactions. As such, these agents need to identify possible interactions with the objects in the environment and measure the consequences of these interactions. This is particularly difficult when the agents never interacted with some of the objects beforehand. This paper describes SOTAI - Smart ObjecT-Agent Interaction, a framework that will help agents to identify possible interactions with unknown objects based on their past experiences. In SOTAI, agents can learn world regularities, like object attributes and frequent relations between attributes. They gather qualitative symbolic descriptions from their sensorial data when interacting with objects and perform inductive reasoning to acquire concepts about them. We implemented an initial case study and the results show that our agents are able to acquire valid conceptual knowledge.

Thalamus: closing the mind-body loop in interactive embodied characters

We present the Thalamus framework, which is based on SAIBA and extends it by adding a perceptual loop. This perceptual loop enables embodied characters to perform continuous interaction. The framework was tested in a case study involving a NAO and an EMYS robots. After showing that our extension works, we point out some issues that were encountered during the development of the case study. We also suggest that the definition of a formal Perception Modelling Language (PML) based on the SAIBA framework can enable SAIBA-compliant embodied characters to perform continuous interaction, while still performing synchronized multimodal behavior based on BML.

Mind the body: filling the gap between minds and bodies in synthetic characters

Interactive virtual environments (IVEs) are inhabited by synthetic characters that guide and engage children in a wide variety of activities, like playing games or learning new things. To build those environments, we need believable autonomous synthetic characters that are able to think and act in very dynamic environments. These characters have often able minds that are limited by the actions that the body can do. In one hand, we have minds capable of creating interesting non-linear behaviour; on the other hand, we have bodies that are limited by the number of animations they can perform. This usually leads to a large planning effort to anticipate possible situations and define which animations are necessary. When we aim at non-linear narrative and non-deterministic plots, there is an obvious gap between what minds can think and what bodies can do. We propose smart bodies as way to fill this gap between minds and bodies. A smart body extends the notion of standard body since it is enriched with semantic information and can do things on its own. The mind still decides what the character should do, but the body chooses how it is done. Smart bodies, like standard bodies, have a model and a collection of animations which are provided by a graphics engine. But they also have access to knowledge about other elements in the world like locations, interaction information and particular attributes. At this point, the notions of interaction spot and action trigger come into play. Interaction spots are specific positions around smart bodies or items where other smart bodies can do particular interactions. Action triggers define automatic reactions which are triggered by smart bodies when certain actions or interactions occur. We use both these constructs to create abstract references for physical elements, to act as a resource and pre-condition mechanisms, and to simulate physics using rule-based reactions. Smart bodies use all this information to create high-level actions which are used by the minds. Thus, minds operate at a higher level and do not have to deal with low-level body geometry or physics. Smart bodies were used in FearNot!, an anti-bullying application. In FearNot! children experience virtual stories generated in real-time where they can witness (from a third-person perspective) a series of bullying situations towards a character. Clearly, in such an emergent narrative scenario, minds need to work at a higher-level of abstraction without worrying with bodies and how a particular action is carried out at low-level. Smart bodies provided this abstraction layer. We performed a small study to validate our work in FearNot! with positive results. We believe there may be other applications where smart bodies have much to offer, particularly when using unscripted and non-linear narrative approaches.

FantasyA – The Duel of Emotions

FantasyA is a computer game where two characters face each other in a duel and emotions are used as the driving elements in the action decision of the characters. In playing the game, the user influences the emotional state of his or her semi-autonomous avatar using a tangible interface for affective input, the SenToy. In this paper we show how we approached the problem of modelling the emotional states of the synthetic characters, and how to combine them with the perception of the emotions of the opponents in the game. This is done by simulating the opponents action tendencies in order to predict their possible actions. For the user to play, he or she must understand the emotional state of his opponent which is achieved through animations (featuring affective body expressions) of the character. FantasyA was evaluated with 30 subjects from different ages and the preliminary results showed that the users liked the game and were able to influence the emotional states of their characters, in particular the young users.