Iván Gris

Building rapport between human and ECA: A pilot study

This study is part of a longer-term project to provide embodied conversational agents (ECAs) with behaviors that enable them to build and maintain rapport with their human partners. We focus on paralinguistic behaviors, and especially nonverbal behaviors, and their role in communicating rapport. Using an ECA that guides its players through a speech-controlled game, we attempt to measure the familiarity built between humans and ECAs across several interactions based on paralinguistic behaviors. In particular, we studied the effect of differences in the amplitude of nonverbal behaviors by an ECA interacting with a human across two conversational sessions. Our results suggest that increasing amplitude of nonverbal paralinguistic behaviors may lead to an increased perception of physical connectedness between humans and ECAs.

Recorded speech, virtual environments, and the effectiveness of embodied conversational agents

Development of embodied conversational agents (ECAs) has tended to focus on the character’s dialog capabilities, with less research on the design and effect of the agent’s voice and of the virtual environments in which the agent exists. For a study of human-ECA rapport, we iteratively developed three versions of a game featuring an ECA, where each version of the game had a different combination of speech generation and virtual environment. Evaluations of the users’ interactions with the different versions of the game enabled us to assess the effects of changes in the agent’s voice and of changes in the agent’s virtual world.

Animation guidelines for believable embodied conversational agent gestures

In animating embodied conversational agents (ECAs), run-time blending of animations can provide a large library of movements that increases the appearance of naturalness while decreasing the number of animations to be developed. This approach avoids the need to develop a costly full library of possible animations in advance of use. Our principal scientific contribution is the development of a model for gesture constraints that enables blended animations to represent naturalistic movement. Rather than creating over-detailed, fine-grained procedural animations or hundreds of motion-captured animation files, animators can include sets of their own animations for agents, blend them, and easily reuse animations, while constraining the ECA to use motions that would occur and transition naturally.

A Mark-Up Language and Interpreter for Interactive Scenes for Embodied Conversational Agents

Our research seeks to provide embodied conversational agents (ECAs) with behaviors that enable them to build and maintain rapport with human users. To conduct this research, we need to build agents and systems that can maintain high levels of engagement with humans over multiple interaction sessions. These sessions can potentially extend to longer periods of time to examine long-term effects of the virtual agent’s behaviors. Our current ECA interacts with humans in a game called “Survival on Jungle Island.” Throughout this game, users interact with our agent across several scenes. Each scene is composed of a collection of speech input, speech output, gesture input, gesture output, scenery, triggers, and decision points. Our prior system was developed with procedural code, which did not lend itself to rapid extension to new game scenes. So to enable effective authoring of the scenes for the “Jungle” game, we adopted a declarative approach. We developed ECA middleware that parses, interprets, and executes XML files that define the scenes. This paper presents the XML coding scheme and its implementation and describes the functional back-end enabled by the scene scripts.

Empirical Analysis of Playability vs. Usability in a Computer Game

This paper reports our experience in applying an empirical user-experience testing method to improve a computer game. We sought to understand the differences in practice between usability and playability, and correspondingly to assess the usefulness of different approaches to taxonomies for playability. Our experience suggests that the evaluation technique for playability can be the same as for usability and that some existing taxonomies for playability do not provide effective support for translating experience of evaluation into heuristics for design. For formative evaluation, understanding episodes for playability depends on specific circumstances of each episode; taxonomies may be more useful for summative evaluation.

Bigger (gesture) isn’t always better

The literature suggests that familiarity and rapport are enhanced by larger, more extraverted gestures. However, the sizes of the increases in amplitude have not been reported. We sought to determine whether this relationship holds true for interaction between humans and embodied conversational agents. To this end, we conducted an experiment in which we increased gesture amplitude, with quantification of the gesture sizes. We hypothesized that rapport would be increased in the larger-gesture condition. However, our results were exactly the opposite: Rapport fell significantly in the larger-gesture condition. This means that larger may not always be better for building human-agent rapport. Our unexpected results may be because our agent’s gestures were simply too big, odd, awkward, or strange, or because of a statistical anomaly.

Young Merlin: an embodied conversational agent in virtual reality

This paper describes a system for embodied conversational agents developed by Inmerssion and one of the applications—Young Merlin: Trial by Fire —built with this system. In the Merlin application, the ECA and a human interact with speech in virtual reality. The goal of this application is to provide engaging VR experiences that build rapport through storytelling and verbal interactions. The agent is fully automated, and his attitude towards the user changes over time depending on the interaction. The conversational system was built through a declarative approach that supports animations, markup language, and gesture recognition. Future versions of Merlin will implement multi-character dialogs, additional actions, and extended interaction time.

An Exploratory Analysis of ECA Characteristics

To help guide design and development of embodied conversational agents, this study reviews the evolving qualities of naturalistic agents by identifying and tracking their most relevant features. The study extends prior taxonomies of characteristics of ECAs, developing a rubric that distinguishes agents’ visual and functional characteristics. The study applies the rubric to 15 agents representing different genres of games, distinguishing agents in terms of their naturalistic qualities. The study explores changes in qualities of agents as a function of time.