Divesh Lala

VISIE: A spatially immersive interaction environment using real-time human measurement

In this paper we present VISIE, a software used to create immersive environments which utilize social and cultural interaction. We use the concept of a spatially immersive display to project information about the virtual world to the user in all directions. The user is able to interact in this world using spatial cognition as they do in the real world. So far, a pressure sensor has been integrated into the system to allow the user to navigate through the environment by walking in place. Additionally, multiple displays can be connected to allow users to communicate with each other in the virtual world. We also outline how we can evaluate and use this system by designing scenarios and experiments. Further work to improve the prototype includes using infrared sensors to track human gesture and behavior that can in turn influence the environment.

Towards a virtual environment for capturing behavior in cultural crowds

Cultural behavior is an area of research that can allow us to further cross-cultural understanding, and is now starting to integrate itself within the field of information technology. One domain that expresses these behaviors is inside a crowd, however the analysis of micro-level crowd behavior is impractical in a real-world setting as passive observation has limitations on understanding true behavior. By using a virtual environment to simulate a crowd situation, measuring an individuals in-crowd behavior becomes feasible. This paper introduces the development of a virtual environment which enables the creation of different types of cultural crowds with which the user may interact. The parameterization of the crowds is based on the famous cultural dimensions put forward by Hofstede. One of the cultural dimensions, individualism/collectivism, was mapped to agent characteristics during a series of simulations and it was found that two distinct types of crowd could be generated. For the dimensions have not yet been examined, the proposed environment provides an ideal opportunity to address this gap in the research as well as becoming a tool with which other types of experimentation can be performed.

Talking with ERICA, an autonomous android

We demonstrate dialogues with an autonomous android ERICA, who has an appearance like a human being. Currently, ERICA plays two social roles: a laboratory guide and a counselor. It is designed to follow the protocols of human dialogue to make the user comfortable: (1) having a chat before the main talk, (2) proactively asking questions, and (3) conveying proper feedbacks. The combination of the human-like appearance and the appropriate behaviors according to her social roles allows for symbiotic human-robot interaction.

ICIE: Immersive Environment for Social Interaction Based on Socio-spacial Information

Spacial information plays an important role in social interaction with people. The ICIE is a platform which can present socio-spacial information, obtain human behavior with non-contact sensors and have components to interpret the socio-spacial information. In this paper, we explain the framework of ICIE and main architectures to capture human behavior and to provide virtual space in ICIE. We discuss socio-spacial interaction by using ICIE and introduce some applications and studies using ICIE.

Synthetic Evidential Study as Primordial Soup of Conversation

Synthetic evidential study (SES for short) is a novel technology-enhanced methodology for combining theatrical role play and group discussion to help people spin stories by bringing together partial thoughts and evidences. SES not only serves as a methodology for authoring stories and games but also exploits the framework of game framework to help people sustain in-depth learning. In this paper, we present the conceptual framework of SES, a computational platform that supports the SES workshops, and advanced technologies for increasing the utility of SES. The SES is currently under development. We discuss conceptual issues and technical details to delineate how much we can implement the idea with our technology and how much challenges are left for the future work.

Synthetic Evidential Study as Augmented Collective Thought Process – Preliminary Report

Synthetic evidential study (SES) is a novel approach to understanding and augmenting collective thought process through substantiation by interactive media. It consists of a role-play game by participants, projecting the resulting play into a shared virtual space, critical discussions with mediated role-play, and componentization for reuse. We present the conceptual framework of SES, initial findings from a SES workshop, supporting technologies for SES, potential applications of SES, and future challenges.

Joint activity theory as a framework for natural body expression in autonomous agents

Creating agents which utilize natural verbal and non-verbal communication is an appropriate goal for many researchers involved in human-computer interaction. Using these types of agents enhances their capabilities as a communication tool for teaching humans inside a virtual environment. This paper describes how Herbert Clarks theory of joint activities can be applied to agents, extending it to a domain in which communication is weighted towards the non-verbal, specifically body expressions. The major framework for this implementation is the joint action ladder, in which a communicative act is checked at several levels before it can be understood and responded to by the receiver of the act. A proposed environment has been created in the form of a basketball game, where a user can interact with his team through natural body movements. Through this type of interaction, the user has the potential to learn gestural expressions through observation as well as less explicit concepts such as the internal state of agents. Several avenues of research that can be followed through the use of the basketball game are also described.

Annotation and analysis of listener's engagement based on multi-modal behaviors

We address the annotation of engagement in the context of human-machine interaction. Engagement represents the level of how much a user is being interested in and willing to continue the current interaction. The conversational data used in the annotation work is a human-robot interaction corpus where a human subject talks with the android ERICA, which is remotely operated by another human subject. The annotation work was done by multiple third-party annotators, and the task was to detect the time point when the level of engagement becomes high. The annotation results indicate that there are agreements among the annotators although the numbers of annotated points are different among them. It is also found that the level of engagement is related to turn-taking behaviors. Furthermore, we conducted interviews with the annotators to reveal behaviors used to show a high level of engagement. The results suggest that laughing, backchannels and nodding are related to the level of engagement.

Enhancing Communication through Distributed Mixed Reality

A navigable mixed reality system where humans and agents can communicate and interact with each other in a virtual environment can be an appropriate tool for analyzing multi-human and multi-agent communication. We propose a prototype of our system, FCWorld, which has been developed to meet these requirements. FCWorld integrates various technologies with a focus on allowing natural human communication. In this paper we discuss the requirements for FCWorld, the technical issues which it must address, and our proposed solutions. We intend it to become a novel tool for a variety of communication tasks such as real-time analysis and facilitation.

A Conversational Dialogue Manager for the Humanoid Robot ERICA

We present a dialogue system for a conversational robot, Erica. Our goal is for Erica to engage in more human-like conversation, rather than being a simple question-answering robot. Our dialogue manager integrates question-answering with a statement response component which generates dialogue by asking about focused words detected in the user’s utterance, and a proactive initiator which generates dialogue based on events detected by Erica. We evaluate the statement response component and find that it produces coherent responses to a majority of user utterances taken from a human-machine dialogue corpus. An initial study with real users also shows that it reduces the number of fallback utterances by half. Our system is beneficial for producing mixed-initiative conversation.