Charles Rich

Explorations in engagement for humans and robots

This paper explores the concept of engagement, the process by which individuals in an interaction start, maintain and end their perceived connection to one another. The paper reports on one aspect of engagement among human interactors--the effect of tracking faces during an interaction. It also describes the architecture of a robot that can participate in conversational, collaborative interactions with engagement gestures. Finally, the paper reports on findings of experiments with human participants who interacted with a robot when it either performed or did not perform engagement gestures. Results of the human-robot studies indicate that people become engaged with robots: they direct their attention to the robot more often in interactions where engagement gestures are present, and they find interactions more appropriate when engagement gestures are present than when they are not.

Collagen: applying collaborative discourse theory to human-computer interaction

We describe an approach to intelligent user interfaces, based on the idea of making the computer a collaborator, and an application-independent technology for implementing such interfaces.

COLLAGEN: A Collaboration Manager for Software Interface Agents

We have implemented an application-independent collaboration manager, called Collagen, based on the SharedPlan theory of discourse, and used it to build a software interface agent for a simple air travel application. The software agent provides intelligent, mixed initiative assistance without requiring natural language understanding. A key benefit of the collaboration manager is the automatic construction of an interaction history which is hierarchically structured according to the users and agents goals and intentions.

The Programmer's Apprentice: a research overview

The long-term goal of the Programmers Apprentice project is to develop a theory of how expert programmers analyze, synthesize, modify, explain, specify, verify, and document programs. The authors present their vision of the Programmers Apprentice, the principles and techniques underlying it, and their progress toward it. The primary vehicle for this exposition is three scenarios illustrating the use of the Apprentice in three phases of the programming task: implementation, design, and requirements. The first scenario is taken from a completed working prototype. The second and third scenarios are the targets for prototype systems currently under construction.<<ETX>>

COLLAGEN: when agents collaborate with people

We take the position that autonomous agents, when they interact with people, should be governed by the same principles that underlie human collaboration. These principles come from research in computational linguistics, specifically collaborative discourse theory, which describes how people communicate and coordinate their activities in the context of shared tasks. We have implemented a prototype toolkit, called Collagen, which embodies collaborative discourse principles, and used it to build a collaborative interface agent for a simple air travel application. The potential benefits of this approach include application-independence, naturalness of use, and ease of learning, without requiring natural language understanding by the agent. Superseded by TR97-21. First International Conference on Autonomous Agents, Marina del Rey, CA, February, 1997, pp. 284-291 and reprinted in M. Huhns and M. Singh, editors, Readings in Agents, Morgan Kaufmann, San Francisco, CA, 1997, pp. 117–124 This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c ©Mitsubishi Electric Research Laboratories, Inc., 1996 201 Broadway, Cambridge, Massachusetts 02139

Using plan recognition in human-computer collaboration

Human-computer collaboration provides a practical and useful application for plan recognition techniques. We describe a plan recognition algorithm which is tractable by virtue of exploiting properties of the collaborative setting, namely: the focus of attention, the use of partially elaborated hierarchical plans, and the possibility of asking for clarification. We demonstrate how the addition of our plan recognition algorithm to an implemented collaborative system reduces the amount of communication required from the user.

Non-Verbal Cues for Discourse Structure

This paper addresses the issue of designing embodied conversational agents that exhibit appropriate posture shifts during dialogues with human users. Previous research has noted the importance of hand gestures, eye gaze and head nods in conversations between embodied agents and humans. We present an analysis of human monologues and dialogues that suggests that postural shifts can be predicted as a function of discourse state in monologues, and discourse and conversation state in dialogues. On the basis of these findings, we have implemented an embodied conversational agent that uses Collagen in such a way as to generate postural shifts.

Recognizing engagement in human-robot interaction

Based on a study of the engagement process between humans, we have developed and implemented an initial computational model for recognizing engagement between a human and a humanoid robot. Our model contains recognizers for four types of connection events involving gesture and speech: directed gaze, mutual facial gaze, conversational adjacency pairs and backchannels. To facilitate integrating and experimenting with our model in a broad range of robot architectures, we have packaged it as a node in the open-source Robot Operating System (ROS) framework. We have conducted a preliminary validation of our computational model and implementation in a simple human-robot pointing game.

Automatic programming: myths and prospects

The authors consider five common myths about automatic programming and expose the fallacies on which they rest. They attempt to provide an accurate picture of these systems in terms of what the user sees, how the system works, and what the system knows. They describe commercially available systems and discuss what is on the horizon.<<ETX>>

Initial Report on a Lisp Programmer's Apprentice

This paper reports on the initial design and partial implementation of an interactive programming environment to be used by expert programmers. The system is based on three forms of program description: 1) definition of structured data objects, their parts, properties, and relations between them, 2) input–output specification of the behavior of program segments, and 3) a hierarchical representation of the internal structure of programs (plans). The plan representation is of major theoretical interest because it includes not only data flow and control flow relationships between subsegments of a program, but also goal-subgoal, prerequisite, and other logical dependencies between the specifications of the subsegments. Plans are utilized both for describing particular programs and in the compilation of a knowledge base of more abstract knowledge about programming, such as the concept of a loop and various specializations, such as enumeration loops and search loops. We also describe a deductive system which can verify the correctness of plans involving side effects on complex data with structure sharing.