Joseph Bates

An architecture for action, emotion, and social behavior

The Oz project at Carnegie Mellon is studying the construction of artistically effective simulated worlds. Such worlds typically include several agents, which must exhibit broad behavior. To meet this need, we are developing an agent architecture, called Tok, that presently supports reactivity, goals, emotions, and social behavior. Here we briefly introduce the requirements of our application, summarize the Tok architecture, and describe a particular social agent we have constructed.

Virtual reality, art, and entertainment

Most existing research on virtual reality concerns issues close to the interface, primarily how to present an underlying simulated world in a convincing fashion. However, for virtual reality to achieve its promise as a rich and popular artistic form, as have the novel, cinema, and television, we believe it will be necessary to explore well beyond the interface, to those issues of content and style that have made traditional media so powerful. We present a case for the importance of this research, then outline several topics we believe are central to the inquiry: developing computational theories for cognitive-emotional agents, presentation style, and drama.

Dramatic presence

Let us consider the presentation by computers of rich, highly interactive worlds that are inhabited by dynamic and complex characters, and shaped by aesthetically pleasing stories. We shall call this interactive drama, and we believe that it requires strong characters, aesthetic presentation, and long-term dramatic structure. This paper describes an experiment designed to help us understand how to create interactive drama. Three principal questions are addressed. One, how does it feel to be immersed in a dramatic virtual world filled with characters and story? Two, what is required of the characters (actors) in such a virtual world? Three, what is required of the story and its director? We present an introduction to interactive drama, summarize the Oz system designed to create and present such experiences, and describe our experiment in detail. Finally, drawing from the experiment, we suggest several hypotheses about interactive drama.

Personality-rich believable agents that use language

We are studying how to create believable agents that perform actions and use natural language in interactive, animated, real-time worlds. Believable agents are autonomous agents that have specific, rich personalities like characters in movies and animation. We have extended Hap, the behavior-based architecture used by the Oz group to construct non-linguistic believable agents, to support natural language text generation. These extensions allow us to tightly integrate text generation with other aspects of the agent, including action, perception, inference and emotion. We describe our approach, and show how it leads to agents with properties we believe important for believability, such as: using language and action together to accomplish communication goals; using perception to help make linguistic choices; varying generated text according to emotional state; varying generated text to express the specific personality; and issuing the text in real-time with pauses, restarts and other breakdownsvisible. Besides being useful in constructing believable agents, we feel these extensions may interest researchers seeking to generate language in other action architectures.

Broad agents

The Oz project at Carnegie Mellon is developing technology for dramatic virtual worlds. One requirement of such worlds is the presence of broad, though perhaps shallow, agents. To support our needs, we are developing an agent architecture that provides goals and goal directed reactive behavior, emotional state and its effects on behavior, some natural language abilities (especially pragmatics based language generation), and some memory and inference abilities. We are limiting each of these capacities whenever necessary to allow us to build a broadly capable, integrated agent.In attempting to construct a broad agent, constraints seem to arise between components of the architecture. In this brief note, we discuss some of these constraints.

System for authoring highly interactive, personality-rich interactive characters

We describe an innovative system for authoring expressive, fully autonomous interactive characters. The focus of our work is creating a system to allow rich authoring that captures as much of the artistic intent of the author in procedural form as we can, and that provides automatic support for expressive execution of that content. The system is composed of two parts: (1)a programming language with unusual language features including concurrency, reflection, backtracking, continuously monitored expressions, and a model of emotion, that was created for the expression of interactive self-animating characters; and (2) a motion synthesis system that combines hand-animated motion data with artistically authored procedures for generalizing the motion while preserving the artistic intent. This system has been used to create over a dozen interactive characters, which have been shown at juried venues, as well as being deployed commercially. We describe how artistic qualities important to interactive characters are encoded and supported using this system, and demonstrate the system with an implemented interactive character.

Integrated Natural Language Generation Systems

Many existing natural language generation systems can be characterized according to their modularization as either pipelined or interleaved. In these separated systems, the generator is divided into several modules (e.g., planning and realization), with control and information passing between the modules during the generation process. This paper proposes a third type of generator, which we call integrated, that unifies the modules into a single mechanism. The mechanism uses a small set of orthogonal basic operations to produce planned and grammatical language output. Integrated systems are conceptually attractive and may support generation of pragmatic effects more effectively than other systems. After discussing the advantages of the integrated approach, we summarize GLINDA, an integrated generator currently under development at Carnegie Mellon. GLINDA is the generator used for narration and intercharacter communication in the Oz Interactive Fiction and Virtual Reality Project.

Interface and narrative arts: contributions from narrative, drama, and film

This panel explores both theoretical and practical contributions from the disciplines of narrative, drama, and film to the field of interface design. Topics include . How can the natural human abilities to tell and understand stories be used in human-computer interaction? . What can be gained from the use of such techniques as poly-linear structure and point of view in narrative and film? ● What kinds of manipulations of time and space in representational worlds would be useful? “ What are some candidate architectures for implementing narrative and dramatic styles of human-computer interaction? Examples from applications in information retrieval, art, education, simulation, entertainment, and programming are presented.

Recognizing substrings of LR(k) languages in linear time

LR parsing techniques have long been studied as efficient and powerful methods for processing context free languages. A linear time algorithm for recognizing languages representable by LR(k) grammars has long been known. Recognizing substrings of a context-free language is at least as hard as recognizing full strings of the language, as the latter problem easily reduces to the former. In this paper we present a linear time algorithm for recognizing substrings of LR(k) languages, thus showing that the substring recognition problem for these languages is no harder than the full string recognition problem. An interesting data structure, the Forest Structured Stack, allows the algorithm to track all possible parses of a substring without loosing the efficiency of the original LR parser. We present the algorithm, prove its correctness, analyze its complexity, and mention several applications that have been constructed.