Steven J. Mead

Character-based interactive storytelling

Interactive storytelling is a privileged application of intelligent visual actor technology. The authors introduce their character-based interactive storytelling prototype that uses hierarchical task network planning techniques, which support story generation and any-time user intervention.

Agents' Interaction in Virtual Storytelling

In this paper we describe a fully implemented prototype for interactive storytelling using the Unreal™ engine. Using a sit-com like scenario as an example of how the dynamic interactions between agents and/or the user dramatise the emerging story. Hierarchical Task Networks (HTNs) are formalised using AND/OR graphs, which are used to describe the many possible variations of the story at a sub-goal level, and the set of all behaviours (from a narrative perspective) of the primary actors at a terminal action level. We introduce real-time variant of the heuristic search algorithm AO* that has been implemented to provide a mechanism for planning (and re-planning) and discuss how the chosen heuristic evaluation function is used to describe narrative concepts. We provide early results of several examples of how the same basic plot can have many differing story instantiations as a result of the dynamic interaction within the virtual set and the personalities of the primary characters, and detail the steps required in the plan generation.

AI-based animation for interactive storytelling

In this paper, we describe a method for implementing AI-based animation of artificial actors in the context of interactive storytelling. We have developed a fully implemented prototype based on the Unreal/sup TM/ game engine and carried out experiments with a simple sitcom-like scenario. We discuss the central role of artificial actors in interactive storytelling and how real-time generation of their behaviour participates to the creation of a dynamic storyline. We follow previous work describing the behaviour of artificial actors through AI planning formalisms, and adapt it to the context of narrative representation. The set of all possible behaviours, accounting for various instantiations of a basic plot, can be represented through an AND/OR graph. A real-time variant of the AO* algorithm can be used to interleave planning and action, thus allowing characters to interact between themselves and with the user. Finally, we present several examples of short plots and situations generated by the system from the dynamic interaction of artificial actors.

Character-driven story generation in interactive storytelling

We describe a fully implemented prototype for interactive storytelling using the Unreal/sup TM/ engine. We describe the important mechanisms involved in the variability of plot instantiations, within a scenario of sitcom genre. We also provide an evaluation of the concepts of how the dynamic interactions between agents and/or the user influence the generation of story, with first results of examples.

Interacting with Virtual Agents in Mixed Reality Interactive Storytelling

User interaction with virtual agents generally takes place in virtual environments in which there is clear separation between the virtual actors and the user, due to the fact that in most cases, the user is in some way external to the virtual world. In Mixed-Reality Interactive Storytelling, the user’s video image is captured in real time and inserted into a virtual world populated by autonomous synthetic actors with which the user interacts. The user in turn watches the composite world projected on a large screen, following a ”magic mirror” metaphor. This context leads to re-investigating the techniques by which the user interacts with virtual actors, as well as exploring specific research problems. In this paper, we discuss some specificities of user interaction with virtual actors in Mixed Reality Interactive Storytelling. After a brief introduction to our system’s architecture and the example scenario supporting our experiments, we describe various techniques supporting multi-modal interaction with virtual actors.

Users acting in mixed reality interactive storytelling

Entertainment systems promise to be a significant application for Mixed Reality. Recently, a growing number of Mixed Reality applications have included interaction with synthetic characters and storytelling. However, AI-based Interactive Storytelling techniques have not yet been explored in the context of Mixed Reality. In this paper, we describe a first experiment in the adaptation of an Interactive Storytelling technique to a Mixed Reality system. After a description of the real time image processing techniques that support the creation of a hybrid environment, we introduce the storytelling technique and the specificities of user interaction in the Mixed Reality context. We illustrate these experiments by discussing examples obtained from the system.

Developing Re-Usable Interactive Storytelling Technologies

Despite the growing interest in Interactive Storytelling (IS), there have been only a small number of implemented demonstrators and few have attempted at developing a re-usable IS technology. In this paper we describe such an IS engine, which is the result of several years of experimentation in the field. The system is based on a game engine for its visualisation component, while the narrative generation component implements a variant of HTN Planning. After an introduction to the principles underlying the system, we introduce the associated production process and discuss authoring problems as well as tools we have developed to facilitate the use of the technology.

From computer games to interactive stories: interactive storytelling

Interactive storytelling can be based either on explicit plot representations or on the autonomous behaviour of artificial characters. In such a character‐based approach, the dynamic interaction between characters generates the actual plot from a generic storyline. Characters’ behaviours are implemented through real‐time search‐based planning techniques. However, the top‐down planning systems that control artificial actors need to be complemented with appropriate mechanisms dealing with emerging (“bottom‐up”) situations of narrative relevance. After discussing the determinants that account for the emergence of narrative situations, we introduce additional mechanisms for coping with these situations. These comprise situated reasoning and action repair: we also illustrate the concepts through detailed examples.

Under the Influence: Using Natural Language in Interactive Storytelling

Interacting in natural language with virtual actors is an important aspect of the development of future Interactive Storytelling systems. We describe a paradigm for speech interfaces in interactive storytelling based on the notion of influence. In this paradigm, the user is mainly a spectator who is however able to interfere with the course of action by issuing advice to the characters. This is achieved by recognising corresponding speech acts and mapping them to the plans which implement characters’ behaviours in the story. We discuss some examples based on a preliminary, yet fully implemented, prototype.

Non-instructional linguistic communication with virtual actors

We explore a new paradigm for natural language communication with autonomous agents. While the dominant paradigm is to use natural language instructions from which the agent behaviour is generated, we investigate how natural language input can influence a pre-existing plan-based behaviour, by interfering with the various types of sub-goals in such a plan. Using as a testbed a fully implemented interactive storytelling application based on virtual actors, we identify various forms of communicative actions and how these can influence actors behaviours. We report early results from the use of a speech processing system used within the storytelling application to influence the behaviour of the artificial actors.