Fred Charles

Character-based interactive storytelling

Interactive storytelling is a privileged application of intelligent visual actor technology. The authors introduce their character-based interactive storytelling prototype that uses hierarchical task network planning techniques, which support story generation and any-time user intervention.

Madame bovary on the holodeck: immersive interactive storytelling

In this paper, we describe a small-scale, yet complete, integration of a real-time immersive interactive storytelling system. While significant progress has been achieved in recent years on the individual component technologies of interactive storytelling, the main objective of this work is to investigate the concept of interactive storytelling in a fully immersive context. We describe each individual component of immersive interactive storytelling from a technical perspective. We have used a commercial game engine as a development environment, supporting real-time visualisation as well as the inclusion of Artificial Intelligence components controlling virtual actors. This visualisation engine has been ported to an immersive setting using dedicated software and hardware supporting real-time stereoscopic visualisation. The hardware platform is built around a 4-sided CAVE-like immersive display operated by a PC-cluster. The interactive storytelling engine is constituted by a planning system based on characters motivations and emotional states. The user can interact with the virtual world using multimodal interaction. We illustrate the systems behaviour on the implementation of excerpts from Madame Bovary, a classic XIXth century novel, and demonstrate the ability for the user to play the role of one of the characters and influence the unfolding of the story by his actions.

Applying planning to interactive storytelling: Narrative control using state constraints

We have seen ten years of the application of AI planning to the problem of narrative generation in Interactive Storytelling (IS). In that time planning has emerged as the dominant technology and has featured in a number of prototype systems. Nevertheless key issues remain, such as how best to control the shape of the narrative that is generated (e.g., by using narrative control knowledge, i.e., knowledge about narrative features that enhance user experience) and also how best to provide support for real-time interactive performance in order to scale up to more realistic sized systems. Recent progress in planning technology has opened up new avenues for IS and we have developed a novel approach to narrative generation that builds on this. Our approach is to specify narrative control knowledge for a given story world using state trajectory constraints and then to treat these state constraints as landmarks and to use them to decompose narrative generation in order to address scalability issues and the goal of real-time performance in larger story domains. This approach to narrative generation is fully implemented in an interactive narrative based on the “Merchant of Venice.” The contribution of the work lies both in our novel use of state constraints to specify narrative control knowledge for interactive storytelling and also our development of an approach to narrative generation that exploits such constraints. In the article we show how the use of state constraints can provide a unified perspective on important problems faced in IS.

Agents' Interaction in Virtual Storytelling

In this paper we describe a fully implemented prototype for interactive storytelling using the Unreal™ engine. Using a sit-com like scenario as an example of how the dynamic interactions between agents and/or the user dramatise the emerging story. Hierarchical Task Networks (HTNs) are formalised using AND/OR graphs, which are used to describe the many possible variations of the story at a sub-goal level, and the set of all behaviours (from a narrative perspective) of the primary actors at a terminal action level. We introduce real-time variant of the heuristic search algorithm AO* that has been implemented to provide a mechanism for planning (and re-planning) and discuss how the chosen heuristic evaluation function is used to describe narrative concepts. We provide early results of several examples of how the same basic plot can have many differing story instantiations as a result of the dynamic interaction within the virtual set and the personalities of the primary characters, and detail the steps required in the plan generation.

AI-based animation for interactive storytelling

In this paper, we describe a method for implementing AI-based animation of artificial actors in the context of interactive storytelling. We have developed a fully implemented prototype based on the Unreal/sup TM/ game engine and carried out experiments with a simple sitcom-like scenario. We discuss the central role of artificial actors in interactive storytelling and how real-time generation of their behaviour participates to the creation of a dynamic storyline. We follow previous work describing the behaviour of artificial actors through AI planning formalisms, and adapt it to the context of narrative representation. The set of all possible behaviours, accounting for various instantiations of a basic plot, can be represented through an AND/OR graph. A real-time variant of the AO* algorithm can be used to interleave planning and action, thus allowing characters to interact between themselves and with the user. Finally, we present several examples of short plots and situations generated by the system from the dynamic interaction of artificial actors.

Character-driven story generation in interactive storytelling

We describe a fully implemented prototype for interactive storytelling using the Unreal/sup TM/ engine. We describe the important mechanisms involved in the variability of plot instantiations, within a scenario of sitcom genre. We also provide an evaluation of the concepts of how the dynamic interactions between agents and/or the user influence the generation of story, with first results of examples.

Interacting with Virtual Agents in Mixed Reality Interactive Storytelling

User interaction with virtual agents generally takes place in virtual environments in which there is clear separation between the virtual actors and the user, due to the fact that in most cases, the user is in some way external to the virtual world. In Mixed-Reality Interactive Storytelling, the user’s video image is captured in real time and inserted into a virtual world populated by autonomous synthetic actors with which the user interacts. The user in turn watches the composite world projected on a large screen, following a ”magic mirror” metaphor. This context leads to re-investigating the techniques by which the user interacts with virtual actors, as well as exploring specific research problems. In this paper, we discuss some specificities of user interaction with virtual actors in Mixed Reality Interactive Storytelling. After a brief introduction to our system’s architecture and the example scenario supporting our experiments, we describe various techniques supporting multi-modal interaction with virtual actors.

Interactive storytelling via video content recombination

In the paper we present a prototype of video-based storytelling that is able to generate multiple story variants from a baseline video. The video content for the system is generated by an adaptation of forefront video summarisation techniques that decompose the video into a number of Logical Story Units (LSU) representing sequences of contiguous and interconnected shots sharing a common semantic thread. Alternative storylines are generated using AI Planning techniques and these are used to direct the combination of elementary LSU for output. We report early results from experiments with the prototype in which the reordering of video shots on the basis of their high-level semantics produces trailers giving the illusion of different storylines.

Immersive FPS games: user experience and performance

Computer games are ideally placed to form the content of future Immersive Media, but this prospect is faced with both technical and usability issues. This paper describes an experiment in immersive gaming using a state-of-the-art computer First Person Shooter (FPS) game, in which we analyze user experience and performance through a combination of in-game metrics, questionnaires and subjective reports. We describe the evaluation of a major commercial computer game as a real-time immersive stereoscopic experience based on a four-screen CAVE-like installation. The implementation is based on a bespoke VR middleware developed on top of the games own engine. Our results show an overwhelming subjective preference for the immersive version despite a decrease in performance attributed to a more realistic aiming mechanism. More importantly, metrics suggest that users took advantage of the immersive context rather than simply transposing their desktop gaming skills.

Users acting in mixed reality interactive storytelling

Entertainment systems promise to be a significant application for Mixed Reality. Recently, a growing number of Mixed Reality applications have included interaction with synthetic characters and storytelling. However, AI-based Interactive Storytelling techniques have not yet been explored in the context of Mixed Reality. In this paper, we describe a first experiment in the adaptation of an Interactive Storytelling technique to a Mixed Reality system. After a description of the real time image processing techniques that support the creation of a hybrid environment, we introduce the storytelling technique and the specificities of user interaction in the Mixed Reality context. We illustrate these experiments by discussing examples obtained from the system.