Jean-Luc Lugrin

Madame bovary on the holodeck: immersive interactive storytelling

In this paper, we describe a small-scale, yet complete, integration of a real-time immersive interactive storytelling system. While significant progress has been achieved in recent years on the individual component technologies of interactive storytelling, the main objective of this work is to investigate the concept of interactive storytelling in a fully immersive context. We describe each individual component of immersive interactive storytelling from a technical perspective. We have used a commercial game engine as a development environment, supporting real-time visualisation as well as the inclusion of Artificial Intelligence components controlling virtual actors. This visualisation engine has been ported to an immersive setting using dedicated software and hardware supporting real-time stereoscopic visualisation. The hardware platform is built around a 4-sided CAVE-like immersive display operated by a PC-cluster. The interactive storytelling engine is constituted by a planning system based on characters motivations and emotional states. The user can interact with the virtual world using multimodal interaction. We illustrate the systems behaviour on the implementation of excerpts from Madame Bovary, a classic XIXth century novel, and demonstrate the ability for the user to play the role of one of the characters and influence the unfolding of the story by his actions.

Making sense of virtual environments: action representation, grounding and common sense

The development of complex interactive 3D systems raises the need for representations supporting more abstract descriptions of world objects, their behaviour and the world dynamics. The inclusion of Artificial Intelligence representations and their use within 3D graphic worlds face both fundamental and technical issues due to the difference in representational logic between computer graphics and knowledge-based systems. We present a framework for such an integration illustrated by a first prototype.

The CaveUT system: immersive entertainment based on a game engine

We describe recent developments in the CaveUT software, which supports immersive virtual reality installations based on the Unreal Tournament game engine. CaveUT implements several high-end VR features such as real-time stereoscopy with head and hand tracking. We demonstrate the use of CaveUT in the SAS Cube™, a PC-based CAVE™-like immersive four-screen display. One of the main advantages of the system is to support fully immersive VR while retaining the game engines advanced features for interaction and behavioral (or AI) systems. We illustrate the use of CaveUT on two installations: an artistic VR installation and an immersive interactive storytelling system.

Avatar anthropomorphism and illusion of body ownership in VR

In this paper we present a novel experiment to explore the impact of avatar realism on the illusion of virtual body ownership (IVBO) in immersive virtual environments, with full-body avatar embodiment and freedom of movement. We evaluated four distinct avatars (a humanoid robot, a block-man, and both male and female human adult) presenting an increasing level of anthropomorphism in their detailed compositions Our results revealed that each avatar elicited a relatively high level of illusion. However both machine-like and cartoon-like avatars elicited an equivalent IVBO, slightly superior to the human-ones. A realistic human appearance is therefore not a critical top-down factor of IVBO, and could lead to an Uncanney Valley effect.

Anthropomorphism and illusion of virtual body ownership

In this paper we present a novel experiment to explore the impact of avatar realism on the illusion of virtual body ownership (IVBO) in immersive virtual environments, with full-body avatar embodiment and freedom of movement. We evaluated four distinct avatars presenting an increasing level of anthropomorphism in their detailed compositions. Our results revealed that each avatar elicited a relatively high level of illusion. However both machine-like and cartoon-like avatars elicited an equivalent IVBO, slightly superior to the human-ones. A realistic human appearance is therefore not a critical top-down factor of IVBO, and could lead to an Uncanny Valley effect.

Intelligent virtual environments for virtual reality art

The development of virtual reality (VR) art installations is faced with considerable difficulties, especially when one wishes to explore complex notions related to user interaction. We describe the development of a VR platform, which supports the development of such installations, from an art+science perspective. The system is based on a CAVE(TM)-like immersive display using a game engine to support visualisation and interaction, which has been adapted for stereoscopic visualisation and real-time tracking. In addition, some architectural elements of game engines, such as their reliance on event-based systems have been used to support the principled definition of alternative laws of Physics. We illustrate this research through the development of a fully implemented artistic brief that explores the notion of causality in a virtual environment. After describing the hardware architecture supporting immersive visualisation we show how causality can be redefined using artificial intelligence technologies inspired from action representation in planning and how this symbolic definition of behaviour can support new forms of user experience in VR.

Exploring the usability of immersive interactive storytelling

The Entertainment potential of Virtual Reality is yet to be fully realised. In recent years, this potential has been described through the Holodeck#8482; metaphor, without however addressing the issue of content creation and gameplay. Recent progress in Interactive Narrative technology makes it possible to envision immersive systems. Yet, little is known about the usability of such systems or which paradigms should be adopted for gameplay and interaction. We report user experiments carried out with a fully immersive Interactive Narrative system based on a CAVE-like system, which explore two interactivity paradigms for user involvement (Actor and Ghost). Our results confirm the potential of immersive Interactive Narratives in terms of performance but also of user acceptance.

Immersive FPS games: user experience and performance

Computer games are ideally placed to form the content of future Immersive Media, but this prospect is faced with both technical and usability issues. This paper describes an experiment in immersive gaming using a state-of-the-art computer First Person Shooter (FPS) game, in which we analyze user experience and performance through a combination of in-game metrics, questionnaires and subjective reports. We describe the evaluation of a major commercial computer game as a real-time immersive stereoscopic experience based on a four-screen CAVE-like installation. The implementation is based on a bespoke VR middleware developed on top of the games own engine. Our results show an overwhelming subjective preference for the immersive version despite a decrease in performance attributed to a more realistic aiming mechanism. More importantly, metrics suggest that users took advantage of the immersive context rather than simply transposing their desktop gaming skills.

Causality and virtual reality art

In this paper, we discuss how a cognitive concept, causality, can be used for the conceptual underpinning of Virtual Reality Art installations. Causality plays an important role in our construction of reality and, as such, it makes sense to use it as a principle to define VR experiences. We have developed a VR platform using cognitive data on causal perception to create artificial event co-occurrences in virtual worlds, which can be perceived as possible outcomes for user actions. After a preliminary validation of this technology by user experiments, it has been used to implement prototypes of artistic installations by two different artists. We describe the technical approach behind the elicitation of causal perception in virtual reality, and illustrate its use through the two artistic installations being developed with this new VR platform.

Alternative reality: a new platform for virtual reality art

Virtual Reality Art involves the design of artificial worlds that offer new experiences to spectators. An important aspect for the development of VR Art installations is the principled definition of behaviour for the environment as a whole, which would facilitate experiments with alternative laws of physics, time, and causality. We describe the first results of an ongoing project dedicated to the development of software tools for the use of Intelligent Virtual Environments in VR Art. Using the architecture of a state-of-the-art game engine, we have developed Artificial Intelligence techniques that support the definition of alternative laws of physics. After discussing the principles behind alternative reality we describe two complementary modes of description for alternative behaviour: qualitative physics and causal simulation. This is illustrated by examples integrated into the virtual environment.