Christopher E. Peters

Developing serious games for cultural heritage: a state-of-the-art review

Although the widespread use of gaming for leisure purposes has been well documented, the use of games to support cultural heritage purposes, such as historical teaching and learning, or for enhancing museum visits, has been less well considered. The state-of-the-art in serious game technology is identical to that of the state-of-the-art in entertainment games technology. As a result, the field of serious heritage games concerns itself with recent advances in computer games, real-time computer graphics, virtual and augmented reality and artificial intelligence. On the other hand, the main strengths of serious gaming applications may be generalised as being in the areas of communication, visual expression of information, collaboration mechanisms, interactivity and entertainment. In this report, we will focus on the state-of-the-art with respect to the theories, methods and technologies used in serious heritage games. We provide an overview of existing literature of relevance to the domain, discuss the strengths and weaknesses of the described methods and point out unsolved problems and challenges. In addition, several case studies illustrating the application of methods and technologies used in cultural heritage are presented.

A model of attention and interest using Gaze behavior

One of the major problems of users interaction with Embodied Conversational Agents (ECAs) is to have the conversation last more than few second: after being amused and intrigued by the ECAs, users may find rapidly the restrictions and limitations of the dialog systems, they may perceive the repetition of the ECAs animation, they may find the behaviors of ECAs to be inconsistent and implausible, etc. We believe that some special links, or bonds, have to be established between users and ECAs during interaction. It is our view that showing and/or perceiving interest is the necessary premise to establish a relationship. In this paper we present a model of an ECA able to establish, maintain and end the conversation based on its perception of the level of interest of its interlocutor.

An exploration of user engagement in HCI

Engagement is a concept of the utmost importance in human computer interaction, not only for informing the design and implementation of interfaces, but also for enabling more sophisticated interfaces capable of adapting to users. While the notion of engagement is actively being studied in a diverse set of domains, the term has been used to refer to a number of related, but different concepts. This paper represents a first attempt at exploring a number of important concepts that the term has been used to refer to, of relevance to both human-human and human-machine interaction modelling.

Direction of attention perception for conversation initiation in virtual environments

We consider the role of gaze and direction of attention for providing embodied agents with the capability of visually perceiving the attention of others in a virtual environment. Such a capability is of importance in social environments where the directions in which others orient themselves provides information necessary for detecting important social cues and serving as a basis for inferring information about their possible motives, desires and intentions. Our real-time model uses synthetic vision and memory to implement a perceptually-based theory of mind that considers the direction of the eyes, head, body and locomotion of others. These contribute to metrics that describe the awareness and amount of interest that another is deemed to have in the self. We apply this capability to an automated conversation initiation scenario where an agent who encounters a potential interaction partner considers not only its own interaction goal, but also its theory of the goal of the other. Our aim is to improve the plausibility of animated social interaction and is inspired by human social behaviour, where one generally wishes to avoid the embarrassing situation of committing to a conversation with an unwilling participant.

Investigating shared attention with a virtual agent using a gaze-based interface

This paper investigates the use of a gaze-based interface for testing simple shared attention behaviours during an interaction scenario with a virtual agent. The interface is non-intrusive, operating in real-time using a standard web-camera for input, monitoring users’ head directions and processing them in real-time for resolution to screen coordinates. We use the interface to investigate user perception of the agent’s behaviour during a shared attention scenario. Our aim is to elaborate important factors to be considered when constructing engagement models that must account not only for behaviour in isolation, but also for the context of the interaction, as is the case during shared attention situations.

Expressive Copying Behavior for Social Agents: A Perceptual Analysis

Successful human interaction commonly involves prototypical exchanges where interactors are engaged, synchronized, and harmonious in their behaviors. The copying of aspects of the others behavior, at different levels, seems central to establishing and maintaining such empathic connections. Yet, many questions remain unanswered, particularly how it is possible to reflect the same affective content back to the other when the actual motion itself is not exactly the same as theirs. This paper presents a perceptual study in which emotional gestures conducted by an actor were mapped onto synthesized versions generated by an embodied virtual agent. Copying is at the expressive level, where qualities such as the fluidity or expansiveness of gestures are considered, rather than exact low-level motion matching. Participants were later asked to rate the emotional content of video recordings of both the original and the synthesized gestures. A statistical analysis shows that, in most cases, participants associated the emotional content of the agents gestures with that intended to be expressed by the original actor. The results suggest that a combination of the type of movement performed and its quality is important for successfully communicating emotions.

Graphics for Serious Games: A head movement propensity model for animating gaze shifts and blinks of virtual characters

An automatic model is presented for animating gaze shifts of virtual characters towards target locations in a virtual environment. Two connected components are described: an eye-head controller and a blinking controller. The gaze control model is based on results from neuroscience, and dictates the contributions of the eyes and head to a gaze shift according to an individuals head movement propensity; that is, their tendency to recruit their head when making gaze motions under different conditions. The blink controller simulates gaze-evoked blinking, a specific category of behaviours that accompany gaze shifts. The probability of occurrence of such blinks, and their amplitude, is related to the gaze shift. The model forms the basis for a number of experiments investigating the impact of blinking, eye-head ratio and direction of head movements on user perception. In addition to other application domains, the findings are of significance to serious games environments, where the perceived quality of a characters gaze may affect engagement, immersion and learning outcomes.

The HUMAINE Database

The HUMAINE Database is grounded in HUMAINE’s core emphasis on considering emotion in a broad sense – ‘pervasive emotion’ – and engaging with the way it colours action and interaction. The aim of the database is to provide a resource to which the community can go to see and hear the forms that emotion takes in everyday action and interaction, and to look at the tools that might be relevant to describing it. Earlier chapters in this handbook describe the techniques and models underpinning the collection and labelling of such data. This chapter focuses on conveying the range of forms that emotion takes in the database, the ways that they can be labelled and the issues that the data raises. The HUMAINE Database provides naturalistic clips which record that kind of material, in multiple modalities, and labelling techniques that are suited to describing it. It was clear when the HUMAINE project began that work on databases should form part of it. However there were very different directions that the work might have taken. They were encapsulated early on in the contrast between ‘supportive’ and ‘provocative’ approaches, introduced in an earlier chapter in this handbook. The supportive option was to assemble a body of data whose size and structure allowed it to be used directly to build systems for recognition and/or synthesis. The provocative option was to assemble a body of data that encapsulated the challenges that the field faces.

A level of interaction framework for exploratory learning with characters in virtual environments

This paper investigates a range of challenges faced in the design of a serious game aimed at teaching history in situ, through the use of an immersive, open virtual environment. In the context of this paper, such an environment is described as an exploratory, expansive virtual world within which a user may interact in a non-linear and situated fashion with the virtual characters that populate it. The main contribution of this paper consists of the introduction of the Levels of Interaction (LoI) framework, designed to assist in the creation of multiple forms of interaction between a user-driven avatar and synthetic characters. The LoI approach addresses the necessity for balancing computational efficiency with the need to provide believable and interactive virtual characters, allowing varying degrees of visual, interactive and behavioural fidelity. The Roma Nova project demonstrates a first implementation of the concept, showing in practice how the LoI are likely to foster more natural interactions between the player and the non-playing characters.

On the Provision of a Comprehensive Computer Graphics Education in the Context of Computer Games: An Activity-Led Instruction Approach

Over the past decade the development of computer games – which originated in academia with the creation of Spacewar at MIT in 1961 – has evolved into an accepted academic discipline, closely related to the field of computer graphics. Games courses can be found embedded in traditional computer science degrees or as dedicated degree programmes for students aiming to work in the games industry. In this paper we present a student-centred, activity-led approach to teaching computer graphics in the context of a computer games technology undergraduate degree. We describe our computer graphics related courses and demonstrate how they are formed by the activity-led teaching methodology.