Michael Nitsche

Tangible interfaces for real-time 3D virtual environments

Emergent game formats, such as machinima, that use game worlds as expressive 3D performance spaces have new expressive powers with an increase of the quality of their underlying graphic and animation systems. Nevertheless, they still lack intuitive control mechanisms. Set direction and acting are limited by tools that were designed to create and play video games rather than produce expressive performance pieces. These tools do a poor job of capturing the performative expression that characterizes more mature media such as film. Tangible interfaces can help open up the game systems for more intuitive character control needed for a greater level of expression in the digital real-time world.n The TUI3D project (Tangible User Interfaces for Real-Time 3D) addresses production and performative challenges involved in creating machinima through the development of tangible interfaces for controlling 3D virtual actors and environments in real-time. In this paper, we present the design and implementation of a tangible puppet prototype for virtual character control in the Unreal game engine and discuss initial user feedback.

I'm in the game: embodied puppet interface improves avatar control

We have developed an embodied puppet interface that translates a players body movements to a virtual character, thus enabling the player to have a fine grained and personalized control of the avatar. To test the efficacy and short-term effects of this control interface, we developed a two-part experiment, where the performance of users controlling an avatar using the puppet interface was compared with users controlling the avatar using two other interfaces (Xbox controller, keyboard). Part 1 examined aiming movement accuracy in a virtual contact game. Part 2 examined changes of mental rotation abilities in users after playing the virtual contact game. Results from Part 1 revealed that the puppet interface group performed significantly better in aiming accuracy and response time, compared to the Xbox and keyboard groups. Data from Part 2 revealed that the puppet group tended to have greater improvement in mental rotation accuracy as well. Overall, these results suggest that the embodied mapping between a player and avatar, provided by the puppet interface, leads to important performance advantages.

Giving your self to the game: transferring a player's own movements to avatars using tangible interfaces

We investigate the cognitive connection players create between their own bodies and the virtual bodies of their game avatars through tangible interfaces. The work is driven by experimental results showing that execution, perception and imagination of movements share a common coding in the brain, which allows people to recognize their own movements better. Based on these results, we hypothesize that players would identify and coordinate better with characters that encode their own movements. We tested this hypothesis in a series of four studies (n=20) that tracked different levels of movement perception abstraction, from own body to that of an avatars body controlled by the participant, to see in which situations people recognize their own movements. Results show that participants can recognize their movements even in abstracted and distorted presentations. This recognition of own movements occurs even when people do not see themselves, but just see a puppet they controlled. We conclude that players - if equipped with the appropriate interfaces - can indeed project and decipher their own body movements in a game character.

Creativity support for novice digital filmmaking

Machinima is a new form of creative digital filmmaking that leverages the real time graphics rendering of computer game engines. Because of the low barrier to entry, machinima has become a popular creative medium for hobbyists and novices while still retaining borrowed conventions from professional filmmaking. Can novice machinima creators benefit from creativity support tools? A preliminary study shows novices generally have difficulty adhering to cinematographic conventions. We identify and document four cinematic conventions novices typically violate. We report on a Wizard-of-Oz study showing a rule-based intelligent system that can reduce the frequency of errors that novices make by providing information about rule violations without prescribing solutions. We discuss the role of error reduction in creativity support tools.

Game design education: integrating computation and culture

Game studies, as a humanistic discipline, has no defined limits to its coverage, no single methodology, and no clear historical boundary. In short, the creation of a curriculum around games is an emerging practice in which research and education, theory and practice, art and commerce, and existing disciplinary boundaries all continue changing in challenging and unpredictable ways. Although many universities focus their digital media curricula on game production and game studies, Georgia Techs commitment to humanistic frameworks connect this emerging environment to longer traditions of human culture by emphasizing research into the expressive potential of games

Recognizing self in puppet controlled virtual avatars

Recent work in neuroscience suggests that there is a common coding in the brain between perception, imagination and execution of movement. Further, this common coding is considered to allow people to recognize their own movements when presented as abstract representations, and coordinate with these movements better. We are investigating how this own movement effect could be extended to improve the interaction between players and game avatars, and how it might be leveraged to augment players cognition. To examine this question, we have designed and developed a tangible puppet interface and 3D virtual environment that are tailored to investigate the mapping between player and avatar movements. In a set of two experiments, we show that when the puppet interface is used to transfer players movements to the avatar, the players are able to recognize their own movements, when presented alongside others movements. In both experiments, players did not observe their movements being transferred to the avatar, and the recognition occurred after a week of the transfer. Since the recognition effect persisted even with these two handicaps, we conclude that this is a robust effect, and the puppet interface is effective in personalizing an avatar, by transferring a players own movements to the virtual character.

Soundscape composition and field recording as a platform for collaborative creativity

In this paper, the authors describe and discuss UrbanRemix, a platform consisting of mobile-device applications and web-based tools to facilitate collaborative field recording, sound exploration, and soundscape creation. Reflecting on its use at workshops, festivals and community events, they evaluate the project in terms of its ability to enable participants to engage with their aural environments and to uncover their own creativity in the process.

Distributed creative cognition in digital filmmaking

This paper reports on an empirical study that uses a Grounded Theory approach to investigate the creative practices of Machinima filmmakers. Machinima is a new digital film production technique that uses the 3D graphics and real time rendering capability of video game engines to create films. In contrast to practices used in traditional film production, weve found that Machinima filmmakers explore and evaluate ideas in real time. These filmmakers generate vague and underspecified mental images, which are then explored and refined using the real time rendering capabilities of game engines. The game engine assists the filmmaker to fill in indeterminate details, which allows creative exploration of scenes through playfully experimenting with parameters such as camera angle and position, lighting, and character position. Creative exploration distributes the cognitive task of evaluation between the human user and the Machinima tool to enable evaluation through exploring possible scene configurations.

Experiments in the use of game technology for pre-visualization

This overview paper outlines the value of real-time 3D engines for pre-visualization. Pre-visualization is a standard tool during pre-production of many modern film productions. First, the parallels between the two increasingly digitized technologies are discussed. Second, the paper outlines the special needs and problems posed by pre-visualization. It argues that animation control and camera control are the two main areas that need to be addressed. Finally, it presents a range of experiments that provide different practical approaches to these two core questions and utilize available game technology. The approach of these tests was to keep the rendering real-time -- liquid -- as long as possible. This follows original machinima-like production pipelines. Ultimately, the here presented prototypes illustrate the value of real-time game engines for pre-visualization as well as still prevailing limitations.

Stories in space: The concept of the story map

While 3D space has become almost ubiquitous in computer games that apply narrative techniques, theoretical frameworks and practical experiments about the use of virtual space are underdeveloped compared to the number of works that deal with literary textual pieces such as MUDs. Offering one element to fill this gap, the notion of a Story Map is introduced in this paper. The interactor’s experience of space and of the events in a Real-Time 3-Dimensional Virtual Environment (RT 3D VE) form a constant discourse and Story Maps are seen as a form of the interactor’s comprehension of this discourse. The Common Tales research project exemplifies the development of this theory before the value of the Story Map for MMORPG’s will be outlined.