Stefan Rank

An interdisciplinary VR-architecture for 3D chatting with non-verbal communication

The communication between avatar and agent has already been treated from different but specialized perspectives. In contrast, this paper gives a balanced view of every key architectural aspect: from text analysis to computer graphics, the chatting system and the emotional model. Non-verbal communication, such as facial expression, gaze, or head orientation is crucial to simulate realistic behavior, but is still an aspect neglected in the simulation of virtual societies. In response, this paper aims to present the necessary modularity to allow virtual humans (VH) conversation with consistent facial expression -either between two users through their avatars, between an avatar and an agent, or even between an avatar and a Wizard of Oz. We believe such an approach is particularly suitable for the design and implementation of applications involving VHs interaction in virtual worlds. To this end, three key features are needed to design and implement this system entitled 3D-emoChatting. First, a global architecture that combines components from several research fields. Second, a real-time analysis and management of emotions that allows interactive dialogues with non-verbal communication. Third, a model of a virtual emotional mind called emoMind that allows to simulate individual emotional characteristics. To conclude the paper, we briefly present the basic description of a user-test which is beyond the scope of the present paper.

Applying a Text-Based Affective Dialogue System in Psychological Research: Case Studies on the Effects of System Behaviour, Interaction Context and Social Exclusion

This article presents two studies conducted with an affective dialogue system in which text-based system–user communication was used to model, generate and present different affective and social interaction scenarios. We specifically investigated the influence of interaction context and roles assigned to the system and the participants, as well as the impact of pre-structured social interaction patterns that were modelled to mimic aspects of “social exclusion” scenarios. The results of the first study demonstrate that both the social context of the interaction and the roles assigned to the system influence the system evaluation, interaction patterns, textual expressions of affective states, as well as emotional self-reports. The results observed for the second study show the system’s ability to partially exclude a participant from a triadic conversation without triggering significantly different affective reactions or a more negative system evaluation. The experimental evidence provides insights on the perception, modelling and generation of affective and social cues in artificial systems that can be realized in different modalities, including the text modality, thus delivering valuable input for applying affective dialogue systems as tools for studying affect and social aspects in online communication.

ON THE NATURE OF ENGINEERING SOCIAL ARTIFICIAL COMPANIONS

The literature on social agents has put forward a number of requirements that social agents need to fulfill. In this paper we analyze the kinds of reasons and motivations that lie behind the statement of these requirements. In a second part of the paper, we look at how one can go about engineering the social agents. We introduce a general language in which to express dialogue rules and some tools that support the development of dialogue systems.

Torchless: Asymmetry in a Shared Screen Dungeon Crawler

Torchless is a procedurally generated competitive dungeon crawling game where players have distinct asymmetric roles while sharing the same screen. Utilizing asymmetry, Torchless explores the relationship between players by experimenting with available information, player power, and player objectives.

Responsive Environmental Diegetic Audio Feedback for Hand Gestures in Audio-Only Games

Touch-less hand gestures require a structured approach towards the design of audio feedback, especially in audio-only environments. We present an implementation approach for responsive audio feedback geared towards hand gestures in audio-only games, focusing on diegetic environmental feedback before, during, and after gestures. The developed framework is used in a project investigating the usefulness of different feedback designs. Preliminary results suggest that the implemented game variants are both rated highly in terms of usability and enjoyment, and that replacing explicit audio instructions for hand positions and movements with responsive audio feedback for suggesting interaction methods using environmental story-related audio cues leads to higher immersion.

Zooming in: Studying Collective Emotions with Interactive Affective Systems

Computer-mediated communication between humans is at the center of the formation of collective emotions on the Internet. This chapter presents how interactive affective systems can be applied in order to study the role of emotion in online communication at the micro-scale, i.e. between individual users or between users and artificial communication partners. Specifically, we report on the effect of a simulated conversational partner’s affective profile, the use of fine-grained communication scenarios and social interaction context on changes in emotional states and expressed affect of users as well as their communication patterns. Based on these findings, we propose applications for such systems focused on supporting different e-communities with real-time information and discuss ethical implications of such systems.

Studying the Impact of Spatial Involvement on Training Mental Rotation with Minecraft

Previous research demonstrates a shift from negative behavioral consequences to positive cognitive development in understanding the effects of computer gameplay across multiple age groups. However, there are still questions about which aspects of gameplay have an impact on those positive consequences. This work-in-progress focuses on the role of spatial involvement via different control schemes by implementing a training regimen through Minecraft for mental rotation in order to understand the impact of aspects of involvement on the training outcome. The project utilizes Minecraft to recreate a proven engaging experience for players, making training mental rotation more accessible. Comparing different levels of spatial involvement will be used to understand factors of the experiences impact. Further, the work extends game training endeavors in psychology using digital media. A target outcome is an improved understanding of what aspects of digital games are relevant for successful training applications.

Designing for Emotional Complexity in Games: The Interplay of Positive and Negative Affect

People play games for the experience, and one of the aims of player experience research is to understand what constitutes and contributes to positive gaming experiences. Emotionally challenging and uncomfortable game play experiences have been largely neglected, as they are seemingly at odds with the fields focus on fun and positive affect. We argue that the positively-biased perspective on desirable emotions in games misses out on opportunities that the interplay between positive and negative emotions offers. A previous workshop at CHI PLAY 2015 covered this missed opportunity by focusing on the false dichotomy between positive and negative affect, and identified a number of factors, both personal and contextual, which determine when players will value emotional game experiences that go beyond the purely positive. The present workshop is a continuation of this effort, putting the spotlight on the complexity of emotional experience and how it evolves throughout game play. Crucially, a central aspect of this workshop is to get participants thinking more about the design and evaluation of these types of experiences, by allowing hands-on game design exercise for the examined emotional experiences.

You're the Camera!: Physical Movements For Transitioning Between Environments in VR

Virtual reality (VR) enables unusual experiences, including the physically impossible, in an immersive environment. As new media such as VR are developed, designers tend to remediate aspects from previous media, but not every aspect fits. Several areas in VR design warrant scientific investigation in that regard. This paper specifically addresses transitioning between environments: when transitioning in a virtual world, will camera movements made simultaneously and in sync with movements from the user produce a preferred scene transition experience compared to virtual camera movement that is less directly coupled? We report on a within-subject experiment where participants were tasked with transitioning between different environments. One set of transitions required the full physical motion of the user to complete, for the other set completing the transition was triggered after a part of the physical movement was performed up to a threshold. Results showed a clear preference for the second variant and thus for less control over the virtual camera.

Spatial Audio Feedback for Hand Gestures in Games

We use game prototypes to investigate different audio feedback designs for hand gestures for encouraging immersion. Replacing explicit audio instructions for hand positions and movements with responsive audio feedback, suggestive of interaction methods by using environmental story-related audio cues, leads to measurably higher immersion. We present the design of responsive audio feedback for hand gestures in the context of such a game. The spatial organization of diegetic environmental feedback before, during, and after gesture input is supported by a structured approach towards the design of audio cues.