Hanna Schraffenberger

Interaction Models for Audience-Artwork Interaction: Current State and Future Directions

Interactive art is of great relevance to the arts, sciences and technology alike. A common field of interest among researchers of different disciplines, practising artists and art institutes is the interaction between audience and artwork. This paper reviews existing research concerning interaction in interactive art and discusses its applicability for describing and classifying audience-artwork interaction. In pointing out possible future directions, we identify a need for models describing the relation between the audience’s and artwork’s actions and reactions as well as the necessity for future research looking at interaction as a continuous bi-directional process between work and audience.

Multimodal augmented reality: the norm rather than the exception

Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participants view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case for an experience-focused and modalities-encompassing understanding of AR. We argue that multi-modality in AR is the norm rather than the exception, as AR environments consist of both virtual content and our real, physical, multimodal world. We explore the role multi-modal and non-visual aspects of our physical reality can play when creating AR scenarios and the possibilities and challenges that emerge when approaching AR from a modalities-encompassing perspective.

Sensory Augmentation: Toward a Dialogue Between the Arts and Sciences

People sense the world by exploiting correlations between their physical actions and the changing sensory input that results from those actions. Interfaces that translate non-human sensor data to signals that are compatible with the human senses can therefore augment our abilities to make sense of the world. This insight has recently sparked an increase in projects that explore sensemaking and the creation of novel human experiences across scientific and artistic disciplines. However, there currently exists no constructive dialogue between artists and scientists that conduct research on this topic. In this position paper, we identify the theory and practice of sensory augmentation as a domain that could benefit from such a dialogue. We argue that artistic and scientific methods can complement each other within research on sensory augmentation and identify six thematic starting points for a dialogue between the arts and sciences. We conducted a case study to explore these conjectures, in which we instigated such a dialogue on a small scale. The case study revealed that the six themes we identified as relevant for a dialogue on sensory augmentation emerge rather spontaneously in such a dialogue and that such an exchange may facilitate progress on questions that are central to the theory and practice of sensory augmentation. Overall, this position paper contributes preliminary evidence for the potential of, and a starting point for, a dialogue between the arts and sciences that advances our understanding of sensory augmentation and the development of applications that involve it.

Towards Novel Relationships between the Virtual and the Real in Augmented Reality

In Augmented Reality (AR), virtual and real content coexist in the same physical environment. However, in order to create AR, solely adding virtual content to a real space does not suffice. In this paper we argue that an augmentation adds and relates something virtual to something real. Subsequently, we discuss both existing and promising future relationships between the virtual and the real. We explore what AR is and what it could possibly include from a technology-independent and conceptual point of view. By comparing our take on AR with common manifestations of AR, we identify possible directions for future research and AR (art) works, such as the use of non-visual modalities and the design of novel interactions between the virtual and the real.

Augmented Metacognition: Exploring Pupil Dilation Sonification to Elicit Metacognitive Awareness

Metacognitive awareness enables people to make conscious decisions about their own cognitions, and adapt to meet task performance goals. Despite the role of metacognition in task performance, technologies that effectively augment metacognition are scarce. We explore a novel approach to augment metacognition based on making the eyes pupil dilations, which associate with a variety of cognitions, audible via sonification in real-time. In this exploratory study, we investigated whether pupil dilation sonification can elicit metacognitive awareness. Our findings suggest that correlations between a variety of cognitions, e.g., attentional focus and depth of thinking, and sounds generated by the sonification can emerge spontaneously and by instruction. This justifies further research into the use of pupil dilation sonification as a means to augment metacognitive abilities.

Reconsidering Registration: New Perspectives on Augmented Reality

Augmented reality (AR) projects typically involve interactive systems that align virtual objects with the real world. This process is called registration and can make it seem as if virtual objects existed in the otherwise real environment. Registration is widely accepted as a defining and necessary characteristic of augmented reality. In this paper, we reconsider the need for registration on two levels. First of all, we argue that the intended presence of virtual objects in real space can be achieved without registration by an interactive AR system. Secondly, we suggest that the perceived spatial presence of virtual content in real space is not necessary for AR in the first place. We illustrate both points with examples and propose a more encompassing view of AR that focuses on relationships between the virtual and the real rather than on registration.