Greg Corness

What the body knows: Exploring the benefits of embodied metaphors in hybrid physical digital environments

A recent trend in ubiquitous computing is the development of new forms of interfaces, which rely on embodied interaction. We focus on the definition of embodiment that refers to the ways in which abstract concepts rely on metaphorical extensions of embodied schemata shaped by processes below the level of conscious awareness as explored by Lakoff and Johnson [Lakoff, G., Johnson, M., 1980. Metaphors We Live By. Chicago Press, Chicago, IL, USA]. Our inquiry focuses on understanding the role embodied metaphors may play in supporting people to understand the possibilities for physical interaction in augmented spaces. We explore this issue through the development and evaluation of an interactive audio environment. We instantiate metaphor theory by using embodied schemata as the basis for the interactional metaphor that relates full-body input actions to audio output responses. We demonstrate and explore the benefits of this approach through a comparative experiment in which adults and children learn to use our audio environment. The results from our experiment indicated that embodied metaphors improve usability however, other factors including discoverability, perceivability of feedback and duplicity of structural isomorphism may mediate these metaphor-based benefits. We have generalized our main findings as a set of suggestions for the design of embodied style interfaces that rely on physical interaction.

Playing with the sound maker: do embodied metaphors help children learn?

In this paper we present the results of a comparative study that explores the potential benefits of using embodied interaction to help children, aged 7 to 10, learn abstract concepts related to musical sounds. Forty children learned to create musical sound sequences using an interactive sound making environment. Half the children used a version of the system that instantiated a body-based metaphor in the mapping layer connecting body movements to output sounds. The remaining children used a version of the same environment that did not instantiate a metaphor in the mapping layer. In general, children were able to more accurately demonstrate sound sequences in the embodied metaphor based system version. However, we observed that children often resorted to spatial rather than body-based metaphors and that the mapping must be easily discoverable as well as metaphorical to provide benefit.

Human-computer-intuition? Exploring the cognitive basis for intuition in embodied interaction

One of the claimed benefits of embodied interaction is that it is an intuitive form of human-computer interaction. While this claim seems to be widely accepted, few studies explore the underlying cognitive mechanisms of intuition in the context of tangible and embedded interaction design. What is intuitive interaction? What makes an interface intuitive to use? We explore these questions in the context of a responsive auditory environment. We propose that intuitive interaction can be facilitated by instantiating an embodied metaphor in the mapping layer between movement-based input actions and auditory system responses. We search for evidence of benefit through a comparative study of the same responsive auditory environment implemented with and without an embodied metaphor in the interactional mapping layer. Qualitative findings about the complexities and limitations of designing intuitive interaction are summarised and the implications for the design of embodied interaction discussed.

Kurio: a museum guide for families

We discuss three design strategies for improving the quality of social interaction and learning with interactive museum guides: 1) embodied interaction; 2) game-learning; 3) a hybrid system. We used these strategies in our prototype Kurio, which is aimed at supporting families visiting museums. The results of our evaluation show positive implications of implementing the design strategies: closing the social gap, naturalizing technology, and supporting exploration and discovery in learning.

Springboard: Designing Image Schema Based Embodied Interaction for an Abstract Domain

In this paper, we describe the theoretical framing, design, and user study of a whole body interactive environment called Springboard. Springboard supports users to explore the concept of balance in the abstract domain of social justice through embodied interaction. We present the foundational theory of embodied conceptual metaphor, focusing on the twin-pan balance schema, which can be enacted spatially or physically. We describe how these enactments of the balance schema and the conceptual metaphor of balance in social justice can be used to design the interaction model for a whole body interactive environment. We present the results of our qualitative interview style user study with 45 participants. The study was conceived to explore how participants enact, perceive, and understand spatial, physical, and conceptual balance through whole body interaction with an abstract domain such as social justice. We conclude with a discussion of implications for whole body interaction design.

Balancing justice: comparing whole body and controller-based interaction for an abstract domain

In this paper, we present a quantitative, comparative study of a multimedia environment about social justice that users can control using whole body interaction or a simple control device. We explore the efficacy of using embodied metaphor-based whole body interaction compared to controller-based interaction for an abstract domain (social justice). We describe how conceptual metaphor theory can be applied to the design of a whole body interaction model, focusing on the twin-pan balance image schema and its metaphorical elaboration that structures the concept of balance in social justice. We describe the Springboard system, our methodology and results from a study with 76 participants. Our results indicate that participants were able to interact with our system using both input approaches. However, participants in the whole body group were more deeply impacted by their experiences related to social justice than those in the control device group.

The Musical Experience through the Lens of Embodiment

ABSTRACT The author addresses the impression that digital media is diminishing the engagement of the body in our musical experience. Combining theories from the disciplines of philosophy and psychology, he constructs a framework for examining the experience of listening to music. A link between research in mirror neurons and the act of perception, as described by Merleau-Ponty, is used to demonstrate the role of embodiment in the listening experience. While acknowledging that hearing and viewing a musical performance do not provide the same musical experience, he aims to demonstrate how our embodied existence ensures the bodys engagement in any musical experience.

Springboard: exploring embodiment, balance and social justice

In this paper we describe the theory and design of a prototype interactive environment called Springboard. Springboard supports users to explore concepts in social justice through embodied interaction. We present the foundational theory of embodied conceptual metaphor, focusing on the twin-pan balance schema. We describe the application of balance metaphors in the design of the interaction model for our interactive environment. We conclude with a discussion of design choices and describe future research based on our prototype.

Experience Structuring Factors Affecting Learning in Family Visits to Museums

This paper describes the design and evaluation of an adaptive museum guide for families. In the Kurio system, a mixture of embedded and tangible technology imbues the museum space with additional support for learning and interaction, accessible via tangible user interfaces. Families engage in an educational game where family members are assigned individual challenges and their progress is monitored and coordinated by the family member with a PDA. After each round of challenges, the family returns to a tabletop display to review their progress. In this paper we present the overall evaluation result of Kurio and, using the model discovery approach, we determine which experience structuring factors have a substantial influence on the learning experience.

Ziklo: bicycle navigation through tactile feedback

Ziklo is a tactile interface for wayfinding devices designed for cyclists. It is made up of two wristbands that vibrate to signal left and right turns. These wristbands are wirelessly connected to the users mobile device via Bluetooth. An application on the mobile device hosts the wayfinding functionality and interface to control the wristbands. The wristbands consists of three vibration motors each, allowing for different vibration patterns and strengths to send different notifications. Ziklos goal is to create an alternative interface for wayfinding devices that does not hinder the users awareness when they are engaged in situations that demand their visual and/or auditory attention.