Milena Droumeva

What the body knows: Exploring the benefits of embodied metaphors in hybrid physical digital environments

A recent trend in ubiquitous computing is the development of new forms of interfaces, which rely on embodied interaction. We focus on the definition of embodiment that refers to the ways in which abstract concepts rely on metaphorical extensions of embodied schemata shaped by processes below the level of conscious awareness as explored by Lakoff and Johnson [Lakoff, G., Johnson, M., 1980. Metaphors We Live By. Chicago Press, Chicago, IL, USA]. Our inquiry focuses on understanding the role embodied metaphors may play in supporting people to understand the possibilities for physical interaction in augmented spaces. We explore this issue through the development and evaluation of an interactive audio environment. We instantiate metaphor theory by using embodied schemata as the basis for the interactional metaphor that relates full-body input actions to audio output responses. We demonstrate and explore the benefits of this approach through a comparative experiment in which adults and children learn to use our audio environment. The results from our experiment indicated that embodied metaphors improve usability however, other factors including discoverability, perceivability of feedback and duplicity of structural isomorphism may mediate these metaphor-based benefits. We have generalized our main findings as a set of suggestions for the design of embodied style interfaces that rely on physical interaction.

Playing with the sound maker: do embodied metaphors help children learn?

In this paper we present the results of a comparative study that explores the potential benefits of using embodied interaction to help children, aged 7 to 10, learn abstract concepts related to musical sounds. Forty children learned to create musical sound sequences using an interactive sound making environment. Half the children used a version of the system that instantiated a body-based metaphor in the mapping layer connecting body movements to output sounds. The remaining children used a version of the same environment that did not instantiate a metaphor in the mapping layer. In general, children were able to more accurately demonstrate sound sequences in the embodied metaphor based system version. However, we observed that children often resorted to spatial rather than body-based metaphors and that the mapping must be easily discoverable as well as metaphorical to provide benefit.

Hands on what?: comparing children's mouse-based and tangible-based interaction

We investigate the similarities and differences -- in terms of quantitative performance and qualitative behaviors -- between how children solve an object manipulation task using mouse-based input versus tangible-based input. This work examines the assumption common in tangible computing that direct physical manipulation is beneficial for certain spatial tasks. We describe an ecologically valid comparison of mouse-based versus tangible-based input for a jigsaw puzzle task in order to better understand the tradeoffs in choosing input and interaction styles. We include a traditional cardboard puzzle for comparative purposes. The results of an experiment with 132 children indicate children are more successful and faster at solving puzzles using a tangible-based approach. Detailed temporal analysis indicates that pairs in the tangible group spend most of their time using a combination of epistemic and pragmatic actions which support mental problem solving. Conversely, pairs in the mouse group use an ineffective trial and error strategy.

Human-computer-intuition? Exploring the cognitive basis for intuition in embodied interaction

One of the claimed benefits of embodied interaction is that it is an intuitive form of human-computer interaction. While this claim seems to be widely accepted, few studies explore the underlying cognitive mechanisms of intuition in the context of tangible and embedded interaction design. What is intuitive interaction? What makes an interface intuitive to use? We explore these questions in the context of a responsive auditory environment. We propose that intuitive interaction can be facilitated by instantiating an embodied metaphor in the mapping layer between movement-based input actions and auditory system responses. We search for evidence of benefit through a comparative study of the same responsive auditory environment implemented with and without an embodied metaphor in the interactional mapping layer. Qualitative findings about the complexities and limitations of designing intuitive interaction are summarised and the implications for the design of embodied interaction discussed.

Making sense of group interaction in an ambient intelligent environment for physical play

This paper presents the results of a study on group interaction with a prototype known as socio-ec(h)o. socio-ec(h)o explores the design of sensing and display, user modeling, and interaction in an embedded interaction system utilizing a game structure. Our study involved the playing of our prototype system by thirty-six (36) participants grouped into teams of four (4). Our aim was to determine heuristics that we could use to further design the interaction and user model approaches for group and embodied interaction systems. We analyzed group interaction and performance based on factors of team cohesion and goal focus. We found that with our system, these factors alone could not explain performance. However, when transitions in the degrees of each factor, i.e. high, medium or low are considered, a clearer picture for performance emerges. The significance of the results is that they describe recognizable factors for positive group interaction.

An ambient intelligence platform for physical play

This paper describes an ambient intelligent prototype known as socio-ec(h)o. socio-ec(h)o explores the design and implementation of a system for sensing and display, user modeling, and interaction models based on a game structure. The game structure includes, word puzzles, levels, body states, goals and game skills. Body states are body movements and positions that players must discover in order to complete a level and in turn represent a learned game skill. The paper provides an overview of background concepts and related research. We describe the prototype and game structure, provide a technical description of the prototype and discuss technical issues related to sensing, reasoning and display. The paper contributes by providing a method for constructing group parameters from individual parameters with real-time motion capture data; and a model for mapping the trajectory of participants actions in order to determine an intensity level used to manage the experience flow of the game and its representation in audio and visual display. We conclude with a discussion of known and outstanding technical issues, and future research.

Exploring ambient sound techniques in the design of responsive environments for children

This paper describes the theoretical framework, design, implementation and results from an exploratory informant workshop that examines an alternative approach to sound feedback in the design of responsive environments for children. This workshop offers preliminary directions and models for using intensity-based ambient sound display in the design of interactive learning environments for children that offer assistance in task-oriented activities. We see the value of this research in developing a more cohesive and ecological model for use of audio feedback in the design of embedded interactions for children. The approach presented here takes the design of multi-modal feedback beyond being experiential, to one that supports learning and problem solving.

Thinking with hands: an embodied approach to the analysis of children's interaction with computational objects

We present the theory and mixed methods approach for analyzing how childrens hands can help them think during interaction with computational objects. The approach was developed for a study investigating the benefits of different input methods for object manipulation activities in digitally supported problem solving. We propose a classification scheme based on the notions of complementary and epistemic actions in spatial problem solving. In order to overcome inequities in number of access points when comparing different input methods, we develop a series of relative measures based on our classification scheme.

Springboard: exploring embodiment, balance and social justice

In this paper we describe the theory and design of a prototype interactive environment called Springboard. Springboard supports users to explore concepts in social justice through embodied interaction. We present the foundational theory of embodied conceptual metaphor, focusing on the twin-pan balance schema. We describe the application of balance metaphors in the design of the interaction model for our interactive environment. We conclude with a discussion of design choices and describe future research based on our prototype.

Sound intensity gradients in an ambient intelligence audio display

This paper describes the prototype of a real-time responsive audio display for an ambient intelligent game named socio-ec(h)o. The audio display relies on a gradient response to represent and anticipate player action. We describe the audio display schema, and discuss results of our current experimentation in guiding player actions through types of audio feedback, for creating sound recognition, perceptions of change and sound intensity.