Kai Tuuri

Formulating a Revised Taxonomy for Modes of Listening

Abstract Listening to sounds or music is not a homogeneous act of grasping meanings by hearing. Yet it is often portrayed as such, especially when the intentional stance of a listener is overlooked. This paper distinguishes listening as the action-oriented intentional activity of making sense of the world. It is proposed that the multifaceted and heterogeneous nature of ‘understanding by listening’ can be outlined in terms of distinct modes of listening. Building upon previous accounts, a revised taxonomy of nine listening modes (reflexive, kinaesthetic, connotative, causal, empathetic, functional, semantic, reduced and critical listening) is proposed and illustrated by examples. Modes refer to different constituents of meaning-creation in the process of listening. In the taxonomy, they are schematically arranged into three levels (experiential, denotative and reflective). The theoretical framework of this revised taxonomy utilizes an embodied cognition paradigm. The experiential basis of meaning in listening is theoretically conceived of as emerging resonances between experiential patterns of sensations, structured patterns of recurrent sensorimotor experiences (action–sound couplings) and the projection of action-relevant mental images. The proposed taxonomy of listening modes is discussed in terms of its implications for perception and cognition research on sounds and music.

Beyond clicks and beeps: in pursuit of an effective sound design methodology

Designing effective non-speech audio elements for a user-interface is a challenging task due to the complex nature of sounds and the changing contexts of non-visual interfaces. In this paper we present a design method, which is intended to take into account the complexity of audio design as well as the existing audio environment and the functional context of use. Central to this method is a rich use scenario, presented in the form of a radio play, which is used as a basis for the work of design panels. A previous version of the design method is analysed and specific practical issues are identified. Solutions to these issues are presented in the form of a modified version of the method. In the current version of the method, special attention has been paid to the development of a rich use scenario and the underlying personage. A case study is presented to illustrate the practical implementation of the modified design method and to support the proposed guidelines for its use.

Leaping across modalities: speed regulation messages in audio and tactile domains

This study examines three design bases for speed regulation messages by testing their ability to function across modalities. Two of the design bases utilise a method originally intended for sound design and the third uses a method meant for tactile feedback. According to the experimental results, all designs communicate the intended meanings similarly in audio and tactile domains. It was also found that melodic (frequency changes) and rhythmic (segmentation) features of stimuli function differently for each type of message.

Gestural attributions as semantics in user interface sound design

This paper proposes a gesture-based approach to user interface sound design, which utilises projections of body movements in sounds as meaningful attributions. The approach is founded on embodied conceptualisation of human cognition and it is justified through a literature review on the subject of interpersonal action understanding. According to the resulting hypothesis, stereotypical gestural cues, which correlate with, e.g., a certain communicative intention, represent specific non-linguistic meanings. Based on this theoretical framework, a model of a process is also outlined where stereotypical gestural cues are implemented in sound design.

In Search for an Integrated Design Basis for Audio and Haptics

Audio and haptics as interaction modalities share properties, which make them highly appropriate to be handled within a single conceptual framework. This paper outlines such framework, gaining ingredients from the literature concerning cross-modal integration and embodied cognition. The resulting framework is bound up with a concept of physical embodiment, which has been introduced within several scientific disciplines to reveal the role of bodily experience and the corresponding mental imagery as the core of meaning-creation. In addition to theoretical discussion, the contribution of the proposed approach in design is outlined.

Who Controls Who? Embodied Control Within Human–Technology Choreographies

In this paper, we explore issues of embodied control that relate to current and future technologies in which body movements function as an instrument of control. Instead of just seeing ourselves in control, it is time to consider how these technologies actually control our moving bodies and transform our lived spaces. By shifting the focus from devices to choreographies among devices, we perform a theoretical analysis of the multidimensional aspects that reside within embodied interaction with technology. We suggest that it is beneficial to acknowledge and reformulate the phenomena of embodied control that go beyond the instrumental user-todevice control scheme. Drawing upon the phenomenology of the body, ecological psychology and embodied cognitive science, we identify three different dimensions of embodied control: instrumental, experiential and infrastructural. Design implications of this theoretical model are also discussed.

Calm down - exploiting sensorimotor entrainment in breathing regulation application

Various phenomena in human life are related to different kinds of rhythms. Not only are our bodily functions based on rhythms, but also much of the interaction with our environment is related to them. In this study, we explore breathing regulation and how it could be supported with an interactive application. The application is based on the concept of entrainment, in which two interacting entities adjust to a common rhythm. The focus is in the design of interaction elements which support entrainment process. A user study of a prototype application is also reported in the paper. The results indicate that the approach is promising and has potential in opening new perspectives to humancomputer interaction.

Design and evaluation of prosody-based non-speech audio feedback for physical training application

Abstract Methodological support for the design of non-speech user interface sounds for human–computer interaction is still fairly scarce. To meet this challenge, this paper presents a sound design case which, as a practical design solution for a wrist-computer physical training application, outlines a prosody-based method for designing non-speech user interface sounds. The principles used in the design are based on nonverbal communicative functions of prosody in speech acts, exemplifying an interpersonal approach to sonic interaction design. The stages of the design process are justified with a theoretical analysis and three empirical sub-studies, which comprise production and recognition tasks involving four communicative functions. The final evaluation study indicates that the resulting sounds of the design process successfully served these functions. In all, this study suggests that prosody-based sound design provides widely applicable means to attribute meaningful, interaction-derived qualities to non-speech sounds for interactive applications.

Human-technology choreographies: re-thinking body, movement and space in interaction design

Bodily movements have traditionally had mostly instrumental value in interaction design. However, movements can also be given a central role in understanding behaviour and in designing technology for humans. This workshop is aiming at taking a fresh, movement-oriented look at the design and evaluation of technology in a wide variety of contexts.

Imagining between ourselves: a group interview approach in exploring listening experiences

In this paper listening experiences are explored through verbal interactions in a group interview situation. By focusing on (1) ordinary and (2) evocative modes of dealing with listening experiences, the present study investigated how imagination mediates between pre-reflective and reflective consciousness and how imagining is shared between persons. Analysis revealed that participants utilised both modes in their discussions for either music or everyday sound samples, although the ordinary mode was the dominant one. The evocative mode was utilised relatively more frequently with the music samples. Group dynamics had an effect on how the mental images and meanings were described and shared within the group interviews. With regard to the sharing of listening experiences, three degrees of group agreement were unveiled.