Marcelo M. Wanderley

The Importance of Parameter Mapping in Electronic Instrument Design

This paper presents a review of a series of experiments which have contributed towards the understanding of the mapping layer in electronic instruments. It challenges the assumption that an electronic instrument consists solely of an interface and a sound generator. It emphasises the importance of the mapping between input parameters and sound parameters, and suggests that this can define the very essence of an instrument. The terms involved with mapping are defined, and existing literature reviewed and summarised. A model for understanding the design of such mapping strategies for electronic instruments is put forward, along with a roadmap of ongoing research focussing on the testing and evaluation of such mapping strategies.

Evaluation of Input Devices for Musical Expression: Borrowing Tools from HCI

This paper reviews the existing literature on input device evaluation and design in human-computer interaction (HCI) and discusses possible applications of this knowledge to the design and evaluation of new interfaces for musical expression. Specifically, a set of musical tasks is suggested to allow the evaluation of different existing controllers.

Mapping performer parameters to synthesis engines

This paper considers the issues involved in the design of electronic and computer interfaces, specifically mapping - the designed link between an instruments playing interface and its sound source. It defines the problem area, reviews the literature, and gives examples of specific system mappings. A general model is presented, with the aim of providing a framework for future discussions on what makes an effective mapping. Several guidelines for mapping strategies are given, based on existing work.

Action-based effects on music perception

The classical, disembodied approach to music cognition conceptualizes action and perception as separate, peripheral processes. In contrast, embodied accounts of music cognition emphasize the central role of the close coupling of action and perception. It is a commonly established fact that perception spurs action tendencies. We present a theoretical framework that captures the ways in which the human motor system and its actions can reciprocally influence the perception of music. The cornerstone of this framework is the common coding theory, postulating a representational overlap in the brain between the planning, the execution, and the perception of movement. The integration of action and perception in so-called internal models is explained as a result of associative learning processes. Characteristic of internal models is that they allow intended or perceived sensory states to be transferred into corresponding motor commands (inverse modeling), and vice versa, to predict the sensory outcomes of planned actions (forward modeling). Embodied accounts typically refer to inverse modeling to explain action effects on music perception (Leman, 2007). We extend this account by pinpointing forward modeling as an alternative mechanism by which action can modulate perception. We provide an extensive overview of recent empirical evidence in support of this idea. Additionally, we demonstrate that motor dysfunctions can cause perceptual disabilities, supporting the main idea of the paper that the human motor system plays a functional role in auditory perception. The finding that music perception is shaped by the human motor system and its actions suggests that the musical mind is highly embodied. However, we advocate for a more radical approach to embodied (music) cognition in the sense that it needs to be considered as a dynamical process, in which aspects of action, perception, introspection, and social interaction are of crucial importance.

Quantitative Analysis of Non-obvious Performer Gestures

This article presents preliminary quantitative results from movement analysis of several clarinet performers with respect to non-obvious or ancillary gestures produced while playing a piece. The comparison of various performances of a piece by the same clarinetist shows a high consistency of movement patterns. Different clarinetists show different overall patterns, although clear similarities may be found, suggesting the existence of various levels of information in the resulting movement. The relationship of these non-obvious gestures to material/physiological, structural and interpretative parameters is highlighted.

Musical gestures: Concepts and methods in research

In the last decade, cognitive science underwent a change of paradigm by bringing human movement into the focus of research. Concepts such as ‘embodiment’ and ‘enactive’ have been proposed as core concepts reflecting the role of the human body in complex processes such as action and perception, and the interaction of mind and physical environment (Varela et al., 1991; Noë, 2004). In music research, human movement has often been related with the notion of gesture. The reason is that many musical activities (performance, conducting, dancing) involve body movements that evoke meanings, and therefore these movements are called gestures. In Camurri et al. (2005), musical gestures are addressed from the viewpoint of their expressive character. However, there are many ways in which music-related body movements can be approached, measured, described and applied. Accordingly, there are many ways in which musical gestures are meaningful. Given the different contexts in which gestures appear, and their close relationship to movement and meaning, one may be tempted to say that the notion of gesture is too broad, ill-defined and perhaps too vague. Yet the use of this notion is very convenient in modern music research, because it allows making a bridge between movement and meaning. A closer look at the term gesture reveals its potential as a core notion that provides access to central issues in action/perception processes and in mind/environment interactions.

The importance of parameter mapping in electronic instrument design

In this paper we challenge the assumption that an electronic instrument consists solely of an interface and a sound generator. We emphasise the importance of the mapping between input parameters and system parameters, and claim that this can define the very essence of an instrument.

Non-obvious Performer Gestures in Instrumental Music

This paper deals with the gestural language of instrumentalists playing wind instruments. It discusses the role of nonobvious performer gestures that may nevertheless influence the final sound produced by the acoustic instrument. These gestures have not commonly been considered in sound synthesis, although they are an integral part of the instrumentalists full gestural language. The structure of this paper will be based on an analysis of these non-obvious gestures followed by some comments on how to best classify them according to existing research on gesture reviewed in the introduction; finally, the influence of these gestures on the sound produced by the instrument will be studied and measurement and simulation results presented.

ESCHER-modeling and performing composed instruments in real-time

This article presents ESCHER, a sound synthesis environment based on Ircams real-time audio environment jMax. ESCHER is a modular system providing synthesis-independent prototyping of gesturally-controlled instruments by means of parameter interpolation. The system divides into two components: gestural controller and synthesis engine. Mapping between components takes place on two independent levels, coupled by an intermediate abstract parameter layer. This separation allows a flexible choice of controllers and/or sound synthesis methods.

Performance Gestures of Musicians: What Structural and Emotional Information Do They Convey?

This paper investigates how expressive gestures of a professional clarinetist contribute to the perception of structure and affect in musical performance. The thirty musically trained subjects saw, heard, or both saw and heard the performance. All subjects made the same judgments including a real-time judgment of phrasing, which targeted the experience of structure, and a real-time judgment of tension, which targeted emotional experience. In addition to standard statistical methods, techniques in the field of Functional Data Analysis were used to interpret the data. These new techniques model data drawn from continuous processes and explore the hidden structures of the data as they change over time. Three main findings add to our knowledge of gesture and movement in music: 1) The visual component carries much of the same structural information as the audio. 2) Gestures elongate the sense of phrasing during a pause in the sound and certain gestures cue the beginning of a new phrase. 3) The importance of visual information to the experience of tension changes with certain structural features in the sound. When loudness, pitch height, and note density are relatively low, the effect of removing the visual component is to decrease the experience of tension.