Rolf Inge Godøy

Gestural-Sonorous Objects: embodied extensions of Schaeffer's conceptual apparatus

One of the most remarkable achievements of Pierre Schaeffers musical thought is his proposal of the sonorous object as the focus of research. The sonorous object is a fragment of sound, typically in the range of a few seconds (often even less), perceived as a unit. Sonorous objects are constituted, studied, and evaluated according to various criteria, and sonorous objects that are found suitable are regarded as musical objects that may be used in musical composition. In the selection and qualification of these sonorous objects, we are encouraged to practise what Schaeffer called ‘reduced listening’, meaning disregarding the original context of the sound, including its source and signification, and instead focus our listening on the sonorous features.However, it can be argued that this principle of ‘reduced listening’ is not in conflict with more fundamental principles of embodied cognition, and that the criteria for the constitution, and the various feature qualifications, of sonorous objects can be linked to gestural images. Also, there are several similarities between studying sound and gestures from a phenomenological perspective, and it is suggested that Schaeffers theoretical concepts may be extended to what is called gestural-sonorous objects.

Playing “air instruments”: mimicry of sound-producing gestures by novices and experts

Both musicians and non-musicians can often be seen making sound-producing gestures in the air without touching any real instruments. Such “air playing” can be regarded as an expression of how people perceive and imagine music, and studying the relationships between these gestures and sound might contribute to our knowledge of how gestures help structure our experience of music.

Musical gestures: Concepts and methods in research

In the last decade, cognitive science underwent a change of paradigm by bringing human movement into the focus of research. Concepts such as ‘embodiment’ and ‘enactive’ have been proposed as core concepts reflecting the role of the human body in complex processes such as action and perception, and the interaction of mind and physical environment (Varela et al., 1991; Noë, 2004). In music research, human movement has often been related with the notion of gesture. The reason is that many musical activities (performance, conducting, dancing) involve body movements that evoke meanings, and therefore these movements are called gestures. In Camurri et al. (2005), musical gestures are addressed from the viewpoint of their expressive character. However, there are many ways in which music-related body movements can be approached, measured, described and applied. Accordingly, there are many ways in which musical gestures are meaningful. Given the different contexts in which gestures appear, and their close relationship to movement and meaning, one may be tempted to say that the notion of gesture is too broad, ill-defined and perhaps too vague. Yet the use of this notion is very convenient in modern music research, because it allows making a bridge between movement and meaning. A closer look at the term gesture reveals its potential as a core notion that provides access to central issues in action/perception processes and in mind/environment interactions.

Chunking in Music by Coarticulation

In our own and other research on music-related actions, findings suggest that perceived action and sound are broken down into a series of chunks in peoples minds when they perceive or imagine music. Chunks are here understood as holistically conceived and perceived fragments of action and sound, typically with durations in the 0.5 to 5 seconds range. There is also evidence suggesting the occurrence of coarticulation within these chunks, meaning the fusion of small-scale actions and sounds into more superordinate actions and sounds. Various aspects of chunking and coarticulation are discussed in view of their role in the production and perception of music, and it is suggested that coarticulation is an integral element of music and should be more extensively explored in the future.

Images of sonic objects

Based on innumerable informal accounts and a number of scientific studies, there can be no doubt that people often have quite vivid images of musical sound in their minds, and that this is the case regardless of levels of musical training. Various introspective accounts and more recent neurocognitive research seem to converge in suggesting that imagery for music is closely linked with imagery for music-related actions. In this paper, the consequences of sound–action links for our notions of the sonic image are discussed, with a particular focus on the relationship between sonic objects and action chunks. In conclusion, the exploitation of action imagery is seen as holding great promise in enhancing our means for musical imagery in various creative, research and educational contexts.

Gestural Imagery in the Service of Musical Imagery

There seem to be strong links between gestural imagery and musical imagery, and it is suggested that gestural imagery can be instrumental in triggering and sustaining mental images of musical sound. Gestural images are seen as integral to most experiences of music, and several practical and theoretical musical disciplines could profit from focusing on these gestural images. Research in support of this is reviewed, and some topics for future research are presented.

Knowledge in Music Theory by Shapes of Musical Objects and Sound-Producing Actions

Music theory must try to deal with emergent qualities, such as contour, texture, timbre and tone semantics, and this necessitates recognizing musical objects as holistic entities. Representations by shapes can be useful here, as shapes are inherently holistic. The idea of shapes is seen as applicable to several aspects and modalities at work in musical imagery, providing images at various levels of resolution of both the unfolding sounds and the sound-producing actions. The paradigm of shapes is seen as well supported by several contemporary domains of thought, but in need of extensive development as an alternative to more abstract approaches in music theory.

Analyzing correspondence between sound objects and body motion

Links between music and body motion can be studied through experiments called sound-tracing. One of the main challenges in such research is to develop robust analysis techniques that are able to deal with the multidimensional data that musical sound and body motion present. The article evaluates four different analysis methods applied to an experiment in which participants moved their hands following perceptual features of short sound objects. Motion capture data has been analyzed and correlated with a set of quantitative sound features using four different methods: (a) a pattern recognition classifier, (b) t-tests, (c) Spearmans ρ correlation, and (d) canonical correlation. This article shows how the analysis methods complement each other, and that applying several analysis techniques to the same data set can broaden the knowledge gained from the experiment.

Understanding Coarticulation in Musical Experience

The term coarticulation designates the fusion of small-scale events, such as single sounds and single sound-producing actions, into larger units of combined sound and body motion, resulting in qualitative new features at what we call the chunk timescale in music, typically in the 0.5.–5 s duration range. Coarticulation has been extensively studied in linguistics and to a certain extent in other domains of human body motion as well as in robotics, but so far not so much in music, so the main aim of this paper is to provide a background for how we can explore coarticulation in both the production and perception of music. The contention is that coarticulation in music should be understood as based on a number of physical, biomechanical and cognitive constraints, and that coarticulation is an essential factor in the shaping of several perceptually salient features of music.

A statistical approach to analyzing sound tracings

This paper presents an experiment on sound tracing, meaning an experiment on how people relate motion to sound. 38 participants were presented with 18 short sounds, and instructed to move their hands in the air while acting as though the sound was created by their hand motion. The hand motion of the participants was recorded, and has been analyzed using statistical tests, comparing results between different sounds, between different subjects, and between different sound classes. We have identified several relationships between sound and motion which are present in the majority of the subjects. A clear distinction was found in onset acceleration for motion to sounds with an impulsive dynamic envelope compared to non-impulsive sounds. Furthermore, vertical movement has been shown to be related to sound frequency, both in terms of spectral centroid and pitch. Moreover, a significantly higher amount of overall acceleration was observed for non-pitched sounds as compared to pitched sounds.