Michiel Demey

Assessing a clarinet player's performer gestures in relation to locally intended musical targets

Abstract Musicianship is known to display high-level skills, which involve different aspects of mental processing and corporeal control. Of particular interest is the match between the musicians mental focus on musical targets (the so-called musical intentions) and the expressive (or so-called auxiliary) body movements. To what extent are these related to each other? And what does this relationship reveal about mind–body connections? To approach these questions, a case study was set up around a clarinet solo performance played from score, covering a style of music unfamiliar to the player. The clarinetists movements were recorded with an optical movement tracking system. A statistical analysis method was developed, to account for movement data in relation to the potential musical intentions and targets. The bottom-up movement analysis method was validated with the performers annotations of targets in the musical score and the performers annotations of communicative/sound facilitating gestures in the performance video. The results reveal that the mental focus on musical targets is related to bodily expression. This finding supports the idea of an embodied model of musical syntax processing, which is strongly related to corporeal gestures.

Concepts, Technology, and Assessment of the Social Music Game "Sync-in-Team'

Music offers an excellent domain in which advanced forms of non-verbal communication can be explored. The first part of this paper introduces the research concepts behind the idea of a social interactive music game, which is based on the no-tions of ‘embodiment’ and ‘mediation technology’. The second part reviews the development of the ‘Sync-in-Team’ game, and its assessment in four different settings, including noisy eco-logical settings. The third part reviews the technological backbone of the game, and the fourth part discusses further developments. A user-oriented approach, based on concepts from embodied music cognition, may offer a valid contribution to the development of novel music-driven games that foster the sense for social interaction, body movement, collaboration, and competition.

Does Social Interaction Activate Music Listeners

Over the last years, embodiment has gained a lot of interest in the field of music research. Researchers began to focus on the study of body movements and gestures in relationship with music. In the study presented here, we empirically quantified the impact of social interaction on movements made by groups of subjects listening and moving to music. Both children (average age 9) and adolescents (average age 16) were tested. The methodology was based on motion capturing using wireless Wii Nintendo Remote sensors, and subsequent statistical analysis. Participants were asked to move along with the beat of the music in two conditions: Individual, without social contact, and in groups of four, encouraging social interaction. Data analysis shows that the influence of the social environment has an effect that can be measured and quantified. In general, the social context stimulates participants to move more intensively to the music. Furthermore, adolescent participants even show a significantly improved synchronization with the beat of the music in the social condition, illustrating social facilitation.

An embodied music cognition approach to multilevel interactive sonification

In this paper, a new conceptual framework and related implementation for interactive sonification is introduced. The conceptual framework consists of a combination of three components, namely, gestalt-based electroacoustic composition techniques (sound), user and body-centered spatial exploration (body), and corporeal mediation technology (tools), which are brought together within an existing paradigm of embodied music cognition. The implementation of the conceptual framework is based on an iterative process that involves the development of several use cases. Through this methodology, it is possible to investigate new approaches for structuring and to interactively explore multivariable data through sound.

DanSync: A Platform to Study Entrainment and Joint-Action during Spontaneous Dance in the Context of a Social Music Game

This paper presents a social music game, named DanSync, as a platform to study joint-action. This game context proves to be an effective manner to study spontaneous dance of players in a laboratory setting. Because of the gameplay participants are engaged in dancing to music with a strong motivation. Performance of dance synchronization to music is studied throughout the gameplay. Joint-action in a dyad is quantified in terms of correlation and phase-locking. Furthermore, entrainment and social bonding in small groups is studied by introducing perturbations in the music stimulus.

The Influence of an Audience on Performers: A Comparison Between Rehearsal and Concert Using Audio, Video and Movement Data

Abstract For many people, live concerts occupy an important position in their experience of music. The interaction between audience and performers creates a special tension, which also influences the performing musicians. This paper presents a method to study how the presence of an audience influences performers. To study this influence, a concert with a singer and a viola da gamba player was recorded using audio, video and acceleration sensors (invisibly) attached to wrists and back of the performers. These data were compared to the general rehearsal, recorded in identical settings. This enabled a scientifically valid comparison, without challenging ecological validity. General rehearsal and concert performances were relatively similar, which shows that performers are able to reproduce their interpretation. However, comparison between conditions revealed some interesting differences. Tempo analysis showed that the pieces in a slower, more flexible tempo were performed slower in concert, while the faster, more dance-like tempi were performed slightly faster. Gesture analysis suggested a tendency for the singer to use more open, communicative postures during the concert, to change posture more often and take more time in the transitions. The movement analysis showed an overall increase in intensity of the hand movements of the singer. In summary, we may conclude that the different methods of analysis demonstrated an intensification of the performance while interacting with the audience.

BilliARt - AR Carom Billiards

This paper presents a framework for processing visual and auditory textures in an augmented reality environment that enables real-time artistic creativity without imposing predefined interaction rules or constraints. It integrates multiple problem domain knowledge in sonification, real-time rendering, object tracking and object recognition in a collaborative art installation using a familiar Carom Billiard game table, motion tracking cameras, a table-top digital projector and a digital audio installation. A demonstrator was presented at a 10-day annual innovation exhibition in Belgium and was perceived as innovative, intuitive and very easy to interact with.

Synchronizing Music and Movement with BeatLED: an Interactive Musical Social Game

Abstract To study the rhythmical and social synchronization, also known as entrainment, that occurs when people move to music in a social context, novel applications that motivate and measure this behaviour are needed. In this paper, we describe the development of such an application, BeatLED, a social game that uses music, movement and luminescent textile. We analyse the requirements of such a musical social game, as well as the possible application fields, such as musicology research, team-building and health care. We developed novel game dynamics and integrated them into a proof-of-concept implementation, which was evaluated positively by 84% of the test users.