Bart Moens

Encouraging Spontaneous Synchronisation with D-Jogger, an Adaptive Music Player That Aligns Movement and Music

In this study we explore how music can entrain human walkers to synchronise to the musical beat without being instructed to do so. For this, we use an interactive music player, called D-Jogger, that senses the users walking tempo and phase. D-Jogger aligns the music by manipulating the timing difference between beats and footfalls. Experiments are reported that led to the development and optimisation of four alignment strategies. The first strategy matched the musics tempo continuously to the runners pace. The second strategy matched the musics tempo at the beginning of a song to the runners pace, keeping the tempo constant for the remainder of the song. The third alignment starts a song in perfect phase synchrony and continues to adjust the tempo to match the runners pace. The fourth and last strategy additionally adjusts the phase of the music so each beat matches a footfall. The first two strategies resulted in a minor increase of steps in phase synchrony with the main beat when compared to a random playlist, the last two strategies resulted in a strong increase in synchronised steps. These results may be explained in terms of phase-error correction mechanisms and motor prediction schemes. Finding the phase-lock is difficult due to fluctuations in the interaction, whereas strategies that automatically align the phase between movement and music solve the problem of finding the phase-locking. Moreover, the data show that once the phase-lock is found, alignment can be easily maintained, suggesting that less entrainment effort is needed to keep the phase-lock, than to find the phase-lock. The different alignment strategies of D-Jogger can be applied in different domains such as sports, physical rehabilitation and assistive technologies for movement performance.

Interacting with the Music Paint Machine: Relating the constructs of flow experience and presence

In this paper we report on the results of an experiment on the experience of flow and presence while engaging with an interactive music system, the Music Paint Machine. This music system provides a game-like environment in which a musician can create a digital painting by playing an acoustic musical instrument, by moving the body in different directions, and by selecting colours using a pressure mat. The experiment aimed at getting a better insight into the possible relationship between flow experience and presence. Based on the definition of flow as a combination of the highest level of presence (presence-as-feeling) and a positive emotional state (Riva et al., 2004a), we hypothesized that presence has a predictive value for flow. Sixty-five musicians, both amateur and professional, participated in the experiment. Flow experience was measured with the Flow State Scale (Jackson and Eklund, 2004). Presence was measured with an in-house designed presence questionnaire. Results showed a significantly strong correlation between flow and presence. Moreover, the scores for presence significantly predicted the Flow State Scale, and explained a significant proportion of variance in the Flow State Scale. Furthermore, many significant associations were found between flow and presence variables, among which the most significant were the strong correlation (Spearmans rank) between the naturalness of using the system and the Flow State Scale and between the feeling of non-mediation and the Flow State Scale.

The Music Paint Machine: Stimulating Self-monitoring Through the Generation of Creative Visual Output Using a Technology-enhanced Learning Tool

Abstract In this paper, we discuss the pedagogically grounded and research-based design of a technology-enhanced learning tool, the Music Paint Machine. This interactive music system introduces a musical experience in which the musician creates a digital painting by playing an acoustic musical instrument and by moving the body on a coloured pressure mat. As a learning tool it aims at the development of musical creativity, at the stimulation of embodied understanding of music and at the development of an intimate relationship with the musical instrument. First, the methodological approach is outlined and pedagogical and theoretical backgrounds are discussed. Then, we report on an experiment in which 51 amateur musicians participated. The experiment aimed at probing the applications potential to induce a flow experience and to learn about how participants evaluate the didactic relevance of the Music Paint Machine. Results suggest that the Music Paint Machine has the potential to evoke a flow experience. Furthermore participants acknowledged its didactic relevance with regard to learning to improvise, to developing understanding of musical parameters and to stimulating creativity.

Alignment strategies for the entrainment of music and movement rhythms

Theories of entrainment assume that spontaneous entrainment emerges from dynamic laws that operate via mediators on interactions, whereby entrainment is facilitated if certain conditions are fulfilled. In this study, we show that mediators can be built that affect the entrainment of human locomotion to music. More specifically, we built D‐Jogger, a music player that functions as a mediator between music and locomotion rhythms. The D‐Jogger makes it possible to manipulate the timing differences between salient moments of the rhythms (beats and footfalls) through the manipulation of the musical period and phase, which affect the condition in which entrainment functions. We conducted several experiments to explore different strategies for manipulating the entrainment of locomotion and music. The results of these experiments showed that spontaneous entrainment can be manipulated, thereby suggesting different strategies on how to embark. The findings furthermore suggest a distinction among different modalities of entrainment: finding the beat (the most difficult part of entrainment), keeping the beat (easier, as a temporal scheme has been established), and being in phase (no entrainment is needed because the music is always adapted to the human rhythm). This study points to a new avenue of research on entrainment and opens new perspectives for the neuroscience of music.

Sound-induced stabilization of breathing and moving.

In humans and other animals, the locomotor and respiratory systems are coupled together through mechanical, neurophysiological, and informational interactions. At a macroscopic observer–environment level, these three types of interactions produce locomotor–respiratory coupling (LRC), whose dynamics are evaluated in this paper. A formal analysis of LRC is presented, exploiting tools from synchronization theories and nonlinear dynamics. The results of two recent studies, in which participants were instructed to cycle or exhale at a natural frequency or in synchrony with an external rhythmic sound, are discussed. The metronome was either absent or present (study 1) and close to or far from the natural frequency of the cycling and breathing systems (study 2). The results evidenced a stabilization of cycling, breathing, and LRC when sound was present compared to when it was absent. A decrease in oxygen consumption was also observed, accompanying the increase in sound‐induced LRC stabilization. These results obtained with a simple rhythmic metronome beat have consequences for exercising while listening to music; the consequences are further explored here.

Spontaneous Velocity Effect of Musical Expression on Self-Paced Walking

The expressive features of music can influence the velocity of walking. So far, studies used instructed (and intended) synchronization. But is this velocity effect still present with non-instructed (spontaneous) synchronization? To figure that out, participants were instructed to walk in their own comfort tempo on an indoor track, first in silence and then with tempo-matched music. We compared velocities of silence and music conditions. The results show that some music has an activating influence, increasing velocity and motivation, while other music has a relaxing influence, decreasing velocity and motivation. The influence of musical expression on the velocity of self-paced walking can be predicted with a regression model using only three sonic features explaining 56% of the variance. Phase-coherence between footfall and beat did not contribute to the velocity effect, due to its implied fixed pacing. The findings suggest that the velocity effect depends on vigor entrainment that influences both stride length and pacing. Our findings are relevant for preventing injuries, for gait improvement in walking rehabilitation, and for improving performance in sports activities.

Gamified music improvisation with BilliArT: a multimodal installation with balls

This paper presents the concept and the realisation of an interactive multimedia installation, called BilliArT, together with an explorative user study conducted on the data gathered during a public exhibition of the installation. The study concerns functional properties of the installation (e.g. usability, design quality) and subjective qualities of the sonic and visual feedback of the installation. The installation consists in a collaborative environment based on the carambole billiards game, which allows the users to engage in a user-driven machine-based jazz-inspired music improvisation, augmented with visual feedback. The installation is designed to promote the interaction among the users and the billiard game, stimulating the motivation to engage in the game by balancing predictable and unpredictable output, and reinforcing the feeling of reward, irrespective of their level of musical training. BilliArT introduces a new framework for expressive interaction related to the concepts of motivation and reward. The exploratory study proved the ability of the installation to activate the users’ sense of aesthetic reward, leading to a more active and satisfactory engagement in the game. Future studies may exploit these results to the advantage of the world of the arts, as well as of studies in human-computer interaction, interface design, and cultural heritage preservation.

Embodied, Participatory Sense-Making in Digitally-Augmented Music Practices: Theoretical Principles and the Artistic Case “SoundBikes”

ABSTRACT Electronic and digital technologies open immense opportunities for music composition, listening, interaction, and participation. However, at the same time, they critically challenge some of the most basic principles that drive human engagement and interaction with music. This article first presents a theoretical discussion of two of these principles, namely sensorimotor control and participatory sense-making. Thereafter, it presents SoundBikes, a music installation that implements these theoretical considerations. SoundBikes is rooted in the idea that collective music-making is a form of participatory sense-making that emerges from embodied, dynamical and collaborative interactions between co-performers. The core components of SoundBikes include an EMS Synthi 100 and two stationary bikes equipped with sensors. To stimulate social interaction and collaboration between cyclist-performers, we designed SoundBikes in a way that performers could exert control over expressive features in the playback of music compositions, by coordinating their (cycling) movements with one another. This functionality is integrated in a gameplay—to further stimulate social collaboration and competition—and a visually attractive environment—to provide visual feedback and to create ambiance.