Joren Six

Synchronizing multimodal recordings using audio-to-audio alignment

Research on the interaction between movement and music often involves analysis of multi-track audio, video streams and sensor data. To facilitate such research a framework is presented here that allows synchronization of multimodal data. A low cost approach is proposed to synchronize streams by embedding ambient audio into each data-stream. This effectively reduces the synchronization problem to audio-to-audio alignment. As a part of the framework a robust, computationally efficient audio-to-audio alignment algorithm is presented for reliable synchronization of embedded audio streams of varying quality. The algorithm uses audio fingerprinting techniques to measure offsets. It also identifies drift and dropped samples, which makes it possible to find a synchronization solution under such circumstances as well. The framework is evaluated with synthetic signals and a case study, showing millisecond accurate synchronization.

MIRchiving: Challenges and opportunities of connecting MIR research and digital music archives

This study is a call for action for the music information retrieval (MIR) community to pay more attention to collaboration with digital music archives. The study, which resulted from an interdisciplinary workshop and subsequent discussion, matches the demand for MIR technologies from various archives with what is already supplied by the MIR community. We conclude that the expressed demands can only be served sustainably through closer collaborations. Whereas MIR systems are described in scientific publications, usable implementations are often absent. If there is a runnable system, user documentation is often sparse---posing a huge hurdle for archivists to employ it. This study sheds light on the current limitations and opportunities of MIR research in the context of music archives by means of examples, and highlights available tools. As a basic guideline for collaboration, we propose to interpret MIR research as part of a value chain. We identify the following benefits of collaboration between MIR researchers and music archives: new perspectives for content access in archives, more diverse evaluation data and methods, and a more application-oriented MIR research workflow.

Adopting a music-to-heart rate alignment strategy to measure the impact of music and its tempo on human heart rate

Music is frequently used as a means of relaxation. Conversely, it is used as a means of arousal in sports and exercise contexts. Previous research suggests that tempo is one of the most significant determinants of music-related arousal and relaxation effects. Here we investigate the specific effect of music tempo, but also more generally, the influence of music on human heart rate. We took the pulses of 32 participants in silence, and then we played them non-vocal, ambient music at a tempo corresponding to their heart rates. Finally, we played the same music again, either with the tempo increased or decreased by a factor of 45%, 30%, or 15%; or maintaining the same tempo as in the first playing. Mixed-design ANOVA tests revealed a significant increase in heart rate while listening to the music as compared with silence (p < .05). In addition, substantial decreases in tempo (-45% or -30%) could account for smaller subsequent heart rate reductions (p < .05). We neither found links between increases in tempo (+15%, +30%, and +45%) and heart rate change, nor small decreases (-15%). In addition, neither effects of gender, music training, nor of musical preference were found. This indicates that during passive music listening, music exerts a general arousal effect on human heart rate, which might be regulated by tempo. These results are a major contribution to the way in which music may be used in everyday activities.

Embodied, Participatory Sense-Making in Digitally-Augmented Music Practices: Theoretical Principles and the Artistic Case “SoundBikes”

ABSTRACT Electronic and digital technologies open immense opportunities for music composition, listening, interaction, and participation. However, at the same time, they critically challenge some of the most basic principles that drive human engagement and interaction with music. This article first presents a theoretical discussion of two of these principles, namely sensorimotor control and participatory sense-making. Thereafter, it presents SoundBikes, a music installation that implements these theoretical considerations. SoundBikes is rooted in the idea that collective music-making is a form of participatory sense-making that emerges from embodied, dynamical and collaborative interactions between co-performers. The core components of SoundBikes include an EMS Synthi 100 and two stationary bikes equipped with sensors. To stimulate social interaction and collaboration between cyclist-performers, we designed SoundBikes in a way that performers could exert control over expressive features in the playback of music compositions, by coordinating their (cycling) movements with one another. This functionality is integrated in a gameplay—to further stimulate social collaboration and competition—and a visually attractive environment—to provide visual feedback and to create ambiance.

Beyond documentation – The digital philology of interaction heritage

ABSTRACT A philologists approach to heritage is traditionally based on the curation of documents, such as text, audio and video. However, with the advent of interactive multimedia, heritage becomes floating and volatile, and not easily captured in documents. We propose an approach to heritage that goes beyond documents. We consider the crucial role of institutes for interactive multimedia (as motor of a living culture of interaction) and propose that the digital philologists task will be to promote the collective/shared responsibility of (interactive) documenting, engage engineering in developing interactive approaches to heritage, and keep interaction-heritage alive through the education of citizens.

Applications of Duplicate Detection in Music Archives: From Metadata Comparison to Storage Optimisation

This work focuses on applications of duplicate detection for managing digital music archives. It aims to make this mature music information retrieval (MIR) technology better known to archivists and provide clear suggestions on how this technology can be used in practice. More specifically applications are discussed to complement meta-data, to link or merge digital music archives, to improve listening experiences and to re-use segmentation data. To illustrate the effectiveness of the technology a case study is explored. The case study identifies duplicates in the archive of the Royal Museum for Central Africa, which mainly contains field recordings of Central Africa. Duplicate detection is done with an existing Open Source acoustic fingerprinter system. In the set, 2.5% of the recordings are duplicates. It is found that meta-data differs dramatically between original and duplicate showing that merging meta-data could improve the quality of descriptions. The case study also shows that duplicates can be identified even if recording speed is not the same for original and duplicate.

Applications of duplicate detection: linking meta-data and merging music archives: The experience of the IPEM historical archive of electronic music

This work focuses on applications of duplicate detection for managing digital music archives. It aims to make this mature music information retrieval (MIR) technology better known to archivists and provide clear suggestions on how this technology can be used in practice. More specifically applications are discussed to complement meta-data, to link or merge digital music archives, to improve listening experiences and to re-use segmentation data. The IPEM archive, a digitized music archive containing early electronic music, provides a case study.