Kristian Nymoen

Chunking in Music by Coarticulation

In our own and other research on music-related actions, findings suggest that perceived action and sound are broken down into a series of chunks in peoples minds when they perceive or imagine music. Chunks are here understood as holistically conceived and perceived fragments of action and sound, typically with durations in the 0.5 to 5 seconds range. There is also evidence suggesting the occurrence of coarticulation within these chunks, meaning the fusion of small-scale actions and sounds into more superordinate actions and sounds. Various aspects of chunking and coarticulation are discussed in view of their role in the production and perception of music, and it is suggested that coarticulation is an integral element of music and should be more extensively explored in the future.

Analyzing sound tracings: a multimodal approach to music information retrieval

This paper investigates differences in the gestures people relate to pitched and non-pitched sounds respectively. An experiment has been carried out where participants were asked to move a rod in the air, pretending that moving it would create the sound they heard. By applying and interpreting the results from Canonical Correlation Analysis we are able to determine both simple and more complex correspondences between features of motion and features of sound in our data set. Particularly, the presence of a distinct pitch seems to influence how people relate gesture to sound. This identification of salient relationships between sounds and gestures contributes as a multi-modal approach to music information retrieval.

Analyzing correspondence between sound objects and body motion

Links between music and body motion can be studied through experiments called sound-tracing. One of the main challenges in such research is to develop robust analysis techniques that are able to deal with the multidimensional data that musical sound and body motion present. The article evaluates four different analysis methods applied to an experiment in which participants moved their hands following perceptual features of short sound objects. Motion capture data has been analyzed and correlated with a set of quantitative sound features using four different methods: (a) a pattern recognition classifier, (b) t-tests, (c) Spearmans ρ correlation, and (d) canonical correlation. This article shows how the analysis methods complement each other, and that applying several analysis techniques to the same data set can broaden the knowledge gained from the experiment.

Survey on synchronization mechanisms in machine-to-machine systems

People have always tried to understand natural phenomena. In computer science natural phenomena are mostly used as a source of inspiration for solving various problems in distributed systems such as optimization, clustering, and data processing. In this paper we will give an overview of research in field of computer science where fireflies in nature are used as role models for time synchronization. We will compare two models of oscillators that explain firefly synchronization along with other phenomena of synchrony in nature (e.g., synchronization of pacemaker cells of the heart and synchronization of neuron networks of the circadian pacemaker). Afterwards, we will present Mirollo and Strogatzs pulse coupled oscillator model together with its limitations. As discussed by the authors of the model, this model lacks of explanation what happens when oscillators are nonidentical. It also does not support mobile and faulty oscillators. Finally, it does not take into consideration that in communication among oscillators there are communication delays. Since these limitations prevent Mirollo and Strogatzs model to be used in real-world environments (such as Machine-to-Machine systems), we will sum up related work in which scholars investigated how to modify the model in order for it to be applicable in distributed systems. However, one has to bear in mind that there are usually large differences between mathematical models in theory and their implementation in practice. Therefore, we give an overview of both mathematical models and mechanisms in distributed systems that were designed after them.

A statistical approach to analyzing sound tracings

This paper presents an experiment on sound tracing, meaning an experiment on how people relate motion to sound. 38 participants were presented with 18 short sounds, and instructed to move their hands in the air while acting as though the sound was created by their hand motion. The hand motion of the participants was recorded, and has been analyzed using statistical tests, comparing results between different sounds, between different subjects, and between different sound classes. We have identified several relationships between sound and motion which are present in the majority of the subjects. A clear distinction was found in onset acceleration for motion to sounds with an impulsive dynamic envelope compared to non-impulsive sounds. Furthermore, vertical movement has been shown to be related to sound frequency, both in terms of spectral centroid and pitch. Moreover, a significantly higher amount of overall acceleration was observed for non-pitched sounds as compared to pitched sounds.

Exploring Sound-Motion Similarity in Musical Experience

People tend to perceive many and also salient similarities between musical sound and body motion in musical experience, as can be seen in countless situations of music performance or listening to music, and as has been documented by a number of studies in the past couple of decades. The so-called motor theory of perception has claimed that these similarity relationships are deeply rooted in human cognitive faculties, and that people perceive and make sense of what they hear by mentally simulating the body motion thought to be involved in the making of sound. In this paper, we survey some basic theories of sound-motion similarity in music, and in particular the motor theory perspective. We also present findings regarding sound-motion similarity in musical performance, in dance, in so-called sound-tracing (the spontaneous body motions people produce in tandem with musical sound), and in sonification, all in view of providing a broad basis for understanding sound-motion similarity in music.

Evaluating Input Devices for Dance Research

Recording music-related motions in ecologically valid situations can be challenging. We investigate the performance of three devices providing 3D acceleration data, namely Axivity AX3, iPhone 4s and a Wii controller tracking rhythmic motions. The devices are benchmarked against an infrared motion capture system, tested on both simple and complex music-related body motions, and evaluations are presented of the data quality and suitability for tracking music-related motions in real-world situations. The various systems represent different trade-offs with respect to data quality, user interface and physical attributes.

The Challenge of Decentralised Synchronisation in Interactive Music Systems

Synchronization is an important part of collaborative music systems, and with such systems implemented on mobile devices, the implementation of algorithms for synchronization without central control becomes increasingly important. Decentralised synchronization has been researched in many areas, and some challenges are solved. However, some of the assumptions that are often made in this research are not suitable for mobile musical systems. We present an implementation of a firefly-inspired algorithm for synchronization of musical agents with fixed and equal tempo, and lay out the road ahead towards synchronization between agents with large differences in tempo. The effect of introducing human-controlled nodes in the network of otherwise agent-controlled nodes is examined.

Self-awareness in Active Music Systems

Self-aware and self-expressive technologies may be used to improve user experience in interactive music systems. This chapter presents how the concepts and techniques from the first parts of the book may be exploited to develop better technologies for active music. Nature-inspired and socially-inspired methods, as introduced in Chapter 7, are used to allow music listeners to influence high-level parameters of the music, such as mood or tempo, without requiring the skill of a professional musician. Several of the examples presented in this chapter utilise the same algorithms as presented for the multi-camera networks in Chapter 13, thus demonstrating the broad application domain of these algorithms. The chapter is organised as a discussion of three example systems. First, a mechanism for conflict resolution in a distributed active music system is presented. The second and third example presented take inspiration from the pheromone mechanism used in Ant Colony Optimisation. The approach is first used for continuous classification of the movement patterns of listeners to incorporate adaptive mapping between sensor data and musical output. In the third example the same mechanism enables a system to remember the preferences of a user when navigating in a musical space.

Common Techniques for Self-awareness and Self-expression

Chapter 5 has provided step-by-step guidelines on how to design selfaware and self-expressive systems, including several architectural patterns with different levels of self-awareness. Chapter 6 has explained important features in self-aware and self-expressive systems, including adaptivity, robustness, multiobjectivity and decentralisation. To allow such self-aware capabilities in each design pattern and enable those system features, this chapter introduces the common techniques that have been used and can be used in self-aware (SA) and selfexpressive (SE) systems, including online learning, nature-inspired learning and socially-inspired learning in collective systems. Online learning allows learning in real time and thus has great flexibility and adaptivity. Nature-inspired learning provides tools to optimise SA/SE systems that can be used to reduce system complexity and costs. Socially-inspired learning is inspired by common social behaviours to facilitate learning, particularly in multi-agent systems that are commonly seen in SA/SE systems. How these techniques contribute to SA/SE systems is explained through several case studies. Their potentials and limitations are widely discussed at different self-awareness levels.