Kristoffer Jensen

Conducting Audio Files via Computer Vision

This paper presents a system to control the playback of audio files by means of the standard classical conducting technique. Computer vision techniques are developed to track a conductor’s baton, and the gesture is subsequently analysed. Audio parameters are extracted from the sound-file and are further processed for audio beat tracking. The sound-file playback speed is adjusted in order to bring the audio beat points into alignment with the gesture beat points. The complete system forms all parts necessary to simulate an orchestra reacting to a conductor’s baton.

Real-Time Beat EstimationUsing Feature Extraction

This paper presents a novel method for the estimation of beat interval from audio files. As a first step, a feature extracted from the waveform is used to identify note onsets. The estimated note onsets are used as input to a beat induction algorithm, where the most probable beat interval is found. Several enhancements over existing beat estimation systems are proposed in this work, including methods for identifying the optimum audio feature and a novel weighting system in the beat induction algorithm. The resulting system works in real-time, and is shown to work well for a wide variety of contemporary and popular rhythmic music. Several real-time music control systems have been made using the presented beat estimation method.

A hybrid resynthesis model for hammer-string interaction of piano tones

This paper presents a source/resonator model of hammer-string interaction that produces realistic piano sound. The source is generated using a subtractive signal model. Digital waveguides are used to simulate the propagation of waves in the resonator. This hybrid model allows resynthesis of the vibration measured on an experimental setup. In particular, the nonlinear behavior of the hammer-string interaction is taken into account in the source model and is well reproduced. The behavior of the model parameters (the resonant part and the excitation part) is studied with respect to the velocities and the notes played. This model exhibits physically and perceptually related parameters, allowing easy control of the sound produced. This research is an essential step in the design of a complete piano model.

Beat estimation on the beat

The paper presents a novel method for the estimation of beat interval, and the exact location of the beats, from audio files. As a first step, a feature extracted from the waveform is used to identify note onsets. The estimated note onsets are used as input to a beat induction algorithm, where the most probable beat intervals are found. The note onsets corresponding to the beat locations are then identified. Several enhancements are proposed, including methods for identifying the optimum audio feature, a novel weighting system in the beat induction algorithm and a simple robust method for identifying the beat locations. The resulting system runs in real-time, and is shown to work well for a wide variety of contemporary and popular rhythmic music.

Exploring Music Contents

This book constitutes the thoroughly refereed post-proceedings of the 7th International Symposium on Computer Music Modeling and Retrieval, CMMR 2010, held in Malaga, Spain, in June 2010. The 22 revised full papers presented were specially reviewed and revised for inclusion in this proceedings volume. The book is divided in five main chapters which reflect the present challenges within the field of computer music modeling and retrieval. The chapters range from music interaction, composition tools and sound source separation to data mining and music libraries. One chapter is also dedicated to perceptual and cognitive aspects that are currently subject to increased interest in the MIR community.

Atomic noise

Stochastic, unvoiced sounds are abundant in music and musical sounds. Without irregularities, the music and sounds become dull and lifeless. This paper presents work on unvoiced sounds that is believed to be useful in noise music. Several methods for obtaining a gradual change towards static white noise are presented. The random values (Dice), random events (Geiger) and random frequencies (Cymbal) noise types are shown to produce many useful sounds. Atomic noise encompasses all three noise types, while adding much more subtle variations and more life to the noise. Methods for obtaining a harmonic sound from the noise are introduced. These methods take advantage of the stochastic nature of the model, facilitating a gradual change from the stochastic sound to the noisy harmonic sound. In addition, the frozen noise repetitions are shown to produce unexpected pitch jumps with a potentially useful musical structure.

A causal rhythm grouping

This paper presents a method to identify segment boundaries in music. The method is based on a hierarchical model; first a features is measured from the audio, then a measure of rhythm is calculated from the feature (the rhythmogram), the diagonal of a self-similarity matrix is calculated from the rhythmogram, and finally the segment boundaries are found on a smoothed novelty measure, calculated from the diagonal of the self-similarity matrix. All the steps of the model have been accompanied with an informal evaluation, and the final system is tested on a variety of rhythmic songs with good results. The paper introduces a new feature that is shown to work significantly better than previously used features, a robust rhythm model and a robust, relatively cheap method to identify structure from the novelty measure.

Speech, Sound and Music Processing: Embracing Research in India

Important aspects of singing ability include musical accuracy and voice quality. In the context of Indian classical music, not only is the correct sequence of notes important to musical accuracy but also the nature of pitch transitions between notes. These transitions are essentially related to gamakas (ornaments) that are important to the aesthetics of the genre. Thus a higher level of singing skill involves achieving the necessary expressiveness via correct rendering of ornamentation, and this ability can serve to distinguish a welltrained singer from an amateur. We explore objective methods to assess the quality of ornamentation rendered by a singer with reference to a model rendition of the same song. Methods are proposed for the perceptually relevant comparison of complex pitch movements based on cognitively salient features of the pitch contour shape. The objective measurements are validated via their observed correlation with subjective ratings by human experts. Such an objective assessment system can serve as a useful feedback tool in the training of amateur singers.

Pitch independent prototyping of musical sounds

This paper presents a method of pitch-independent prototyping, classification and creation of musical sounds. Based on the spectral envelope, temporal envelope and irregularities of the quasi-harmonic partials, a simple timbre model is used for the prototyping, classification and synthesis of musical sounds. Indications are given how to modify the timbre parameters so as to have a natural sounding sound when modifying, for instance, the pitch. Furthermore, the sound can be perfected by modifying the individual parameters of each partial. Finally, several methods for searching sounds in a large database of sounds are proposed.

Music genre classification using an auditory memory model

Audio feature estimation is potentially improved by including the auditory short-term memory (STM) model. A new paradigm of audio feature estimation is obtained by adding the influence of notes in the STM. These notes are identified using the directional spectral flux, and the spectral content that is increased by the new note is added to the STM. The STM is exponentially fading with time span and number of elements, and each note only belongs to the STM for a limited time. Initial investigations regarding the behavior of the STM shows promising results, and an initial experiment with sensory dissonance has been undertaken with good results. The parameters obtained from the auditory memory model, along with the dissonance measure, are shown here to be of interest in music genre classification.