Tue Haste Andersen

Conducting Audio Files via Computer Vision

This paper presents a system to control the playback of audio files by means of the standard classical conducting technique. Computer vision techniques are developed to track a conductor’s baton, and the gesture is subsequently analysed. Audio parameters are extracted from the sound-file and are further processed for audio beat tracking. The sound-file playback speed is adjusted in order to bring the audio beat points into alignment with the gesture beat points. The complete system forms all parts necessary to simulate an orchestra reacting to a conductor’s baton.

Real-Time Beat EstimationUsing Feature Extraction

This paper presents a novel method for the estimation of beat interval from audio files. As a first step, a feature extracted from the waveform is used to identify note onsets. The estimated note onsets are used as input to a beat induction algorithm, where the most probable beat interval is found. Several enhancements over existing beat estimation systems are proposed in this work, including methods for identifying the optimum audio feature and a novel weighting system in the beat induction algorithm. The resulting system works in real-time, and is shown to work well for a wide variety of contemporary and popular rhythmic music. Several real-time music control systems have been made using the presented beat estimation method.

Feel the beat: direct manipulation of sound during playback

We present a tangible user interface for direct manipulation of sound during playback. The interface was inspired by observing DJs and musicians working with computers where looping of sound takes on an important role. Through exploration using hardware and software prototypes we have realized direct mapping of perceptually important sound parameters to a motorized slider, enabling users to monitor and manipulate sound during playback.

Beat estimation on the beat

The paper presents a novel method for the estimation of beat interval, and the exact location of the beats, from audio files. As a first step, a feature extracted from the waveform is used to identify note onsets. The estimated note onsets are used as input to a beat induction algorithm, where the most probable beat intervals are found. The note onsets corresponding to the beat locations are then identified. Several enhancements are proposed, including methods for identifying the optimum audio feature, a novel weighting system in the beat induction algorithm and a simple robust method for identifying the beat locations. The resulting system runs in real-time, and is shown to work well for a wide variety of contemporary and popular rhythmic music.

Searching for music: how feedback and input-control change the way we search

The growing amount of digital music available at desktop computers and portable media players increases the need for interfaces that facilitate efficient music navigation. Search patterns are quantified and evaluated across types of feedback and input controllers in an experiment with 12 participants. The way music is searched and the subjective factors varied significantly across input device and type of audio feedback. However, no difference in task completion time was found for the evaluated interfaces. Based on the experiments, we propose several ways in which future designs may improve searching and browsing in recorded music.