Mark Levy

Structural Segmentation of Musical Audio by Constrained Clustering

We describe a method of segmenting musical audio into structural sections based on a hierarchical labeling of spectral features. Frames of audio are first labeled as belonging to one of a number of discrete states using a hidden Markov model trained on the features. Histograms of neighboring frames are then clustered into segment-types representing distinct distributions of states, using a clustering algorithm in which temporal continuity is expressed as a set of constraints modeled by a hidden Markov random field. We give experimental results which show that in many cases the resulting segmentations correspond well to conventional notions of musical form. We show further how the constrained clustering approach can easily be extended to include prior musical knowledge, input from other machine approaches, or semi-supervision.

Music Information Retrieval Using Social Tags and Audio

In this paper we describe a novel approach to applying text-based information retrieval techniques to music collections. We represent tracks with a joint vocabulary consisting of both conventional words, drawn from social tags, and audio muswords, representing characteristics of automatically-identified regions of interest within the signal. We build vector space and latent aspect models indexing words and muswords for a collection of tracks, and show experimentally that retrieval with these models is extremely well-behaved. We find in particular that retrieval performance remains good for tracks by artists unseen by our models in training, and even if tags for their tracks are extremely sparse.

Extraction of High-Level Musical Structure From Audio Data and Its Application to Thumbnail Generation

A method for segmenting musical audio with a hierarchical timbre model is introduced. New evidence is presented to show that music segmentation can be recast as clustering of timbre features, and a new clustering algorithm is described. A prototype thumbnail-generating application is described and evaluated. Experimental results are given, including comparison of machine and human segmentations

Learning Latent Semantic Models for Music from Social Tags

Abstract In this paper we describe how to build a variety of information retrieval models for music collections based on social tags. We discuss the particular nature of social tags for music and apply latent semantic dimension reduction methods to co-occurrence counts of words in tags given to individual tracks. We evaluate the performance of various latent semantic models in relation to both previous work and a simple full-rank vector space model based on tags. We investigate the extent to which our low-dimensional semantic spaces respect traditional catalogue organization by artist and genre, and how well they generalize to unseen tracks, and we illustrate some of the concepts expressed by the learned dimensions.

Lightweight measures for timbral similarity of musical audio

Timbral similarity measures basedon Mel-Frequency Cepstral Coefficients have been widely reported as the basis for a possible general music similarity function, which would have wide application to searching, browsing and recommendation. Many of the reported methods, however, have computational requirements that make them impractical for searching realistic collections using current hardware. We compare lightweight measures that appear to perform equally well, and introduce a simplification that reduces memory requirements and execution time by a further order of magnitude. This yields a similarity measure that will scale easily to large commercial collections. We give comparative results over two contrasting music collections, one of which has been widely studied, allowing direct comparison with previous work.

A Comparison of Timbral and Harmonic Music Segmentation Algorithms

Four music segmentation algorithms are presented, one based on purely timbral features, one on purely harmonic features, and two on different combinations of features. They are compared against each other and against human annotations of two albums by The Beatles. Example segmentations are given together with a quantitative measure of boundary accuracy. No algorithm is found to be clearly superior, although examples suggest that the combined algorithms can offer improved boundary detection.

Signal-based Music Searching and Browsing

This paper describes an approach to the problem of finding songs in some sense similar to a query song. This is a problem of increasing importance, because consumers owning large digital music collections wish to navigate these and to add new songs by searching on-line. The technological approach is described, leading to the description of a simple demonstrator.