Jordan B. L. Smith

Audio Properties of Perceived Boundaries in Music

Data mining tasks such as music indexing, information retrieval, and similarity search, require an understanding of how listeners process music internally. Many algorithms for automatically analyzing the structure of recorded music assume that a large change in one or another musical feature suggests a section boundary. However, this assumption has not been tested: while our understanding of how listeners segment melodies has advanced greatly in the past decades, little is known about how this process works with more complex, full-textured pieces of music, or how stable this process is across genres. Knowing how these factors affect how boundaries are perceived will help researchers to judge the viability of algorithmic approaches with different corpora of music. We present a statistical analysis of a large corpus of recordings whose formal structure was annotated by expert listeners. We find that the acoustic properties of boundaries in these recordings corroborate findings of previous perceptual experiments. Nearly all boundaries correspond to peaks in novelty functions, which measure the rate of change of a musical feature at a particular time scale. Moreover, most of these boundaries match peaks in novelty for several features at several time scales. We observe that the boundary-novelty relationship can vary with listener, time scale, genre, and musical feature. Finally, we show that a boundary profile derived from a collection of novelty functions correlates with the estimated salience of boundaries indicated by listeners.

Emergent formal structures of factor oracle-driven musical improvisations

In this article, improvisations created with the factor oracle, a commonly used data structure in machine models of musical improvisation, are shown to exhibit certain formal structures independent of the musical content. We posit that these structures are in fact emergent properties of the behavior of the factor oracle itself. An expert improviser (the first author) performed a series of improvisations with Mimi, a factor oracle-driven multimodal system for human-machine improvisation, and the formal structures of each performance was independently analyzed by the performer and an experienced music structure annotator (the second author). Quantitative assessment of the similarity between the performers and the listeners analyses was carried out using techniques from the field of automatic structure analysis. Supported by a comparison to baseline analysis approaches, the results suggest a high level of agreement between the two sets of analyses. Drawing upon this foundation of evidence, we discuss these analyses and their relationship to common classical forms, including canon- and rondo-like forms, as well as forms based on the juxtaposition of rhythmic cells.