Olivier Lartillot

A Matlab Toolbox for Music Information Retrieval

We present MIRToolbox, an integrated set of functions written in Matlab, dedicated to the extraction from audio files of musical features related, among others, to timbre, tonality, rhythm or form. The objective is to offer a state of the art of computational approaches in the area of Music Information Retrieval (MIR). The design is based on a modular framework: the different algorithms are decomposed into stages, formalized using a minimal set of elementary mechanisms, and integrating different variants proposed by alternative approaches — including new strategies we have developed —, that users can select and parametrize. These functions can adapt to a large area of objects as input.

Using machine-learning methods for musical style modeling

The ability to construct a musical theory from examples presents a great intellectual challenge that, if successfully met, could foster a range of new creative applications. Inspired by this challenge, we sought to apply machine-learning methods to the problem of musical style modeling. Our work so far has produced examples of musical generation and applications to a computer-aided composition system. Machine learning consists of deriving a mathematical model, such as a set of stochastic rules, from a set of musical examples. The act of musical composition involves a highly structured mental process. Although it is complex and difficult to formalize, it is clearly far from being a random activity. Our research seeks to capture some of the regularity apparent in the composition process by using statistical and information theoretic tools to analyze musical pieces. The resulting models can be used for inference and prediction and, to a certain extent, to generate new works that imitate the style of the great masters.

Generalizability and Simplicity as Criteria in Feature Selection: Application to Mood Classification in Music

Classification of musical audio signals according to expressed mood or emotion has evident applications to content-based music retrieval in large databases. Wrapper selection is a dimension reduction method that has been proposed for improving classification performance. However, the technique is prone to lead to overfitting of the training data, which decreases the generalizability of the obtained results. We claim that previous attempts to apply wrapper selection in the field of music information retrieval (MIR) have led to disputable conclusions about the used methods due to inadequate analysis frameworks, indicative of overfitting, and biased results. This paper presents a framework based on cross-indexing for obtaining realistic performance estimate of wrapper selection by taking into account the simplicity and generalizability of the classification models. The framework is applied on sets of film soundtrack excerpts that are consensually associated with particular basic emotions, comparing Naive Bayes, k-NN, and SVM classifiers using both forward selection (FS) and backward elimination (BE). K-NN with BE yields the most promising results - 56.5% accuracy with only four features. The most useful feature subset for k-NN contains mode majorness and key clarity, combined with dynamical, rhythmical, and structural features.

A Musical Pattern Discovery System Founded on a Modeling of Listening Strategies

Music is a domain of expression that conveys a paramount degree of complexity. The musical surface, composed of a multitude of notes, results from the elaboration of numerous structures of different types and sizes. The composer constructs this structural complexity in a more or less explicit way. The listener, faced by such a complex phenomenon, is able to reconstruct only a limited part of it, mostly in a non-explicit way. One particular aim of music analysis is to objectify such complexity, thus offering to the listener a tool for enriching the appreciation of music (Lartillot and SaintJames, 2004). The trouble is, traditional musical analysis, although offering a valuable understanding of musical style, does not go into the deepest details of this complexity. Some approaches of 20th-century musicology, such as the thematic analysis by Rudolph Reti (1951), were aimed at a better awareness of complexity. However, their scope was still restricted to a particular aspect of musical structure. For instance, Reti’s approach was founded on the hypothesis that a musical work is built on a single motive. And even within such limited scope, the search cannot be undertaken exhaustively, owing to the unreachable combinatory structure of musical works. Even worse, the results of such analyses do not meet a consensus agreement (Cook 1987), which questions the relevance of the underlying methods. Nicolas Ruwet formalized motivic analysis as a set of partially detailed operations that carry out a top–down hierarchical segmentation of the musical work (Ruwet 1987). However, he never actually followed his model when applying it to concrete examples, but rather he relied implicitly on his own intuitions. In fact, a careful application of this method easily leads to absurd results that invalidate the model (Lartillot 2004). All this points to the necessity of a computational modeling of the discovery processes themselves. Indeed, a computer implementation of the modeling can explicitly show its potential capacities and pertinence and can be validated—or invalidated—according to its operational efficiency. Moreover, non-computational modeling implicitly tends to limit the complexity of algorithms and data structures for practical reasons. For instance, the grouping structure proposed by Lerdahl and Jackendoff (1983), which can successfully be implemented on a computer (Temperley 2001), relies however on the foundations of simple preference rules and the idea of a unique hierarchical system, which, as will be explained below, limits a detailed understanding of musical structure. The necessary simplifications that are required by the non-computer modeling question the use of modeling itself, according to Reti:

Multi-Dimensional motivic pattern extraction founded on adaptive redundancy filtering

Abstract We present a computational model for discovering repeated patterns in symbolic representations of monodic music. Patterns are discovered through an incremental adaptive identification along a multi-dimensional parametric space. The difficulties of pattern discovery mainly come from combinatorial redundancies, that our model is able to control efficiently. A specificity relation is defined between pattern descriptions, unifying suffix and inclusion relations and enabling a filtering of redundant descriptions. Combinatorial proliferation caused by successive repetitions of patterns is managed using cyclic patterns. The modelling of these redundancy control mechanisms enables an automation of musicology-relevant analyses of musical databases.

Motivic matching strategies for automated pattern extraction

This article proposes an approach to the problem of automated extraction of motivic patterns in monodies. Different musical dimensions, restricted in current approaches to the most prominent melodic and rhythmic features at the surface level, are defined. The proposed strategy of detection of repeated patterns consists of an exact matching of the successive parameters forming the motives. We suggest a generalization of the multiple-viewpoint approach that allows a variability of the types of parameters (melodic, rhythmic, etc.) defining each successive extension of these motives. This enables us to take into account a more general class of motives, called heterogeneous motives, which includes interesting motives beyond the scope of previous approaches. Besides, this heterogeneous representation of motives may offer more refined explanations concerning the impact of gross contour representation in motivic analysis. This article also shows that the main problem aroused by the pattern extraction task is related to the control of the combinatorial redundancy of musical structures. Two main strategies are presented, that ensure an adaptive filtering of the redundant structures, and which are based on the notions of closed and cyclic patterns. The method is illustrated with the analysis of two pieces: a medieval Geisslerlied and a Bach Invention.

Generalized Musical Pattern Discovery by Analogy from Local Viewpoints

Musical knowledge discovery, an important issue of digital network processing, is also a crucial question for music. Indeed, music may be considered as a kind of network. A new approach for Musical Pattern Discovery is proposed, which tries to consider musical discourse in a general polyphonic framework. We suggest a new vision of automated pattern analysis that generalizes the multiple viewpoint approach. Sharing the idea that pattern emerges from repetition, analogy-based modeling of music understanding adds the idea of a permanent induction of global hypotheses from local perception. Through a chronological scanning of the score, analogies are inferred between local relationships - namely, notes and intervals - and global structures - namely, patterns - whose paradigms are stored inside an abstract pattern trie. Basic mechanisms for inference of new patterns are described. Such an elastic vision of music enables a generalized understanding of its plastic expression.

Taxonomic categorisation of motivic patterns

The issue of pattern description in computational models for motivic analysis is closely related to the cognitive debate on categorisation, in which are traditionally opposed “well-defined” and “ill-defined” categorisations. The ill-defined conceptualisation has been considered as a suitable framework for the formalisation of musical categorisation as it takes into account motivic variations. It seems that computational models rely rather on well-defined categorisation, due to its better controllability. The computational model we previously presented (Lartillot & Toiviainen, 2007) strikes a balance by developing a new flexible framework allowing the taking into account of unrestricted variability, but in the same time ensuring a precise description of whole categories, including their underlying variations. For that purpose, categories are not described by one single prototype, but on the contrary by a taxonomy of subcategories forming a multi-levelled hierarchy. The proposed computational model forms a complex system of interdependencies: its behaviour cannot be predicted but can only be observed through a computational running on actual musical examples. The behaviour emerging from the model offers hence a possible explanation of listeners’ cognitive capabilities and might indicate necessary conditions for cognitively relevant modelling.

Modeling musical attributes to characterize ensemble recordings using rhythmic audio features

In this paper, we present the results of a pre-study on music performance analysis of ensemble music. Our aim is to implement a music classification system for the description of live recordings, for instance to help musicologist and musicians to analyze improvised ensemble performances. The main problem we deal with is the extraction of a suitable set of audio features from the recorded instrument tracks. Our approach is to extract rhythm-related audio features and to apply them for regression-based modeling of eight more general musical attributes. The model based on Partial Least-Squares Regression without preceding Principal Component Analysis performed best for all of the eight attributes.

Reflections towards a Generative Theory of Musical Parallelism

Parallelism plays a core role in Lerdahl and Jackendoffs (1983) GTTM, as it rules the emergence of motivic, metrical, grouping and even formal structures. Due to the high amount of detail and complexity characterising associational structures, neither explicit model nor systematic methodology of parallelism-based structural inference has been included into the GTTM. This paper develops a methodological and computational answer to this problem founded on a computational modelling of pattern extraction operations. The paper focuses in particular on the methodological interest of the pattern mining formalism, and in particular its application to the formalisation of grouping and metrical structure inference and to the understanding of the possible interdependences, between these two levels of representation, but also more generally between parallelism and phenomenal accents. Concerning grouping structures, an attenuation of the strong hierarchy postulate is suggested, opening the framework to the study of structural polysemy and polyphony. A generalisation of the pattern mining formalism indicates a bottom-up understanding of musical variation and ornamentation, through the construction of a syntagmatic network whose connectivity is constrained by basic tonal principles.