W. Bas de Haas

Functional modelling of musical harmony: an experience report

Music theory has been essential in composing and performing music for centuries. Within Western tonal music, from the early Baroque on to modern-day jazz and pop music, the function of chords within a chord sequence can be explained by harmony theory. Although Western tonal harmony theory is a thoroughly studied area, formalising this theory is a hard problem. We present a formalisation of the rules of tonal harmony as a Haskell (generalized) algebraic datatype. Given a sequence of chord labels, the harmonic function of a chord in its tonal context is automatically derived. For this, we use several advanced functional programming techniques, such as type-level computations, datatype-generic programming, and error-correcting parsers. As a detailed example, we show how our model can be used to improve content-based retrieval of jazz songs. We explain why Haskell is the perfect match for these tasks, and compare our implementation to an earlier solution in Java. We also point out shortcomings of the language and libraries that limit our work, and discuss future developments which may ameliorate our solution.

A functional approach to automatic melody harmonisation

Melody harmonisation is a centuries-old problem of long tradition, and a core aspect of composition in Western tonal music. In this work we describe FHarm, an automated system for melody harmonisation based on a functional model of harmony. Our system first generates multiple harmonically well-formed chord sequences for a given melody. From the generated sequences, the best one is chosen, by picking the one with the smallest deviation from the harmony model. Unlike all existing systems, FHarm guarantees that the generated chord sequences follow the basic rules of tonal harmony. We carry out two experiments to evaluate the quality of our harmonisations. In one experiment, a panel of harmony experts is asked to give its professional opinion and rate the generated chord sequences for selected melodies. In another experiment, we generate a chord sequence for a selected melody, and compare the result to the original harmonisation given by a harmony scholar. Our experiments confirm that FHarm generates realistic chords for each melody note. However, we also conclude that harmonising a melody with individually well-formed chord sequences from a harmony model does not guarantee a well-sounding coherence between the chords and the melody. We reflect on the experience gained with our experiment, and propose future improvements to refine the quality of the harmonisation.

Finding Repeated Patterns in Music: State of Knowledge, Challenges, Perspectives

This paper discusses the current state of knowledge on musical pattern finding. Various studies propose computational methods to find repeated musical patterns. Our detailed review of these studies reveals important challenges in musical pattern finding research: different methods have not yet been directly compared, and the influence of music representation and filtering on the results has not been assessed. Moreover, we need a thorough understanding of musical patterns as perceived by human listeners. A sound evaluation methodology is still lacking. Consequently, we suggest perspectives for musical pattern finding: future research can provide a comparison of different methods, and an assessment of different music representations and filtering criteria. A combination of quantitative and qualitative methods can overcome the lacking evaluation methodology. Musical patterns identified by human listeners form a reference, but also an object of study, as computational methods can help us understand the criteria underlying human notions of musical repetition.

Automatic functional harmonic analysis

Music scholars have been studying tonal harmony intensively for centuries, yielding numerous theories and models. Unfortunately, a large number of these theories are formulated in a rather informal fashion and lack mathematical precision. In this article we present HarmTrace, a functional model of Western tonal harmony that builds on well-known theories of tonal harmony. In contrast to other approaches that remain purely theoretical, we present an implemented system that is evaluated empirically. Given a sequence of symbolic chord labels, HarmTrace automatically derives the harmonic relations between chords. For this, we use advanced functional programming techniques that are uniquely available in the Haskell programming language. We show that our system is fast, easy to modify and maintain, robust against noisy data, and that its harmonic analyses comply with Western tonal harmony theory.

A geometrical distance measure for determining the similarity of musical harmony

In the last decade, digital repositories of music have undergone an enormous growth. Therefore, the availability of scalable and effective methods that provide content-based access to these repositories has become critically important. This study presents and tests a new geometric distance function that quantifies the harmonic distance between two pieces of music. Harmony is one of the most important aspects of music and we will show in this paper that harmonic similarity can significantly contribute to the retrieval of digital music. Yet, within the music information retrieval field, harmonic similarity measures have received far less attention compared to other similarity aspects. The distance function we present, the Tonal pitch step distance, is based on a cognitive model of tonality and captures the change of harmonic distance to the tonal center over time. This distance is compared to two other harmonic distance measures. We show that it can be efficiently used for retrieving similar chord sequences, and that it significantly outperforms a baseline string matching approach. Although the proposed method is not the best performing distance measure, it offers the best quality–runtime ratio. Furthermore, we demonstrate in a case study how our harmonic similarity measure can contribute to the musicological discussion of the melody and harmony in large-scale corpora.

Structural Segmentation of Music Based on Repeated Harmonies

In this paper we present a simple, yet powerful method for deriving the structural segmentation of a musical piece based on repetitions in chord sequences, called FORM. Repetition in harmony is a fundamental factor in constituting musical form. However, repeated pattern discovery in music still remains an open problem, and it has not been addressed before in chord sequences. FORM relies on a suffix tree based algorithm to find repeated patterns in symbolic chord sequences that are either provided by machine transcriptions or musical experts. This novel approach complements other segmentation methods, which generally use a self-distance matrix based on other musical features describing timbre, instrumentation, rhythm, or melody. We evaluate the segmentation quality of FORM on 649 popular songs, and show that FORM outperforms two baseline approaches. With FORM we explore new ways of exploiting musical repetition for structural segmentation, yielding a flexible and practical algorithm, and a better understanding of musical repetition.

Automatic chord label personalization through deep learning of shared harmonic interval profiles

Current automatic chord estimation systems are trained and tested using datasets that contain single reference annotations , i.e., for each corresponding musical segment (e.g., audio frame or section), the reference annotation contains a single chord label. Nevertheless, theoretical insights on harmonic ambiguity from harmony theory, experimental studies on annotator subjectivity in harmony annotations, and the availability of vast amounts of heterogeneous (subjective) harmony annotations in crowd-sourced repositories make the notion of a single-harmonic “ground truth” reference annotation a tenuous one. Recent studies suggest that subjectivity is intrinsic to harmonic reference annotations that should be embraced in automatic chord estimation rather than resolved. We introduce the first approach to automatic chord label personalization by modeling annotator subjectivity through harmonic interval-based chord representations. We integrate these representations from multiple annotators and deep learn them from audio. From a single trained model and the annotators’ chord-label vocabulary, we can accurately personalize chord labels for individual annotators. Furthermore, we show that chord personalization using multiple reference annotations outperforms using just a single reference annotation. Our results show that annotator subjectivity should inform future research on automatic chord estimation to improve the state of the art.