Tim Crawford

Polyphonic Score Retrieval Using Polyphonic Audio Queries: A Harmonic Modeling Approach

This paper extends the familiar “query by humming” music retrieval framework into the polyphonic realm. As humming in multiple voices is quite difficult, the task is more accurately described as “query by audio example,” onto a collection of scores. To our knowledge, we are the first to use polyphonic audio queries to retrieve from polyphonic symbolic collections. Furthermore, as our results will show, we will not only use an audio query to retrieve a known item symbolic piece, but we will use it to retrieve an entire set of real-world composed variations on that piece, also in the symbolic format. The harmonic modeling approach which forms the basis of this work is a new and valuable technique which has both wide applicability and future potential.

Investigating music collections at different scales with AudioDB

Abstract Content-based search of music collections presents differing challenges at different scales and according to the task at hand. In this paper, we consider a number of different use cases for content-based similarity search, at scales ranging between a detailed investigation of a single track to searching for fragments of a track against a collection of millions of media items. We pay particular attention to the varying tradeoff between precision and recall in these contexts, both from the point of view of system evaluation and from the point of view of a user of a system searching an unknown collection. We present the audioDB software for content-based search, and describe how it has been used to address use cases across these different collection sizes; in addition we show that the interpretation of similarity as a distance which can be modelled statistically, initially motivated by our desire to achieve sublinear retrieval time on large databases, can be used to improve the precision of searches over small and medium-sized collections.

Capturing the workflows of music information retrieval for repeatability and reuse

Many solutions for the reuse and re-purposing of Music Information Retrieval (MIR) methods, and the tools implementing those methods, have been introduced over recent years. Proposals for achieving interoperability between systems have ranged from shared software libraries and interfaces, through common frameworks and portals, to standardised file formats and metadata. Here we assess these solutions for their suitability to be reused and combined as repurposable components within assemblies (or workflows) that can be used in novel and possibly more ambitious ways. Reuse and repeatability also have great implications for the process of MIR research: the encapsulation of any algorithm and its operation—including inputs, parameters, and outputs—is fundamental to the repeatability and reproducibility of an experiment. This is desirable both for the open and reliable evaluation of algorithms and for the advancement of MIR by building more effectively upon prior research. At present there is no clear best practice widely adopted by the field. Based upon our analysis of contemporary systems and their adoption we reflect as to whether this should be considered a failure. Are there limits to interoperability unique to MIR, and how might they be overcome? Beyond workflows how much research context can, and should, be captured? We frame our assessment within the emerging notion of Research Objects for reproducible research in other domains, and describe how their adoption could serve as a route to reuse in MIR.

Explorations in linked data practice for early music corpora

Exploring connections between pieces, people and places and relating them to culture as a whole is a central activity of musicology. As libraries increase the availability of musical information in digital form, the data available for such research also expands, but to take such resources together and combine them with others that are relevant a further step of alignment and linkage is needed. We describe here the process and tools we applied to two corpora of early modern music: Early Music Online, which comprises catalogue metadata in MarcXML and facsimile images for approximately 8,500 items of early printed music; and the Electronic Corpus of Lute Music, containing over 1,000 pieces with supporting metadata. A supervised process with automated elements assists the musicologist to create a linked and extensible knowledge structure, aligning entities within and between corpora and to external Linked Data. Finally, we reflect upon how we believe these methods integrate with, and indeed form a crucial element of, the transformed process of modern digital scholarship.

An e-Research approach to Web-scale music analysis

The growing quantity of digital recorded music available in large-scale resources such as the Internet archive provides an important new resource for musical analysis. An e-Research approach has been adopted in order to create a very substantive web-accessible corpus of musical analyses in a common framework for use by music scholars, students and beyond, and to establish a methodology and tooling that will enable others to add to the resource in the future. The enabling infrastructure brings together scientific workflow and Semantic Web technologies with a set of algorithms and tools for extracting features from recorded music. It has been used to deliver a prototype system, described here, that demonstrates the utility of Linked Data for enhancing the curation of collections of music signal data for analysis and publishing results that can be simply and readily correlated to these and other sources. This paper describes the motivation, infrastructure design and the proof-of-concept case study and reflects on emerging e-Research practice as researchers embrace the scale of the Web.

Expert-guided semantic linking of music-library metadata for study and reuse

The process of aligning datasets that lack mutually-shared identifiers is fraught with ambiguity and difficult to automate. Manual performance of such a process may be time-consuming and error-prone. We present the Semantic Alignment and Linking Tool (SALT) that addresses this problem by applying semantic technologies and Linked Data approaches in order to produce candidate alignment suggestions that may be confirmed or disputed by a user with domain expertise. These decisions are integrated back into the knowledge base and are available for further iterative comparison by the user; the complete RDF graph is published and can be queried through the same SPARQL endpoint that also underlies the SALT user interface. Provenance of the musicologists judgement is captured and added to the descriptive graph, supporting further discourse and counter-proposals. We report on a use case and perform an evaluation of this tool within a musicological context, joining metadata from the British Library and other sources with programme data from BBC Radio 3 in a project focusing on early music.

Breathy, Resonant, Pressed – Automatic Detection of Phonation Mode from Audio Recordings of Singing

Abstract In this paper we present an experiment on automatic detection of phonation modes from recordings of sustained sung vowels. We created an open dataset specifically for this experiment, containing recordings of nine vowels from multiple languages, sung by a female singer on all pitches in her vocal range in phonation modes breathy, neutral, flow (resonant) and pressed. The dataset is available under a Creative Commons license at http://www.proutskova.de/phonation-modes. First, glottal flow waveform is estimated via inverse filtering (IAIF) from audio recordings. Then six parameters of the glottal flow waveform are calculated. A 4-class Support Vector Machine classifier is constructed to separate these features into phonation mode classes. We automated the IAIF approach by computing the values of the input arguments – lip radiation and formant count – leading to the best-performing SVM classifiers (average classification accuracy over 60%), yielding a physical model for the articulation of the vowels. We examine the steps needed to generalize and extend the experimental work presented in this paper in order to apply this method in ethnomusicological investigations.

Abstracting musical queries: towards a musicologist's workbench

In this paper, we propose a paradigm for computer-based music retrieval and analysis systems that employs one or more explicit abstraction layers between the user and corpus– and representation–specific tools. With illustrations drawn from “battle music”, a genre popular throughout Renaissance Europe, we show how such an approach may not only be more obviously useful to a user, but also offer extra power through the ability to generalise classes of tasks across collections.

On providing semantic alignment and unified access to music library metadata

A variety of digital data sources—including institutional and formal digital libraries, crowd-sourced community resources, and data feeds provided by media organisations such as the BBC—expose information of musicological interest, describing works, composers, performers, and wider historical and cultural contexts. Aggregated access across such datasets is desirable as these sources provide complementary information on shared real-world entities. Where datasets do not share identifiers, an alignment process is required, but this process is fraught with ambiguity and difficult to automate, whereas manual alignment may be time-consuming and error-prone. We address this problem through the application of a Linked Data model and framework to assist domain experts in this process. Candidate alignment suggestions are generated automatically based on textual and on contextual similarity. The latter is determined according to user-configurable weighted graph traversals. Match decisions confirming or disputing the candidate suggestions are obtained in conjunction with user insight and expertise. These decisions are integrated into the knowledge base, enabling further iterative alignment, and simplifying the creation of unified viewing interfaces. Provenance of the musicologist’s judgement is captured and published, supporting scholarly discourse and counter-proposals. We present our implementation and evaluation of this framework, conducting a user study with eight musicologists. We further demonstrate the value of our approach through a case study providing aligned access to catalogue metadata and digitised score images from the British Library and other sources, and broadcast data from the BBC Radio 3 Early Music Show.

Duplicate Detection in Facsimile Scans of Early Printed Music

There is a growing number of collections of readily available scanned musical documents, whether generated and managed by libraries, research projects, or volunteer efforts. They are typically digital images; for computational musicology we also need the musical data in machine-readable form. Optical Music Recognition (OMR) can be used on printed music, but is prone to error, depending on document condition and the quality of intermediate stages in the digitization process such as archival photographs. This work addresses the detection of one such error—duplication of images—and the discovery of other relationships between images in the process.