Matija Marolt

A connectionist approach to automatic transcription of polyphonic piano music

In this paper, we present a connectionist approach to automatic transcription of polyphonic piano music. We first compare the performance of several neural network models on the task of recognizing tones from time-frequency representation of a musical signal. We then propose a new partial tracking technique, based on a combination of an auditory model and adaptive oscillator networks. We show how synchronization of adaptive oscillators can be exploited to track partials in a musical signal. We also present an extension of our technique for tracking individual partials to a method for tracking groups of partials by joining adaptive oscillators into networks. We show that oscillator networks improve the accuracy of transcription with neural networks. We also provide a short overview of our entire transcription system and present its performance on transcriptions of several synthesized and real piano recordings. Results show that our approach represents a viable alternative to existing transcription systems.

A Mid-Level Representation for Melody-Based Retrieval in Audio Collections

Searching audio collections using high-level musical descriptors is a difficult problem, due to the lack of reliable methods for extracting melody, harmony, rhythm, and other such descriptors from unstructured audio signals. In this paper, we present a novel approach to melody-based retrieval in audio collections. Our approach supports audio, as well as symbolic queries and ranks results according to melodic similarity to the query. We introduce a beat-synchronous melodic representation consisting of salient melodic lines, which are extracted from the analyzed audio signal. We propose the use of a 2D shift-invariant transform to extract shift-invariant melodic fragments from the melodic representation and demonstrate how such fragments can be indexed and stored in a song database. An efficient search algorithm based on locality-sensitive hashing is used to perform retrieval according to similarity of melodic fragments. On the cover song detection task, good results are achieved for audio, as well as for symbolic queries, while fast retrieval performance makes the proposed system suitable for retrieval in large databases.

Transcription of polyphonic piano music with neural networks

This paper presents our experiences in building a system for transcription of polyphonic piano music. By transcription we mean the conversion of an audio recording of a polyphonic piano performance to a series of notes and their starting times. Our final goal is to build a transcription system that would transcribe polyphonic piano music over the entire piano range and with large polyphony. The system consists of three main stages. We first use a cochlear model based on the gammatone filterbank to transform an audio signal of a piano performance into time-frequency space. In the second stage we use a network of coupled adaptive oscillators to extract partial tracks from the output of the cochlear model and in the third stage we employ artificial neural networks acting as pattern recognisers to extract notes from the output of the oscillator network. The system uses several networks each trained to recognize the occurrence of a specific note in the input signal.

Networks of Adaptive Oscillators for Partial Tracking and Transcription of Music Recordings

In this paper, we present a technique for tracking partials in musical signals, based on networks of adaptive oscillators. We show how synchronization of adaptive oscillators can be utilized to detect periodic patterns in outputs of a human auditory model and thus track stable frequency components (partials) in musical signals. The model is further extended to track groups of harmonically related partials by grouping oscillators into networks. We present the integration of the partial tracking model into a system for transcription of polyphonic piano music. The transcription system is based on a connectionist architecture that employs networks of adaptive oscillators for partial tracking and feed forward neural networks for associating partial groups with notes. We provide a short overview of our entire transcription system and present its performance on transcriptions of several synthesized and real piano recordings.

On detecting note onsets in piano music

This paper presents an overview of our researches in the use of connectionist systems for transcription of polyphonic piano music and concentrates on the issue of onset detection in musical signals. We propose a new technique for detecting onsets in a piano performance. The technique is based on a combination of a bank of auditory filters, a network of integrate-and-fire neurons and a multilayer perceptron. Such a structure introduces several advantages over the standard peak-picking onset detection approach and we present its performance on several synthesized and real piano recordings. Results show that our approach represents a viable alternative to existing onset detection algorithms.

Audio Melody Extraction Based on Timbral Similarity of Melodic Fragments

The presented study deals with extraction of melodic line(s) from polyphonic audio recordings. Our approach is based on finding significant melodic fragments throughout the analyzed piece of music and clustering these fragments according to their timbral similarity. Fragments within clusters are taken to represent fragments belonging to different melodic lines. Holes between significant fragments within each cluster are filled-in by a shortest-path approach over all melodic fragments. The paper presents our study in more detail and provides results on real recordings

Educational hypermedia: an evaluation study

The role of computers in education has changed significantly in the last years. Educational systems have exceeded passive teaming systems and now actively participate in the teaming process. Web-based educational systems, for example, supported by incorporated intelligent tutoring techniques, are able to adapt information and its presentation to each individual user, and dynamically support navigation through the hypermedia material. This paper deals with the evaluation of educational hypermedia. An adaptive hypermedia system was developed and implemented for this purposes. The system is based on a concept domain model and also considers the elements of knowledge uncertainty in the process of user modelling. Both models are used for system adaptation, which builds on adaptive link insertion in addition to traditional adaptive navigation support technologies, like annotation and direct guidance. The system has been evaluated in a real environment and the results of the experiments are discussed here. In the paper, adaptive hypermedia systems in general are described first. Then our system is described, which was used in the evaluation study. In the end, the experiments are described and the results analysed in details.

Automatic Transcription of Bell Chiming Recordings

Bell chiming is a folk music tradition that involves performers playing rhythmic patterns on church bells. The paper presents a method for automatic transcription of bell chiming recordings, where the goal is to detect the bells that were played and their onset times. We first present an algorithm that estimates the number of bells in a recording and their approximate spectra. The algorithm uses a modified version of the intelligent k-means algorithm, as well as some prior knowledge of church bell acoustics to find clusters of partials with synchronous onsets in the time-frequency representation of a recording. Cluster centers are used to initialize non-negative matrix factorization that factorizes the time-frequency representation into a set of basis vectors (bell spectra) and their activations. To transcribe a recording, we propose a probabilistic framework that integrates factorization and onset detection data with prior knowledge of bell chiming performance rules. Both parts of the algorithm are evaluated on a set of bell chiming field recordings.

Ethnomuse: Archiving folk music and dance culture

The paper presents the development of EthnoMuse: multimedia digital library of Slovenian folk music and dance culture. The main scope of the project concerns the digitization of production and post-production processes that relate to collecting, documenting and archiving of folk heritage and development of multimedia applications for various content types (folk song, music and dance) and formats (image, audio, video, notation, MIDI etc.). The main objective of this paper is to discuss the former, focusing on the conceptual design of a flexible data model the archive is based on. We also briefly describe the tools developed to support the workflow of researchers involved in collecting and archiving of folk music related contents.

The EthnoMuse digital library: conceptual representation and annotation of ethnomusicological materials

The paper presents two vital aspects of the EthnoMuse digital library. We first present the development of a flexible data model through FRBRoo and CIDOC CRM conceptualization of processes and relations in folk song and music realizations. The approach is novel in that it conceptualizes and integrates various folkloristic and ethnomusicological materials, and also standardizes the workflow of production and post-production processes related to recording and documenting of folk song and music. We also present how novel music information retrieval techniques were integrated into the library to provide support for annotation of its contents. Two case studies are presented: automatic segmentation and labeling of field recordings, and transcription of bell chiming recordings.