Ana Rebelo

Optical music recognition: state-of-the-art and open issues

For centuries, music has been shared and remembered by two traditions: aural transmission and in the form of written documents normally called musical scores. Many of these scores exist in the form of unpublished manuscripts and hence they are in danger of being lost through the normal ravages of time. To preserve the music some form of typesetting or, ideally, a computer system that can automatically decode the symbolic images and create new scores is required. Programs analogous to optical character recognition systems called optical music recognition (OMR) systems have been under intensive development for many years. However, the results to date are far from ideal. Each of the proposed methods emphasizes different properties and therefore makes it difficult to effectively evaluate its competitive advantages. This article provides an overview of the literature concerning the automatic analysis of images of printed and handwritten musical scores. For self-containment and for the benefit of the reader, an introduction to OMR processing systems precedes the literature overview. The following study presents a reference scheme for any researcher wanting to compare new OMR algorithms against well-known ones.

Optical recognition of music symbols: A comparative study

Many musical works produced in the past are still currently available only as original manuscripts or as photocopies. The preservation of these works requires their digitalization and transformation into a machine-readable format. However, and despite the many research activities on optical music recognition (OMR), the results for handwritten musical scores are far from ideal. Each of the proposed methods lays the emphasis on different properties and therefore makes it difficult to evaluate the efficiency of a proposed method. We present in this article a comparative study of several recognition algorithms of music symbols. After a review of the most common procedures used in this context, their respective performances are compared using both real and synthetic scores. The database of scores was augmented with replicas of the existing patterns, transformed according to an elastic deformation technique. Such transformations aim to introduce invariances in the prediction with respect to the known variability in the symbols, particularly relevant on handwritten works. The following study and the adopted databases can constitute a reference scheme for any researcher who wants to confront a new OMR algorithm face to well-known ones.

Staff Detection with Stable Paths

The preservation of musical works produced in the past requires their digitalization and transformation into a machine-readable format. The processing of handwritten musical scores by computers remains far from ideal. One of the fundamental stages to carry out this task is the staff line detection. We investigate a general-purpose, knowledge-free method for the automatic detection of music staff lines based on a stable path approach. Lines affected by curvature, discontinuities, and inclination are robustly detected. Experimental results show that the proposed technique consistently outperforms well-established algorithms.

A connected path approach for staff detection on a music score

The preservation of many music works produced in the past entails their digitalization and consequent accessibility in an easy-to-manage digital format. Carrying this task manually is very time consuming and error prone. While optical music recognition systems usually perform well on printed scores, the processing of handwritten musical scores by computers remain far from ideal. One of the fundamental stages to carry out this task is the staff line detection. In this paper a new method for the automatic detection of music staff lines based on a connected path approach is presented. Lines affected by curvature, discontinuities, and inclination are robustly detected. Experimental results show that the proposed technique consistently outperforms well-established algorithms.

Music score binarization based on domain knowledge

Image binarization is a common operation in the preprocessing stage in most Optical Music Recognition (OMR) systems. The choice of an appropriate binarization method for handwritten music scores is a difficult problem. Several works have already evaluated the performance of existing binarization processes in diverse applications. However, no goal-directed studies for music sheets documents were carried out. This paper presents a novel binarization method based in the content knowledge of the image. The method only needs the estimation of the staffline thickness and the vertical distance between two stafflines. This information is extracted directly from the gray level music score. The proposed binarization procedure is experimentally compared with several state of the art methods.

Robust Staffline Thickness and Distance Estimation in Binary and Gray-Level Music Scores

The optical recognition of handwritten musical scores by computers remains far from ideal. Most OMR algorithms rely on an estimation of the staff line thickness and the vertical line distance within the same staff. Subsequent operation can use these values as references, dismissing the need for some predetermined threshold values. In this work we improve on previous conventional estimates for these two reference lengths. We start by proposing a new method for binarized music scores and then extend the approach for gray-level music scores. An experimental study with 50 images is used to assess the interest of the novel method.

Staff Line Detection and Removal in the Grayscale Domain

The detection of staff lines is the first step of most Optical Music Recognition (OMR) systems. Its great significance derives from the ease with which we can then proceed with the extraction of musical symbols. All OMR tasks are usually achieved using binary images by setting thresholds that can be local or global. These techniques however, may remove relevant information of the music sheet and introduce artifacts which will degrade results in the later stages of the process. It arises therefore a need to create a method that reduces the loss of information due to the binarization. The baseline for the methodology proposed in this paper follows the shortest path algorithm proposed in [CardosoTPAMI08]. The concept of strong staff pixels (SSPs), which is a set of pixels with a high probability of belonging to a staff line, is proposed to guide the cost function. The SSP allows to overcome the results of the binary based detection and to generalize the binary framework to grayscale music scores. The proposed methodology achieves good results.

A new optical music recognition system based on combined neural network

This work is financed by Fund of Doctoral Program of the Ministry of Education (Approval No. 20110161110035) and National Natural Science Foundation of China (Approval No. 61174140, 61203016 and 61174050).

Global Constraints for Syntactic Consistency in OMR: An Ongoing Approach

Optical Music Recognition (OMR) systems are an indispensable tool to transform the paper-based music scores and manuscripts into a machine-readable symbolic format. A system like this potentiates search, retrieval and analysis. One of the problematic stages is the musical symbols detection where operations to localize and to isolate musical objects are developed. The complexity is caused by printing and digitalization, as well as the paper degradation over time. Distortions inherent in staff lines, broken, connected and overlapping symbols, differences in sizes and shapes, noise, and zones of high density of symbols is even worst when we are dealing with handwritten music scores. In this paper the exploration of an optimization approach to support semantic and syntactic consistency after the music symbols extraction phase is proposed. The inclusion of this ongoing technique can lead to better results and encourage further experiences in the field of handwritten music scores recognition.

A Fuzzy C-Means Algorithm for Fingerprint Segmentation

Fingerprint segmentation is a crucial step of an automatic fingerprint identification system, since an accurate segmentation promote both the elimination of spurious minutiae close to the foreground boundaries and the reduction of the computation time of the following steps. In this paper, a new, and more robust fingerprint segmentation algorithm is proposed. The main novelty is the introduction of a more robust binarization process in the framework, mainly based on the fuzzy C-means clustering algorithm. Experimental results demonstrate significant benchmark progress on three existing FVC datasets.