Nicola Montecchio

Adaptive Gesture Recognition with Variation Estimation for Interactive Systems

This article presents a gesture recognition/adaptation system for human--computer interaction applications that goes beyond activity classification and that, as a complement to gesture labeling, characterizes the movement execution. We describe a template-based recognition method that simultaneously aligns the input gesture to the templates using a Sequential Monte Carlo inference technique. Contrary to standard template-based methods based on dynamic programming, such as Dynamic Time Warping, the algorithm has an adaptation process that tracks gesture variation in real time. The method continuously updates, during execution of the gesture, the estimated parameters and recognition results, which offers key advantages for continuous human--machine interaction. The technique is evaluated in several different ways: Recognition and early recognition are evaluated on 2D onscreen pen gestures; adaptation is assessed on synthetic data; and both early recognition and adaptation are evaluated in a user study involving 3D free-space gestures. The method is robust to noise, and successfully adapts to parameter variation. Moreover, it performs recognition as well as or better than nonadapting offline template-based methods.

A unified approach to real time audio-to-score and audio-to-audio alignment using sequential Montecarlo inference techniques

We present a methodology for the real time alignment of music signals using sequential Montecarlo inference techniques. The alignment problem is formulated as the state tracking of a dynamical system, and differs from traditional Hidden Markov Model - Dynamic Time Warping based systems in that the hidden state is continuous rather than discrete. The major contribution of this paper is addressing both problems of audio-to-score and audio-to-audio alignment within the same framework in a real time setting. Performances of the proposed methodology on both problems are then evaluated and discussed.

Statistical Music Modeling Aimed at Identification and Alignment

This paper describes a methodology for the statistical modeling of music works. Starting from either the representation of the symbolic score or the audio recording of a performance, a hidden Markov model is built to represent the corresponding music work. The model can be used to identify unknown recordings and to align them with the corresponding score. Experimental evaluation using a collection of classical music recordings showed that this approach is effective in terms of both identification and alignment. The methodology can be exploited as the core component for a set of tools aimed at accessing and actively listening to a music collection.

Automatic Alignment of Music Performances with Scores Aimed at Educational Applications

We present a system for automatic real time alignment of an acoustic music performance with a digital representation of its score, a problem which is usually defined score following. The alignment is based on an application of hidden Markov models. A model is automatically built from a music score, while decoding is used to compute the most probable location of the performance along the score model. The effectiveness of the proposed approach has been tested with a collection of recordings of orchestral music. Even if the typical application of a score following system is automatic accompaniment, in this paper we propose a set of novel applications that are targeted also to non musicians, for educational use.

Content-Based Cover Song Identification in Music Digital Libraries

In this paper we report the status of our research on the problem of content-based cover song identification in music digital libraries. An approach which exploits both harmonic and rhythmic facets of music is presented and evaluated against a test collection. Directions for future work are proposed, and particular attention is given to the scalability challenge.

A scalable cover identification engine

This paper describes the implementation of a content-based cover song identification system which has been released under an open source license. The system is centered around the Apache Lucene text search engine library, and proves how classic techniques derived from textual Information Retrieval, in particular the bag-of-words paradigm, can successfully be adapted to music identification. The paper focuses on extensive experimentation on the most influential system parameters, in order to find an optimal tradeoff between retrieval accuracy and speed of querying.

FALCON: FAst Lucene-based Cover sOng identification

We present FALCON, an open-source engine for content-based cover song identification written in Java. The popular Lucene search engine library is used as the core of the software, proving that textual methods in information retrieval can be successfully adapted to multimedia tasks. An overview of the system methodology and of the implementation are provided, along with experimental results on a medium-size test collection