Sankalp Gulati

ESSENTIA: an open-source library for sound and music analysis

We present Essentia 2.0, an open-source C++ library for audio analysis and audio-based music information retrieval released under the Affero GPL license. It contains an extensive collection of reusable algorithms which implement audio input/output functionality, standard digital signal processing blocks, statistical characterization of data, and a large set of spectral, temporal, tonal and high-level music descriptors. The library is also wrapped in Python and includes a number of predefined executable extractors for the available music descriptors, which facilitates its use for fast prototyping and allows setting up research experiments very rapidly. Furthermore, it includes a Vamp plugin to be used with Sonic Visualiser for visualization purposes. The library is cross-platform and currently supports Linux, Mac OS X, and Windows systems. Essentia is designed with a focus on the robustness of the provided music descriptors and is optimized in terms of the computational cost of the algorithms. The provided functionality, specifically the music descriptors included in-the-box and signal processing algorithms, is easily expandable and allows for both research experiments and development of large-scale industrial applications.

Rāga Recognition based on Pitch Distribution Methods

Abstract Rāga forms the melodic framework for most of the music of the Indian subcontinent. Thus automatic rāga recognition is a fundamental step in the computational modelling of the Indian art-music traditions. In this work, we investigate the properties of rāga and the natural processes by which people identify it. We bring together and discuss the previous computational approaches to rāga recognition correlating them with human techniques, in both Karṇāṭaka (south Indian) and Hindustānī (north Indian) music traditions. The approaches which are based on first-order pitch distributions are further evaluated on a large comprehensive dataset to understand their merits and limitations. We outline the possible short and mid-term future directions in this line of work.

Automatic Tonic Identification in Indian Art Music: Approaches and Evaluation

Abstract The tonic is a fundamental concept in Indian art music. It is the base pitch, which an artist chooses in order to construct the melodies during a rāg(a) rendition, and all accompanying instruments are tuned using the tonic pitch. Consequently, tonic identification is a fundamental task for most computational analyses of Indian art music, such as intonation analysis, melodic motif analysis and rāg recognition. In this paper we review existing approaches for tonic identification in Indian art music and evaluate them on six diverse datasets for a thorough comparison and analysis. We study the performance of each method in different contexts such as the presence/absence of additional metadata, the quality of audio data, the duration of audio data, music tradition (Hindustani/Carnatic) and the gender of the singer (male/female). We show that the approaches that combine multi-pitch analysis with machine learning provide the best performance in most cases (90% identification accuracy on average), and are robust across the aforementioned contexts compared to the approaches based on expert knowledge. In addition, we also show that the performance of the latter can be improved when additional metadata is available to further constrain the problem. Finally, we present a detailed error analysis of each method, providing further insights into the advantages and limitations of the methods.

Mining Melodic Patterns in Large Audio Collections of Indian Art Music

Discovery of repeating structures in music is fundamental to its analysis, understanding and interpretation. We present a data-driven approach for the discovery of short-time melodic patterns in large collections of Indian art music. The approach first discovers melodic patterns within an audio recording and subsequently searches for their repetitions in the entire music collection. We compute similarity between melodic patterns using dynamic time warping (DTW). Furthermore, we investigate four different variants of the DTW cost function for rank refinement of the obtained results. The music collection used in this study comprises 1,764 audio recordings with a total duration of 365 hours. Over 13 trillion DTW distance computations are done for the entire dataset. Due to the computational complexity of the task, different lower bounding and early abandoning techniques are applied during DTW distance computation. An evaluation based on expert feedback on a subset of the dataset shows that the discovered melodic patterns are musically relevant. Several musically interesting relationships are discovered, yielding further scope for establishing novel similarity measures based on melodic patterns. The discovered melodic patterns can further be used in challenging computational tasks such as automatic raga recognition, composition identification and music recommendation.

An evaluation of methodologies for melodic similarity in audio recordings of Indian art music

We perform a comparative evaluation of methodologies for computing similarity between short-time melodic fragments of audio recordings of Indian art music. We experiment with 560 different combinations of procedures and parameter values. These include the choices made for the sampling rate of the melody representation, pitch quantization levels, normalization techniques and distance measures. The dataset used for evaluation consists of 157 and 340 annotated melodic fragments of Carnatic and Hindustani music recordings, respectively. Our results indicate that melodic fragment similarity is particularly sensitive to distance measures and normalization techniques. Sampling rates do not have a significant impact for Hindustani music, but can significantly degrade the performance for Carnatic music. Overall, the performed evaluation provides a better understanding of the processing steps and parameter settings for melodic similarity in Indian art music. Importantly, it paves the way for developing unsupervised melodic pattern discovery approaches, whose evaluation is a challenging and, many times, ill-defined task.

Meter detection from audio for indian music

The meter of a musical excerpt provides high-level rhythmic information and is valuable in many music information retrieval tasks. We investigate the use of a computationally efficient approach to metrical analysis based on psycho-acoustically motivated decomposition of the audio signal. A two-stage comb filter-based approach, originally proposed for double/ triple meter estimation, is extended to a septuple meter (such as 7/8 time-signature) and its performance evaluated on a sizable Indian music database. We find that this system works well for Indian music and the distribution of musical stress/accents across a temporal grid can be utilized to obtain the metrical structure of audio automatically.

Automatic genre classification of North Indian devotional music

The automatic classification of musical genre from audio signals has been a topic of active research in recent years. Although the identification of genre is a subjective task that likely involves high-level musical attributes such as instrumentation, style, rhythm and melody, low-level acoustic features have been widely applied to the automatic task with varying degrees of success. In this work, we consider the genres of the music of northern India, in particular the devotional music sub-genres of bhajan and qawwali. Both are rooted in the framework of North Indian classical music and are similar in the sense of serving the identical socio-cultural function even if for different religious communities of the same region. Features representing timbre, as well as temporal characteristics in the form of tempo and modulation spectra of timbral features, are shown to be potentially effective discriminators as seen by classification experiments performed on a database of excerpts drawn from the two music genres.

Rhythm pattern representations for tempo detection in music

Detection of perceived tempo of music is an important aspect of music information retrieval. Perceived tempo depends in a complex manner on the rhythm structure of the audio signal. Machine learning approaches, proposed recently, avoid peak picking and use rhythm pattern matching with stored tempo annotated songs in the database. We investigate different signal processing methods for rhythm pattern extraction and evaluate these for the music tempo detection task. We also investigate the effect of using additional information about the rhythmic style on the performance of the tempo detection system. The different systems are comparatively evaluated on a standard Ballroom Dance music database and an Indian music database.

Phrase-based rĀga recognition using vector space modeling

Automatic raga recognition is one of the fundamental computational tasks in Indian art music. Motivated by the way seasoned listeners identify ragas, we propose a raga recognition approach based on melodic phrases. Firstly, we extract melodic patterns from a collection of audio recordings in an unsupervised way. Next, we group similar patterns by exploiting complex networks concepts and techniques. Drawing an analogy to topic modeling in text classification, we then represent audio recordings using a vector space model. Finally, we employ a number of classification strategies to build a predictive model for raga recognition. To evaluate our approach, we compile a music collection of over 124 hours, comprising 480 recordings and 40 ragas. We obtain 70% accuracy with the full 40-raga collection, and up to 92% accuracy with its 10-raga subset. We show that phrase-based raga recognition is a successful strategy, on par with the state of the art, and sometimes outperforms it. A by-product of our approach, which arguably is as important as the task of raga recognition, is the identification of raga-phrases. These phrases can be used as a dictionary of semantically-meaningful melodic units for several computational tasks in Indian art music.

Melodic shape stylization for robust and efficient motif detection in Hindustani vocal music

In Hindustani classical music, melodic phrases are identified not only by the stable notes at precise pitch intervals but also by the shapes of the continuous transient pitch segments connecting these. Time-series matching via subsequence dynamic time warping (DTW) facilitates the equal contribution of stable notes and transients to the computation of similarity between pitch contour segments corresponding to melodic phrases. In the interest of reducing computational complexity it is advantageous to replace time-series DTW with low-dimensional string matching provided a principled approach to the time-series to symbolic string conversion is available. While the stable notes easily lend themselves to quantization, we address the compact representation of the transient pitch segments in this work. We analyze the design considerations at each stage: pitch curve fitting, normalization (with respect to pitch interval and duration), shape dictionary generation, inter-symbol proximity measure and string matching cost functions. A combination of domain knowledge- and data-driven optimization on a database of raga music is exploited to design the melodic representation of a raga phrase that enables a performance comparable to the time series based matching in an audio search by query task at significantly lower computational cost.