Shlomo Dubnov

Automatic Classification of Musical Instrument Sounds

We present an exhaustive review of research on automatic classification of sounds from musical instruments. Two different but complementary approaches are examined, the perceptual approach and the taxonomic approach. The former is targeted to derive perceptual similarity functions in order to use them for timbre clustering and for searching and retrieving sounds by timbral similarity. The latter is targeted to derive indexes for labeling sounds after culture- or user-biased taxonomies. We review the relevant features that have been used in the two areas and then we present and discuss different techniques for similarity-based clustering of sounds and for classification into pre-defined instrumental categories.

Using Factor Oracles for Machine Improvisation

We describe variable markov models we have used for statistical learning of musical sequences, then we present the factor oracle, a data structure proposed by Crochemore & al for string matching. We show the relation between this structure and the previous models and indicate how it can be adapted for learning musical sequences and generating improvisations in a real-time context.

Using machine-learning methods for musical style modeling

The ability to construct a musical theory from examples presents a great intellectual challenge that, if successfully met, could foster a range of new creative applications. Inspired by this challenge, we sought to apply machine-learning methods to the problem of musical style modeling. Our work so far has produced examples of musical generation and applications to a computer-aided composition system. Machine learning consists of deriving a mathematical model, such as a set of stochastic rules, from a set of musical examples. The act of musical composition involves a highly structured mental process. Although it is complex and difficult to formalize, it is clearly far from being a random activity. Our research seeks to capture some of the regularity apparent in the composition process by using statistical and information theoretic tools to analyze musical pieces. The resulting models can be used for inference and prediction and, to a certain extent, to generate new works that imitate the style of the great masters.

Maximum a-posteriori probability pitch tracking in noisy environments using harmonic model

Modern speech processing applications require operation on signal of interest that is contaminated by high level of noise. This situation calls for a greater robustness in estimation of the speech parameters, a task which is hard to achieve using standard speech models. In this paper, we present an optimal estimation procedure for sound signals (such as speech) that are modeled by harmonic sources. The harmonic model achieves more robust and accurate estimation of voiced speech parameters. Using maximum a posteriori probability framework, successful tracking of pitch parameters is possible in ultra low signal to noise conditions (as low as -15 dB). The performance of the method is evaluated using the Keele pitch detection database with realistic background noise. The results show best performance in comparison to other state-of-the-art pitch detectors. Application of the proposed algorithm in a simple speaker identification system shows significant improvement in the performance.

Synthesizing sound textures through wavelet tree learning

Natural sounds are complex phenomena because they typically contain a mixture of events localized in time and frequency. Moreover, dependencies exist across different time scales and frequency bands, which are important for proper sound characterization. Historically, acoustical theorists have represented sound in numerous ways. Our research has focused on a granular method of sonic analysis, which views sound as a series of short, distinct bursts of energy. Using that theory, this article presents a statistical learning algorithm for synthesizing new random instances of natural sounds.

Structural and affective aspects of music from statistical audio signal analysis

Understanding and modeling human experience and emotional response when listening to music are important for better understanding of the stylistic choices in musical composition. In this work, we explore the relation of audio signal structure to human perceptual and emotional reactions. Memory, repetition, and anticipatory structure have been suggested as some of the major factors in music that might influence and possibly shape these responses. The audio analysis was conducted on two recordings of an extended contemporary musical composition by one of the authors. Signal properties were analyzed using statistical analyses of signal similarities over time and information theoretic measures of signal redundancy. They were then compared to Familiarity Rating and Emotional Force profiles, as recorded continually by listeners hearing the two versions of the piece in a live-concert setting. The analysis shows strong evidence that signal properties and human reactions are related, suggesting applications of these techniques to music understanding and music information-retrieval systems.

Generalization of spectral flatness measure for non-Gaussian linear processes

We present an information-theoretic measure for the amount of randomness or stochasticity that exists in a signal. This measure is formulated in terms of the rate of growth of multi-information for every new signal sample of the signal that is observed over time. In case of a Gaussian statistics it is shown that this measure is equivalent to the well-known spectral flatness measure that is commonly used in audio processing. For nonGaussian linear processes a generalized spectral flatness measure is developed, which estimates the excessive structure that is present in the signal due to the nonGaussianity of the innovation process. An estimator for this measure is developed using Negentropy approximation to the non-Gaussian signal and the innovation process statistics. Applications of this new measure are demonstrated for the problem of voiced/unvoiced determination, showing improved performance.

Generalized Likelihood Ratio Test for Voiced-Unvoiced Decision in Noisy Speech Using the Harmonic Model

In this paper, a novel method for voiced-unvoiced decision within a pitch tracking algorithm is presented. Voiced-unvoiced decision is required for many applications, including modeling for analysis/synthesis, detection of model changes for segmentation purposes and signal characterization for indexing and recognition applications. The proposed method is based on the generalized likelihood ratio test (GLRT) and assumes colored Gaussian noise with unknown covariance. Under voiced hypothesis, a harmonic plus noise model is assumed. The derived method is combined with a maximum a-posteriori probability (MAP) scheme to obtain a pitch and voicing tracking algorithm. The performance of the proposed method is tested using several speech databases for different levels of additive noise and phone speech conditions. Results show that the GLRT is robust to speaker and environmental conditions and performs better than existing algorithms.

Spectral Anticipations

Introduction This paper deals relations between randomness and structure in audio and musical sounds. Randomness, in layman sense, would be something that has an element of variation or surprise to it, while structure would be more predictable, rule-based or even deterministic. Dealing with noise, which is the “purest” type of randomness, one usually adopts the canonical physical or engineering definition of noise as signal with white spectrum, i.e. comprised of equal or almost equal energies in all frequencies. This seems to imply that noise is a complex phenomenon simply by the fact that it contains very many frequency components (mathematically, in order to qualify as random or stochastic process, the density of the frequency components has to be such that the signal would have a continuous spectrum, while periodic components would be spectral lines or delta functions). In contradiction to this reasoning comes the fact that to our perception noise is a rather simple signal, and in terms of its musical use it does not allow much structural manipulation and organization. Musical notes or other repeating or periodic acoustic components in music are closer to being deterministic and could be considered as “structure”. But then, complex musical signals, such as polyphonic or orchestral music that contain simultaneous contributions from multiple instrumental sources, often have a spectrum so dense that it seems to approach a noise-like spectrum. In such situation, the ability to determine the structure of the signal can not be revealed by looking at signal spectrum alone. Therefore, the physical definition of noise as a

A New Nonparametric Pairwise Clustering Algorithm Based on Iterative Estimation of Distance Profiles

We present a novel pairwise clustering method. Given a proximity matrix of pairwise relations (i.e. pairwise similarity or dissimilarity estimates) between data points, our algorithm extracts the two most prominent clusters in the data set. The algorithm, which is completely nonparametric, iteratively employs a two-step transformation on the proximity matrix. The first step of the transformation represents each point by its relation to all other data points, and the second step re-estimates the pairwise distances using a statistically motivated proximity measure on these representations. Using this transformation, the algorithm iteratively partitions the data points, until it finally converges to two clusters. Although the algorithm is simple and intuitive, it generates a complex dynamics of the proximity matrices. Based on this bipartition procedure we devise a hierarchical clustering algorithm, which employs the basic bipartition algorithm in a straightforward divisive manner. The hierarchical clustering algorithm copes with the model validation problem using a general cross-validation approach, which may be combined with various hierarchical clustering methods.We further present an experimental study of this algorithm. We examine some of the algorithms properties and performance on some synthetic and ‘standard’ data sets. The experiments demonstrate the robustness of the algorithm and indicate that it generates a good clustering partition even when the data is noisy or corrupted.