Pierre Leveau

Instrument-Specific Harmonic Atoms for Mid-Level Music Representation

Several studies have pointed out the need for accurate mid-level representations of music signals for information retrieval and signal processing purposes. In this paper, we propose a new mid-level representation based on the decomposition of a signal into a small number of sound atoms or molecules bearing explicit musical instrument labels. Each atom is a sum of windowed harmonic sinusoidal partials whose relative amplitudes are specific to one instrument, and each molecule consists of several atoms from the same instrument spanning successive time windows. We design efficient algorithms to extract the most prominent atoms or molecules and investigate several applications of this representation, including polyphonic instrument recognition and music visualization.

Adaptation of source-specific dictionaries in Non-Negative Matrix Factorization for source separation

This paper concerns the adaptation of spectrum dictionaries in audio source separation with supervised learning. Supposing that samples of the audio sources to separate are available, a filter adaptation in the frequency domain is proposed in the context of Non-Negative Matrix Factorization with the Itakura-Saito divergence. The algorithm is able to retrieve the acoustical filter applied to the sources with a good accuracy, and demonstrates significantly higher performances on separation tasks when compared with the non-adaptive model.

A comparison of two extensions of the matching pursuit algorithm for the harmonic decomposition of sounds

In the framework of audio signal analysis, it is desired to obtain sparse representations that are able to reflect the harmonic structures, e.g., issued from musical instruments. In this paper, we compare two approaches which introduce some explicit models of harmonic features into the matching pursuit analysis framework. The first approach is the harmonic matching pursuit (HMP), where the harmonic structures are modeled by sets of harmonically related Gabor atoms which are directly optimized in the analysis loop. The second approach, called meta-molecular matching pursuit (M3P), is based on the a posteriori agglomeration of elementary features coming from a short time Fourier transform. We discuss the pros and cons of each method through experiments involving different audio signals, and conclude on possible approaches for combining the two methods.

Speech-guided source separation using a pitch-adaptive guide signal model

In this paper, we present a new method to perform underdetermined audio source separation using a spoken or sung reference signal to inform the separation process. This method explicitly models possible differences between the spoken reference and the target signal, such as pitch differences and time lag. We show that the proposed algorithm outperforms state-of-the art methods.

Convolutive common audio signal extraction

This paper addresses the extraction of a common signal among several mono audio tracks when this common signal undergoes a track-specific filtering. This problem arises in the extraction of a common music and effects track from a set of soundtracks in different languages. To this aim, a novel approach is proposed. The method is based on the dictionary modeling of track-specific and common signals, and is compared to a previous one proposed by the authors based on geometric considerations. The approach is integrated into a Non-Negative Matrix Factorization framework using the Itakura-Saito divergence. The method is evaluated on a synthetic database composed of filtered music and effects tracks, the filters being track-specific, and track-specific dialogs. The results show that this task becomes tractable, while the previously introduced method could not handle track-specific filtering.

Introducing a simple fusion framework for audio source separation

We propose in this paper a simple fusion framework for un-derdetermined audio source separation. This framework can be applied to a wide variety of source separation algorithms providing that they estimate time-frequency masks. Fusion principles have been successfully implemented for classification tasks. Although it is similar to classification, audio source separation does not usually take advantage of such principles. We thus introduce some general fusion rules inspired by classification and we evaluate them in the context of voice extraction. Experimental results are promising as our proposed fusion rule can improve separation results up to 1 dB in SDR.

Geometric multichannel common signal separation with application to music and effects extraction from film soundtracks

We address the task of separation of music and effects from dialogs in film or television soundtracks. This is of interest for film studios wanting to release films in new, previously unavailable languages when the original separated music and effects track is not available. For this purpose, we propose several methods for common signal extraction from a set of soundtracks in different languages, which are multichannel extensions of previous methods for center signal extraction from stereo signals. The proposed methods are simple, effective, and have an intuitive geometrical interpretation. Experiments show that the proposed methods improve the results provided by our previously proposed methods based on basic filtering techniques.

Sound enhancement using sparse approximation with speclets

This paper addresses an innovative approach to informed enhancement of damaged sound. It uses sparse approximations with a learned dictionary of atoms modeling the main components of the undamaged source spectra. The decomposition process aims at finding which of the atoms could constitute the decomposition of the undamaged source in order to recover it. The decomposition of the damaged signal is done with a Matching Pursuit algorithm and involves an adaptation of the dictionary learned on undamaged sources. Evaluation is performed on a bandwidth extension task for various classes of signals.

Using Stereo Information for Instrument Identification in Polyphonic Mixtures

This paper discusses the localization of music instruments in the stereo space. The signal, composed of two channels, is decomposed into a linear combination of Stereo Instrument-Specific Harmonic atoms, that model the harmonic structure of instrument notes as a whole and whose individual angles give clues about the real angle of the sources. To get such decompositions, a Stereo Matching Pursuit algorithm has been implemented, with a phase adaptation for each signal channel. This decomposition gives neat source localizations for instantaneous mixes, and the extension to realistic convolutive mixes seems possible with adequate post-processing.