Francisco J. Cañadas-Quesada

Musical Instrument Sound Multi-Excitation Model for Non-Negative Spectrogram Factorization

This paper presents theoretical and experimental results about constrained non-negative matrix factorization (NMF) to model the excitation of the musical instruments. These excitations represent vibrating objects, while the filter represents the resonance structure of the instrument, which colors the produced sound. We propose to model the excitations as the weighted sum of harmonically constrained basis functions, whose parameters are tied across different pitches of an instrument. An NMF-based framework is used to learn the model parameters. We assume that the excitations of a well-tempered instrument should possess an identifiable characteristic structure whereas the conditions of the music scene might produce variations in the filter. In order to test the reliability of our proposal, we evaluate our method for a music transcription task in two scenarios. On the first one, comparison with state-of-the-art methods has been performed over a dataset of piano recordings obtaining more accurate results than other NMF-based algorithms. On the second one, two woodwind instrument databases have been used to demonstrate the benefits of our model in comparison with previous excitation-filter model approaches.

Constrained non-negative sparse coding using learnt instrument templates for realtime music transcription

Abstract In this paper, we present a realtime signal decomposition method with single-pitch and harmonicity constrains using instrument specific information. Although the proposed method is designed for monophonic music transcription, it can be used as a candidate selection technique in combination with other realtime transcription methods to address polyphonic signals. The harmonicity constraint is particularly beneficial for automatic transcription because, in this way, each basis can define a single pitch. Furthermore, restricting the model to have a single-nonzero gain at each frame has been shown to be a very suitable constraint when dealing with monophonic signals. In our method, both harmonicity and single-nonzero gain constraints are enforced in a deterministic manner. A realtime factorization procedure based on Non-negative sparse coding (NNSC) with Beta-divergence and fixed basis functions is proposed. In this paper, the basis functions are learned using a supervised process to obtain spectral patterns for different musical instruments. The proposed method has been tested for music transcription of both monophonic and polyphonic signals and has been compared with other state-of-the-art transcription methods, and in these tests, the proposed method has obtained satisfactory results in terms of accuracy and runtime.

Note-event Detection in Polyphonic Musical Signals based on Harmonic Matching Pursuit and Spectral Smoothness

Abstract Harmonic Matching Pursuit (HMP) is an interesting signal processing tool to detect simultaneous notes in musical audio signals. HMP decomposes an audio signal into harmonic atoms, a suitable choice due to their strong harmonic content. However, HMP provides an inaccurate decomposition when notes with a rational frequency relation are simultaneously played (the overlapping partial problem). An example of such a situation is the common occurrence in Western music of consonant intervals, i.e. perfect fifth (3:2), perfect fourth (4:3) or major third (5:4). For these cases, HMP often obtains harmonic atoms whose fundamental frequencies are the greatest common divisor of the active fundamental frequencies. To solve this problem, we propose a processing algorithm that works over the harmonic atoms obtained by HMP. This algorithm is based on maximizing the smoothness of the spectral envelope for each harmonic atom provided by the HMP decomposition. Results show that our proposal achieves a good performance. Competitive accuracy and error measures in a musical transcription application are reported in relation to the state of the art transcription systems.

Constrained non-negative matrix factorization for score-informed piano music restoration

In this work, we propose a constrained non-negative matrix factorization method for the audio restoration of piano music using information from the score. In the first stage (instrument training), spectral patterns for the target source (piano) are learned from a dataset of isolated piano notes. The model for the piano is constrained to be harmonic because, in this way, each pattern can define a single pitch. In the second stage (noise training), spectral patterns for the undesired source (noise) are learned from the most common types of vinyl noises. To obtain a representative model for the vinyl noise, a cross-correlation-based constraint that minimizes the cross-talk between different noise components is used. In the final stage (separation), we use the trained instrument and noise models in an NMF framework to extract the clean audio signal from undesired non-stationary noise. To improve the separation results, we propose a novel score-based constraint to avoid activations of notes or combinations that are not present in the original score. The proposed approach has been evaluated and compared with commercial audio restoration softwares, obtaining competitive results.

Monophonic constrained non-negative sparse coding using instrument models for audio separation and transcription of monophonic source-based polyphonic mixtures

In this paper we propose a monophonic constrained signal decomposition model applied to polyphonic signals composed of several monophonic sources from different musical instruments. The harmonic constraint is particularly effective for tonal instruments because each note is associated with a unique basis. The monophonic constraint is implemented by enforcing single-non-zero gains per instrument in the factorization process. The proposed method uses previously trained instrument models with a supervised procedure. Both constraints (harmonic and monophonic) are implemented in a deterministic manner. The proposed method has been tested for two audio signal applications, Sound Source Separation and Automatic Music Transcription. Comparison with other state-of-the-art methods using a dataset of polyphonic mixtures composed of monophonic sources has produced competitive and promising results.

Improving multiple-F0 estimation by onset detection for polyphonic music transcription

In a monaural polyphonic context, music transcription and specifically, multiple-F0 estimation systems have achieved promising results in the last decade. However, most of these systems present intermittent misses of pitch within a note or inaccurate definitions about onsets and offsets due to frame-by-frame analysis. In this paper, we propose a multiple-F0 estimation system which extracts a set of active pitches at each frame (analysis frame) but note tracking is performed defining temporal intervals by an accurate onset detector. Our system shows promising results, in terms of onset and multiple-F0 estimation, to be evaluated using real-world and synthesized polyphonic music recordings taken from MAPS music database.

Gunshot detection and localization based on Non-negative Matrix Factorization and SRP-hat

This paper describes a sound source detection and localization system where the sound source is a gunshot. The proposed system is composed of two consecutive stages: detection and localization. In this manner, the localization system is activated when a gunshot is detected. The detection stage is based on a Non-negative Matrix Factorization (NMF) approach. The localization stage proposes a modification of the method called Steered Response Power Phase transform (SRP-Phat). Firstly, we estimate the delay between multichannel signals that are captured by a six microphone array. Generalized Cross-Correlation (GCC), Generalized Cross-Correlation Phase transform (GCC-Phat), Steered Response Power Phase transform (SRP-Phat) and the proposed localization method are compared. Evaluation indicates that the proposed detection and localization method provides promising results in many acoustic scenarios including noisy environments with low signal-to-noise ratios. The proposed application is designed following the paradigm of smart sound processing and is focused on security needs in a real life scenario.

Compositional model for speech denoising based on source/filter speech representation and smoothness/sparseness noise constraints

Semi-supervised NMF with source/filter speech model and constrained noise parameters.Evaluated on the 3rd CHiME and SiSEC 2013 datasets for speech and noise separation.Proposed noise constraints help to improve the isolation of speech with real noises.All tested environments but one could be modeled by the constraints.Better separation results than conventional semi-supervised sparse NMF. We present a speech denoising algorithm based on a regularized non-negative matrix factorization (NMF), in which several constraints are defined to describe the background noise in a generic way. The observed spectrogram is decomposed into four signal contributions: the voiced speech source and three generic types of noise. The speech signal is represented by a source/filter model which captures only voiced speech, and where the filter bases are trained on a database of individual phonemes, resulting in a small dictionary of phoneme envelopes. The three remaining terms represent the background noise as a sum of three different types of noise (smooth noise, impulsive noise and pitched noise), where each type of noise is characterized individually by imposing specific spectro-temporal constraints, based on sparseness and smoothness restrictions. The method was evaluated on the 3rd CHiME Speech Separation and Recognition Challenge development dataset and compared with conventional semi-supervised NMF with sparse activations. Our experiments show that, with a similar number of bases, source/filter modeling of speech in conjunction with the proposed noise constraints produces better separation results than sparse training of speech bases, even though the system is only designed for voiced speech and the results may still not be practical for many applications.

Wheezing Sound Separation Based on Constrained Non-Negative Matrix Factorization

Auscultation remains the first clinical examination that a physician performs to detect respiratory diseases originated by wheezes, which are the most specific asthmatic symptoms. It is common that respiratory sounds (normal breath sounds) acoustically interfere wheezes with both frequency and time domain. As a result, the physicians cognitive ability is reduced causing a misdiagnosis or inability to clearly hear all significant sounds to detect a pulmonary disease. This paper presents a constrained non-negative matrix factorization (NMF) approach to separate wheezes from respiratory sounds applied to single-channel mixtures. The proposed constraints, smoothness and sparseness, attempts to model common spectral behaviour shown by wheezes and normal breath sounds. Specifically, the spectrogram of a wheeze can be modelled as a narrowband spectrum (sparseness in frequency). However, the spectrogram of a normal breath sound can be modelled as a wideband spectrum (smoothness in frequency) with a slow temporal variation (smoothness in time). Experimental results report that the proposed method improves the audio quality of the wheezes removing most of the respiratory sounds, being a novel way to successfully apply a NMF approach to a wheeze/respiratory sound separation.