Julian Mathias Becker

NMF-based informed source separation

Informed Source Separation (ISS) is a topic unifying the research fields of both source separation and source coding. Its main objective is to recover audio objects out of a mixture with a source separation step assisted by a set of compact parameters extracted with complete knowledge of the sources. ISS can be used for applications such as active listening and remixing of music (e.g. karaoke). In this paper, we propose a new ISS method which includes a semi-blind source separation (SBSS) step in the ISS decoder to decrease the amount of parameter bit rate. SBSS is conducted by factorizing the mixture in time-frequency domain by nonnegative matrix factorization (NMF). The transmitted parameters consist of a compact NMF initialization as well as residuals calculated in the NMF domain. We show in simulations that using SBSS in the decoder increases the separation quality and that our scheme improves the rate-distortion performance in comparison to a state-of-the art method.

Generalized constraints for NMF with application to informed source separation

Nonnegative matrix factorization (NMF) is a widely used method for audio source separation. Additional constraints supporting e.g. temporal continuity or sparseness adapt NMF to the structure of audio signals even further. In this paper, we propose generalized NMF constraints which make use of prior information gathered for each component individually. In general, this information could be obtained blindly or by a training step. Here we make use of these novel constraints in an algorithm for informed audio source separation (ISS). ISS uses source separation to code audio objects by assisting a source separation step in the decoder with parameters extracted with knowledge of the sources in the encoder. In [1], a novel algorithm for ISS was proposed which makes use of an NMF step in the decoder. We show in experiments that the generalized constraints enhance the separation quality while keeping the additionally needed bit rate very low.

A probability-based combination method for unsupervised clustering with application to blind source separation

Unsupervised clustering algorithms can be combined to improve the robustness and the quality of the results, e.g. in blind source separation. Before combining the results of these clustering methods the corresponding clusters have to be aligned, but usually it is not known which clusters of the employed methods correspond to each other. In this paper, we present a method to avoid this correspondence problem using probability theory. We also present an application of our method in blind source separation. Our approach is better expandable than other state-of-the-art separation algorithms while leading to slightly better results.

Component-Adaptive Priors for NMF

Additional priors for nonnegative matrix factorization NMF are a powerful way of adapting NMF to specific tasks, such as for example audio source separation. For this application, priors supporting sparseness or temporal continuity have been proposed. However, these priors are not helpful for all kinds of signals and should therefore only be used when needed. For some mixtures, only some components of the mixtures should be supported by these priors. We present an easy, but efficient method of adapting priors to different components. We show, that the separation results are improved, while the computational complexity is even slightly reduced. We also show, that our method is a helpful modification for the combination of different priors.

Custom sized non-negative matrix factor deconvolution for sound source separation

Non-negative Matrix Factorization (NMF) is frequently used for audio source separation. One downside of the NMF is, that it is not able to capture temporal structure of sound events. NMF splits these events into different components. In this paper we present an extension to NMF, which is capable of representing sound events with temporal structure in only one component. We also present an algorithm, which uses this method efficiently. We show that this algorithm leads to a more compact factorization (i.e. with less components) compared to NMF, without losing separation quality.

Quantization-aware parameter estimation for audio upmixing

Upmixing consists in extracting audio objects out of their downmix, given some parameters computed beforehand at a coding stage. It is an important task in audio processing with many applications in the entertainment industry. One particularly successful approach for this purpose is to compress the audio objects through nonnegative matrix factorization (NMF) parameters at the coder, to be used for separating the downmix at the decoder. In this paper, we focus on such NMF methods for audio compression, which operate at very low parameter bitrates. In existing methods, parameter estimation and quantization are conducted independently. Here, we propose two extensions: first, we jointly estimate and quantize the parameters at the coder to ensure good reconstruction at the decoder. Second, we propose a parameter refinement method operated at the decoder, that benefits from priors induced by quantization to yield better performance. We show that our contributions outperform existing baseline methods.

Extended semantic initialization for NMF-based audio source separation

Nonnegative matrix factorization (NMF) is often used for source separation of audio signals. In most of these algorithms, the initialization step of the NMF, which has a strong impact on the separation performance, is based on random values or deterministic methods such as singular value decomposition (SVD). Another deterministic initialization approach, which is used e.g. for score-informed source separation algorithms, makes use of synthesized magnitude spectra of harmonic notes. It was shown that this semantic method leads to good separation results in blind source separation (BSS) as well; not only for harmonic but also for percussive mixtures with some harmonic components. In this paper, we present an extension to the semantic approach to enhance the separation quality for arbitrary audio mixtures. We evaluate this extension in a BSS scenario and compare it to other initialization schemes.

Adaptive weights for NMF with additional priors

Nonnegative matrix factorization (NMF) has become a very popular method in various signal processing applications. Supporting NMF with additional cost functions, so called priors, is very helpful to adapt the factorization to specific tasks. Additional priors are usually multiplied by fixed weights to adjust the influence of the prior. The question how to adapt these weights to the needs of specific factorization scenarios is yet unsolved. In this paper, we present a method to adjust the weights iteratively throughout the NMF process. We evaluate our method in an audio source separation environment and show, that it is more robust than the recently used method with fixed weights and that it leads to better separation results.