Minje Kim

Nonnegative matrix partial co-factorization for drum source separation

We address a problem of separating drums from polyphonic music containing various pitched instruments as well as drums. Nonnegative matrix factorization (NMF) was successfully applied to spectrograms of music to learn basis vectors, followed by support vector machine (SVM) to classify basis vectors into ones associated with drums (rhythmic source) only and pitched instruments (harmonic sources). Basis vectors associated with pitched instruments are used to reconstruct drum-eliminated music. However, it is cumbersome to construct a training set for pitched instruments since various instruments are involved. In this paper, we propose a method which only incorporates prior knowledge on drums, not requiring such training sets of pitched instruments. To this end, we present nonnegative matrix partial co-factorization (NMPCF) where the target matrix (spectrograms of music) and drum-only-matrix (collected from various drums a priori) are simultaneously decomposed, sharing some factor matrix partially, to force some portion of basis vectors to be associated with drums only. We develop a simple multiplicative algorithm for NMPCF and show its usefulness empirically, with numerical experiments on real-world music signals.

Nonnegative Matrix Partial Co-Factorization for Spectral and Temporal Drum Source Separation

We address a problem of separating drum sources from monaural mixtures of polyphonic music containing various pitched instruments as well as drums. We consider a spectrogram of music, described by a matrix where each row is associated with intensities of a frequency over time. We employ a joint decomposition to several spectrogram matrices that include two or more column-blocks of the mixture spectrograms (columns of mixture spectrograms are partitioned into 2 or more blocks) and a drum-only (drum solo playing) matrix constructed from various drums a priori. To this end, we apply nonnegative matrix partial co-factorization (NMPCF) to these target matrices, in which column-blocks of mixture spectrograms and the drum-only matrix are jointly decomposed, sharing a factor matrix partially, in order to determine common basis vectors that capture the spectral and temporal characteristics of drum sources. Common basis vectors learned by NMPCF capture spectral patterns of drums since they are shared in the decomposition of the drum-only matrix and accommodate temporal patterns of drums because repetitive characteristics are captured by factorizing column-blocks of mixture spectrograms (each of which is associated with different time periods). Experimental results on real-world commercial music signal demonstrate the performance of the proposed method.

Monaural music source separation: nonnegativity, sparseness, and shift-invariance

In this paper we present a method for polyphonic music source separation from their monaural mixture, where the underlying assumption is that the harmonic structure of a musical instrument remains roughly the same even if it is played at various pitches and is recorded in various mixing environments. We incorporate with nonnegativity, shift-invariance, and sparseness to select representative spectral basis vectors that are used to restore music sources from their monaural mixture. Experimental results with monaural instantaneous mixture of voice/cello and monaural convolutive mixture of saxophone/viola, are shown to confirm the validity of our proposed method.

Blind rhythmic source separation: Nonnegativity and repeatability

An unsupervised method is proposed aiming at extracting rhythmic sources from commercial polyphonic music whose number of channels is limited to one. Commercial music signals are not usually provided with more than two channels while they often contain multiple instruments including singing voice. Therefore, instead of using conventional ways, such as modeling mixing environments or statistical characteristics, we should introduce other source-specific characteristics for separating or extracting the sources. In this paper, we concentrate on extracting rhythmic sources from the mixture with the other harmonic sources. An extension of nonnegative matrix factorization (NMF) is used to analyze multiple relationships between spectral and temporal properties in the given input matrices. Moreover, temporal repeatability of the rhythmic sound sources is implicated as common rhythmic property among segments of an input mixture signal. The proposed method shows acceptable, but not superior separation quality to the referred drum source separation systems. However, it has better applicability due to its blind manner in separation.

On spectral basis selection for single channel polyphonic music separation

In this paper we present a method of separating musical instrument sound sources from their monaural mixture, where we take the harmonic structure of music into account and use the sparseness and the overlapping NMF to select representative spectral basis vectors which are used to reconstruct unmixed sound. A method of spectral basis selection is illustrated and experimental results with monaural instantaneous mixtures of voice/cello and saxophone/viola, are shown to confirm the validity of our proposed method.

ICA-Based Clustering for Resolving Permutation Ambiguity in Frequency-Domain Convolutive Source Separation

Permutation ambiguity is an inherent limitation in independent component analysis, which is a bottleneck in frequency-domain methods of convolutive source separation. In this paper we present a method for resolving this permutation ambiguity, where we group vectors of estimated frequency responses into clusters in such a way that each cluster contains frequency responses associated with the same source. The clustering is carried out, applying independent component analysis to estimated frequency responses. In contrast to existing methods, the proposed method does not require any prior information such as the geometric configuration of microphone arrays or distances between sources and microphones. Experimental results confirm the validity of our method