Yu Shiu

Similarity matrix processing for music structure analysis

The structure analysis of pop and rock songs from audio signals is conducted via similarity matrix processing in this work. The similarity matrix offers pairwise similarity between any two short intervals of fixed length in a song. We use two similarity matrices to show their diverse characteristics. The characteristics are explained by musical chord successions. Then, several similarity matrix processing techniques are developed for music structure analysis. First, an algorithm is proposed to check the boundaries and periods of repetitive chord successions with short periods. Second, the Viterbi algorithm is applied to detect straight segments in sub-diagonal lines of the similarity matrix. Periods of repeating chord successions are used to refine the state space to enhance the detection performance. Furthermore, a post-processing technique is used to map detected segments into sections in a song. Experimental results from test musical audio data are given to demonstrate the performance of the proposed method.

Audio Source Separation with Matching Pursuit and Content-Adaptive Dictionaries (MP-CAD)

A single-channel audio source separation algorithm based on the matching pursuit (MP) technique with content-adaptive dictionaries (CAD) is proposed in this work. The proposed MP-CAD algorithm uses content-dependent atoms that capture inherent characteristics of audio signals effectively. As compared with previous methods based on spectral decomposition and clustering in the time-frequency domain, the MP-CAD algorithm projects the time-domain audio signals onto a subspace spanned by content-adaptive atoms efficiently for their concise representation and separation. The effectiveness of the MP-CAD algorithm in audio signal approximation and single-channel source separation is demonstrated by computer simulation.

Robust on-line beat tracking with kalman filtering and probabilistic data association (KF-PDA)

A Kalman filtering (KF) approach to on-line musical beat tracking with probabilistic data association (PDA) is investigated in this work. We first formulate the beat tracking process as a linear dynamic system of beat progression, and then apply the Kalman filtering algorithm to the dynamic system in estimating the time-varying tempo and beat locations. Musical beat tracking using traditional Kalman filtering is however not reliable in the presence of tempo fluctuations and expressive timing deviations. To address this problem, we adopt data association techniques to assign probability masses to all possible beat interpretations, and then locate the true beat according to the weighting. Two methods are proposed. The first one (PDA-I) weighs the distance between the candidate observation and the predicted beat location while the second method (PDA-II) considers not only the distance but also the onset intensity in weight selection. Superior performance of the proposed beat tracking algorithm is demonstrated with simulation results on the Music Information Retrieval Evaluation Exchange (MIREX) 2006 beat tracking competition practice dataset and the Billboard Top-10 database.

Efficient music representation with content adaptive dictionaries

An efficient music representation based on the matching pursuit (MP) technique with content-adaptive dictionaries (CADs) is investigated in this work. The Gabor atoms are commonly adopted in the MP-based signal representation due to their excellent time-frequency localization property. However, the Gabor dictionary may not yield a concise representation for music signals. Music signals have special characteristics which are specified by pitches and durations of music notes. In this work, we exploit music characteristics to create a set of content-adaptive elementary functions called atoms that efficiently capture the inherent structure of music signals. As a result, we are able to project musical signals onto a subspace spanned by atoms from CAD for a concise and efficient representation. The proposed CAD representation technique is applied to music enhancement with noisy background to demonstrate the power of the proposed representation.

Musical structure analysis using similarity matrix and dynamic programming

Automatic music segmentation and structure analysis from audio waveforms based on a three-level hierarchy is examined in this research, where the three-level hierarchy includes notes, measures and parts. The pitch class profile (PCP) feature is first extracted at the note level. Then, a similarity matrix is constructed at the measure level, where a dynamic time warping (DTW) technique is used to enhance the similarity computation by taking the temporal distortion of similar audio segments into account. By processing the similarity matrix, we can obtain a coarse-grain music segmentation result. Finally, dynamic programming is applied to the coarse-grain segments so that a song can be decomposed into several major parts such as intro, verse, chorus, bridge and outro. The performance of the proposed music structure analysis system is demonstrated for pop and rock music.

Audio fingerprint extraction for content identification

In this work, we present an audio content identification system that identifies some unknown audio material by comparing its fingerprint with those extracted off-line and saved in the music database. We will describe in detail the procedure to extract audio fingerprints and demonstrate that they are robust to noise and content-preserving manipulations. The main feature in the proposed system is the zero-crossing rate extracted with the octave-band filter bank. The zero-crossing rate can be used to describe the dominant frequency in each subband with a very low computational cost. The size of audio fingerprint is small and can be efficiently stored along with the compressed files in the database. It is also robust to many modifications such as tempo change and time-alignment distortion. Besides, the octave-band filter bank is used to enhance the robustness to distortion, especially those localized on some frequency regions.

On-line Music Beat Tracking with Kalman Filtering and Probability Data Association (KF-PDA)

An on-line music beat tracking algorithm with Kalman filtering (KF) and probability data association (PDA) is proposed. The KF-PDA method not only estimates the music tempo and beat pulse positions in real-time but also provides a temporal metrical unit for human-perception-like processing. The superior performance of the proposed KF-PDA beat tracking system is demonstrated using the MIREX beat tracking competitions practice data set.

An Improved Technique for Blind Audio Source Separation

A blind audio source separation technique with an ill-posed mixing matrix and additive noise is proposed in this work. With this technique, we divide the solution into two steps. The first step is to estimate the ill-posed mixing matrix and the second step is to separate original sources. To estimate the ill-posed mixing matrix, an enhanced soft-assignment method is used in the first step. Then, the generalized p-norm optimization method is adopted in the second step, which can yield a solution sparser than the l1-norm minimization technique. Experimental results on synthetic mixtures and real-world mixtures are used to demonstrate the efficiency of the proposed technique in the presence of an ill-posed mixing matrix and additive noise.

A Musical Onset Detection System with Scalable Complexity

An onset detection system based on linear prediction with scalable complexity is proposed in this work. One unique feature of the proposed onset detection algorithm is that it can offer a trade-off between complexity and detection accuracy by adjusting its parameters. Consequently, it can be used in consumer electronics such as karaoke performance evaluation and automatic visual effect generation in portable media players.

On-Line Musical Beat Tracking with Phase-Locked-Loop (PLL) Techniique

An on-line beat tracking algorithm based on the digital phase-locked-loop (PLL) technique is proposed in this work, which estimates both the music tempo and the position of beat pulses in real time. In addition to a detailed description of the proposed algorithm, we show that the PLL-technique is effective in its tracking performance and simple to implement.