Kyeongok Kang

A compressed domain scheme for classifying block edge patterns

Detecting and classifying edge components in images find a wide range of applications from perceptually improved coding schemes to content-based searches. A fast and systematic scheme is proposed to classify the edge orientation of each block in discrete cosine transform (DCT)-compressed images. Derived from a simple, but acceptable, pixel-domain algorithm, the proposed scheme performs directly on DCT coefficient domain and, thus, saves the arithmetic operations by a considerable amount. The performance of the proposed method is validated by the experiments against various natural images.

Nonnegative matrix partial co-factorization for drum source separation

We address a problem of separating drums from polyphonic music containing various pitched instruments as well as drums. Nonnegative matrix factorization (NMF) was successfully applied to spectrograms of music to learn basis vectors, followed by support vector machine (SVM) to classify basis vectors into ones associated with drums (rhythmic source) only and pitched instruments (harmonic sources). Basis vectors associated with pitched instruments are used to reconstruct drum-eliminated music. However, it is cumbersome to construct a training set for pitched instruments since various instruments are involved. In this paper, we propose a method which only incorporates prior knowledge on drums, not requiring such training sets of pitched instruments. To this end, we present nonnegative matrix partial co-factorization (NMPCF) where the target matrix (spectrograms of music) and drum-only-matrix (collected from various drums a priori) are simultaneously decomposed, sharing some factor matrix partially, to force some portion of basis vectors to be associated with drums only. We develop a simple multiplicative algorithm for NMPCF and show its usefulness empirically, with numerical experiments on real-world music signals.

Nonnegative Matrix Partial Co-Factorization for Spectral and Temporal Drum Source Separation

We address a problem of separating drum sources from monaural mixtures of polyphonic music containing various pitched instruments as well as drums. We consider a spectrogram of music, described by a matrix where each row is associated with intensities of a frequency over time. We employ a joint decomposition to several spectrogram matrices that include two or more column-blocks of the mixture spectrograms (columns of mixture spectrograms are partitioned into 2 or more blocks) and a drum-only (drum solo playing) matrix constructed from various drums a priori. To this end, we apply nonnegative matrix partial co-factorization (NMPCF) to these target matrices, in which column-blocks of mixture spectrograms and the drum-only matrix are jointly decomposed, sharing a factor matrix partially, in order to determine common basis vectors that capture the spectral and temporal characteristics of drum sources. Common basis vectors learned by NMPCF capture spectral patterns of drums since they are shared in the decomposition of the drum-only matrix and accommodate temporal patterns of drums because repetitive characteristics are captured by factorizing column-blocks of mixture spectrograms (each of which is associated with different time periods). Experimental results on real-world commercial music signal demonstrate the performance of the proposed method.

Summarization of news video and its description for content‐based access

A video summary abstracts the entirety with the gist without losing the essential content of the original video and also facilitates efficient content‐based access to the desired content. In this article, we propose a novel method for summarizing a news video based on multimodal analysis of the content. The proposed method exploits the closed caption (CC) data to locate semantically meaningful highlights in a news video and speech signals in an audio stream to align the CC data with the video in a time‐line. Then, the extracted highlights are described in a multilevel structure using the MPEG‐7 Summarization Description Scheme (DS). Specifically, we use the HierarchicalSummary DS that allows efficient accessing of the content through such functionalities as multilevel abstracts and navigation guidance in a hierarchical fashion. Intensive experiments with our prototypical systems are presented to demonstrate the validity and reliability of the proposed method in real applications.

Blind rhythmic source separation: Nonnegativity and repeatability

An unsupervised method is proposed aiming at extracting rhythmic sources from commercial polyphonic music whose number of channels is limited to one. Commercial music signals are not usually provided with more than two channels while they often contain multiple instruments including singing voice. Therefore, instead of using conventional ways, such as modeling mixing environments or statistical characteristics, we should introduce other source-specific characteristics for separating or extracting the sources. In this paper, we concentrate on extracting rhythmic sources from the mixture with the other harmonic sources. An extension of nonnegative matrix factorization (NMF) is used to analyze multiple relationships between spectral and temporal properties in the given input matrices. Moreover, temporal repeatability of the rhythmic sound sources is implicated as common rhythmic property among segments of an input mixture signal. The proposed method shows acceptable, but not superior separation quality to the referred drum source separation systems. However, it has better applicability due to its blind manner in separation.

Variable Subband Analysis for High Quality Spatial Audio Object Coding

Spatial Audio Object Coding (SAOC) handles a number of audio objects to provide a user with active audio services. It represents all objects as a stereo downmixed signal with some side information, and the bitrate can be significantly reduced compared to that of the conventional audio coders. In spite of the advantage of bitrate reduction, the SAOC has the severe problem such as the degradation of sound quality. To solve the problem while minimizing increase of the bitrate, variable subband analysis and the revised SAOC structure called two-step structure were proposed in this paper. Through a subjective listening test, it was confirmed that the proposed method improves sound quality by a small bitrate increase.

Spatial Audio Object Coding With Two-Step Coding Structure for Interactive Audio Service

An interactive audio service is a new conceptual audio service that provides the users with opportunities for a variety of experiences on the alternative and advanced audio services. In the interactive audio service, users can freely control various audio objects to make their own audio sounds. A spatial audio object coding (SAOC) is a useful technology that can support most parts of the interactive audio service with a relatively low bit-rate, but is very poor to perfect gain control of a certain audio object, i.e., the target audio object. In this paper, the SAOC with a two-step coding structure is proposed to efficiently handle the target audio object as well as the normal audio objects. A transform coded excitation (TCX) based residual coding scheme is presented in the context of the sound quality enhancement. From experimental results, it can be noted that the various audio objects can be successfully handled with respect to the bit-rate and the sound quality by using the proposed two-step coding structure SAOC.

A model of sound localisation applied to the evaluation of systems for stereophony

In this paper, a model of human sound localisation is described, and its prediction is compared to the results of listening tests. The model takes binaural signals as the input, processing them in a series of signal processing modules, which simulate the peripheral, binaural and the central stages of spatial hearing. In particular, the central processor of the model considers the excitation-inhibition (EI) cell activity patterns as the internal representation of available cues, and the source location estimates are obtained by using a simple pattern-matching procedure. In the listening tests, stereophonic images were presented to the listeners front, where the stimulus was either broadband or 1/3 octave band noise at 7 centre frequencies from 0.5 kHz to 6 kHz. The subjective responses compared well to the model prediction across frequency except for some cases where the image location was overestimated. Also, the prediction for the localisation of broadband phantom images agreed well with the test results, where the model prediction was integrated across frequency according to a tentatively suggested weighting function. Although the neuroscientific background is weak for the model, the good agreement with the subjective responses suggests that the model is worth investigating further.

The MPEG Interactive Music Application Format Standard [Standards in a Nutshell]

The music industry is going through a transformation, and new interactive music services have emerged. It is envisaged that this new concept of digital music content will dominate the next generation of music services. A standardized file format is inevitably required to provide the interoperability between various interactive music players and interactive music albums. This issue is addressed in a new standard by the Moving Picture Experts Group (MPEG), known as the MPEG-A Interactive Music Application Format (IM AF). IM AF integrates multiple audio tracks with appropriate additional information, enabling users to experience various preset mixes and to make their own mixes complying with interactivity rules imposed by the music composers with the aim of fitting their artistic creation.

Content filtering for broadcasting media

In modern digital broadcasting environment, broadcasting content filtering could provide a useful function that a TV viewer can find or store personal desired scenes from programs of multiple channels and it can be done even when one is watching a program from other channel. To achieve the filtering function in live broadcast, real-time processing is needed basically. In this paper, a broadcasting content filtering algorithm is proposed and filtering system requirements for the real-time content filtering are analyzed. To achieve real-time content processing, a buffer control algorithm is proposed as well. The usefulness of the broadcasting content filtering is demonstrated with experiments on a test-bed system.