Munchurl Kim

Moving object segmentation in video sequences by user interaction and automatic object tracking

Abstract The new MPEG-4 video coding standard consists of object based coding schemes for multimedia and enables content based functionalities. In order to apply the MPEG-4 visual standard, each frame of video sequences should be represented in terms of video object planes (VOPs). Video objects in still pictures or video sequences should be first identified before the encoding process starts. It requires a prior decomposition of sequences into VOPs so that each VOP represents an object. This paper addresses a user-assisted video segmentation method for separating moving objects from the background in video sequences. The proposed method comprises of intra-frame and inter-frame segmentation modules. The intra-frame segmentation incorporates user interaction in defining a high level semantic object of interest to be segmented and detects precise object boundary. The inter-frame segmentation involves boundary and region tracking to capture temporal coherence of moving objects with accurate object boundary information. The proposed method exhibits a consistent and faithful segmentation performance for various types of video sequences.

Moving object tracking in H.264/AVC bitstream

Data broadcasting services are required to provide user interactivity through connecting additional contents such as object information to audio-visual contents. H.264/AVC-based metadata authoring tools include functions which identify and track position and motion of objects. In this work, we propose a method for tracking the target object by using partially decoded texture data and motion vectors extracted directly from H.264/AVC bitstream. This method achieves low computational complexity and high performance through the dissimilarity energy minimization algorithm which tracks feature points adaptively according to these characteristics. The experiment has shown that the proposed method had high performance with fast processing time.

A target advertisement system based on TV viewer's profile reasoning

The traditional broadcasting services such as terrestrial, satellite and cable broadcasting have been unidirectional mass media regardless of TV viewer’s preferences. Recently rich media streaming has become possible via the broadband networks. Furthermore, since bidirectional communication is possible, personalcasting such as personalized streaming service has been emerging by taking into account the user’s preference on content genres, viewing times and actors/actresses etc. Accordingly personal media becomes an important means for content provision service in addition to the traditional broadcasting service as mass media. In this paper, we introduce a user profile reasoning method for TV viewers. The user profile reasoning is made in terms of genre preference and TV viewing times for TV viewer’s groups in different genders and ages. For user profiling reasoning, the TV viewing history data is used to train the proposed user profiling reasoning algorithm which allows for target advertisement for different age/gender groups. To show the effectiveness of our proposed user profile reasoning method, we present plenty of the experimental results by using real TV usage history.

Summarization of news video and its description for content‐based access

A video summary abstracts the entirety with the gist without losing the essential content of the original video and also facilitates efficient content‐based access to the desired content. In this article, we propose a novel method for summarizing a news video based on multimodal analysis of the content. The proposed method exploits the closed caption (CC) data to locate semantically meaningful highlights in a news video and speech signals in an audio stream to align the CC data with the video in a time‐line. Then, the extracted highlights are described in a multilevel structure using the MPEG‐7 Summarization Description Scheme (DS). Specifically, we use the HierarchicalSummary DS that allows efficient accessing of the content through such functionalities as multilevel abstracts and navigation guidance in a hierarchical fashion. Intensive experiments with our prototypical systems are presented to demonstrate the validity and reliability of the proposed method in real applications.

Real-time detection and tracking of multiple objects with partial decoding in H.264|AVC bitstream domain

In this paper, we show that we can apply probabilistic spatiotemporal macroblock filtering (PSMF) and partial decoding processes to effectively detect and track multiple objects in real time in H.264|AVC bitstreams with stationary background. Our contribution is that our method cannot only show fast processing time but also handle multiple moving objects that are articulated, changing in size or internally have monotonous color, even though they contain a chaotic set of non-homogeneous motion vectors inside. In addition, our partial decoding process for H.264|AVC bitstreams enables to improve the accuracy of object trajectories and overcome long occlusion by using extracted color information.

A Metadata Schema Design on Representation of Sensory Effect Information for Sensible Media and its Service Framework using UPnP

With advent of various media services and development of audio and video devices, we can enjoy the media more effectively and realistically. Conventional media content is presented mostly via speakers, TV and LCD monitors. Beyond the media rendering only, if the media contents interlink with peripheral devices when being playbacked, it is possible to make fascinating effects on audiovisual media contents. In this paper, we suggest a device rendered sensible media and metadata schema for representing the effect and control information and design a service framework for device rendered sensible media based on UPnP framework.

Automatic user preference learning for personalized electronic program guide applications

In this article, we introduce a user preference model contained in the User Interaction Tools Clause of the MPEG-7 Multimedia Description Schemes, which is described by a UserPreferences description scheme (DS) and a UsageHistory description scheme (DS). Then we propose a user preference learning algorithm by using a Bayesian network to which weighted usage history data on multimedia consumption is taken as input. Our user preference learning algorithm adopts a dynamic learning method for learning real-time changes in a users preferences from content consumption history data by weighting these choices in time. Finally, we address a user preference-based television program recommendation system on the basis of the user preference learning algorithm and show experimental results for a large set of realistic usage-history data of watched television programs. The experimental results suggest that our automatic user reference learning method is well suited for a personalized electronic program guide (EPG) application.

Target advertisement service using TV viewers’ profile inference

Due to the limitation of broadcasting service, in general, TV programs with commercial advertisements are scheduled to be broadcasted by demographics. The uniformly provided commercial can not draw many TV viewers’ interest, which is not correspondent to the goal of the commercial. In order to solve the problem, a novel target advertisement technique is proposed in this paper. The target advertisement is a personalized advertisement according to TV viewers’ profile such as their age, gender, occupation, etc. However, viewers are usually reluctant to inform their profile to the TV program provider or the advertisement company because their information can be used on some bad purpose by unknown people. Our target advertisement technique estimates a viewer’s profile using Normalized Distance Sum and Inner product method. In the experiment, our method is evaluated for estimating the TV viewers’ profile using TV usage history provided by AC Neilson Korea.

Modeling the user preference on broadcasting contents using Bayesian networks

In this paper, we introduce a new supervised learning method of a Bayesian network for user preference models. Unlike other preference models, our method traces the trend of a user preference as time passes. It allows us to do online learning so we do not need the exhaustive data collection. The tracing of the trend can be done by modifying the frequency of attributes in order to force the old preference to be correlated with the current preference under the assumption that the current preference is correlated with the near future preference. The objective of our learning method is to force the mutual information to be reinforced by modifying the frequency of the attributes in the old preference by providing weights to the attributes. With developing mathematical derivation of our learning method, experimental results on the learning and reasoning performance on TV genre preference using a real set of TV program watching history data.

An efficient block mode decision for temporal scalability in scalable video coding

In this paper, a fast intermode decision scheme which is suitable for the hierarchical B-picture structure in which much computational power is spent for combined variable block sizes and bi-predictive motion estimation is introduced. The hypothesis testing considering the characteristics of the hierarchical B-picture structure in the proposed method is performed on 16x16 and 8x8 blocks to have early termination for RD computation of all possible modes. The early termination in intermode decision is performed by comparing the pixel values of current blocks and corresponding motion-compensated blocks. When the hypothesis tests are performed, the confidence intervals to accept the null hypothesis or not are decided according to the temporal scalability levels under the consideration of properties of hierarchical B-pictures. The proposed scheme exhibits effective early termination behavior in intermode decision of temporal scalabilities and leads to a significant reduction up to 69% in computational complexity with slight increment in bit amounts. The degradation of visual quality turns out to be negligible in terms of PSNR values.