Daehee Kim

A VOP generation tool: automatic segmentation of moving objects in image sequences based on spatio-temporal information

The new MPEG-4 video coding standard enables content-based functionalities. In order to support the philosophy of the MPEG-4 visual standard, each frame of video sequences should be represented in terms of video object planes (VOPs). In other words, video objects to be encoded in still pictures or video sequences should be prepared before the encoding process starts. Therefore, it requires a prior decomposition of sequences into VOPs so that each VOP represents a moving object. This paper addresses an image segmentation method for separating moving objects from the background in image sequences. The proposed method utilizes the following spatio-temporal information. (1) For localization of moving objects in the image sequence, two consecutive image frames in the temporal direction are examined and a hypothesis testing is performed by comparing two variance estimates from two consecutive difference images, which results in an F-test. (2) Spatial segmentation is performed to divide each image into semantic regions and to find precise object boundaries of the moving objects. The temporal segmentation yields a change detection mask that indicates moving areas (foreground) and nonmoving areas (background), and spatial segmentation produces spatial segmentation masks. A combination of the spatial and temporal segmentation masks produces VOPs faithfully. This paper presents various experimental results.

Color Image Segmentation Using Anisotropic Diffusion and Agglomerative Hierarchical Clustering

A new color image segmentation scheme is presented in this paper. The proposed algorithm consists of image simplification, region labeling and color clustering. The vector-valued diffusion process is performed in the perceptually uniform LUV color space. We present a discrete 3-D diffusion model for easy implementation. The statistical characteristics of each labeled region are employed to estimate the number of total clusters and agglomerative hierarchical clustering is performed with the estimated number of clusters. Since the proposed clustering algorithm counts each region as a unit, it does not generate oversegmentation along region boundaries.

Image Segmentation Based on Shape Space Modeling

In this paper, we propose a new image segmentation method based on the active contour. If we define a shape space as a set of all possible variations from the initial curve and we assume that the shape space is linear, it can be decomposed into the column space and the left null space of the shape matrix. In the proposed method, the shape space vector in the column space describes changes from the initial curve to the imaginary feature curve, and a dynamic graph search algorithm describes the detailed shape of the object in the left null space. Since we employ the shape matrix and the SUSAN operator to outline object boundaries, the proposed algorithm can ignore unwanted feature points generated by low-level image processing operations and is therefore applicable to images of the complex background. We can also compensate for limitations of the shape matrix with the dynamic graph search algorithm.

A method for blocking effect reduction based on optimal filtering

In block-based coding schemes, the input image is segmented into small blocks that are processed independently; therefore, blocking effects occur along block boundaries. Various methods have been developed to reduce such blocking effects. In this paper, we propose a method for blocking effect reduction based on optimal filtering, and we compare its performance with those of others.

User-assisted segmentation algorithm using B-spline curves

Most automatic segmentation techniques have difficulties in extracting individual video objects in a single frame. They are somewhat premature to obtain desirable segmentation results from various kinds of image sequences. However, if the user can provide semantic information of video objects for the first frame in a user-assisted manner, improved segmentation results can be obtained in the following picture frames. The user-assisted or semi-automatic approach for video segmentation is more practical in generating VOPs of moving objects. In this paper, we propose a new semi-automatic video segmentation scheme using an active contour algorithm. In most active contour algorithms, samples of the contour convey insufficient information about the curve shape between samples due to finite difference approximation. In addition, since most active contour algorithms have been developed for images of homogeneous simple background, they are not applicable to moving objects in the complex background. Therefore, this paper addresses the B-spline representation of curves and the morphological feature extraction for video object segmentation.

Blocking effect reduction based on optimal filtering

In block-based coding schemes, the input image is segmented into small blocks that are processed independently: therefore, blocking effects occur along block boundaries. Various methods have been developed to reduce such blocking effects. In this paper, we propose a new blocking effect reduction method based on optimal filtering and we compare its performance with those of others.

A rate control algorithm using macroblock types

ISO/IEC MPEG-3 Test Model 5 (TM5) describes a rate control scheme which consists of three steps: bit allocation, rate control and modulation. However, there are basically two problems with the TM5 rate control. First, macroblocks in a picture may not all be treated equally. Secondly, the TM5 rate control does not handle scene changes properly. Although several schemes have been developed to solve these problems of TM5, none of them exploits the information of the macroblocks to be coded. We propose a new rate control strategy by using coded types of macroblocks within a picture. With the proposed algorithm, we can get a more consistent picture quality, and avoid buffer overflow and underflow.

Video segmentation using vector-valued diffusion and clustering

In this paper, we propose a user-assisted video segmentation algorithm based on color information to alleviate oversegmentation problems. We perform intra-frame segmentation by image simplification, region labeling, and color clustering. In this paper, we also present a discrete three dimensional diffusion model for easy implementation. The statistical property of each labeled region is used to estimate the number of total clusters, and agglomerative hierarchical clustering is performed with the estimated number of clusters. Since the proposed clustering algorithm counts each region as a unit, it does not generate oversegmentation problems along region boundaries. For inter-frame segmentation, we employ a look-up table for foreground color clusters, track the foreground regions, and utilize those information to extract moving objects.

Predefined object tracking method for video segmentation

In order to support the philosophy of the MPEG-4 video coding standard, we have to represent each frame of video sequences in terms of video object planes (VOPs). Several automatic methods for segmenting of moving objects have been developed. Such algorithms separate the foreground from the background with a change detection mask, which is obtained by the difference image between two successive frames. Thus, these techniques cannot represent each individual video object in a single frame separately, i.e., object correspondence problem. In addition, those algorithms are somewhat premature to obtain desirable segmentation results from all kinds of image sequences because the mathematical model or the similarity measure for the extraction of the video object has not been defined adequately. However, if the user can define video objects in the first frame or newly appeared video objects by a partially or completely user-assisted method like the snakes algorithm, we may obtain good segmentation results over the following successive frames. This semi-automatic segmentation may be more practical in generating VOPs of moving objects. In this paper, we propose a new user-assisted video segmentation algorithm. This algorithm consists of two steps: intra-frame segmentation and inter-frame segmentation. The intra-frame segmentation is applied to the first frame of the image sequence or the frames that have newly appeared video objects. The user can manually define the newly appeared video objects in the image sequence. The inter-frame segmentation is applied to the following consecutive frames. In the inter-frame segmentation, user-defined video objects are segmented automatically by object tracking.