Wei Zeng

Robust moving object segmentation on H.264/AVC compressed video using the block-based MRF model

Moving object segmentation in compressed domain plays an important role in many real-time applications, e.g. video indexing, video transcoding, video surveillance, etc. Because H.264/AVC is the up-to-date video-coding standard, few literatures have been reported in the area of video analysis on H.264/AVC compressed video. Compared with the former MPEG standard, H.264/AVC employs several new coding tools and provides a different video format. As a consequence, moving object segmentation on H.264/AVC compressed video is a new task and challenging work. In this paper, a robust approach to extract moving objects on H.264/AVC compressed video is proposed. Our algorithm employs a block-based Markov Random Field (MRF) model to segment moving objects from the sparse motion vector field obtained directly from the bitstream. In the proposed method, object tracking is integrated in the uniform MRF model and exploits the object temporal consistency simultaneously. Experiments show that our approach provides the remarkable performance and can extract moving objects efficiently and robustly. The prominent applications of the proposed algorithm are object-based transcoding, fast moving object detection, video analysis on compressed video, etc.

A robust text detection algorithm in images and video frames

In this paper, an algorithm for detecting text in images and video frames is proposed. The algorithm contains two steps: initial detection and verification. In the first step, edge feature and morphology operation are employed to locate edge-dense image blocks. Empirically rules are applied on these blocks to get candidate text. In the second step, wavelet-based features are employed to represent the texture property of text. A SVM classifier is used to identify text from the candidate ones. Experiments show that this algorithm has 93.9% detection rate for English text and a 92.4% detection rate for Chinese text. The algorithm is robust to language, font-color and size.

Color image segmentation using density-based clustering

Color image segmentation is an important but still open problem in image processing. We propose a method for this problem by integrating the spatial connectivity and color features of the pixels. Considering that an image can be regarded as a dataset in which each pixel has a spatial location and a color value, color image segmentation can be obtained by clustering these pixels into different groups of coherent spatial connectivity and color. To discover the spatial connectivity of the pixels, density-based clustering is employed, which is an effective clustering method used in data mining for discovering spatial databases. The color similarity of the pixels is measured in Munsell (HVC) color space whose perceptual uniformity ensures the color change in the segmented regions is smooth in terms of human perception. Experimental results using the proposed method demonstrate encouraging performance.

Shape-based adult images detection

This paper reports an investigation on adult images detection based on the shape features of skin regions. In order to accurately detect skin regions, we propose a skin detection method using multi-Bayes classifiers in the paper. Based on skin color detection results, shape features are extracted and fed into a boosted classifier to decide whether or not the skin regions represent a nude. We evaluate adult image detection performance using different boosted classifiers and different shape descriptors. Experimental results show that classification using boosted C4.5 classifier and combination of different shape descriptors outperforms other classification schemes.

SHAPE-BASED ADULT IMAGE DETECTION

This paper investigates adult images detection based on the shape features of skin regions. In order to accurately detect skin regions, we propose a skin detection method using multi-Bayes classifiers in the paper. Based on skin color detection results, shape features are extracted and fed into a boosted classifier to decide whether or not the skin regions represent a nude. We evaluate adult image detection performance using different boosted classifiers and different shape descriptors. Experimental results show that classification using boosted C4.5 classifier and combination of different shape descriptors outperforms other classification schemes.

Adaptive relevance feedback based on Bayesian inference for image retrieval

Relevance feedback can be considered as a Bayesian classification problem. For retrieving images efficiently, an adaptive relevance feedback approach based on the Bayesian inference, rich get richer (RGR), is proposed. If the feedback images in current iteration are consistent with the previous ones, the images that are similar to the query target are assigned to high probabilities. Therefore, the images that are similar to the users ideal target are emphasized step by step. The experiments showed that the average precision of RGR improves 5-20% on each interaction compared with non-RGR. When compared with MARS. the proposed approach greatly reduces the users efforts for composing a query and captures users, intention efficiently

Automatic moving object extraction in MPEG video

In this paper, we propose a moving object extraction technique for MPEG coded data directly. It is a change-based motion object extraction approach, which discriminates background and moving objects by means of the higher-order statistics (HOS) performed on the interframe differences of DC image. The DC image is partly decoded picture from the compressed video for the rapid reconstruction of image data. In order to employ an optimal threshold in moving object detection stage, the background is detected by the Moment-preserving thresholding technique for each frame. Based on the background statistic, the proportion of background variance is employed to extract the final object mask by comparison with the fourth moment measure and the variance. Experimental results have demonstrated that the proposed approach worked efficiently and showed a robust result for object extraction in compressed video.

Shot change detection on H.264/AVC compressed video

This paper deals with the problem of shot-change detection on H.264/AVC compressed video. As H.264/AVC employs several new coding tools, the statistic information of the macroblock is different from the former MPEG video. How to detect shot-changes efficiently on H.264/AVC compressed video is a challenging work. In this paper, we propose a novel scheme, a macroblock type analysis plus the intra-mode statistical constraint rule, to detect shot-changes for P-frame and B-frame coded video of H.264/AVC. For detecting shot-changes in I-frame coded video, we propose an intra mode histogram and use the weighted city block distance to measure the similarities among I-frames. Experimental results show that the proposed algorithms achieve satisfactory performance and very fast detection.

Video indexing by motion activity maps

Motion based video indexing is an important and active research area in content-based video retrieval. It explores the dynamic characteristics of video content and provides techniques for video representation and retrieval. In this paper, a new motion-based approach is proposed, in which the image generated from motion activity is used to index video. The proposed approach firstly computes the accumulation measurement of motion activity on the grids of video frames along the time axis. Then, the computed measurement is quantized into several gray levels and a gray image is generated from all measurements on the grids. The generated image called motion activity map (MAM) reserves the motion information on different spatial locations of video. The intensity of MAM is corresponding to the magnitude of motion activity. All MAMs are organized into a hierarchical tree according to the video structure. Therefore, user can browse video by the MAMs tree. The proposed approach provides a hierarchical, coarse to fine view of video and thus makes interactive video retrieval more simply and intuitively.

A novel compressed domain shot segmentation algorithm on H.264/AVC

This paper presents a novel shot segmentation algorithm on the H.264/AVC video, which operates in the compressed domain. First, the algorithm exploits the intra prediction mode histogram to locate those potential GOPs, where shot transitions occur with great probability. Secondly, to further find shot boundaries at the frame level, we count the number of macroblocks with different inter prediction modes as the features and exploit HMMs to automatically model different cases in which shot transitions can occur among I, P and B frames. Since H.264/AVC provides more motion compensation modes, using HMMs can avoid the tediousness of manually tuning multiple thresholds simultaneously. The experimental results show that the algorithm is efficient and robust and it can not only locate cuts, but also work for gradual shot transitions.