Ming-eh Chi

ROI video coding based on H.263+ with robust skin-color detection technique

The ability to give higher priority to regions-of-interest (ROI) is the emerging functionality for present day video coding. A simple and fast method for face detection is proposed to define ROIs dynamically in real time applications. We use the color information Cr and RGB variance to determine the skin-color pixels. Because the two color spaces are commonly used in most hardware and video codec standards, there is no extra computation overhead required for conversion. Low-pass filtering is applied to the background to reduce used bits. For the video coding system, a region-based video codec based on H.263+ with the option mode of modified quantization is set up. We adjust the distortion weight parameter and variance at the macroblock layer to control the qualities at different regions. Experimental results show that the proposed methods can significantly improve quality at the ROI. Our methods are suitable for real time videoconferencing.

Efficient multi-frame motion estimation algorithms for MPEG-4 AVC/JVT/H.264

In H.264 standard, a lot of computational complexity is consumed in the encoder for motion estimation. It allows seven block sizes to perform the motion/compensation, and refers to previous five frames for searching the best motion vector to achieve lower bitrate and higher quality. Since the H.264 reference software uses the full search scheme to obtain the best performance, it spends a lot of searching time. In this paper we propose efficient searching algorithms by reuse of the motion vector information from the last reference frame. The proposed algorithms use the stored motion vectors to compose the current motion vector without performing the full search in each reference frame. Therefore, our proposed algorithms can obtain the speed up ratio 4 in average for encoding, which benefits from the prediction of the motion vector for reference frames in advance and maintain a good performance. Any fast search algorithm can be utilized to further largely reduce the computational load for the motion estimation from previous one frame.

Temporal error concealment algorithm by recursive block-matching principle

Compressed video bitstream is sensitive to channel errors that may degrade the reconstructed pictures severely. Error concealment is an effective approach to reduce the impact of errors, and it is implemented at the decoders. Its aim is to mask the effect of the missing blocks to create image or video of subjectively acceptable quality by using the spatial or/and temporal correlation. This paper proposes a temporal error concealment method that exploits the temporal redundancy of a video sequence using a block-matching approach. A recursive block-matching algorithm, based on the two-step block-matching principle, is presented to refine the concealed video. The proposed algorithm can also conceal the multiple erroneous slices. The proposed algorithm outperforms the algorithms presented in previous studies.

Region-of-interest video coding based on rate and distortion variations for H.263+

Region-of-interest (ROI) is an essential task that one must undertake in low bit-rate multimedia communications because of the limited bandwidth of the channels and the transcoder between different standards. In this paper, an effective ROI determination method is proposed. The skin-color extraction is first employed to determine the ROI macroblocks; then, the framework for the adjustment of the quantization parameters (QPs) of ROI macroblocks according to the distortion and bit-rate variations is proposed to fit the target bit rate. In view of the residual distortion information, the fuzzy logic controller proposed in our framework can adaptively adjust the weighting factor for corresponding QPs by the distortion variation in the macroblock layer. A linear prediction formula, derived from the rate variation, is proposed to allocate appropriate bits for each ROI macroblock and maintain the target bit rate and buffer fullness. Experiment results on TMN8 reveal that the proposed ROI video coding can significantly enhance the quality at ROIs. Furthermore, the proposed framework obtains about 0.5dB gain in the objective performance and also provides the better subjective quality compared with previous works.

Fast Mode Decision Algorithm Through Inter-View Rate-Distortion Prediction for Multiview Video Coding System

Multiview video coding (MVC) has attracted great attention from industries and research institutes. MVC is used to encode stereoscopic video streams for 3D playout systems such as 3D television, digital cinema, and IP network applications. MVC is an extended version of H.264/AVC that improves the performance of multiview videos. Yet, when compared with single-view video coding, MVC consumes much more time when encoding large amounts of data. Speed-up algorithms, therefore, are essential for realizing related applications. This paper presents a fast mode decision algorithm to avoid the high computational complexity of MVC. The proposed approach aims to reduce candidate modes and make mode decision process more efficient. The minimum and maximum values of rate-distortion cost (RD cost) in the previously encoded view are used to compute a threshold for each mode in the current view. Compared with joint multiview video coding, the experimental results demonstrate that the proposed algorithm provides an average of 79% in time savings with negligible bit rate increase and peak signal-to-noise ratio decrease.

Robust Region-of-Interest Determination Based on User Attention Model Through Visual Rhythm Analysis

Region-of-interest (ROI) determination is very important for video processing and it is desirable to find a simple method to identify the ROI. Along this direction, this paper investigates a user attention model based on visual rhythm analysis for automatic determination of ROI in a video. The visual rhythm, which is an abstraction of a video, is a thumbnail version of a video by a 2-D image that captures the temporal information of a video sequence. Four sampling lines, including diagonal, anti-diagonal, vertical, and horizontal lines, are employed to obtain four visual rhythm maps in order to analyze the location of the ROI from video data. Via the variation on visual rhythms, object and camera motions can be efficiently distinguished. As for hardware design consideration, the proposed scheme can accurately extract ROI with very low computational complexity for real-time applications. The promising results from the experiments demonstrate that the moving object is effectively and efficiently extracted. Finally, we present a way to use flexible macroblock ordering in combination with ROI determination as a preprocessing step for H.264/AVC video coding, and experimental results show the quality of ROI regions is significantly enhanced.

Computation-scalable algorithm for scalable video coding

This paper presents a strategy that efficiently allocates the limited resource and still maintains the coding performance for the spatial scalability of H.264/SVC. An integrated computation-scalable algorithm is proposed based on mode decision and adaptive search range. The RD-cost of the base layer is used as the priority to decide the mode in the enhancement layer, and the motion vector difference in the base layer is employed to determine the search range as well. Through the proposed approach, computation complexity of the enhancement layer could be reduced efficiently and well allocated. Experimental results demonstrate that the saving time reaches up to 82% and the computation scalability is realized.

Prediction Error Prioritizing Strategy for Fast Normalized Partial Distortion Motion Estimation Algorithm

A prediction error prioritizing-based normalized partial distortion search algorithm for fast motion estimation is proposed in this letter. The distortion behavior of each pixel in a macroblock is first analyzed to point out the priority/order of sum of absolute difference calculation. Afterward, the normalized partial distortion search algorithm is applied for half-stop of the distortion calculation. In addition, a dynamic search range decision algorithm is adopted for automatically changing the size of the search range to further increase the motion estimation speed. The computational complexity can be reduced significantly through the proposed algorithm, though leaving a PSNR degradation that could be dismissed.

Region-of-Interest video coding by fuzzy control for H.263+ standard

This paper proposes an Region-of-Interest(ROI) video coding algorithm based on H.263+. The ability to give higher priority to ROI is the emerging functionality for video coding nowadays. In most videoconferencing and videophony, most existing compression schemes treat the entire image uniformly. However, the background is much less critical than the foreground, i.e., human face, moving objects, etc., for communication. A region-based video codec based on the H.263+ with the optional mode of modified quantization is set up. A new algorithm based on TMN8 utilizes a fuzzy logic control method to give adaptively weighted factors to each macroblock(MB) and allocate different bits to each MB according to the complexity and the variance variation in the rate-and-distortion model. From experimental results, the proposed method can significantly improve the quality at ROI and multiple ROI functionality also has been implemented. Our method is suitable for real time videoconferencing.

High performance error concealment algorithm by motion vector refinement for MPEG-4 video

A new error concealment algorithm is proposed using recursive motion vector refinement. The proposed method utilizes the top/bottom motion vectors of lost macroblocks in current and reference frames and refines motion vectors recursively. Simulation results based on the MPEG-4 codec present a superior subjective and objective performance of the proposed technique compared with conventional temporal concealment techniques.