Ming-Ting Sun

A family of VLSI designs for the motion compensation block-matching algorithm

A family of modular VLSI architectures and chip implementations of the motion-compensation full-search block-matching algorithm are described. This set of application-specific integrated circuits is motivated by the intensive computations required to perform motion compensation in real time. The architectures are based on data-flow designs, which allow sequential inputs but perform parallel processing with 100% efficiency. On the basis of these architectures, a programmable chip can be designed for motion vector estimation with different block sizes. The chips can be cascaded for a larger tracking range or for a video source with a higher pixel sampling rate. A chip-pair design is also derived for calculating fractional motion vectors with quarter-pel precision. The chip-pair design has been laid out, and the chip characteristics are given. Test circuitry is also included to increase the testability of the chips. >

Digital Video Transcoding

Video transcoding, due to its high practical values for a wide range of networked video applications, has become an active research topic. We outline the technical issues and research results related to video transcoding. We also discuss techniques for reducing the complexity, and techniques for improving the video quality, by exploiting the information extracted from the input video bit stream.

Motion vector refinement for high-performance transcoding

In transcoding, simply reusing the motion vectors extracted from an incoming video bit stream may not result in the best quality. In this paper, we show that the incoming motion vectors become nonoptimal due to the reconstruction errors. To achieve the best video quality possible, a new motion estimation should be performed in the transcoder. We propose a fast-search adaptive motion vector refinement scheme that is capable of providing video quality comparable to that can be achieved by performing a new full-scale motion estimation but with much less computation. We discuss the case when some incoming frames are dropped for frame-rate conversions, and propose motion vector composition method to compose a motion vector from the incoming motion vectors. The composed motion vector can also be refined using the proposed motion vector refinement scheme to achieve better results.

An entropy coding system for digital HDTV applications

Run-length coding (RLC) and variable-length coding (VLC) are widely used techniques for lossless data compression. A high-speed entropy coding system using these two techniques is considered for digital high definition television (HDTV) applications. Traditionally, VLC decoding is implemented through a tree-searching algorithm as the input bits are received serially. For HDTV applications, it is very difficult to implement a real-time VLC decoder of this kind due to the very high data rate required. A parallel structured VLC decoder which decodes each codeword in one clock cycle regardless of its length is introduced. The required clock rate of the decoder is thus lower, and parallel processing architectures become easy to adopt in the entropy coding system. The parallel entropy coder and decoder are designed for implementation in two experimental prototype chips which are designed to encode and decode more than 52 million samples/s. Some related system issues, such as the synchronization of variable-length codewords and error concealment, are also discussed. >

Modeling DCT coefficients for fast video encoding

Digital video coding standards such as H.263 and MPEG are becoming more and more important for multimedia applications. Due to the huge amount of computations required, there are significant efforts to speed up the processing of video encoders. Previously, the efforts were mainly focused on the fast motion-estimation algorithm. However, as the motion-estimation algorithm becomes optimized, to speed up the video encoders further we also need to optimize other functions such as the discrete cosine transform (DCT) and inverse DCT (IDCT). In this paper, we propose a theoretical model for DCT coefficients. Based on the model, we develop an adaptive algorithm to reduce the computations of DCT, IDCT, quantization, and inverse quantization. We also present a fast DCT algorithm to speed up the calculations of DCT further when the quantization step size is large. We show, by simulations, that significant improvement in the processing speed can be achieved with negligible video-quality degradation, We also implement the algorithm in a real-time PC-based platform to show that it is effective and practical.

Fast multiple reference frame motion estimation for H.264/AVC

Multiple reference frame motion compensation is a new feature introduced in H.264/MPEG-4 AVC to improve video coding performance. However, the computational cost of multiple reference frame motion estimation (MRF-ME) is very high. In this paper, we propose an algorithm that takes into account the correlation/continuity of motion vectors among different reference frames. We show that the algorithm effectively reduces the computations of MRF-ME, and achieves similar coding gain compared to the motion search approaches in the reference software

Robust Detection of Abandoned and Removed Objects in Complex Surveillance Videos

Tracking-based approaches for abandoned object detection often become unreliable in complex surveillance videos due to occlusions, lighting changes, and other factors. We present a new framework to robustly and efficiently detect abandoned and removed objects based on background subtraction (BGS) and foreground analysis with complement of tracking to reduce false positives. In our system, the background is modeled by three Gaussian mixtures. In order to handle complex situations, several improvements are implemented for shadow removal, quick-lighting change adaptation, fragment reduction, and keeping a stable update rate for video streams with different frame rates. Then, the same Gaussian mixture models used for BGS are employed to detect static foreground regions without extra computation cost. Furthermore, the types of the static regions (abandoned or removed) are determined by using a method that exploits context information about the foreground masks, which significantly outperforms previous edge-based techniques. Based on the type of the static regions and user-defined parameters (e.g., object size and abandoned time), a matching method is proposed to detect abandoned and removed objects. A person-detection process is also integrated to distinguish static objects from stationary people. The robustness and efficiency of the proposed method is tested on IBM Smart Surveillance Solutions for public safety applications in big cities and evaluated by several public databases, such as The Image library for intelligent detection systems (i-LIDS) and IEEE Performance Evaluation of Tracking and Surveillance Workshop (PETS) 2006 datasets. The test and evaluation demonstrate our method is efficient to run in real-time, while being robust to quick-lighting changes and occlusions in complex environments.

A rate-control scheme for video transport over wireless channels

We investigate the scenario of using the automatic repeat request (ARQ) retransmission scheme for two-way video communications over wireless Rayleigh fading channels. Video quality is the major concern of these applications. We show that, during the retransmissions of error packets, due to the reduced channel throughput, the video encoder buffer may fill-up quickly and cause the TMN8 rate-control algorithm to significantly reduce the bits allocated to each video frame. This results in PSNR degradation and many skipped frames. To minimize the number of frames skipped, we propose improved rate-control schemes that take into consideration the effects of the video buffer fill-up, an a priori channel model, and the channel feedback information. We also show that a discrete cosine transform (DCT) coefficient soft-thresholding scheme can be applied to further improve video quality. As a result, our proposed rate-control schemes encode the video sequences with less frame skipping and with higher PSNR compared to TMN8.

Fast variable block-size motion estimation algorithm based on merge and slit procedures for H.264/MPEG-4 AVC

This paper proposes and investigates fast variable block-size motion estimation algorithms based on merge and split procedure for H.264/MPEG-4 AVC video encoding. The algorithms take advantage of the correlation of the motion vectors (MVs) of the different block-size modes, to achieve good computation reduction. Experimental results show that the number of search point can be reduced to about 4% of that using full-search motion estimation, with negligible quality degradation.

Fast multiple reference frame motion estimation for H.264

Multiple reference frame motion compensation is a new feature introduced in H.264/MPEG-4 AVC to improve video coding performance. However, the computational cost of multiple reference frame motion estimation (MRF-ME) is very high. We propose an algorithm that takes into account the correlation/continuity of motion vectors among different reference frames. We also show that the algorithm effectively reduces the computations of MRF-ME, and achieves similar coding gain compared to the full-search approach.