Mei-Yin Shen

Rate Control for H.264 Video With Enhanced Rate and Distortion Models

A new rate control scheme for H.264 video encoding with enhanced rate and distortion models is proposed in this work. Compared with existing H.264 rate control schemes, our scheme has offered several new features. First, the inter-dependency between rate-distortion optimization (RDO) and rate control in H.264 is resolved via quantization parameter estimation and update. Second, since the bits of the header information may occupy a larger portion of the total bit budget, which is especially true when being coded at low bit rates, a rate model for the header information is developed to estimate header bits more accurately. The number of header bits is modeled as a function of the number of nonzero motion vector (MV) elements and the number of MVs. Third, a new source rate model and a distortion model are proposed. For this purpose, coded 4 times 4 blocks are identified and the number of source bits and distortion are modeled as functions of the quantization stepsize and the complexity of coded 4 times 4 blocks. Finally, a R-D optimized bit allocation scheme among macroblocks (MBs) is proposed to improve picture quality. Built upon the above ideas, a rate control algorithm is developed for the H.264 baseline-profile encoder under the constant bit rate constraint. It is shown by experimental results that the new algorithm can control bit rates accurately with the R-D performance significantly better than that of the rate control algorithm implemented in the H.264 software encoder JM8.1a

Artifact reduction in low bit rate wavelet coding with robust nonlinear filtering

A postprocessing algorithm for compression artifact reduction in low-bit-rate wavelet coding is proposed in this work. We first formulate the artifact reduction problem as a robust estimation problem. Under this framework, the artifact-free image is obtained by minimizing a cost function that accounts for the smoothness constraint as well as image fidelity. Unlike the traditional approach that adopts gradient descent search for optimization, a set of nonlinear filters is used to calculate the approximate global minimum. The nonlinear filtering approach reduces the number of the objective function evaluation and increases the speed of convergence. It is shown by experimental results that the proposed approach can alleviate wavelet coding artifacts efficiently with a low computational cost.

A Robust Nonlinear Filtering Approach to Inverse Halftoning

A new blind inverse halftoning algorithm based on a nonlinear filtering technique of low computational complexity and low memory requirement is proposed in this research. It is called blind since we do not require the knowledge of the halftone kernel. The proposed scheme performs nonlinear filtering in conjunction with edge enhancement to improve the quality of an inverse halftoned image. Distinct features of the proposed approach include efficiently smoothing halftone patterns in large homogeneous areas, additional edge enhancement capability to recover the edge quality, and an excellent PSNR performance with only local integer operations and a small memory buffer.

Fast inter-prediction mode decision and motion search for H.264

A fast inter-prediction mode decision and motion search algorithm is proposed for the H.264 video coding standard. The multi-resolution motion estimation scheme and an adaptive rate-distortion model are employed with early termination rules to accelerate the search process. With the new algorithm, the amount of computation involved in the motion search can be substantially reduced. Experimental results show that the proposed algorithm can achieve a speed-up factor ranging from 60 to 150 times as compared to the full-search algorithm with little quality degradation

A Quad-Tree Decomposition Approach to Cartoon Image Compression

A quad-tree decomposition approach is proposed for cartoon image compression in this work. The proposed algorithm achieves excellent coding performance by using a unique quad-tree decomposition and shape coding method along with a GIF like color indexing technique to efficiently encode large areas of the same color, which appear in a cartoon-type image commonly. To reduce complexity, the input image is partitioned into small blocks and the quad-tree decomposition is independently applied to each block instead of the entire image. The LZW entropy coding method can be performed as a postprocessing step to further reduce the coded file size. It is demonstrated by experimental results that the proposed method outperforms several well-known lossless image compression techniques for cartoon images that contain 256 colors or less

Fast compression artifact reduction technique based on nonlinear filtering

A computationally efficient postprocessing technique to reduce compression artifacts in low-bit-rate video coding is presented in this research. Blocking and ringing effects are two major compression artifacts observed in the block DCT-based video codec, such as H.263, when the coding bit rate becomes low. Reduction of these artifacts can significantly improve the overall visual quality of decoded video. A new table-lookup method and a nonlinear filtering approach are adopted to remove the blocking and ringing effects, respectively. Experiments employing H.263+TMN8 anchor bit-streams are performed to show that the proposed postprocessing technique can alleviate the coding artifacts efficiently with a low computational complexity.

Techniques for flexible image/video resolution conversion with heterogeneous terminals

Multimedia capturing and display devices of different resolutions and aspect ratios can be easily connected by networks and, thus, there is a great need to develop techniques that facilitate flexible image/video format conversion and content adaptation among these heterogeneous terminals. Quality degradation due to downsampling, up-sampling, coding/decoding, and some content adaptation mechanism (say, image mosaicking) in the transmission process is inevitable. It is desirable that multimedia contents can be easily captured, displayed, and seamlessly composed. Challenges and techniques to achieve this goal are reviewed first. Then, two specific topics, i.e., image/video mosaicking and super resolution (SR) conversion, are highlighted. As compared with previous work developed for these problems, the challenge under the current context is to strike a balance between low computational complexity and high quality of resultant image/video. Several new developments along this line are discussed

Fast motion search with efficient inter-prediction mode decision for H.264

Abstract A fast inter-prediction mode decision and motion search algorithm is proposed for the H.264 video coding standard. The multi-resolution motion estimation scheme and an adaptive rate-distortion model are employed with early termination rules to accelerate the search process. With the new algorithm, the amount of computation involved in the motion search can be substantially reduced. Experimental results show that the proposed algorithm can achieve a speed-up factor ranging from 60 to 150 times as compared to the full-search algorithm with little quality degradation.

Real-time compression artifact reduction via robust nonlinear filtering

Low complexity postprocessing algorithms to reduce compression artifacts by using a robust nonlinear filtering approach and a table lookup method are investigated in this research. We first formulate the compression artifact reduction problem as a robust estimation problem. Under this framework, an enhanced image can be obtained by minimizing a cost function that accounts for the image smoothness as well as the image fidelity constraints. However, unlike traditional methods that adopt a gradient descent method to search for the optimal solution, we determine the approximate solution via the evaluation of a set of nonlinear cost functions. This nonlinear filtering process is performed to reduce the computational complexity of the postprocessing operation so that it can be implemented in real time. In the case of video postprocessing, a table lookup method is adopted to further reduce the complexity. The proposed approach is generic and flexible. It can be applied to different compression schemes with minor finetuning. We have tested the developed algorithm on several compressed video or images which are obtained by JPEG 2000 VM, and H.263+. It has been demonstrated that the proposed method can reduce compression artifacts efficiently with a low computational complexity.

Pixel- and compressed-domain color matching techniques for video mosaic applications

Several color matching algorithms are proposed to merge two or more video inputs of smaller sizes into one single video output of a larger size with a wider field of view for the video mosaic application. The main challenge is to remove apparent seam lines between image boundaries. All developed algorithms share the same basic idea with different implementation details. That is, color differences between input images are first compensated using either histogram equalization or polynomial-based contrast stretching techniques. Then, a linear filtering technique is adopted to remove seam lines between image boundaries. The algorithms are developed in both the pixel and the DCT domains. The compressed-domain processing is attractive since it reduces the computational complexity. It is shown by experimental results that the color matching problem can be solved satisfactorily even in the compressed domain.