Haoming Chen

New Transforms Tightly Bounded by DCT and KLT

It is well known that the discrete cosine transform (DCT) and Karhunen-Loève transform (KLT) are two good representatives in image and video coding: the first can be implemented very efficiently while the second offers the best R-D coding performance. In this work, we attempt to design some new transforms with two goals: i) approaching to the KLTs R-D performance and ii) maintaining the implementation cost no bigger than that of DCT. To this end, we follow a cascade structure of multiple butterflies to develop an iterative algorithm: two out of N nodes are selected at each stage to form a Givens rotation (which is equivalent to a butterfly); and the best rotation angle is then determined by maximizing the resulted coding gain. We give the closed-form solutions for the node-selection as well as the angle-determination, together with some design examples to demonstrate their superiority.

Improvements on Intra Block Copy in natural content video coding

The Intra Block Copy (IntraBC) is a newly adopted tool in the HEVC extension for the screen content video coding. The IntraBC tool efficiently encodes repeating patterns in a picture. The current IntraBC scheme achieves about 1.0% bit-rate reduction on average and up to 4.3% bitrate reduction on natural content video for a database consisting of 2K, 4K, and 8K sequences. In this paper, we propose to improve the IntraBC with a template matching block vector and a fractional search IntraBC. With these two tools, the gain on natural content video coding can be further improved by 0.5% on average and up to 2.0%.

Nearest-neighbor intra prediction for screen content video coding

Screen content video coding is becoming increasingly important in various applications, such as desktop sharing, video conferencing, and remote education. In general, compared to natural camera-captured content, screen content has different characteristics, such as sharp edges. In this paper, we propose a novel intra prediction scheme for screen content video. In the proposed scheme, bilinear interpolation in angular intra prediction in HEVC is selectively replaced by nearest-neighbor (NN) interpolation to preserve the sharp edges in screen content video. We present two different variants of NN interpolation. In the first implicit pixel-based method, both the encoder, and the decoder determine whether to perform NN interpolation based on the prediction pixels. The second method comprises of the encoder performing a Rate-Distortion search at a block-level, and explicitly signaling a flag to the decoder to indicate when to use the NN interpolation. Both the proposed variants provide significant gains over HEVC, and simulation results show that average gains of 3.3% BD-bitrate are achieved for screen content video. The HEVC proposal of this method was accepted in the core experiments, and would be a technology under consideration in the ongoing Screen Content Coding extension of HEVC scheduled to begin in March 2014.

Design of non-separable transforms for directional 2-D sources

Traditionally, a 2-D block-based transform is always implemented through two separate 1-D transforms along each blocks vertical and horizontal dimensions. Such a framework is however not highly suitable for a directional 2-D source in which the dominant directional information is neither horizontal nor vertical. On the other hand, the R-D performance upper bound for all block-based transform coding schemes applied on such 2-D directional sources can be obtained by the non-separable Karhunen-Loève transform (KLT) — which is unfortunately very expensive computationally. In this paper, we present a new framework for designing some non-separable transforms that offer an R-D performance closer to that of the KLT, but can be implemented with nearly the same complexity as that of the discrete cosine transform (DCT).

Improving Intra Prediction in High-Efficiency Video Coding

Intra prediction is an important tool in intra-frame video coding to reduce the spatial redundancy. In current coding standard H.265/high-efficiency video coding (HEVC), a copying-based method based on the boundary (or interpolated boundary) reference pixels is used to predict each pixel in the coding block to remove the spatial redundancy. We find that the conventional copying-based method can be further improved in two cases: 1) the boundary has an inhomogeneous region and 2) the predicted pixel is far away from the boundary that the correlation between the predicted pixel and the reference pixels is relatively weak. This paper performs a theoretical analysis of the optimal weights based on a first-order Gaussian Markov model and the effects when the pixel values deviate from the model and the predicted pixel is far away from the reference pixels. It also proposes a novel intra prediction scheme based on the analysis that smoothing the copying-based prediction can derive a better prediction block. Both the theoretical analysis and the experimental results show the effectiveness of the proposed intra prediction method. An average gain of 2.3% on all intra coding can be achieved with the HEVC reference software.

Hybrid angular intra/template matching prediction for HEVC intra coding

In the latest HEVC video coding standard, angular intra prediction (AIP) applies 35 modes including 33 angular modes which can handle blocks with direction information well, and other 2 modes (DC and planar) which are used to predict smooth blocks. However, for blocks with complex texture, these modes may not give good predictions. Some complex blocks can be predicted well by template matching prediction (TMP) which was proposed to predict blocks having similar patterns in the coded regions of the same frame without the cost of large overheads. In this paper, a novel hybrid AIP/TMP is proposed to improve the prediction efficiency. Experimental results show that the proposed method can achieve a coding gain of about 2.5% for high resolution sequences on average compared to the AIP in HEVC. The gain can be up to 4.2%.

Design of low-complexity, non-separable 2-D transforms based on butterfly structures

The transform used in most image and video coding standards is the separable 2-D discrete cosine transform (DCT), which has been proven to be a robust approximation of the optimal Karhunen-Loève transform (KLT) for the 1st-order Markov sources with a large correlation coefficient. However, such separable 2-D DCT surely is not the best choice when it is applied on some residual or directional signals. Based on the butterfly architecture for DCTs fast implementation, we present in this paper a novel design of non-separable 2-D transforms that get much closer to the KLT but at the implementation cost no bigger than that of the DCT. The critical issue in our design is how to pair all node-variables in various stages of the butterfly structure. We propose a near-optimal pairing strategy to solve this problem and present some examples to demonstrate its effectiveness.

Adaptive transform with HEVC intra coding

This paper proposes a content adaptive transform based on singular value decomposition (SVD) with HEVC intra coding. In the latest video coding standard HEVC, a combination of Discrete Cosine Transform (DCT) and Discrete Sine Transform (DST) is adopted to transform the residuals from intra prediction. The DCT and DST are fixed transforms which are derived based on the Gauss-Markov model. Since the residual blocks contain various distributions which may not be compacted well by the fixed DCT/DST, we propose an adaptive transform method based on SVD which is adaptive to residuals for angular intra coding. Experimental results show that the proposed method can achieve rate-distortion performance improvements compared to HEVC intra coding (up to 1.9% bit rate savings).

A capture-to-display delay measurement system for visual communication applications

We propose an effective method to measure the capture-to-display delay (CDD) of a visual communication application. The method does not require modifications to the existing system, nor require the encoder and decoder clocks to be synchronized. Furthermore, we propose a solution to solve the multiple overlapped-timestamp problems due to the response time of the display and the exposure time of the camera. We implemented the method in software to measure the capture-to-display delay of a cellphone video chat application over various types of networks. Experiments confirmed the effectiveness of our proposed methods.

Capture-to-display delay measurement for visual communication applications

We propose a method to measure the capture-to-display delay (CDD) of a visual communication application. The method does not require modifications to the existing system, nor require the encoder and decoder clocks be synchronized. Furthermore, we propose a solution to solve the multiple-overlapped-timestamp problem due to the exposure time of the camera. We analyze the measurement error, and implement the method in software to measure the CDD of a cellphone video chat application over various types of networks. Experiments confirm the effectiveness of our proposed method.