Luheng Jia

Local saliency detection based fast mode decision for HEVC intra coding

The High Efficiency Video Coding (HEVC) is the next generation video coding standard beyond H.264/AVC. Compared with only up to 9 modes for intra prediction in H.264/AVC, HEVC provides 35 intra prediction modes (IPM) to improve coding efficiency, which inevitably poses a huge complexity burden to the encoder. To speed up the HEVC encoder, a novel fast mode decision (FMD) algorithm for HEVC intra prediction is proposed. In the proposed algorithm, we analyzed the costs generated by rough mode decision (RMD), which has already been incorporated in the HM software. We found that the RMD costs listed by mode number generally follow the same trend with the rate-distortion optimization (RDO) costs. Further, the local salient modes, whose RMD costs have a significant drop compared with adjacent modes, tend to be promising competitors for the optimal mode. Based on these observations, we further reduced the number of the candidates for the RDO process. Experimental results show that our proposed algorithm achieves averagely 19.0% (up to 33.6%) encoding time saving whilst causing negligible RD performance loss (0.4% BD-Rate increase on average) compared with HM 7.0 anchor.

Color clustering matting

Natural image matting refers to the problem of extracting regions of interest such as foreground object from an image based on user inputs like scribbles or trimap. More specifically, we need to estimate the color information of background, foreground and the corresponding opacity, which is an ill-posed problem inherently. Inspired by closed-form matting and KNN matting, in this paper, we extend the local color line model which is based on the assumption of linear color clustering within a small local window, to nonlocal feature space neighborhood. New affinity matrix is defined to achieve better clustering. Further, we demonstrate that good clustering ensures better prediction of alpha matte. Experimental evaluations on benchmark datasets and comparisons show that our matting algorithm is of higher accuracy and better visual quality than some state-of-the-art matting algorithms.

Content based fast prediction unit quadtree depth decision algorithm for HEVC

The nested quadtree based partitioning scheme of HEVC contributes a lot to the coding efficiency improvement, however, it adds significant complexity to the encoder. This paper introduces a fast prediction unit (PU) level quadtree depth decision (FPDD) algorithm. It is achieved by making use of the inherited correlation of PU quadtree structure between current largest coding unit (LCU) and its spatial and temporal neighbors. To reduce error propagation, we also propose a confidence grading scheme to prevent LCUs with bad prediction from being referred to by others. Results show that our proposed algorithm provides averagely 20.0% (up to 39.3%) encoding time reduction whilst causing negligible RD performance loss (0.2% BD-Rate increase on average) compared with HM 7.0.

A diamond search windowbased adaptive search range algorithm

In this paper, a novel diamond search window (SW) based full search algorithm is proposed. To dynamically adjust the size of the diamond search window, a frame level adaptive search range (SR) algorithm is presented. Laplace distribution is used to model the optimal motion vector difference (MVD) distribution within a frame and the model parameters are estimated by maximum likelihood estimation (MLE). The data used for MLE is also carefully studied. With the estimated distribution model, proper SR is obtained dynamically frame by frame. Experiment results show that a diamond SW outperforms the traditional squared SW. Also, the proposed adaptive search range algorithm properly adjusts the diamond SW size. The proposed diamond SW based adaptive search range (ASR+DSW-FS) algorithm obtains the same or even better performance compared with conventional full search algorithm, while significantly reduces the computational complexity.

Fast and efficient intra-frame deinterlacing using observation model based bilateral filter

Recently, a few bilateral filter based interpolation and intraframe deinterlacing algorithms have been proposed, but these algorithms only use prior information (bilateral filter). In this paper, we propose an efficient and fast intra-frame deinterlacing algorithm using an observation model based bilateral filter (using both likelihood and prior information). Our proposed algorithm is also able to use approximated horizontal pixels for the deinterlacing, which results into the better prediction of the edges. From extensive experiments, it is observed that the proposed algorithm has the capability of provide satisfactory results in terms of both objective and subjective quality.

A tutorial on image/video coding standards

The field of image and video compression has gone through rapid growth during the past thirty years, leading to various coding standards. The main goal of continuous efforts on image/video coding standardization is to achieve low bit rate for data storage and transmission, while maintaining acceptable distortion. In this paper, various developmental stages of image and video compression standards are reviewed, including JPEG and JPEG 2000 image standards, MPEG-1, MPEG-2, MPEG-4, H.261, H.263, H.264/MPEG-4 AVC, and the latest international video standard HEVC as well as Chinese video coding standard AVS. Key features and major applications of the standards will be briefly introduced and the compression performance of the standards at each stage will be compared and discussed.

Palette-based compound image compression in HEVC by exploiting non-local spatial correlation

Non-camera captured images (also known as compound image) contain a mixture of camera-captured natural images and computer-generated graphics and texts. Nowadays, there are more and more applications calling for non-camera captured image/video compression scheme. However, current video coding standards, which are designed for natural video, treat non-camera captured video less carefully. For example, the state-of-the-art video coding standard High Efficiency Video Coding (HEVC) may blur or even remove edges in text/graphic region. A lot of schemes are proposed to preserve direction property of texts and graphics, such as palette-based intra coding. In this paper, a novel palette coding scheme is proposed for palette-based intra coding in HEVC. The palette in a block is predicted from an adaptive palette template, which records the statistical non-local spatial correlation of an image. Every block chooses its own palette using the palette template as the prediction in a rate-distortion optimized manner. Experimental results show that the proposed scheme can achieve up to 5.2% bit-rate saving compared to the state-of-the-art palette-based coding scheme in HEVC.

Symmetrical predictor structure based integrated lossy, near lossless/lossless coding of images

Prediction based algorithms reported in the literature are not able to integrate lossy and near-lossless/lossless coding and uses only causal pixels (non-symmetrical predictor structure) for prediction. A non-symmetrical predictor structure, however, is not able to efficiently adapt near the intensity varying areas, which results into poor prediction. Hence, we propose a novel two-stage algorithm for lossy, near lossless/lossless compression using a symmetrical predictor structure is proposed. In the first stage, the proposed algorithm encodes and decodes the given image using the JPEG-2000 standard algorithm (lossy coding). This JPEG-2000 decoded image in the first stage, enables us to use the symmetrical predictor (using both causal and non-causal pixels) for prediction in the second stage. A performance evaluation shows that our algorithm is significantly better in terms of compression performance as compared to some of the computationally complex methods.

Improved sample adaptive offset for HEVC

High-Efficiency Video Coding (HEVC) is the newest video coding standard which can significantly reduce the bit rate by 50% compared with existing standards. One new efficient tool is sample adaptive offset (SAO), which classifies reconstructed samples into different categories, and reduces the distortion by adding an offset to samples of each category. Two SAO types are adopted in HEVC: edge offset (EO) and band offset (BO). Four 1-D directional edge patterns are used in edge offset type, and only one is selected for each CTB. However, single directional pattern cannot remove artifacts effectively for the CTBs, which contain edges in different directions. Therefore, we analyze the performance of each edge pattern applied on this kind of CTB, and propose to take advantage of existing edge classes and combine some of the them as a new edge offset class, which can adapt to multiple edge directions. All the combinations are tested, and the results show that for Low Delay P condition, they can achieve 0.2% to 0.5% bit rate reduction.

Lossless/near lossless compression using bilateral filter and symmetrical structure based bias cancellation

Recently, a few symmetrical predictor structure (SPS) based lossless / near lossless compression algorithms have been proposed, which can efficiently exploit the information from the neighboring pixels. Prediction stage of existing SPS algorithms uses least squares optimization, which is computationally expensive and only causal pixels are used for the bias cancellation stage. In this paper, we propose an efficient and computationally simple lossless / near-lossless compression algorithm using the bilateral filter. Moreover we propose to use both causal and non-causal pixels for bias cancellation. From extensive experiments, it is observed that the proposed algorithm has the capability of provide better prediction and compression performance.