Zhaoqing Pan

Efficient Motion and Disparity Estimation Optimization for Low Complexity Multiview Video Coding

The use of variable block-size motion estimation (ME), disparity estimation (DE), and multiple reference frames selection aims to improve the coding efficiency of multiview video coding (MVC), however, this is at the cost of high computational complexity of these advanced coding techniques, which are not suitable for real-time video broadcasting applications. In this paper, we propose an efficient ME and DE algorithm for reducing the computational complexity of MVC. Firstly, according to the characteristics of the coded block pattern and rate distortion (RD) cost, an early DIRECT mode decision algorithm is proposed. Then, based on the characteristics of the initial search point in the ME/DE process and the observation that the best point is center-biased, an early ME/DE termination strategy is proposed. If the ME/DE early termination is not satisfied, the ME/DE search window will be reduced by applying the optimal theory. At last, two block matching search strategies are proposed to predict the best point for the ME/DE. Experimental results show that the proposed algorithm can achieve 50.05% to 77.61%, 64.83% on average encoding time saving. Meanwhile, the RD performance degradation is negligible. Especially, the proposed algorithm can be applied to not only the odd views but also the even views.

Fast Motion Estimation Based on Content Property for Low-Complexity H.265/HEVC Encoder

The high definition (HD) and ultra HD videos can be widely applied in broadcasting applications. However, with the increased resolution of video, the volume of the raw HD visual information data increases significantly, which becomes a challenge for storage, processing, and transmitting the HD visual data. The state-of-the-art video compression standard-H.265/High Efficiency Video Coding (HEVC) compresses the raw HD visual data efficiently, while the high compression rate comes at the cost of heavy computation load. Hence, reducing the encoding complexity becomes vital for the H.265/HEVC encoder to be used in broadcasting applications. In this paper, based on the best motion vector selection correlation among the different size prediction modes, we propose a fast motion estimation (ME) method to reduce the encoding complexity of the H.265/HEVC encoder. First, according to the prediction unit (PU) partition type, all PUs are classified into two classes, parent PU and children PUs, respectively. Then, based on the best motion vector selection correlation between the parent PU and children PUs, the block matching search process of the children PUs is adaptively skipped if their parent PU chooses the initial search point as its final optimal motion vector in the ME process. Experimental results show that the proposed method achieves an average of 20% ME time saving as compared with the original HM-TZSearch. Meanwhile, the rate distortion performance degradation is negligible.

Early MERGE Mode Decision Based on Motion Estimation and Hierarchical Depth Correlation for HEVC

The high efficiency video coding (HEVC) is the latest video coding standard, which adopts the quadtree structure based coding tree unit (CTU) to improve the coding efficiency. In the HEVC encoding process, the CTU is recursively split into the 8×8 size coding units (CUs) from the 64×64 size CU. Along with the increased number of the sizes of the CUs, the number of coding modes has been greatly increased, which results in high computational complexity in the HEVC encoder. In this paper, we propose an early MERGE mode decision algorithm to reduce the computational complexity of the HEVC encoder. Firstly, based on the all-zero block (AZB) and the motion estimation (ME) information of the INTER 2N×2N mode, an early MERGE mode decision is proposed for the root CUs (i.e., 64×64 size CUs). Then, an early MERGE mode decision is proposed for the children CUs (i.e., 32×32, 16×16, and 8×8 size CUs) by considering the mode selection correlation between the root CU and the children CUs. To maximize the computational complexity reduction, when the root CUs are encoded in the non-MERGE modes, the AZB and the ME information are also used for early termination of the children CUs. Experimental results demonstrate that compared to the state-of-the-art published method, the proposed algorithm can achieve about 35% encoding time on average saving while the rate distortion performance degradation is negligible.

Fast reference frame selection based on content similarity for low complexity HEVC encoder

A fast reference frame selection algorithm for the HEVC encoder is proposed.A relationship between the content similarity and reference frame selection is derived.The content similarity is studied without any extra computational complexity.Experimental results show that the proposed algorithm efficiently removes the encoding complexity of the best reference frame decision process. The high efficiency video coding (HEVC) is the state-of-the-art video coding standard, which achieves about 50% bit rate saving while maintaining the same visual quality as compared to the H.264/AVC. This achieved coding efficiency benefits from a set of advanced coding tools, such as the multiple reference frames (MRF) based interframe prediction, which efficiently improves the coding efficiency of the HEVC encoder, while it also increases heavy computation into the HEVC encoder. The high encoding complexity becomes a bottleneck for the high definition videos and HEVC encoder to be widely used in real-time and low power multimedia applications. In this paper, we propose a content similarity based fast reference frame selection algorithm for reducing the computational complexity of the multiple reference frames based interframe prediction. Based the large content similarity between the parent prediction unit (Inter_2N2N) and the children prediction units (Inter_2NN, Inter_N2N, Inter_NN, Inter_2NnU, Inter_2NnD, Inter_nL2N, and Inter_nR2N), the reference frame selection information of the children prediction units are obtained by learning the results of their parent prediction unit. Experimental results show that the proposed algorithm can reduce about 54.29% and 43.46% MRF encoding time saving for the low-delay-main and random-access-main coding structures, respectively, while the rate distortion performance degradation is negligible.

Machine Learning-Based Coding Unit Depth Decisions for Flexible Complexity Allocation in High Efficiency Video Coding

In this paper, we propose a machine learning-based fast coding unit (CU) depth decision method for High Efficiency Video Coding (HEVC), which optimizes the complexity allocation at CU level with given rate-distortion (RD) cost constraints. First, we analyze quad-tree CU depth decision process in HEVC and model it as a three-level of hierarchical binary decision problem. Second, a flexible CU depth decision structure is presented, which allows the performances of each CU depth decision be smoothly transferred between the coding complexity and RD performance. Then, a three-output joint classifier consists of multiple binary classifiers with different parameters is designed to control the risk of false prediction. Finally, a sophisticated RD-complexity model is derived to determine the optimal parameters for the joint classifier, which is capable of minimizing the complexity in each CU depth at given RD degradation constraints. Comparative experiments over various sequences show that the proposed CU depth decision algorithm can reduce the computational complexity from 28.82% to 70.93%, and 51.45% on average when compared with the original HEVC test model. The Bjøntegaard delta peak signal-to-noise ratio and Bjøntegaard delta bit rate are -0.061 dB and 1.98% on average, which is negligible. The overall performance of the proposed algorithm outperforms those of the state-of-the-art schemes.

Early termination for TZSearch in HEVC Motion Estimation

The TZSearch algorithm was adopted in the high efficiency video coding reference software HM as a fast Motion Estimation (ME) algorithm for its excellent performance in reducing ME time and maintaining a comparable Rate Distortion (RD) performance. However, the multiple initial search point decision and the hybrid block matching search contribute a relatively high computational complexity to TZSearch. In this paper, based on the statistical analysis of the probability of median predictor to be selected as the final best point in the large Coding Units (CUs) (64×64, 32×32) and small CUs (16×16, 8×8) as well as the center-biased characteristic of the final best search point in ME process, we propose two early terminations for TZSearch. Experimental results show that the proposed early terminations can achieve 38.96% encoding time saving, while the RD performance degradation is quite acceptable.

Multiview Coding Mode Decision With Hybrid Optimal Stopping Model

In a generic decision process, optimal stopping theory aims to achieve a good tradeoff between decision performance and time consumed, with the advantages of theoretical decision-making and predictable decision performance. In this paper, optimal stopping theory is employed to develop an effective hybrid model for the mode decision problem, which aims to theoretically achieve a good tradeoff between the two interrelated measurements in mode decision, as computational complexity reduction and rate-distortion degradation. The proposed hybrid model is implemented and examined with a multiview encoder. To support the model and further promote coding performance, the multiview coding mode characteristics, including predicted mode probability and estimated coding time, are jointly investigated with inter-view correlations. Exhaustive experimental results with a wide range of video resolutions reveal the efficiency and robustness of our method, with high decision accuracy, negligible computational overhead, and almost intact rate-distortion performance compared to the original encoder.

Fast mode decision based on texture---depth correlation and motion prediction for multiview depth video coding

The motion estimation and disparity estimation are used to remove the temporal and inter-view redundancies in multiview plus depth video coding, however, the variable block-size ME and DE make the computational complexity increase dramatically. This drawback limits it to be applied in real-time applications. In this paper, based on the mode correlations between depth video and its corresponding texture video, motion prediction and coded block pattern, we propose a fast mode decision algorithm to reduce the computational complexity of multiview depth video coding. Experimental results show that the proposed algorithm can achieve 67.18 and 69.90 % encoding time saving for even and odd views, respectively, while maintaining a comparable rate-distortion performance. In addition, with the dramatic encoding time reduction, the proposed algorithm becomes more suitable for real-time applications.

Fast Mode Decision Using Inter-View and Inter-Component Correlations for Multiview Depth Video Coding

With the development of three-dimensional (3-D) display technologies, 3-D video has attracted more and more interest. Multiview video plus depth (MVD) is one of the most popular representation formats of 3-D video. In MVD coding system, multiview depth video needs to be coded and transmitted in addition to the texture video. This paper presents a novel fast mode decision (FMD) method for odd views in multiview depth video coding. First, the inter-view and inter-component coding correlations are analyzed to provide efficient reference information. Then, with a view to the characteristics of different types of frames, different early termination strategies are proposed. For the nonanchor frame, the early termination criterion is based on the rate-distortion cost information of the even views and the coded block pattern information. For the anchor frame, the criterion is set stricter to maintain the coding accuracy. Experimental results show that the proposed method can reduce 78.07% coding time on average, without significant loss of video quality.

A Direction-Based Unsymmetrical-Cross Multi-Hexagon-Grid Search Algorithm for H.264/AVC Motion Estimation

The Unsymmetrical-cross Multi-hexagon-grid Search (UMHexagonS) is one of the best fast Motion Estimation (ME) algorithm in H.264/AVC (Advanced Video Coding) reference software. It achieves an excellent coding performance by using multiple initial search point predictors and hybrid block matching search pattern. However, the hybrid search pattern makes the computational complexity of ME increased. In this paper, we propose a Direction-based UMHexagonS (DBUMHexagonS) to further reduce the computational complexity of UMHexagonS. Each block matching search pattern of UMHexagonS is divided into four direction-based sub-search patterns, one of four directions is selected according to the difference between the Motion Vector (MV) of current block and the MV of its collocated block in previous frame, such a direction is applied to all following search patterns. As a result, the number of total search points will be dramatically reduced. Experimental results show that compared to the best UMHexagonS algorithm, the proposed algorithm can save the ME time up to 30.094 % while the rate-distortion performance is not compromised.