Yanbo Gao

Lagrangian Multiplier Adaptation for Rate-Distortion Optimization With Inter-Frame Dependency

Rate-distortion optimization (RDO) is widely used in video coding, which plays a critical role in enhancing the coding efficiency substantially. Currently, the RDO process is performed in a way that coding efficiency of each coding unit (CU) is maximized independently without considering the dependency among CUs. As we know, in the current hybrid video coding structure, spatial/temporal prediction techniques are extensively used, which introduce strong dependency among CUs. In this paper, we investigate RDO with inter-frame dependency, where the impact of coding performance of the current CU on that of the following frames is considered. Accordingly, an RDO scheme taking the inter-frame dependency into account is proposed by adapting the Lagrangian multiplier. The experimental results show that the proposed scheme can achieve about 3.22% and 3.19% BD-rate saving in average over the state-of-the-art High Efficiency Video Coding (HEVC) reference software HM15.0 in the low-delay

Inter-frame dependent rate-distortion optimization using lagrangian multiplier adaption

P

Temporally Dependent Rate-Distortion Optimization for Low-Delay Hierarchical Video Coding

(LDP) and low-delay

Hierarchical temporal dependent rate-distortion optimization for low-delay coding

B

Texture-Aware Depth Prediction in 3D Video Coding

(LDB) coding structures, respectively, with no extra encoding time. The proposed scheme can obtain a significantly higher coding gain than the multiple quantization parameter (MQP) (±3) optimization technique that would greatly increase the encoding time by a factor of about six. Coupled with MQP optimization, the proposed scheme can further achieve about 5.96% and 5.57% BD-rate savings in average over the HEVC and about 4.03% and 4.07% over the HEVC with MQP optimization, under the specified common test conditions for LDP and LDB coding structures, respectively.

Layer-based temporal dependent rate-distortion optimization in Random-Access hierarchical video coding

It is known that, in the current hybrid video coding structure, spatial and temporal prediction techniques are extensively used which introduce strong dependency among coding units. Such dependency poses a great challenge to perform a global rate-distortion optimization (RDO) when encoding a video sequence. RDO is usually performed in a way that coding efficiency of each coding unit is optimized independently without considering dependeny among coding units, leading to a suboptimal coding result for the whole sequence. In this paper, we investigate the inter-frame dependent RDO, where the impact of coding performance of the current coding unit on that of the following frames is considered. Accordingly, an inter-frame dependent rate-distortion optimization scheme is proposed and implemented on the newest video coding standard High Efficiency Video Coding (HEVC) platform. Experimental results show that the proposed scheme can achieve about 3.19% BD-rate saving in average over the state-of-the-art HEVC codec (HM15.0) in the low-delay B coding structure, with no extra encoding time. It obtains a significantly higher coding gain than the multiple QP (±3) optimization technique which would greatly increase the encoding time by a factor of about 6. Coupled with the multiple QP optimization, the proposed scheme can further achieve a higher BD-rate saving of 5.57% and 4.07% in average than the HEVC codec and the multiple QP optimization enabled HEVC codec, respectively.

A frame-level rate control scheme for low delay video coding in HEVC

Low-delay hierarchical coding structure (LD-HCS), as one of the most important components in the latest High Efficiency Video Coding (HEVC) standard, greatly improves coding performance. It groups consecutive P/B frames into different layers and encodes them with different quantization parameters (QPs) and reference mechanisms in such a way that temporal dependency among frames can be exploited. However, due to varying characteristics of video contents, temporal dependency among coding units differs significantly from each other in the same or different layers, while a fixed LD-HCS scheme cannot take full advantage of the dependency, leading to a substantial loss in coding performance. This paper addresses the temporally dependent rate distortion optimization (RDO) problem by attempting to exploit varying temporal dependency of different units. First, the temporal relationship of different frames under the LD-HCS is examined, and hierarchical temporal propagation chains are constructed to represent the temporal dependency among coding units in different frames. Then, a hierarchical temporally dependent RDO scheme is developed specifically for the LD-HCS based on a source distortion propagation model. Experimental results show that our proposed scheme can achieve 2.5% and 2.3% BD-rate gain in average compared with the HEVC codec under the same configuration of P and B frames, respectively, with a negligible increase in encoding time. Furthermore, coupled with QP adaption, our proposed method can achieve higher coding gains, e.g., with multi-QP optimization, about 5.4% and 5.0% BD-rate saving in average over the HEVC codec under the same setting of P and B frames, respectively.

Source Distortion Temporal Propagation Analysis for Random-Access Hierarchical Video Coding Optimization

Hierarchical coding structure (HCS) is one of the most important components in High Efficiency Video Coding (HEVC) that improves the coding performance greatly, especially for Low-Delay (LD) coding. It groups frames into different layers and enc odes them with different quantization parameters (QP) and different reference mechanisms. Due to the extensively used inter-prediction, the coding of frames in different layers is highly dependent and an appropriate QP and reference selection scheme may significantly improve the performance by taking advantage of such temporal dependency. However in the current HEVC codec, a predefined HCS, such as the Low-Delay HCS (LD-HCS), is performed without considering the different characteristic of different video contents, thus leading to a suboptimal coding solution. In this paper, the hierarchical temporal relationship under LD-HCS is first investigated and a hierarchical temporal propagation chain is constructed to describe the temporal dependency among frames. Then a hierarchical temporal dependent rate-distortion optimization scheme is developed specifically for the LD-HCS in HEVC. Experiments results show that the proposed scheme achieves BD-rate saving of 2.9% and 2.8% in average against HEVC codec under LD-HCS of P and B frames, respectively, with a negligible increase in encoding time.

Independently Recurrent Neural Network (IndRNN): Building a Longer and Deeper RNN

3D video has raised great interest in the last decade and currently a new 3D video coding standard, known as 3D video coding extension of High Efficiency Video Coding (3D-HEVC), has been developed. The standard investigates the coding of multiview video plus depth, which consists of texture videos and depth videos of multiple views. Depth video, as a description of geometry information of a scene, is generally composed of large flat regions separated by sharp edges. The conventional video coding may fail to generate an accurate prediction for units with sharp edges due to its block-based prediction which cannot compensate (minor) boundary changes well. In order to attack the problem, a new texture-aware depth inter-prediction method is proposed, which incorporates pixel-oriented weighting in the bi-prediction process by exploiting motion and structure similarities between texture and depth videos. Furthermore, such pixel-oriented weighting scheme can be extended to the uni-prediction process by considering more prediction blocks with small motion vector displacements. Experimental results demonstrate that the adapted 3D-HEVC codec with the proposed method can achieve better rate-distortion performance compared to the original 3D-HEVC standard codec.

A Fully Trainable Network with RNN-based Pooling.

Rate-distortion optimization (RDO) plays an important part in improving the coding efficiency of High Efficiency Video Coding (HEVC), especially for the hierarchical coding structure defined in the Random-Access (RA) configuration, noted as Random-Access Hierarchical Video Coding (RA-HVC), where different frames are assigned to different temporal layers and further coded with different coding parameters. Due to the inter-frame prediction, coding result of one unit may affect the coding performance of the following temporally related units. Therefore, the temporal dependency among units needs to be considered in the coding process. However, the RDO process in the current video codec is performed without considering the varying temporal dependency, thus compromising the rate-distortion performance significantly. To address this problem, a layer-based temporal dependent RDO method is proposed in this paper where the temporal dependency among different frames in the same or different layers is examined. By reformulating the temporal dependent RDO for the RA-HVC, we show that it can be implemented in a way of simply refining the Lagrange multiplier. Experimental results show that the proposed method achieves, in average, about 1.4% BD-rate savings with a negligible increase in encoding time for the random-access configuration.