Do-Kyoung Kwon

Rate Control for H.264 Video With Enhanced Rate and Distortion Models

A new rate control scheme for H.264 video encoding with enhanced rate and distortion models is proposed in this work. Compared with existing H.264 rate control schemes, our scheme has offered several new features. First, the inter-dependency between rate-distortion optimization (RDO) and rate control in H.264 is resolved via quantization parameter estimation and update. Second, since the bits of the header information may occupy a larger portion of the total bit budget, which is especially true when being coded at low bit rates, a rate model for the header information is developed to estimate header bits more accurately. The number of header bits is modeled as a function of the number of nonzero motion vector (MV) elements and the number of MVs. Third, a new source rate model and a distortion model are proposed. For this purpose, coded 4 times 4 blocks are identified and the number of source bits and distortion are modeled as functions of the quantization stepsize and the complexity of coded 4 times 4 blocks. Finally, a R-D optimized bit allocation scheme among macroblocks (MBs) is proposed to improve picture quality. Built upon the above ideas, a rate control algorithm is developed for the H.264 baseline-profile encoder under the constant bit rate constraint. It is shown by experimental results that the new algorithm can control bit rates accurately with the R-D performance significantly better than that of the rate control algorithm implemented in the H.264 software encoder JM8.1a

Full-reference quality assessment of stereopairs accounting for rivalry

We develop a framework for assessing the quality of stereoscopic images that have been afflicted by possibly asymmetric distortions. An intermediate image is generated which when viewed stereoscopically is designed to have a perceived quality close to that of the cyclopean image. We hypothesize that performing stereoscopic QA on the intermediate image yields higher correlations with human subjective judgments. The experimental results confirm the hypothesis and show that the proposed framework significantly outperforms conventional 2D QA metrics when predicting the quality of stereoscopically viewed images that may have been asymmetrically distorted.

Fast intra block copy (IntraBC) search for HEVC screen content coding

Coding of screen content video is becoming important because of applications such as wireless displays, remote desktop, remote gaming, automotive infotainment, cloud computing, distance education etc. Video in these applications often has mixed content consisting of natural video, text and graphics in the same picture. Intra block copy (IntraBC) is a new coding tool being studied for the HEVC Range Extensions (RExt) standard. IntraBC is a block matching technique where in a Coding Unit (CU) is predicted as a displacement from already reconstructed block of samples from neighboring region in the same picture. IntraBC is very effective for screen content video since it removes redundancy from repeating patterns which typically occur in text and graphics regions. IntraBC provides a significant bit rate savings (up to 44%) for screen content sequences but at the cost of encoding time increase because of the search involved in Intra block matching. This paper presents fast encoding techniques for early skipping of IntraBC search which result in about 21% - 24% encoding time reduction for Intra coding.

Study of subject agreement on stereoscopic video quality

We describe a study that aims towards enhancing our understanding of the perception of H.264/AVC compressed stereoscopic 3D videos, in particular spatial video quality, depth quality, visual comfort and overall 3D video quality. The results of this study indicate that the human subjects have diverse opinions on depth quality scores but a high agreement on spatial video quality. Their agreement on overall 3D video quality is intermediate relative to that on spatial video quality and depth quality. Based on our analysis, we propose to use separate quality assessment models: spatial video quality models and depth quality models.

Improved sample adaptive offset for HEVC

Sample adaptive offset (SAO) is the new in-loop filter in High Efficiency Video Coding (HEVC) standard. In this paper, the problems in the early version of SAO technique is discussed, and it is shown how the proposed methods improve its performance. It is proposed to restrict edge offset sign according to edge shape to reduce visual artifact caused by edge offset. It is also proposed to reduce number of band offset to facilitate implementation. Experimental results shows the proposed methods effectively reduce the artifact caused by edge offset, and reduce number of band offset by half without compromising coding efficiency.

H.264/SVC temporal bit allocation with dependent distortion model

The bit allocation problem for hierarchical B-pictures in H.264/SVC is studied with a GOP-based dependent distortion model in this work. Inter-dependency between temporal layers of H.264/SVC is often neglected because of the complexity involved, which often leads to poorer rate control performance. To address this shortcoming, we propose a distortion model that takes inter-dependency into consideration while preserving the low complexity of the encoding process. It is demonstrated by experimental results that the new distortion model results in a highly efficient bit allocation scheme, which outperforms the rate control algorithm in the JSVM 9.12 reference codec by a significant margin.

Intra motion compensation and entropy coding improvements for HEVC screen content coding

Applications such as wireless displays, automotive infotainment, remote desktop, remote gaming, distance education, cloud computing etc. are becoming popular. Video in these applications often has mixed content consisting of natural video and screen content (text, graphics etc.) in the same picture. In text and graphics regions, patterns such as text characters, icons, lines etc. can repeat within a picture. Also since the graphics and text regions have sharp edges that are sometimes not predicted well by Intra prediction tools, the probability of prediction error having high amplitude increases. This paper presents two tools for improving the intra coding efficiency of the High Efficiency Video Coding (HEVC) standard when coding screen content. The first tool is a Coding Unit (CU)-level Intra motion compensation tool to eliminate redundancy from repeating patterns in text and graphics regions. The second tool is a modification to HEVC cRiceParam update process of coeff_abs_level_remaining entropy coding to adapt better to larger prediction errors in text and graphics regions. The combination of the two tools achieves an average bit-rate/Bjontegaard Delta-Rate savings in the range of 0.8% to 28.1% over HEVC Range Extensions Test Model on screen content test video sequences used in Range Extensions core experiments by the Joint Collaborative Team for Video Coding (JCT-VC).

GOP-based Rate Control for H.264/SVC with Hierarchical B-pictures

A GOP-based rate control algorithm for hierarchical B-pictures in H.264/SVC is proposed in this work. The problem is firstly formulated as a bit allocation problem among different temporal layers. Due to the dependency of these layers, we propose a Q-selection tree solution framework. The monotonicity property is exploited to prune the Q-selection tree. Then, the rate-distortion (R-D) data is calculated at a selected set of leaf nodes in any admissible Q-selection tree. The R-D data of other leaf nodes are interpolated using the log-linearity property of a local R-D model. It is shown by experiments that the proposed algorithm achieves a significant coding gain improvement at different target bit rates.

Joint quality-temporal (Q-T) bit allocation for H.264/SVC

A joint quality-temporal (Q-T) bit allocation scheme is proposed for H.264/SVC in this work. First, rate and distortion (R-D) models for dependent quality layers are derived, where the complex inter-layer dependency is considered. Then, the joint Q-T bit allocation problem is formulated as an optimization problem using the Lagrange method and solved numerically with the derived R-D models. As a result, we develop a low-complexity bit allocation scheme for the joint Q-T scalability of H.264/SVC. It is demonstrated by experimental results that the new R-D models results in a highly efficient bit allocation scheme, which outperforms the JSVM benchmark by a significant margin.

Dependent R/D Modeling Techniques and Joint T-Q Layer Bit Allocation for H.264/SVC

We investigate dependent rate/distortion (R/D) modeling techniques for H.264/SVC videos. We introduce a self-domain (S-domain) analysis method for characterizing the dependent R/D behaviors, where the R/D characteristics of a base layer are employed as the observation domain for those of dependent layers. Based on S-domain observations, we propose empirical dependent R/D models and analyze physical implications of the proposed models. As an application of the proposed R/D models, we examine a joint temporal-quality layer bit allocation algorithm formulated as a Lagrange optimization problem. The proposed R/D models enable us to derive an analytical solution to the joint optimization problem. Finally, it is demonstrated by experimental results that our bit allocation algorithm outperforms JSVM benchmark by a significant margin (10%-20%) at various bit rates.