Tao-Sheng Ou

Perceptual Rate-Distortion Optimization Using Structural Similarity Index as Quality Metric

The rate-distortion optimization (RDO) framework for video coding achieves a tradeoff between bit-rate and quality. However, objective distortion metrics such as mean squared error traditionally used in this framework are poorly correlated with perceptual quality. We address this issue by proposing an approach that incorporates the structural similarity index as a quality metric into the framework. In particular, we develop a predictive Lagrange multiplier estimation method to resolve the chicken and egg dilemma of perceptual-based RDO and apply it to H.264 intra and inter mode decision. Given a perceptual quality level, the resulting video encoder achieves on the average 9% bit-rate reduction for intra-frame coding and 11% for inter-frame coding over the JM reference software. Subjective test further confirms that, at the same bit-rate, the proposed perceptual RDO indeed preserves image details and prevents block artifact better than traditional RDO.

SSIM-Based Perceptual Rate Control for Video Coding

The quality of video is ultimately judged by human eye; however, mean squared error and the like that have been used as quality metrics are poorly correlated with human perception. Although the characteristics of human visual system have been incorporated into perceptual-based rate control, most existing schemes do not take rate-distortion optimization into consideration. In this paper, we use the structural similarity index as the quality metric for rate-distortion modeling and develop an optimum bit allocation and rate control scheme for video coding. This scheme achieves up to 25% bit-rate reduction over the JM reference software of H.264. Under the rate-distortion optimization framework, the proposed scheme can be easily integrated with the perceptual-based mode decision scheme. The overall bit-rate reduction may reach as high as 32% over the JM reference software.

Fast Decision of Block Size, Prediction Mode, and Intra Block for H.264 Intra Prediction

The spatial-domain intra prediction scheme of H.264 has high computational complexity, especially for the High Profile as it incorporates the additional intra 8 × 8 prediction mode. To address this issue, we explore the hierarchy of H.264 mode decision process in this paper and adopt an approach that is in synchrony with the mode decision hierarchy. In particular, we propose a variance-based algorithm for block size decision, an improved filter-based algorithm for prediction mode decision using contextual information, and a selection algorithm for intra block decision that exploits the relation between the rate-distortion characteristic and the best coding type. Performance comparison is provided to show the improvement of the proposed algorithms over previous methods.

Improving video coding quality by perceptual rate-distortion optimization

The goal of this work is to seek a feasible direction of video coding that can provide a significant quality improvement over H.264. In light of the well-known findings that the distortion metric for video quality has a profound impact on video coding performance and that traditional metrics such as mean square error are poorly correlated with human perception, we identify perceptual video coding, more specifically, perceptual-based rate-distortion optimization, as a sensible approach that has the potential to help drive the performance of video coding to a significantly higher quality level. This technology assessment is supported by experiments with various video sequences, bit-rates, and encoding profiles. The results show that the perceptual-based RDO can indeed bring significant quality improvement for H.264.

A perceptual-based approach to bit allocation for H.264 encoder

Since the ultimate receivers of encoded video are human eyes, the characteristics of human visual system should be taken into consideration in the design of bit allocation to improve the perceptual video quality. In this paper, we incorporate the structural similarity index as a distortion metric and propose a novel rate-distortion model to characterize the relationship between rate and the structural similarity index. Based on the model, we develop an optimum bit allocation and rate control scheme for H.264 encoders. Experimental results show that up to 25% bitrate reduction over the JM reference software can be achieved. Subjective evaluation further confirms that the proposed scheme preserves more structural information and improves the perceptual quality of the encoded video.

Perceptual-based coding mode decision

The framework of rate-distortion optimization (RDO) has been widely adopted for video coding to achieve a good trade-off between bit-rate and distortion. However, objective distortion metrics such as mean square error traditionally used in this framework are poorly correlated with perceptual video quality. To address this issue, we incorporate the structural similarity index as a quality metric into the framework and develop a predictive Lagrange multiplier selection technique to resolve the chicken-and-egg dilemma of perceptual-based RDO. The resulting perceptual-based RDO is then applied to H.264 intra mode decision as an illustration of the application of the proposed technique. Given a perceptual quality level, 5%–10% bit rate reduction over the JM reference software of H.264 is achieved. Subjective evaluation further confirms that, at the same bit-rate, the proposed perceptual RDO preserves image details and prevents block artifact better than the traditional RDO.

Quality Improvement of Video Codec by Rate-Distortion Optimized Quantization

Conventional quantization methods consider only the distortion between original and reconstructed video as the cost of compression. Considering the time-varying nature of network bandwidth for multimedia services, we believe a video coding system can provide a better quality of experience if it takes the bit rate of the compressed bit stream into consideration as well when optimizing the quantization. In this paper we present a rate-distortion optimization approach to the quantization of video coding. This approach is able to balance between rate and distortion for quantization and enhance the overall quality of the entire coding system, with only a slight increase in computational overhead. We implement this method in H.264/AVC, and the extensive experimental data obtained under various test conditions show that the performance of the R-D optimized quantization is indeed better than the H.264 reference software.

Efficient MB and prediction mode decisions for intra prediction of H.264 High Profile

The rate-distortion optimization framework employed in H.264 for the selection of best coding mode is computationally expensive, although it achieves remarkable coding efficiency. In this paper, an integrated intra prediction algorithm for intra-frame coding of the H.264 High Profile is proposed. It consists of two components: a variance-based method for macroblock mode decision and an improved filter-based method for prediction mode decision using contextual information. The two components can work independently or jointly to achieve higher efficiency. The integrated algorithm achieves an average saving of 51.3% (maximum 66.0%) in encoding time for intra coded frames, with negligible effect on PSNR and bit-rate.

Fast H.264 selective intra mode decision for inter-frame coding

Despite its remarkable coding efficiency, the computational complexity of the spatial-domain intra prediction scheme of H.264 remains a challenging issue especially for applications of the High Profile, as it incorporates the additional intra 8×8 prediction mode. To address this issue, we propose in this paper a fast selective intra mode decision algorithm that exploits a newly discovered property of inter-frame coding that the difference in rate-distortion cost between the best inter mode and the intra 16×16 mode is highly related to the type of the best coding mode. The proposed algorithm has high prediction accuracy, low computational overhead, and negligible effect on PSNR and bit-rate. Performance comparison is provided to show the superiority of the algorithm.

Recent progress on perceptual video coding

The purpose of this technical demonstration is to show how the compressed video quality can be improved by taking the characteristics of human visual system into account in the rate distortion optimization and rate control processes of a video encoder. Both the R-D performance of the video encoder and the picture quality can be evaluated subjectively by the viewers in this demonstration to witness that perceptual video coding is indeed a promising direction for next generation video coding.