Kaifang Yang

An efficient algorithm to eliminate temporal pumping artifact in video coding with hierarchical prediction structure

A metric for just noticeable temporal pumping artifact (TPA) is established.An efficient TPA eliminating algorithm in video coding is proposed.The proposed algorithm achieves a more smooth subjective video quality.The proposed algorithm improves the observe comfort of the human eye. At low bit-rates, hierarchical prediction structure using quantization parameter cascading strategy will introduce a new temporal distortion, i.e., the temporal pumping artifact (TPA) presenting itself as a stumbling effect which seriously affect the subjective video reconstructive quality. This paper analyzes one important key factor which greatly influence the critical perceptual of the TPA and proposes a metric for just noticeable temporal pumping artifact (JNTPA) based on spatial-temporal masking effects in human visual system (HVS). Finally, an efficient TPA eliminating algorithm in video coding based on the JNTPA has been proposed. The subsequent experiments demonstrate that the estimated JNTPA values using the proposed metric are in line with the HVS perception and comparable with JVT-P104, the proposed TPA eliminating algorithm achieves a more smooth subjective video quality and improves the observe comfort of the human eye. At the same bit-rate, better subjective video quality can be observed using the eliminating algorithm.

Perceptual based SAO rate-distortion optimization method with a simplified JND model for H.265/HEVC

In the latest H.265/High Efficiency Video Coding (HEVC) standard, the sample adaptive offset (SAO) filter technique is adopted to improve the quality of the reconstructed video. But so far, the research efforts related to the SAO optimization to date have mainly centered on objective rate-distortion (R-D) performance without considering its visual quality. In this paper, the human visual characteristics (represented by a JND model) are introduced into the SAO optimization process for the first time, and a new human visual perceptual-based SAO R-D optimization method, referred to as P-SAO, for H.265/HEVC is proposed. Simultaneously, considering the SAO R-D optimization in H.265/HEVC is complex and in order to use the JND model more effectively and to minimize the calculation time of the introduced JND in SAO, a simplified JND model is proposed based on a modified Sobel operator. Experimental results show that compared with the latest JND model in pixel domain, the proposed JND model can achieve similar subjective quality with significantly reduced computational complexity (i.e., an average processing time reduction of 89.35%). Compared with the original SAO R-D method in the reference software model of the H.265/HEVC, the P-SAO method can achieve better image subjective quality with performance gain of up to 0.2505dB in terms of Δ P S P N R without comprising the R-D performance in Δ P S N R . The human visual characteristic is first introduced into SAO optimization process.A new simplified just noticeable distortion (JND) model is proposed.A modified Sobel operator is proposed.The proposed method can achieve better image subjective quality.

Perception-based Quantitative Definition of Temporal Pumping Artifact

Temporal noises or distortions, such as flickering, jerkiness, and mosquito noise, have been studied for many years in video coding, but a completely mathematical definition describing the perception characters of these temporal noises has not been investigated nor established. In this paper, a distinct type of temporal coding distortion, i.e., the temporal pumping artifact (TPA) is addressed, which manifests itself as visually annoying pumping and stumbling effects caused by applying inappropriate quantization parameter assignment in the state-of-the-art video coding standards with the hierarchical prediction structure. In this paper, the TPA is treated as a spatio-temporal signal and a detailed and comprehensive perception-based quantitative definition in terms of its phase, amplitude, and frequency, respectively is given for the fist time. It is both useful for understanding, or proposing algorithms to alleviate, or even eliminate the TPA and useful for the related studies for other temporal noises or distortions.

Efficient bitstream extraction for scalable video based on simulated annealing

This paper presents an efficient method for bitstream extraction for scalable video based on simulated annealing. Following the same spirit as annealing, the proposed method searches for the optimized combination of quality layers for extraction through slowly reducing the simulated temperature according to the characteristics of frames in different temporal levels. Experimental results show that the proposed method provides an optimized performance, which is significantly higher than that of the basic extraction method. When compared with the quality layer‐based extraction method in the reference software model of H.264/SVC (Joint Scalable Video Model), the proposed method can achieve a similar rate‐distortion performance with a significantly reduced computational complexity. Furthermore, the proposed method can obtain a more smoothed video quality, which is always preferable by the end user. Copyright

Detection and estimation of supra-threshold distortion levels of pictures based on just-noticeable difference

A subjective assessment method is described to determine picture quality levels in the supra-threshold region for processed images, with reference to their original counterparts, based on just-noticeable difference (JND) detection experiment. It has been found that the range of JND levels is dependent on picture contents and can be predicted as a function of texture masking factor computed in the pixel domain. The experimental data obtained also reveal that relationship of JND levels in the supra-threshold region and the MSE (mean squared error) can be approximated by a linear function whose slope is modeled as a function of edge and texture contrast masking factors. The model is devised to predict JND levels which provide subjective picture quality rating discernible by human viewers and can be used for visual quality regulated image/video coding, as well as evaluating the capacity of existing objective metrics in predicting picture quality and/or distortion relative to JND based quality/distortion rating categories.

Adaptive quantization parameter cascading for random-access prediction in H.265/HEVC based on dependent R-D models

In H.265/HEVC, the random-access prediction structure helps to increase the coding efficiency and provides the temporal scalability. However, the quantization parameter cascading (QPC) strategy being used is not optimized in terms of the rate-distortion performance. This paper analyzes the rate and distortion dependency between temporal layers, and proposes a PSNR metric-based distortion model and a piecewise rate model to optimize the QPC. Experimental results demonstrate that the proposed algorithm achieves an average APSNR gain of 0.12dB with the maximum APSNR gain being 0.37dB. Meanwhile, an up to 8.7% BD-rate reduction with the average BD-rate reduction being 3.3% can be obtained.

A fast algorithm of bitstream extraction using distortion prediction based on simulated annealing

Scalable video streams can be extracted to meet the bandwidth limitation of different networks and end-users. Bitstream extraction is usually performed at the network proxy or gateway during transmission, where a low computational complexity is always preferred. How to quickly and accurately select the best resolution combination for a video to meet different bandwidth requirements by each user is crucial in bitstream extraction. In this paper a fast algorithm of bitstream extraction for scalable video is proposed. The interlayer dependency between the base quality layer and the first quality layer was used to predict the distortion of higher quality layers. When quality of every layer is available, the proposed method searches for the optimized combination of quality layers based on simulated annealing. Experimental results show that the proposed method provides an optimized performance, which is significantly higher than that can be achieved by the basic extraction method. Compared to the quality layer based extraction method in the reference software model of H.264/SVC (i.e., JSVM), the proposed algorithm can greatly decrease the decoding times from 2NT to only 2 without losing rate-distortion performance. Furthermore, the proposed method obtains a more smoothed video quality which is always favorable to the observer.

A Visual-Masking-Based Estimation Algorithm for Temporal Pumping Artifact Region Prediction

This paper investigates the temporal pumping artifact (TPA) induced by digital video coding using H.264/AVC and H.265/HEVC standards and proposes a visual-masking-based method to estimate regions with perceptible TPA, referred to as VM-TPA-PRE, for head-and-shoulder video sequences which are common in video messaging, video conferencing and video telephony applications. In digitally coded head-and-shoulder video sequences, the TPA manifests itself as a stumbling effect caused by severe quality fluctuations from frame to frame among adjacent pictures which are most likely to be perceived in regions that the human visual system (HVS) is sensitive to. Considering the object-based or region-of-interest-based video coding theory, accurately estimating regions of the TPA perceivable to the HVS is the key to effective assessment and processing of the TPA and to improve visual quality of videos impaired by the TPA. Experimental results clearly show that the estimation by the VM-TPA-PRE is accurate and in line with human perception.

Fast sample adaptive offset for H.265/HEVC based on temporal dependency

The newly introduced in-loop filter in H.265/HEVC, i.e., sample adaptive offset (SAO), can effectively improve the coding efficiency as well as the subjective quality of encoded videos. However, complex operations are needed in the procedure of SAO which imposes challenges for real-time applications. In this paper, the temporal relationships of two SAO parameters between the temporal base layer and higher temporal layers are explored and utilized to propose a fast SAO method for reducing the computational complexity. Experimental results show that, compared to the original SAO implementation in the reference software model of H.265/HEVC, i.e., HM13.0, the proposed fast SAO method achieves an average of 72.28% reduction in SAO processing time with the average BD-rate increment being 0.56% for LD-P, LD-B and RA configurations.

Temporal distortions in H.265/HEVC

In the newly established video coding standard H.265/High Efficiency Video Coding (HEVC), temporal distortions are introduced under certain coding configurations, which are perceptually annoying at medium and low bitrates. In this paper, two typical temporal distortions due to temporal quality fluctuations in HEVC, i.e., temporal flicker artifact (TFA) and temporal pumping artifact (TPA) are explored in terms of their perception. The coding parameters influencing the perception of TPA and TFA are analysed. The related work in this paper is useful for understanding the fundamental reasons of temporal distortions in HEVC and for HEVC coding to alleviate, or even eliminate the TPA and TFA.