Jingjing Dai

Color video denoising based on adaptive color space conversion

Denoising is one of the most common and important task in video processing systems and abundant efforts have been made on video denoising nowadays. Multihypothesis motion compensated filter (MHMCF) is an effective video denoising method, which combines multiple hypotheses obtained from motion estimation through a number of reference frames by weighted average to suppress noise. However, MHMCF only considers denoising of grayscale video signal. In this paper, we apply MHMCF to color video denoising, where the RGB video is first transformed to the luminance-color difference space before denoising. Instead of using traditional YCbCr color conversion, we propose a novel color conversion matrix which is adaptive to the noise variance in R, G, B channels. Simulation results demonstrate that our proposed color space conversion method can successfully improve the denoising performance for color video.

Film grain noise removal and synthesis in video coding

In this paper, we propose new techniques for film grain noise removal and synthesis, which can be applied in video coding. Film grain noise is clearly noticeable in high-definition video, and should be preserved for the sake of natural look. However, film grain noise tends to reduce the coding efficiency because of its random nature. In our work, prior to video encoding, essential parameters of film grain noise are estimated and the noise is removed by temporal filtering; at the decoder size, film grain noise is modeled by an autoregressive (AR) model, synthesized using the estimated parameters, and added back to the decoded video. Simulation results show that the proposed algorithms can considerably reduce the bitrate and at the same time achieve good subjective quality.

Motion vector coding based on predictor selection and boundary-matching estimation

In most of recent video coding standards based on block-based hybrid coding scheme, especially in the state-of-the-art H.264/AVC standard, motion vector information occupies a considerable portion of the whole compressed bitstream. There-fore, the efficient coding of motion vectors has become an essential objective to further reduce the bitrate. In this paper, we propose a novel motion vector coding method based on predictor selection and estimation. First, the optimal motion vector predictor (MVP) is chosen from a predefined predictor candidate set consisting of both spatial predictors and temporal predictors to minimize the number of bits used for encoding motion vector difference (MVD). Then, in order to avoid sending extra bits for informing the decoder which candidate is selected by the encoder, boundary-matching (BM) estimation is applied at the decoder side to find out the optimal predictor. The basic principle of BM estimation is to preserve the spatial continuity of boundaries between currently reconstructed block and its neighbors. Simulation results show that compared to the original H.264/AVC codec, the proposed scheme improves coding efficiency for various video sequences.

An Analytic Framework for Frame-Level Dependent Bit Allocation in Hybrid Video Coding

In this paper, an analytical framework for frame-level dependent bit allocation (DBA) in hybrid video coding is proposed. First, the dependency of neighboring frames is quantitatively measured with the proposed inter-frame dependency model (IFDM). Based on the proposed IFDM, the problem of frame-level DBA among a number of frames of different frame types is studied, and the optimal solution is achieved through successive convex optimization. The prove the validity of the proposed framework, a case study of current state-of-the-art standard H.264/AVC is conducted. Experimental results show that significant gain of up to 0.9dB in PSNR can be obtained.

Rate-Distortion Optimized Transforms Based on the Lloyd-Type Algorithm for Intra Block Coding

The directional intra prediction (IP) in H.264/AVC and HEVC tends to cause the residue to be anisotropic. To transform the IP residue, Mode Dependent Directional Transform (MDDT) based on Karhunen Loève transform (KLT) can achieve better energy compaction than DCT, with one transform assigned to each prediction mode. However, due to the data variation, different residue blocks with the same IP mode may not have the same statistical properties. Instead of constraining one transform for each IP mode, in this paper, we propose a novel rate-distortion optimized transform (RDOT) scheme which allows a set of specially trained transforms to be available to all modes, and each block can choose its preferred transform to minimize the rate-distortion (RD) cost. We define a cost function which is an estimate of the true RD cost and use the Lloyd-type algorithm (a sequence of transform optimization and data reclassification alternately) to find the optimal set of transforms. The proposed RDOT scheme is implemented in HM9.0 software of HEVC. Experimental results suggest that RDOT effectively achieves 1.6% BD-Rate reduction under the Intra Main condition and 1.6% BD-Rate reduction under the Intra High Efficiency (HE) 10bit condition.

An efficient motion vector coding algorithm based on adaptive predictor selection

Motion Estimation is a core part of modern video coding standards, which significantly improves the compression efficiency. On the other hand, motion information takes considerable portion of compressed bit stream, especially in low bit rate situation. In this paper, an efficient motion vector prediction algorithm is proposed to minimize the bits used for coding the motion information. Several spatial and temporal neighboring motion vectors are selected as the motion vector predictor (MVP) candidates. By applying template matching to each block, a near-optimal MVP can be obtained both at the encoder and decoder side, thus no predictor index is needed to signal to the decoder. We also embed the MVP into current motion estimation process. Furthermore, a correction technique is executed as a remedy when template matching picks out a non-efficient predictor. Simulation results indicate that a bit rate reduction of up to 7.29% over H.264/AVC is achieved by the proposed scheme.

A fast NL-Means method in image denoising based on the similarity of spatially sampled pixels

As one of the best image denoising methods, the Non-Local Means(NL-Means)algorithm[5] proposed by Buades et al. generates state-of-the-art performance. However, due to the high computational complexity, it is difficult to be directly used in practical applications. In this paper, a novel fast algorithm based on the similarity of spatially sampled pixels is introduced. Compared with other fast approaches, the result has shown that our method always uses the shortest time. Meanwhile, it keeps a similar or even better visual results. A maximum of 0.9dB improvement can be attained in comparison with the original method when the noise variance is small.

Rate distortion optimized transform for intra block coding for HEVC

The Discrete Cosine Transform is statistically optimal for first order Markov signals, which is widely used in image and video coding. However, in the intra frame coding of H.264/AVC, it is known that after directional intra prediction, there is still anisotropic features left in residue signals. And the features are related to the intra prediction modes. In order to represent the corresponding features, mode dependent directional transform (MDDT), which is based on Karhunen Loeve transform (KLT), was derived and adopted into JMKTA software, which is a preliminary software platform for High Efficiency Video Coding (HEVC)[2]. Within the MDDT scheme, each prediction mode has its correspondent transform. However, due to the data variation, this mode dependent classification may not lead to optimal residual data separation. It means that even though the residue blocks are using the same prediction mode, they may exhibit different statistical or structural properties. Sometimes, the mode dependent transform basis functions can not represent the residue signal very well. Therefore, in this paper, we propose a rate distortion optimized transform scheme, which provides a transform selection capability. The proposed scheme is implemented in HM0.9 for HEVC, achieving 3.2% BDBR in Intra Low Complexity (LoCo) condition and 2.0% BDBR in Intra High Efficiency (HE) condition.

Frame-level dependent bit allocation via geometric programming

In this paper, a novel frame-level dependent bit allocation (DBA) method is proposed. Our contribution is two-fold: First, the dependency between adjacent frames is quantitatively measured by the introduced inter-frame dependency model(IFDM). The IFDM not only holds for slow video sequences, but remains valid for video sequences of median and high motion as well. Second, based on the IFDM, the conventional DBA problem is revisited and finally it is categorized to a geometric programming problem which can be optimally and effectively solved using interior-point methods. Experimental results show that a significant gain of up to 0.5dB in PSNR can be obtained.

Generalized multihypothesis motion compensated filter for grayscale and color video denoising

This work deals with the additive white Gaussian noise reduction in grayscale and color video sequences. The first main contribution of this paper is the generalized multihypothesis motion compensated filter (GMHMCF) which combines the merits of the traditional time-recursive filter and non-recursive filter in the sense that the reference frame buffer of GMHMCF consists of the denoised previous frames as well as the noisy future frames, such that both the backward and the forward inter-frame correlation can be well exploited. To establish the temporal correspondence between neighboring frames, GMHMCF employs a noise-robust motion estimation (ME) with a pre-defined motion vector (MV) regularization term to construct multiple temporal predictions (hypotheses), which are combined with the current noisy observation through a linear optimal estimator to restore the noise-free signal. The denoising performance of different reference frame configurations is analytically discussed and experimentally tested. Another important contribution of this paper is to extend the GMHMCF to color noise reduction. We examine the primary factors that affect the denoising error of the linear estimator and derive an adaptive optimal luminance-chrominance space such that, when the RGB samples are converted to that new space, GMHMCF can be applied to the individual color components to achieve the minimum overall denoising error. The experiments conducted on representative test video sequences demonstrate that the proposed method provides promising results and is competitive with other state-of-the-art denoising techniques both in terms of objective metric and in perceptual quality.