Mengyao Ma

Edge-Directed Error Concealment

In this paper we propose an edge-directed error concealment (EDEC) algorithm, to recover lost slices in video sequences encoded by flexible macroblock ordering. First, the strong edges in a corrupted frame are estimated based on the edges in the neighboring frames and the received area of the current frame. Next, the lost regions along these estimated edges are recovered using both spatial and temporal neighboring pixels. Finally, the remaining parts of the lost regions are estimated. Simulation results show that compared to the existing boundary matching algorithm [1] and the exemplar-based inpainting approach [2] , the proposed EDEC algorithm can reconstruct the corrupted frame with both a better visual quality and a higher decoder peak signal-to-noise ratio.

Temporal Video Denoising Based on Multihypothesis Motion Compensation

Denoising module is required by any practical video processing systems. Most existing denoising schemes are spatio-temporal filters which operate on data over three dimensions. However, to limit the number of inputs, these filters only utilize one reference frame and cannot fully exploit temporal correlation. In this paper, a recursive temporal denoising filter named multihypothesis motion compensated filter (MHMCF) is proposed. To fully exploit temporal correlation, MHMCF performs motion estimation in a number of reference frames to construct multiple hypotheses (temporal predictions) of the current pixel. These hypotheses are combined by weighted averaging to suppress noise and estimate the actual current pixel value. Based on the multihypothesis motion compensated residue model presented in this paper, we investigate the efficiency of MHMCF, and some numerical evaluations are revealed. Experimental results show that MHMCF demonstrates quite good denoising performance while the inputs are much fewer than spatio-temporal filters. Moreover, as a purely temporal filter, it can well preserve spatial details and achieve satisfactory visual quality.

Integration of Recursive Temporal LMMSE Denoising Filter Into Video Codec

The presence of noise can dramatically affect the efficiency of video compression systems. For performance improvement, most practical video compression systems adopt a denoising filter as a pre-processing module for the video encoder, or as a post-processing module for the video decoder, but the complexity introduced by denoising can be very high. This paper first presents a recursive temporal linear minimum mean squared error (LMMSE) filter for video denoising. Based on the analysis of the hybrid video compression process, two novel schemes are presented, one for video encoding and the other for video decoding, in which the proposed recursive temporal LMMSE filter is seamlessly integrated into the encoding and the decoding processes, respectively. For both of these two schemes, the denoising is implemented with nearly no extra computation introduced. Experimental results validate the effectiveness of the proposed schemes on encoding and decoding noisy video sequences.

Bidirectionally decodable Wyner-Ziv video coding

Inter frame prediction technique significantly improves the compression efficiency in the hybrid video coding schemes. However, this technique causes the decoding dependency of each inter frame on all of its reference frames. This dependency complicates the reverse play operation which is the most common video cassette recording (VCR) functions. This dependency also causes error propagation when the video is transmitted over error prone channel. In this paper, we propose a novel bidirectionally decodable Wyner-Ziv video coding scheme which relaxes this inter frame dependency. The proposed bidirectionally decodable Wyner-Ziv frame can be decoded by using whether forward prediction or backward prediction as side information at the decoder, i.e. the proposed stream supports forward decoding and backward decoding simultaneously. Compared with the other schemes which support reverse playback, our scheme requires much lower bandwidth and smaller storage space. In error resilient test, our scheme outperforms H.264 up to 4dB at same bitrate. Our proposed frames also support video splicing and stream switching at arbitrary time point like I-frames.

A Novel Analytic Quantization-Distortion Model for Hybrid Video Coding

A proper theoretical quantization-distortion model for hybrid video coding is always desirable, since this allows us to explain the behavior of existing codecs and to design better ones. However, due to the existence of motion-compensated prediction, hybrid video coding introduces interframe dependency into the encoded video, which makes its quantization-distortion characteristics difficult to analyze. In this paper, a joint analysis of quantization and motion-compensated prediction is presented. For a complete analysis, we investigate not only the distortion that quantization introduces into video signal, but also its effect on motion-compensated prediction. Based on the joint analysis, a quantization-distortion model of hybrid video coding is proposed. Our extensive experimental results show that the proposed model can estimate the quantization-distortion curve of hybrid video coding with high accuracy. Furthermore, the estimation accuracy remains high for various video sequences and encoder configurations.

Error Concealment for Frame Losses in MDC

Multiple description coding (MDC) is an effective error resilience (ER) technique for video coding. In case of frame loss, error concealment (EC) techniques can be used in MDC to reconstruct the lost frame, with error, from which subsequent frames can be decoded directly. With such direct decoding, the subsequent decoded frames will gradually recover from the frame loss, though slowly. In this paper we propose a novel algorithm using multihypothesis error concealment (MHC) to improve the error recovery rate of any EC in the temporal subsampling MDC. In MHC, the simultaneous temporal-interpolated frame is used as an additional hypothesis to improve the reconstructed video quality after the lost frame. Both subjective and objective results show that MHC can achieve significantly better video quality than direct decoding.

Image deblocking using convex optimization

Images encoded at low-bit rate may suffer from blocking artifacts, which can dramatically degrade the visual quality. In this paper, a novel approach to image deblocking is presented. Based on the analysis of image coding process and the property of natural images, an objective function and a set of constraint functions are proposed, and image deblocking is formulated as a convex optimization problem which can be easily solved using numerical methods. The feasibility of the convex optimization problem is utilized to detect the true object edges and avoid blurring. Experimental results demonstrates the effectiveness of the proposed approach.

Fast Multi-Hypothesis Motion Compensated Filter for Video Denoising

Multi-Hypothesis motion compensated filter (MHMCF) utilizes a number of hypotheses (temporal predictions) to estimate the current pixel which is corrupted with noise. While showing remarkable denoising results, MHMCF is computationally intensive as full search is employed in the expectation of finding good temporal predictions in the presence of noise. In the frame of MHMCF, a fast denoising algorithm FMHMCF is proposed in this paper. With edge preserved low-pass prefiltering and noise-robust fast multihypothesis search, FMHMCF could find reliable hypotheses while checking very few search locations, so that the denoising process can be dramatically accelerated. Experimental results show that FMHMCF can be 10 to 14 times faster than MHMCF, while achieving the same or even better denoising performance with up to 1.93 dB PSNR (peak-signal-noise-ratio) improvement.

An Encoder-Embedded Video Denoising Filter Based on the Temporal LMMSE Estimator

Noise not only degrades the visual quality of video contents, but also significantly affects the coding efficiency. Based on the temporal linear minimum mean square error (LMMSE) estimator, an innovative denoising filter is proposed in this paper. The proposed filter only requires simple operations manipulating on the individual residue coefficients and can be seamlessly integrated into video encoders. Compared to traditional filter-encoder cascaded scheme, embedding the proposed filter into the video encoder can save a large amount of computation. The experimental results show that with the proposed filter embedded, both the noise suppression capability and the coding efficiency of the video encoder can be dramatically improved. Furthermore, as a purely temporal filter, it can well preserve the fine details of video contents and satisfactory visual quality can be achieved

A new motion compensation approach for error resilient video coding

Multihypothesis motion-compensated prediction (MHMCP) can be used as an error resilience technique for video coding. Motivated by MHMCP, we propose a new error resilience approach named alternative motion-compensated prediction (AMCP), where two-hypothesis and one-hypothesis predictions are alternatively used with some mechanism. Both theory and simulation results show that in case of one frame loss, the expected converged error using AMCP is smaller than that using two-hypothesis MCP.