Jozef Škorupa

Rate-distortion driven decoder-side bitplane mode decision for distributed video coding

Distributed video coding (DVC) features simple encoders but complex decoders, which lies in contrast to conventional video compression solutions such as H.264/AVC. This shift in complexity is realized by performing motion estimation at the decoder side instead of at the encoder, which brings a number of problems that need to be dealt with. One of these problems is that, while employing different coding modes yields significant coding gains in classical video compression systems, it is still difficult to fully exploit this in DVC without increasing the complexity at the encoder side. Therefore, in this paper, instead of using an encoder-side approach, techniques for decoder-side mode decision are proposed. A rate-distortion model is derived that takes into account the position of the side information in the quantization bin. This model is then used to perform mode decision at the coefficient level and bitplane level. Average rate gains of 13-28% over the state-of-the-art DISCOVER codec are reported, for a GOP of size four, for several test sequences.

Efficient Low-Delay Distributed Video Coding

Distributed video coding (DVC) is a video coding paradigm that allows for a low-complexity encoding process by exploiting the temporal redundancies in a video sequence at the decoder side. State-of-the-art DVC systems exhibit a structural coding delay since exploiting the temporal redundancies through motion-compensated interpolation requires the frames to be decoded out of order. To alleviate this problem, we propose a system based on motion-compensated extrapolation that allows for efficient low-delay video coding with low complexity at the encoder. The proposed extrapolation technique first estimates the motion field between the two most recently decoded frames using the Lucas-Kanade algorithm. The obtained motion field is then extrapolated to the current frame using an extrapolation grid. The proposed techniques are implemented into a novel architecture featuring hybrid block-frequency Wyner-Ziv coding as well as mode decision. Results show that having references from both temporal directions in interpolation provides superior rate-distortion performance over a single temporal direction in extrapolation, as expected. However, the proposed extrapolation method is particularly suitable for low-delay coding as it performs better than H.264/AVC intra, and it is even able to outperform the interpolation-based DVC codec from DISCOVER for several sequences.

Accounting for quantization noise in online correlation noise estimation for Distributed Video Coding

In Distributed Video Coding (DVC), compression is achieved by exploiting correlation between frames at the decoder, instead of at the encoder. More specifically, the decoder uses already decoded frames to generate side information Y for each Wyner-Ziv frame X, and corrects errors in Y using error correcting bits received from the encoder. For efficient use of these bits, the decoder needs information about the correlation between X available at the encoder and Y at the decoder. While several techniques for online estimation of correlation noise X - Y have been proposed, the quantization noise in Y has not been taken into account. As a solution, in this paper, we calculate the quantization noise of intra frames at the encoder and use this information at the decoder to improve the accuracy of the correlation noise estimation. Results indicate averageWyner-Ziv bit rate reductions up to 19.5% (Bjøntegaard delta) for coarse quantization.

Stopping criterions for turbo coding in a Wyner-Ziv video codec

Distributed video coding (DVC) targets video coding applications with low encoding complexity by generating a prediction of the video signal at the decoder. One of the most common architectures uses turbo codes to correct errors in this prediction. Unfortunately, a rigorous analysis of turbo coding in the context of DVC is missing. We have targeted one particular aspect of turbo coding: the stopping criterion. The stopping criterion indicates whether decoding was successful, i.e., whether the errors in the prediction signal have been corrected. In this paper we describe and compare several stopping criterions known from the field of channel coding and criterions currently used in DVC. As our results suggest the choice of the stopping criterion has a significant impact on the overall video-coding performance. Moreover, we have found that there are even better performing criterions than those currently used in DVC.

Exploiting quantization and spatial correlation in virtual-noise modeling for distributed video coding

Aiming for low-complexity encoding, video coders based on Wyner-Ziv theory are still unsuccessfully trying to match the performance of predictive video coders. One of the most important factors concerning the coding performance of distributed coders is modeling and estimating the correlation between the original video signal and its temporal prediction generated at the decoder. One of the problems of the state-of-the-art correlation estimators is that their performance is not consistent across a wide range of video content and different coding settings. To address this problem we have developed a correlation model able to adapt to changes in the content and the coding parameters by exploiting the spatial correlation of the video signal and the quantization distortion. In this paper we describe our model and present experiments showing that our model provides average bit rate gains of up to 12% and average PSNR gains of up to 0.5dB when compared to the state-of-the-art models. The experiments suggest that the performance of distributed coders can be significantly improved by taking video content and coding parameters into account.

Introducing skip mode in distributed video coding

Although it was proven in the 1970s already by Wyner and Ziv and Slepian and Wolf that, under certain conditions, the same rate-distortion boundaries exist for distributed video coding (DVC) systems as for traditional predicting systems, until now no practical DVC system has been developed that even comes close to the performance of state-of-the-art video codecs such as H.264/AVC in terms of rate-distortion. Some important factors for this are the lower accuracy of the motion estimation performed at the decoder, the inaccurate modeling of the correlation between the side information and the original frame, and the absence in most state-of-the-art DVC systems of anything conceptually similar to the notion of skipped macroblocks in predictive coding systems. This paper proposes an extension of a state-of-the-art transform domain residual DVC system with an implementation of skip mode. The skip mode has an impact at two different places: in the turbo decoder, more specifically the soft input, soft output (SISO) convolutional decoder, and in the puncturing of the parity bits. Results show average bitrate gains up to 39% (depending on the sequence) achieved by combining both approaches. Furthermore, a hybrid video codec is presented where the motion estimation task is shifted back to the encoder. This results in a drastic increase in encoder complexity, but also in a drastic performance gain in terms of rate-distortion, with average bitrate savings up to 60% relative to the distributed video codec. In the hybrid video codec, smaller but still important average bitrate gains are achieved by implementing skip mode: up to 24%.

Accurate Correlation Modeling for Transform-Domain Wyner-Ziv Video Coding

In Wyner-Ziv (WZ) video coding, low-complexity encoding is achieved by generating the prediction signal only at the decoder. An accurate model of the correlation between the original frame and its prediction is necessary for efficient coding. Firstly, we propose an improvement for the pixel-domain correlation estimation. In transform-domain WZ video coding current models estimate the necessary correlation parameters directly in the transform domain. We propose an alternative approach, where an accurate estimation in the pixel domain is followed by a novel method of transforming the pixel-domain estimation into the transform domain. The experimental results show that our model leads to average bit-rate gain of 3.5---8%.

Flexible distribution of complexity by hybrid predictive-distributed video coding

There is currently limited flexibility for distributing complexity in a video coding system. While rate-distortion-complexity (RDC) optimization techniques have been proposed for conventional predictive video coding with encoder-side motion estimation, they fail to offer true flexible distribution of complexity between encoder and decoder since the encoder is assumed to have always more computational resources available than the decoder. On the other hand, distributed video coding solutions with decoder-side motion estimation have been proposed, but hardly any RDC optimized systems have been developed. To offer more flexibility for video applications involving multi-tasking or battery-constrained devices, in this paper, we propose a codec combining predictive video coding concepts and techniques from distributed video coding and show the flexibility of this method in distributing complexity. We propose several modes to code frames, and provide complexity analysis illustrating encoder and decoder computational complexity for each mode. Rate distortion results for each mode indicate that the coding efficiency is similar. We describe a method to choose which mode to use for coding each inter frame, taking into account encoder and decoder complexity constraints, and illustrate how complexity is distributed more flexibly.

Decoder-driven mode decision in a block-based distributed video codec

Distributed Video Coding (DVC) is a video coding paradigm in which the computational complexity is shifted from the encoder to the decoder. DVC is based on information theoretic results suggesting that, under ideal conditions, the same rate-distortion performance can be achieved as for traditional video codecs. In practice however, there is still a significant performance gap between the two coding architectures. One of the main reasons for this gap is the lack of multiple coding modes in current DVC solutions. In this paper, we propose a block-based distributed video codec that supports three coding modes: Wyner–Ziv, skip, and intra. The mode decision process is entirely decoder-driven. Skip blocks are selected based on the estimated accuracy of the side information. The choice between intra and Wyner–Ziv coding modes is made on a rate-distortion basis, by selecting the coding mode with the lowest rate while assuring equal distortion for both modes. Experimental results illustrate that the proposed block-based architecture has some advantages over classical bitplane-based approaches. Introducing skip and intra coded blocks yields average bitrate gains of up to 33.7% over our basic configuration supporting Wyner–Ziv mode only, and up to 29.7% over the reference bitplane-based DISCOVER codec.

Distributed Video Coding With Feedback Channel Constraints

Many of the distributed video coding (DVC) systems described in the literature make use of a feedback channel from the decoder to the encoder to determine the rate. However, the number of requests through the feedback channel is often high, and as a result the overall delay of the system could be unacceptable in practical applications. As a solution, feedback-free DVC systems have been proposed, but the problem with these solutions is that they incorporate a difficult trade-off between encoder complexity and compression performance. Recognizing that a limited form of feedback may be supported in many video-streaming scenarios, in this paper we propose a method for constraining the number of feedback requests to a fixed maximum number of N requests for an entire Wyner-Ziv (WZ) frame. The proposed technique estimates the WZ rate at the decoder using information obtained from previously decoded WZ frames and defines the N requests by minimizing the expected rate overhead. Tests on eight sequences show that the rate penalty is less than 5% when only five requests are allowed per WZ frame (for a group of pictures of size four). Furthermore, due to improvements from previous work, the system is able to perform better than or similar to DISCOVER even when up to two requests per WZ frame are allowed. The practical usefulness of the proposed approach is studied by estimating end-to-end delay and encoder buffer requirements, indicating that DVC with constrained feedback can be an important solution in the context of video-streaming scenarios.