Shankar L. Regunathan

Video coding with optimal inter/intra-mode switching for packet loss resilience

Resilience to packet loss is a critical requirement in predictive video coding for transmission over packet-switched networks, since the prediction loop propagates errors and causes substantial degradation in video quality. This work proposes an algorithm to optimally estimate the overall distortion of decoder frame reconstruction due to quantization, error propagation, and error concealment. The method recursively computes the total decoder distortion at pixel level precision to accurately account for spatial and temporal error propagation. The accuracy of the estimate is demonstrated via simulation results. The estimate is integrated into a rate-distortion (RD)-based framework for optimal switching between intra-coding and inter-coding modes per macroblock. The cost in computational complexity is modest. The framework is further extended to optimally exploit feedback/acknowledgment information from the receiver/network. Simulation results both with and without a feedback channel demonstrate that precise distortion estimation enables the coder to achieve substantial and consistent gains in PSNR over known state-of-the-art RD- and non-RD-based mode switching methods.

Toward optimality in scalable predictive coding

A method is proposed for efficient scalability in predictive coding, which overcomes known fundamental shortcomings of the prediction loop at enhancement layers. The compression efficiency of an enhancement-layer is substantially improved by casting the design of its prediction module within an estimation-theoretic framework, and thereby exploiting all information available at that layer for the prediction of the signal, and encoding of the prediction error. While the most immediately important application is in video compression, the method is derived in a general setting and is applicable to any scalable predictive coder. Thus, the estimation-theoretic approach is first developed for basic DPCM compression and demonstrates the power of the technique in a simple setting that only involves straightforward prediction, scalar quantization, and entropy coding. Results for the scalable compression of first-order Gauss-Markov and Laplace-Markov signals illustrate the performance. A specific estimation algorithm is then developed for standard scalable DCT-based video coding. Simulation results show consistent and substantial performance gains due to optimal estimation at the enhancement-layers.

On zero-error source coding with decoder side information

Let (X,Y) be a pair of random variables distributed over a finite product set V/spl times/W according to a probability distribution P(x,y). The following source coding problem is considered: the encoder knows X, while the decoder knows Y and wants to learn X without error. The minimum zero-error asymptotic rate of transmission is shown to be the complementary graph entropy of an associated graph. Thus, previous results in the literature provide upper and lower bounds for this minimum rate (further, these bounds are tight for the important class of perfect graphs). The algorithmic aspects of instantaneous code design are considered next. It is shown that optimal code design is NP-hard. An optimal code design algorithm is derived. Polynomial-time suboptimal algorithms are also presented, and their average and worst case performance guarantees are established.

The asymptotic closed-loop approach to predictive vector quantizer design with application in video coding

The basic vector quantization (VQ) technique employed in video coding belongs to the category of predictive vector quantization (PVQ), as it involves quantization of the (motion compensated) frame prediction error. It is well known that the design of PVQ suffers from fundamental difficulties, due to the prediction loop, which have an impact on the convergence and the stability of the design procedure. We propose an approach to PVQ design that enjoys the stability of open-loop design while it ensures ultimate optimization of the closed-loop system. The method is derived for general predictive quantization, and we demonstrate it on video compression at low bit rates, where it provides substantial improvement over standard open and closed loop design techniques. Further, the approach outperforms standard DCT-based video coding.

End-to-end distortion estimation for RD-based robust delivery of pre-compressed video

Applications where packetized video is streamed over the Internet, must be designed to achieve robustness to packet loss as well as compression efficiency. Whenever possible, the ideal solution to this problem is to jointly optimize the adaptation of the compression and error protection strategies to the network status, so as to minimize the expected end-to-end distortion of reconstructed video at the receiver. However, in the case of pre-compressed video streaming, compression is performed without knowledge of the network condition, and conversely the delivery is performed without access to the original signal. It is hence difficult for the transmitter to estimate and minimize the end-to-end distortion during delivery. This paper addresses this problem by deriving an algorithm which enables the transmitter, or other intermediate nodes, to estimate the overall end-to-end distortion while delivering pre-compressed video. This estimate fully accounts for the effects of (prior) quantization, packet loss and error propagation, as well as error concealment. The accuracy of the estimate is demonstrated by simulation results. The algorithm requires storage of minimal side-information that is computed during compression. The algorithm is of low complexity, and is applicable to virtually all coding techniques, including the standard (predictive) video coders. The paper also discusses the use of this estimate to adapt a variety of packet-loss resilience techniques for pre-compressed video streaming. The considerable potential gains of this approach are illustrated via the example of an FEC-based streaming video system.

Multiple description quantization by deterministic annealing

The design of vector quantizers for diversity-based communication over two or more channels of possibly differing capacities and failure probabilities, is considered. The crucial dependence of current design techniques on initialization, especially of index assignment, is well recognized. Instead, we propose to pursue a deterministic annealing approach which is independent of initialization, does not assume any prior knowledge of the source density, and avoids many poor local minima of the cost surface. The approach consists of iterative optimization of a random encoder at gradually decreasing levels of randomness as measured by the Shannon entropy. At the limit of zero entropy, a hard multiple description (MD) quantizer is obtained. This process is directly analogous to annealing processes in statistical physics. Via an alternative derivation, we show that it may also be interpreted as approximating the minimum rate sums among points on the convex hull of the MD achievable rate-distortion region of El Gamal and Cover, subject to constraints on the sizes of the reproduction alphabets. To illustrate the potential of our approach, we present simulation results that show substantial performance gains over existing design techniques.

On zero-error coding of correlated sources

The problem of separate zero-error coding of correlated sources is considered. Inner and outer single-letter bounds are established for the achievable rate region, and conditions for their coincidence are investigated. It is shown that successive encoding combined with time sharing is not always an optimal coding strategy. Conditions for its optimality are derived. The inner bound to the achievable rate region follows as a special case of the single-letter characterization of a generalized zero-error multiterminal rate-distortion problem. The applications of this characterization to a problem of remote computing are also explored. Other results include (i) a product-space characterization of the achievable rates, (ii) bounds for finite block length, and (iii) asymptotic fixed-length rates.

Optimized video streaming over lossy networks with real-time estimation of end-to-end distortion

This paper is concerned with adaptive and robust video streaming over lossy networks. A general system is described which is compatible with virtually all application settings. The delivery procedure is optimized within a rate-distortion framework with the aid of an efficient real-time end-to-end distortion estimation algorithm. The estimate fully accounts for the effects of (prior) quantization, packet loss and error propagation, as well as error concealment. It features high accuracy and low complexity, and requires a small amount of pre-computed side information. It is shown that various error-resilient schemes can be optimized within the proposed framework. In particular, unequal error protection through forward error control is employed in the simulations to demonstrate the achievable performance gains.

Efficient prediction in multiple description video coding

This work is concerned with the design of multiple description (MD) compression systems with emphasis on video coding. Although inter-frame prediction is critical to the performance of video coders, the problem of efficient prediction in MD systems has not been satisfactorily resolved. We focus on quantizer based MD coding and propose an estimation theoretic (ET) approach to prediction and reconstruction. The advantage of ET prediction is two fold: (1) It takes into account all the information available at each decoder for an optimal estimate; (2) It mitigates the degradation due to quantization in the prediction feedback loop. The ET approach is first shown to achieve significant gains in the simpler setting of predictive MD coding of synthetic Gauss-Markov and Laplace-Markov processes. We then present performance results for MD compression of video sequences and demonstrate consistent and substantial PSNR gains over conventional techniques.

Scalable video coding with robust mode selection

We propose to improve the packet loss resilience of scalable video coding. An algorithm for optimal coding mode selection for the base and enhancement layers is developed, which limits error propagation due to packet loss, while retaining compression efficiency. We first derive a method to estimate the overall decoder distortion, which includes the effects of quantization, packet loss and error concealment employed at the decoder. The estimate accounts for temporal and spatial error propagation due to motion compensated prediction, and computes the expected distortion precisely per pixel. The distortion estimate is incorporated within a rate-distortion framework to optimally select the coding mode as well as quantization step size for the macroblocks in each layer. Simulation results show substantial performance gains for both base and enhancement layers.