Bernd Girod

Distributed Video Coding

Distributed coding is a new paradigm for video compression, based on Slepian and Wolfs and Wyner and Zivs information-theoretic results from the 1970s. This paper reviews the recent development of practical distributed video coding schemes. Wyner-Ziv coding, i.e., lossy compression with receiver side information, enables low-complexity video encoding where the bulk of the computation is shifted to the decoder. Since the interframe dependence of the video sequence is exploited only at the decoder, an intraframe encoder can be combined with an interframe decoder. The rate-distortion performance is superior to conventional intraframe coding, but there is still a gap relative to conventional motion-compensated interframe coding. Wyner-Ziv coding is naturally robust against transmission errors and can be used for joint source-channel coding. A Wyner-Ziv MPEG encoder that protects the video waveform rather than the compressed bit stream achieves graceful degradation under deteriorating channel conditions without a layered signal representation.

Analysis of video transmission over lossy channels

A theoretical analysis of the overall mean squared error (MSE) in hybrid video coding is presented for the case of error prone transmission. Our model covers the complete transmission system including the rate-distortion performance of the video encoder, forward error correction, interleaving, and the effect of error concealment and interframe error propagation at the video decoder. The channel model used is a 2-state Markov model describing burst errors on the symbol level. Reed-Solomon codes are used for forward error correction. Extensive simulation results using an H.263 video codec are provided for verification. Using the model, the optimal tradeoff between INTRA and INTER coding as well as the optimal channel code rate can be determined for given channel parameters by minimizing the expected MSE at the decoder. The main focus of this paper is to show the accuracy of the derived analytical model and its applicability to the analysis and optimization of an entire video transmission system.

Watermarking of uncompressed and compressed video

Abstract In this paper, methods for embedding additive digital watermarks into uncompressed and compressed video sequences are presented. The basic principle borrows from spread spectrum communications. It consists of addition of an encrypted, pseudo-noise signal to the video that is invisible, statistically unobtrusive, and robust against manipulations. For practical applications, watermarking schemes operating on compressed video are desirable. A method for watermarking of MPEG-2 encoded video is presented. The scheme is a compatible extension of the scheme operating on uncompressed video. The watermark is generated exactly in the same manner as for uncompressed video, transformed using the discrete cosine transform (DCT) and embedded into the MPEG-2 bit-stream without increasing the bit-rate. The watermark can be retrieved from the decoded video and without knowledge of the original, unwatermarked video. Although an existing MPEG-2 bit-stream is partly altered, the scheme avoids visible artifacts by addition of a drift compensation signal. The proposed method is robust and of much lower complexity than a complete decoding process followed by watermarking in the pixel domain and re-encoding. Fast implementations exist which have a complexity comparable to a video decoder. Experimental results are given. The scheme is also applicable to other hybrid transform coding schemes like MPEG-1, MPEG-4, H.261, and H.263.

Wyner-Ziv coding of motion video

In current interframe video compression systems, the encoder performs predictive coding to exploit the similarities of successive frames. The Wyner-Ziv theorem on source coding with side information available only at the decoder suggests that an asymmetric video codec, where individual frames are encoded separately, but decoded conditionally (given temporally adjacent frames) could achieve similar efficiency. We report the first results on a Wyner-Ziv coding scheme for motion video that uses intraframe encoding, but interframe decoding.

Transform-domain Wyner-Ziv Codec for Video

In current interframe video compression systems, the encoder performs predictive coding to exploit the similarities of successive frames. The Wyner-Ziv Theorem on source coding with side information available only at the decoder suggests that an asymmetric video codec, where individual frames are encoded separately, but decoded conditionally (given temporally adjacent frames) could achieve similar efficiency. We propose a transformdomain Wyner-Ziv coding scheme for motion video that uses intraframe encoding, but interframe decoding. In this system, the transform coefficients of a Wyner-Ziv frame are encoded independently using a scalar quantizer and turbo coder. The decoder uses previously reconstructed frames to generate side information to conditionally decode the Wyner-Ziv frames. Simulation results show significant gains above DCT-based intraframe coding and improvements over the pixel-domain Wyner-Ziv video coder.

Compression with side information using turbo codes

We show that turbo codes can come close to the Slepian-Wolf bound in lossless distributed source coding. In the asymmetric scenario considered, X and Y are statistically dependent signals and X is encoded with no knowledge of Y. However, Y is known as side information at the decoder. We use a system based on turbo codes to send X at a rate close to H(X|Y). We apply our system to binary sequences and simulations show performance close to the information-theoretic limit. For distributed source coding of Gaussian sequences, our results show significant improvement over previous work. The scheme also performs well for joint source-channel coding.

Long-term memory motion-compensated prediction

Long-term memory motion-compensated prediction extends the spatial displacement vector utilized in block-based hybrid video coding by a variable time delay permitting the use of more frames than the previously decoded one for motion compensated prediction. The long-term memory covers several seconds of decoded frames at the encoder and decoder. The use of multiple frames for motion compensation in most cases provides significantly improved prediction gain. The variable time delay has to be transmitted as side information requiring an additional bit rate which may be prohibitive when the size of the long-term memory becomes too large. Therefore, me control the bit rate of the motion information by employing rate constrained motion estimation. Simulation results are obtained by integrating long-term memory prediction into an H.263 codec. Reconstruction PSNR improvements up to 2 dB for the Foreman sequence and 1.5 dB for the Mother-Daughter sequence are demonstrated in comparison to the TMN-2.0 H.263 coder. The PSNR improvements correspond to bit-rate savings up to 34 and 30%, respectively. Mathematical inequalities are used to speed up motion estimation while achieving full prediction gain.

Scalar Costa scheme for information embedding

Research on information embedding, particularly information hiding techniques, has received considerable attention within the last years due to its potential application in multimedia security. Digital watermarking, which is an information hiding technique where the embedded information is robust against malicious or accidental attacks, might offer new possibilities to enforce the copyrights of multimedia data. In this paper, the specific case of information embedding into independent identically distributed (IID) data and attacks by additive white Gaussian noise (AWGN) is considered. The original data is not available to the decoder. For Gaussian data, in 1983, Costa proposed a scheme that theoretically achieves the capacity of this communication scenario. However, Costas scheme is not practical. Thus, several research groups have proposed suboptimal practical communication schemes based on Costas idea. The goal of this paper is to give a complete performance analysis of the scalar Costa scheme (SCS), which is a suboptimal technique using scalar embedding and reception functions. Information theoretic bounds and simulation results with state-of-the-art coding techniques are compared. Further, reception after amplitude scaling attacks and the invertibility of SCS embedding are investigated.

Outdoors augmented reality on mobile phone using loxel-based visual feature organization

We have built an outdoors augmented reality system for mobile phones that matches camera-phone images against a large database of location-tagged images using a robust image retrieval algorithm. We avoid network latency by implementing the algorithm on the phone and deliver excellent performance by adapting a state-of-the-art image retrieval algorithm based on robust local descriptors. Matching is performed against a database of highly relevant features, which is continuously updated to reflect changes in the environment. We achieve fast updates and scalability by pruning of irrelevant features based on proximity to the user. By compressing and incrementally updating the features stored on the phone we make the system amenable to low-bandwidth wireless connections. We demonstrate system robustness on a dataset of location-tagged images and show a smart-phone implementation that achieves a high image matching rate while operating in near real-time.

Cross-layer design of ad hoc networks for real-time video streaming

Cross-layer design breaks away from traditional network design where each layer of the protocol stack operates independently. We explore the potential synergies of exchanging information between different layers to support real-time video streaming. In this new approach information is exchanged between different layers of the protocol stack, and end-to-end performance is optimized by adapting to this information at each protocol layer. We discuss key parameters used in the cross-layer information exchange along with the associated cross-layer adaptation. Substantial performance gains through this cross-layer design are demonstrated for video streaming.