Christophe Tillier

Motion compensation and scalability in lifting-based video coding

Abstract Motion-compensated temporal filtering subband video codecs have attracted recently a lot of attention, due to their compression performance comparable with that of state-of-the-art hybrid codecs and due to their additional scalability features. In this paper, we present a scalable video codec based on a 5/3 adaptive temporal lifting decomposition. Different adaptation criteria for coping with the occluded areas are discussed and new criteria for optimizing the temporal prediction are introduced. For our simulations, we use a memory-constraint “on-the-fly” implementation. We also evaluate the temporal scalability properties of this video coding structure.

A motion-compensated overcomplete temporal decomposition for multiple description scalable video coding

We present a new multiple-description coding (MDC) method for scalable video, designed for transmission over error-prone networks. We employ a redundant motion-compensated scheme derived from the Haar multiresolution analysis, in order to build temporally correlated descriptions in a t+2D video coder. Our scheme presents a redundancy which decreases with the resolution level. This is achieved by additionally subsampling some of the wavelet temporal subbanbds. We present an equivalent four-band lifting implementation leading to simple central and side decoders as well as a packet-based reconstruction strategy in order to cope with random packet losses.

3D, 3-band, 3-tap temporal lifting for scalable video coding

Recently, a new generation of video compression techniques gained the interest of the research engineering community, by proposing state-of-the-art compression performance in addition to temporal/spatial/SNR scalability features. They exploit interframe redundancy through a temporal wavelet transform in the motion direction. While classical two-band decompositions offer dyadic factors of scalability, in this paper we present a three-band temporal lifting structure providing a factor 3 of scalability in a motion-compensated subband video coding scheme. More flexibility is thus available in order to adapt the bitstream to the bandwidth variations in heterogeneous networks. We also show improved compression performance compared with a dyadic Haar MCTF analysis.

Distributed temporal multiple description coding for robust video transmission

The problem of multimedia communications over best-effort networks is addressed here with multiple description coding (MDC) in a distributed framework. In this paper, we first compare four video MDC schemes based on different time splitting patterns and temporal two- or three-band motion-compensated temporal filtering (MCTF). Then, the latter schemes are extended with systematic lossy description coding where the original sequence is separated into two subsequences, one being coded as in the latter schemes, and the other being coded with a Wyner-Ziv (WZ) encoder. This amounts to having a systematic lossy Wyner-Ziv coding of every other frame of each description. This error control approach can be used as an alternative to automatic repeat request (ARQ) or forward error correction (FEC), that is, the additional bitstream can be systematically sent to the decoder or can be requested, as in ARQ. When used as an FEC mechanism, the amount of redundancy is mostly controlled by the quantization of the Wyner-Ziv data. In this context, this approach leads to satisfactory rate-distortion performance at the side decoders, however it suffers from high redundancy which penalizes the central description. To cope with this problem, the approach is then extended to the use of MCTF for the Wyner-Ziv frames, in which case only the low-frequency subbands are WZ-coded and sent in the descriptions.

Optimization of the predict operator in lifting-based motion-compensated temporal filtering

The motion-compensated temporal filtering is an essential part of a scalable wavelet-based video coding scheme, which applies a temporal wavelet transform in the motion direction over the frames of a video sequence. The lifting structure of the temporal filter bank tackled in this paper involves a predict operator which makes use of two motion vector fields to bidirectionally predict frames from their neighbouring ones. We show in this paper that there exists an optimal algorithm allowing to estimate jointly these two motion vector fields subject to an optimization criterion directly related to the coding of the detail subbands. We provide an iterative suboptimal form of this algorithm implementing this approach. We show that this algorithm provides substantial gains in terms of PSNR, with the same complexity as a separate estimation of the two motion vector fields.

Redundant multiresolution analysis for multiple description video coding

Multiple description coding (MDC) is a joint source-channel coding technique specifically designed for real-time multimedia applications over best effort switched packet networks (such as Internet), in order to cope with packet losses due to transmission errors or network congestion. Error resilience of transmitted bitstreams is thus significantly increased, but this does not solve problems like bitstream adaptation to bandwidth variations or receiver characteristics, which are in turn addressed by scalable coding techniques. In this paper, we present a new method of multiple description coding of scalable video, combining the scalability features with MDC. We propose a redundant motion-compensated temporal scheme related to Haar multiresolution analysis. We also present an equivalent lifting implementation leading to simple central and lateral decoders.

Joint wavelet packets for groups of frames in MCTF

Wavelet packets provide a flexible representation of data, which has been proved to be very useful in a lot of applications in signal, image and video processing. In particular, in image and video coding, their ability to best capture the input content features can be exploited by designing appropriate optimization criteria. In this paper, we introduce joint wavelet packets for groups of frames, allowing to provide a unique best basis representation for several frames and not one basis per frame, as classically done. Two main advantages are expected from this joint representation. On the one hand, bitrate is spared, since a single tree description is sent instead of 31 per group of frames (GOP) - when a GOP contains, for example, 32 frames. On the other hand, this common description can characterize the spatio-temporal features of the given video GOP and this way can be exploited as a valuable feature for video classification and video database searching. A second contribution of the paper is to provide insight into the modifications necessary in the best basis algorithm (BBA) in order to cope with biorthogonal decompositions. A computationally efficient algorithm is deduced for an entropy-based criterion.