Alexis M. Tourapis

3D-TV Content Storage and Transmission

There exist a variety of ways to represent 3D content, including stereo and multiview video, as well as frame-compatible and depth-based video formats. There are also a number of compression architectures and techniques that have been introduced in recent years. This paper provides an overview of relevant 3D representation and compression formats. It also analyzes some of the merits and drawbacks of these formats considering the application requirements and constraints imposed by different storage and transmission systems.

Weighted prediction methods for improved motion compensation

Weighted prediction in the H.264/MPEG-4 AVC video coding standard is a coding tool used primarily for improving coding efficiency during illumination changes, such as fading transitions. In this paper, we extend the usage of weighted prediction within the context of motion compensation in H.264, to further improve coding efficiency even for scenes with constant illumination characteristics. This is done by better exploiting coding tools already available in H.264 such as reference list reordering and multiple reference prediction.

Efficient block-based intra prediction for image coding with 2D geometrical manipulations

A novel block-based intra-prediction scheme is proposed for efficient image (or intra-frame) coding, where we apply various 2D geometrical manipulations to reference image blocks to enrich the pool of prediction blocks for a given target block. As compared with the traditional line-based intra prediction in H.264/AVC, the new scheme offers a significant coding gain (about 0.24-1.23dB in the PSNR value with the same bit rate) at the cost of higher complexity. Several techniques to reduce the search complexity are also discussed.

Correcting unsynchronized zoom in 3D video

When capturing 3D video with a stereoscopic camera setup, it is important for the cameras to be precisely aligned and synchronized. This is particularly difficult in transitions such as zooming where the camera parameters must be changed in unison, or else the perceived 3D effect will be degraded. In this paper we study the problem of unsynchronized zooming in 3D video. First, we present a subjective study that shows that the perceived quality of stereo video is greatly reduced if the two views are zoomed by different amounts. Next, we present a method for correcting zoom mismatch by applying cropping and scaling to ones of the views. Our method involves finding matching points between the left and right views, and performing least-squares regressions to estimate the amount of scaling and cropping required to make the views consistent. Experiments were performed on videos with digitally introduced zoom mismatch and videos with optical unsynchronized zoom. In both cases the results show that our method is highly accurate and produces videos without size differences or vertical parallax between the two views.

Rate control for video coding with slice type dependencies

Rate control is an integral part of a video coding system as it ensures the generation of compressed bit streams that satisfy bandwidth and buffering constraints. Traditional rate control algorithms maintain a rate control model for a single picture type that is applied on pictures of this type. The remaining pictures are allocated a quantization parameter with no consideration of bit rate constraints. Video sequences may be coded with complex prediction structures that affect coding requirements. In this paper, we improve rate control for I- and B-coded pictures, especially in the context of hierarchical coding. We also describe a new rate control paradigm for coding video sequences with multiple slice types that achieves the bit target for all pictures. Both contributions improve rate-distortion performance and rate control accuracy.

Buffered adaptive interpolation filters

H.264/MPEG-4 AVC uses motion prediction with fractional-pixel precision to reduce the temporal redundancy that often exists in a video signal. Previously, it has been shown that Adaptive Interpolation Filter (AIF) methods can significantly improve coding performance compared to the fixed interpolation filters used by existing codecs by adaptively tuning the motion interpolation filters to the video signals statistics. In this work, a new AIF scheme, named the Buffered Adaptive Interpolation Filter (BAIF) scheme, is presented. The proposed scheme allows the video codec to better exploit the statistical similarities that usually exist among consecutive pictures, thereby improving performance. Specifically, filter buffers are established to store the adaptive interpolation filters previously sent in the bitstream and use them to code subsequent pictures. Further, when multiple reference pictures are used, the proposed scheme allows different interpolation filters to be used for different reference pictures. Simulation results show that the proposed scheme consistently achieves better coding performance than existing AIF schemes. Compared to the current state-of-the-art Enhanced Adaptive Interpolation Filter (EAIF) scheme, the proposed scheme achieves additional coding gain of up to 3%. Compared to H.264/AVC, coding performance is improved by up to 20%.

Multiple Reference Motion Compensation: A Tutorial Introduction and Survey

Motion compensation exploits temporal correlation in a video sequence to yield high compression efficiency. Multiple reference frame motion compensation is an extension of motion compensation that exploits temporal correlation over a longer time scale. Devised mainly for increasing compression efficiency, it exhibits useful properties such as enhanced error resilience and error concealment. In this survey, we explore different aspects of multiple reference frame motion compensation, including multihypothesis prediction, global motion prediction, improved error resilience and concealment for multiple references, and algorithms for fast motion estimation in the context of multiple reference frame video encoders.

Motion estimation with entropy coding considerations in H.264/AVC

In many modern video encoders, motion estimation is usually formulated as a Lagrangian cost function that balances motion prediction accuracy and the rate to transmit motion vectors. In H.264/AVC video encoders, including the H.264 reference software, in an effort to reduce computational complexity, the motion vector rate is often estimated using a look-up table based on universal variable length coding regardless of the entropy coding method used. However, the estimated rate could be different from the actual rate used for encoding. In this paper we present an empirical study on the motion vector rate estimation mismatch caused by different entropy coding assumptions and discuss its impact on motion estimation and coding efficiency. We also present a more accurate motion vector rate estimation method for encoding with the context adaptive binary arithmetic coding (CABAC) method while still maintaining low computational complexity. The proposed motion vector rate estimation can provide up to 2.26% in rate savings for low bit rate applications when all modes are considered for the encoding. Results for single mode encoding, which can be useful for fast encoding applications, show that we can achieve benefits of up to 27.15% and 7.39% in rate savings for low and medium to high bit rate applications respectively.

Drift Characterization of Intra Prediction and Quantization in H.264

The combination of intra prediction, transform, and quantization in the H.264/AVC video coding standard offers high compression performance in picture areas where inter prediction is unavailable or fails. Motivated from coding experiments with the reference software that resulted in significant visual distortion with certain types of content, we investigated intra prediction further. Intra prediction may propagate distortion to subsequent blocks or macroblocks. As a result, rate-distortion optimized mode decision needs to account for this distortion. Furthermore, rounding operations in H.264/AVC accumulate small but non-trivial errors. This is especially true for the encoding process of the 16 times 16 intra prediction mode, which involves a combination of DCT and Hadamard transforms. We investigate, theoretically and experimentally, the problem of visual distortion caused by sub-optimal intra prediction coding decisions by modeling distortion propagation across subsequent blocks. We propose two algorithms that address the problem.

A low complexity architecture for video coding with overlapped block motion compensation

Video coding standards, such as H.264/MPEG-4 AVC, have adopted the hybrid codec paradigm that consists of intra and inter prediction, followed by transform and quantization of the prediction residual. Inter prediction uses block-based motion compensation (BMC) to predict samples in the current picture. BMC, however, suffers from blocking artifacts and discontinuities. Along with deblocking, overlapped block motion compensation (OBMC) has been proposed in the past, primarily to suppress blocking artifacts. However, issues that include decoding complexity and implementation have been impediments to practical applications of OBMC. Here1, we propose a new decoder- and encoder-friendly architecture for inter prediction with OBMC. Furthermore, we show preliminary experimental results that point to coding efficiency gains over the BMC architecture of H.264/MPEG-4 AVC.