Jakub Stankowski

High Efficiency 3D Video Coding Using New Tools Based on View Synthesis

We propose a new coding technology for 3D video represented by multiple views and the respective depth maps. The proposed technology is demonstrated as an extension of the recently developed high efficiency video coding (HEVC). One base views are compressed into a standard bitstream (like in HEVC). The remaining views and the depth maps are compressed using new coding tools that mostly rely on view synthesis. In the decoder, those views and the depth maps are derived via synthesis in the 3D space from the decoded baseview and from data corresponding to small disoccluded regions. The shapes and locations of those disoccluded regions can be derived by the decoder without any side information transmitted. To achieve high compression efficiency, we propose several new tools such as depth-based motion prediction, joint high frequency layer coding, consistent depth representation, and nonlinear depth representation. The experiments show high compression efficiency of the proposed technology. The bitrate needed for transmission of two side views with depth maps is mostly less than 50% than that of the bitrate for a single-view video.

Coding of multiple video+depth using HEVC technology and reduced representations of side views and depth maps

During the last two decades, a new technology generation of video compression was introduced about each 9 years. Each new compression-technology generation provides halving of necessary bitrates as compared to the last previous generation. This increasing single-view compression performance is related to increasing compression performance of multiview video coding. For multiview video with associated depth maps, additional significant bitrate reduction may be achieved. The paper reports the original compression technology that was designed and developed at Poznań University of Technology in response to MPEG Call for Proposals on 3D Video Coding Technology. The main idea of this technique is to predict very efficiently the side views and the depth maps from the base view.

Extensions of the HEVC technology for efficient multiview video coding

The paper deals with multiview video coding using the new technology of High Efficiency Video Coding (HEVC). Implementation of multiview video coding in the framework of HEVC is described together with new specific tools proposed by the authors. Extensive experimental results are reported for compression performance comparison of MVC (ISO 14496-10), HEVC simulcast and two versions of proposed “multiview HEVC”. For “multiview HEVC” the results indicate significant bitrate reduction of about 50%, as compared to the state-of-the-art MVC technology standardized as a part of AVC (MPEG-4, H.264).

3D video compression by coding of disoccluded regions

In this paper, we present the most efficient coding tools that are used in a new video compression technology for multiple views with the depth maps. This very well performing technology was designed and developed in response to MPEG Call for Proposals on 3D Video Coding Technology. The proposed technology exploits a tool that reduces the side views and the side depth maps to small disoccluded regions. This way only one central view and one depth map are coded in the HEVC syntax while the remaining views and the depth maps are synthesized in the decoder from the small disoccluded regions and from the central-view data. Therefore, the bitrate needed for a side view is mostly below 20% of the bitrate for single-view video.

3D-HEVC extension for circular camera arrangements

The paper presents an extension of 3D-HEVC for the circular camera arrangements. It generalizes the derivation of the disparity vectors from depth data for sequences captured using cameras located on an arc. The general equation for disparity calculations has been implemented instead of the simplified equation used in 3D-HEVC. Experiments have been performed on widely recognized multiview test sequences with both circular and linear camera arrangements. In the case of sequences obtained using cameras located on an arc, the proposed extension provides about 6% bitrate reduction when compared to 3D HEVC, and does not influence the 3D-HEVC performance for linear view arranged sequences.

New results in free-viewpoint television systems for horizontal virtual navigation

The paper presents the concept of a practical free-viewpoint television system with purely optical depth estimation. The system consists of camera modules that contain pairs or triples of cameras together with the respective microphones. The camera modules can be sparsely located in arbitrary positions around a scene. Each camera module is equivalent to a video camera with a depth sensor and microphones. The hardware requirements, the video and audio processing algorithms and the preliminary experimental results are reported. In particular, for such systems, a compression technique is discussed that is more efficient than the new 3D-HEVC technology. A set of new test sequences obtained with the use of camera pairs are presented.

Bitrate distribution of syntax elements in the HEVC encoded video

The paper focuses on two problems: 1) analysis of bitrate contribution of I and B frames in encoded HEVC data stream, and 2) analysis of distribution of HEVC syntax elements in I and B frames. The relationship between sizes of I and B frames located at a different levels in hierarchical coding scheme was experimentally determined, as well as contribution of HEVC syntax elements in encoded streams of I, B0, B1, B2, and B3 frames. The results are presented in the paper. All presented results concern a wide range of QP values in a video encoder.

Fast watermarking of MPEG-4 AVC/H.264 encoded HDTV video bitstreams

The paper describes watermarking of compressed video bitstreams. This technique is proven to be very fast and robust against various attacks including camcorder recording attack. The watermark is embedded using indirect modifications of transform coefficients in selected P- and B-frames. In the watermarked bitstream there exist sequences of watermarked pictures interleaved with sequences of unmodified pictures. The unmodified pictures serve as a reference for the watermark detection. The experimental results described for real-world MPEG-4 AVC/H.264 encoded HDTV video bitstreams are included.

Adaptation of the 3D-HEVC coding tools to arbitrary locations of cameras

In the paper, we describe the extensions of the 3D-HEVC compression technology aimed at improved compression efficiency for multiview sequences acquired from arbitrarily located cameras. Our proposal refines the inter-view prediction by replacing the horizontal shifts with the true mapping in the 3D space. This implies changes in several coding tools, which we describe in details. The paper also reports experimental results on the comparison of the proposed solution to the 3D-HEVC standard codec. We also discuss the influence of the number of views and the view coding order on the compression efficiency.

Rate-distortion optimized quantization in HEVC: Performance limitations

Rate-Distortion Optimized Quantization (RDOQ) is an encoding optimization technique that may be modified with no implications to the bitstream compliance with the standard. In the paper, we propose an RDOQ variant with exact cost estimation resulting in higher compression ratios as compared to HEVC Test Model. This improvement is obtained at the cost of the complexity increase that may be scaled by replacing the individual steps of exact estimation by the simplified ones from HEVC Test Model. In that way, a family of RDOQ variants is created. The performance of the exact variant from this paper defines a kind of the performance limit for RDOQ.