Jakub Siast

High Efficiency 3D Video Coding Using New Tools Based on View Synthesis

We propose a new coding technology for 3D video represented by multiple views and the respective depth maps. The proposed technology is demonstrated as an extension of the recently developed high efficiency video coding (HEVC). One base views are compressed into a standard bitstream (like in HEVC). The remaining views and the depth maps are compressed using new coding tools that mostly rely on view synthesis. In the decoder, those views and the depth maps are derived via synthesis in the 3D space from the decoded baseview and from data corresponding to small disoccluded regions. The shapes and locations of those disoccluded regions can be derived by the decoder without any side information transmitted. To achieve high compression efficiency, we propose several new tools such as depth-based motion prediction, joint high frequency layer coding, consistent depth representation, and nonlinear depth representation. The experiments show high compression efficiency of the proposed technology. The bitrate needed for transmission of two side views with depth maps is mostly less than 50% than that of the bitrate for a single-view video.

Coding of multiple video+depth using HEVC technology and reduced representations of side views and depth maps

During the last two decades, a new technology generation of video compression was introduced about each 9 years. Each new compression-technology generation provides halving of necessary bitrates as compared to the last previous generation. This increasing single-view compression performance is related to increasing compression performance of multiview video coding. For multiview video with associated depth maps, additional significant bitrate reduction may be achieved. The paper reports the original compression technology that was designed and developed at Poznań University of Technology in response to MPEG Call for Proposals on 3D Video Coding Technology. The main idea of this technique is to predict very efficiently the side views and the depth maps from the base view.

Extensions of the HEVC technology for efficient multiview video coding

The paper deals with multiview video coding using the new technology of High Efficiency Video Coding (HEVC). Implementation of multiview video coding in the framework of HEVC is described together with new specific tools proposed by the authors. Extensive experimental results are reported for compression performance comparison of MVC (ISO 14496-10), HEVC simulcast and two versions of proposed “multiview HEVC”. For “multiview HEVC” the results indicate significant bitrate reduction of about 50%, as compared to the state-of-the-art MVC technology standardized as a part of AVC (MPEG-4, H.264).

Network-on-Multi-Chip (NoMC) for Multi-FPGA Multimedia Systems

Some applications, especially in the area of multimedia processing, need to be implemented in a multichip platform, due to their size. An efficient communication infrastructure for such systems may be designed with the use of the Networks-on-Chip (NoCs). However, a network for multi-chip systems require a scalable architecture. Moreover, for multimedia purposes, such NoC should support a multicast transmission mode. In order to meet this requirements, we propose the NoMC (Network-on-Multi-Chip) which is a hierarchical interconnect system, designed for multi-chip systems. A performance of the proposed network is assessed utilizing a model of the MVC (Multiview Video Coding) coder. In such system, the multicast transmission mode may yield an overall bandwidth gain up to 30%. Moreover, the synthesis results show that the proposed network elements are easily synthesizable for the FPGA devices.

3D video compression by coding of disoccluded regions

In this paper, we present the most efficient coding tools that are used in a new video compression technology for multiple views with the depth maps. This very well performing technology was designed and developed in response to MPEG Call for Proposals on 3D Video Coding Technology. The proposed technology exploits a tool that reduces the side views and the side depth maps to small disoccluded regions. This way only one central view and one depth map are coded in the HEVC syntax while the remaining views and the depth maps are synthesized in the decoder from the small disoccluded regions and from the central-view data. Therefore, the bitrate needed for a side view is mostly below 20% of the bitrate for single-view video.

Experiments on acquisition and processing of video for free-viewpoint television

The paper describes an experimental multiview video production, processing and delivery chain developed at Poznan University of Technology for research on free-viewpoint television. The multiview-video acquisition system consists of HD camera units with wireless synchronization, wireless control, video storage and power supply units. Therefore no cabling is needed in the system, which is important for shooting real-world events. The system is mostly used for nearly circular setup of cameras but the locations of cameras are arbitrary, and the procedures for system calibration and multiview video correction are consid-ered. The paper deals also with adoption for circular camera arrangement of the techniques implemented in Depth Estimation Reference Software and View Synthesis Reference Software.

Bitrate distribution of syntax elements in the HEVC encoded video

The paper focuses on two problems: 1) analysis of bitrate contribution of I and B frames in encoded HEVC data stream, and 2) analysis of distribution of HEVC syntax elements in I and B frames. The relationship between sizes of I and B frames located at a different levels in hierarchical coding scheme was experimentally determined, as well as contribution of HEVC syntax elements in encoded streams of I, B0, B1, B2, and B3 frames. The results are presented in the paper. All presented results concern a wide range of QP values in a video encoder.

Digital watermarking with local strength adjustment for AVC-compressed HDTV bitstreams

A computationally inexpensive watermark schema for MPEG-4 AVC/H.264 streams that use local mark strength adjustment minimizing visible artifacts is presented. The local mark strength adjustment mechanism and watermark embedding mechanism operate on residual data. This approach minimize the number of operations needed for video bitstream decoding and subsequent watermarked video data encoding. The experimental results included herein, described for real-world MPEG4 AVC/H.264 encoded HDTV video bitstreams prove its robustness to complex attacks, especially camcorder recording attack.

Fast depth estimation on mobile platforms and FPGA devices

In this paper, we propose a fast technique for real-time depth estimation that is implementable on mobile devices like smartphones and tablets. Moreover, the FPGA-based implementations of this technique are also reported. The idea of this technique is to use small-block matching and exploit the recently estimated disparity values in order to enhance spatial consistency of the output disparity map. Using the Middlebury stereoscopic test images, the proposed real-time technique is compared to other techniques known from the references. The obtained results demonstrate high efficiency of the proposed technique implemented both on mobile platforms and FPGA devices.

Depth Map’s 2D Histogram Assisted Occlusion Handling in Video Object Tracking

The paper describes new algorithm for automatic video object tracking. Proposed architecture consists of two loops of Kalman filter. In the loop of the tracking process, the information achieved from video and from 2D histogram based on depth map is used. Two loops work simultanously and the parameters between the loops are interchanged when the occlusion occurs. The 2D histogram representation of the depth map has unique properties that can be used to improve the tracking eficiency especially in the case of occlusions of the objects in the image. Experimental results prove that the proposed system can accurately track multiple objects in complex scenes.