Deyang Liu

Disparity compensation based 3D holoscopic image coding using HEVC

3D holoscopic image can provide true 3D content by reproducing the light rays of the 3D scene and is regarded as a promising technique for future 3D TV. Disparity information is paramount intrinsic characteristic of the 3D holoscopic image. In this paper, a disparity compensation based 3D holoscopic image coding algorithm using HEVC is put forward. In order to drive an authentic disparity, we expand the available area outwards in the disparity matching process instead of dividing the current coding block into four parts. Experimental results demonstrate that the proposed method can obtain considerable gains over original HEVC intra-prediction and the classical TMP algorithm with acceptable complexity increasing compared to original HEVC standard.

3D holoscopic image coding scheme using HEVC with Gaussian process regression

3D holoscopic image, also known as integral imaging, light field imaging and plenoptic imaging, can provide a natural and fatigue-free 3D visualization. However, a large amount of data is required to represent the 3D holoscopic content. Therefore, efficient coding schemes for such particular type of image are needed. In this paper, we propose a Gaussian process regression based prediction scheme to compress the 3D holoscopic image. In the proposed scheme, the coding block and its prediction supports are modeled as a Gaussian process (GP) and Gaussian process regression (GPR) is used to obtain a better prediction of the coding block. Limited searching windows in horizontal and vertical directions are used to obtain the prediction supports, and a filtration method is designed to judge the reliability of the obtained prediction supports. Moreover, in order to alleviate the high complexity caused by GPR, a sparsification method is also put forward. Experimental results demonstrate the advantage of the proposed scheme for 3D holoscopic image coding in terms of different quality metrics as well as the visual quality of the views rendered from decompressed 3D holoscopic content, compared to the HEVC intra-prediction method and several other prediction methods in this field. HighlightsAn efficient 3D holoscopic video coding solution based on Gaussian Process Regression using HEVC is proposed.The coding block and its prediction supports are modeled as a Gaussian process and Gaussian process regression is used to obtain a better prediction of the coding block.Superior coding efficiency is shown compared to the HEVC intra-prediction method and several other prediction methods in this field.A better visual quality of the rendered views can be obtained.

Three-dimensional holoscopic image coding scheme using high-efficiency video coding with kernel-based minimum mean-square-error estimation

Abstract. Three-dimensional (3-D) holoscopic imaging, also known as integral imaging, light field imaging, or plenoptic imaging, can provide natural and fatigue-free 3-D visualization. However, a large amount of data is required to represent the 3-D holoscopic content. Therefore, efficient coding schemes for this particular type of image are needed. A 3-D holoscopic image coding scheme with kernel-based minimum mean square error (MMSE) estimation is proposed. In the proposed scheme, the coding block is predicted by an MMSE estimator under statistical modeling. In order to obtain the signal statistical behavior, kernel density estimation (KDE) is utilized to estimate the probability density function of the statistical modeling. As bandwidth estimation (BE) is a key issue in the KDE problem, we also propose a BE method based on kernel trick. The experimental results demonstrate that the proposed scheme can achieve a better rate-distortion performance and a better visual rendering quality.

Coding of 3D holoscopic image by using spatial correlation of rendered view images

Holoscopic imaging is a prospective acquisition and display solution for providing natural and fatigue-free 3D visualization. However, large amount of data is required to represent the 3D holoscopic content. Therefore, efficient coding schemes for this particular type of image are needed. In this paper, an effective coding scheme is proposed by exploring the spatial correlation among the view images with different perspectives rendered from 3D holoscopic image. We utilize the interlaced view image to descript such spatial correlation. A linear prediction method is used on the interlaced view image instead of the original holoscopic image directly. Experimental results show that the proposed coding scheme performs better than HEVC intra standard and screen content coding extension of HEVC with around 2.41dB and 0.42 dB average quality improvement respectively.

Scalable coding of 3D holoscopic image by using a sparse interlaced view image set and disparity map

Abstract3D holoscopic imaging, also known as integral imaging, light field imaging or plenoptic imaging, can provide natural and fatigue-free 3D visualization. Holoscopic contents captured by the plenoptic camera contain both spatial and angular information of a 3D scene. Therefore, view images with different perspectives can be rendered from the holoscopic contents. A coding scheme to compress the 3D holoscopic image by exploiting the high spatial correlation among the rendered view images will be advantageous. Therefore, in this paper, an efficient scalable coding scheme is proposed to compress the 3D holoscopic image by utilizing such high spatial correlation. We firstly re-arrange the holoscopic contents to form an interlaced view image. A sparse format is then proposed to express the interlaced view image. With the reconstructed image derived by disparity map based sifting and interpolation, the full interlaced view image is coded by using the reconstructed image as a reference frame. As an outcome of the representation, a scalable structure with three layers can be provided by the proposed scheme. Experimental results demonstrate that the 3D holoscopic image can be compressed efficiently with over 44 percent bit rate reduction compared with HEVC. Meanwhile, the proposed scheme can also surpass several other prediction schemes in this field.

Hybrid Kernel-Based Template Prediction and Intra Block Copy for Light Field Image Coding

Light field (LF) technology can capture both spatial and angular information of a 3D scene, and enable new possibilities for digital imaging. However, one problem that occupies an important position to deal with the LF data is the sheer size of data volume. Therefore, in this paper, we propose an effective LF image coding scheme by combining the kernel-based template prediction method and intra block copy method. In the proposed method, intra block copy method is used to predict the unknown blocks where kernel-based template prediction method fails under the Rate-Distortion (RD) based decision mechanism. Experimental results show that the LF data can be efficiently compressed by the proposed method and the execution time in decoder side can also be reduced by about 35% compared to kernel-based template prediction method.

Bit allocation for 3D video coding based on lagrangian multiplier adjustment

Abstract Multi-view video plus depth (MVD) format 3D video consists of color texture image and gray depth map, the depth map provides the scene geometry information and is utilized to synthesize the virtual views through Depth Image Based Rendering (DIBR) technique. The quality of the synthesized virtual views is related to the qualities of both texture image and depth map, thus bit allocation between texture image and depth map is very important in 3D video coding. In this paper, we propose to optimally allocate bits between texture image and depth map by adjusting the Lagrangian Multiplier (LM) in depth map coding, we adjust the LM based on the difference between texture image coding Quantization Parameter (QP) and depth map QP. Experimental results show that the proposed method can achieve optimal 3D video coding performance for different sequences under different bitrates, and the complexity of our method is extremely low.

Adaptive Bit Allocation for 3D Video Coding

In the multi-view video plus depth 3D video coding, texture image and depth map are coded jointly. The texture image is utilized for displaying and synthesizing the virtual view as reference image. The depth map provides the scene geometry information and is utilized to synthesize the virtual view at the terminal through Depth-Image Based Rendering technique. The distortion of the compressed texture image and depth map will be propagated to the synthesized virtual view. Besides the coding efficiency of texture image and depth map, bit allocation between texture image and depth map also has a great effect on the synthesized virtual view quality. Several methods are proposed for bit allocation between texture image and depth map, but most of them attempt to allocate a fixed target bitrate based on virtual view distortion model to achieve optimal synthesized virtual view quality, and the modeling process brings extra complexity. In practical application, the video sequence has different contents and fixed bit ratio cannot achieve optimal performance. In this paper, we propose an adaptive bit allocation algorithm for 3D video coding. First, we present a model to estimate the synthesized virtual view distortion, and then adjust the bit ratio between adjacent views and between texture image and depth map at Group of Picture level based on the virtual view quality fluctuation. We adjust the bit ratio to achieve the optimal virtual view quality for different video contents. Experimental results demonstrate that the proposed algorithm can optimally allocate bits to achieve optimal virtual view quality under different target bitrates and for different video contents, and the computational complexity of the proposed algorithm is extremely low.

Depth map coding based on virtual view quality

Multi-view video plus depth (MVD) is a 3D video representation. In MVD, the depth map provides the scene distance information and is used to render the virtual view through Depth Image Based Rendering (DIBR) technique. The depth map coding error will induce distortion in the rendered virtual views. This paper proposes a mathematic model that can estimate the synthesized virtual view distortion induced by depth map compression, and the model is employed to the rate distortion optimization (RDO) in the depth map coding. Based on the rendered virtual view quality, a Lagrangian optimization adjustment scheme at Coding Unit (CU) level is proposed to improve the depth map encoding efficiency. Experimental results demonstrate that the proposed method can improve the BD-PSNR of virtual view for 0.62 dB, and the encoding complexity reduces compared with the view synthesis optimization (VSO) technique in the 3D-HEVC Test Model (HTM).

An Improved 3D Holoscopic Image Coding Scheme Using HEVC Based on Gaussian Mixture Models

3D holoscopic system can provide continuous motion parallax throughout the viewing zone with precise convergence and depth perception, for which it is regarded as a promising technique for future 3D TV. In this paper, a 3D holoscopic image coding scheme based on Gaussian mixture models (GMM) is introduced firstly, taking full advantage of the intrinsic characteristic of such particular type of content. Due to the shortcomings of GMM based method, an improved method is thereafter put forward, in which many parameters that are insignificant in the final estimator of GMM based method are avoided, and more surrounding pixels are used to obtain the model parameters with the help of the least square method. Experimental results indicate that the improved method can obtain considerable gains over HEVC intra prediction and several other prediction methods.