Lijun Zhao

Two-stage filtering of compressed depth images with Markov Random Field

Virtual view synthesis and image comprehension have become easier with the aid of depth information. However, when a depth image is compressed, severe distortions along boundaries may occur, thus leading to performance degradation. To solve this problem, we propose in this paper a two-stage filtering that consists of binary segmentation-based depth filtering and the reconstruction using a Markov Random Field (MRF) model. The MRF model adopted in our work consists of a data term and a smoothness term so as to preserve the boundary and maintain the smoothness simultaneously. We notice that directly applying the MRF model to a distorted depth image is usually unable to produce a satisfactory performance. Then, we propose that binary segmentation based depth filtering is used to remove artifacts over discontinuous regions in the distorted depth image. Experimental results show that, through our processing, the compressed depth image can render better quality for the synthesized images than many existing depth filtering methods. HighlightsTwo stage filtering is proposed to process HEVC-compressed depth image.The binary segmentation-based filtering is proposed to achieve a pre-filtering.The pre-filtering could well retrieve object boundaries with low-complexity.Depth image is reconstructed by MRF with reliable pixels treated as measured values.

Joint iterative guidance filtering for compressed depth images

In the general 3D scene, the correlation of depth image and corresponding color image exists, so many filtering methods have been proposed to improve the quality of depth images according to this correlation. Unlike the conventional methods, in this paper both depth and color information can be jointly employed to improve the quality of compressed depth image by the way of iterative guidance. Firstly, due to noises and blurring in the compressed image, a depth pre-filtering method is essential to remove artifact noises. Considering that the received geometry structure in the distorted depth image is more reliable than its color image, the color information is merged with depth image to get depth-merged color image. Then the depth image and its corresponding depth-merged color image can be used to refine the quality of the distorted depth image using joint iterative guidance filtering method. Therefore, the efficient depth structural information included in the distorted depth images are preserved relying on depth itself, while the corresponding color structural information are employed to improve the quality of depth image. We demonstrate the efficiency of the proposed filtering method by comparing objective and visual quality of the synthesized image with many existing depth filtering methods.

Cascaded reconstruction network for compressive image sensing

The theory of compressed sensing (CS) has been successfully applied to image compression in the past few years, whose traditional iterative reconstruction algorithm is time-consuming. Fortunately, it has been reported deep learning-based CS reconstruction algorithms could greatly reduce the computational complexity. In this paper, we propose two efficient structures of cascaded reconstruction networks corresponding to two different sampling methods in CS process. The first reconstruction network is a compatibly sampling reconstruction network (CSRNet), which recovers an image from its compressively sensed measurement sampled by a traditional random matrix. In CSRNet, deep reconstruction network module obtains an initial image with acceptable quality, which can be further improved by residual reconstruction network module based on convolutional neural network. The second reconstruction network is adaptively sampling reconstruction network (ASRNet), by matching automatically sampling module with corresponding residual reconstruction module. The experimental results have shown that the proposed two reconstruction networks outperform several state-of-the-art compressive sensing reconstruction algorithms. Meanwhile, the proposed ASRNet can achieve more than 1 dB gain, as compared with the CSRNet.

Single depth image super-resolution with multiple residual dictionary learning and refinement

Learning-based image super-resolution methods often use large datasets to learn texture features. When these methods are applied to depth images, emphasis should be given on learning the geometrical structures at object boundaries, since depth images do not have much texture information. In this paper, we develop a scheme to learn multiple residual dictionaries from only one external image. After depth image super-resolution, some artifacts may appear. An adaptive depth map refinement method is then proposed to remove these artifacts along the depth edges, based on the shape-adaptive weighted median filtering method. Experimental results demonstrate the advantage of the proposed method over many other methods.

Iterative range-domain weighted filter for structural preserving image smoothing and de-noising

The filtering weights from both spatial domain and range domain in the bilateral filtering always restrict filtering output value highly related to very close neighboring pixels, which results in very small changes before and after filtering. In order to better resolve the problem of piece-wise smoothness image’s de-noising, such as artifact removal of compressed depth image, we firstly propose an iterative range-domain weighted filter method. The filtering weights of the proposed method are calculated within a fixed window in an iterative way according to both pixel similarity in the range domain and image’s pixel occurring frequency, but there is no filtering weight from the spatial domain. Secondly, the proposed method is combined with Gaussian filtering as an engine in order to finish the task of image smoothing, because image smoothing for extracting structures is often sensitive to image’s fine details with strong gradients during suppressing image’s textures. To demonstrate the efficiency, we have applied the proposed method into many applications. For example, the proposed method has better performances on compressed depth artifact removal than BF, CVBF, and ADTF. Meanwhile, the proposed method is used for capture-noise removal of depth image. Additionally, the proposed method performs better performance on structural information preservation for image smoothing, as compared to several existing methods.