Chuanmin Jia

Nonlocal In-Loop Filter: The Way Toward Next-Generation Video Coding?

In-loop filtering has emerged as an essential coding tool since H.264/AVC, due to its delicate design, which reduces different kinds of compression artifacts. However, existing in-loop filters rely only on local image correlations, largely ignoring nonlocal similarities. In this article, the authors explore the design philosophy of in-loop filters and discuss their vision for the future of in-loop filter research by examining the potential of nonlocal similarities. Specifically, the group-based sparse representation, which jointly exploits an images local and nonlocal self-similarities, lays a novel and meaningful groundwork for in-loop filter design. Hard- and soft-thresholding filtering operations are applied to derive the sparse parameters that are appropriate for compression artifact reduction. Experimental results show that this in-loop filter design can significantly improve the compression performance of the High Efficiency Video Coding (HEVC) standard, leading us in a new direction for improving compression efficiency.

Non-Local Structure-Based Filter for Video Coding

The deblocking filtering (DF) in HEVC is only applied to the boundaries between the coding units, prediction units, or transform units, which actually exists two issues. On one hand, the simple DF in HEVC does not fully exploit structure information in video. On the other hand, DF in HEVC does not consider the inside areas, which often suffers from quantization distortion. To alleviate the above issues, in this paper, a non-local structure-based filter (NLSF) is proposed by simultaneously enforcing the intrinsic local sparsity and the non-local self-similarity of each frame in video. NLSF not only deals with the boundaries, but also deals with the inside areas, which is able to effectively reduce the artifacts while enhance the quality of the deblocking frames. Experimental results demonstrate that, compared with the original HEVC reference encoder implementation in AI configuration, the proposed NLSF can achieve up to 7.3% BD-rate saving by substituting for DF in HEVC.

Light Field Image Compression with Sub-apertures Reordering and Adaptive Reconstruction

Light field (LF) attracts tremendous attention due to its capability of recording the intensity of scene objects as well as the direction of the light ray, which also dramatically increases the amount of redundant data. In this paper, we explore the structure of the light field images, and propose a pseudo-sequence based light field image compression with sub-aperture reordering and adaptive reconstruction to efficiently improve the coding performances. In the proposed method, we firstly decompose the lenslet image into sub-aperture images, and then design an optimized sub-aperture scan order to rearrange them sequentially as a pseudo-sequence. Third, we take advantage of the state-of-the-art video codec to compress the pseudo-sequence by leveraging both intra- and inter-view correlations. Considering the interpolation and transform induced by the reconstruction procedure from sub-aperture images to lenslet image, we propose an enhanced reconstruction method by applying region-based non-local adaptive filters which extracts the non-local similarities for collaborative filtering to promote the quality of reconstructed lenslet images. Extensive experimental results show that the proposed method achieves up to 15.7% coding gain in terms of BD-rate.

Deep convolutional network based image quality enhancement for low bit rate image compression

In this contribution, a novel image quality enhancement algorithm based on convolutional network is proposed for low bit rate image compression. Specifically, a downsample procedure is performed to generate lower resolution image for low bit rate compression. While the decoder side, upsample is to be performed firstly to the original resolution. Image quality is further enhanced by the proposed convolutional deep network. In particular, an optional image quality improvement network can be utilized for further enhancement after the first network. With the help of deep network, more detailed and high-frequency information can be recovered while maintaining the consistency of contour area, leading to better visual quality. Another benefit of this approach lies in that the proposed approach is fully compatible with all third-party image codec pipeline. Experimental result shows that the proposed scheme significantly outperforms JPEG in low bit rate image compression.

Structure-driven Adaptive Non-local Filter for High Efficiency Video Coding (HEVC)

Deblocking filter (DF) Is High Efficiency Video Coding (HEVC) is Only Applied to all Samples Adjacent to prediction units (PU), or transform units (TU), which actually exists two issues. The first one is that DF in HEVC does not fully exploit nonlocal similarity structure information in video. The second one is that DF is HEVC does not consider the inside pixels, which often suffer from quantization distrotion. To alleviate these issues, in this paper, a structure-driven adaptive non-local filter (SANF) Is Proposed By Simultaneously Enforcing The Intrinsic Local Sparsity And The Non-Local Self-Similarity Of Each Frame. Not only SANF deals with the boundary pixels, but also the inside area, which is able to effectively reduce block artifacts while enhancing the quality of the deblocked frames. Applying SANF to luma and chroma components after DF, simulation results demonstrate that the proposed SANF can save BD-rate reduction up to 10.3% with ALF off. For luma component, SANF achieves 4.1%. 3.3%, 4.4% BD-rate saving for all intra, low delay B and random access configurations, respectively with ALF off. furthermore, the performance with ALF on is also discussed.