Hangfan Liu

Image denoising via adaptive soft-thresholding based on non-local samples

This paper proposes a new image denoising approach using adaptive signal modeling and adaptive soft-thresholding. It improves the image quality by regularizing all the patches in image based on distribution modeling in transform domain. Instead of using a global model for all patches, it employs content adaptive models to address the non-stationarity of image signals. The distribution model of each patch is estimated individually and can vary for different transform bands and for different patch locations. In particular, we allow the distribution model for each individual patch to have non-zero expectation. To estimate the expectation and variance parameters for the transform bands of a particular patch, we exploit the non-local correlation of image and collect a set of similar patches as data samples to form the distribution. Irrelevant patches are excluded so that this non-local based modeling is more accurate than global modeling. Adaptive soft-thresholding is employed since we observed that the distribution of non-local samples can be approximated by Laplacian distribution. Experimental results show that the proposed scheme outperforms the state-of-the-art denoising methods such as BM3D and CSR in both the PSNR and the perceptual quality.

G-CAST: Gradient Based Image SoftCast for Perception-Friendly Wireless Visual Communication

Conventional image and video communication systems are usually designed with the objective being to maximize the fidelity of reconstructed images measured by mean square errors (MSE). It is well known that the fidelity metric MSE may not reflect the visual quality perceived by human eyes. Recent advancements in image quality assessment tell us that the structural similarity (SSIM), especially the gradient similarity, reveals the perceptual fidelity of images more reliably. Inspired by this observation, this paper proposes a new image communication approach, which conveys the visual information in an image by transmitting the image gradients and recovers the image from the received gradient data at decoder side using statistical image prior knowledge. In particular, we designed a gradient-based image SoftCast scheme for wireless scenarios. Experimental results show that the proposed scheme can produce reconstruction images with much better perceptual quality. The advantage in perceptual quality is verified by the quality improvement measured by the metrics SSIM and gradient signal-to-noise ratio (GSNR).

Image Denoising via Bandwise Adaptive Modeling and Regularization Exploiting Nonlocal Similarity

This paper proposes a new image denoising algorithm based on adaptive signal modeling and regularization. It improves the quality of images by regularizing each image patch using bandwise distribution modeling in transform domain. Instead of using a global model for all the patches in an image, it employs content-dependent adaptive models to address the non-stationarity of image signals and also the diversity among different transform bands. The distribution model is adaptively estimated for each patch individually. It varies from one patch location to another and also varies for different bands. In particular, we consider the estimated distribution to have non-zero expectation. To estimate the expectation and variance parameters for every band of a particular patch, we exploit the nonlocal correlation in image to collect a set of highly similar patches as the data samples to form the distribution. Irrelevant patches are excluded so that such adaptively learned model is more accurate than a global one. The image is ultimately restored via bandwise adaptive soft-thresholding, based on a Laplacian approximation of the distribution of similar-patch group transform coefficients. Experimental results demonstrate that the proposed scheme outperforms several state-of-the-art denoising methods in both the objective and the perceptual qualities.This paper proposes a new image denoising algorithm based on adaptive signal modeling and regularization. It improves the quality of images by regularizing each image patch using bandwise distribution modeling in transform domain. Instead of using a global model for all the patches in an image, it employs content-dependent adaptive models to address the non-stationarity of image signals and also the diversity among different transform bands. The distribution model is adaptively estimated for each patch individually. It varies from one patch location to another and also varies for different bands. In particular, we consider the estimated distribution to have non-zero expectation. To estimate the expectation and variance parameters for every band of a particular patch, we exploit the nonlocal correlation in image to collect a set of highly similar patches as the data samples to form the distribution. Irrelevant patches are excluded so that such adaptively learned model is more accurate than a global one. The image is ultimately restored via bandwise adaptive soft-thresholding, based on a Laplacian approximation of the distribution of similar-patch group transform coefficients. Experimental results demonstrate that the proposed scheme outperforms several state-of-the-art denoising methods in both the objective and the perceptual qualities.

Nonlocal Gradient Sparsity Regularization for Image Restoration

Total variation (TV) regularization is widely used in image restoration to exploit the local smoothness of image content. Essentially, the TV model assumes a zero-mean Laplacian distribution for the gradient at all pixels. However, real-world images are nonstationary in general, and the zero-mean assumption of pixel gradient might be invalid, especially for regions with strong edges or rich textures. This paper introduces a nonlocal (NL) extension of TV regularization, which models the sparsity of the image gradient with pixelwise content-adaptive distributions, reflecting the nonstationary nature of image statistics. Taking advantage of the NL similarity of natural images, the proposed approach estimates the image gradient statistics at a particular pixel from a group of nonlocally searched patches, which are similar to the patch located at the current pixel. The gradient data in these NL similar patches are regarded as the samples of the gradient distribution to be learned. In this way, more accurate estimation of gradient is achieved. Experimental results demonstrate that the proposed method outperforms the conventional TV and several other anchors remarkably and produces better objective and subjective image qualities.

Gradient based image/video softcast with grouped-patch collaborative reconstruction

Inspired by the recent image quality assessment (IQA) studies which indicate that the image gradient data reflects the visual information more reliably than the image pixels, gradient based transmission scheme was recently proposed to pursue better perceptual quality for wireless visual communication. This paper develops an effective method to reconstruct high quality image from the received noisy gradient data. The proposed method utilizes both local correlation and non-local similarity within the image signal to regularize the reconstruction image. Principle component analysis (PCA) is employed to learn signal-adaptive two-dimensional (2D) transform basis, and 3D transform is performed on grouped similar patches to further decorrelate the coefficients. In this way, distortions can be effectively suppressed via adaptive collaborative shrinkage on the transform coefficients. Experimental results demonstrate that the proposed method improves the reconstruction performance remarkably compared with the existing schemes.

Non-local extension of total variation regularization for image restoration

Total-variation (TV) regularization is widely adopted in image restoration problems to exploit the feature that natural images are smooth with small gradient values at most regions. Basic TV method assumes identical zero-mean Laplacian distribution for the gradients at all pixels. However, for real-world images, the statistics of gradients may not be stationary, and the zero-mean assumption of gradients may not be valid either for a specific pixel. This paper presents a non-local extension of TV regularization for image restoration, called Non-Local Gradient Sparsity Regularization (NGSR). The NGSR model employs a separate gradient value distribution for each pixel. To figure out the distribution parameters, the NGSR method exploits a set of patches which are similar to the patch centered at current pixel and estimates the distribution parameter adaptively. Experimental results demonstrate that the proposed NGSR outperforms traditional TV remarkably for image restoration.

Gradient based image transmission and reconstruction using non-local gradient sparsity regularization

Most existing image coding and communication systems aim to minimize the mean square error (MSE) of the pixels reconstructed at receivers. However, the quality metric MSE has long been criticized for not being consistent with the perception of human vision systems. This paper considers a gradient-based image SoftCast (G-Cast) scheme, based on the recent advancements in image quality assessment which indicate that gradient similarity is highly correlated with perceptual image quality. To reconstruct the image from the received noisy gradient data, we exploit the statistical characteristics of image gradients. Instead of using the very simple Laplacian distribution for image gradient as in the total variation (TV) model, we further exploit the non-local similarity of image patches. A non-local gradient sparsity regularization (NLGSR) method is developed and solved using augmented Lagrangian method. Experimental results show that the proposed scheme provides promising perceptual image quality, and the NLGSR reconstruction scheme outperforms the existing schemes remarkably.

Content-adaptive low rank regularization for image denoising

Prior knowledge plays an important role in image denoising tasks. This paper utilizes the data of the input image to adaptively model the prior distribution. The proposed scheme is based on the observation that, for a natural image, a matrix consisted of its vectorized non-local similar patches is of low rank. We use a non-convex smooth surrogate for the low-rank regularization, and view the optimization problem from the empirical Bayesian perspective. In such framework, a parameter-free distribution prior is derived from the grouped non-local similar image contents. Experimental results show that the proposed approach is highly competitive with several state-of-art denoising methods in PSNR and visual quality.

Wireless Image SoftCast Using Compressive Gradient

Based on observations that the visual quality has strong correlation with image gradients, gradient based image SoftCast (G-Cast) advocates to convey visual information by delivering image gradients. In G-Cast, both horizontal gradients and vertical gradients needs to be transmitted, even if the channel bandwidth is insufficient. This paper propose to send out the random projection measurements of the gradients instead of delivering gradients directly, so that the number of transmitted data can be adjusted according to bandwidth conditions. Experimental results validate the effectiveness of the the proposed transmission scheme under various channel signal-to-noise ratio (CSNR) and bandwidth conditions.

Image Denoising via Low Rank Regularization Exploiting Intra and Inter Patch Correlation

In image restoration tasks, image priors generally utilize correlation within image contents to predict the latent image signal. In this paper, we propose to jointly exploit both intra- and inter-patch correlation of the input image, so as to further reduce the uncertainty of the unknown signal, and thus improve the prediction of the latent image. The proposed scheme evolves from the low-rank regularization for non-local highly-correlated image contents. Since the underlying cost function to pursue minimal rank is hard to solve, we use non-convex smooth surrogates for the rank penalty. Two such surrogates are utilized in order to incorporate both intra- and inter-patch correlation. To tackle the optimization problem, we use iterative alternating direction technique to divide the problem into two subproblems, each of which is solved via an empirical Bayesian procedure built upon variational approximation. Experimental results on image denoising show that the proposed approach outperforms several state-of-the-art methods in terms of peak signal-to-noise ratio, structural similarity, and perceptual quality.