Mading Li

Joint image denoising using adaptive principal component analysis and self-similarity

The non-local means (NLM) has attracted enormous interest in image denoising problem in recent years. In this paper, we propose an efficient joint denoising algorithm based on adaptive principal component analysis (PCA) and self-similarity that improves the predictability of pixel intensities in reconstructed images. The proposed algorithm consists of two successive steps without iteration: the low-rank approximation based on parallel analysis, and the collaborative filtering. First, for a pixel and its nearest neighbors, the training samples in a local search window are selected to form the similar patch group by the block matching method. Next, it is factorized by singular value decomposition (SVD), whose left and right orthogonal basis denote local and non-local image features, respectively. The adaptive PCA automatically chooses the local signal subspace dimensionality of the noisy similar patch group in the SVD domain by the refined parallel analysis with Monte Carlo simulation. Thus, image features can be well preserved after dimensionality reduction, and simultaneously the noise is almost eliminated. Then, after the inverse SVD transform, the denoised image is reconstructed from the aggregate filtered patches by the weighted average method. Finally, the collaborative Wiener filtering is used to further remove the noise. The experimental results validate its generality and effectiveness in a wide range of the noisy images. The proposed algorithm not only produces very promising denoising results that outperforms the state-of-the-art methods in most cases, but also adapts to a variety of noise levels.

Adaptive General Scale Interpolation Based on Weighted Autoregressive Models

The autoregressive (AR) model has been widely used in signal processing for its effective estimation, especially in image processing. Many dedicated 2× interpolation algorithms adopt the AR model to describe the strong correlation between low-resolution (LR) pixels and high-resolution (HR) pixels. However, these AR model-based methods closely depend on the fixed relative position between LR pixels and HR pixels that are nonexistent in the general scale interpolation. In this paper, we present an adaptive general scale interpolation algorithm that is capable of arbitrary scaling factors considering the nonstationarity of natural images. Different from other dedicated 2× interpolation methods, the proposed AR terms are modeled by pixels with their adjacent unknown HR neighbors. To compensate for the information loss caused by mismatches of AR models, we consider a weighting scheme suitable for general scale situations based on the pixel similarity to increase accuracy of the estimation. Comprehensive experiments demonstrate the effectiveness of the proposed method on general scaling factors. The maximum gain of peak signal-to-noise ratio is 2.07 dB compared with segment adaptive gradient angle in 1.5× enlargements. To evaluate the performance in resolution adaptive video coding, we have also tested our method on Joint Scalable Video Model codec and obtained better subjective quality and rate-distortion performance.

Image Blocking Artifacts Reduction via Patch Clustering and Low-Rank Minimization

Summary form only given. Block-based Discrete Cosine Transform (BDCT) has been widely used in image and video compression due to its energy compacting property and relative ease of implementation. However, BDCT has a major drawback, which is usually referred to as blocking artifacts. Blocking artifacts appear as grid noise along the block boundaries because each block is transformed and quantized independently. Image deblocking techniques can reduce these distortions and alleviate the conflict between bit rate reduction and visual quality preservation. Many state-of-the-art image deblocking algorithms treated the blocking artifacts reduction of the compressed image as an inverse restoration problem. Natural image prior models are well utilized into the blocking artifacts reduction processing, such as the local sparsity prior model and non-local similarity property of natural images. These two local and non-local models characterize the image prior information in two complementary perspectives. Therefore, it is necessary to combine these two models in a unified framework. In this paper, we propose a novel method to reduce the blocking artifacts of blockcoded images via patch clustering and low-rank minimization, which simultaneously exploits the local and non-local sparse representations in a unified framework. First, the whole compressed image are divided into small patches. For each patch, we perform patch clustering to collect similar patches into a group. Then the whole group are simultaneously reconstructed by a low-rank minimization approach. Singular value thresholding (SVT) algorithm is employed to solve the low-rank minimization problem. To further improve the performance of the proposed algorithm, we adopt an iterative procedure to utilize the newly output data in each iteration and update the noise and signal variance adaptively. Experimental results show that the proposed method achieves higher PSNR and SSIM than the state-of-the-art methods. Comparing to the state-of-theart algorithms and, the proposed algorithm achieves about 0.37dB and 0.11dB improvement on average. For visual quality assessment, the deblocking images produced by the proposed algorithm reveal much more sharp edge structures and richer textures.

Isophote-Constrained Autoregressive Model With Adaptive Window Extension for Image Interpolation

The autoregressive (AR) model is widely used in image interpolations. Traditional AR models consider utilizing the dependence between pixels to model the image signal. However, they ignore the valuable patch-level information for image modeling. In this paper, we propose to integrate both the pixel-level and patch-level information to depict the relationship between high-resolution and low-resolution pixels and obtain better image interpolation results. In particular, we propose an isophote-constrained AR (ICAR) model to perform AR-flavored interpolation within an identified joint stable region and further develop an AR interpolation with an adaptive window extension. Considering the smoothness along the isophote curve, the ICAR model searches only several successive similar patches along the isophote curve over a large region to construct an adaptive window. These overlapped patches, representing the patch-level structure similarity, are used to construct a joint AR model. To better characterize the piecewise stationarity and determine whether a pixel is suitable for AR estimation, we further propose pixel-level and patch-level similarity metrics and embed them into the ICAR model, introducing a weighted ICAR model. Comprehensive experiments demonstrate that our method can effectively reconstruct the edge structures and suppress jaggy or ringing artifacts. In the objective quality evaluation, our method achieves the best results in terms of both peak signal-to-noise ratio and structural similarity for both simple size doubling (two times) and for arbitrary scale enlargements.

Patch-based image deblocking using geodesic distance weighted low-rank approximation

Transform coding based on the discrete cosine transform (DCT) has been widely used in image coding standards. However, the coded images often suffer from severe visual distortions such as blocking artifacts. In this paper, we propose a novel image deblocking method to address the blocking artifacts reduction problem in a patch-based scheme. Image patches are clustered and reconstructed by the low-rank approximation, which is weighted by the geodesic distance. Experimental results show that the proposed method achieves higher PSNR than the state-of-the-art deblocking and denoising methods and the processed images present good visual quality.

Postprocessing of block-coded videos for deflicker and deblocking

In this paper, we propose a novel postprocessing method to suppress both the flickering and blocking artifacts in block-coded videos. For reducing the flickering effect between adjacent frames, we propose an adaptive multi-scale motion filtering method to maintain the motion coherence of processed video. For blocking artifacts suppression, we adopt a patch-based scheme in which similar patches are grouped in a spatio-temporal domain and each patch group is recovered by solving a low rank matrix completion problem. Experimental results show that the proposed method can significantly reduce the flickering and blocking artifacts in the decoded videos.

Structure-guided image completion via regularity statistics

In this paper, we propose a novel hierarchical image completion approach using regularity statistics, considering structure features. Guided by dominant structures, the target image is used to generate reference images in a self-reproductive way by image data enhancement. The structure-guided image data enhancement allows us to expand the search space for samples. A Markov Random Field model is used to guide the enhanced image data combination to globally reconstruct the target image. For lower computational complexity and more accurate structure estimation, a hierarchical process is implemented. Experiments demonstrate the effectiveness of our method comparing to several state-of-the-art image completion techniques.

General scale interpolation based on fine-grained isophote model with consistency constraint

In this paper, we propose a fine-grained isophote model with consistency constraint to characterize the piecewise-stationarity of image signals. According to this model, we present a novel interpolation algorithm. In this model, the displacement coefficient is used to model the isophote. Then fine-grained pixel intensity information is introduced to correct the displacement calculation and make the isophote estimation more robust. In order to handle the piecewise-stationarity, we force the isophote direction consistent in the local window when an interpolated line is piecewise-stationary. The proposed algorithm can accommodate the general scale enlargement. Experimental results demonstrate that the proposed approach achieves better performances in both objective and subjective quality assessment.

Multi-frame Super Resolution Using Refined Exploration of Extensive Self-examples

The multi-frame super resolution (SR) problem is to generate high resolution (HR) images by referring to a sequence of low resolution (LR) images. However, traditional multi-frame SR methods fail to take full advantage of the redundancy in LR images. In this paper, we present a novel algorithm using a refined example-based SR framework to cope with this problem. The refined framework includes two innovative points. First, based upon a thorough study of multi-frame and single frame statistics, we extend the single frame example-based scheme to multi-frame. Instead of training an external dictionary, we search for examples in the image pyramids of the LR inputs, i.e., a set of multi-resolution images derived from the input LRs. Second, we propose a new metric to find similar image patches, which not only considers the intensity and structure features of a patch but also adaptively balances between these two parts. With the refined framework, we are able to make the utmost of the redundancy in LR images to facilitate the SR process. As can be seen from the experiments, it is efficient in preserving structural features. Experimental results also show that our algorithm outperforms state-of-the-art methods on test sequences, achieving the average PSNR gain by up to 1.2dB.

Deep Combined Image Denoising with Cloud Images

Image denoising methods essentially lose some high-frequency (HF) information in denoising. To address this issue, we propose an end-to-end trainable deep network to additionally utilize online retrieved cloud images to compensate for the HF information loss based on the internal inferred results. In particular, the noise inference network first infers a noise map from the noisy image and derives an intermediate image by removing the noise map from the noisy image. Then the external online compensation is performed based on the intermediate image. The final results are obtained by fusing the intermediate image with external HF maps extracted by the external HF compensation network. Extensive experimental results demonstrate that our method achieves notably better performance than state-of-the-art denoising methods.