Honggang Chen

Single Image Super-Resolution via Adaptive Transform-Based Nonlocal Self-Similarity Modeling and Learning-Based Gradient Regularization

Single image super-resolution (SISR) is a challenging work, which aims to recover the missing information in an observed low-resolution (LR) image and generate the corresponding high-resolution (HR) version. As the SISR problem is severely ill-conditioned, effective prior knowledge of HR images is necessary to well pose the HR estimation. In this paper, an effective SISR method is proposed via the local structure-adaptive transform-based nonlocal self-similarity modeling and learning-based gradient regularization (LSNSGR). The LSNSGR exploits both the natural and learned priors of HR images, thus integrating the merits of conventional reconstruction-based and learning-based SISR algorithms. More specifically, on the one hand, we characterize nonlocal self-similarity prior (natural prior) in transform domain by using the designed local structure-adaptive transform; on the other hand, the gradient prior (learned prior) is learned via the jointly optimized regression model. The former prior is effective in suppressing visual artifacts, while the latter performs well in recovering sharp edges and fine structures. By incorporating the two complementary priors into the maximum a posteriori-based reconstruction framework, we optimize a hybrid L1- and L2-regularized minimization problem to achieve an estimation of the desired HR image. Extensive experimental results suggest that the proposed LSNSGR produces better HR estimations than many state-of-the-art works in terms of both perceptual and quantitative evaluations.

SGCRSR: Sequential gradient constrained regression for single image super-resolution

Abstract Single image super-resolution (SISR), which aims to produce an image with higher resolution and better visual quality from the given single low-resolution (LR) image, has attracted extensive attention in recent years. In particular, the regression-based SISR approaches, which learn the mapping between LR and high-resolution (HR) patch pairs, are efficient and effective as a whole. However, the super-resolved images produced by this kind of method often suffer from visual artifacts as no extra constraints or priors are enforced. To alleviate these shortcomings, we propose a Sequential Gradient Constrained Regression-based single image Super-Resolution (SGCRSR) framework, which provides an effective way to combine the conventional learning-based and reconstruction-based approaches. Firstly, we improve the performance of the well-known super-resolution (SR) method A+ by addressing its deficiencies in both training and testing stages and propose the enhanced A+ (EA+). Then, the EA+ model is applied in dual intensity–gradient domain to construct the Gradient Constrained Regression (GCR)-based SR unit. Finally, a GCR-based sequential SR framework, namely the SGCRSR, is established to improve the quality of super-resolved images gradually. Extensive experiments show that the proposed SGCRSR achieves better performance than several state-of-the-art SR algorithms.

CISRDCNN: Super-resolution of compressed images using deep convolutional neural networks

In recent years, much research has been conducted on image super-resolution (SR). To the best of our knowledge, however, few SR methods were concerned with compressed images. The SR of compressed images is a challenging task due to the complicated compression artifacts, while many images suffer from them in practice. The intuitive solution for this difficult task is to decouple it into two sequential but independent subproblems, i.e., compression artifacts reduction (CAR) and SR. Nevertheless, some useful details may be removed in CAR stage, which is contrary to the goal of SR and makes the SR stage more challenging. In this paper, an end-to-end trainable deep convolutional neural network is designed to perform SR on compressed images (CISRDCNN), which reduces compression artifacts and improves image resolution jointly. Experiments on compressed images produced by JPEG (we take the JPEG as an example in this paper) demonstrate that the proposed CISRDCNN yields state-of-the-art SR performance on commonly used test images and imagesets. The results of CISRDCNN on real low quality web images are also very impressive, with obvious quality enhancement. Further, we explore the application of the proposed SR method in low bit-rate image coding, leading to better rate-distortion performance than JPEG.

An Iterative Framework of Cascaded Deblocking and Superresolution for Compressed Images

Superresolution (SR) of compressed images is chall-enging due to the combination of resolution loss and compression artifacts. To solve these intertwined problems, the conventional cascading framework splits the solution into independent deblocking and SR subprocesses, where some existing high-frequency (HF) components are often oversmoothed during deblocking and information exchange between cascaded deblocking and SR remains untouched. In this paper, we propose an iterative cascading framework after analyzing the correlation between the two subprocesses. Deblocking is provided with a shape-adaptive low-rank prior to well preserve edges and an extra prior to restore the lost HF components. The latter prior represents an important feedback link from SR to deblocking, which is a novel design in this framework. To provide an accurate and noise-robust feedback of the extra prior, an SR method via singular value decomposition projection is also developed. The extensive experimental results demonstrate the superior performance of the proposed method.

Single image super resolution based on feature enhancement

In most of the existing regression-based methods, mapping matrices are directly learnt from features which are extracted from the interpolation results of low-resolution (LR) images. Nevertheless, this kind of features usually suffer from many artifacts which may produce bad effects on image super-resolution (SR) reconstruction. In this paper, we propose an effective single image super-resolution (SISR) method, which restores a high-resolution (HR) image through the feature enhancement mapping matrices and detail enhancement mapping matrices. The method bases on adjusted anchored neighborhood regression to get more accurate features and better mapping matrices. In the proposed framework, we train the linear mapping matrices twice. In the first stage, we extract interpolated LR image features and HR image features to learn feature enhancement mapping matrices, then these learnt matrices are used to enhance the extracted features. In the second stage, we learn detail enhancement mapping matrices from enhanced LR image features and HR image patches. With the two-stage learning strategy, more accurate mapping matrices can be obtained and thus better SR results can be achieved. Experimental results verify the effectiveness of our method compared to other state-of-the-art methods.

Video super-resolution using joint regularization

Video super-resolution (SR) is an inverse problem, and with this method, we can reconstruct a high-resolution (HR) version of a low-resolution (LR) video sequence. Because regularization-based method can solve the pathological problem in super-resolution, so it is widely used. However, in many traditional regularization terms, only the intra-image correlation will be taken into consideration so that the redundancy between adjacent frames is not be utilized. In order to make full use of both inter-image correlation and intra-image correlation, we combine compensation-based TV (CTV) regularization term with multi-non-local low-rank (MNLR) regularization term in our algorithm. Moreover, we utilize a weight matrix to reduce the negative impacts which is caused by registration residuals in CTV, and the weight matrix is based on spatial information filtering and clustering. The experiments show that we can get better results than the compared methods by the proposed algorithm in visual quality and objective effective evaluation.

Single image super-resolution based on deep learning and gradient transformation

In this paper, an effective single image super-resolution method based on deep learning and gradient transformation is proposed. Firstly, the low-resolution image is upscaled by convolutional neural network. Then we calculate the gradients of the upscaled image, and transform them into desired gradients by using gradient transformation network. The transformed gradients are utilized as a constraint to establish the reconstruction energy function. Finally, we optimize this energy function to estimate the high-resolution image. Experimental results show that our proposed algorithm can produce sharp high-resolution images with few ringing or jaggy artifacts, and our results have high values of the objective assessment parameters.