Binyu Yan

A new framework for remote sensing image super-resolution

Jointly representing an image with different types of features is proposed in feature extraction stage.Multi-type feature dictionaries are obtained in sparse representation stage to capture the different structures of the image.Multiple HR patches can be estimated with multi-type feature dictionaries from one LR patch.A strategy is proposed to integrate those estimated HR patches by adaptively adjusting the weights. Remote sensing images play an important role in many practical applications, however, due to the physical limitations of remote sensing devices, it is difficult to obtain images at an expecting high resolution level. Acquiring high-resolution(HR) images from the original low-resolution(LR) ones with super-resolution(SR) methods has always been an attractive proposition in embedded systems including various kinds of tablet PC and smart phone. SR methods based on sparse representation have been successfully used in processing remote sensing images, however, they have two major problems in common. First, they use only one type of image features to represent the low resolution(LR) images. However, one single type of features cannot accurately represent an image due to the diverse structures of the image, as a result, artifacts would be produced simultaneously. Second, many dictionary learning methods try to build a universal dictionary with only one single type of features. However, apparently, a dictionary with a single type of features is not enough to capture the different structures of a remote sensing image, without any doubt, the resultant image would turn out to be a poor one. To overcome the problems above, we propose a new framework for remote sensing image super resolution: sparse representation-based SR method by processing dictionaries with multi-type features. First, in order to represent the remote sensing image more accurately, different types of features are extracted from images. Second, to achieve a better performance, various dictionaries with multi-type features are learned to capture the essential structures of the image. Then, its proposed to adaptively control the weights of the high resolution(HR) patches obtained by different dictionaries. Numerous experiments validate that this proposed framework brings better results in terms of both objective quantitation and visual perception than other compared algorithms.

Remote Sensing Image Super-resolution Using Dual-Dictionary Pairs Based on Sparse Presentation and Multiple Features

In this paper, a super-resolution method based on sparse dictionary and multiple futures is proposed for remote sensing images. Super-resolution aims to reconstruct the high-frequency detail from the low resolution image. In this paper, high frequency is decomposed into two parts: primary high-frequency and residual high frequency. We proposed dual-dictionary pairs, i.e. primitive sparse dictionary pair and residual sparse dictionary pair to recover primary high-frequency and residual high frequency respectively. To describe the image more precise, we use multiple features to describe the structure of the image, and combine them together to present the image. Then use the combination futures to train the dictionary. The experimental results show that the proposed algorithm has a good performance, and the high-resolution image generated by the proposed method is with better subjective and objective quality compared with other methods.

Fast multisensor infrared image super-resolution scheme with multiple regression models

We propose a fast super-resolution scheme with multiple regression models.This study combines the information from VI images and IR images.This study divides training patches into multiple clusters by K-means clustering.We propose a soft-assignment based multiple regression method. High resolution (HR) infrared (IR) images play an important role in many areas. However, it is difficult to obtain images at a desired resolution level because of the limitation of hardware and image environment. Therefore, improving the spatial resolution of infrared images has become more and more urgent. Methods based on sparse coding have been successfully used in single-image super-resolution (SR) reconstruction. However, the existing sparse representation-based SR method for infrared (IR) images usually encounter three problems. First, IR images always lack detailed information, which leads to unsatisfying IR image reconstruction results with conventional method. Second, the existing dictionary learning methods in SR aim at learning a universal and over-complete dictionary to represent various image structures. A large number of different structural patterns exist in an image, whereas one dictionary is not capable of capturing all of the different structures. Finally, the optimization for dictionary learning and image reconstruction requires a highly intensive computation, which restricts the practical application in real-time systems. To overcome these problems, we propose a fast IR image SR scheme. Firstly, we integrate the information from visible (VI) images and IR images to improve the resolution of IR images because images acquired by different sensors provide complementary information for the same scene. Second, we divide the training patches into several clusters, then the multiple dictionaries are learned for each cluster in order to provide each patch with a more accurate dictionary. Finally, we propose an method of Soft-assignment based Multiple Regression (SMR). SMR reconstructs the high resolution (HR) patch by the dictionaries corresponding to its K nearest training patch clusters. The method has a low level of computational complexity and may be readily suitable for real-time processing applications. Numerous experiments validate that this scheme brings better results in terms of quantization and visual perception than many state-of-the-art methods, while at the same time maintains a relatively low level of time complexity. Since the main computation of this scheme is matrix multiplication, it will be easily implemented in FPGA system.

Multiscale and Multitopic Sparse Representation for Multisensor Infrared Image Superresolution

Methods based on sparse coding have been successfully used in single-image superresolution (SR) reconstruction. However, the traditional sparse representation-based SR image reconstruction for infrared (IR) images usually suffers from three problems. First, IR images always lack detailed information. Second, a traditional sparse dictionary is learned from patches with a fixed size, which may not capture the exact information of the images and may ignore the fact that images naturally come at different scales in many cases. Finally, traditional sparse dictionary learning methods aim at learning a universal and overcomplete dictionary. However, many different local structural patterns exist. One dictionary is inadequate in capturing all of the different structures. We propose a novel IR image SR method to overcome these problems. First, we combine the information from multisensors to improve the resolution of the IR image. Then, we use multiscale patches to represent the image in a more efficient manner. Finally, we partition the natural images into documents and group such documents to determine the inherent topics and to learn the sparse dictionary of each topic. Extensive experiments validate that using the proposed method yields better results in terms of quantitation and visual perception than many state-of-the-art algorithms.

Remote Sensing Images Super-resolution Based on Sparse Dictionaries and Residual Dictionaries

In this paper, a sensing image super-resolution (SR) reconstruction method is proposed. Sparse dictionary dealing with remote sensing image SR problem is introduced in this work. The sparse dictionary is based on a sparsity model where the dictionary atoms have sparse representation over a basic dictionary. The sparse dictionary consists of two parts: basic dictionary and atom representation matrix. The sparse dictionary leads to compact representation and it is both adaptive and efficient. Furthermore, compared with conventional SR methods, two dictionary pairs, i.e. primitive sparse dictionary pair and residual sparse dictionary pair, are proposed. The primitive sparse dictionary pair is learned to reconstruct initial high-resolution (HR) remote sensing image from a single low-resolution (LR) input. However, the initial HR remote sensing image loses some details compare with the corresponding original HR image completely. Therefore, residual sparse dictionary pair is learned to reconstruct residual information. The proposed method is tested on remote sensing images, and the experimental results indicate that the proposed algorithm can provide substantial improvement in resolution of remote sensing images, and the results are superior in quality to the results produced by other methods.

Infrared Image Super-Resolution with Parallel Random Forest

Infrared imaging has the advantage of all-weather working ability. Due to the limitation of the hardware and the high cost, the resolution of infrared image (IR) is very low. To improve the resolution of IR images, this paper exploits super-resolution (SR) method for IR images. A new SR framework by using random forests is proposed in this paper. Existing methods adopts single regression model for SR. However, which single regression model tends to overfit training data, and would lead to a poor performance. Furthermore, the existing methods are not suitable for real-time system due to the heavy time consuming. To resolve this problem, an ensemble regression model, i.e. random forests rather than single regression model is adopted in this paper. In addition, to achieve better results multi-regression models rather than a single regression model are trained on the clustered training data. Moreover, the features used in many SR methods cannot extract features on diagonal orientation. To resolve this problem, we adopt a second order derivative filter, which can extract features on diagonal orientation. The experimental results demonstrate the availability of the proposed method.

Implementing real-time RCF-Retinex image enhancement method using CUDA

RCF-Retinex is a novel Retinex-based image enhancement method which can improve contrast, eliminate noise, and enhance details simultaneously. It utilizes region covariance filter (RCF) to estimate the illumination. However, RCF-Retinex encounters time-consuming problem, since the region covariance filter is computationally intensive, which restricts the practical application in real-time systems. Therefore, it is necessary to decrease the computational complexity by parallelization. This paper proposes a GPU-based RCF-Retinex, which can accelerate region covariance filter using CUDA. It is feasible to use CUDA to parallel the region covariance filter due to its consecutive convolution operations, thus we can obtain the illumination image fast. Experiments have proved the improvement of running time and the enhancement results are similar with those using the unaccelerated RCF-Retinex method.

A sparse representation based pansharpening method

Abstract Insufficient information captured by a single satellite sensor can hardly be fit real applications. Pansharpening is a hot topic in remote sensing region, which combines the spectral information of multispectral image and spatial details of panchromatic image to obtain high spatial resolution multispectral image. In this paper, we present a novel sparse representation-based pansharpening method, which consists three stages: dictionary construction, panchromatic image decomposition, and high spatial resolution multispectral image reconstruction. First, we use multispectral images as training set and calculate intensity channels of multispectral images. Then we obtain the high-frequency components and low-frequency components of intensity channels. Second, we sparsely decompose the panchromatic image by using a pair of dictionaries to obtain high-frequency components and low-frequency components of the panchromatic image. Third, the optimized high-frequency components of the panchromatic image will be integrated into the multispectral image to generate the final high resolution multispectral image. The quantitative and subjective evaluations show that the proposed method performs better effectiveness and practicality than the existing sparse representation-based methods.

An Efficient Image Enlargement Method for Image Sensors of Mobile in Embedded Systems

Main challenges for image enlargement methods in embedded systems come from the requirements of good performance, low computational cost, and low memory usage. This paper proposes an efficient image enlargement method which can meet these requirements in embedded system. Firstly, to improve the performance of enlargement methods, this method extracts different kind of features for different morphologies with different approaches. Then, various dictionaries based on different kind of features are learned, which represent the image in a more efficient manner. Secondly, to accelerate the enlargement speed and reduce the memory usage, this method divides the atoms of each dictionary into several clusters. For each cluster, separate projection matrix is calculated. This method reformulates the problem as a least squares regression. The high-resolution (HR) images can be reconstructed based on a few projection matrixes. Numerous experiment results show that this method has advantages such as being efficient and real-time and having less memory cost. These advantages make this method easy to implement in mobile embedded system.

Infrared Image Super-Resolution by Using Sparse Dictionary and Nonsubsampled Contourlet Transform

Due to the limitation of hardware, Infrared (IR) image has low-resolution (LR) and poor visual quality. Infrared image super-resolution (SR) is a good solution for this problem. However, the conventional SR methods have some drawbacks. Firstly, the trained dictionary is an unstructured dictionary, which may lead to worse results. Secondly, the representation of the image is too simple to effectively represent image. To resolve these problems, in this paper, firstly, the sparse dictionary is introduced into the IR image SR to get better results. Secondly, nonsubsampled contour let transform (NSCT) is employed in the proposed method to obtain a better representation of IR image. The experiment results indicate that the subjective visual effect and objective evaluation are acquired excellent performance in the proposed method. Besides, this method is superior to other methods in the paper.