Chang Li

Infrared and visible image fusion via gradient transfer and total variation minimization

We propose a new IR/visible fusion method based on gradient transfer and TV minimization.It can keep both the thermal radiation and the appearance information in the source images.We generalize the proposed method to fuse image pairs without pre-registration.Our fusion results look like sharpened IR images with highlighted target and abundant textures.To the best of our knowledge, the proposed fusion strategy has not yet been studied. In image fusion, the most desirable information is obtained from multiple images of the same scene and merged to generate a composite image. This resulting new image is more appropriate for human visual perception and further image-processing tasks. Existing methods typically use the same representations and extract the similar characteristics for different source images during the fusion process. However, it may not be appropriate for infrared and visible images, as the thermal radiation in infrared images and the appearance in visible images are manifestations of two different phenomena. To keep the thermal radiation and appearance information simultaneously, in this paper we propose a novel fusion algorithm, named Gradient Transfer Fusion (GTF), based on gradient transfer and total variation (TV) minimization. We formulate the fusion problem as an ?1-TV minimization problem, where the data fidelity term keeps the main intensity distribution in the infrared image, and the regularization term preserves the gradient variation in the visible image. We also generalize the formulation to fuse image pairs without pre-registration, which greatly enhances its applicability as high-precision registration is very challenging for multi-sensor data. The qualitative and quantitative comparisons with eight state-of-the-art methods on publicly available databases demonstrate the advantages of GTF, where our results look like sharpened infrared images with more appearance details.

Hyperspectral Image Classification With Robust Sparse Representation

Recently, the sparse representation-based classification (SRC) methods have been successfully used for the classification of hyperspectral imagery, which relies on the underlying assumption that a hyperspectral pixel can be sparsely represented by a linear combination of a few training samples among the whole training dictionary. However, the SRC-based methods ignore the sparse representation residuals (i.e., outliers), which may make the SRC not robust for outliers in practice. To overcome this problem, we propose a robust SRC (RSRC) method which can handle outliers. Moreover, we extend the RSRC to the joint robust sparsity model named JRSRC, where pixels in a small neighborhood around the test pixel are simultaneously represented by linear combinations of a few training samples and outliers. The JRSRC can also deal with outliers in hyperspectral classification. Experiments on real hyperspectral images demonstrate that the proposed RSC and JRSRC have better performances than the orthogonal matching pursuit (OMP) and simultaneous OMP, respectively. Moreover, the JRSRC outperforms some other popular classifiers.

Hyperspectral image denoising using the robust low-rank tensor recovery.

Denoising is an important preprocessing step to further analyze the hyperspectral image (HSI), and many denoising methods have been used for the denoising of the HSI data cube. However, the traditional denoising methods are sensitive to outliers and non-Gaussian noise. In this paper, by utilizing the underlying low-rank tensor property of the clean HSI data and the sparsity property of the outliers and non-Gaussian noise, we propose a new model based on the robust low-rank tensor recovery, which can preserve the global structure of HSI and simultaneously remove the outliers and different types of noise: Gaussian noise, impulse noise, dead lines, and so on. The proposed model can be solved by the inexact augmented Lagrangian method, and experiments on simulated and real hyperspectral images demonstrate that the proposed method is efficient for HSI denoising.

Hyperspectral Unmixing with Robust Collaborative Sparse Regression

Recently, sparse unmixing (SU) of hyperspectral data has received particular attention for analyzing remote sensing images. However, most SU methods are based on the commonly admitted linear mixing model (LMM), which ignores the possible nonlinear effects (i.e., nonlinearity). In this paper, we propose a new method named robust collaborative sparse regression (RCSR) based on the robust LMM (rLMM) for hyperspectral unmixing. The rLMM takes the nonlinearity into consideration, and the nonlinearity is merely treated as outlier, which has the underlying sparse property. The RCSR simultaneously takes the collaborative sparse property of the abundance and sparsely distributed additive property of the outlier into consideration, which can be formed as a robust joint sparse regression problem. The inexact augmented Lagrangian method (IALM) is used to optimize the proposed RCSR. The qualitative and quantitative experiments on synthetic datasets and real hyperspectral images demonstrate that the proposed RCSR is efficient for solving the hyperspectral SU problem compared with the other four state-of-the-art algorithms.

Infrared and visible image fusion methods and applications: A survey

Abstract Infrared images can distinguish targets from their backgrounds based on the radiation difference, which works well in all-weather and all-day/night conditions. By contrast, visible images can provide texture details with high spatial resolution and definition in a manner consistent with the human visual system. Therefore, it is desirable to fuse these two types of images, which can combine the advantages of thermal radiation information in infrared images and detailed texture information in visible images. In this work, we comprehensively survey the existing methods and applications for the fusion of infrared and visible images. First, infrared and visible image fusion methods are reviewed in detail. Meanwhile, image registration, as a prerequisite of image fusion, is briefly introduced. Second, we provide an overview of the main applications of infrared and visible image fusion. Third, the evaluation metrics of fusion performance are discussed and summarized. Fourth, we select eighteen representative methods and nine assessment metrics to conduct qualitative and quantitative experiments, which can provide an objective performance reference for different fusion methods and thus support relative engineering with credible and solid evidence. Finally, we conclude with the current status of infrared and visible image fusion and deliver insightful discussions and prospects for future work. This survey can serve as a reference for researchers in infrared and visible image fusion and related fields.

GBM-Based Unmixing of Hyperspectral Data Using Bound Projected Optimal Gradient Method

The generalized bilinear model (GBM) has been widely used for the nonlinear unmixing of hyperspectral images, and traditional GBM solvers include the Bayesian algorithm, the gradient descent algorithm, the semi-nonnegative-matrix-factorization algorithm, etc. However, they suffer from one of the following problems: high computational cost, sensitive to initialization, and the pixelwise algorithm hinders us from applying to large hyperspectral images. In this letter, we apply Nesterovs optimal gradient method to solve the least-square problem under the bound constraint, which is named as the bound projected optimal gradient method (BPOGM). The BPOGM can achieve the optimal convergence rate of O(1/k2), with k denoting the number of iterations in BPOGM. We further apply the BPOGM to solve the GBM-based unmixing problem. Experiments on both synthetic data sets and real hyperspectral images demonstrate that the BPOGM is efficient for solving the GBM-based unmixing problem.

Hyperspectral image denoising with superpixel segmentation and low-rank representation

Recently, low-rank representation (LRR) based hyperspectral image (HSI) restoration method has been proven to be a powerful tool for simultaneously removing different types of noise, such as Gaussian, dead pixels and impulse noise. However, the LRR based method just adopts the square patch denoising strategy, which makes it not able to excavate the spatial information in HSI. This paper integrates superpixel segmentation (SS) into LRR and proposes a novel denoising method called SSLRR. First, the principal component analysis (PCA) is adopted to obtain the first principal component of HSI. Then the SS is adopted to the first principal component of HSI to get the homogeneous regions. Since we excavate the spatial-spectral information of HSI by combining PCA with SS, it is better than simply dividing the HSI into square patches. Finally, we employ the LRR to each homogeneous region of HSI, which enable us to remove all the above mentioned different types of noise simultaneously. Extensive experiments conducted on synthetic and real HSIs indicate that the SSLRR is efficient for HSI denoising.

Sparse Unmixing of Hyperspectral Data with Noise Level Estimation

Recently, sparse unmixing has received particular attention in the analysis of hyperspectral images (HSIs). However, traditional sparse unmixing ignores the different noise levels in different bands of HSIs, making such methods sensitive to different noise levels. To overcome this problem, the noise levels at different bands are assumed to be different in this paper, and a general sparse unmixing method based on noise level estimation (SU-NLE) under the sparse regression framework is proposed. First, the noise in each band is estimated on the basis of the multiple regression theory in hyperspectral applications, given that neighboring spectral bands are usually highly correlated. Second, the noise weighting matrix can be obtained from the estimated noise. Third, the noise weighting matrix is integrated into the sparse regression unmixing framework, which can alleviate the impact of different noise levels at different bands. Finally, the proposed SU-NLE is solved by the alternative direction method of multipliers. Experiments on synthetic datasets show that the signal-to-reconstruction error of the proposed SU-NLE is considerably higher than those of the corresponding traditional sparse regression unmixing methods without noise level estimation, which demonstrates the efficiency of integrating noise level estimation into the sparse regression unmixing framework. The proposed SU-NLE also shows promising results in real HSIs.

Infrared ultraspectral signature classification based on a restricted Boltzmann machine with sparse and prior constraints

The state-of-the-art ultraspectral technology brings a new hope for the high precision applications due to its high spectral resolution. However, it comes with new challenges brought by the improvement of spectral resolution such as the Hughes phenomenon and over-fitting issue, and our work is aimed at addressing these problems. As new Markov random field (MRF) models, the restricted Boltzmann machines (RBMs) have been used as generative models for many different pattern recognition and artificial intelligence applications showing promising and outstanding performance. In this article, we propose a new method for infrared ultraspectral signature classification based on the RBMs, which adopt the regularization-based techniques to improve the classification accuracy and robustness to noise compared to traditional RBMs. First, we add an arctan-like term to the objective function as a sparse constraint to improve the classification accuracy. Second, we utilize a Gaussian prior to avoid the over-fitting problem. Third, to further improve the classification performance, a multi-layer RBM model, a deep belief network (DBN), is adopted for infrared ultraspectral signature classification. Experiments using different spectral libraries provided by the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) and the Environmental Protection Agency (EPA) were performed to evaluate the performance of the proposed method by comparing it with other traditional methods, including spectral coding-based classifiers (binary coding (BC), spectral feature-based binary coding (SFBC), and spectral derivative feature coding (SDFC) matching methods), a novel feature extraction method termed crosscut feature extraction matching (CF), and three machine learning methods (artificial deoxyribonucleic acid (DNA)-based spectral matching (ADSM), DBN, and sparse deep belief network (SparseDBN)). Experimental results demonstrate that the proposed method is superior to the other methods with which it was compared and can simultaneously improve the accuracy and robustness of classification.

Robust Sparse Hyperspectral Unmixing With

Sparse unmixing (SU) of hyperspectral data have recently received particular attention for analyzing remote sensing images, which aims at finding the optimal subset of signatures to best model the mixed pixel in the scene. However, most SU methods are based on the commonly admitted linear mixing model, which ignores the possible nonlinear effects (i.e., nonlinearity), and the nonlinearity is merely treated as outlier. Besides, the traditional SU algorithms often adopt the <inline-formula> <tex-math notation=LaTeX>