Hongyi Liu

A New Pan-Sharpening Method With Deep Neural Networks

A deep neural network (DNN)-based new pansharpening method for the remote sensing image fusion problem is proposed in this letter. Research on representation learning suggests that the DNN can effectively model complex relationships between variables via the composition of several levels of nonlinearity. Inspired by this observation, a modified sparse denoising autoencoder (MSDA) algorithm is proposed to train the relationship between high-resolution (HR) and low-resolution (LR) image patches, which can be represented by the DNN. The HR/LR image patches only sample from the HR/LR panchromatic (PAN) images at hand, respectively, without requiring other training images. By connecting a series of MSDAs, we obtain a stacked MSDA (S-MSDA), which can effectively pretrain the DNN. Moreover, in order to better train the DNN, the entire DNN is again trained by a back-propagation algorithm after pretraining. Finally, assuming that the relationship between HR/LR multispectral (MS) image patches is the same as that between HR/LR PAN image patches, the HR MS image will be reconstructed from the observed LR MS image using the trained DNN. Comparative experimental results with several quality assessment indexes show that the proposed method outperforms other pan-sharpening methods in terms of visual perception and numerical measures.

Variational Bayesian Method for Retinex

In this paper, we propose a variational Bayesian method for Retinex to simulate and interpret how the human visual system perceives color. To construct a hierarchical Bayesian model, we use the Gibbs distributions as prior distributions for the reflectance and the illumination, and the gamma distributions for the model parameters. By assuming that the reflection function is piecewise continuous and illumination function is spatially smooth, we define the energy functions in the Gibbs distributions as a total variation function and a smooth function for the reflectance and the illumination, respectively. We then apply the variational Bayes approximation to obtain the approximation of the posterior distribution of unknowns so that the unknown images and hyperparameters are estimated simultaneously. Experimental results demonstrate the efficiency of the proposed method for providing competitive performance without additional information about the unknown parameters, and when prior information is added the proposed method outperforms the non-Bayesian-based Retinex methods we compared.

Hyperspectral Imagery Super-Resolution by Compressive Sensing Inspired Dictionary Learning and Spatial-Spectral Regularization

Due to the instrumental and imaging optics limitations, it is difficult to acquire high spatial resolution hyperspectral imagery (HSI). Super-resolution (SR) imagery aims at inferring high quality images of a given scene from degraded versions of the same scene. This paper proposes a novel hyperspectral imagery super-resolution (HSI-SR) method via dictionary learning and spatial-spectral regularization. The main contributions of this paper are twofold. First, inspired by the compressive sensing (CS) framework, for learning the high resolution dictionary, we encourage stronger sparsity on image patches and promote smaller coherence between the learned dictionary and sensing matrix. Thus, a sparsity and incoherence restricted dictionary learning method is proposed to achieve higher efficiency sparse representation. Second, a variational regularization model combing a spatial sparsity regularization term and a new local spectral similarity preserving term is proposed to integrate the spectral and spatial-contextual information of the HSI. Experimental results show that the proposed method can effectively recover spatial information and better preserve spectral information. The high spatial resolution HSI reconstructed by the proposed method outperforms reconstructed results by other well-known methods in terms of both objective measurements and visual evaluation.

Local brightness adaptive image colour enhancement with Wasserstein distance

Colour image enhancement is an important preprocessing phase of many image analysis tasks such as image segmentation, pattern recognition and so on. This study presents a new local brightness adaptive variational model using Wasserstein distance for colour image enhancement. Under the perceptually inspired variational framework, the proposed energy functional consists of an improved contrast energy term and a Wasserstein dispersion energy term. To better adjust image dynamic range, the authors propose a local brightness adaptive contrast energy term using the average brightness of image local patch as the local brightness indicator. To restore image true colours, a Wasserstein distance-based dispersion energy term is used to measure the statistical similarity between the original image and the enhanced image. The proposed energy functional is minimised by using a gradient descent algorithm. Two objective measures are used to quantitatively measure the enhancement quality. Experimental results demonstrate the efficiency of the proposed model for removing colour cast and haze, enhancing contrast, recovering details and equalising low key images.

A novel hyperspectral image anomaly detection method based on low rank representation

This paper presents a novel method for anomaly detection in hyperspectral image(HSI) based on low-rank representation. In the observed HSI, the anomalies can be separated from the background. Since each pixel in the background can be approximately represented by a background dictionary, and the representation coefficients of the background pixels are correlative, a low-rank representation model is used to model the background part. Besides, to gain robust representation coefficient, the sum-to-one constraint is added. The advantage of the proposed low-rank representation sum-to-one (LRRSTO) method is that it makes use of the global correlation of the background and strength the robustness of the representation. Experiments results have been conducted using both simulated and real data sets. These experiments indicated that our algorithm achieves very promising performance.

Spatial-spectral compressive sensing for hyperspectral images super-resolution over learned dictionary

This paper proposes a new hyperspectral images superresolution (HSI-SR) method based on compressive sensing (CS) theory, spatial sparsity and spectral similarity prior. First, according to sparsity and incoherence of CS theory, we propose a new dictionary learning method, ensuring that the learned dictionary not only has less dimensionality to speed up the sparse decomposition, but also satisfies sparsity well. Then, we introduce the spatial sparsity and spectral similarity regularizations into HSI-SR model, which can recover the spatial information effectively and preserve the spectral information well. The experimental results show the proposed method outperforms other well-known methods in terms of both objective measurements and visual evaluation.

Color demosaicking via nonlocal tensor representation

A single sensor camera can capture scenes by means of color filter array. Each pixel samples only one of the three primary colors. Color demosaicking (CDM) is a process of reconstruction a full color image from this sensor data. In this paper, we propose a novel CDM scheme based on learned simultaneous sparse coding over nonlocal tensor representation. First, similar 2D patches are grouped to form a three-order tensor, that is, 3D array. Then, three sub-dictionaries, which characterize the coherent structures that appear in each dimension of the grouped tensor, are learned jointly by using Tucker decomposition. The consequent coefficient tensor is imposed by the grouped-block-sparsity constraint, which forces the similar patches to share the same atoms of the dictionaries in their sparse decomposition. Experimental results demonstrate the effectiveness both in the average CPSNR and visual quality.

Adaptive tensor matrix based kernel regression for hyperspectral image denoising

Kernel regression has been shown to be a powerful image denoising technique. In this paper, a three-dimensional (3-D) kernel regression hyperspectral image (HSI) denoising mechanism is proposed. The main contributions of this paper can be summarized as follows: Three orientation vectors and the corresponding coefficients are presented, which are adaptive for each pixel based on the innovation of 2-D structure tensor. An adaptive-driven 3-D tensor matrix is proposed for kernel regression, in which the spatial geometric structure and spectrum continuity are both considered. The proposed adaptive kernel regression is applied to HSI denoising. Both stimulated and real data experiments indicate that the proposed method can work well in detail preservation and noise removal.

Image Demosaicking Using Chromatic-Preserved Kernel Regression

In this paper, a color image demosaicking (CDM) method is proposed. For preserving the chromatic information, the original RGB plane is transformed to HSI plane, and then an anisotropic tensor matrix kernel regression based interpolation technique is processed in I-component. Finally, a directional interpolation based on R-G/B-G difference is used for R and B channel refinements. Experimental results show that our proposed demosaicking method outperforms other available methods.

A structure-preserved nonlocal iterative regularization model for image denoising

Non-local Means(NLM) is increasingly popular in image denoising. In this paper, the nonlocal structure similarity of images obtained by the iteration is exploited. By combining the nonlocal similarity constraints with total variation regularization, an iterative regularized variational model is proposed, in which the nonlocal weight depends on local structure of patches. An effective algorithm is also presented to solve the optimization problem based on split Bregman iteration. Experimental results reveal that the proposed method is competitive with the state-of-art denoising algorithms, especially for images with strong noise.