Xiaoqiang Lu

Image quality assessment

Full reference image quality assessment is very important for many image processing applications. The challenge of image quality assessment lies in two aspects: (1) formulating perceptual meaningful features and (2) finding a way to pool them into a single quality score. A novel two-step approach is proposed to address these problems. In the first step, sparse representations of local image patches are computed to simulate the low level characteristic of the human vision system (HVS) and represent the meaningful image structures. The differences between the representations of distorted and undistorted patches are utilized to measure the local distortion. In the second step, these local distortion measurements are fused into a single image quality score by using kernel ridge regression (KRR). Kernel ridge regression can mimic the complex high level behaviors of human vision system and is shown to be an effective way to learn the relationship between local quality measurements and quality score. The contributions of this paper would be summarized as follows: (1) extracting approximate perceptual meaningful features in image quality assessment is transformed as a sparse representation problem. In this case, the sparse representation coefficients can reflect the salient local structures and give local quality assessments. (2) The KRR is utilized to pool the local quality assessments into a single image quality score. Thus, the nonlinear relationship between the objective model outputs and the subjective quality ratings can be learned by exploiting the KRR. (3) Extensive experiments are conducted on six public databases. Compared with other approaches, the proposed approach has achieved the best performance, which demonstrates the effectiveness and robustness of the proposed approach.

Manifold Regularized Sparse NMF for Hyperspectral Unmixing

Hyperspectral unmixing is one of the most important techniques in analyzing hyperspectral images, which decomposes a mixed pixel into a collection of constituent materials weighted by their proportions. Recently, many sparse nonnegative matrix factorization (NMF) algorithms have achieved advanced performance for hyperspectral unmixing because they overcome the difficulty of absence of pure pixels and sufficiently utilize the sparse characteristic of the data. However, most existing sparse NMF algorithms for hyperspectral unmixing only consider the Euclidean structure of the hyperspectral data space. In fact, hyperspectral data are more likely to lie on a low-dimensional submanifold embedded in the high-dimensional ambient space. Thus, it is necessary to consider the intrinsic manifold structure for hyperspectral unmixing. In order to exploit the latent manifold structure of the data during the decomposition, manifold regularization is incorporated into sparsity-constrained NMF for unmixing in this paper. Since the additional manifold regularization term can keep the close link between the original image and the material abundance maps, the proposed approach leads to a more desired unmixing performance. The experimental results on synthetic and real hyperspectral data both illustrate the superiority of the proposed method compared with other state-of-the-art approaches.

Graph-Regularized Low-Rank Representation for Destriping of Hyperspectral Images

Hyperspectral image destriping is a challenging and promising theme in remote sensing. Striping noise is a ubiquitous phenomenon in hyperspectral imagery, which may severely degrade the visual quality. A variety of methods have been proposed to effectively alleviate the effects of the striping noise. However, most of them fail to take full advantage of the high spectral correlation between the observation subimages in distinct bands and consider the local manifold structure of the hyperspectral data space. In order to remedy this drawback, in this paper, a novel graph-regularized low-rank representation (LRR) destriping algorithm is proposed by incorporating the LRR technique. To obtain desired destriping performance, two sides of performing destriping are included: 1) To exploit the high spectral correlation between the observation subimages in distinct bands, the technique of LRR is first utilized for destriping, and 2) to preserve the intrinsic local structure of the original hyperspectral data, the graph regularizer is incorporated in the objective function. The experimental results and quantitative analysis demonstrate that the proposed method can both remove striping noise and achieve cleaner and higher contrast reconstructed results.

Remote Sensing Image Scene Classification: Benchmark and State of the Art

Remote sensing image scene classification plays an important role in a wide range of applications and hence has been receiving remarkable attention. During the past years, significant efforts have been made to develop various data sets or present a variety of approaches for scene classification from remote sensing images. However, a systematic review of the literature concerning data sets and methods for scene classification is still lacking. In addition, almost all existing data sets have a number of limitations, including the small scale of scene classes and the image numbers, the lack of image variations and diversity, and the saturation of accuracy. These limitations severely limit the development of new approaches especially deep learning-based methods. This paper first provides a comprehensive review of the recent progress. Then, we propose a large-scale data set, termed “NWPU-RESISC45,” which is a publicly available benchmark for REmote Sensing Image Scene Classification (RESISC), created by Northwestern Polytechnical University (NWPU). This data set contains 31 500 images, covering 45 scene classes with 700 images in each class. The proposed NWPU-RESISC45 1) is large-scale on the scene classes and the total image number; 2) holds big variations in translation, spatial resolution, viewpoint, object pose, illumination, background, and occlusion; and 3) has high within-class diversity and between-class similarity. The creation of this data set will enable the community to develop and evaluate various data-driven algorithms. Finally, several representative methods are evaluated using the proposed data set, and the results are reported as a useful baseline for future research.

Scene Recognition by Manifold Regularized Deep Learning Architecture

Scene recognition is an important problem in the field of computer vision, because it helps to narrow the gap between the computer and the human beings on scene understanding. Semantic modeling is a popular technique used to fill the semantic gap in scene recognition. However, most of the semantic modeling approaches learn shallow, one-layer representations for scene recognition, while ignoring the structural information related between images, often resulting in poor performance. Modeled after our own human visual system, as it is intended to inherit humanlike judgment, a manifold regularized deep architecture is proposed for scene recognition. The proposed deep architecture exploits the structural information of the data, making for a mapping between visible layer and hidden layer. By the proposed approach, a deep architecture could be designed to learn the high-level features for scene recognition in an unsupervised fashion. Experiments on standard data sets show that our method outperforms the state-of-the-art used for scene recognition.

Geometry constrained sparse coding for single image super-resolution

The choice of the over-complete dictionary that sparsely represents data is of prime importance for sparse coding-based image super-resolution. Sparse coding is a typical unsupervised learning method to generate an over-complete dictionary. However, most of the sparse coding methods for image super-resolution fail to simultaneously consider the geometrical structure of the dictionary and corresponding coefficients, which may result in noticeable super-resolution reconstruction artifacts. In this paper, a novel sparse coding method is proposed to preserve the geometrical structure of the dictionary and the sparse coefficients of the data. Moreover, the proposed method can preserve the incoherence of dictionary entries, which is critical for sparse representation. Inspired by the development on non-local self-similarity and manifold learning, the proposed sparse coding method can provide the sparse coefficients and learned dictionary from a new perspective, which have both reconstruction and discrimination properties to enhance the learning performance. Extensive experimental results on image super-resolution have demonstrated the effectiveness of the proposed method.

Semi-Supervised Multitask Learning for Scene Recognition

Scene recognition has been widely studied to understand visual information from the level of objects and their relationships. Toward scene recognition, many methods have been proposed. They, however, encounter difficulty to improve the accuracy, mainly due to two limitations: 1) lack of analysis of intrinsic relationships across different scales, say, the initial input and its down-sampled versions and 2) existence of redundant features. This paper develops a semi-supervised learning mechanism to reduce the above two limitations. To address the first limitation, we propose a multitask model to integrate scene images of different resolutions. For the second limitation, we build a model of sparse feature selection-based manifold regularization (SFSMR) to select the optimal information and preserve the underlying manifold structure of data. SFSMR coordinates the advantages of sparse feature selection and manifold regulation. Finally, we link the multitask model and SFSMR, and propose the semi-supervised learning method to reduce the two limitations. Experimental results report the improvements of the accuracy in scene recognition.

AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification

Aerial scene classification, which aims to automatically label an aerial image with a specific semantic category, is a fundamental problem for understanding high-resolution remote sensing imagery. In recent years, it has become an active task in the remote sensing area, and numerous algorithms have been proposed for this task, including many machine learning and data-driven approaches. However, the existing data sets for aerial scene classification, such as UC-Merced data set and WHU-RS19, contain relatively small sizes, and the results on them are already saturated. This largely limits the development of scene classification algorithms. This paper describes the Aerial Image data set (AID): a large-scale data set for aerial scene classification. The goal of AID is to advance the state of the arts in scene classification of remote sensing images. For creating AID, we collect and annotate more than 10000 aerial scene images. In addition, a comprehensive review of the existing aerial scene classification techniques as well as recent widely used deep learning methods is given. Finally, we provide a performance analysis of typical aerial scene classification and deep learning approaches on AID, which can be served as the baseline results on this benchmark.

Alternatively Constrained Dictionary Learning For Image Superresolution

Dictionaries are crucial in sparse coding-based algorithm for image superresolution. Sparse coding is a typical unsupervised learning method to study the relationship between the patches of high-and low-resolution images. However, most of the sparse coding methods for image superresolution fail to simultaneously consider the geometrical structure of the dictionary and the corresponding coefficients, which may result in noticeable superresolution reconstruction artifacts. In other words, when a low-resolution image and its corresponding high-resolution image are represented in their feature spaces, the two sets of dictionaries and the obtained coefficients have intrinsic links, which has not yet been well studied. Motivated by the development on nonlocal self-similarity and manifold learning, a novel sparse coding method is reported to preserve the geometrical structure of the dictionary and the sparse coefficients of the data. Moreover, the proposed method can preserve the incoherence of dictionary entries and provide the sparse coefficients and learned dictionary from a new perspective, which have both reconstruction and discrimination properties to enhance the learning performance. Furthermore, to utilize the model of the proposed method more effectively for single-image superresolution, this paper also proposes a novel dictionary-pair learning method, which is named as two-stage dictionary training. Extensive experiments are carried out on a large set of images comparing with other popular algorithms for the same purpose, and the results clearly demonstrate the effectiveness of the proposed sparse representation model and the corresponding dictionary learning algorithm.

Double Constrained NMF for Hyperspectral Unmixing

Given only the collected hyperspectral data, unmixing aims at obtaining the latent constituent materials and their corresponding fractional abundances. Recently, many nonnegative matrix factorization (NMF)-based algorithms have been developed to deal with this issue. Considering that the abundances of most materials may be sparse, the sparseness constraint is intuitively introduced into NMF. Although sparse NMF algorithms have achieved advanced performance in unmixing, the result is still susceptible to unstable decomposition and noise corruption. To reduce the aforementioned drawbacks, the structural information of the data is exploited to guide the unmixing. Since similar pixel spectra often imply similar substance constructions, clustering can explicitly characterize this similarity. Through maintaining the structural information during the unmixing, the resulting fractional abundances by the proposed algorithm can well coincide with the real distributions of constituent materials. Moreover, the additional clustering-based regularization term also lessens the interference of noise to some extent. The experimental results on synthetic and real hyperspectral data both illustrate the superiority of the proposed method compared with other state-of-the-art algorithms.