Rob Heylen

A Review of Nonlinear Hyperspectral Unmixing Methods

In hyperspectral unmixing, the prevalent model used is the linear mixing model, and a large variety of techniques based on this model has been proposed to obtain endmembers and their abundances in hyperspectral imagery. However, it has been known for some time that nonlinear spectral mixing effects can be a crucial component in many real-world scenarios, such as planetary remote sensing, intimate mineral mixtures, vegetation canopies, or urban scenes. While several nonlinear mixing models have been proposed decades ago, only recently there has been a proliferation of nonlinear unmixing models and techniques in the signal processing literature. This paper aims to give an historical overview of the majority of nonlinear mixing models and nonlinear unmixing methods, and to explain some of the more popular techniques in detail. The main models and techniques treated are bilinear models, models for intimate mineral mixtures, radiosity-based approaches, ray tracing, neural networks, kernel methods, support vector machine techniques, manifold learning methods, piece-wise linear techniques, and detection methods for nonlinearity. Furthermore, we provide an overview of several recent developments in the nonlinear unmixing literature that do not belong into any of these categories.

Fully Constrained Least Squares Spectral Unmixing by Simplex Projection

We present a new algorithm for linear spectral mixture analysis, which is capable of supervised unmixing of hyperspectral data while respecting the constraints on the abundance coefficients. This simplex-projection unmixing algorithm is based upon the equivalence of the fully constrained least squares problem and the problem of projecting a point onto a simplex. We introduce several geometrical properties of high-dimensional simplices and combine them to yield a recursive algorithm for solving the simplex-projection problem. A concrete implementation of the algorithm for large data sets is provided, and the algorithm is benchmarked against well-known fully constrained least squares unmixing (FCLSU) techniques, on both artificial data sets and real hyperspectral data collected over the Cuprite mining region. Unlike previous algorithms for FCLSU, the presented algorithm possesses no optimization steps and is completely analytical, severely reducing the required processing power.

Non-Linear Spectral Unmixing by Geodesic Simplex Volume Maximization

Spectral mixtures observed in hyperspectral imagery often display nonlinear mixing effects. Since most traditional unmixing techniques are based upon the linear mixing model, they perform poorly in finding the correct endmembers and their abundances in the case of nonlinear spectral mixing. In this paper, we present an unmixing algorithm that is capable of extracting endmembers and determining their abundances in hyperspectral imagery under nonlinear mixing assumptions. The algorithm is based upon simplex volume maximization, and uses shortest-path distances in a nearest-neighbor graph in spectral space, hereby respecting the nontrivial geometry of the data manifold in the case of nonlinearly mixed pixels. We demonstrate the algorithm on an artificial data set, the AVIRIS Cuprite data set, and a hyperspectral image of a heathland area in Belgium.

A Multilinear Mixing Model for Nonlinear Spectral Unmixing

In hyperspectral unmixing, bilinear and linear-quadratic models have become popular recently, and also the polynomial postnonlinear model shows promising results. These models do not consider endmember interactions involving more than two endmembers, although such interactions might compose a nontrivial part of the observed spectrum in scenarios involving bright materials and complex geometrical structures, such as vegetation and intimate mixtures. In this paper, we present an extension of these models to include an infinite number of interactions. Several technical problems, such as divergence of the resulting series, can be avoided by introducing an optical interaction probability, which becomes the only free parameter of the model in addition to the abundances. We present an unmixing strategy based on this multilinear mixing (MLM) model; present comparisons with the bilinear models and the Hapke model for intimate mixing; and show that, in several scenarios, the MLM model obtains superior results.

Hyperspectral Intrinsic Dimensionality Estimation With Nearest-Neighbor Distance Ratios

The first task to be performed in most hyperspectral unmixing chains is the estimation of the number of endmembers. Several techniques for this problem have already been proposed, but the class of fractal techniques for intrinsic dimensionality estimation is often overlooked. In this paper, we study an intrinsic dimensionality estimation technique based on the known scaling behavior of nearest-neighbor distance ratios, and its performance on the spectral unmixing problem. We present the relation between intrinsic manifold dimensionality and the number of endmembers in a mixing model, and investigate the effects of denoising and the statistics on the algorithm. The algorithm is compared with several alternative methods, such as Hysime, virtual dimensionality, and several fractal-dimension based techniques, on both artificial and real data sets. Robust behavior in the presence of noise, and independence of the spectral dimensionality, is demonstrated. Furthermore, due to its construction, the algorithm can be used for non-linear mixing models as well.

Nonlinear Spectral Unmixing With a Linear Mixture of Intimate Mixtures Model

We present a new model for nonlinear spectral mixing observed in hyperspectral imagery and demonstrate how this model can be used for unmixing and obtaining abundance maps. The model is based on the idea that a single pixel can contain several spatially segregated areas containing different mineral mixtures and fuses Hapkes radiative transfer model for intimate mineral mixtures with the traditional linear mixing model. The resulting model allows great flexibility for generating spectra, provides abundance coefficients in terms of total relative ground cover for each endmember, and can be reduced to several other nonlinear mixing models by an appropriate choice of the parameters. Experiments on laboratory mineral mixtures and real hyperspectral imagery show reduced reconstruction errors and more accurate abundance coefficients compared with the linear mixing model or the recently introduced multimixture pixel model. Moreover, the reconstruction error improvement can be used as a per-pixel measure of the size of the intimate mixing component.

Detecting the Adjacency Effect in Hyperspectral Imagery With Spectral Unmixing Techniques

The adjacency effect is an interesting phenomenon characterized by the occurrence of path interferences between the reflectances coming from different ground-cover materials. The effect is caused by atmospheric scattering, hence a typical approach to its detection has been the modeling of radiation transfer and spectral correspondence at particular wavelengths. In this paper, we investigate the detection of adjacency effects as being a general unmixing problem. This means that we opt to use spectral unmixing to separate the true signature of a pixel from the background scatter reflected from its adjacent neighborhood. To account for different types of atmospheric scattering, we consider several unmixing methods. These include the established linear- and a recently studied generalized bilinear model, as well as a more data-driven unmixing that could implicitly address nonlinearities not covered by the first mentioned approaches. We evaluate these unmixing models by comparing their results with those obtained from a specialized treatment of the adjacency effect in turbid waters surrounded by vegetated land. This comparison is demonstrated on real data acquired under varying atmospheric conditions.

Calculation of Geodesic Distances in Nonlinear Mixing Models: Application to the Generalized Bilinear Model

Recently, several nonlinear techniques have been proposed in hyperspectral image processing for classification and unmixing applications. A popular data-driven approach for treating nonlinear problems employs the geodesic distances on the data manifold as property of interest. These geodesic distances are approximated by the shortest path distances in a nearest neighbor graph constructed in the data cloud. Although this approach often works well in practical applications, the graph-based approximation of these geodesic distances often fails to capture correctly the true nonlinear structure of the manifold, causing deviations in the subsequent algorithms. On the other hand, several model-based nonlinear techniques have been introduced as well and have the advantage that one can, in theory, calculate the geodesic distances analytically. In this letter, we demonstrate how one can calculate the true geodesics, and their lengths, on any manifold induced by a nonlinear hyperspectral mixing model. We introduce the required techniques from differential geometry, show how the constraints on the abundances can be integrated in these techniques, and present a numerical method for finding a solution of the geodesic equations. We demonstrate this technique on the recently developed generalized bilinear model, which is a flexible model for the nonlinearities introduced by secondary reflections. As an application of the technique, we demonstrate that multidimensional scaling applied to these geodesic distances can be used as a preprocessing step to linear unmixing, yielding better unmixing results on nonlinear data when compared to principal component analysis and outperforming ISOMAP.

Fusion of Hyperspectral and Multispectral Images Using Spectral Unmixing and Sparse Coding

Unlike multispectral (MSI) and panchromatic (PAN) images, generally the spatial resolution of hyperspectral images (HSI) is limited, due to sensor limitations. In many applications, HSI with a high spectral as well as spatial resolution are required. In this paper, a new method for spatial resolution enhancement of a HSI using spectral unmixing and sparse coding (SUSC) is introduced. The proposed method fuses high spectral resolution features from the HSI with high spatial resolution features from an MSI of the same scene. Endmembers are extracted from the HSI by spectral unmixing, and the exact location of the endmembers is obtained from the MSI. This fusion process by using spectral unmixing is formulated as an ill-posed inverse problem which requires a regularization term in order to convert it into a well-posed inverse problem. As a regularizer, we employ sparse coding (SC), for which a dictionary is constructed using high spatial resolution MSI or PAN images from unrelated scenes. The proposed algorithm is applied to real Hyperion and ROSIS datasets. Compared with other state-of-the-art algorithms based on pansharpening, spectral unmixing, and SC methods, the proposed method is shown to significantly increase the spatial resolution while perserving the spectral content of the HSI.

Multidimensional Pixel Purity Index for Convex Hull Estimation and Endmember Extraction

One of the earliest endmember extraction algorithms employed in hyperspectral image processing is the pixel purity index (PPI) algorithm. This algorithm is still popular today but suffers from several drawbacks, such as a large computational cost. Many recent papers focus on improving the speed of the PPI algorithm with high-performance computing or combinatorial methods. In this paper, we present a computationally efficient way of calculating the PPI scores, based on the geometrical interpretation of the PPI sampling process. We first demonstrate the equivalence with Monte Carlo sampling of the polar cones of the convex hull of the data set. Next, we introduce a more efficient sampling method, where we use higher dimensional subspaces to sample these polar cones instead of 1-D skewers. The resulting algorithm can be used to quickly estimate the most important convex hull vertices of the data set, determine the corresponding PPI scores, and produce a list of endmember candidates. An unweighted version of this algorithm is introduced as well, which is simpler to implement, has a higher computational performance, and yields similar endmembers. If the subspace dimension is chosen to be one, both algorithms reduce to the PPI algorithm. We demonstrate the properties of these algorithms, such as convergence speed and accuracy, on artificial and real hyperspectral data and show that the results correspond to those obtained with PPI. The proposed algorithms, however, are up to three orders of magnitude faster and can generate representative PPI scores in less than a second on real hyperspectral data sets.