Shen-En Qian

Denoising of Hyperspectral Imagery Using Principal Component Analysis and Wavelet Shrinkage

In this paper, a new denoising method is proposed for hyperspectral data cubes that already have a reasonably good signal-to-noise ratio (SNR) (such as 600 : 1). Given this level of the SNR, the noise level of the data cubes is relatively low. The conventional image denoising methods are likely to remove the fine features of the data cubes during the denoising process. We propose to decorrelate the image information of hyperspectral data cubes from the noise by using principal component analysis (PCA) and removing the noise in the low-energy PCA output channels. The first PCA output channels contain a majority of the total energy of a data cube, and the rest PCA output channels contain a small amount of energy. It is believed that the low-energy channels also contain a large amount of noise. Removing noise in the low-energy PCA output channels will not harm the fine features of the data cubes. A 2-D bivariate wavelet thresholding method is used to remove the noise for low-energy PCA channels, and a 1-D dual-tree complex wavelet transform denoising method is used to remove the noise of the spectrum of each pixel of the data cube. Experimental results demonstrated that the proposed denoising method produces better denoising results than other denoising methods published in the literature.

Noise reduction of hyperspectral imagery using hybrid spatial-spectral derivative-domain wavelet shrinkage

In this paper, a new noise reduction algorithm is introduced and applied to the problem of denoising hyperspectral imagery. This algorithm resorts to the spectral derivative domain, where the noise level is elevated, and benefits from the dissimilarity of the signal regularity in the spatial and the spectral dimensions of hyperspectral images. The performance of the new algorithm is tested on two different hyperspectral datacubes: an Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) datacube that is acquired in a vegetation-dominated site and a simulated AVIRIS datacube that simulates a geological site. The new algorithm provides signal-to-noise-ratio improvement up to 84.44% and 98.35% in the first and the second datacubes, respectively.

Fast three‐dimensional data compression of hyperspectral imagery using vector quantization with spectral‐feature‐based binary coding

A fast lossy 3-D data compression scheme using vector quantization (VQ) is presented that exploits the spatial and the spectral redundancy in hyperspectral imagery. Hyperspectral imagery may be viewed as a 3-D array of samples in which two dimensions correspond to spatial position and the third to wavelength. Unlike traditional 2-D VQ, where spatial blocks of n3m pixels are taken as vectors, we define one spectrum, corresponding to a profile taken along the wavelength axis, as a vector. This constitution of vectors makes good use of the high corre- lation in the spectral domain and achieves a high compression ratio. It also leads to fast codebook generation and fast codevector matching. A coding scheme for fast vector matching called spectral-feature-based binary coding (SFBBC) is used to encode each spectral vector into a simple and efficient set of binary codes. The generation of the codebook and the matching of codevectors are performed by matching the binary codes produced by the SFBBC. The experiments were carried out using a test hyperspectral data cube from the Compact Airborne Spectro- graphic Imager. Generating a codebook is 39 times faster with the SF- BBC than with conventional VQ, and the data compression is 30 to 40 times faster. Compression ratios greater than 192 : 1 have been achieved with peak signal-to-noise ratios of the reconstructed hyper- spectral sequences exceeding 45.2 dB.

Hyperspectral data compression using a fast vector quantization algorithm

A fast vector quantization algorithm for data compression of hyperspectral imagery is proposed in this paper. It makes use of the fact that in the full search of the generalized Lloyd algorithm (GLA) a training vector does not require a search to find the minimum distance partition if its distance to the partition is improved in the current iteration compared to that of the previous iteration. The proposed method has the advantage of being simple, producing a large computation time saving and yielding compression fidelity as good as the GLA. Four hyperspectral data cubes covering a wide variety of scene types were tested. The loss of spectral information due to compression was evaluated using the spectral angle mapper and a remote sensing application.

Vector quantization using spectral index-based multiple subcodebooks for hyperspectral data compression

This paper describes a spectral index (SI)-based multiple subcodebook algorithm (MSCA) for lossy hyperspectral data compression. The scene of a hyperspectral dataset to be compressed is delimited into n regions by segmenting its SI image. The spectra in each region have similar spectral characteristics. The dataset is then separated into n subsets, corresponding to the n regions. While keeping the total number of codevectors the same (i.e. the same compression ratio), not just a single codebook, but n smaller and more efficient subcodebooks are generated. Each subcodebook is used to compress the spectra in the corresponding region. With the MSCA, both the codebook generation time (CGT) and coding time (CT) can be improved by a factor of around n at almost no loss of fidelity. Four segmentation methods for delimiting the scene of the data cube were studied. Three hyperspectral vector quantization data compression systems that use the improved techniques were simulated and tested. The simulation results show that the CGT could be reduced by more than three orders of magnitude, while the quality of the codebooks remained good. The overall processing speed of the compression systems could be improved by a factor of around 1000 at an average fidelity penalty of 1.0 dB.

Near lossless data compression onboard a hyperspectral satellite

To deal with the large volume of data produced by hyperspectral sensors, the Canadian Space Agency (CSA) has developed and patented two near lossless data compression algorithms for use onboard a hyperspectral satellite: successive approximation multi-stage vector quantization (SAMVQ) and hierarchical self-organizing cluster vector quantization (HSOCVQ). This paper describes the two compression algorithms and demonstrates their near lossless feature. The compression error introduced by the two compression algorithms was compared with the intrinsic noise of the original data that is caused by the instrument noise and other noise sources such as calibration and atmospheric correction errors. The experimental results showed that the compression error was not larger than the intrinsic noise of the original data when a test data set was compressed at a compression ratio of 20:1. The overall noise in the reconstructed data that contains both the intrinsic noise and the compression error is even smaller than the intrinsic noise when the data is compressed using SAMVQ. A multi-disciplinary user acceptability study has been carried out in order to evaluate the impact of the two compression algorithms on hyperspectral data applications. This paper briefly summarizes the evaluation results of the user acceptability study. A prototype hardware compressor that implements the two compression algorithms has been built using field programmable gate arrays (FPGAs) and benchmarked. The compression ratio and fidelity achieved by the hardware compressor are similar to those obtained by software simulation

Recent Developments in the Hyperspectral Environment and Resource Observer (HERO) Mission

In 1997, the Canadian Space Agency (CSA) and Canadian industry began developing enabling technologies for hyperspectral satellites. Since then, the CSA has conducted mission and payload concept studies in preparation for launch of the first Canadian hyperspectral earth observation satellite. This Canadian hyperspectral remote sensing project is now named the Hyperspectral Environment and Resource Observer (HERO) Mission. In 2005, the Preliminary System Requirement Review (PSRR) and the Phase A (Preliminary Mission Definition) were concluded. Recent developments regarding the payload include an extensive comparison of potential optical designs. The payload uses separate grating spectrometers for the visible near-infrared and short-wave infrared portions of the spectrum. The instrument covers a swath of >30 km, has a ground sampling distance of 30 m, a spectral range of 400-2500 nm, and a spectral sampling interval of 10 nm. Smile and keystone are minimized. Recent developments regarding the mission include requirements simplification, data compression studies, and hyperspectral data simulation capability. In addition, a Prototype Data Processing Chain (PDPC) has been defined for 3 key hyperspectral applications. These are: geological mapping in the arctic environment, dominant species identification for forestry, and leaf area index for estimating foliage cover as well as forecasting crop growth and yield in agriculture.

A new nonlinear dimensionality reduction method with application to hyperspectral image analysis

In this paper, we propose a new nonlinear dimensionality reduction method by combining Locally Linear Embedding (LLE) with Laplacian Eigenmaps, and apply it to hyperspectral data. LLE projects high dimensional data into a low-dimensional Euclidean space while preserving local topological structures. However, it may not keep the relative distance between data points in the dimension-reduced space as in the original data space. Laplacian Eigenmaps, on the other hand, can preserve the locality characteristics in terms of distances between data points. By combining these two methods, a better locality preserving method is created for nonlinear dimensionality reduction. Experiments conducted in this paper confirms the feasibility of the new method for hyperspectral dimensionality reduction. The new method can find the same number of endmembers as PCA and LLE, but it is more accurate than them in terms of endmember location. Moreover, the new method is better than Laplacian Eigenmap alone because it identifies more pure mineral endmembers.

3D data compression of hyperspectral imagery using vector quantization with NDVI-based multiple codebooks

This paper describes a new vector quantization based algorithm that uses the remote sensing knowledge Normalized Difference Vegetation Index (NDVI) to reduce the codebook generation time (CGT) and coding time (CT). The experimental results showed that it yielded an improvement in both CGT and CT of 14.1 and 14.8 times when the scene of a data set is segmented into 16 classes, while the reconstruction fidelity was almost as same as that by the conventional vector quantization algorithm. The PSNR of the reconstructed data reached 43.31 dB when the compression ratio was of 81:1.

A multidisciplinary user acceptability study of hyperspectral data compressed using an on‐board near lossless vector quantization algorithm

To deal with the extremely high data rate and huge data volume generated on‐board a hyperspectral satellite, the Canadian Space Agency (CSA) has developed two fast on‐board data compression techniques for hyperspectral imagery. The CSA is planning to place a data compressor on‐board a proposed Canadian hyperspectral satellite using these techniques to reduce the requirement for on‐board storage and provide a better match to available downlink capacity. Since the compression techniques are lossy, it is essential to assess the usability of the compressed data and the impact on remote sensing applications. In this paper, 11 hyperspectral data users covering a wide range of application areas and a variety of hyperspectral sensors assessed the usability of the compressed data using their well understood datasets and predefined evaluation criteria. Double blind testing was adopted to eliminate bias in the evaluation. Four users had ground truth available. They qualitatively and quantitatively compared the products derived from the compressed data to the ground truth at compression ratios from 10 : 1 to 50 : 1 to examine whether the compressed data provided the same amount of information as the original for their applications. They accepted all the compressed data. The users who did not have ground truths available evaluated the compression impact by comparing the products derived from the compressed data with those derived from the original data. They accepted most of the compressed data.