Allan Hollinger

Fast three‐dimensional data compression of hyperspectral imagery using vector quantization with spectral‐feature‐based binary coding

A fast lossy 3-D data compression scheme using vector quantization (VQ) is presented that exploits the spatial and the spectral redundancy in hyperspectral imagery. Hyperspectral imagery may be viewed as a 3-D array of samples in which two dimensions correspond to spatial position and the third to wavelength. Unlike traditional 2-D VQ, where spatial blocks of n3m pixels are taken as vectors, we define one spectrum, corresponding to a profile taken along the wavelength axis, as a vector. This constitution of vectors makes good use of the high corre- lation in the spectral domain and achieves a high compression ratio. It also leads to fast codebook generation and fast codevector matching. A coding scheme for fast vector matching called spectral-feature-based binary coding (SFBBC) is used to encode each spectral vector into a simple and efficient set of binary codes. The generation of the codebook and the matching of codevectors are performed by matching the binary codes produced by the SFBBC. The experiments were carried out using a test hyperspectral data cube from the Compact Airborne Spectro- graphic Imager. Generating a codebook is 39 times faster with the SF- BBC than with conventional VQ, and the data compression is 30 to 40 times faster. Compression ratios greater than 192 : 1 have been achieved with peak signal-to-noise ratios of the reconstructed hyper- spectral sequences exceeding 45.2 dB.

Vector quantization using spectral index-based multiple subcodebooks for hyperspectral data compression

This paper describes a spectral index (SI)-based multiple subcodebook algorithm (MSCA) for lossy hyperspectral data compression. The scene of a hyperspectral dataset to be compressed is delimited into n regions by segmenting its SI image. The spectra in each region have similar spectral characteristics. The dataset is then separated into n subsets, corresponding to the n regions. While keeping the total number of codevectors the same (i.e. the same compression ratio), not just a single codebook, but n smaller and more efficient subcodebooks are generated. Each subcodebook is used to compress the spectra in the corresponding region. With the MSCA, both the codebook generation time (CGT) and coding time (CT) can be improved by a factor of around n at almost no loss of fidelity. Four segmentation methods for delimiting the scene of the data cube were studied. Three hyperspectral vector quantization data compression systems that use the improved techniques were simulated and tested. The simulation results show that the CGT could be reduced by more than three orders of magnitude, while the quality of the codebooks remained good. The overall processing speed of the compression systems could be improved by a factor of around 1000 at an average fidelity penalty of 1.0 dB.

Near lossless data compression onboard a hyperspectral satellite

To deal with the large volume of data produced by hyperspectral sensors, the Canadian Space Agency (CSA) has developed and patented two near lossless data compression algorithms for use onboard a hyperspectral satellite: successive approximation multi-stage vector quantization (SAMVQ) and hierarchical self-organizing cluster vector quantization (HSOCVQ). This paper describes the two compression algorithms and demonstrates their near lossless feature. The compression error introduced by the two compression algorithms was compared with the intrinsic noise of the original data that is caused by the instrument noise and other noise sources such as calibration and atmospheric correction errors. The experimental results showed that the compression error was not larger than the intrinsic noise of the original data when a test data set was compressed at a compression ratio of 20:1. The overall noise in the reconstructed data that contains both the intrinsic noise and the compression error is even smaller than the intrinsic noise when the data is compressed using SAMVQ. A multi-disciplinary user acceptability study has been carried out in order to evaluate the impact of the two compression algorithms on hyperspectral data applications. This paper briefly summarizes the evaluation results of the user acceptability study. A prototype hardware compressor that implements the two compression algorithms has been built using field programmable gate arrays (FPGAs) and benchmarked. The compression ratio and fidelity achieved by the hardware compressor are similar to those obtained by software simulation

Recent Developments in the Hyperspectral Environment and Resource Observer (HERO) Mission

In 1997, the Canadian Space Agency (CSA) and Canadian industry began developing enabling technologies for hyperspectral satellites. Since then, the CSA has conducted mission and payload concept studies in preparation for launch of the first Canadian hyperspectral earth observation satellite. This Canadian hyperspectral remote sensing project is now named the Hyperspectral Environment and Resource Observer (HERO) Mission. In 2005, the Preliminary System Requirement Review (PSRR) and the Phase A (Preliminary Mission Definition) were concluded. Recent developments regarding the payload include an extensive comparison of potential optical designs. The payload uses separate grating spectrometers for the visible near-infrared and short-wave infrared portions of the spectrum. The instrument covers a swath of >30 km, has a ground sampling distance of 30 m, a spectral range of 400-2500 nm, and a spectral sampling interval of 10 nm. Smile and keystone are minimized. Recent developments regarding the mission include requirements simplification, data compression studies, and hyperspectral data simulation capability. In addition, a Prototype Data Processing Chain (PDPC) has been defined for 3 key hyperspectral applications. These are: geological mapping in the arctic environment, dominant species identification for forestry, and leaf area index for estimating foliage cover as well as forecasting crop growth and yield in agriculture.

3D data compression of hyperspectral imagery using vector quantization with NDVI-based multiple codebooks

This paper describes a new vector quantization based algorithm that uses the remote sensing knowledge Normalized Difference Vegetation Index (NDVI) to reduce the codebook generation time (CGT) and coding time (CT). The experimental results showed that it yielded an improvement in both CGT and CT of 14.1 and 14.8 times when the scene of a data set is segmented into 16 classes, while the reconstruction fidelity was almost as same as that by the conventional vector quantization algorithm. The PSNR of the reconstructed data reached 43.31 dB when the compression ratio was of 81:1.

A multidisciplinary user acceptability study of hyperspectral data compressed using an on‐board near lossless vector quantization algorithm

To deal with the extremely high data rate and huge data volume generated on‐board a hyperspectral satellite, the Canadian Space Agency (CSA) has developed two fast on‐board data compression techniques for hyperspectral imagery. The CSA is planning to place a data compressor on‐board a proposed Canadian hyperspectral satellite using these techniques to reduce the requirement for on‐board storage and provide a better match to available downlink capacity. Since the compression techniques are lossy, it is essential to assess the usability of the compressed data and the impact on remote sensing applications. In this paper, 11 hyperspectral data users covering a wide range of application areas and a variety of hyperspectral sensors assessed the usability of the compressed data using their well understood datasets and predefined evaluation criteria. Double blind testing was adopted to eliminate bias in the evaluation. Four users had ground truth available. They qualitatively and quantitatively compared the products derived from the compressed data to the ground truth at compression ratios from 10 : 1 to 50 : 1 to examine whether the compressed data provided the same amount of information as the original for their applications. They accepted all the compressed data. The users who did not have ground truths available evaluated the compression impact by comparing the products derived from the compressed data with those derived from the original data. They accepted most of the compressed data.

Study of real-time lossless data compression for hyperspectral imagery

This paper describes a study of real-time lossless data compression of hyperspectral imagery using prediction and entropy encoding. The main effort in developing a compression system, is to study predictors that can yield the best reduction of entropy and can be easily implemented in real-time. The Consultative Committee for Space Data System (CCSDS) recommended lossless algorithm is selected as the entropy encoder. Four predictor schemes have been selected for study. Three typical hyperspectral data sets acquired by the Airborne Visible/Infrared imaging Spectrometer (AVIRIS) and three acquired by the Compact Airborne Spectrographic Imager (casi) were used as test data. A lossless compression system with different predictors has been simulated and tested with the test data.

3D Data Compression Systems Based on Vector Quantization for Reducing the Data Rate of Hyperspectral Imagery

The next generation of satellite-based remote sensing instruments will produce an unprecedented volume of data. Imaging spectrometers, also known as hyperspectral imagers, are prime examples. They collect image data in hundreds of spectral bands simultaneously from the near ultraviolet through the short wave infrared, and are capable of providing direct identification of surface materials. A schematic diagram illustrating the concept of an imaging spectrometer is given in Fig.11. The volume and complexity of data produced by these instruments offers a significant challenge to downlink transmission and traditional image analysis methods. Since they produce 3-dimensional (3D) data cubes in which two dimensions correspond to spatial position and the third to wavelength, raw data rates can easily exceed the available downlink capacity or on-board storage capacity. Often, therefore, a portion of the data collected on board is discarded before transmission. This data reduction process may involve: 1) reducing the duty cycle, 2) reducing the spatial or spectral resolution, and 3) reducing the spatial or spectral range. Obviously, in such cases large amounts of information are lost.

Evaluation and comparison of JPEG2000 and vector quantization based onboard data compression algorithm for hyperspectral imagery

This paper evaluates and compares JPEG 2000 and Successive Approximation Multi-stage Vector Quantization (SAMVQ) compression algorithms for hyperspectral imagery. PSNR was used to measure the statistical performance of the two compression algorithms. The SAMVQ outperforms JPEG 2000 by 17 dB of PSNR at the same compression ratios. The preservation of both spatial and spectral features was evaluated qualitatively and quantitatively. The SAMVQ outperforms JPEG 2000 in both spatial and spectral features preservation.

Imaging Spectrometry As A Tool For Botanical Mapping

During the summers of 1985 and 1986 a Programmable Multispectral Imager (PMI), also known as the Fluorescence Line Imager (FLI), has been used to collect airborne data over a number of forested targets in Canada and the United States. The sites were selected on the basis of suspected localized vegetation stress due to possible excess metal uptake or reported regional forest decline due to suspected acid deposition damage. This paper focuses on the characteristics of the spectral/image data available from this new sensor along with results of preliminary analysis of some of these data. Stable pixel to pixel vegetation spectral properties provide a verification of sensor calibration methods. Comparison of FLI vegetation spectra with ground-based spectra of vegetation samples show good correspondence for a variety of species studied, including spectral properties of the red edge.