Aaron Kiely

Exploiting Calibration-Induced Artifacts in Lossless Compression of Hyperspectral Imagery

Algorithms for compression of hyperspectral data are commonly evaluated on a readily available collection of Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) images. These images are the end product of processing raw data from the instrument, and their sample value distributions contain artificial regularities that are introduced by the conversion of raw data values to radiance units. It is shown that some of the best reported lossless compression results for these images are achieved by algorithms that significantly exploit these artifacts. This fact has not been widely reported and may not be widely recognized. Compression performance comparisons involving such algorithms and these standard AVIRIS images can be misleading if they are extrapolated to images that lack such artifacts, such as unprocessed hyperspectral images. In fact, two of these algorithms are shown to achieve rather unremarkable compression performance on a set of more recent AVIRIS images that do not contain appreciable calibration-induced artifacts. This newer set of AVIRIS images, which contains both calibrated and raw images, is made available for compression experiments. To underscore the potential impact of exploiting calibration-induced artifacts in the standard AVIRIS data sets, a compression algorithm is presented that achieves noticeably smaller compressed sizes for these data sets than is reported for any other algorithm.

The new CCSDS image compression recommendation

The consultative committee for space data systems (CCSDS) data compression working group has recently adopted a recommendation for image data compression, with a final release expected in 2005. The algorithm adopted in the recommendation consists of a two-dimensional discrete wavelet transform of the image, followed by progressive bit-plane coding of the transformed data. The algorithm can provide both lossless and lossy compression, and allows a user to directly control the compressed data volume or the fidelity with which the wavelet-transformed data can be reconstructed. The algorithm is suitable for both frame-based image data and scan-based sensor data, and has applications for near-Earth and deep-space missions. The standard will be accompanied by free software sources on a future Web site. An application-specific integrated circuit (ASIC) implementation of the compressor is currently under development. This paper describes the compression algorithm along with the requirements that drove the selection of the algorithm. Performance results and comparisons with other compressors are given for a test set of space images

Optimized Autonomous Space In-Situ Sensor Web for Volcano Monitoring

In response to NASAs announced requirement for Earth hazard monitoring sensor-web technology, a multidisciplinary team involving sensor-network experts (Washington State University), space scientists (JPL), and Earth scientists (USGS Cascade Volcano Observatory (CVO)), have developed a prototype of dynamic and scalable hazard monitoring sensor-web and applied it to volcano monitoring. The combined Optimized Autonomous Space - In-situ Sensor-web (OASIS) has two-way communication capability between ground and space assets, uses both space and ground data for optimal allocation of limited bandwidth resources on the ground, and uses smart management of competing demands for limited space assets. It also enables scalability and seamless infusion of future space and in-situ assets into the sensor-web. The space and in-situ control components of the system are integrated such that each element is capable of autonomously tasking the other. The ground in-situ was deployed into the craters and around the flanks of Mount St. Helens in July 2009, and linked to the command and control of the Earth Observing One (EO-1) satellite.

An efficient variable length coding scheme for an IID source

In this article we examine a scheme that uses two alternating Huffman codes to encode a discrete independent and identically distributed source with a dominant symbol. One Huffman code encodes the length of runs of the dominant symbol, the other encodes the remaining symbols. We call this combined strategy alternating runlength Huffman (ARH) coding. This is a popular scheme, used for example in the efficient pyramid image coder (EPIC) subband coding algorithm. Since the runlengths of the dominant symbol are geometrically distributed, they can be encoded using the Huffman codes identified by Golomb (1966) and later generalized by Gallager and Van Voorhis (1975). This runlength encoding allows the most likely symbol to be encoded using less than one bit per sample, providing a simple method for overcoming a drawback of prefix codes-that the redundancy approaches one as the largest symbol probability P approaches one. For ARH coding, the redundancy approaches zero as P approaches one. Comparing the average code rate of ARH with direct Huffman coding we find that: 1. If P<1/3, ARH is less efficient than Huffman coding. 2. If 1/3/spl les/P<2/5, ARH is less than or equally efficient as Huffman coding, depending on the source distribution. 3. If 2/5/spl les/P/spl les/0.618, ARH and Huffman coding are equally efficient. 4. If P>0.618, ARH is more efficient than Huffman coding. We give examples of applying ARH coding to some specific sources.

GPU lossless hyperspectral data compression system for space applications

On-board lossless hyperspectral data compression reduces data volume in order to meet NASA and DoD limited downlink capabilities. At JPL, a novel, adaptive and predictive technique for lossless compression of hyperspectral data, named the Fast Lossless (FL) algorithm, was recently developed. This technique uses an adaptive filtering method and achieves state-of-the-art performance in both compression effectiveness and low complexity. Because of its outstanding performance and suitability for real-time onboard hardware implementation, the FL compressor is being formalized as the emerging CCSDS Standard for Lossless Multispectral & Hyperspectral image compression. The FL compressor is well-suited for parallel hardware implementation. A GPU hardware implementation was developed for FL targeting the current state-of-the-art GPUs from NVIDIA®. The GPU implementation on a NVIDIA® GeForce® GTX 580 achieves a throughput performance of 583.08 Mbits/sec (44.85 MSamples/sec) and an acceleration of at least 6 times a software implementation running on a 3.47 GHz single core Intel® Xeon™ processor. This paper describes the design and implementation of the FL algorithm on the GPU. The massively parallel implementation will provide in the future a fast and practical real-time solution for airborne and space applications.

Adaptive Linear Filtering Compression on realtime sensor networks

We present a lightweight lossless compression algorithm for realtime sensor networks. Our proposed Adaptive Linear Filtering Compression (ALFC) algorithm performs predictive compression, using adaptive linear filtering to predict sample values followed by entropy coding of prediction residuals, encoding a variable number of samples into fixed-length packets. Adaptive prediction eliminates the need to determine prediction coefficients a priori and, more importantly, allows compression to dynamically adjust to a changing source. The algorithm requires only integer arithmetic operations and thus is compatible with sensor platforms that do not support floating-point operations. Significant robustness to packets losses is provided by including small but sufficient overhead data to allow samples in each packet to be independently decoded. Real-world evaluations on seismic data from a wireless sensor network testbed show that ALFC provides more effective compression and uses less resources than some other lossless compression approaches such as S-LZW. Experiments in a multi-hop sensor network also show that ALFC can significantly improve raw data throughput and energy efficiency.

Performance impact of parameter tuning on the CCSDS-123 lossless multi- and hyperspectral image compression standard

Abstract Multi-spectral and hyperspectral image data payloads have large size and may be challenging to download from remote sensors. To alleviate this problem, such images can be effectively compressed using specially designed algorithms. The new CCSDS-123 standard has been developed to address onboard lossless coding of multi-spectral and hyperspectral images. The standard is based on the fast lossless algorithm, which is composed of a causal context-based prediction stage and an entropy-coding stage that utilizes Golomb power-of-two codes. Several parts of each of these two stages have adjustable parameters. CCSDS-123 provides satisfactory performance for a wide set of imagery acquired by various sensors; but end-users of a CCSDS-123 implementation may require assistance to select a suitable combination of parameters for a specific application scenario. To assist end-users, this paper investigates the performance of CCSDS-123 under different parameter combinations and addresses the selection of an adequate combination given a specific sensor. Experimental results suggest that prediction parameters have a greater impact on the compression performance than entropy-coding parameters.

Spectral Ringing Artifacts in Hyperspectral Image Data Compression

When using 3-D wavelet transforms for hyperspectral image compression, systematic variations in signal level of different spectral bands can cause widely-varying mean values in spatial planes of spatially low-pass subbands. Failing to account for this phenomenon can have detrimental effects on image compression, including reduced effectiveness in compressing spatially low-pass subband data, and biases in some reconstructed spectral bands.

Applying Radiation Hardening by Software to Fast Lossless compression prediction on FPGAs

As scientists endeavor to learn more about the worlds ecosystems, engineers are pushed to develop more sophisticated instruments. With these advancements comes an increase in the amount of data generated. For satellite based instruments the additional data requires sufficient bandwidth be available to transmit the data. Alternatively, compression algorithms can be employed to reduce the bandwidth requirements. This work is motivated by the proposed HyspIRI mission, which includes two imaging spectrometers measuring from visible to short wave infrared (VSWIR) and thermal infrared (TIR) that saturate the projected bandwidth allocations. We present a novel investigation into the capability of using FPGAs integrated with embedded PowerPC processors to adequately perform the predictor function of the Fast Lossless (FL) compression algorithm for multispectral and hyperspectral imagery. Furthermore, our design includes a multi-PowerPC implementation which incorporates recently developed Radiation Hardening by Software (RHBSW) techniques to provide software-based fault tolerance to commercial FPGA devices. Our results show low performance overhead (4-8%) while achieving a speedup of 1.97× when utilizing both PowerPCs. Finally, the evaluation of the proposed system includes resource utilization, performance metrics, and an analysis of the vulnerability to Single Event Upsets (SEU) through the use of a hardware based fault injector.

Error containment in compressed data using sync markers

We examine a specific strategy of using sync markers for error containment in compressed data, using a model that separates the data compression and error containment stages.