David S. Kittle

Compressive Coded Aperture Spectral Imaging: An Introduction

Imaging spectroscopy involves the sensing of a large amount of spatial information across a multitude of wavelengths. Conventional approaches to hyperspectral sensing scan adjacent zones of the underlying spectral scene and merge the results to construct a spectral data cube. Push broom spectral imaging sensors, for instance, capture a spectral cube with one focal plane array (FPA) measurement per spatial line of the scene [1], [2]. Spectrometers based on optical bandpass filters sequentially scan the scene by tuning the bandpass filters in steps. The disadvantage of these techniques is that they require scanning a number of zones linearly in proportion to the desired spatial and spectral resolution. This article surveys compressive coded aperture spectral imagers, also known as coded aperture snapshot spectral imagers (CASSI) [1], [3], [4], which naturally embody the principles of compressive sensing (CS) [5], [6]. The remarkable advantage of CASSI is that the entire data cube is sensed with just a few FPA measurements and, in some cases, with as little as a single FPA shot.

Multiframe image estimation for coded aperture snapshot spectral imagers

A coded aperture snapshot spectral imager (CASSI) estimates the three-dimensional spatiospectral data cube from a snapshot two-dimensional coded projection, assuming that the scene is spatially and spectrally sparse. For less spectrally sparse scenes, we show that the use of multiple nondegenerate snapshots can make data cube recovery less ill-posed, yielding improved spatial and spectral reconstruction fidelity. Additionally, data acquisition can be easily scaled to meet the time/resolution requirements of the scene with little modification or extension of the original CASSI hardware. A multiframe reconstruction of a 640 × 480 × 53 voxel datacube with 450-650 nm white-light illumination of a scene reveals substantial improvement in the reconstruction fidelity, with limited increase in acquisition and reconstruction time.

Multiscale gigapixel photography

Pixel count is the ratio of the solid angle within a camera’s field of view to the solid angle covered by a single detector element. Because the size of the smallest resolvable pixel is proportional to aperture diameter and the maximum field of view is scale independent, the diffraction-limited pixel count is proportional to aperture area. At present, digital cameras operate near the fundamental limit of 1–10 megapixels for millimetre-scale apertures, but few approach the corresponding limits of 1–100 gigapixels for centimetre-scale apertures. Barriers to high-pixel-count imaging include scale-dependent geometric aberrations, the cost and complexity of gigapixel sensor arrays, and the computational and communications challenge of gigapixel image management. Here we describe the AWARE-2 camera, which uses a 16-mm entrance aperture to capture snapshot, one-gigapixel images at three frames per minute. AWARE-2 uses a parallel array of microcameras to reduce the problems of gigapixel imaging to those of megapixel imaging, which are more tractable. In cameras of conventional design, lens speed and field of view decrease as lens scale increases, but with the experimental system described here we confirm previous theoretical results suggesting that lens speed and field of view can be scale independent in microcamera-based imagers resolving up to 50 gigapixels. Ubiquitous gigapixel cameras may transform the central challenge of photography from the question of where to point the camera to that of how to mine the data.

Coded aperture compressive temporal imaging

We use mechanical translation of a coded aperture for code division multiple access compression of video. We discuss the compressed videos temporal resolution and present experimental results for reconstructions of > 10 frames of temporal data per coded snapshot.

3D imaging spectroscopy for measuring hyperspectral patterns on solid objects

Sophisticated methods for true spectral rendering have been developed in computer graphics to produce highly accurate images. In addition to traditional applications in visualizing appearance, such methods have potential applications in many areas of scientific study. In particular, we are motivated by the application of studying avian vision and appearance. An obstacle to using graphics in this application is the lack of reliable input data. We introduce an end-to-end measurement system for capturing spectral data on 3D objects. We present the modification of a recently developed hyperspectral imager to make it suitable for acquiring such data in a wide spectral range at high spectral and spatial resolution. We capture four megapixel images, with data at each pixel from the near-ultraviolet (359 nm) to near-infrared (1,003 nm) at 12 nm spectral resolution. We fully characterize the imaging system, and document its accuracy. This imager is integrated into a 3D scanning system to enable the measurement of the diffuse spectral reflectance and fluorescence of specimens. We demonstrate the use of this measurement system in the study of the interplay between the visual capabilities and appearance of birds. We show further the use of the system in gaining insight into artifacts from geology and cultural heritage.

Coded Hyperspectral Imaging and Blind Compressive Sensing

Blind compressive sensing (CS) is considered for reconstruction of hyperspectral data imaged by a coded aperture camera. The measurements are manifested as a superposition of the coded wavelength-dependent data, with the ambient three-dimensional hyperspectral datacube mapped to a two-dimensional measurement. The hyperspectral datacube is recovered using a Bayesian implementation of blind CS. Several demonstration experiments are presented, including measurements performed using a coded aperture snapshot spectral imager (CASSI) camera. The proposed approach is capable of efficiently reconstructing large hyperspectral datacubes. Comparisons are made between the proposed algorithm and other techniques employed in compressive sensing, dictionary learning, and matrix factorization.

Joint segmentation and reconstruction of hyperspectral data with compressed measurements

This work describes numerical methods for the joint reconstruction and segmentation of spectral images taken by compressive sensing coded aperture snapshot spectral imagers (CASSI). In a snapshot, a CASSI captures a two-dimensional (2D) array of measurements that is an encoded representation of both spectral information and 2D spatial information of a scene, resulting in significant savings in acquisition time and data storage. The reconstruction process decodes the 2D measurements to render a three-dimensional spatio-spectral estimate of the scene and is therefore an indispensable component of the spectral imager. In this study, we seek a particular form of the compressed sensing solution that assumes spectrally homogeneous segments in the two spatial dimensions, and greatly reduces the number of unknowns, often turning the underdetermined reconstruction problem into one that is overdetermined. Numerical tests are reported on both simulated and real data representing compressed measurements.

Design and fabrication of an ultraviolet-visible coded aperture snapshot spectral imager

We describe the design and performance of a coded aperture spectral imager with a wide spectral range of 320 to 700 nm over 87 channels and 1988-by-1988 pixels of spatial resolution. A custom-designed relay lens was designed and built for the system, including a dispersive prism element in the collimated space of the relay lens. The optical design process, prescription, and performance are reported for the entire system, including calibration and alignment. Simulations of high-resolution spectral images are conducted to verify the reconstruction algorithm and relative resolution of the instrument compared to ground truth data. Measured data were taken with the instrument using both a random coded aperture and standard slit for spatial resolution comparisons. Finally, reconstructed spectral images from the instrument are presented of a sunlight-illuminated flower from 360 to 700 nm.

Temporal Compressive Sensing for Video

Video camera architects must design cameras capable of high-quality, dynamic event capture, while adhering to power and communications constraints. Though modern imagers are capable of both simultaneous spatial and temporal resolutions at micrometer and microsecond scales, the power required to sample at these rates is undesirable. The field of compressive sensing (CS) has recently suggested a solution to this design challenge. By exploiting physical-layer compression strategies, one may overlay the original scene with a coding sequence to sample at sub-Nyquist rates with virtually no additional power requirement. The underlying scene may be later estimated without significant loss of fidelity. In this chapter, we cover a variety of such strategies taken to improve an imager’s temporal resolution. Highlighting a new low-power acquisition paradigm, we show how a video sequence of high temporal resolution may be reconstructed from a single video frame taken with a low-framerate camera.

Compressive Sensing for Video Using a Passive Coding Element

We present a prototype system that utilizes mechanical translation of a passive coding element to compress high-speed temporal information into low-framerate video sequences. Reconstructions of 148 frames per experimental coded snapshot are reported.