Hoover Rueda

Higher-order computational model for coded aperture spectral imaging.

Coded aperture snapshot spectral imaging systems (CASSI) sense the three-dimensional spatio-spectral information of a scene using a single two-dimensional focal plane array snapshot. The compressive CASSI measurements are often modeled as the summation of coded and shifted versions of the spectral voxels of the underlying scene. This coarse approximation of the analog CASSI sensing phenomena is then compensated by calibration preprocessing prior to signal reconstruction. This paper develops a higher-order precision model for the optical sensing in CASSI that includes a more accurate discretization of the underlying signals, leading to image reconstructions less dependent on calibration. Further, the higher-order model results in improved image quality reconstruction of the underlying scene than that achieved by the traditional model. The proposed higher precision computational model is also more suitable for reconfigurable multiframe CASSI systems where multiple coded apertures are used sequentially to capture the hyperspectral scene. Several simulations and experimental measurements demonstrate the benefits of the discretization model.

DMD-based implementation of patterned optical filter arrays for compressive spectral imaging.

Compressive spectral imaging (CSI) captures multispectral imagery using fewer measurements than those required by traditional Shannon-Nyquist theory-based sensing procedures. CSI systems acquire coded and dispersed random projections of the scene rather than direct measurements of the voxels. To date, the coding procedure in CSI has been realized through the use of block-unblock coded apertures (CAs), commonly implemented as chrome-on-quartz photomasks. These apertures block or permit us to pass the entire spectrum from the scene at given spatial locations, thus modulating the spatial characteristics of the scene. This paper extends the framework of CSI by replacing the traditional block-unblock photomasks by patterned optical filter arrays, referred to as colored coded apertures (CCAs). These, in turn, allow the source to be modulated not only spatially but spectrally as well, entailing more powerful coding strategies. The proposed CCAs are synthesized through linear combinations of low-pass, high-pass, and bandpass filters, paired with binary pattern ensembles realized by a digital micromirror device. The optical forward model of the proposed CSI architecture is presented along with a proof-of-concept implementation, which achieves noticeable improvements in the quality of the reconstruction.

Multi-spectral compressive snapshot imaging using RGB image sensors

Compressive sensing is a powerful sensing and reconstruction framework for recovering high dimensional signals with only a handful of observations and for spectral imaging, compressive sensing offers a novel method of multispectral imaging. Specifically, the coded aperture snapshot spectral imager (CASSI) system has been demonstrated to produce multi-spectral data cubes color images from a single snapshot taken by a monochrome image sensor. In this paper, we expand the theoretical framework of CASSI to include the spectral sensitivity of the image sensor pixels to account for color and then investigate the impact on image quality using either a traditional color image sensor that spatially multiplexes red, green, and blue light filters or a novel Foveon image sensor which stacks red, green, and blue pixels on top of one another.

Compressive Hyperspectral Imaging via Approximate Message Passing

We consider a compressive hyperspectral imaging reconstruction problem, where three-dimensional spatio-spectral information about a scene is sensed by a coded aperture snapshot spectral imager (CASSI). The CASSI imaging process can be modeled as suppressing three-dimensional coded and shifted voxels and projecting these onto a two-dimensional plane, such that the number of acquired measurements is greatly reduced. On the other hand, because the measurements are highly compressive, the reconstruction process becomes challenging. We previously proposed a compressive imaging reconstruction algorithm that is applied to two-dimensional images based on the approximate message passing (AMP) framework. AMP is an iterative algorithm that can be used in signal and image reconstruction by performing denoising at each iteration. We employed an adaptive Wiener filter as the image denoiser, and called our algorithm “AMP-Wiener.” In this paper, we extend AMP-Wiener to three-dimensional hyperspectral image reconstruction, and call it “AMP-3D-Wiener.” Applying the AMP framework to the CASSI system is challenging, because the matrix that models the CASSI system is highly sparse, and such a matrix is not suitable to AMP and makes it difficult for AMP to converge. Therefore, we modify the adaptive Wiener filter and employ a technique called damping to solve for the divergence issue of AMP. Our approach is applied in nature, and the numerical experiments show that AMP-3D-Wiener outperforms existing widely-used algorithms such as gradient projection for sparse reconstruction (GPSR) and two-step iterative shrinkage/thresholding (TwIST) given a similar amount of runtime. Moreover, in contrast to GPSR and TwIST, AMP-3D-Wiener need not tune any parameters, which simplifies the reconstruction process.

Spatial super-resolution in code aperture spectral imaging

The Code Aperture Snapshot Spectral Imaging system (CASSI) senses the spectral information of a scene using the underlying concepts of compressive sensing (CS). The random projections in CASSI are localized such that each measurement contains spectral information only from a small spatial region of the data cube. The goal of this paper is to translate high-resolution hyperspectral scenes into compressed signals measured by a low-resolution detector. Spatial super-resolution is attained as an inverse problem from a set of low-resolution coded measurements. The proposed system not only offers significant savings in size, weight and power, but also in cost as low resolution detectors can be used. The proposed system can be efficiently exploited in the IR region where the cost of detectors increases rapidly with resolution. The simulations of the proposed system show an improvement of up to 4 dB in PSNR. Results also show that the PSNR of the reconstructed data cubes approach the PSNR of the reconstructed data cubes attained with high-resolution detectors, at the cost of using additional measurements.

Single Aperture Spectral+ToF Compressive Camera: Toward Hyperspectral+Depth Imagery

Spectral imaging involves the sensing of a large amount of spatial information across a multitude of wavelengths. Conventional approaches rely on scanning techniques to construct a spectral data cube. Recently, compressive spectral imaging (CSI) has allowed to estimate spectral images with as few as a single coded snapshot. On a different front, 3-D ranging imaging often involves scanning along one of the spatial dimensions to estimate the depth of an scene using structured light, or the use of two cameras as required by stereo-imaging techniques. Recently, Time-of-Flight (ToF) snapshot imaging has gained considerable attention, due to its accuracy and speed. To date, however, these imaging modalities (CSI and ToF) have been realized and implemented by separate independent imaging sensors. This paper presents the development of a single aperture compressive spectral + depth imaging camera that employs a commodity 3-D range ToF sensor as the sensing device of a coded-aperture-based compressive spectral imager. The proposed system uses a single aperture/single sensor; thus, representing a significant improvement over existing RGB+D cameras that integrate two separate image sensors, one for RGB and another for depth. In addition, the observable wavelength range of the CSI device is expanded from the visible to the near-infrared, resolving up to as many as 16 independent channels. The proposed system allows the addition of side-information by placing a grayscale or RGB camera in the same path of the single-aperture system, such that the quality of the spectral estimation is improved, while maintaining high-frame rates. We demonstrate the proposed ideas through real experimentation conducted on an assembled CSI+ToF testbed camera system.

Compressive spectral integral imaging using a microlens array

In this paper, a compressive spectral integral imaging system using a microlens array (MLA) is proposed. This system can sense the 4D spectro-volumetric information into a compressive 2D measurement image on the detector plane. In the reconstruction process, the 3D spatial information at different depths and the spectral responses of each spatial volume pixel can be obtained simultaneously. In the simulation, sensing of the 3D objects is carried out by optically recording elemental images (EIs) using a scanned pinhole camera. With the elemental images, a spectral data cube with different perspectives and depth information can be reconstructed using the TwIST algorithm in the multi-shot compressive spectral imaging framework. Then, the 3D spatial images with one dimensional spectral information at arbitrary depths are computed using the computational integral imaging method by inversely mapping the elemental images according to geometrical optics. The simulation results verify the feasibility of the proposed system. The 3D volume images and the spectral information of the volume pixels can be successfully reconstructed at the location of the 3D objects. The proposed system can capture both 3D volumetric images and spectral information in a video rate, which is valuable in biomedical imaging and chemical analysis.

Compressive spectral testbed imaging system based on thin-film color-patterned filter arrays

Compressive spectral imaging systems can reliably capture multispectral data using far fewer measurements than traditional scanning techniques. In this paper, a thin-film patterned filter array-based compressive spectral imager is demonstrated, including its optical design and implementation. The use of a patterned filter array entails a single-step three-dimensional spatial-spectral coding on the input data cube, which provides higher flexibility on the selection of voxels being multiplexed on the sensor. The patterned filter array is designed and fabricated with micrometer pitch size thin films, referred to as pixelated filters, with three different wavelengths. The performance of the system is evaluated in terms of references measured by a commercially available spectrometer and the visual quality of the reconstructed images. Different distributions of the pixelated filters, including random and optimized structures, are explored.

Robust tracking and anomaly detection in Video Surveillance sequences

In this paper, the authors examine the problem of tracking people in both bright and dark video sequences. In particular, this problem is treated as a background/foreground decomposition problem, where the static part corresponds to the background, and moving objects to the foreground. Having this into account, the problem is formulated as a rank minimization problem of the form X = L + S + E, where X is the captured scene, L is the low-rank part (background), S is the sparse part (foreground) and E is the corrupting uniform noise introduced in the capture process. Actually, low-rank and sparse structures are widely studied and some areas such as Robust Principal Component Analysis (RPCA) and Matrix Completion (MC) have emerged to solve this kind of problems. Here we compare the performance of three different methods in solving the RPCA optimization problem for background separation: augmented lagrange multiplier method, Bayesian markov dependency method, and bilateral random projections method. Furthermore, a preprocessing light normalization stage and a mathematical morphology based post-processing stage are proposed to obtain better results.

3D compressive spectral integral imaging.

A novel compressive 3D imaging spectrometer based on the coded aperture snapshot spectral imager (CASSI) is proposed. By inserting a microlens array (MLA) into the CASSI system, one can capture spectral data of 3D objects in a single snapshot without requiring 3D scanning. The 3D spatio-spectral sensing phenomena is modelled by computational integral imaging in tandem with compressive coded aperture spectral imaging. A set of focal stack images is reconstructed from a single compressive measurement, and presented as images focused on different depth planes where the objects are located. The proposed optical system is demonstrated with simulations and experimental results.