Michael K. Yetzbacher

Designing manufacturable filters for a 16-band plenoptic camera using differential evolution

A 16-band plenoptic camera allows for the rapid exchange of filter sets via a 4x4 filter array on the lenss front aperture. This ability to change out filters allows for an operator to quickly adapt to different locales or threat intelligence. Typically, such a system incorporates a default set of 16 equally spaced at-topped filters. Knowing the operating theater or the likely targets of interest it becomes advantageous to tune the filters. We propose using a modified beta distribution to parameterize the different possible filters and differential evolution (DE) to search over the space of possible filter designs. The modified beta distribution allows us to jointly optimize the width, taper and wavelength center of each single- or multi-pass filter in the set over a number of evolutionary steps. Further, by constraining the function parameters we can develop solutions which are not just theoretical but manufacturable. We examine two independent tasks: general spectral sensing and target detection. In the general spectral sensing task we utilize the theory of compressive sensing (CS) and find filters that generate codings which minimize the CS reconstruction error based on a fixed spectral dictionary of endmembers. For the target detection task and a set of known targets, we train the filters to optimize the separation of the background and target signature. We compare our results to the default 16 at-topped non-overlapping filter set which comes with the plenoptic camera and full hyperspectral resolution data which was previously acquired.

Imaging with multi-spectral mosaic-array cameras.

The emerging class of multi-spectral mosaic-array cameras combines opportunities of spectral data processing and full-motion video color display. We explore capabilities of such sensors and propose the novel demosaicking algorithm capable of enhancing resolution of equally sampled multi-spectral mosaic imagery. We present experimental results of the proposed processing using the imagery acquired with a nine-band short-wave infrared mosaic-array camera.

Super-Resolution Based Demosaicking For Full Motion Video SWIR Multi-Spectral Sensor

We present novel demosaicking algorithm for the first full motion video SWIR 9-band sensor based on pixel size filter array. Spatial resolution enhancement of each spectral band is achieved using modified super-resolution technique.

Multiple-order staircase etalon spectroscopy

Traditional Fabry-Perot (FP) spectroscopy is bandwidth limited to avoid mixing signals from different transmission orders of the interferometer. Unlike Fourier transformation, the extraction of spectra from multiple-order interferograms resulting from multiplexed optical signals is in general an ill-posed problem. Using a Fourier transform approach, we derive a generalized Nyquist limit appropriate to signal recovery from FP interferograms. This result is used to derive a set of design rules giving the usable wavelength range and spectral resolution of FP interferometers or etalon arrays given a set of accessible physical parameters. Numerical simulations verify the utility of these design rules for moderate resolution spectroscopy with bandwidths limited by the detector spectral response. Stable and accurate spectral recovery over more than one octave is accomplished by simple matrix multiplication of the interferogram. In analogy to recently developed single-order micro-etalon arrays (Proc. of SPIE v.8266, no. 82660Q), we introduce Multiple-Order Staircase Etalon Spectroscopy (MOSES), in which micro-arrays of multiple order etalons can be bonded to or co-fabricated with a sensor array. MOSES enables broader bandwidth multispectral and hyperspectral instruments than single-order etalon arrays while keeping a physical footprint insignificantly different from that of the detection array.

Demosaicking for full motion video 9-band SWIR sensor

Short wave infrared (SWIR) spectral imaging systems are vital for Intelligence, Surveillance, and Reconnaissance (ISR) applications because of their abilities to autonomously detect targets and classify materials. Typically the spectral imagers are incapable of providing Full Motion Video (FMV) because of their reliance on line scanning. We enable FMV capability for a SWIR multi-spectral camera by creating a repeating pattern of 3x3 spectral filters on a staring focal plane array (FPA). In this paper we present the imagery from an FMV SWIR camera with nine discrete bands and discuss image processing algorithms necessary for its operation. The main task of image processing in this case is demosaicking of the spectral bands i.e. reconstructing full spectral images with original FPA resolution from spatially subsampled and incomplete spectral data acquired with the choice of filter array pattern. To the best of authors knowledge, the demosaicking algorithms for nine or more equally sampled bands have not been reported before. Moreover all existing algorithms developed for demosaicking visible color filter arrays with less than nine colors assume either certain relationship between the visible colors, which are not valid for SWIR imaging, or presence of one color band with higher sampling rate compared to the rest of the bands, which does not conform to our spectral filter pattern. We will discuss and present results for two novel approaches to demosaicking: interpolation using multi-band edge information and application of multi-frame super-resolution to a single frame resolution enhancement of multi-spectral spatially multiplexed images.

An analytic method for spectrum recovery from wedge or staircase spectrometers

Wedge or staircase micro-optics have become important components for building miniature optical spectrometers. These devices create spectral discrimination through interference between beams resulting from reflections at the surfaces of the optic. The literature has examples of low reflectance wedge spectrometer system where the Fourier transform is used to recover the spectrum (with no inherent bandwidth limit), and high-reflectance, band-limited simplex spectrometers where no data processing is required. Instruments in the first category tend to be for the thermal infrared range, and instruments in the second category are more often encountered in the visible band. This second category includes linear variable filters and discrete etalon staircases. Though in practice, the signal treatment for these two types of spectrometers is radically different, the underlying interference mechanism is identical. It follows, that a single signal processing algorithm must exist which correctly treats the two types of signals. We present a mathematical description of the signal model for such spectrometers. We show that in the case of spectrally uniform reflectance, the signal has a specific relationship to the spectrum’s Fourier transform. We cast the spectral recovery problem as a matrix inversion, and derive formulas for calculating the solution matrix. The solution matrix is shown to yield the exact spectrum when applied to modeled wedge spectrometer signals in both low and high reflectance cases.

The effect of lens aperture for remote sensing of trace gases using Fabry-Perot interferometer-based cameras

We report an approximate simplified calculation of transmission spectra for a telecentric cone of light impinging on a Fabry-Perot interferometer. We model sulfur dioxide sensing and show that the F-number affects the optimum parameters. OCIS codes: (120.2230) Fabry-Perot; (280.1120) Air pollution monitoring

Selecting an Optimized COTS Filter Set for Multispectral Plenoptic Sensing

A 16-band plenoptic camera allows for the rapid exchange of filter sets via a 4x4 filter array on the lens’s front aperture thus allowing an operator to quickly adapt to a different locale or threat intelligence. Typically, such a system incorporates a default set of 16 equally spaced, non-overlapping, flat-topped filters. Knowing the operating theater or the likely targets of interest it becomes advantageous to tune the filters; we propose a differential evolution approach to search over a set of commercial off-the-shelf (COTS) filters for an optimal collection of filters. We examine two independent tasks: general spectral sensing and target detection. For general spectral sensing, we utilize compressive sensing and find filters that generate codings which minimize the reconstruction error. For target detection, we select filters to optimize the separation between the background and a set of targets. We compare the results obtained using the selected COTS filters to the default filter set and full spectral resolution hyperspectral (HS) filter set for target detection and general spectral sensing on a previously obtained HS image.

Increasing Resolution of Multi-Spectral Mosaic-Array Cameras

We present approaches to spatial resolution recovery of multi-spectral mosaic-array sensors based on pixel size filter arrays. Resolution enhancement is demonstrated using imagery collected with 9-band SWIR full motion video camera.

Multiple height calibration reference for nano-metrology

Modern nano-metrology instruments require calibration references with nanometer accuracy in the x, y, and z directions. A common problem is the accurate calibration in the z direction (height). For example, it is generally not difficult to obtain accurate x- and y- calibration references for a Scanning Probe Microscope (SPM). It is, however, much more difficult to obtain accurate z-axis results. It is difficult to control z-axis piezo dynamics because during scanning in the xy-plane the x-and y-axes move at a constant rate whiles the z axis does not. Furthermore due to the high cost of producing calibration standards, the microscope is often calibrated at only one height. However, if the relationship between the measured z height and the actual z height is not linear, then the height measurements will not be correct. In this paper, we will present a method for the fabrication of calibration references with: (i) sub-10 nm features and (ii) multiple step heights on one reference, allowing for better calibration of the non-linearity in the z direction.