Kenneth J. Barnard

Joint MAP registration and high-resolution image estimation using a sequence of undersampled images

In many imaging systems, the detector array is not sufficiently dense to adequately sample the scene with the desired field of view. This is particularly true for many infrared focal plane arrays. Thus, the resulting images may be severely aliased. This paper examines a technique for estimating a high-resolution image, with reduced aliasing, from a sequence of undersampled frames. Several approaches to this problem have been investigated previously. However, in this paper a maximum a posteriori (MAP) framework for jointly estimating image registration parameters and the high-resolution image is presented. Several previous approaches have relied on knowing the registration parameters a priori or have utilized registration techniques not specifically designed to treat severely aliased images. In the proposed method, the registration parameters are iteratively updated along with the high-resolution image in a cyclic coordinate-descent optimization procedure. Experimental results are provided to illustrate the performance of the proposed MAP algorithm using both visible and infrared images. Quantitative error analysis is provided and several images are shown for subjective evaluation.

High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system

Some imaging systems employ detector arrays which are not su‐ciently dense so as to meet the Nyquist criteria during image acquisition. This is particularly true for many staring infrared imagers. Thus, the full resolution afiorded by the optics is not being realized in such a system. This paper presents a technique for estimating a high resolution image, with reduced aliasing, from a sequence of undersampled rotated and translationally shifted frames. Such an image sequence can be obtained if an imager is mounted on a moving platform, such as an aircraft. Several approaches to this type of problem have been proposed in the literature. Here we extend some of this previous work. In particular, we deflne an observation model which incorporates knowledge of the optical system and detector array. The high resolution image estimate is formed by minimizing a new regularized cost function which is based on the observation model. We show that with the proper choice of a tuning parameter, our algorithm exhibits robustness in the presence of noise. We consider both gradient descent and conjugate gradient optimization procedures to minimize the cost function. Detailed experimental results are provided to illustrate the performance of the proposed algorithm using digital video from an infrared imager.

Design and performance of the Civil Air Patrol ARCHER hyperspectral processing system

The Civil Air Patrol (CAP) is procuring Airborne Real-time Cueing Hyperspectral Enhanced Reconnaissance (ARCHER) systems to increase their search-and-rescue mission capability. These systems are being installed on a fleet of Gippsland GA-8 aircraft, and will position CAP to gain realworld mission experience with the application of hyperspectral sensor and processing technology to search and rescue. The ARCHER system design, data processing, and operational concept leverage several years of investment in hyperspectral technology research and airborne system demonstration programs by the Naval Research Laboratory (NRL) and Air Force Research Laboratory (AFRL). Each ARCHER system consists of a NovaSol-designed, pushbroom, visible/near-infrared (VNIR) hyperspectral imaging (HSI) sensor, a co-boresighted visible panchromatic high-resolution imaging (HRI) sensor, and a CMIGITS-III GPS/INS unit in an integrated sensor assembly mounted inside the GA-8 cabin. ARCHER incorporates an on-board data processing system developed by Space Computer Corporation (SCC) to perform numerous real-time processing functions including data acquisition and recording, raw data correction, target detection, cueing and chipping, precision image geo-registration, and display and dissemination of image products and target cue information. A ground processing station is provided for post-flight data playback and analysis. This paper describes the requirements and architecture of the ARCHER system, with emphasis on data processor design, components, software, interfaces, and displays. Key sensor performance characteristics and real-time data processing features are discussed. The use of the system for detecting and geo-locating ground targets in real-time is demonstrated using test data collected in Southern California in the fall of 2004.

Fast Super-Resolution Using an Adaptive Wiener Filter with Robustness to Local Motion

We present a new adaptive Wiener filter (AWF) super-resolution (SR) algorithm that employs a global background motion model but is also robust to limited local motion. The AWF relies on registration to populate a common high resolution (HR) grid with samples from several frames. A weighted sum of local samples is then used to perform nonuniform interpolation and image restoration simultaneously. To achieve accurate subpixel registration, we employ a global background motion model with relatively few parameters that can be estimated accurately. However, local motion may be present that includes moving objects, motion parallax, or other deviations from the background motion model. In our proposed robust approach, pixels from frames other than the reference that are inconsistent with the background motion model are detected and excluded from populating the HR grid. Here we propose and compare several local motion detection algorithms. We also propose a modified multiscale background registration method that incorporates pixel selection at each scale to minimize the impact of local motion. We demonstrate the efficacy of the new robust SR methods using several datasets, including airborne infrared data with moving vehicles and a ground resolution pattern for objective resolution analysis.

Review of infrared scene projector technology-1993

The importance of testing IR imagers and missile seekers with realistic IR scenes warrants a review of the current technologies used in dynamic infrared scene projection. These technologies include resistive arrays, deformable mirror arrays, mirror membrane devices, liquid crystal light valves, laser writers, laser diode arrays, and CRTs. Other methods include frustrated total internal reflection, thermoelectric devices, galvanic cells, Bly cells, and vanadium dioxide. A description of each technology is presented along with a discussion of their relative benefits and disadvantages. The current state of each methodology is also summarized. Finally, the methods are compared and contrasted in terms of their performance parameters.

Nonmechanical microscanning using optical space-fed phased arrays

A method for microscanning in imaging sensors is developed that allows liquid-crystal beam steerers to be used as nonmechanical microscan devices. This submicroscanning method involves using the beam steerers to shift images on a focal-plane array by a fraction of the amount used in typical microscan methods. Interpolation techniques based on interlaced sampling are used to produce images free of aliasing out to twice the Nyquist frequency determined by the focal-plane array. Since a continuous phase ramp is produced by the liquid-crystal beam steerer, dispersion effects due to the gratinglike nature of the devices are avoided. Simulations for both one- and two-dimensional cases are presented, as well as experimental results using a 3- to 5-μm imaging sensor and a liquid-crystal beam steerer designed for operation at 1.064 μm.

Fast super-resolution with affine motion using an adaptive Wiener filter and its application to airborne imaging

Fast nonuniform interpolation based super-resolution (SR) has traditionally been limited to applications with translational interframe motion. This is in part because such methods are based on an underlying assumption that the warping and blurring components in the observation model commute. For translational motion this is the case, but it is not true in general. This presents a problem for applications such as airborne imaging where translation may be insufficient. Here we present a new Fourier domain analysis to show that, for many image systems, an affine warping model with limited zoom and shear approximately commutes with the point spread function when diffraction effects are modeled. Based on this important result, we present a new fast adaptive Wiener filter (AWF) SR algorithm for non-translational motion and study its performance with affine motion. The fast AWF SR method employs a new smart observation window that allows us to precompute all the needed filter weights for any type of motion without sacrificing much of the full performance of the AWF. We evaluate the proposed algorithm using simulated data and real infrared airborne imagery that contains a thermal resolution target allowing for objective resolution analysis.

Modulation transfer function of hexagonal staring focal plane arrays

The modulation transfer functions (MTFs) of hexagonally sampled arrays with both rectangular and hexagonal pixel shapes are derived from spatial averaging considerations. In one direction, the hexagonal pixel shape is shown to provide a 13.6% improvement in MTF at the Nyquist bandlimit over an equivalent rectangular shape. For the orthogonal direction, the hexagonal shape has a slightly worse MTF, which is 4.8% less than the MTF of the rectangular shape at the Nyquist bandlimit.

Analysis of hyperspectral change detection as affected by vegetation and illumination variations

This study examines the effectiveness of specific hyperspectral change detection algorithms on scenes with different illumination conditions such as shadows, low sun angles, and seasonal vegetation changes with specific emphasis placed on background suppression. When data sets for the same spatial scene on different occasions exist, change detection algorithms utilize linear predictors such as chronochrome and covariance equalization in an attempt to suppress background and improve detection of atypical manmade changes. Using a push-broom style imaging spectrometer mounted on a pan and tilt platform, visible to near infrared data sets of a scene containing specific objects are gathered. Hyperspectral system characterization and calibration is performed to ensure the production of viable data. Data collection occurs over a range of months to capture a myriad of conditions including daily illumination change, seasonal illumination change, and seasonal vegetation change. Choosing reference images, the degree of background suppression produced for various time-2 scene conditions is examined for different background classes. A single global predictor produces a higher degree of suppression when the conditions between the reference and time-2 remain similar and decreases as drastic illumination and vegetation alterations appear. Manual spatial segmentation of the scene coupled with the application of a different linear predictor for each class can improve suppression.

Fourier Spectral Filter Array for Optimal Multispectral Imaging

Limitations to existing multispectral imaging modalities include speed, cost, range, spatial resolution, and application-specific system designs that lack versatility of the hyperspectral imaging modalities. In this paper, we propose a novel general-purpose single-shot passive multispectral imaging modality. Central to this design is a new type of spectral filter array (SFA) based not on the notion of spatially multiplexing narrowband filters, but instead aimed at enabling single-shot Fourier transform spectroscopy. We refer to this new SFA pattern as Fourier SFA, and we prove that this design solves the problem of optimally sampling the hyperspectral image data.