Russell C. Hardie

Joint MAP registration and high-resolution image estimation using a sequence of undersampled images

In many imaging systems, the detector array is not sufficiently dense to adequately sample the scene with the desired field of view. This is particularly true for many infrared focal plane arrays. Thus, the resulting images may be severely aliased. This paper examines a technique for estimating a high-resolution image, with reduced aliasing, from a sequence of undersampled frames. Several approaches to this problem have been investigated previously. However, in this paper a maximum a posteriori (MAP) framework for jointly estimating image registration parameters and the high-resolution image is presented. Several previous approaches have relied on knowing the registration parameters a priori or have utilized registration techniques not specifically designed to treat severely aliased images. In the proposed method, the registration parameters are iteratively updated along with the high-resolution image in a cyclic coordinate-descent optimization procedure. Experimental results are provided to illustrate the performance of the proposed MAP algorithm using both visible and infrared images. Quantitative error analysis is provided and several images are shown for subjective evaluation.

High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system

Some imaging systems employ detector arrays which are not su‐ciently dense so as to meet the Nyquist criteria during image acquisition. This is particularly true for many staring infrared imagers. Thus, the full resolution afiorded by the optics is not being realized in such a system. This paper presents a technique for estimating a high resolution image, with reduced aliasing, from a sequence of undersampled rotated and translationally shifted frames. Such an image sequence can be obtained if an imager is mounted on a moving platform, such as an aircraft. Several approaches to this type of problem have been proposed in the literature. Here we extend some of this previous work. In particular, we deflne an observation model which incorporates knowledge of the optical system and detector array. The high resolution image estimate is formed by minimizing a new regularized cost function which is based on the observation model. We show that with the proper choice of a tuning parameter, our algorithm exhibits robustness in the presence of noise. We consider both gradient descent and conjugate gradient optimization procedures to minimize the cost function. Detailed experimental results are provided to illustrate the performance of the proposed algorithm using digital video from an infrared imager.

A new computationally efficient CAD system for pulmonary nodule detection in CT imagery

Early detection of lung nodules is extremely important for the diagnosis and clinical management of lung cancer. In this paper, a novel computer aided detection (CAD) system for the detection of pulmonary nodules in thoracic computed tomography (CT) imagery is presented. The paper describes the architecture of the CAD system and assesses its performance on a publicly available database to serve as a benchmark for future research efforts. Training and tuning of all modules in our CAD system is done using a separate and independent dataset provided courtesy of the University of Texas Medical Branch (UTMB). The publicly available testing dataset is that created by the Lung Image Database Consortium (LIDC). The LIDC data used here is comprised of 84 CT scans containing 143 nodules ranging from 3 to 30mm in effective size that are manually segmented at least by one of the four radiologists. The CAD system uses a fully automated lung segmentation algorithm to define the boundaries of the lung regions. It combines intensity thresholding with morphological processing to detect and segment nodule candidates simultaneously. A set of 245 features is computed for each segmented nodule candidate. A sequential forward selection process is used to determine the optimum subset of features for two distinct classifiers, a Fisher Linear Discriminant (FLD) classifier and a quadratic classifier. A performance comparison between the two classifiers is presented, and based on this, the FLD classifier is selected for the CAD system. With an average of 517.5 nodule candidates per case/scan (517.5+/-72.9), the proposed front-end detector/segmentor is able to detect 92.8% of all the nodules in the LIDC/testing dataset (based on merged ground truth). The mean overlap between the nodule regions delineated by three or more radiologists and the ones segmented by the proposed segmentation algorithm is approximately 63%. Overall, with a specificity of 3 false positives (FPs) per case/patient on average, the CAD system is able to correctly identify 80.4% of the nodules (115/143) using 40 selected features. A 7-fold cross-validation performance analysis using the LIDC database only shows CAD sensitivity of 82.66% with an average of 3 FPs per CT scan/case.

Rank conditioned rank selection filters for signal restoration

A class of nonlinear filters called rank conditioned rank selection (RCRS) filters is developed and analyzed in this paper. The RCRS filters are developed within the general framework of rank selection (RS) filters, which are filters constrained to output an order statistic from the observation set. Many previously proposed rank order based filters can be formulated as RS filters. The only difference between such filters is in the information used in deciding which order statistic to output. The information used by RCRS filters is the ranks of selected input samples, hence the name rank conditioned rank selection filters. The number of input sample ranks used is referred to as the order of the RCRS filter. The order can range from zero to the number of samples in the observation window, giving the filters valuable flexibility. Low-order filters can give good performance and are relatively simple to optimize and implement. If improved performance is demanded, the order can be increased but at the expense of filter simplicity. In this paper, many statistical and deterministic properties of the RCRS filters are presented. A procedure for optimizing over the class of RCRS filters is also presented. Finally, extensive computer simulation results that illustrate the performance of RCRS filters in comparison with other techniques in image restoration applications are presented.

LUM filters: a class of rank-order-based filters for smoothing and sharpening

A new class of rank-order-based filters, called lower-upper-middle (LUM) filters, is introduced. The output of these filters is determined by comparing a lower- and an upper-order statistic to the middle sample in the filter window. These filters can be designed for smoothing and sharpening, or outlier rejection. The level of smoothing done by the filter can range from no smoothing to that of the median filter. This flexibility allows the LUM filter to be designed to best balance the tradeoffs between noise smoothing and signal detail preservation. LUM filters for enhancing edge gradients can be designed to be insensitive to low levels of additive noise and to remove impulsive noise. Furthermore, LUM filters do not cause overshoot or undershoot. Some statistical and deterministic properties of the LUM filters are developed, and a number of experimental results are presented to illustrate the performance. These experiments include applications to 1D signals and to images. >

Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames

Forward looking infrared (FLIR) detector arrays generally produce spatially undersampled images because the FLIR arrays cannot be made dense enough to yield a sufficiently high spatial sampling frequency. Multi-frame techniques, such as microscanning, are an effective means of reducing aliasing and increasing resolution in images produced by staring imaging systems. These techniques involve interlacing a set of image frames that have been shifted with respect to each other during acquisition. The FLIR system is mounted on a moving platform, such as an aircraft, and the vibrations associated with the platform are used to generate the shifts. Since a fixed number of image frames is required, and the shifts are random, the acquired frames will not fall on a uniformly spaced grid. Furthermore, some of the acquired frames may have almost similar shifts thus making them unusable for high-resolution image reconstruction. In this paper, we utilize a gradient-based registration algorithm to estimate the shifts between the acquired frames and then use a weighted nearest-neighbor approach for placing the frames onto a uniform grid to form a final high-resolution image. Blurring by the detector and optics of the imaging system limits the increase in image resolution when microscanning is attempted at sub-pixel movements of less than half the detector width. We resolve this difficulty by the application of the Wiener filter, designed using the modulation transfer function (MTF) of the imaging system, to the high-resolution image. Simulation and experimental results are presented to verify the effectiveness of the proposed technique. The techniques proposed herein are significantly faster than alternate techniques, and are found to be especially suitable for real-time applications.

Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions

A maximum a posteriori (MAP) estimation method for improving the spatial resolution of a hyperspectral image using a higher resolution auxiliary image is extended to address several practical remote sensing situations. These include cases where: 1) the spectral response of the auxiliary image is unknown and does not match that of the hyperspectral image; 2) the auxiliary image is multispectral; and 3) the spatial point spread function for the hyperspectral sensor is arbitrary and extends beyond the span of the detector elements. The research presented follows a previously reported MAP approach that makes use of a stochastic mixing model (SMM) of the underlying spectral scene content to achieve resolution enhancement beyond the intensity component of the hyperspectral image. The mathematical formulation of a generalized form of the MAP/SMM estimate is described, and the enhancement algorithm is demonstrated using various image datasets.

A Fast Image Super-Resolution Algorithm Using an Adaptive Wiener Filter

A computationally simple super-resolution algorithm using a type of adaptive Wiener filter is proposed. The algorithm produces an improved resolution image from a sequence of low-resolution (LR) video frames with overlapping field of view. The algorithm uses subpixel registration to position each LR pixel value on a common spatial grid that is referenced to the average position of the input frames. The positions of the LR pixels are not quantized to a finite grid as with some previous techniques. The output high-resolution (HR) pixels are obtained using a weighted sum of LR pixels in a local moving window. Using a statistical model, the weights for each HR pixel are designed to minimize the mean squared error and they depend on the relative positions of the surrounding LR pixels. Thus, these weights adapt spatially and temporally to changing distributions of LR pixels due to varying motion. Both a global and spatially varying statistical model are considered here. Since the weights adapt with distribution of LR pixels, it is quite robust and will not become unstable when an unfavorable distribution of LR pixels is observed. For translational motion, the algorithm has a low computational complexity and may be readily suitable for real-time and/or near real-time processing applications. With other motion models, the computational complexity goes up significantly. However, regardless of the motion model, the algorithm lends itself to parallel implementation. The efficacy of the proposed algorithm is demonstrated here in a number of experimental results using simulated and real video sequences. A computational analysis is also presented.

Application of the stochastic mixing model to hyperspectral resolution enhancement

A maximum a posteriori (MAP) estimation method is described for enhancing the spatial resolution of a hyperspectral image using a higher resolution coincident panchromatic image. The approach makes use of a stochastic mixing model (SMM) of the underlying spectral scene content to develop a cost function that simultaneously optimizes the estimated hyperspectral scene relative to the observed hyperspectral and panchromatic imagery, as well as the local statistics of the spectral mixing model. The incorporation of the stochastic mixing model is found to be the key ingredient for reconstructing subpixel spectral information in that it provides the necessary constraints that lead to a well-conditioned linear system of equations for the high-resolution hyperspectral image estimate. Here, the mathematical formulation of the proposed MAP method is described. Also, enhancement results using various hyperspectral image datasets are provided. In general, it is found that the MAP/SMM method is able to reconstruct subpixel information in several principal components of the high-resolution hyperspectral image estimate, while the enhancement for conventional methods, like those based on least squares estimation, is limited primarily to the first principal component (i.e., the intensity component).

An Algebraic Algorithm for Nonuniformity Correction in Focal-Plane Arrays

A scene-based algorithm is developed to compensate for bias nonuniformity in focal-plane arrays. Nonuniformity can be extremely problematic, especially for mid- to far-infrared imaging systems. The technique is based on use of estimates of interframe subpixel shifts in an image sequence, in conjunction with a linear-interpolation model for the motion, to extract information on the bias nonuniformity algebraically. The performance of the proposed algorithm is analyzed by using real infrared and simulated data. One advantage of this technique is its simplicity; it requires relatively few frames to generate an effective correction matrix, thereby permitting the execution of frequent on-the-fly nonuniformity correction as drift occurs. Additionally, the performance is shown to exhibit considerable robustness with respect to lack of the common types of temporal and spatial irradiance diversity that are typically required by statistical scene-based nonuniformity correction techniques.