Thomas J. Grycewicz

Estimation bias from using nonlinear Fourier plane correlators for sub-pixel image shift measurement and implications for the binary joint transform correlator

When using Fourier plane digital algorithms or an optical correlator to measure the correlation between digital images, interpolation by center-of-mass or quadratic estimation techniques can be used to estimate image displacement to the sub-pixel level. However, this can lead to a bias in the correlation measurement. This bias shifts the sub-pixel output measurement to be closer to the nearest pixel center than the actual location. The paper investigates the bias in the outputs of both digital and optical correlators, and proposes methods to minimize this effect. We use digital studies and optical implementations of the joint transform correlator to demonstrate optical registration with accuracies better than 0.1 pixels. We use both simulations of image shift and movies of a moving target as inputs. We demonstrate bias error for both center-of-mass and quadratic interpolation, and discuss the reasons that this bias is present. Finally, we suggest measures to reduce or eliminate the bias effects. We show that when sub-pixel bias is present, it can be eliminated by modifying the interpolation method. By removing the bias error, we improve registration accuracy by thirty percent.

Joint transform optical correlation applied to sub-pixel image registration

The binary joint transform correlator (BJTC) can provide sub-pixel correlation location accuracy for a pair of almost identical inputs, as is the case when computing the registration offset between two overlapping images from the same sensor. Applications include noise cancellation, motion compensation, super-resolution processing, and image splicing. We experimentally demonstrated sub-pixel registration and image co-addition. Our results show a resolution improves by a factor of almost two compared to normal integration. This paper details early results in an ongoing project.

Fourier plane and optical processing for sub-pixel image registration

Accurate registration of image pairs is critical to a number of image processing tasks, including image splicing, change detection, image co-addition, and super-resolution processing. We have applied digital Fourier plane correlation and optical joint-transform correlation to image registration. We demonstrated RMS registration accuracy of 0.09 pixels for a digital system and 0.1 pixels for an optical system. The experimental system uses an electrically addressed spatial light modulator (SLM) at the input plane of a joint transform correlator and an optically addressed SLM at the Fourier plane. We operate our optical system in burst mode at the rate of 50 correlations per second. The paper describes digital and experimental implementations of the image registration system. We discuss preprocessing algorithms used to prepare inputs for correlation, post-processing algorithms to interpolate the output, and the effect of these processing variations on system performance.

Focal plane resolution and overlapped array TDI imaging

In this paper we model sub-pixel image registration for a generic earth-observing satellite system with a focal plane using two offset Time Delay and Integrate (TDI) arrays in the focal plane to improve the achievable ground resolution over the resolution achievable with a single array. The modeling process starts with a high-resolution image as ground truth. The Parameterized Image Chain Analysis & Simulation SOftware (PICASSO) modeling tool is used to degrade the images to match the optical transfer function, sampling, and noise characteristics of the target system. The model outputs a pair of images with a separation close to the nominal half-pixel separation between the overlapped arrays. A registration estimation algorithm is used to measure the offset for image reconstruction. The two images are aligned and summed on a grid with twice the capture resolution. We compare the resolution in images between the inputs before overlap, the reconstructed image, and a simulation for the image which would have been captured on a focal plane with twice the resolution. We find the performance to always be better than the lower resolution baseline, and to approach the performance of the high-resolution array in the ideal case. We show that the overlapped array imager significantly outperforms both the conventional high- and low-resolution imagers in conditions with high image smear.

PICASSO: an end-to-end image simulation tool for space and airborne imaging systems

The design of any modern imaging system is the end result of many trade studies, each seeking to optimize image quality within real world constraints such as cost, schedule and overall risk. Image chain analysis - the prediction of image quality from fundamental design parameters - is an important part of this design process. At The Aerospace Corporation we have been using a variety of image chain analysis tools for many years, the Parameterized Image Chain Analysis & Simulation SOftware (PICASSO) among them. In this paper we describe our PICASSO tool, showing how, starting with a high quality input image and hypothetical design descriptions representative of the current state of the art in commercial imaging satellites, PICASSO can generate standard metrics of image quality in support of the decision processes of designers and program managers alike.

Avoiding stair-step artifacts in image registration for GOES-R navigation and registration assessment

In developing software for independent verification and validation (IV and V) of the Image Navigation and Registration (INR) capability for the Geostationary Operational Environmental Satellite – R Series (GOES-R) Advanced Baseline Imager (ABI), we have encountered an image registration artifact which limits the accuracy of image offset estimation at the subpixel scale using image correlation. Where the two images to be registered have the same pixel size, subpixel image registration preferentially selects registration values where the image pixel boundaries are close to lined up. Because of the shape of a curve plotting input displacement to estimated offset, we call this a stair-step artifact. When one image is at a higher resolution than the other, the stair-step artifact is minimized by correlating at the higher resolution. For validating ABI image navigation, GOES-R images are correlated with Landsat-based ground truth maps. To create the ground truth map, the Landsat image is first transformed to the perspective seen from the GOES-R satellite, and then is scaled to an appropriate pixel size. Minimizing processing time motivates choosing the map pixels to be the same size as the GOES-R pixels. At this pixel size image processing of the shift estimate is efficient, but the stair-step artifact is present. If the map pixel is very small, stair-step is not a problem, but image correlation is computation-intensive. This paper describes simulation-based selection of the scale for truth maps for registering GOES-R ABI images.

Focal plane resolution and overlapped array time delay and integrate imaging

In this paper we model sub-pixel image registration for a generic earth-observing satellite system with a focal plane using two offset time delay and integrate (TDI) arrays in the focal plane to improve the achievable ground resolution over the resolution achievable with a single array. The modeling process starts with a high-resolution image as ground truth. The Parameterized Image Chain Analysis & Simulation Software (PICASSO) modeling tool is used to degrade the images to match the optical transfer function, sampling, and noise characteristics of the target system. The model outputs a pair of images with a separation close to the nominal half-pixel separation between the overlapped arrays. A registration estimation algorithm is used to measure the offset for image reconstruction. The two images are aligned and summed on a grid with twice the capture resolution. We compare the resolution in images between the inputs before overlap, the reconstructed image, and a simulation for the image which would have been captured on a focal plane with twice the resolution. We find the performance to always be better than the lower resolution baseline, and to approach the performance of the high-resolution array in the ideal case. We show that the overlapped array imager significantly outperforms both the conventional high- and low-resolution imagers in conditions with high image smear.

Single lens joint transform correlator for high-resolution position location

This paper demonstrates the use of the single lens joint transform correlator (SLJTC) to precisely determine the target location for on-axis correlation with the targets superimposed. The SLJTC is a two stage processor with an input stage identical to the chirp-encoded joint transform correlator. The first stage computes the chirp-modulated joint power spectrum. The correlation signal in the chirp-encoded joint power spectrum is an amplitude encoded lens function of Fresnel zone plate. The correlation output is in the focal plane of this zone plate. The location of the center of the zone plate is proportional to the correlation location. The magnification of the correlation plane is determined by the chirp displacement. This is used to amplify small shifts in the correlation location. The SLJTC has been experimentally demonstrated with an output plane magnification of 6.3 and a peak-to-noise ratio of 16.7 dB.