Victor Tom

Morphology-based algorithm for point target detection in infrared backgrounds

A morphology-based algorithm has been developed for point target detection in IRST applications. It exhibits comparable detection and false alarm performance to a median filter. The morphology-based algorithm has an efficient computational paradigm based on combinations of simple nonlinear grayscale operations, which makes it ideally suited to real- time, high data rate IRST applications. A detection filter based on morphological background estimation exhibits spatial high-pass characteristics emphasizing target-like peaks in the data and suppressing all other clutter. Example cases are presented which point out the detection performance differences between the morphological and median approaches. Overall performance results were generated in the form of ROC curves for cloud, horizon and sea clutter IRAMMP backgrounds.

Automatic Geolocation Correction of Satellite Imagery

Modern satellites tag their images with geolocation information using GPS and star tracking systems. Depending on the quality of the geopositioning equipment, errors may range from a few meters to tens of meters on the ground. At the current state of art, there is no established method to automatically correct these errors limiting the large-scale joint utilization of cross-platform satellite images. In this paper, an automatic geolocation correction framework that corrects images from multiple satellites simultaneously is presented. As a result of the proposed correction process, all the images are effectively registered to the same absolute geodetic coordinate frame. The usability and the quality of the correction framework are demonstrated through a 3-D surface reconstruction application. The 3-D surface models given by original satellite geopositioning metadata, and the corrected metadata, are compared. The quality difference is measured through an entropy-based metric applied to the orthographic height maps given by the 3-D surface models. Measuring the absolute accuracy of the framework is harder due to lack of publicly available high-precision ground surveys. However, the geolocation of images of exemplar satellites from different parts of the globe are corrected, and the road networks given by OpenStreetMap are projected onto the images using original and corrected metadata to demonstrate the improved quality of alignment.

A Theory of Automatic Parameter Selection for Feature Extraction With Application to Feature-Based Multisensor Image Registration

This paper generalizes the previously developed automated edge-detection parameter selection algorithm of Yitzhaky and Peli. We generalize the approach to arbitrary multidimensional, continuous or discrete parameter spaces, and feature spaces. This generalization enables use of the parameter selection approach with more general image features, for use in feature-based multisensor image registration applications. We investigate the problem of selecting a suitable parameter space sampling density in the automated parameter selection algorithm. A real-valued sensitivity measure is developed which characterizes the effect of parameter space sampling on feature set variability. Closed-form solutions of the sensitivity measure for special feature set relationships are derived. We conduct an analysis of the convergence properties of the sensitivity measure as a function of increasing parameter space sampling density. For certain parameter space sampling sequence types, closed-form expressions for the sensitivity measure limit values are presented. We discuss an approach to parameter space sampling density selection which uses the sensitivity measure convergence behavior. We provide numerical results indicating the utility of the sensitivity measure for selecting suitable parameter values.

Morphology-based algorithms for target detection/segmentation in FLIR imagery

This paper describes a morphology-based hierarchical process for the detection and segmentation of low and high contrast targets in second generation FLIR imagery. The computational framework is based on the application of simple non-linear binary and grayscale operations that lead to real-time implementations. The process consists of two major processing steps: target-cueing/coarse-segmentation and contour refinement. Our multi-stage detection/segmentation process was applied to both real and simulated FLIR imagery. Preliminary results indicate that the developed morphology-based detector exhibits excellent detection performance for both low and high contrast targets in complex backgrounds while maintaining a low false alarm rate. Contour refinement is based on the watershed transform that is applied in a hierarchical fashion. In addition, our segmenter extracted accurate target outlines under poor conditions in which edge-based techniques or traditional watershed algorithms would have failed.

Application of a dynamic feature selection algorithm to multi-sensor image registration

Image registration is usually a required first processing step for such activities as surveillance, video tracking, change detection, and remote sensing. Often, different sensors are used for the collection of the test and reference imagery. The sensor phenomenology differences can present problems for automatic selection of registration algorithm parameters because of different cross-sensor feature manifestation. In previous work involving edge-based multisensor image registration, we applied a previously-developed automated approach to parameter selection, designed specifically for edge detection. In this work, we adapt and apply a dynamic feature selection algorithm (DFSA) that we recently developed for use in registration algorithm selection for registering images with varying scene content type. We adapt and apply the DFSA to the problem of selecting appropriate registration algorithm parameter values in an edge-based registration algorithm. The approach involves generating test-to-reference feature match scores over a sampling of the transform hypothesis space. The approach is scene-adaptive thereby requiring no a priori information on image scene content. Furthermore, in the DFSA we leverage prior match score calculation generated in a hierarchical grid search to reduce additional computational expense. We give a brief overview of the registration algorithmic framework, and present a description of the dynamic feature selection algorithm. Numerical results are presented for performing test SAR-to-reference EO image registration to show the registration convergence performance improvement resulting from use of the DFSA. Numerical results are generated over images exhibiting different scene content types. We also evaluate the effect of match score normalization on the registration convergence performance improvement.

A comparison of stereo and multiview 3-D reconstruction using cross-sensor satellite imagery

High-resolution and accurate Digital Elevation Model (DEM) generation from satellite imagery is a challenging problem. In this work, a stereo 3-D reconstruction framework is outlined that is applicable to nonstereoscopic satellite image pairs that may be captured by different satellites. The orthographic height maps given by stereo reconstruction are compared to height maps given by a multiview approach based on Probabilistic Volumetric Representation (PVR). Height map qualities are measured in comparison to manually prepared ground-truth height maps in three sites from different parts of the world with urban, semi-urban and rural features. The results along with strengths and weaknesses of the two techniques are summarized.

Dynamic Algorithm Selection for Multi-Sensor Image Registration

Modern sensors have a range of modalities including SAR, EO, and IR. Registration of multimodal imagery from such sensors is becoming an increasingly common pre-processing step for various image exploitation activities such as image fusion for ATR. Over the past decades, several approaches to multisensor image registration have been developed. However, performance of these image registration algorithms is highly dependent on scene content and sensor operating conditions, with no single algorithm working well across the entire operating conditions space. To address this problem, in this paper we present an approach for dynamic selection of an appropriate registration algorithm, tuned to the scene content and feature manifestation of the imagery under consideration. We consider feature-based registration using Harris corners, Canny edge detection, and CFAR features, as well as pixel-based registration using cross-correlation and mutual information. We develop an approach for selecting the optimal combination of algorithms to use in the dynamic selection algorithm. We define a performance measure which balances contributions from convergence redundancy and convergence coverage components calculated over sample imagery, and optimize the measure to define an optimal algorithm set. We present numerical results demonstrating the improvement in registration performance through use of the dynamic algorithm selection approach over results generated through use of a fixed registration algorithm approach. The results provide registration convergence probabilities for geo-registering test SAR imagery against associated EO reference imagery. We present convergence results for various match score normalizations used in the dynamic selection algorithm.

A Progressive Refinement Approach to Aerial Image Registration Using Local Transform Perturbations

Spatial domain log-polar approaches have demonstrated success for video image registration. However, the log-polar representation is sensitive to origin location. This drawback often necessitates performing a parameter sweep over log-polar origin location, which can be time-consuming. In this paper we present an alternative approach that is appropriate for small to moderate scale, rotational, and skew misalignments but allows large translational offset. We use a form of robust phase correlation to estimate the gross translation, then perform a local search over log-polar origin to fine tune the translation. We sequentially estimate affine transform parameters by maximizing a measure of registration solution verity. We also investigate the effect of scale and rotational initial alignment errors on the robustness of the initial phase correlation to estimate gross translation. We present results using video imagery acquired from a real aerial video surveillance system.

A smart iterative algorithm for multisensor image registration

Realtime multisensor image registration algorithms must be computationally efficient. Often, simplifying assumptions are made to reduce computational time. However, these simplifications usually trade registration convergence performance for reduced runtime. For non-realtime applications where computational resources are not severely limited, this tradeoff may be reversed to improve convergence performance at the expense of increased computational cost. To this end we introduce a smart iterative approach to minimize mis-registrations and thus optimize registration convergence probability. The approach involves performing a registration sweep over a smart sampling of parameters governing feature generation. This approach involves use of two components; a feature sensitivity measure (FSM) and a registration verification metric (VM). The FSM measures the effect of parameter values on feature set variability. This measure enables choice of a suitable parameter sampling density to use for performing iterative registration solution search. The VM provides feedback on the registration solution verity in the absence of ground truth and is used to identify a converged solution. First, we provide an overview of the registration framework used to generate convergence results. Next we introduce the FSM and present mathematical properties. We then describe the VM and present the iterative algorithm. We present numerical results illustrating FSM convergence with increasing parameter sampling density for Canny edge features in SAR imagery. We illustrate use of FSM convergence behavior to select a suitable parameter sampling density for use in the iterative algorithm. Finally, SAR-to-EO registration performance results are presented showing improved convergence probability.

A verification metric for multi-sensor image registration

Accurate geo-location of imagery produced from airborne imaging sensors is a prerequisite for precision targeting and navigation. However, the geo-location metadata often has significant errors which can degrade the performance of applications using the imagery. When reference imagery is available, image registration can be performed as part of a bundle-adjustment procedure to reduce metadata errors. Knowledge of the metadata error statistics can be used to set the registration transform hypothesis search space size. In setting the search space size, a compromise is often made between computational expediency and search space coverage. It therefore becomes necessary to detect cases in which the true registration solution falls outside of the initial search space. To this end, we develop a registration verification metric, for use in a multisensor image registration algorithm, which measures the verity of the registration solution. The verification metric value is used in a hypothesis testing problem to make a decision regarding the suitability of the search space size. Based on the hypothesis test outcome, we close the loop on the verification metric in an iterative algorithm. We expand the search space as necessary, and re-execute the registration algorithm using the expanded search space. We first provide an overview of the registration algorithm, and then describe the verification metric. We generate numerical results of the verification metric hypothesis testing problem in the form of Receiver Operating Characteristics curves illustrating the accuracy of the approach. We also discuss normalization of the metric across scene content.