Hai-Wen Chen

Target detection improvements using temporal integrations and spatial fusion

Our previous results [1], [2] of spatio-temporal fusion for target classification have been further developed for target detection in this paper. In our previous results for target classifIcation, the fusion is conducted in the likelihood function reading domain. In general, the likelihood functions (pdfs) are obtained from training data based on single sensor and single frame measurements. Therefore, if we conducted fusion using the likelihood readings of the features extracted from measurements of single sensor and frame, we only need to store one set of likelihood functions for single sensor and frame, no matter how many sensors and frames we will use for fusion. On the other hand, since the detection process uses thresholding technique instead of likelihood functions, we can direct fuse the features values from different sensors and time frames in thefeature domain for target detection. As discussed in our previous papers, the so-called spatial fusion is defined as the fusion between different sensors, and the temporal fusion is defined as the temporal integration across different time frames within a single sensor. Different spatial fusion and temporal integration (fusion) strategies have been developed and compared, including pre-detection integration (such as additive, multiplicative, MAX, and MIN fusions), as well as the traditional post-detection integration (the persistency test). The pre-detection integration is conducted by fusing the feature values from different time frames before the thresholding process (the detection process), while the post-detection integration is conducted after the thresholding process. Although our techniques are aimed for improving target detection, these techniques can bed used for other applications involving thresholding techniques. In target recognition, ATR (automatic target recognition) is a research area with high attention. One popular ATR approach uses the matched filtering/correlation techniques, and the resulting features after the correlation (e.g., the peak-to-sidelobe-ratio) will subject a threshold-screening to pick the recognized targets [6]. Therefore, both the preand post-detection temporal integration methods can be used to improve target recognition when multiple temporal frames are involved. In our 2nd study, temporal correlation and non-stationary properties of sensor noise have been investigated using sequences of imagery collected by an JR (256x256) sensor looking at different scenes (trees, grass, roads, buildings, etc.). The natural noise extracted from the JR sensor, as well as noise generated by a computer with Gaussian and Rayleigh distributions have been used to test and compare different temporal integration strategies. The simulation results show that both the preand post-detection temporal integrations can considerably improve target detection by integrating only 3—5 time frames (tested by real sensor noise as well as computer generated noise). Moreover, the detection results can be further improved by combining both the preand post-detection temporal integrations.

Integrated spatiotemporal multiple sensor fusion system design

In many missile and fire control applications, targets of interest may be acquired and tracked over some finite period of time with one or more sensors. This allows for the collection of sequential segments or frames of temporal information per sensor as well as across various sensors. By appropriately processing this information, target detection and classification performance can be considerably increased. Furthermore, we have developed new and different fusion strategies (additive and MINMAX fusion) in addition to the traditional strategies. Our test and analysis results show that temporal fusion can improve target classification as well as spatial fusion. In this work we have developed an optimal and novel design for an integrated spatio-temporal multi-sensor fusion system that combines inputs from different sensors as well as from the different time frames of each sensor.

Target detection and recognition improvements by use of spatiotemporal fusion

We developed spatiotemporal fusion techniques for improving target detection and automatic target recognition. We also investigated real IR (infrared) sensor clutter noise. The sensor noise was collected by an IR (256 x 256) sensor looking at various scenes (trees, grass, roads, buildings, etc.). More than 95% of the sensor pixels showed near-stationary sensor clutter noise that was uncorrelated between pixels as well as across time frames. However, in a few pixels (covering the grass near the road) the sensor noise showed nonstationary properties (with increasing or decreasing mean across time frames). The natural noise extracted from the IR sensor, as well as the computer-generated noise with Gaussian and Rayleigh distributions, was used to test and compare different spatiotemporal fusion strategies. Finally, we proposed two advanced detection schemes: the double-thresholding the reverse-thresholding techniques. These techniques may be applied to complicated clutter situations (e.g., very-high clutter or nonstationary clutter situations) where the traditional constant-false-alarm-ratio technique may fail.

Adaptive spatiotemporal multiple sensor fusion

We have developed and applied a spatiotemporal fusion framework that uses different fusion strategies across time frames (temporal fusion) as well as between sensors (spatial fusion). We have developed, at the feature level, new and different fusion strategies (additive and minmax fusion) in addition to the traditional strategies (multiplicative, min, and max fusion). These different fusion strategies are compared and predicted by their receiver operating characteristics performance in the likelihood-reading domain. Furthermore, by applying the additive fusion strategy, we have developed methods for adaptive sensor weighting using reliability functions to improve fusion performance. Our simulated test and analysis results show that temporal fusion (as well as spatial fusion) can considerably improve target classification, and also show that the adaptive sensor weighting using reliability functions can significantly improve fusion performance when one sensor is much more reliable than the other. Finally, we propose an optimal integrated spatiotemporal multiple sensor fusion system that includes two new processors: the adaptive processor and the fusion selection processor.

Smart DRM (dynamic range management) for optimal IR seeker sensitivity and dynamic range control

For an IR (infrared) sensor, the raw digital images coming out from the FPA (focal plane array) A/D converter contain strong non-uniformity/fixed pattern noise (FPN) as well as permanent and blinking dead pixels. Before performing the target detection and tracking functions, these raw images are processed by a CWF (chopper-wheel-free) MBPF NUC (Measurement-Based-Parametric-Fitting Non-Uniformity Correction) system to replace the dead pixels and to remove or reduce the FPN, as shown in Figure 1. The input to MBPF NUC is RIMi,j(the raw image), where 1⩽i, j⩽256, and the output is CIMi,j, the corrected image. It is important to note that as shown in Figure 1 the IT (integration time) for the FPA input capacitors is a critical parameter to control the sensors sensitivity and temperature DR (dynamic range). From the results of our FPN measurement, the STD (standard deviation) of FPN from a raw uncorrected image can be as high as 300-400 counts. This high count FPN will severely reduce the sensors sensitivity (we would like to detect a weak target as low as a couple of counts) and hamper the target tracking and/or ATR functions because of the high counts FPN artifacts. Therefore, the major purpose of the NUC system is to reduce FPN for early target detection, and the secondary purpose is to reduce FPN artifacts for reliable target tracking and ATR.

A chopper-free measurement-based parametric-fitting nonuniformity correction system

In traditional designs of a NUC (non-uniformity correction) system, a rotating chopper-wheel (or a blurring/deform lens) is used to separate the outside scene and the inside FPN (fixed pattern noise) on the FPA (focal plane array). To design a NUC system removing the chopper-wheel (chopper-free) and its control electronics and hardware will not only considerably reduce the cost, but also require less space to fit the NUC system. In this paper, we describe a recently developed CF (chopper-free) NUC system. This system is simpler to build, costs less, and requires less space, as compared with traditional designs.