Belur V. Dasarathy

Multi-level sensor fusion for improved target discrimination

A project was sponsored by the US Army Space and Strategic Defense Command (USASSDC) to develop, test, and demonstrate sensor fusion techniques for target recognition. The sensor fusion architecture developed for this program exploits the use of sensor fusion at all levels (signal, feature, and decision levels) to improve target recognition capability against tactical ballistic missile (TBM) targets. It was hypothesized that use of fusion at all these levels is necessary to give a system robustness to noise and sensor degradation and to accommodate a wide variety of mission and target types. This would address problems such as ineffective sensors, missing data, and invalid classifier assumptions. Thus, the critical objective of the new fusion system architecture is to permit graceful degradation rather than catastrophic failure whenever some components of the system, whether hardware or software,fail to perform adequately. Two tests were conducted on the trained discrimination algorithms: a blind test with simulated noisy signatures collected at angles different from those in the training database and a test using actual radar field test data collected during flight of a representative TBM target. The test results demonstrate a high degree of recognition accuracy. The paper describes the training and testing techniques used, show the fusion strategy employed, and illustrate the advantages of exploiting multi-level fusion.

Intelligent fusion processing in BMD applications

Intelligent processing techniques are applied to a ballistic missile defense (BMD) application, focused on classifying the objects in a typical threat complex, using fused IR and ladar sensors. These techniques indicate the potential to improve designation robustness against off-normal/unexpected conditions, or when sensor data or classifier performance degrades. A fused sensor discrimination (FuSeD) simulation testbed was assembled for designation experiments, to evaluate test and simulation data, assess intelligent processor and classification algorithms, and evaluate sensor performance. Results were produced for a variety of neural net and other nonlinear classifiers, yielding high designation performance and low false alarm rates. Most classifiers yield a few percent in false alarm rate; rates are further improved when multiple techniques are applied vi a majority based fusion technique. Example signatures, features, classifier descriptions, intelligent controller design, and architecture are included. Work was performed for the discriminating interceptor technology program (DITP).

Metric sensitivity of the multisensor information fusion process under instance-based learning

The study investigated the sensitivity of the instance-based- learning (IBL) driven multi-source information fusion process to the underlying distance metric. An audio-visual system for recognition of spoken French vowels is used as an example for this investigation. Three different distance measures, namely, Euclidian, city block and chess board metrics, are employed for this initial foray into metric sensitivity analysis. In this example, the test phase encompasses a broader range of noise environments of the audio signal as compared to the training phase. The system is thus exercised in both trained and untrained noise regimes. Under the untrained regime, interpolation as well as extrapolation or off-nominal scenarios are considered. In the former, the signal to noise ratio in the test phase is within the range used in training phase but does not specifically include it. In the latter, the signal to noise ratio in the test phase is outside the range used in the training phase. It is observed that while both of the single-sensor based decision systems individually are not very sensitive to the choice of the metric, the fused decision system is indeed significantly more sensitive to this choice. The city block metric offers better performance as compared to the other two in the case of the fused audio- visual system across most of the spectrum of noise environments, except for the extreme off-nominal conditions wherein the Euclidian offers slightly better performance. The chess board metric offers the lowest performance across the entire test range. The lack of training in the interpolation scenario has a noticeably strong effect on audio performance under the chess board metric.

Intelligent multi-classifier fusion for decision making in ballistic missile defense applications

In the context of a large project on reliable real-time identification of ballistic missile type targets through fusion of information from disparate sensors, a variety of decisions in-decision out (DEI-DEO) fusion processing techniques are being evaluated. This paper describes the experience of employing several conventional as well as approximate reasoning based classifiers and fusing the resulting decisions through these DEI-DEO fusion processors for identification of lethal targets in the presence of decoys and debris using simulated infrared sensor data. While no final conclusions can as yet be drawn based on this limited evaluation, the study confirms the basic premise that intelligent multiclassifier fusion offers the potential for more robust performance in a dynamic environment wherein a priori assumptions made regarding the individual classifiers may not always hold.

A trainable decisions-in decision-out (DEI-DEO) fusion system

Most of the decision fusion systems proposed hitherto in the literature for multiple data source (sensor) environments operate on the basis of pre-defined fusion logic, be they crisp (deterministic), probabilistic, or fuzzy in nature, with no specific learning phase. The fusion systems that are trainable, i.e., ones that have a learning phase, mostly operate in the features-in-decision-out mode, which essentially reduces the fusion process functionally to a pattern classification task in the joint feature space. In this study, a trainable decisions-in-decision-out fusion system is described which estimates a fuzzy membership distribution spread across the different decision choices based on the performance of the different decision processors (sensors) corresponding to each training sample (object) which is associated with a specific ground truth (true decision). Based on a multi-decision space histogram analysis of the performance of the different processors over the entire training data set, a look-up table associating each cell of the histogram with a specific true decision is generated which forms the basis for the operational phase. In the operational phase, for each set of decision inputs, a pointer to the look-up table learnt previously is generated from which a fused decision is derived. This methodology, although primarily designed for fusing crisp decisions from the multiple decision sources, can be adapted for fusion of fuzzy decisions as well if such are the inputs from these sources. Examples, which illustrate the benefits and limitations of the crisp and fuzzy versions of the trainable fusion systems, are also included.

Multilevel fusion exploitation

This paper describes a project that was sponsored by the U.S. Army Space and Strategic Defense Command (USASSDC) to develop, test, and demonstrate sensor fusion algorithms for target recognition. The purpose of the project was to exploit the use of sensor fusion at all levels (signal, feature, and decision levels) and all combinations to improve target recognition capability against tactical ballistic missile (TBM) targets. These algorithms were trained with simulated radar signatures to accurately recognize selected TBM targets. The simulated signatures represent measurements made by two radars (S-band and X- band) with the targets at a variety of aspect and roll angles. Two tests were conducted: one with simulated signatures collected at angles different from those in the training database and one using actual test data. The test results demonstrate a high degree of recognition accuracy. This paper describes the training and testing techniques used; shows the fusion strategy employed; and illustrates the advantages of exploiting multi-level fusion.

CORPS: class overlap region partitioning scheme--a tool for feature assessment

The paper presents a simple but effective feature assessment scheme that can be employed for a quick optimal evaluation of the individual discrimination potentials of a large number of features. The approach, Class Overlap Region Partitioning Scheme (CORPS), can be used either as a stand alone tool or as a front end to more complex combinatorial feature selection procedures such as branch and bound and genetic algorithms. The approach has the flexibility to permit imposition of a bias on the evaluation in favor of reducing either of the two possible types of errors in a binary decision process, for example false alarm or leakage in a target detection problem. Details of the associated algorithmic and operational procedures are furnished to facilitate wide usage of this new tool.

Sensor fusion options for ballistic missile defense interceptor applications

Critical elements of future exoatmospheric interceptor systems are intelligent processing techniques which can effectively combine sensor data from disparate sensors. This paper summarizes the impact on discrimination performance of several feature and classifier fusion techniques, which can be used as part of the overall IP approach. These techniques are implemented either within the fused sensor discrimination testbed, or off-line as building blocks that can be modified to assess differing fusion approaches, classifiers and their impact on interceptor requirements. Several optional approaches for combining the data at the different levels, i.e., feature and classifier levels, are discussed in this paper and a comparison of performance results is shown. Approaches yielding promising results must still operate within the timeline and memory constraints on board the interceptor. A hybrid fusion approach is implemented at the feature level through the use of feature sets input to specific classifiers. The output of the fusion process contains an estimate of the confidence in the data and the discrimination decisions. The confidence in the data and decisions can be used in real time to dynamically select different sensor feature data, classifies, or to request additional sensor data on specific objects that have not been confidently identified as lethal or non-legal. However, dynamic selection requires an understanding of the impact of various combinations of feature sets and classifier options. Accordingly, the paper presents the various tools for exploring these options and illustrates their usage with data sets generated to realistically simulate the world of Ballistic Missile Defense interceptor applications.

Optimal features-in feature-out (FEI-FEO) fusion for decisions in multisensor environments

The study presents a formal methodology to the problem of feature level fusion, that had been previously addressed in the literature mostly in an ad hoc manner on a case by case basis only. The input set of features extracted from multiple sensors (data sources) are optimally fused to derive a synthetic feature so as to enhance the effective discrimination potential among the defined set of decision classes. This `features in - feature out (FEI-FEO) fusion process, unlike most other fusion schemes reported in the literature, is designed through a formal learning phase in which an optimal mapping from the multi-sensor derived feature space to a single unified feature is developed. This learning, accomplished through a new composite random and deterministic search based optimization tool, defines the transformation for the FEI-FEO process. This transformation is applied to the multi-sensor generated feature sets in the operational phase to derive the fused feature values corresponding to the objects under observation. The corresponding classification decisions are made on the basis of relative closeness of these feature values to the different class mean values in the transformed single dimensional feature space. The new methodology has been implemented in MATLAB which, being a vector/matrix oriented language, is an ideal candidate for solving problems in pattern recognition and learning. The method is applied to well-known data sets available on the web for testing pattern recognition algorithms to assess its effectiveness relative to the traditional classification methods from both conceptual as well as computational view points.

Fuzzy evidential-reasoning-based decision fusion

The paper presents a new decision fusion methodology for identifying and tracking of multiple targets in a multisensor environment. The methodology combines concepts from the fuzzy logic and evidential reasoning domains to develop an integrated approach. The core methodology assumes that the sensors provide non-crisp or fuzzy labels, i.e., provide fuzzy class membership estimates corresponding to the different decision choices. However, the methodology can be adapted to environments wherein the sensors do not offer such information but only provide a single label deemed as the most likely. This is accomplished by having a learning phase wherein the performance of the sensors as compared to the ground truth is observed and utilized to derive fuzzy membership estimates corresponding to every individual sensor-decision scenario. The tracking mechanism is designed to also maintain fuzzy membership in different classes until the membership in any one class approaches unity. The fuzzy membership vectors, corresponding to the input sensor data as well as the tracks, include a measure of ignorance. This ignorance continuously decreases for the tracks as more and more track reports are integrated into the tracks. The paper presents details of the algorithmic process along with results as applied to some real-world data that demonstrates the effectiveness of the synergistic exploitation of the fuzzy logic and evidential reasoning concepts.