Alan N. Steinberg

Revisions to the JDL Data Fusion Model

The Data Fusion Model maintained by the Joint Directors of Laboratories (JDL) Data Fusion Group is the most widely-used method for categorizing data fusion-related functions. This paper discusses the current effort to revise the expand this model to facilitate the cost-effective development, acquisition, integration and operation of multi- sensor/multi-source systems. Data fusion involves combining information - in the broadest sense - to estimate or predict the state of some aspect of the universe. These may be represented in terms of attributive and relational states. If the job is to estimate the state of a people, it can be useful to include consideration of informational and perceptual states in addition to the physical state. Developing cost-effective multi-source information systems requires a method for specifying data fusion processing and control functions, interfaces, and associate databases. The lack of common engineering standards for data fusion systems has been a major impediment to integration and re-use of available technology: current developments do not lend themselves to objective evaluation, comparison or re-use. This paper reports on proposed revisions and expansions of the JDL Data FUsion model to remedy some of these deficiencies. This involves broadening the functional model and related taxonomy beyond the original military focus, and integrating the Data Fusion Tree Architecture model for system description, design and development.

Data fusion system engineering

The paper reports on methods for the cost-effective development and integration of multi-sensor fusion technology. The methods presented extend the Project Correlation Data Fusion Engineering Guidelines with significant evolution. The key new insight is in formulating the system engineering process as a resource management problem; allowing the application of the Bowmans model of the duality between data fusion and resource management.

Predictive modeling of interacting agents

A model for characterizing communications behaviors has been generalized to a probabilistic model for intentional behavior. Responsive actions are decomposed into measurement, inference, planning and control components) and are predicted as a function of the estimated capability, opportunity and intent of given agents to perform such component actions. This representational scheme enables the generation of process models for predicting capabilities development (technical intelligence) and tactical operations (operational intelligence).

Open Networks: Generalized Multi-Sensor Characterization

This paper examines issues in characterizing the performance of information sources as necessary for data fusion and coordination in a net-centric environment. In many practical applications, interacting agents have various degrees-and possibly time-varying degrees-of allegiance, common purpose, cooperativeness, information fidelity, controllability, etc. Agents share information with friends, foes and innocent bystanders alike, with varying degrees of cooperativeness and openness. In such cases, each network node needs to explicitly estimate the performance, trustworthiness and allegiance of all other contributing nodes as a part of the general multi-sensor/multi-target state estimation process. A sensors or information systems reporting bias-which may include intentional or unintentional human biases-is distinguished from its measurement bias. The problem is compared with that of measurement bias estimation, e.g. in target tracking. Formulations for estimation of biases in discrete variable reporting-e.g. in target classification or activity state reporting-are explored

Issues and challenges of the applications of context to enhance information fusion: panel summary

During the 2015 SPIE DSS conference, nine panelists were invited to highlight the trends and use of context for information fusion. This paper highlights the common issues and trends presented from the panel discussion. The different panelists highlighted methods of filtering methods, data aggregation, and the importance of context for realtime analytics. Using the panelist perspectives, the review organizes the common issues and themes as well areas of future analysis of content and context enrichment from information fusion.

Formalization of “Context” for Information Fusion

Context exploitation can provide benefits for information fusion by establishing expectations of world states, explaining received data, and resolving ambiguous interpretations; thereby improving process efficiency, reliability, and trustworthiness of the fusion product. While everybody recognizes the importance of considering context in inferencing, designers of information fusion processes only recently have begun to incorporate context explicitly into fusion processes. Effective context exploitation requires a clear understanding of what context is, how to represent it in a formal way, and how to use it for particular information fusion applications. Although these problems are similar to the ones discussed by researchers in many other fields, consideration of context in designing information fusion systems also poses additional challenges such as understanding the relationships between situations and context, utilizing context for understanding and fusion of natural language data, context dynamics, context recognition, and contextual reasoning under the uncertainty inherent in fusion problems. This chapter provides a brief discussion on possible ways of confronting these challenges while designing information fusion systems.

System-Level Use of Contextual Information

A system that exploits information—e.g. to support decision making—can use contextual information both in providing expectations and in resolving uncertain inferences. In the latter case, contextual reasoning involves inferring desired information (values of “problem variables”) on the basis of other available information (“context variables”). Relevant contexts are often not self-evident, but must be discovered or selected as a means to problem-solving. Therefore, context exploitation involves (a) predicting the value of contextual information to meet information needs; (b) selecting information types and sources expected to provide information useful in meeting those needs; (c) determining the relevance and quality of acquired information; and (d) applying selected information to a problem at hand. Fusion of contextual information can improve the quality of inferences, but involves concerns about the quality of the contextual information. The availability and quality of predictive models dictate the ways in which contextual information can be used. Many applications are benefitted by inference systems that adaptively discover and exploit context and refine such models to meet evolving information states and information needs.

Real-time composite feature extraction and visualization using pixel level fusion of active/passive data

A system has been developed whereby active LADAR and passive electro-optic (EO) imaging data are registered in hardware at the pixel level. For the sake of discussion, the sensor is herein called a “LADAR/EO Fusion Sensor” or LEFS. The resulting fully aligned high-dimension feature vector enhances target recognition and permits dense point matching for precise image mosaicking. A significant benefit is in combining the ability of pencil beam active systems to work at long ranges and to penetrate obscurants with the passive array’s wide instantaneous field of view at increased resolution. One application in which this has had enormous benefit is in observation through partial or intermittent obscuration; e.g. with partial cloud cover or foliage. Reflected radiation associated with features within gaps in the obscuration are sensed passively while at the same time the active pencil beam efficiently maps structure within the revealed region. Pixel-level registration of data permits extended regions to be mapped by combining temporally or spatially diverse collections. This data is convenient for estimating the probability of detection and recognition in cluttered environments. Given knowledge of the nature of the clutter provided by the LEFS, the probability of target presence (prior and posterior) can be better estimated. A temporally evolving target detectability map can be produced and overlaid on a target expectation map to facilitate an understanding of the likelihood of false and missed detections. Methods for estimating the length of time for which a targeting decision can be deferred as well as for estimating the probability of resolving ambiguity in time are presented. Military applications for which this technology is being developed or assessed include precision tactical targeting, Precision Controlled Reference Image Base (CRIB) production and Automatic Registration of targeting data into the CRIB. Civil applications include 3D city modelling, real-time airborne mapping, postdisaster reconnaissance, floodplain and coastline mapping, drug interdiction target detection, environmental monitoring, and search and rescue. Introduction The probability of detection and identification of targets in cluttered environments is influenced by clutter density and the associated ability of a remote sensor to see through holes in clutter. When acquiring imagery from low-flying dynamic platforms such as small UAVs or ground-based sensors used by special forces, often only glimpses are possible though holes between trees or into steep canyons. For a given EO/IR image, only a limited number of ground patches might be visible. When a second image is coregistered with the first, the number and size of patches is increased. Given collection from enough viewpoints, it is theoretically possible to piece together enough patches to view a significant portion of the scene behind the clutter and discover otherwise obscured targets. Piecing things together accurately poses a major problem without an accurate determination of the fine details of the clutter geometry, i.e. the shape of trees and the geometry of the intervening holes. The compilation of this complex geometry through stereoscopic techniques is problematic when many points on the ground are only viewable through one hole seen from one vantage point only. This paper presents an approach to extracting EO data from multiple holes in clutter with the assistance of a range sensor such as LADAR that is fused at the pixel-level. The resulting 3D “patch data” provides an opportunity to characterize the viewability of targets in clutter and to compute the