Jean Roy

Ontology alignment in geographical hard-soft information fusion systems

Information fusion exists over many forms of hard data (e.g. from physical sensors) and soft data (e.g. from human reports) to interpret observations of real-world objects. As demonstrated from the Geographical Information Systems (GIS) community, there is a growing need for the linking and alignment of both (1) exploited physical imagery products and (2) derived ontological textual labels (semantic markup). Semantic markup can be done on both exploited data (e.g. automated image segmentation), as well as user reports (e.g. weather forecasts). Since the derived information is collected, stored, and displayed into distinct ontological structures by different agencies; ontological alignment is thus required whenever the semantic information is paired with distinct real-world imagery observations. In this paper, we explore issues of fusing hard and soft data as related to ontology alignment. A maritime domain situational awareness example with geographical imagery and textural ontologies is shown to demonstrate the need for ontology alignment to assist users for pragmatic surveillance.

Rule-based expert system for maritime anomaly detection

Maritime domain operators/analysts have a mandate to be aware of all that is happening within their areas of responsibility. This mandate derives from the needs to defend sovereignty, protect infrastructures, counter terrorism, detect illegal activities, etc., and it has become more challenging in the past decade, as commercial shipping turned into a potential threat. In particular, a huge portion of the data and information made available to the operators/analysts is mundane, from maritime platforms going about normal, legitimate activities, and it is very challenging for them to detect and identify the non-mundane. To achieve such anomaly detection, they must establish numerous relevant situational facts from a variety of sensor data streams. Unfortunately, many of the facts of interest just cannot be observed; the operators/analysts thus use their knowledge of the maritime domain and their reasoning faculties to infer these facts. As they are often overwhelmed by the large amount of data and information, automated reasoning tools could be used to support them by inferring the necessary facts, ultimately providing indications and warning on a small number of anomalous events worthy of their attention. Along this line of thought, this paper describes a proof-of-concept prototype of a rule-based expert system implementing automated rule-based reasoning in support of maritime anomaly detection.

Modeling and simulation in support of the design of a data fusion system

Abstract This paper uses a modeling and simulation approach in the engineering process of designing a data fusion system for military applications. Fundamental issues in the design of a data fusion system are the selection of appropriate architecture and efficient and dedicated real-time algorithms for the application of interest. Selection of data fusion algorithms is performed using a simulation test bed where realistic modeling of sensor performance is being performed. The test bed also allows the use of multiple algorithms in a dynamic sense based on the particular regime in which the fusion system is operating. This paper first presents the data fusion system engineering guidelines that are being used. Then the paper addresses the complex problem of the tactical picture compilation. Finally the paper shows a case study where the selection of data fusion algorithms is performed for an airborne platform.

Fusion of identity declarations from dissimilar sources using the Dempster-Shafer theory

The problem of fusing identity declarations emanating from different sources is explored and decision makers are offered a quanti- tative analysis based on statistical methodology that can enhance their decision making processes regarding the identity of detected objects. The context is naval warfare where commanders and their staff require access to a wide range of information to carry out their duties. This information provides them with the knowledge necessary to determine the position, identity and behavior of the enemy. Statistical analysis rooted in the Dempster-Shafer theory of evidence is used to further the fusion of identity declarations.

Human-computer interface for the study of information fusion concepts in situation analysis and command decision support systems

Situation Awareness (SAW) is essential for commanders to conduct decision-making (DM) activities. Situation Analysis (SA) is defined as a process, the examination of a situation, its elements, and their relations, to provide and maintain a product, i.e., a state of SAW for the decision maker. Operational trends in warfare put the situation analysis process under pressure. This emphasizes the need for a real-time computer-based Situation analysis Support System (SASS) to aid commanders in achieving the appropriate situation awareness, thereby supporting their response to actual or anticipated threats. Data fusion is clearly a key enabler for SA and a SASS. Since data fusion is used for SA in support of dynamic human decision-making, the exploration of the SA concepts and the design of data fusion techniques must take into account human factor aspects in order to ensure a cognitive fit of the fusion system with the decision-maker. Indeed, the tight human factor aspects in order to ensure a cognitive fit of the fusion system with the decision-maker. Indeed, the tight integration of the human element with the SA technology is essential. Regarding these issues, this paper provides a description of CODSI (Command Decision Support Interface), and operational- like human machine interface prototype for investigations in computer-based SA and command decision support. With CODSI, one objective was to apply recent developments in SA theory and information display technology to the problem of enhancing SAW quality. It thus provides a capability to adequately convey tactical information to command decision makers. It also supports the study of human-computer interactions for SA, and methodologies for SAW measurement.

The Canada-Netherlands collaboration on multisensor data fusion and other Canada-NATO MSDF activities

This paper describes a collaborative effort between Canada and The Netherlands in analyzing multi-sensor data fusion systems. In view of the overlapping interest in studying and comparing applicability and performance of advanced state-of-the-art multi-sensor data fusion (MSDF) techniques, the two research establishments involved (DREV and TNO/FEL) have decided to join their efforts in the development of MSDF testbeds. This resulted in the Joint-FACET (Fusion Algorithms and Concepts Exploration Testbed), a highly modular and flexible series of applications that is capable of processing both real and synthetic input data. In addition to the bilateral collaboration with The Netherlands, this paper presents a survey of the other Canadian MSDF activities conducted under NATO Research Study Groups.

Quantitative comparison of sensor fusion architectural approaches in an algorithm-level test bed

This paper presents the results of a quantitative comparison of two architectural options in developing a multi-sensor data fusion system. One option is the centralized architecture: a single track file is maintained and updated using raw sensor measurements. The second option is the autonomous sensor fusion architecture: each sensor maintains its own track file. The sensor tracks are then transmitted to a central processor responsible for fusing this data to form a master track file. Various performance trade-offs will typically be required in the selection of the best multi-sensor data fusion architecture since each approach has different benefits and disadvantages. The emphasis for this study is given to measuring the quality of the fused conducted with the CASE_ATTI (concept analysis and simulation environment for automatic target racking and identification) testbed. This testbed provides the algorithm-level test and replacement capability required to conduct this kind of performance study.

Hierarchical adaptation scheme for multiagent data fusion and resource management in situation analysis

Sensor Management (SM) has to do with how to best manage, coordinate and organize the use of sensing resources in a manner that synergistically improves the process of data fusion. Based on the contextual information, SM develops options for collecting further information, allocates and directs the sensors towards the achievement of the mission goals and/or tunes the parameters for the realtime improvement of the effectiveness of the sensing process. Conscious of the important role that SM has to play in modern data fusion systems, we are currently studying advanced SM Concepts that would help increase the survivability of the current Halifax and Iroquois Class ships, as well as their possible future upgrades. For this purpose, a hierarchical scheme has been proposed for data fusion and resource management adaptation, based on the control theory and within the process refinement paradigm of the JDL data fusion model, and taking into account the multi-agent model put forward by the SASS Group for the situation analysis process. The novelty of this work lies in the unified framework that has been defined for tackling the adaptation of both the fusion process and the sensor/weapon management.

Performance comparison of contact-level and track-level sensor fusion architectures

The results of a quantitative comparison of two architectural options in developing a multisensor data fusion system are presented. One option is the contact-level or centralized architecture: a single track file is maintained and updated using raw sensor measurements. The second option is the autonomous or track-level sensor fusion architec- ture: each sensor maintains its own track file. The sensor tracks are then transmitted to a central processor responsible for fusing this data to form a master track file. The emphasis for this study is given to measuring the quality of the fused product and the bandwidth of the communications required between the sensors and the fusion center. The evaluation is conducted with the Concept Analysis and Simulation Environment for Automatic Target Tracking and Identification testbed. This testbed pro- vides the algorithm-level test and replacement capability required to con- duct this kind of performance study.

Efficient cluster management algorithm for multiple-hypothesis tracking

This paper presents a detailed discussion of clustering as applied to multiple hypothesis tracking (MHT). The combinatorial problem associated with forming multiple data association hypotheses can be reduced significantly by partitioning the entire set of system tracks and input data elements into separate clusters. Cluster management, a process that deals with cluster formation, merging, splitting and deletion, is thus motivated by the idea that a large tracking problem can be divided into a number of smaller problems that can be solved independently. The paper emphasizes on the cluster splitting process since it is the most difficult aspect of clustering while being an often neglected issue in the target tracking literature. The hypothesis dependencies that must be taken into account when one attempts to split the hypothesis tree of a cluster into two or more independent trees are discussed. This is an important issue since the hypotheses within a cluster must not interact with the hypotheses contained within other clusters for the MHT technique to remain consistent. A very efficient algorithm is described that performs a combined split-merge process simultaneously for all the clusters. The algorithm has been designed to avoid a waste of computer resources that may happen when splitting clusters that should have been kept merged according to the most recent input data set. The dynamic data structure that is used to implement the hypothesis tree is described as a key element of the approach efficiency. An example of cluster management is presented.