Achim Kuwertz

Using heterogeneous multilevel swarms of UAVs and high-level data fusion to support situation management in surveillance scenarios

The development and usage of Unmanned Aerial Vehicles (UAVs) quickly increased in the last decades, mainly for military purposes. This technology is also now of high interest in non-military contexts like logistics, environmental studies and different areas of civil protection. While the technology for operating a single UAV is rather mature, additional efforts are still necessary for using UAVs in fleets (or swarms). The Aid to SItuation Management based on MUltimodal, MUltiUAVs, MUltilevel acquisition Techniques (ASIMUT) project which is supported by the European Defence Agency (EDA) aims at investigating and demonstrating dedicated surveillance services based on fleets of UAVs. The aim is to enhance the situation awareness of an operator and to decrease his workload by providing support for the detection of threats based on multi-sensor multi-source data fusion. The operator is also supported by the combination of information delivered by the heterogeneous swarms of UAVs and by additional information extracted from intelligence databases. As a result, a distributed surveillance system increasing detection, high-level data fusion capabilities and UAV autonomy is proposed.

ISR analytics: Architectural and methodic concepts

Prevention and management of damage scenarios require adequate situation awareness to make timely, coordinated, and proactive decisions possible. The stakeholders must be able to access and to comprehend relevant information quickly and with justifiable effort. The resulting challenges for intelligence, surveillance, and reconnaissance (ISR) lie not only in the new and further development of individual sensor and exploitation systems but also in interoperable system networking as well as in the realization of adequate strategies for the collection, processing, dissemination, and presentation of data and information products [1], [2], [3], [4]. In this publication, we present a high level architecture for ISR analytics that complies with these observations. It provides the functionality to customize the system precisely to specific scenarios of the ISR domain. We give a more detailed insight into concepts and approaches that are essential for specific architecture components.

Multi-step sensor management for localizing movable sources of spatially distributed phenomena

Localizing sources of physical quantities is often only possible in an indirect manner by observing the induced continuous phenomena, such as pollution loads of air or water. By employing model-based reconstruction methods, the task of localizing movable sources by distributed sensor measurements can be formulated as a non-linear stochastic parameter estimation problem. A computationally efficient state estimator is applied to this estimation problem for enabling real-time source localization. Furthermore, this paper proposes a novel approach to multistep sensor management for utilizing future sensors measurements in a most informative way. Here, predictive statistical linearization is employed for converting the given nonlinear non-Gaussian sensor management problem into a linear Gaussian one, which can be solved efficiently. By controlling a mobile sensor, it is demonstrated that the proposed method yields accurate source localization results.

ASIMUT project: Aid to SItuation Management based on MUltimodal, MUltiUAVs, MUltilevel acquisition Techniques

This document summarizes the activities and results of the ASIMUT project (Aid to SItuation Management based on MUltimodal, MUltiUAVs, MUltilevel acquisition Techniques) carried out by the consortium composed of Thales, Fraunhofer IOSB, Fly-n-Sense, University of Bordeaux and University of Luxembourg. Funded by the European Defence Agency (EDA), the objectives of the ASIMUT project are to design, implement and validate algorithms that will allow the efficient usage of autonomous swarms of Unmanned Aerial Vehicles (UAVs) for surveillance missions.

High-level data fusion component for drone classification and decision support in counter UAV

Today, drone technology has been made available around the world. Anyone can purchase a drone from an online retailer. Government agencies and military are seeing a rise in drones used for terrorism, destruction and espionage. The emergence of threats caused by unfriendly or hostile drones requires proactive drone detection in order to decide on appropriate defensive actions. In this contribution, a high-level data fusion component for drone classification is presented. The high-level data fusion component is part of our counter UAV system MODEAS including decision support. The component provides well-defined interfaces which allow it to be integrated also into other counter UAV systems. The aim of the high-level data fusion component is to support an operator in his decision making by providing detailed information about detected drones together with assigned threat levels. To identify a detected and tracked drone with sufficient detail, a knowledge-based classification is performed, based on background knowledge like drone model specifications. By fusing the knowledge-based classification results with prior results of a sensor-based classification, the overall classification is improved. The fusion results, in addition to kinematic data, also contain specific capabilities of the respective drone like its maximum payload, endurance, and speed as well as recorded incidents with similar drones or their typical (commercial) usage, if known. Based on these fusion results, a threat analysis is performed. The component’s output then is a ranked list of dossiers for the most probable types of drones with regard to the observation data and their assigned threat levels.

Integrating coalition shared data in a system architecture for high level information management

Globalization has created complex economic and sociological dependencies. The nature of conflicts has changed and nations are confronted with a vast number of new threat scenarios. Information superiority is a question of being able to get the right information at the right time. Technology allows to disseminate information in near real-time and enables both aggressors and defenders to act remotely and network over time and space. Technologies in the areas of sensors and platforms as well as network technology and storage capability have evolved to a level where mass data can be easily shared and disseminated. To benefit fully from these new capabilities, there is a need for systems and services that can interact with each other in a well-defined interoperable way. On an organizational level it is necessary to define common processes to coordinate actors, their activities, the assets available and the data and information created. Security restrictions, (intellectual) property rights as well as data privacy regulations need to be fulfilled. The Coalition Shared Data (CSD) concept supports operational processes as defined by NATO within Joint ISR (Intelligence, Surveillance and Reconnaissance) and the Intelligence Cycle by defining standardized interfaces, data models, services and workflows. To support information provision additionally, techniques of data and information extraction, fusion and visual analysis can be added at the system level. Other available sources can be connected through the usage of semantic world models. To ensure data integrity multilevel security measures need to be combined with the existing concept. The publication introduces the operational processes defined within NATO doctrines and process descriptions and maps the CSD concept to it. It describes the new Edition of STANAG (NATO Standardization Agreement) 4559 Edition 4 that implements the CSD concept and connects it to operational processes. Based on this it introduces a system architecture for ISR Analytics.

Open architecture of smart sensor suites

Experiences from recent conflicts show the strong need for smart sensor suites comprising different multi-spectral imaging sensors as core elements as well as additional non-imaging sensors. Smart sensor suites should be part of a smart sensor network – a network of sensors, databases, evaluation stations and user terminals. Its goal is to optimize the use of various information sources for military operations such as situation assessment, intelligence, surveillance, reconnaissance, target recognition and tracking. Such a smart sensor network will enable commanders to achieve higher levels of situational awareness. Within the study at hand, an open system architecture was developed in order to increase the efficiency of sensor suites. The open system architecture for smart sensor suites, based on a system-of-systems approach, enables combining different sensors in multiple physical configurations, such as distributed sensors, co-located sensors combined in a single package, tower-mounted sensors, sensors integrated in a mobile platform, and trigger sensors. The architecture was derived from a set of system requirements and relevant scenarios. Its mode of operation is adaptable to a series of scenarios with respect to relevant objects of interest, activities to be observed, available transmission bandwidth, etc. The presented open architecture is designed in accordance with the NATO Architecture Framework (NAF). The architecture allows smart sensor suites to be part of a surveillance network, linked e.g. to a sensor planning system and a C4ISR center, and to be used in combination with future RPAS (Remotely Piloted Aircraft Systems) for supporting a more flexible dynamic configuration of RPAS payloads.

Towards Web-Based Semantic Knowledge Completion for Adaptive World Modeling in Cognitive Systems

This contribution proposes web-based concept completion, an approach for enriching concept descriptions created from observations with additional semantic information. The proposed methods are designed as an extension to the Object-Oriented World Model (OOWM), a system for environment representation in cognitive systems using semantic knowledge captured in a domain model. An adaptive version of the OOWM allows extending the domain model with new concept descriptions created from observations. Such sensor observations yet only provide the perceptual information part of a concept description and lack semantic information. To this end, this contribution is concerned with selecting and integrating suitable resources readily available from the World Wide Web into a demonstrator for concept completion.

Modellbasierte Quellenverfolgung in räumlich ausgedehnten Phänomenen mittels Sensoreinsatzplanung

Zusammenfassung Bewegte Quellen können durch Emission räumlich ausgedehnte Phänomene wie beispielsweise Schadstoff- oder Temperaturverteilungen erzeugen. Zur Lokalisierung von Quellen mit unbekannter Position stehen in vielen Aufgabenstellungen Informationen nur indirekt durch die verteilte Vermessung des induzierten Phänomens zur Verfügung — etwa unter Verwendung stationärer oder mobiler Sensoren. Dieser Beitrag stellt modellbasierte Verfahren für eine echtzeitfähige Lokalisierung und Verfolgung von bewegten Quellen vor. Zur gezielten Maximierung des Informationsgehalts der Messungen wird dabei eine vorausschauende Sensoreinsatzplanung genutzt, welche eine hohe Lokalisierungsgüte bei geringem Aufwand ermöglicht. Abstract Space-time continuous phenomena such as pollution loads or temperature distributions often originate from unknown and possibly movable sources. In many real-world scenarios, however, information about the location of such sources can only be gained indirectly by monitoring the induced physical phenomena using distributed sensing systems, e. g., stationary or mobile sensors. In this article, a model-based approach for real-time source localization and target tracking is introduced. To maximize specific information gained from the measurements, this approach strongly relies on a non-myopic sensor management methodology, which allows for tracking moving sources in an efficient and accurate manner.