Sergei Astapov

Data to decision: pushing situational information needs to the edge of the network

Obtaining a high level of situation awareness while maintaining optimal utilization of resources is becoming increasingly important, especially in the context of asymmetric warfare, where information superiority is crucial for maintaining the edge over the opponent. Obtaining an adequate level of situational information from an ISR system is dependent on sensor capabilities as well as the ability to cue the sensors appropriately based on the current information needs and the ability to utilize the collected data with suitable data processing methods. Applying the Data to Decision approach for managing the behavior of sensor systems facilitates optimal use of sensor assets while providing the required level of situational information. The approach presented in the paper combines the Data to Decision approach with the Fog Computing paradigm, where the computation is pushed to the edge of the network. This allows to take advantage of Big Data potentially generated by the sensor systems while keeping the resource requirements in terms of bandwidth manageable. We suggest a System of Systems approach for assembling the ISR system, where individual systems have a high level of autonomy and the computational resources to perform the necessary computation tasks. To facilitate a composition of a System of Systems of sensors for tactical applications the proactive middleware ProWare is applied. The work presented in the paper has been conducted as part of the European Defense Agency project IN4STARS, in the context of which an implementation of a sensor solution is being built, which facilitates on-line sensor cueing and collaboration between sensors by building upon the Fog Computing paradigm and utilizing the Data to Decision concepts.

Simplified acoustic localization by linear arrays for wireless sensor networks

The application of wireless sensor networks (WSN) for the task of acoustic localization provides great opportunities for distributed cooperative tracking of sound sources in large areas. However WSNs are significantly more limited in terms of computational resources and power than typical computer systems. Therefore the methods applied for acoustic localization in WSN must be optimized for minimal resource consumption. This paper builds on the advances of Steered Response Power with Phase Transform (SRP-PHAT) optimization and proposes a further simplification in terms of additional minimization of the initial search volume. By using several linear microphone arrays we are able to estimate the initial region of sound source and reduce the number of computations by at least one order of magnitude. The results of several experiments on real signals confirm the achieved improvements.

A method of real-time mobile vehicle identification by means of acoustic noise analysis implemented on an embedded device

The paper considers a novel method of mobile vehicle identification based on acoustic signal analysis and the implementation of the method on a specific embedded device. The algorithm is designed as a multistage decision-making scheme, which involves frequency domain feature extraction, fuzzy classification, correlation analysis and signal dynamics monitoring. The implementation of the system is tested in real-time conditions on an embedded platform. The results of processing time measurements indicate the ability to operate in real-time with several different signal frame lengths.

Object detection for military surveillance using distributed multimodal smart sensors

This paper considers an autonomous ground Intelligence, Surveillance and Reconnaissance (ISR) system comprising of multiple distributed, wirelessly communicating smart sensors. The ISR system, in turn, is a part of a larger System of Systems (SoS) consisting of aerial, manned, etc. surveillance systems and information collection centers. The smart sensors of the ISR system perform environment monitoring using different modalities and exchange object detection and identification results to assess the situation and provide other SoS components with this information. In the paper we discuss using acoustic, magnetic and Passive Infrared (PIR) sensor information for target detection and identification. We also propose an approach of distributed acoustic source localization and a method of velocity estimation using PIR data. For sensor communication an asynchronous ad-hoc WSN configuration is proposed. The system is implemented on low power smart sensors utilizing Atmel ATmega128RFA1 processors with integrated 2.4GHz IEEE 802.15.4 compliant radio transceivers.

A two-stage approach to 2D DOA estimation for a compact circular microphone array

Microphone arrays and, specifically, circular arrays have been used for sound source localization and multimedia applications for more than a decade. In recent years the development of compact arrays for implementation in Wireless Sensor Networks (WSN) has risen in popularity. This paper considers a 2D Direction of Arrival (DOA) estimation method for a compact circular array, equipped with additional vertically placed microphones. The proposed method is aimed at reducing the computational cost of DOA estimation for implementation on embedded hardware of WSN smart sensors. The method is compared with a well known localization algorithm of SRP-PHAT and is proven to provide adequate DOA estimates, while being more computationally effective.

Collective acoustic localization in a network of dual channel low power devices

Situation awareness is an important aspect of ubiquitous computer systems, as these systems of systems are highly integrated with the physical world and for successful operation they must maintain high awareness of the environment. Acoustic information is one of the most popular modalities, by which the environment states are estimated. Multi-sensor approaches also provide the possibility for acoustic source localization. This paper considers an acoustic localization system of dual channel smart sensors interconnected through a Wireless Sensor Network (WSN). The low computational power of smart sensor devices requires distribution of localization tasks among WSN nodes. The Initial Search Region Reduction method is used in the WSN to meet this requirement. The system is implemented on smart dust motes utilizing Atmel ATmega128RFA1 processors with integrated 2.4GHz IEEE 802.15.4 compliant radio transceivers. The paper discusses complications, introduced by low power hardware, and reviews conditions of real-time operation.

Situation awareness via Internet of things and in-network data processing

Computing on the edge of the Internet of things comprises among other tasks in-sensor signal processing and performing distributed data fusion and aggregation at network nodes. This poses a challenge to distributed sensor networks of low computing power devices that have to do complex fusion, aggregation and signal processing in situ. One of the difficulties lies in ensuring validity of data collected from heterogeneous sources. Ensuring data validity, for example, the temporal and spatial correctness of data, is crucial for correct in-network data fusion and aggregation. The article considers wireless sensor technology in military domain with the aim of improving situation awareness for military operations. Requirements for contemporary intelligence, surveillance and reconnaissance applications are explored and an experimental wireless sensor network, designed to enhance situation awareness to both in-the-field units and remote intelligence operatives, is described. The sensor nodes have the capability to perform in-sensor signal processing and distributed in-network data aggregation and fusion complying with edge computing paradigm. In-network data processing is supported by service-oriented middleware which facilitates run-time sensor discovery and tasking and ad hoc (re)configuration of the network links. The article describes two experiments demonstrating the ability of the wireless sensor network to meet intelligence, surveillance and reconnaissance requirements. The efficiency of distributed data fusion is evaluated and the importance and effect of establishing data validity is shown.

Sound localization and processing for inducing synesthetic experiences in Virtual Reality

Synesthesia is the act of experiencing one sense modality as another, e.g., a person may vividly experience flashes of colors when listening to a series of sounds. Recent technological advances in the Virtual and Augmented Reality field allow to induce such experiences due to the effect of presence achieved in the virtual environment. In this paper, we present initial results related to a particular synesthetic experience; namely, we focus on the visual interpretation of sound. Towards that end, we apply a series of recently developed methods for detecting the source of sound in a three-dimensional space around the listener, and transform the audio signal into a series of visual shapes, such that the size of each shape is determined by the loudness of the sound source, and its color is determined by the dominant spectral component of the sound. Experimental results are provided and discussed.

Detection, identification and tracking of mobile objects with distributed system of systems

Tracking and identification of mobile objects is a challenging task, which is made even more challenging if the task is executed in a distributed manner by a System of Systems (SoS). However, the distributed SoS solution has many advantages when compared to centralized approaches. The computational load can be distributed, the fidelity of the tracking and identification results is potentially increased with the increased number of sensing points and there is a potential to decrease communication loads when compared to centralized approaches that use multiple sensing points. The paper presents a distributed tracking and identification approach where both the computation and sensing are distributed among network nodes using a SoS approach. The results show that such an approach is feasible and the accuracy is satisfactory.

Gunshot acoustic event identification and shooter localization in a WSN of asynchronous multichannel acoustic ground sensors

Gunshot acoustic localization for military and civilian security systems has long been an important topic of research. In recent years the development of Wireless Sensor Network (WSN) systems of independent Unmanned Ground Sensors (UGS) performing distributed cooperative localization has grown in popularity. This paper considers a shooter localization approach based on gunshot Shockwave (SW) and Muzzle Blast (MB) event time and Direction of Arrival (DOA) information. The approach accounts for acoustic events Not-of-Interest (NOI), such as target hit noise, reflections and background noise. UGS perform gunshot acoustic event detection and DOA estimation independently; the information regarding every detected shot instance is sent through the WSN to the fusion node, which performs event identification and calculates the shooter’s position. The paper presents a solution to identifying SW and MB among NOI events at the stage of information fusion. The considered approach treats the information gathered from different UGS separately, and thus does not require precise synchronization between the UGS. For DOA estimation, an algorithm designed for circular microphone arrays is proposed and compared with the SRP-PHAT localization algorithm. It is shown to provide adequate DOA estimates, while being more computationally effective. The proposed shooter localization approach is tested on real signals, acquired during three live shooting experiments. It is shown to succeed in localizing the shooter’s position with a mean accuracy of 0.87 m for 30 shots at the range of 35 m, and just above 7 m for 37 shots at the range of 100 m.