Joakim Rydell

Accurate and reliable soldier and first responder indoor positioning: multisensor systems and cooperative localization

A robust, accurate positioning system with seamless outdoor and indoor coverage is a highly needed tool for increasing safety in emergency response and military urban operations. It must be lightweight, small, inexpensive, and power efficient, and still provide meter-level accuracy during extended operations. GPS receivers, inertial sensors, and local radio-based ranging are natural choices for a multisensor positioning system. Inertial navigation with foot-mounted sensors is suitable as the core system in GPS denied environments, since it can yield meter-level accuracies for a few minutes. However, there is still a need for additional supporting sensors to keep the accuracy at acceptable levels during the duration of typical soldier and first responder operations. Suitable aiding sensors are three-axis magnetometers, barometers, imaging sensors, Doppler radars, and ultrasonic sensors. Further more, cooperative positioning, where first responders exchange position and error estimates in conjunction with performing radio based ranging, is deemed a key technology. This article provides a survey on technologies and concepts for high accuracy soldier and first responder positioning systems, with an emphasis on indoor positioning.

Fusion of acoustic and optical sensor data for automatic fight detection in urban environments

We propose a two-stage method for detection of abnormal behaviours, such as aggression and fights in urban environment, which is applicable to operator support in surveillance applications. The proposed method is based on fusion of evidence from audio and optical sensors. In the first stage, a number of modality-specific detectors perform recognition of low-level events. Their outputs act as input to the second stage, which performs fusion and disambiguation of the firststage detections. Experimental evaluation on scenes from the outdoor part of the PROMETHEUS database demonstrated the practical viability of the proposed approach. We report a fight detection rate of 81% when both audio and optical information are used. Reduced performance is observed when evidence from audio data is excluded from the fusion process. Finally, in the case when only evidence from one camera is used for detecting the fights, the recognition performance is poor.

CHAMELEON: Visual-inertial indoor navigation

This paper presents a study of an indoor navigation system, Chameleon. The system is based on simultaneous localization and mapping (SLAM), using a stereo camera combined with an inertial measurement unit (IMU). Results from a number of navigation experiments are shown. Some experiments were performed in parallel with a foot-mounted system, Centipede, and results from this system are shown as a comparison. Additionally, the ability of the camera-based system to generate building maps is demonstrated.

Crowd behavior analysis under cameras network fusion using probabilistic methods

The use of cameras in surveillance is increasing in the last years due to the low cost of the sensor and the requirement by surveillance in public places. However, the manual analysis of this data is impracticable. Thus, automatic and robust methods to processing this high quantity of data are required. This paper proposes a framework to address this problem. The crowd analysis is achieved in camera networks information by using the optical flow. The Hidden Markov models and Bayesian Networks are compared to understand the agents behavior in the scene. The experimental results are obtained for several sequences where fight and robbery occurs. Results are promise in order to get an automatic system to find abnormal events.

Scenario-based evaluations of high-accuracy personal positioning systems

Foot-mounted inertial sensors combined with GPS-receivers, magnetometers, and barometric pressure sensors have shown great potential in providing high-accuracy positioning systems for first responder and military applications. Several factors, including the type of movement, surface, and the shape of the trajectory, can strongly influence the performance of foot-mounted inertial navigation systems. There is a need for realistic scenario-based evaluations as a complement to the controlled environment tests that have been published in the literature. In this work we evaluate the performance of a foot-mounted inertial navigation system using three-axis accelerometers, gyroscopes and magnetometers during realistic scenario-based measurements. The position accuracy is evaluated by using a camera-based reference system which positions itself towards visual markers placed at pre-surveyed positions, using a slightly modified version of the ARToolKitPlus software. Maximum position errors of 2.5 to 5.5 meters were obtained during four separate high-tempo building clearing operations that lasted approximately three and a half minutes each. Further improvements in accuracy, as well as improved robustness towards different movement patterns, can be achieved by implementing an adaptive stand-still detection algorithm.

Chameleon on fire — Thermal infrared indoor positioning

In this paper we present a system for positioning and mapping, primarily for use in smoke diver applications. The system is based on a stereo pair of thermal infrared cameras and is shown to produce trajectory and mapping estimates while used in environments with sufficient thermal contrast. The system is evaluated in, e.g., a test facility for smoke divers, with good results.

Geometric calibration of thermal cameras

There exist several tools and methods for camera resectioning, i.e. geometric calibration for the purpose of estimating intrinsic and extrinsic parameters. The intrinsic parameters represent the internal properties of the camera such as focal length, principal point and distortion coefficients. The extrinsic parameters relate the cameras position to the world, i.e. how is the camera positioned and oriented in the world. With both sets of parameters known it is possible to relate a pixel in one camera to the world or to another camera. This is important in many applications, for example in stereo vision. The existing methods work well for standard visual cameras in most situations. Intrinsic parameters are usually estimated by imaging a well-defined pattern from different angles and distances. Checkerboard patterns are very often used for calibration since it is a well-defined pattern with easily detectable features. The intersections between the black and white squares form high contrast points which can be estimated with sub pixel accuracy. Knowing the precise dimension and structure of the pattern makes enables calculation of the intrinsic parameters. Extrinsic calibration can be performed in a similar manner if the exact position and orientation of the pattern is known. A common method is to distribute markers in the scene and to measure their exact locations. The key to good calibration is well-defined points and accurate measurements. Thermal cameras are a subset of infrared cameras that work with long wavelengths, usually between 9 and 14 microns. At these wavelengths all objects above absolute zero temperature emit radiation making it ideal for passive imaging in complete darkness and widely used in military applications. The issue that arises when trying to perform a geometric calibration of a thermal camera is that the checkerboard emits more or less the same amount of radiation in the black squares as in the white. In other words, the calibration board that is optimal for calibration of visual cameras might be completely useless for thermal cameras. A calibration board for thermal cameras should ideally be a checkerboard with high contrast in thermal wavelengths. (It is of course possible to use other sorts of objects or patterns but since most tools and software expect a checkerboard pattern this is by far the most straightforward solution.) Depending on the application it should also be more or less portable and work booth in indoor and outdoor scenarios. In this paper we present several years of experience with calibration of thermal cameras in various scenarios. Checkerboards with high contrast both for indoor and outdoor scenarios are presented as well as different markers suitable for extrinsic calibration.

Positioning evaluation) 2

This paper presents a camera-based reference system for positioning. The system provides reliable ground truth data in relatively large indoor environments using a minimum of preinstalled infrastructure. The reference system will be used together with a foot-mounted inertial navigation system (INS) with the purpose to evaluate how errors in the foot-mounted INS grow over time. The reference system itself is evaluated and shown to provide sufficient accuracy.

Sensor fusion for improved indoor navigation

A reliable indoor positioning system providing high accuracy has the potential to increase the safety of first responders and military personnel significantly. To enable navigation in a broad range of environments and obtain more accurate and robust positioning results, we propose a multi-sensor fusion approach. We describe and evaluate a positioning system, based on sensor fusion between a foot-mounted inertial measurement unit (IMU) and a camera-based system for simultaneous localization and mapping (SLAM). The complete system provides accurate navigation in many relevant environments without depending on preinstalled infrastructure. The camera-based system uses both inertial measurements and visual data, thereby enabling navigation also in environments and scenarios where one of the sensors provides unreliable data during a few seconds. When sufficient light is available, the camera-based system generally provides good performance. The foot-mounted system provides accurate positioning when distinct steps can be detected, e.g., during walking and running, even in dark or smoke-filled environments. By combining the two systems, the integrated positioning system can be expected to enable accurate navigation in almost all kinds of environments and scenarios. In this paper we present results from initial tests, which show that the proposed sensor fusion improves the navigation solution considerably in scenarios where either the foot-mounted or camera-based system is unable to navigate on its own.

Threat detection and tracking using 3D FLASH ladar data

Laser-based 3D sensors measure range with high accuracy and allow for detection of objects behind various type of occlusion, e.g., tree canopies. Range information is valuable for detection of small objects that are typically represented by 5-10 pixels in the data set. Range information is also valuable in tracking problems when the tracked object is occluded under parts of its movement and when there are several objects in the scene. In this paper, on-going work on detection and tracking are presented. Detection of partly occluded vehicles is discussed. To detect partly occluded objects we take advantage of the range information for removing foreground clutter. The target detection approach is based on geometric features, for example local surface detection, shadow analysis and height-based detection. Initial results on tracking of humans are also presented. The benefits with range information are discussed. Results are illustrated using outdoor measurements with a 3D FLASH LADAR sensor and a 3D scanning LADAR.