Pablo O. Arambel

Multiple-hypothesis tracking of multiple ground targets from aerial video with dynamic sensor control

The goal of the DARPA Video Verification of Identity (VIVID) program is to develop an automated video-based ground targeting system for unmanned aerial vehicles that significantly improves operator combat efficiency and effectiveness while minimizing collateral damage. One of the key components of VIVID is the Multiple Target Tracker (MTT), whose main function is to track many ground targets simultaneously by slewing the video sensor from target to target and zooming in and out as necessary. The MTT comprises three modules: (i) a video processor that performs moving object detection, feature extraction, and site modeling; (ii) a multiple hypothesis tracker that processes extracted video reports (e.g. positions, velocities, features) to generate tracks of currently and previously moving targets and confusers; and (iii) a sensor resource manager that schedules camera pan, tilt, and zoom to support kinematic tracking, multiple target track association, scene context modeling, confirmatory identification, and collateral damage avoidance. When complete, VIVID MTT will enable precision tracking of the maximum number of targets permitted by sensor capabilities and by target behavior. This paper describes many of the challenges faced by the developers of the VIVID MTT component, and the solutions that are currently being implemented.

Signature-Aided Air-to-Ground Video Tracking

Tracking ground moving objects using aerial video sensors is very challenging when the objects go through periods of occlusion caused by trees or buildings. If the occlusion interval is relatively large, there are confusing objects in the vicinity, or the object performs abrupt maneuvers while occluded, maintaining continuous tracks after the occlusion requires advanced exploitation of the imagery. This paper presents a signature-aided multiple hypothesis tracking system where signatures are extracted during periods of certainty and used after the occlusion to resolve association ambiguity. The discussion focuses on the interaction between the tracker and the signature extraction/exploitation module, as well as other tracking aspects within the signature-aided tracking paradigm

Performance assessment of a video-based air-to-ground multiple target tracker with dynamic sensor control

The goal of the DARPA Video Verification of Identity (VIVID) program is to develop an automated video-based ground targeting system for unmanned aerial vehicles. The system comprises several modules that interact with each other to support tracking of multiple targets, confirmatory identification, and collateral damage avoidance. The Multiple Target Tracking (MTT) module automatically adjusts the camera pan, tilt, and zoom to support kinematic tracking, multi-target track association, and confirmatory identification. The MTT system comprises: (i) a video processor that performs moving object detection and feature extraction, including object position and velocity, (ii) a multiple hypothesis tracker that processes video processor reports to generate and maintain tracks, and (iii) a sensor resource manager that aims the sensor to improve tracking of multiple targets. This paper presents a performance assessment of the current implementation of the MTT under several operating conditions. The evaluation is done using pre-recorded airborne video to assess the ability of the video tracker to detect and track ground moving objects over extended periods of time. The tests comprise a number of different operational conditions such as multiple targets and confusers under various levels of occlusion and target maneuverability, as well as different background conditions. The paper also describes the challenges that still need to be overcome to extend track life over long periods of time.

Markov Chains for the Prediction of Tracking Performance

Highly accurate predictions of tracking performance usually require high fidelity Monte Carlo simulations that entail significant implementation time, run time, and complexity. In this paper we consider the use of Markov Chains as a simpler alternative that models critical aspects of the tracking process and provides reasonable estimates of tracking performance, while maintaining much lower cost and complexity. We describe a general procedure for Markov-Chain based performance prediction, and illustrate the use of this procedure in the context of an airborne system that employs a steerable EO/IR sensor to track single targets or multiple targets in non-overlapping fields of view. We discuss the effects of key model parameters, including measurement sampling rates, track termination, target occlusions, and missed detections. We also present plots of performance as a function of occlusion probability and target recognition probability that exemplify the use of the model.

Multiple Target Tracking with a Steerable Airborne Video Sensor

Tracking multiple surface targets with a single steerable airborne video sensor is accomplished by several interrelated functions: (i) image registration for camera motion compensation and accurate image-to-ground mapping, (ii) video processing for object detection and feature extraction, (iii) target tracking for detection association and track creation and maintenance, (iv) signature extraction and exploitation, and (v) sensor resource management for the generation of sensor steering commands. The first function is often overlooked, but has a significant impact in the performance of the overall system. A rudimentary registration can be achieved by using the platform location and attitude as well as the sensor orientation and field of view information, but the accuracy of this registration is typically poor due to inertial navigation system errors, particularly in small unmanned aerial vehicles with cost and hardware limitations. Successful registration of successive frames enables the use of multiple frame video processing for improved object detection and provides stable image-to-ground mapping for improved data association by the tracker. In systems with a steerable sensor that slews back and forth to track more than one target simultaneously, the image registration module creates and maintains multiple mosaics corresponding to the different tracking areas. In this paper we discuss primarily the image registration module and the system of coordinate frames that is maintained to improve data association and tracking.

Expected track length estimation using track break statistics

We consider the problem of estimating the performance of a system that tracks moving objects on the ground using airborne sensors. Expected Track Life (ETL) is a measure of performance that indicates the ability of a tracker to maintain track for extended periods of time. The most desirable method for computing ETL would involve the use of large sets of real data with accompanying truth. This accurately accounts for sensor artifacts and data characteristics, which are difficult to simulate. However, datasets with these characteristics are difficult to collect because the coverage area of the sensors is limited, the collection time is limited, and the number of objects that can realistically be truthed is also limited. Thus when using real datasets, many tracks are terminated because the objects leave the field of view or the end of the dataset is reached. This induces a bias in the estimation when the ETL is computed directly from the tracks. An alternative to direct ETL computation is the use of Markov-Chain models that use track break statistics to estimate ETL. This method provides unbiased ETL estimates from datasets much shorter than what would be required for direct computation. In this paper we extend previous work in this area and derive an explicit expression of the ETL as a function of track break statistics. An example illustrates the properties and advantages of the method.

Exploiting Multi-Vehicle Interactions to Improve Urban Vehicle Tracking

The subject of traffic flow modeling began over fifty years ago when Lighthill and Whitham used flow continuity equation from fluid dynamics to describe traffic behavior. Since then, a multitude of models, broadly classified into macroscopic, mesoscopic, and microscopic models, has been developed. Macroscopic models describe the space-time evolution of aggregate quantities such as traffic flow density whereas microscopic models describe behavior of individual drivers/vehicles in the presence of other vehicles. In this paper, we consider tracking of vehicles using a specific microscopic model known as the intelligent driver model (IDM). As in other microscopic models, the IDM equations of motion of a vehicle are nonlinearly coupled to those of neighboring vehicles, with the magnitudes of coupling terms becoming larger as vehicles get closer and smaller as vehicles get farther apart. In our approach, the state of weakly coupled groups of vehicles is represented by separated probability distributions. When the vehicles move closer to each other, the state is represented by a joint probability distribution that takes into account the interaction among vehicles. We use a sum of Gaussians approach to represent the underlying interaction structure for state estimation and reduce computational complexity. In this paper we describe our approach and illustrate the approach with simulated examples.

Performance analysis of structured pedigree distributed fusion systems

Structured pedigree is a way to compress pedigree information. When applied to distributed fusion systems, the approach avoids the well known problem of information double counting resulting from ignoring the cross-correlation among fused estimates. Other schemes that attempt to compute optimal fused estimates require the transmission of full pedigree information or raw data. This usually can not be implemented in practical systems because of the enormous requirements in communications bandwidth. The Structured Pedigree approach achieves data compression by maintaining multiple covariance matrices, one for each uncorrelated source in the network. These covariance matrices are transmitted by each node along with the state estimate. This represents a significant compression when compared to full pedigree schemes. The transmission of these covariance matrices (or a subset of these covariance matrices) allows for an efficient fusion of the estimates, while avoiding information double counting and guaranteeing consistency on the estimates. This is achieved by exploiting the additional partial knowledge on the correlation of the estimates. The approach uses a generalized version of the Split Covariance Intersection algorithm that applies to multiple estimates and multiple uncorrelated sources. In this paper we study the performance of the proposed distributed fusion system by analyzing a simple but instructive example.

Feasibility study of a video-based urban traffic tracking system

This paper presents a feasibility study on the problem of tracking vehicles using multiple video cameras placed at known locations. Each video processor extracts time of arrival at the camera, instantaneous velocity, and image features such as color and size. The information is sent to a centralized tracker that estimates the location of the vehicles at each instant of time. Our study showed that our association algorithms across multiple camera views could achieve an 80% detection rate. We discovered that changes in resolution, view angle, elevation, and sun angle did not significantly affect our algorithms performance. Our study also showed that with multiple hypothesis tracking and with adequate camera to camera image matching performance, tracks can be maintained for 10 km with 50% probability for 1600 meter camera spacing and 90% probability for 400 meter camera spacing. The conclusion is that current tracking technology with sufficient video matching performance is likely to give good tracking performance in an urban environment.