Jason L. Williams

Approximate Dynamic Programming for Communication-Constrained Sensor Network Management

Resource management in distributed sensor networks is a challenging problem. This can be attributed to the fundamental tradeoff between the value of information contained in a distributed set of measurements versus the energy costs of acquiring measurements, fusing them into the conditional probability density function (pdf) and transmitting the updated conditional pdf. Communications is commonly the highest contributor among these costs, typically by orders of magnitude. Failure to consider this tradeoff can significantly reduce the operational lifetime of a sensor network. While a variety of methods have been proposed that treat a subset of these issues, the approaches are indirect and usually consider at most a single time step. In the context of object tracking with a distributed sensor network, we propose an approximate dynamic programming approach that integrates the value of information and the cost of transmitting data over a rolling time horizon. We formulate this tradeoff as a dynamic program and use an approximation based on a linearization of the sensor model about a nominal trajectory to simultaneously find a tractable solution to the leader node selection problem and the sensor subset selection problem. Simulation results demonstrate that the resulting algorithm can provide similar estimation performance to that of the common most informative sensor selection method for a fraction of the communication cost.

Cost-function-based gaussian mixture reduction for target tracking

The problem of tracking targets in clutter nat- urally leads to a Gaussian mixture representation of the probability density function of the target state vector. St ate- of-the-art Multiple Hypothesis Tracking (MHT) techniques maintain the mean, covariance and probability weight cor- responding to each hypothesis, yet they rely on ad hoc merging and pruning rules to control the growth of hy- potheses. This paper proposes a structured cost-function- based approach to the hypothesis control problem, utiliz- ing the newly defined Integral Square Difference (ISD) cost measure. The performance of the ISD-based algorithm for tracking a single target in heavy clutter is compared to that of Salmonds joining filter, which previously had provided the highest performance in the scenario examined. For a larger number of mixture components, it is shown that the ISD algorithm outperforms the joining filter remark- ably, yielding an average track life more than double that achievable using the joining filter. Furthermore, it appear s that the performance of the algorithm will continue to grow exponentially as the number of mixture components is in- creased, hence the performance achievable is limited only by the computational resources available.

An approximate dynamic programming approach to a communication constrained sensor management problem

Resource management in distributed sensor networks is a challenging problem. This can be attributed to the fundamental trade-off between the value of information contained in a distributed set of measurements versus the energy costs of acquiring measurements, fusing them into a model of uncertainty, and transmitting the resulting model. Communications is commonly the highest contributor among these costs, typically by orders of magnitude. Failure to consider this trade-off can significantly reduce the operational lifetime of a sensor network. While a variety of methods have been proposed that treat a subset of these issues, the approaches are indirect and usually consider at most a single time step. In the context of object tracking with a distributed sensor network we propose an approximate dynamic programming approach which integrates the value of information and the cost of transmitting data over a rolling time horizon. We formulate this trade-off as a dynamic program, and use an approximation based on a linearization of the sensor model about a nominal trajectory to simultaneously find a tractable solution to the leader node selection problem and the sensor subset selection problem. Simulation results demonstrate that the resulting algorithm can provide similar estimation performance to that of the common most informative sensor selection method for a fraction of the communication cost.

An approximate dynamic programming approach for communication constrained inference

Resource management in distributed sensor networks is a challenging problem. This can be attributed to the fundamental tradeoff between the value of information contained in a distributed set of measurements versus the energy costs of acquiring the measurements, fusing them into a model of uncertainty, and transmitting the resulting model. Communications is commonly the highest contributor among these costs, typically by orders of magnitude. Failure to consider this tradeoff can significantly reduce the operational lifetime of a sensor network. While a variety of methods have been proposed that treat a subset of these issues, the approaches are indirect and usually consider at most a single time step. In the context of target tracking with a distributed sensor network we propose an approximate dynamic programming approach which integrates the value of information and the cost of transmitting data over a rolling time horizon. Specifically, we consider tracking a single target and constrain the problem such that, at any time, a single sensor, referred to as the leader node, is activated to both sense and update the probabilistic model. The issue of selecting which sensor should be the leader at each time is of primary interest, as it directly impacts the trade-off between the estimation accuracy and the cost of communicating the probabilistic model from old leader node to new leader node. We formulate this trade-off as a dynamic program, and use an approximation based on a linearization of the sensor model about a nominal trajectory to find a tractable solution. Simulation results demonstrate that the resulting algorithm can provide similar estimation performance to that of the common most informative sensor election method at a fraction of the communication energy cost

Optimization approaches to dynamic routing of measurements and models in a sensor network object tracking problem

Inter-sensor communication often comprises a significant portion of energy expenditures in a sensor network as compared to sensing and computation. We discuss an integrated approach to dynamically routing measurements and models in a sensor network. Specifically, we examine the problem of tracking objects within a region wherein the responsibility for combining measurements and updating a posterior state distribution is assigned to a single sensor at any given time step. The so called leader node may change over time. Sensor nodes communicate for two reasons: firstly, measurements of target state are transmitted from sensors to the current leader node for incorporation into the state estimate model; secondly, the state model is transmitted between sensors when the leader node changes. The trade-off between these two types of communication is of primary importance to dynamic selection of the leader node. We propose an algorithm based on a dynamic programming roll-out formulation of the minimum cost problem. We obtain a cost function which can be efficiently minimized by simplifying the problem to that of an open loop feedback controller which is an upper bound to the optimal cost. We present empirical results which compare methods previously proposed in the literature to the algorithm presented here.

Detection and Localization of Material Releases with Sparse Sensor Configurations

We consider the problem of detecting and localizing a material release utilizing sparse sensor measurements and formulate the problem as one of abrupt change detection. Methods which rely on single-sensor detection require dense deployment to achieve adequate coverage; costly sensors preclude such approaches. Furthermore, localization requires the fusion of multiple sensor measurements. Fusion in sparse sensor configurations is dependent on the knowledge of the dynamics of particle dispersion, which is, itself, problematic due to the inherent randomness on the wind field. We consider the efficacy of using an approximate dynamic model with coarse parameter estimates for the detection and localization of material releases. Specifically, we consider propagation models consisting of diffusion plus transport according to a Gaussian dispersion model. Assuming a known wind field, unconstrained intersensor communication, and a centralized processor, we derive optimal inference algorithms and provide a hybrid detection-localization hypothesis-testing framework with linear growth in the hypothesis space. We then analyze the probability of detection, time-to-detection, and localization performance as a function of the number of sensors. Furthermore, we examine the impact on performance when the underlying dynamical model deviates from the assumed model. This detailed analysis provides the basis for the design of more sophisticated algorithms for 1) performing robust detection followed by refined nonlinear parameter estimation which provides enhanced localization, and 2) distributed architectures aimed at conserving communication resources in which detections within local clusters are used to trigger more intensive intercluster communication to improve detection and localization

Improved hypothesis selection for multiple hypothesis tracking

The need to track closely-spaced targets in clutter is essential in support of military operations. This paper presents a Multiple Hypothesis Tracking (MHT) algorithm which uses an efficient structure to represent the dependency which naturally arises between targets due to the joint observation process, and an Integral Square Error (ISE) mixture reduction algorithm for hypothesis control. The resulting algorithm, denoted MHT with ISE Reduction (MISER), is tested against performance metrics including track life, coalescence and track swap. The results demonstrate track life performance similar to that of ISE-based methods in the single-target case, and a significant improvement in track swap metric due to the preservation of correlation between targets. The result that correlation reduces the track life performance for formation targets requires further investigation, although it appears to demonstrate that the inherent coupling of dynamics noises for such problems eliminates much of the benefit of representing correlation only due to the joint observation process.

Importance sampling actor-critic algorithms

Importance sampling (IS) and actor-critic are two methods which have been used to reduce the variance of gradient estimates in policy gradient optimization methods. We show how IS can be used with temporal difference methods to estimate a cost function parameter for one policy using the entire history of system interactions incorporating many different policies. The resulting algorithm is then applied to improving gradient estimates in a policy gradient optimization. The empirical results demonstrate a 20-40 times reduction in variance over the IS estimator for an example queueing problem, resulting in a similar factor of improvement in convergence for a gradient search

Performance Guarantees for Information Theoretic Sensor Resource Management

Many estimation problems involve sensors which can be actively controlled to alter the information received and utilized in the underlying inference task. In this paper, we discuss performance guarantees for heuristic algorithms for adaptive sensor control in sequential estimation problems, where the inference criterion is mutual information. We also demonstrate the performance of our tighter online computable performance guarantees through computational simulations. The guarantees may be applied to other estimation criteria including the Cramer-Rao bound.

Cost-function-based hypothesis control techniques for multiple hypothesis tracking

The problem of tracking targets in clutter naturally leads to a Gaussian mixture representation of the probability density function of the target state vector. Modern tracking methods maintain the mean, covariance and probability weight corresponding to each hypothesis, yet they rely on simple merging and pruning rules to control the growth of hypotheses. This paper proposes a structured, cost-function-based approach to the hypothesis control problem, utilizing the Integral Square Error (ISE) cost measure. A comparison of track life performance versus computational cost is made between the ISE-based filter and previously proposed approximations including simple pruning, Singers n-scan memory filter, Salmonds joining filter, and Chen and Lius Mixture Kalman Filter (MKF). The results demonstrate that the ISE-based mixture reduction algorithm provides track life performance which is significantly better than the compared techniques using similar numbers of mixture components, and performance competitive with the compared algorithms for similar mean computation times.