Thomas Hanselmann

Continuous-Time Adaptive Critics

A continuous-time formulation of an adaptive critic design (ACD) is investigated. Connections to the discrete case are made, where backpropagation through time (BPTT) and real-time recurrent learning (RTRL) are prevalent. Practical benefits are that this framework fits in well with plant descriptions given by differential equations and that any standard integration routine with adaptive step-size does an adaptive sampling for free. A second-order actor adaptation using Newtons method is established for fast actor convergence for a general plant and critic. Also, a fast critic update for concurrent actor-critic training is introduced to immediately apply necessary adjustments of critic parameters induced by actor updates to keep the Bellman optimality correct to first-order approximation after actor changes. Thus, critic and actor updates may be performed at the same time until some substantial error build up in the Bellman optimality or temporal difference equation, when a traditional critic training needs to be performed and then another interval of concurrent actor-critic training may resume

Multiple target tracking with asynchronous bearings-only-measurements

An algorithm for detection and tracking of multiple targets using bearings measurements from several sensors is developed. The algorithm is an implementation of a multiple hypothesis tracker with pruning of unlikely hypotheses. Tracking conditional on each hypothesis can be performed using any suitable filtering approximation. In this paper a range- parameterized unscented Kalman filter is used. Each hypothesis describes a track collection with varying number of targets. Final track estimates are obtained by weighted clustering according to hypothesis probabilities and clustered track states. Simulation experiments include arbitrary setup of multiple targets and multiple moving receiver platforms (sensors). The main results are the asynchronous modeling of measurements arrivals which allows an effective and efficient processing in a Bayesian MHT framework.

Adaptive Target Tracking in Slowly Changing Clutter

False track discrimination performance of a target tracking algorithm in a heavy clutter environment depends on the track confirmation and the track termination thresholds. The optimum value of these thresholds depends on the environment, in particular on the given probability of detection and on the existing clutter density. When tracking ground targets the probability of target detection is nominally constant, whereas the clutter measurement density varies significantly. Previously it was shown that, for a wide range of target signal to noise (+clutter) ratio in a uniform clutter density environment, and given the opportunity to set signal detection thresholds, the optimum value of clutter measurement density is almost constant (and the probability of detection will vary). We propose a scheme where the feedback from the target tracking system corrects the detection thresholds for each sensor resolution cell to obtain the constant and optimal clutter measurement density in each cell, when the clutter statistics changes slowly. This results in better false track discrimination capabilities of the tracker and also replaces the CFAR block in the signal processing unit

State dependent detection and object tracking

Target tracking algorithms usually treat the probability of detection as a constant, independent of the target state. In most cases this is not true, one obvious example being the Doppler frequency based clutter rejection, the other is obfuscation (shadowing) of ground based targets. This dependency modulates the measurement likelihood, which in turn introduces measurement non-linearity. In this paper we first present a general algorithm for target tracking in clutter when the probability of detection is target state dependent, and then proceed to an algorithm where both target state estimate and the probability of detection are modeled as Gaussian Mixtures. Probability of target existence is recursively updated as the track quality measure used for false track discrimination. A two sensor based ground target tracking in clutter simulation validates this approach.

Optimal signal detection for false track discrimination

Optimal signal detection for false track discrimination is determined by simulation using the integrated probabilistic data association (IPDA) algorithm. The IPDA algorithm is an efficient probabilistic data association algorithm with estimates of target existence probabilities that can be used to distinguish true and false tracks. The rate of confirmed false tracks is held constant and the optimal signal detection probability is determined by the maximum confirmed true tracks for a given signal to noise ratio. This allows the determination of the optimal detection probability which is reported for a range of signal to noise ratios.

Continuous adaptive critic designs

A continuous formulation of an adaptive critic design (ACD) is investigated. Connections to the discrete case are made, where backpropagation through time (BPTT) and realtime recurrent learning (RTRL) are prevalent. A second order actor adaptation, based on Newtons method, is established for fast actor convergence. Also, a fast critic update for concurrent actor-critic training is outlined that keeps the Bellman optimality correct to first order approximation after actor changes.

Computationally Efficient Transmission Scheduling for Sensor Networks

In this paper a new heuristic for transmission scheduling in sensor networks is proposed, using a model suggested by Chen et al. The performance and complexity of their algorithms are analyzed and it seems that the procedure for optimal scheduling with global channel state information is computationally too intensive for practical networks, while the heuristic methods used result in far from optimal network lifetimes. In this paper, a new heuristic is proposed, called the ratio minimisation heuristic, which has near optimal performance while maintaining linear complexity in the number of sensors.

A decomposition algorithm based on the algebraic perceptron

This paper introduces a prototype algorithm for decomposing a binary image into simpler images, such as regions bounded by low-degree polynomials. The algorithm is based on the algebraic perceptron, but can be extended to use other support vector schemes.