Vikram Krishnamurthy

Particle filters for state estimation of jump Markov linear systems

Jump Markov linear systems (JMLS) are linear systems whose parameters evolve with time according to a finite state Markov chain. In this paper, our aim is to recursively compute optimal state estimates for this class of systems. We present efficient simulation-based algorithms called particle filters to solve the optimal filtering problem as well as the optimal fixed-lag smoothing problem. Our algorithms combine sequential importance sampling, a selection scheme, and Markov chain Monte Carlo methods. They use several variance reduction methods to make the most of the statistical structure of JMLS. Computer simulations are carried out to evaluate the performance of the proposed algorithms. The problems of on-line deconvolution of impulsive processes and of tracking a maneuvering target are considered. It is shown that our algorithms outperform the current methods.

On-line estimation of hidden Markov model parameters based on the Kullback-Leibler information measure

Sequential or online hidden Markov model (HMM) signal processing schemes are derived, and their performance is illustrated by simulation. The online algorithms are sequential expectation maximization (EM) schemes and are derived by using stochastic approximations to maximize the Kullback-Leibler information measure. The schemes can be implemented either as filters or fixed-lag or sawtooth-lag smoothers. They yield estimates of the HMM parameters including transition probabilities, Markov state levels, and noise variance. In contrast to the offline EM algorithm (Baum-Welch scheme), which uses the fixed-interval forward-backward scheme, the online schemes have significantly reduced memory requirements and improved convergence, and they can estimate HMM parameters that vary slowly with time or undergo infrequent jump changes. Similar techniques are used to derive online schemes for extracting finite-state Markov chains imbedded in a mixture of white Gaussian noise (WGN) and deterministic signals of known functional form with unknown parameters. >

Algorithms for optimal scheduling and management of hidden Markov model sensors

The author considers a hidden Markov model (HMM) where a single Markov chain is observed by a number of noisy sensors. Due to computational or communication constraints, at each time instant, one can select only one of the noisy sensors. The sensor scheduling problem involves designing algorithms for choosing dynamically at each time instant which sensor to select to provide the next measurement. Each measurement has an associated measurement cost. The problem is to select an optimal measurement scheduling policy to minimize a cost function of estimation errors and measurement costs. The optimal measurement policy is solved via stochastic dynamic programming. Sensor management issues and suboptimal scheduling algorithms are also presented. A numerical example that deals with the aircraft identification problem is presented.

Performance analysis of a dynamic programming track before detect algorithm

We analyze a dynamic programming (DP)-based track before detect (TBD) algorithm. By using extreme value theory we obtain explicit expressions for various performance measures of the algorithm such as probability of detection and false alarm. Our analysis has two advantages. First the unrealistic Gaussian and independence assumptions used in previous works are not required. Second, the probability of detection and false alarm curves obtained fit computer simulated performance results significantly more accurately than previously proposed analyses of the TBD algorithm.

Expectation maximization algorithms for MAP estimation of jump Markov linear systems

In a jump Markov linear system, the state matrix, observation matrix, and the noise covariance matrices evolve according to the realization of a finite state Markov chain. Given a realization of the observation process, the aim is to estimate the state of the Markov chain assuming known model parameters. Computing conditional mean estimates is infeasible as it involves a cost that grows exponentially with the number of observations. We present three expectation maximization (EM) algorithms for state estimation to compute maximum a posteriori (MAP) state sequence estimates [which are also known as Bayesian maximum likelihood state sequence estimates (MLSEs)]. The first EM algorithm yields the MAP estimate for the entire sequence of the finite state Markov chain. The second EM algorithm yields the MAP estimate of the (continuous) state of the jump linear system. The third EM algorithm computes the joint MAP estimate of the finite and continuous states. The three EM algorithms optimally combine a hidden Markov model (HMM) estimator and a Kalman smoother (KS) in three different ways to compute the desired MAP state sequence estimates. Unlike the conditional mean state estimates, which require computational cost exponential in the data length, the proposed iterative schemes are linear in the data length.

Optimal Joint Session Admission Control in Integrated WLAN and CDMA Cellular Networks with Vertical Handoff

This paper considers optimizing the utilization of radio resources in a heterogeneous integrated system consisting of two different networks: a wireless local area network (WLAN) and a wideband code division multiple access (CDMA) network. We propose a joint session admission control scheme for multimedia traffic that maximizes overall network revenue with quality of service (QoS) constraints over both the WLAN and the CDMA cellular networks. The WLAN operates under the IEEE 802.11e medium access control (MAC) protocol, which supports QoS for multimedia traffic. A novel concept of effective bandwidth is used in the CDMA network to derive the unified radio resource usage, taking into account both physical layer linear minimum mean square error (LMMSE) receivers and characteristics of the packet traffic. Numerical examples illustrate that the network revenue earned in the proposed joint admission control scheme is significantly larger than that when the individual networks are optimized independently with no vertical handoff between them. The revenue gain is also significant over the scheme in which vertical handoff is supported, but admission control is not done jointly. Furthermore, we show that the optimal joint admission control policy is a randomized policy, i.e., sessions are admitted to the system with probabilities in some states

Stochastic sampling algorithms for state estimation of jump Markov linear systems

Jump Markov linear systems are linear systems whose parameters evolve with time according to a finite-state Markov chain. Given a set of observations, our aim is to estimate the states of the finite-state Markov chain and the continuous (in space) states of the linear system. The computational cost in computing conditional mean or maximum a posteriori (MAP) state estimates of the Markov chain or the state of the jump Markov linear system grows exponentially in the number of observations. We present three globally convergent algorithms based on stochastic sampling methods for state estimation of jump Markov linear systems. The cost per iteration is linear in the data length. The first proposed algorithm is a data augmentation (DA) scheme that yields conditional mean state estimates. The second proposed scheme is a stochastic annealing (SA) version of DA that computes the joint MAP sequence estimate of the finite and continuous states. Finally, a Metropolis-Hastings DA scheme based on SA is designed to yield the MAP estimate of the finite-state Markov chain. Convergence results of the three above-mentioned stochastic algorithms are obtained. Computer simulations are carried out to evaluate the performances of the proposed algorithms. The problem of estimating a sparse signal developing from a neutron sensor based on a set of noisy data from a neutron sensor and the problem of narrow-band interference suppression in spread spectrum code-division multiple-access (CDMA) systems are considered.

An improvement to the interacting multiple model (IMM) algorithm

Computing the optimal conditional mean state estimate for a jump Markov linear system requires exponential complexity, and hence, practical filtering algorithms are necessarily suboptimal. In the target tracking literature, suboptimal multiple-model filtering algorithms, such as the interacting multiple model (IMM) method and generalized pseudo-Bayesian (GPB) schemes, are widely used for state estimation of such systems. We derive a reweighted interacting multiple model algorithm. Although the IMM algorithm is an approximation of the conditional mean state estimator, our algorithm is a recursive implementation of a maximum a posteriori (MAP) state sequence estimator. This MAP estimator is an instance of a previous version of the EM algorithm known as the alternating expectation conditional maximization (AECM) algorithm. Computer simulations indicate that the proposed reweighted IMM algorithm is a competitive alternative to the popular IMM algorithm and GPB methods.

Hidden Markov model multiarm bandits: a methodology for beam scheduling in multitarget tracking

We derive optimal and suboptimal beam scheduling algorithms for electronically scanned array tracking systems. We formulate the scheduling problem as a multiarm bandit problem involving hidden Markov models (HMMs). A finite-dimensional optimal solution to this multiarm bandit problem is presented. The key to solving any multiarm bandit problem is to compute the Gittins (1989) index. We present a finite-dimensional algorithm that computes the Gittins index. Suboptimal algorithms for computing the Gittins index are also presented. Numerical examples are presented to illustrate the algorithms.

Structured Threshold Policies for Dynamic Sensor Scheduling—A Partially Observed Markov Decision Process Approach

We consider the optimal sensor scheduling problem formulated as a partially observed Markov decision process (POMDP). Due to operational constraints, at each time instant, the scheduler can dynamically select one out of a finite number of sensors and record a noisy measurement of an underlying Markov chain. The aim is to compute the optimal measurement scheduling policy, so as to minimize a cost function comprising of estimation errors and measurement costs. The formulation results in a nonstandard POMDP that is nonlinear in the information state. We give sufficient conditions on the cost function, dynamics of the Markov chain and observation probabilities so that the optimal scheduling policy has a threshold structure with respect to a monotone likelihood ratio (MLR) ordering. As a result, the computational complexity of implementing the optimal scheduling policy is inexpensive. We then present stochastic approximation algorithms for estimating the best linear MLR order threshold policy.