Iain B. Collings

On-line identification of hidden Markov models via recursive prediction error techniques

An on-line state and parameter identification scheme for hidden Markov models (HMMs) with states in a finite-discrete set is developed using recursive prediction error (RPE) techniques. The parameters of interest are the transition probabilities and discrete state values of a Markov chain. The noise density associated with the observations can also be estimated. Implementation aspects of the proposed algorithms are discussed, and simulation studies are presented to show that the algorithms converge for a wide variety of initializations. In addition, an improved version of an earlier proposed scheme (the Recursive Kullback-Leibler (RKL) algorithm) is presented with a parameterization that ensures positivity of transition probability estimates. >

An adaptive hidden Markov model approach to FM and M -ary DPSK demodulation in noisy fading channels

Abstract In this paper extended Kalman filtering (EKF) and hidden Markov model (HMM) signal processing techniques are coupled in order to demodulate frequency modulated signals in noisy fading channels. The demodulation scheme presented is applied to both digital M-ary differential phase shift keyed (MDPSK) and analog frequency modulated (FM) signals. Adaptive state-and-parameter estimation schemes are devised based on the assumption that the transmission channel introduces time-varying gain-and-phase changes, modelled by a stochastic linear system, and has additive Gaussian noise. An adaptive HMM approach is formulated which consists of a continuous state Kalman filter (KF) coupled with finite-discrete state HMM filters. The technique used is to represent MDPSK and FM signals with state space signal models for which the KF/HMM coupled filters are derived. A key to this approach is that complete information-states are used, instead of the maximum a posteriori estimates of the traditional matched filter approach, or maximum likelihood estimates of the Viterbi algorithm. The case of white observation noise is considered, as well as a generalisation to cope with coloured noise. Simulation studies are also presented.

An information-state approach to linear/risk-sensitive/quadratic/Gaussian control

In this paper we use an information-state approach to obtain the solution to the linear risk-sensitive quadratic Gaussian control problem. With these methods the solution is obtained without appealing to a certainty equivalence principle. Specifically we consider the case of tracking a desired trajectory. The result gives some insight to more general information-state methods for nonlinear systems. Limit results are presented which demonstrate the link to standard linear quadratic Gaussian control. Also, a risk-sensitive filtering result is presented which shows the relationship between tracking and filtering problems. Finally, simulation studies are presented to indicate some advantages gained via a risk-sensitive control approach.<<ETX>>

Adaptive HMM filters for signals in noisy fading channels

Kalman filtering (KF) and hidden Markov model (HMM) signal processing techniques are coupled to demodulate signals transmitted through noisy fading channels. The demodulation scheme presented can be applied to both digital M-ary differential phase shift keyed (MDPSK) and analog frequency modulated (FM) signals. Adaptive state and parameter estimation algorithms are devised based on the assumption that the transmission channel introduces time-varying gain and phase changes, modelled by a stochastic linear system, and has additive Gaussian noise. Our technique is to use an HMM filter, for signal estimation, coupled with a KF, for channel parameter tracking. The approach taken can easily be generalised for other transmission schemes, such as continuous phase modulated (CPM) signals.<<ETX>>

Recursive Prediction Error Techniques for Adaptive Estimation of Hidden Markov Models

Abstract In this paper an adaptive state and parameter estimation scheme for Hidden Markov Models (HMMs) with states in a finite set is developed using HMM processing in conjunction with Recursive Prediction Error (RPE) techniques. The parameters of interest are the transition probabilities and discrete state values of a Markov chain, and the noise density in which the Markov chain is embedded. We assume white Gaussian noise, however in general any independent and identically distributed (LLd.) noise density could be considered. In contrast to the off-line Expectation Maximisation (EM) algorithm which uses a fixed-interval “forward-backward“ multi-pass scheme, our on-line schemes have significantly reduced memory requirements, and with appropriate forgetting, can track slowly varying HMM parameters in an asymptotically efficient manner. Simulation studies are also presented.

Identification of Hidden Markov Models with Grouped State Values

Abstract This paper presents a recursive prediction error algorithm which addresses the problem of computational complexity for on-line identification of hidden Markov models (HMMs). The particular class of HMMs considered has state-values which are clustered into groups. This allows a reformulation of the Markov model and results in a sub-optimal reduced order identification scheme. Actually, an exact definition of clustering is not discussed, rather, a general identification technique is presented for which the computational requirements are greatly reduced when the state-values are divided in some way into groups. The applicability to certain types of cluster patterns is tested via simulation studies.

On-line estimation and identification of HMMs with grouped state values

This paper presents a signal-processing scheme for the class of lumpable or weakly lumpable hidden Markov models (HMMs) which have state values clustered into groups. Attention is focused not only on state estimation for known models but also on on-line model identification. The approach taken employs a new technique whereby separate state estimators are used for each group of state values. The state estimator for each group estimates the discrete states in that group together with an associated flag state which represents all the other groups. The result is that the computational complexity is greatly reduced. Hidden Markov model parameters associated with lumpable or weakly lumpable Markov chains can be identified on-line using available techniques such as the recursive prediction error (RPE) approach taken in this paper. These techniques estimate the transition probabilities and discrete state values of the Markov chain on-line. Other parameters, such as the noise density associated with the observations, can also be identified.