Václav Šmídl

Variational Bayesian Filtering

The use of the variational Bayes (VB) approximation in Bayesian filtering is studied, both as a means to accelerate marginalized particle filtering and as a deterministic local (one-step) approximation. The VB method of approximation is reviewed, together with restrictions that allow various computational savings to be achieved. These variants provide a range of algorithms that can be used in a principled tradeoff between quality of approximation and computational cost. In combination with marginalized particle filtering, they generalize previously published work on variational filtering and extend currently available methods for speeding up stochastic approximations in Bayesian filtering. In particular, the free-form nature of the VB approximation allows optimal selection of moments which summarize the particles. Other Bayesian filtering schemes are developed by replacing the marginalization operator in Bayesian filtering with VB-marginals. This leads to further computational savings at the cost of quality of approximation. The performance of the various VB filtering schemes is illustrated in the context of a Gaussian model with a nonlinear substate, and a hidden Markov model.

On Bayesian principal component analysis

A complete Bayesian framework for principal component analysis (PCA) is proposed. Previous model-based approaches to PCA were often based upon a factor analysis model with isotropic Gaussian noise. In contrast to PCA, these approaches do not impose orthogonality constraints. A new model with orthogonality restrictions is proposed. Its approximate Bayesian solution using the variational approximation and results from directional statistics is developed. The Bayesian solution provides two notable results in relation to PCA. The first is uncertainty bounds on principal components (PCs), and the second is an explicit distribution on the number of relevant PCs. The posterior distribution of the PCs is found to be of the von-Mises-Fisher type. This distribution and its associated hypergeometric function, F10, are studied. Numerical reductions are revealed, leading to a stable and efficient orthogonal variational PCA (OVPCA) algorithm. OVPCA provides the required inferences. Its performance is illustrated in simulation, and for a sequence of medical scintigraphic images.

Mixture-based extension of the AR model and its recursive Bayesian identification

An extension of the AutoRegressive (AR) model is studied, which allows transformations and distortions on the regressor to be handled. Many important signal processing problems are amenable to this Extended AR (i.e., EAR) model. It is shown that Bayesian identification and prediction of the EAR model can be performed recursively, in common with the AR model itself. The EAR model does, however, require that the transformation be known. When it is unknown, the associated transformation space is represented by a finite set of candidates. What follows is a Mixture-based EAR model, i.e., the MEAR model. An approximate identification algorithm for MEAR is developed, using a restricted Variational Bayes (VB) method. This restores the elegant recursive update of sufficient statistics. The MEAR model is applied to the robust identification of AR processes corrupted by outliers and burst noise, respectively, and to click removal for speech.

Robust estimation of autoregressive processes using a mixture-based filter-bank

Abstract A mixture-based framework for robust estimation of ARX-type processes is presented. The ARX process is presumed to suffer from an unknown noise and/or distortion. The approach taken here is to model the overall degraded process via a mixture. Each component of this mixture uses the same ARX model but explores a different noise/distortion process. Estimation of this mixture unifies the preprocessing and process modelling tasks. The quasi-Bayes (QB) procedure for mixture identification is extended to yield a fast recursive update of the estimator statistics. This allows non-stationary noise/distortion effects to be tracked. An application in on-line outlier-robust estimation of an AR process is given.

Bayesian estimation of non-stationary AR model parameters via an unknown forgetting factor

We study Bayesian estimation of the time-varying parameters of a non-stationary AR (autoregressive) process. This is traditionally achieved via exponential forgetting. A numerically tractable solution is available if the forgetting factor is known a priori. This assumption is now relaxed. Instead, we propose joint Bayesian estimation of the AR parameters and the unknown forgetting factor. The posterior distribution is intractable, and is approximated using the variational-Bayes (VB) method. Improved parameter tracking is revealed in simulation.

The Variational Bayes Approximation In Bayesian Filtering

The variational Bayes (VB) approximation is applied in the context of Bayesian filtering, yielding a tractable on-line scheme for a wide range of non-stationary parametric models. This VB-filtering scheme is used to identify a hidden Markov model with an unknown non-stationary transition matrix. In a simulation study involving soft-bit data, reliable inference of the underlying binary sequence is achieved in tandem with estimation of the transition probabilities. The performance compares favourably with a proposed particle filtering approach, and at lower computational cost

Fast variational PCA for functional analysis of dynamic image sequences

Principal component analysis (PCA) is a well-known algorithm used in many areas of science. It is usually taken as the golden standard for dimensionality reduction. However, PCA usually does not provide information about uncertainty of its results, thus preventing further investigation of model structure. A full Bayesian treatment is not feasible. Recently, variational PCA (VPCA) was proposed as an approximate Bayesian solution of the problem. In this paper, we summarise the iterative solution to the PCA problem arising from a variational approach. A new model with orthogonality restrictions is constructed in order to overcome its limitations. Notably, a highly efficient computational algorithm for variational PCA is revealed. It is applied in the analysis of functional medical images, yielding solution in a fraction of the time needed by the conventional technique.

Accelerated Particle Filtering using the Variational Bayes Approximation

In Bayesian filtering, the model may allow analytical marginalization over a subset, θ1,t, of the parameters. The marginalized (Rao-Blackwellized) particle filter (MPF) exploits this, by requiring stochastic sampling only in the remaining parameters, θ2,t, with the potential for major computational and convergence speed-ups. The marginalized filtering distribution in θ1,t is expressed as a mixture of n analytical components, each conditioned on one of the n particle trajectories in θ2,t; i.e. sufficient statistics must be stored and updated for each particle trajectory. In this paper, the variational Bayes (VB) approximation is used as a one-step approximation to extract necessary moments from the n particles in a principled manner, yielding a single-component marginalized filtering distribution. This formalizes and extends a recently reported certainty equivalence approach to accelerating MPFs. The comparative performance of the full and accelerated MPFs is explored via a scalar nonlinear filtering example.