Clare A. McGrory

Variational approximations in Bayesian model selection for finite mixture distributions

Variational methods, which have become popular in the neural computing/machine learning literature, are applied to the Bayesian analysis of mixtures of Gaussian distributions. It is also shown how the deviance information criterion, (DIC), can be extended to these types of model by exploiting the use of variational approximations. The use of variational methods for model selection and the calculation of a DIC are illustrated with real and simulated data. The variational approach allows the simultaneous estimation of the component parameters and the model complexity. It is found that initial selection of a large number of components results in superfluous components being eliminated as the method converges to a solution. This corresponds to an automatic choice of model complexity. The appropriateness of this is reflected in the DIC values.

Variational Bayes for estimating the parameters of a hidden Potts model

Hidden Markov random field models provide an appealing representation of images and other spatial problems. The drawback is that inference is not straightforward for these models as the normalisation constant for the likelihood is generally intractable except for very small observation sets. Variational methods are an emerging tool for Bayesian inference and they have already been successfully applied in other contexts. Focusing on the particular case of a hidden Potts model with Gaussian noise, we show how variational Bayesian methods can be applied to hidden Markov random field inference. To tackle the obstacle of the intractable normalising constant for the likelihood, we explore alternative estimation approaches for incorporation into the variational Bayes algorithm. We consider a pseudo-likelihood approach as well as the more recent reduced dependence approximation of the normalisation constant. To illustrate the effectiveness of these approaches we present empirical results from the analysis of simulated datasets. We also analyse a real dataset and compare results with those of previous analyses as well as those obtained from the recently developed auxiliary variable MCMC method and the recursive MCMC method. Our results show that the variational Bayesian analyses can be carried out much faster than the MCMC analyses and produce good estimates of model parameters. We also found that the reduced dependence approximation of the normalisation constant outperformed the pseudo-likelihood approximation in our analysis of real and synthetic datasets.

A fully Bayesian approach to inference for Coxian phase-type distributions with covariate dependent mean

Phase-type distributions represent the time to absorption for a finite state Markov chain in continuous time, generalising the exponential distribution and providing a flexible and useful modelling tool. We present a new reversible jump Markov chain Monte Carlo scheme for performing a fully Bayesian analysis of the popular Coxian subclass of phase-type models; the convenient Coxian representation involves fewer parameters than a more general phase-type model. The key novelty of our approach is that we model covariate dependence in the mean whilst using the Coxian phase-type model as a very general residual distribution. Such incorporation of covariates into the model has not previously been attempted in the Bayesian literature. A further novelty is that we also propose a reversible jump scheme for investigating structural changes to the model brought about by the introduction of Erlang phases. Our approach addresses more questions of inference than previous Bayesian treatments of this model and is automatic in nature. We analyse an example dataset comprising lengths of hospital stays of a sample of patients collected from two Australian hospitals to produce a model for a patients expected length of stay which incorporates the effects of several covariates. This leads to interesting conclusions about what contributes to length of hospital stay with implications for hospital planning. We compare our results with an alternative classical analysis of these data.

A new variational Bayesian algorithm with application to human mobility pattern modeling

A new variational Bayesian (VB) algorithm, split and eliminate VB (SEVB), for modeling data via a Gaussian mixture model (GMM) is developed. This new algorithm makes use of component splitting in a way that is more appropriate for analyzing a large number of highly heterogeneous spiky spatial patterns with weak prior information than existing VB-based approaches. SEVB is a highly computationally efficient approach to Bayesian inference and like any VB-based algorithm it can perform model selection and parameter value estimation simultaneously. A significant feature of our algorithm is that the fitted number of components is not limited by the initial proposal giving increased modeling flexibility. We introduce two types of split operation in addition to proposing a new goodness-of-fit measure for evaluating mixture models. We evaluate their usefulness through empirical studies. In addition, we illustrate the utility of our new approach in an application on modeling human mobility patterns. This application involves large volumes of highly heterogeneous spiky data; it is difficult to model this type of data well using the standard VB approach as it is too restrictive and lacking in the flexibility required. Empirical results suggest that our algorithm has also improved upon the goodness-of-fit that would have been achieved using the standard VB method, and that it is also more robust to various initialization settings.

The Variational Bayesian Approach to Fitting Mixture Models to Circular Wave Direction Data

AbstractThe emerging variational Bayesian (VB) technique for approximate Bayesian statistical inference is a non-simulation-based and time-efficient approach. It provides a useful, practical alternative to other Bayesian statistical approaches such as Markov chain Monte Carlo–based techniques, particularly for applications involving large datasets. This article reviews the increasingly popular VB statistical approach and illustrates how it can be used to fit Gaussian mixture models to circular wave direction data. This is done by taking the straightforward approach of padding the data; this method involves adding a repeat of a complete cycle of the data to the existing dataset to obtain a dataset on the real line. The padded dataset can then be analyzed using the standard VB technique. This results in a practical, efficient approach that is also appropriate for modeling other types of circular, or directional, data such as wind direction.

Recent Bayesian approaches for spatial analysis of 2-D images with application to environmental modelling

From remote sensing of the environment, to brain scans in medicine, the growth in the use of image data has motivated a parallel increase in statistical techniques for analysing these images. A particular area of growth has been in Bayesian models and corresponding computational methods. Bayesian approaches have been proposed to address the gamut of supervised and unsupervised inferential aims in image analysis. In this article we provide a general review of these approaches, with a focus on unsupervised analysis of 2-D images. Four exemplar methods that canvas the broad aims of image modelling and analysis are described. An exposition of these approaches is provided by applying them to an environmental case study involving the use of satellite data to assess water quality in the Great Barrier Reef, Australia. The techniques considered in detail are hidden Markov random fields (MRF), Gaussian MRF, Poisson/gamma random fields, and Voronoi tessellations. We also consider a variety of enabling computational algorithms, including MCMC, variational Bayes and integrated nested Laplace approximations. We compare the different aims and inferential capabilities of the models and discuss the advantages and drawbacks of the corresponding computational algorithms.

Transdimensional sequential Monte Carlo for hidden Markov models using variational Bayes - SMCVB.

In this paper we outline a transdimensional sequen- tial Monte Carlo algorithm - SMCVB - for fitting hidden Markov models. Sequential Monte Carlo (SMC) involves generating a weighted sample of particles from a sequence of probability distributions with the aim of converging to the target Bayesian posterior distribution. SMCVB makes use of variational Bayes (VB) in combination with SMC principles to create an algorithm which targets the posterior distribution more efficiently thereby saving on time and computational storage requirements. Another key feature of our methodology is that the variational-Bayes- generated proposals can vary in dimension. We have found in our simulation studies that we are able to obtain sensible estimates of the model dimensionality in this one-step procedure. This introduces very valuable additional flexibility in the modelling approach and opens up the potential for use of the algorithm in on-line settings where efficient and reliable estimation of dimensionality and parameters is required.