Sylvia Frühwirth-Schnatter

Markov chain Monte Carlo Estimation of Classical and Dynamic Switching and Mixture Models

Bayesian estimation of a very general model class, where the distribution of the observations depends on a latent process taking values in a discrete state space, is discussed in this article. This model class covers finite mixture modeling, Markov switching autoregressive modeling, and dynamic linear models with switching. The consequences the unidentifiability of this type of model has on Markov chain Monte Carlo (MCMC) estimation are explicitly dealt with. Joint Bayesian estimation of all latent variables, model parameters, and parameters that determine the probability law of the latent process is carried out by a new MCMC method called permutation sampling. The permutation sampler first samples from the unconstrained posterior–which often can be done in a convenient multimove manner–and then applies a permutation of the current labeling of the states of the latent process. In a first run, the random permutation sampler used selected the permutation randomly. The MCMC output of the random permutation sampler is explored to find suitable identifiability constraints. In a second run, the permutation sampler was used to sample from the constrained posterior by imposing identifiablity constraints. This time a suitable permutation is applied if the identifiability constraint is violated. For illustration, two detailed case studies are presented, namely finite mixture modeling of fetal lamb data and Markov switching autoregressive modeling of the U.S. quarterly real gross national product data.

Model-Based Clustering of Multiple Time Series

We propose to pool multiple time series into several groups using finite-mixture models. Within each group, the same econometric model holds. We estimate the groups of time series simultaneously with the group-specific model parameters using Bayesian Markov chain Monte Carlo simulation methods. We discuss model identification and base model selection on marginal likelihoods. With a simulation study, we document the efficiency gains in estimation and forecasting realized relative to overall pooling of the time series. To illustrate the usefulness of the method, we analyze extensions to unobserved heterogeneity and to Markov switching within clusters.

Bayesian inference for finite mixtures of univariate and multivariate skew-normal and skew-t distributions

Skew-normal and skew-t distributions have proved to be useful for capturing skewness and kurtosis in data directly without transformation. Recently, finite mixtures of such distributions have been considered as a more general tool for handling heterogeneous data involving asymmetric behaviors across subpopulations. We consider such mixture models for both univariate as well as multivariate data. This allows robust modeling of high-dimensional multimodal and asymmetric data generated by popular biotechnological platforms such as flow cytometry. We develop Bayesian inference based on data augmentation and Markov chain Monte Carlo (MCMC) sampling. In addition to the latent allocations, data augmentation is based on a stochastic representation of the skew-normal distribution in terms of a random-effects model with truncated normal random effects. For finite mixtures of skew normals, this leads to a Gibbs sampling scheme that draws from standard densities only. This MCMC scheme is extended to mixtures of skew-t distributions based on representing the skew-t distribution as a scale mixture of skew normals. As an important application of our new method, we demonstrate how it provides a new computational framework for automated analysis of high-dimensional flow cytometric data. Using multivariate skew-normal and skew-t mixture models, we could model non-Gaussian cell populations rigorously and directly without transformation or projection to lower dimensions.

Auxiliary mixture sampling with applications to logistic models

A new method of data augmentation for binary and multinomial logit models is described. First, the latent utilities are introduced as auxiliary latent variables, leading to a latent model which is linear in the unknown parameters, but involves errors from the type I extreme value distribution. Second, for each error term the density of this distribution is approximated by a mixture of normal distributions, and the component indicators in these mixtures are introduced as further latent variables. This leads to Markov chain Monte Carlo estimation based on a convenient auxiliary mixture sampler that draws from standard distributions like normal or exponential distributions and, in contrast to more common Metropolis-Hastings approaches, does not require any tuning. It is shown how the auxiliary mixture sampler is implemented for binary or multinomial logit models, and it is demonstrated how to extend the sampler to mixed effect models and time-varying parameter models for binary and categorical data. Finally, an application to Austrian labor market data is discussed.

Improved auxiliary mixture sampling for hierarchical models of non-Gaussian data

The article considers Bayesian analysis of hierarchical models for count, binomial and multinomial data using efficient MCMC sampling procedures. To this end, an improved method of auxiliary mixture sampling is proposed. In contrast to previously proposed samplers the method uses a bounded number of latent variables per observation, independent of the intensity of the underlying Poisson process in the case of count data, or of the number of experiments in the case of binomial and multinomial data. The bounded number of latent variables results in a more general error distribution, which is a negative log-Gamma distribution with arbitrary integer shape parameter. The required approximations of these distributions by Gaussian mixtures have been computed. Overall, the improvement leads to a substantial increase in efficiency of auxiliary mixture sampling for highly structured models. The method is illustrated for finite mixtures of generalized linear models and an epidemiological case study.

Bayesian analysis of switching ARCH models

We consider a time series model with autoregressive conditional heteroscedasticity that is subject to changes in regime. The regimes evolve according to a multistate latent Markov switching process with unknown transition probabilities, and it is the constant in the variance process of the innovations that is subject to regime shifts. The joint estimation of the latent process and all model parameters is performed within a Bayesian framework using the method of Markov chain Monte Carlo (MCMC) simulation. We perform model selection with respect to the number of states and the number of autoregressive parameters in the variance process using Bayes factors and model likelihoods. To this aim, the model likelihood is estimated by the method of bridge sampling. The usefulness of the sampler is demonstrated by applying it to the data set previously used by Hamilton and Susmel (1994) who investigated models with switching autoregressive conditional heteroscedasticity using maximum likelihood methods. The paper concludes with some issues related to maximum likelihood methods, to classical model selection, and to potential straightforward extensions of the model presented here.

On statistically inference for fuzzy data with applications to descriptive statistics

Abstract The object of this paper is to discuss how fuzziness of data is propagated when statistical inference for samples of non-precise data is carried out. First, ‘fuzzy data’ and ‘fuzzy samples’ are defined. Subsequently, the method of propagation of fuzziness as used in previous work for specific stochastic models is formulated in rather general terms. This method may be applied to any statistical method which leads to a result that may be expressed as a function f ( x 1 ,…, x n ) of the data x 1 ,…, x n . It is proved that this method is equal to determining the images of a family of compact subsets of the sample space under the function f (·). This equivalence is utilized in order to perform propagation of fuzziness in practice. Finally, this approach is applied to some concepts of descriptive statistics such as fuzzy sample mean, fuzzy sample variance and to the empirical distribution function of a fuzzy sample.

Bayesian Analysis of the Heterogeneity Model

We consider Bayesian estimation of a finite mixture of models with random effects, which is also known as the heterogeneity model. First, we discuss the properties of various Markov chain Monte Carlo samplers that are obtained from full conditional Gibbs sampling by grouping and collapsing. Whereas full conditional Gibbs sampling turns out to be sensitive to the parameterization chosen for the mean structure of the model, the alternative sampler is robust in this respect. However, the logical extension of the approach to the sampling of the group variances does not further increase the efficiency of the sampler. Second, we deal with the identifiability problem due to the arbitrary labeling within the model. Finally, a case study involving metric conjoint analysis serves as a practical illustration.

Data Augmentation and MCMC for Binary and Multinomial Logit Models

The paper introduces two new data augmentation algorithms for sampling the parameters of a binary or multinomial logit model from their posterior distribution within a Bayesian framework. The new samplers are based on rewriting the underlying random utility model in such away that only differences of utilities are involved. As a consequence, the error term in the logit model has a logistic distribution. If the logistic distribution is approximated by a finite scale mixture of normal distributions, auxiliary mixture sampling can be implemented to sample from the posterior of the regression parameters. Alternatively, a data augmented Metropolis–Hastings algorithm can be formulated by approximating the logistic distribution by a single normal distribution. A comparative study on five binomial and multinomial data sets shows that the new samplers are superior to other data augmentation samplers and to Metropolis–Hastings sampling without data augmentation.

Applied State Space Modelling of Non-Gaussian Time Series using Integration-based Kalman-filtering

The main topic of the paper is on-line filtering for non-Gaussian dynamic (state space) models by approximate computation of the first two posterior moments using efficient numerical integration. Based on approximating the prior of the state vector by a normal density, we prove that the posterior moments of the state vector are related to the posterior moments of the linear predictor in a simple way. For the linear predictor Gauss-Hermite integration is carried out with automatic reparametrization based on an approximate posterior mode filter. We illustrate how further topics in applied state space modelling, such as estimating hyperparameters, computing model likelihoods and predictive residuals, are managed by integration-based Kalman-filtering. The methodology derived in the paper is applied to on-line monitoring of ecological time series and filtering for small count data.