Man-Suk Oh

Adaptive importance sampling in monte carlo integration

An Adaptive Importance Sampling (AIS) scheme is introduced to compute integrals of the form as a mechanical, yet flexible, way of dealing with the selection of parameters of the importance function. AIS starts with a rough estimate for the parameters λ of the importance function g , and runs importance sampling in an iterative way to continually update λ using only linear accumulation. Consistency of AIS is established. The efficiency of the algorithm is studied in three examples and found to be substantially superior to ordinary importance sampling.

Bayesian Multidimensional Scaling and Choice of Dimension

Multidimensional scaling is widely used to handle data that consist of similarity or dissimilarity measures between pairs of objects. We deal with two major problems in metric multidimensional scaling–configuration of objects and determination of the dimension of object configuration–within a Bayesian framework. A Markov chain Monte Carlo algorithm is proposed for object configuration, along with a simple Bayesian criterion, called MDSIC, for choosing their dimension. Simulation results are presented, as are real data. Our method provides better results than does classical multidimensional scaling and ALSCAL for object configuration, and MDSIC seems to work well for dimension choice in the examples considered.

Integration of Multimodal Functions by Monte Carlo Importance Sampling

Abstract Numerical integration of a multimodal integrand f(θ) is approached by Monte Carlo integration via importance sampling. A mixture of multivariate t density functions is suggested as an importance function g(θ), for its easy random variate generation, thick tails, and high flexibility. The number of components in the mixture is determined by the number of modes of f(θ), and the mixing weights and location and scale parameters of the component distributions are determined by numerical minimization of a Monte Carlo estimate of the squared variation coefficient of the weight function f(θ)/g(θ). Stratified importance sampling and control variates are shown to be particularly effective variance reduction techniques in this case. The algorithm is applied to a 10-dimensional example and shown to yield significant improvement over usual integration schemes.

Multivariate receptor models and model uncertainty

Estimation of the number of major pollution sources, the source composition profiles, and the source contributions are the main interests in multivariate receptor modeling. Due to lack of identifiability of the receptor model, however, the estimation cannot be done without some additional assumptions. A common approach to this problem is to estimate the number of sources, q, at the first stage, and then estimate source profiles and contributions at the second stage, given additional constraints (identifiability conditions) to prevent source rotation/ transformation and the assumption that the q-source model is correct. These assumptions on the parameters (the number of sources and identifiability conditions) are the main source of model uncertainty in multivariate receptor modeling. In this paper, we suggest a Bayesian approach to deal with model uncertainties in multivariate receptor models by using Markov chain Monte Carlo (MCMC) schemes. Specifically, we suggest a method which can simultaneously estimate parameters (compositions and contributions), parameter uncertainties, and model uncertainties (number of sources and identifiability conditions). Simulation results and an application to air pollution data are presented. D 2002 Elsevier Science B.V. All rights reserved.

Model-Based Clustering With Dissimilarities: A Bayesian Approach

A Bayesian model-based clustering method is proposed for clustering objects on the basis of dissimilarites. This combines two basic ideas. The first is that the objects have latent positions in a Euclidean space, and that the observed dissimilarities are measurements of the Euclidean distances with error. The second idea is that the latent positions are generated from a mixture of multivariate normal distributions, each one corresponding to a cluster. We estimate the resulting model in a Bayesian way using Markov chain Monte Carlo. The method carries out multidimensional scaling and model-based clustering simultaneously, and yields good object configurations and good clustering results with reasonable measures of clustering uncertainties. In the examples we study, the clustering results based on low-dimensional configurations were almost as good as those based on high-dimensional ones. Thus, the method can be used as a tool for dimension reduction when clustering high-dimensional objects, which may be useful especially for visual inspection of clusters. We also propose a Bayesian criterion for choosing the dimension of the object configuration and the number of clusters simultaneously. This is easy to compute and works reasonably well in simulations and real examples.

Estimation of posterior density functions from a posterior sample

The joint posterior density function of parameters and marginal posterior density functions of subsets of parameters are key quantities in Bayesian inference. Even when the posterior densities are unknown, there are many cases where Markov Chain Monte Carlo methods can generate samples from the joint posterior distribution. This paper proposes a simple and efficient method of estimating the posterior density functions at various points simultaneously by using a posterior sample.

Bayesian analysis of time series Poisson data

This paper provides a practical simulation-based Bayesian analysis of parameter-driven models for time series Poisson data with the AR(1) latent process. The posterior distribution is simulated by a Gibbs sampling algorithm. Full conditional posterior distributions of unknown variables in the model are given in convenient forms for the Gibbs sampling algorithm. The case with missing observations is also discussed. The methods are applied to real polio data from 1970 to 1983.

Bayesian inference and model selection in latent class logit models with parameter constraints: An application to market segmentation

Latent class models have recently drawn considerable attention among many researchers and practitioners as a class of useful tools for capturing heterogeneity across different segments in a target market or population. In this paper, we consider a latent class logit model with parameter constraints and deal with two important issues in the latent class models--parameter estimation and selection of an appropriate number of classes--within a Bayesian framework. A simple Gibbs sampling algorithm is proposed for sample generation from the posterior distribution of unknown parameters. Using the Gibbs output, we propose a method for determining an appropriate number of the latent classes. A real-world marketing example as an application for market segmentation is provided to illustrate the proposed method.

A simple and efficient Bayesian procedure for selecting dimensionality in multidimensional scaling

Multidimensional scaling (MDS) is a technique which retrieves the locations of objects in a Euclidean space (the object configuration) from data consisting of the dissimilarities between pairs of objects. An important issue in MDS is finding an appropriate dimensionality underlying these dissimilarities. In this paper, we propose a simple and efficient Bayesian approach for selecting dimensionality in MDS. For each column (attribute) vector of an MDS configuration, we assume a prior that is a mixture of the point mass at 0 and a continuous distribution for the rest of the parameter space. Then the marginal posterior distribution of each column vector is also a mixture of the same form, in which the mixing weight of the continuous distribution is a measure of significance for the column vector. We propose an efficient Markov chain Monte Carlo (MCMC) method for estimating the mixture posterior distribution. The proposed method is fully Bayesian. It takes parameter estimation error into account when computing penalties for complex models and provides an uncertainty measure for the choice of dimensionality. Also, the MCMC algorithm is computationally very efficient since it visits various dimensional models in one MCMC procedure. A simulation study compares the proposed method with the Bayesian method of Oh and Raftery (2001). Three real data sets are analysed by using the proposed method.

A unified Bayesian inference on treatment means with order constraints

In some applications involving comparison of treatment means, it is known a priori that population means are ordered in a certain way. In such situations, imposing constraints on the treatment means can greatly increase the effectiveness of statistical procedures. This paper proposes a unified Bayesian method which performs a simultaneous comparison of treatment means and parameter estimation in ANOVA models with order constraints on the means. A continuous prior restricted to order constraints is employed, and posterior samples of parameters are generated using a Markov chain Monte Carlo method. Posterior probabilities of all possible hypotheses on the equality/inequality of treatment means are obtained using Savage-Dickey density ratios, for which we propose a simple and computationally efficient estimation method. Posterior densities and HPD intervals of parameters of interest are estimated with almost no extra cost, given some by-products from the test procedure. Simulation study results show that the proposed method outperforms the test without constraints and that the method is powerful in detecting the true hypothesis. The method is applied to the ramus bone sizes of 20 boys, which were measured at four time points. The proposed Bayesian test reveals that there are two growth spurts in the ramus bone size during the observed period, which could not be detected by pairwise comparisons of the means.