Christopher K. Carter

Experiments in Stochastic Computation for High-Dimensional Graphical Models

We discuss the implementation, development and performance of methods of stochastic computation in Gaussian graphical models. We view these methods from the perspective of high-dimensional model search, with a particular interest in the scalability with dimension of Markov chain Monte Carlo (MCMC) and other stochastic search methods. After reviewing the structure and context of undirected Gaussian graphical models and model uncertainty (covariance selection), we discuss prior specifications, including new priors over models, and then explore a number of examples using various methods of stochastic computation. Traditional MCMC methods are the point of departure for this experimentation; we then develop alternative stochastic search ideas and contrast this new approach with MCMC. Our examples range from low (12–20) to moderate (150) dimension, and combine simple synthetic examples with data analysis from gene expression studies. We conclude with comments about the need and potential for new computational methods in far higher dimensions, including constructive approaches to Gaussian graphical modeling and computation.

Efficient Bayesian Inference for Dynamic Mixture Models

Abstract A Bayesian approach is presented for estimating a mixture of linear Gaussian state-space models. Such models are used to model interventions in time series and nonparametric regression. Markov chain Monte Carlo sampling is usually necessary to obtain the posterior distributions of such mixture models, because it is difficult to obtain them analytically. The methodological contribution of the article is to derive a set of recursions for dynamic mixture models that efficiently implement a Markov chain Monte Carlo sampling scheme that converges rapidly to the posterior distribution. The methodology is illustrated by fitting an autoregressive model subject to interventions to zinc concentration in sludge.

Diagnostics for Time Series Analysis

Test statistics are proposed to determine the goodness of fit of a time series model. The test statistics are based on a sequence of random variables that are independent and standard normal if the model is correct. The paper shows how to compute this sequence of random variables efficiently using a combination of Markov chain Monte Carlo and importance sampling. The power of the statistics to detect outliers and level shifts is studied for an autoregressive model. The methodology is illustrated using both simulated and real data.

Semiparametric Bayesian inference for time series with mixed spectra

A Bayesian analysis is presented of a time series which is the sum of a stationary component with a smooth spectral density and a deterministic component consisting of a linear combination of a trend and periodic terms. The periodic terms may have known or unknown frequencies. The advantage of our approach is that different features of the data such as the regression parameters, the spectral density, unknown frequencies, and missing observations are combined in a hierarchical Bayesian framework and estimated simultaneously. A Bayesian test to detect the presence of deterministic components in the data is also constructed. By using an asymptotic approximation to the likelihood, the computation is carried out efficiently using Markov chain Monte Carlo in O(Mn) operations, where n is the sample size and M and is the number of iterations. We show empirically that our approach works well on real and simulated examples.

Remote access methods for exploratory data analysis and statistical modelling: Privacy-Preserving Analytics ®

This paper is concerned with the challenge of enabling the use of confidential or private data for research and policy analysis, while protecting confidentiality and privacy by reducing the risk of disclosure of sensitive information. Traditional solutions to the problem of reducing disclosure risk include releasing de-identified data and modifying data before release. In this paper we discuss the alternative approach of using a remote analysis server which does not enable any data release, but instead is designed to deliver useful results of user-specified statistical analyses with a low risk of disclosure. The techniques described in this paper enable a user to conduct a wide range of methods in exploratory data analysis, regression and survival analysis, while at the same time reducing the risk that the user can read or infer any individual record attribute value. We illustrate our methods with examples from biostatistics using publicly available data. We have implemented our techniques into a software demonstrator called Privacy-Preserving Analytics (PPA), via a web-based interface to the R software. We believe that PPA may provide an effective balance between the competing goals of providing useful information and reducing disclosure risk in some situations.

Understanding sources of variation in syndromic surveillance for early warning of natural or intentional disease outbreaks

Daily counts of computer records of hospital emergency department arrivals grouped according to diagnosis (called here syndrome groupings) can be monitored by epidemiologists for changes in frequency that could provide early warning of bioterrorism events or naturally occurring disease outbreaks and epidemics. This type of public health surveillance is sometimes called syndromic surveillance. We used transitional Poisson regression models to obtain one-day-ahead arrival forecasts. Regression parameter estimates and forecasts were updated for each day using the latest 365 days of data. The resulting time series of recursive estimates of parameters such as the amplitude and location of the seasonal peaks as well as the one-day-ahead forecasts and forecast errors can be monitored to understand changes in epidemiology of each syndrome grouping. The counts for each syndrome grouping were autocorrelated and non-homogeneous Poisson. As such, the main methodological contribution of the article is the adaptation of Cumulative Sum (CUSUM) and Exponentially Weighted Moving Average (EWMA) plans for monitoring non-homogeneous counts. These plans were valid for small counts where the assumption of normally distributed one-day-ahead forecasts errors, typically used in other papers, breaks down. In addition, these adaptive plans have the advantage that control limits do not have to be trained for different syndrome groupings or aggregations of emergency departments. Conventional methods for signaling increases in syndrome grouping counts, Shewhart, CUSUM, and EWMA control charts of the standardized forecast errors were also examined. Shewhart charts were, at times, insensitive to shifts of interest. CUSUM and EWMA charts were only reasonable for large counts. We illustrate our methods with respiratory, influenza, diarrhea, and abdominal pain syndrome groupings.

Bayesian covariance matrix estimation using a mixture of decomposable graphical models

We present a Bayesian approach to estimating axa0covariance matrix by using a prior that is a mixture over all decomposable graphs, with the probability of each graph size specified by the user and graphs of equal size assigned equal probability. Most previous approaches assume that all graphs are equally probable. We show empirically that the prior that assigns equal probability over graph sizes outperforms the prior that assigns equal probability over all graphs in more efficiently estimating the covariance matrix. Thexa0prior requires knowing the number of decomposable graphs for each graph size and we give a simulation method for estimating these counts. We also present a Markov chain Monte Carlo method for estimating the posterior distribution of the covariance matrix that is much more efficient than current methods. Both the prior and the simulation method to evaluate the prior apply generally to any decomposable graphical model.

Variable Selection and Covariance Selection in Multivariate Regression Models

This article provides a general framework for Bayesian variable selection and covariance selection in a multivariate regression model with Gaussian errors. By variable selection we mean allowing certain regression coefficients to be zero. By covariance selection we mean allowing certain elements of the inverse covariance matrix to be zero. We estimate all the model parameters by model averaging using a Markov chain Monte Carlo simulation method. The methodology is illustrated by applying it to four real data sets. The effectiveness of variable selection and covariance selection in estimating the multivariate regression model is assessed by using four loss functions and four simulated data sets. Each of the simulated data sets is based on parameter estimates obtained from a corresponding real data set.

Constructing priors based on model size for nondecomposable Gaussian graphical models: A simulation based approach

A method for constructing priors is proposed that allows the off-diagonal elements of the concentration matrix of Gaussian data to be zero. The priors have the property that the marginal prior distribution of the number of nonzero off-diagonal elements of the concentration matrix (referred to below as model size) can be specified flexibly. The priors have normalizing constants for each model size, rather than for each model, giving a tractable number of normalizing constants that need to be estimated. The article shows how to estimate the normalizing constants using Markov chain Monte Carlo simulation and supersedes the method of Wong et al. (2003) [24] because it is more accurate and more general. The method is applied to two examples. The first is a mixture of constrained Wisharts. The second is from Wong et al. (2003) [24] and decomposes the concentration matrix into a function of partial correlations and conditional variances using a mixture distribution on the matrix of partial correlations. The approach detects structural zeros in the concentration matrix and estimates the covariance matrix parsimoniously if the concentration matrix is sparse.