Robert E. McCulloch

Variable Selection via Gibbs Sampling

Abstract A crucial problem in building a multiple regression model is the selection of predictors to include. The main thrust of this article is to propose and develop a procedure that uses probabilistic considerations for selecting promising subsets. This procedure entails embedding the regression setup in a hierarchical normal mixture model where latent variables are used to identify subset choices. In this framework the promising subsets of predictors can be identified as those with higher posterior probability. The computational burden is then alleviated by using the Gibbs sampler to indirectly sample from this multinomial posterior distribution on the set of possible subset choices. Those subsets with higher probability—the promising ones—can then be identified by their more frequent appearance in the Gibbs sample.

An exact likelihood analysis of the multinomial probit model

Abstract We develop new methods for conducting a finite sample, likelihood-based analysis of the multinomial probit model. Using a variant of the Gibbs sampler, an algorithm is developed to draw from the exact posterior of the multinomial probit model with correlated errors. This approach avoids direct evaluation of the likelihood and, thus, avoids the problems associated with calculating choice probabilities which affect both the standard likelihood and method of simulated moments approaches. Both simulated and actual consumer panel data are used to fit six-dimensional choice models. We also develop methods for analyzing random coefficient and multiperiod probit models.

Bayesian CART model search

Abstract In this article we put forward a Bayesian approach for finding classification and regression tree (CART) models. The two basic components of this approach consist of prior specification and stochastic search. The basic idea is to have the prior induce a posterior distribution that will guide the stochastic search toward more promising CART models. As the search proceeds, such models can then be selected with a variety of criteria, such as posterior probability, marginal likelihood, residual sum of squares or misclassification rates. Examples are used to illustrate the potential superiority of this approach over alternative methods.

A Bayesian analysis of the multinomial probit model with fully identified parameters

We present a new prior and corresponding algorithm for Bayesian analysis of the multinomial probit model. Our new approach places a prior directly on the identified parameter space. The key is the specification of a prior on the covariance matrix so that the (1,1) element if fixed at 1 and it is possible to draw from the posterior using standard distributions. Analytical results are derived which can be used to aid in assessment of the prior.

Bayesian Inference and Prediction for Mean and Variance Shifts in Autoregressive Time Series

Abstract This article is concerned with statistical inference and prediction of mean and variance changes in an autoregressive time series. We first extend the analysis of random mean-shift models to random variance-shift models. We then consider a method for predicting when a shift is about to occur. This involves appending to the autoregressive model a probit model for the probability that a shift occurs given a chosen set of explanatory variables. The basic computational tool we use in the proposed analysis is the Gibbs sampler. For illustration, we apply the analysis to several examples.

Bayesian Treed Models

When simple parametric models such as linear regression fail to adequately approximate a relationship across an entire set of data, an alternative may be to consider a partition of the data, and then use a separate simple model within each subset of the partition. Such an alternative is provided by a treed model which uses a binary tree to identify such a partition. However, treed models go further than conventional trees (e.g. CART, C4.5) by fitting models rather than a simple mean or proportion within each subset. In this paper, we propose a Bayesian approach for finding and fitting parametric treed models, in particular focusing on Bayesian treed regression. The potential of this approach is illustrated by a cross-validation comparison of predictive performance with neural nets, MARS, and conventional trees on simulated and real data sets.

Bayes and big data: the consensus Monte Carlo algorithm

A useful definition of ‘big data’ is data that is too big to process comfortably on a single machine, either because of processor, memory, or disk bottlenecks. Graphics processing units can alleviate the processor bottleneck, but memory or disk bottlenecks can only be eliminated by splitting data across multiple machines. Communication between large numbers of machines is expensive (regardless of the amount of data being communicated), so there is a need for algorithms that perform distributed approximate Bayesian analyses with minimal communication. Consensus Monte Carlo operates by running a separate Monte Carlo algorithm on each machine, and then averaging individual Monte Carlo draws across machines. Depending on the model, the resulting draws can be nearly indistinguishable from the draws that would have been obtained by running a single-machine algorithm for a very long time. Examples of consensus Monte Carlo are shown for simple models where single-machine solutions are available, for large single-layer hierarchical models, and for Bayesian additive regression trees (BART).

Local Model Influence

Abstract This article develops a general method for assessing the influence of model assumptions in a Bayesian analysis. We assume that model choices are indexed by a hyperparameter with some given initial choice. We use the term “model” to encompass both the sampling model and the prior distribution. We wish to assess the effect of changing the hyperparameter away from the initial choice. We are performing a sensitivity analysis, with the hyperparameter defining our perturbations. We use the Kullback—Leibler divergence to measure the difference between posteriors corresponding to different choices of the hyperparameter. We also measure the change in priors. If small changes in the priors lead to large changes in posteriors, the choice of hyperparameter is influential. The second-order difference in the Kullback—Leibler divergence is expressed by Fisher information matrices. The relative change in posteriors compared with priors may be summarized by the relative eigenvalue of the posterior and prior Fishe...

Account-Level Modeling for Trade Promotion: An Application of a Constrained Parameter Hierarchical Model

Abstract We consider the problem of utilizing data at the retail/market level on sales and marketing mix variables to help manufacturers optimize the allocation of trade promotional budgets across areas. Major consumer packaged goods manufacturers budget at least one-half of their total marketing expenses to trade promotions. Trade promotional deals are designed to encourage retailers to promote products by temporarily reducing the price, putting them in in-store displays, or advertising in local media. A profit-based trade promotional allocation system will require estimates of the responsiveness of sales at each retailer to a given promotion. A major barrier to the use of retailer data is the proliferation of incorrectly signed coefficients in standard least squares analyses. Even more sophisticated adaptive shrinkage methods will not remove the problem of improper signs. We propose a hierarchical model to modeling retailer response that uses a first-stage prior with inequality constraints on the regres...

Posterior, predictive, and utility-based approaches to testing the arbitrage pricing theory

Abstract To provide a framework for judging the economic significance of departures from the arbitrage pricing theory, we adopt a utility-based metric based on optimal portfolio choices. This measure is examined using both predictive and posterior analysis. Our predictive analysis shows very large and economically significant departures from the model restrictions. However, the high level of parameter uncertainty suggests that we cannot conclusively either affirm or reject the APT. Our conclusions differ markedly from other studies which employ traditional significance-testing procedures and, in many instances, fail to reject the APT restrictions.