Michael Stanley Smith

Nonparametric regression using Bayesian variable selection

Abstract This paper estimates an additive model semiparametrically, while automatically selecting the significant independent variables and the appropriate power transformation of the dependent variable. The nonlinear variables are modeled as regression splines, with significant knots selected from a large number of candidate knots. The estimation is made robust by modeling the errors as a mixture of normals. A Bayesian approach is used to select the significant knots, the power transformation, and to identify outliers using the Gibbs sampler to carry out the computation. Empirical evidence is given that the sampler works well on both simulated and real examples and that in the univariate case it compares favorably with a kernel-weighted local linear smoother. The variable selection algorithm in the paper is substantially faster than previous Bayesian variable selection algorithms.

Parsimonious Covariance Matrix Estimation for Longitudinal Data

This article proposes a data-driven method to identify parsimony in the covariance matrix of longitudinal data and to exploit any such parsimony to produce a statistically efficient estimator of the covariance matrix. The approach parameterizes the covariance matrix through the Cholesky decomposition of its inverse. For longitudinal data, this is a one-step-ahead predictive representation, and the Cholesky factor is likely to have off-diagonal elements that are zero or close to zero. A hierarchical Bayesian model is used to identify any such zeros in the Cholesky factor, similar to approaches that have been successful in Bayesian variable selection. The model is estimated using a Markov chain Monte Carlo sampling scheme that is computationally efficient and can be applied to covariance matrices of high dimension. It is demonstrated through simulations that the proposed method compares favorably in terms of statistical efficiency with a highly regarded competing approach. The estimator is applied to three real examples in which the dimension of the covariance matrix is large relative to the sample size. The first two examples are from biometry and electricity demand modeling and are longitudinal. The third example is from finance and highlights the potential of our method for estimating cross-sectional covariance matrices.

Nonparametric regression using linear combinations of basis functions

This paper discusses a Bayesian approach to nonparametric regression initially proposed by Smith and Kohn (1996. Journal of Econometrics 75: 317–344). In this approach the regression function is represented as a linear combination of basis terms. The basis terms can be univariate or multivariate functions and can include polynomials, natural splines and radial basis functions. A Bayesian hierarchical model is used such that the coefficient of each basis term can be zero with positive prior probability. The presence of basis terms in the model is determined by latent indicator variables. The posterior mean is estimated by Markov chain Monte Carlo simulation because it is computationally intractable to compute the posterior mean analytically unless a small number of basis terms is used. The present article updates the work of Smith and Kohn (1996. Journal of Econometrics 75: 317–344) to take account of work by us and others over the last three years. A careful discussion is given to all aspects of the model specification, function estimation and the use of sampling schemes. In particular, new sampling schemes are introduced to carry out the variable selection methodology.

Bayesian Modeling and Forecasting of Intraday Electricity Load

The advent of wholesale electricity markets has brought renewed focus on intraday electricity load forecasting. This article proposes a multi-equation regression model with a diagonal first-order stationary vector autoregresson (VAR) for modeling and forecasting intraday electricity load. The correlation structure of the disturbances to the VAR and the appropriate subset of regressors are explored using Bayesian model selection methodology. The full spectrum of finite-sample inference is obtained using a Bayesian Markov chain Monte Carlo sampling scheme. This includes the predictive distribution of load and the distribution of the time and level of daily peak load, something that is difficult to obtain with other methods of inference. The method is applied to several multiequation models of half-hourly total system load in New South Wales, Australia. A detailed model based on 3 years of data reveals trend, seasonal, bivariate temperature/humidity, and serial correlation components that all vary intraday, justifying the assumption of a multiequation approach. Short-term forecasts from simple models highlight the gains that can be made if accurate temperature predictions are exploited. Bayesian predictive means for half-hourly load compare favorably with point forecasts obtained using iterated generalized least squares estimation of the same models.

Spatial Bayesian Variable Selection With Application to Functional Magnetic Resonance Imaging

We propose a procedure to undertake Bayesian variable selection and model averaging for a series of regressions located on a lattice. For those regressors that are in common in the regressions, we consider using an Ising prior to smooth spatially the indicator variables representing whether or not the variable is zero or nonzero in each regression. This smooths spatially the probabilities that each independent variable is nonzero in each regression and indirectly smooths spatially the regression coefficients. We discuss how single-site sampling schemes can be used to evaluate the joint posterior distribution. The approach is applied to the problem of functional magnetic resonance imaging in medical statistics, where massive datasets arise that require prompt processing. Here the Ising prior with a three-dimensional neighborhood structure is used to smooth spatially activation maps from regression models of blood oxygenation. The Ising prior also has the advantage of allowing incorporation of anatomic prior information through the external field. Using a visual experiment, we show how a single-site sampling scheme can provide rapid evaluation of the posterior activation maps and activation amplitudes. The approach is shown to result in maps that are superior to those produced by a recent Bayesian approach using a continuous Markov random field for the activation amplitude.

Modeling Longitudinal Data Using a Pair-Copula Decomposition of Serial Dependence

Copulas have proven to be very successful tools for the flexible modeling of cross-sectional dependence. In this paper we express the dependence structure of continuous-valued time series data using a sequence of bivariate copulas. This corresponds to a type of decomposition recently called a “vine” in the graphical models literature, where each copula is entitled a “pair-copula.” We propose a Bayesian approach for the estimation of this dependence structure for longitudinal data. Bayesian selection ideas are used to identify any independence pair-copulas, with the end result being a parsimonious representation of a time-inhomogeneous Markov process of varying order. Estimates are Bayesian model averages over the distribution of the lag structure of the Markov process. Using a simulation study we show that the selection approach is reliable and can improve the estimates of both conditional and unconditional pairwise dependencies substantially. We also show that a vine with selection outperforms a Gaussian copula with a flexible correlation matrix. The advantage of the pair-copula formulation is further demonstrated using a longitudinal model of intraday electricity load. Using Gaussian, Gumbel, and Clayton pair-copulas we identify parsimonious decompositions of intraday serial dependence, which improve the accuracy of intraday load forecasts. We also propose a new diagnostic for measuring the goodness of fit of high-dimensional multivariate copulas. Overall, the pair-copula model is very general and the Bayesian method generalizes many previous approaches for the analysis of longitudinal data. Supplemental materials for the article are also available online.

Modeling Multivariate Distributions Using Copulas: Applications in Marketing

In this research we introduce a new class of multivariate probability models to the marketing literature. Known as “copula models,” they have a number of attractive features. First, they permit the combination of any univariate marginal distributions that need not come from the same distributional family. Second, a particular class of copula models, called “elliptical copula,” has the property that they increase in complexity at a much slower rate than existing multivariate probability models as the number of dimensions increase. Third, they are very general, encompassing a number of existing multivariate models and providing a framework for generating many more. These advantages give copula models a greater potential for use in empirical analysis than existing probability models used in marketing. We exploit and extend recent developments in Bayesian estimation to propose an approach that allows reliable estimation of elliptical copula models in high dimensions. Rather than focusing on a single marketing problem, we demonstrate the versatility and accuracy of copula models with four examples to show the flexibility of the method. In every case, the copula model either handles a situation that could not be modeled previously or gives improved accuracy compared with prior models.

Where, When, and How Long: Factors That Influence the Redemption of Mobile Phone Coupons

The use of coupons delivered by mobile phone, so-called “m-coupons,” is growing rapidly. In this study, the authors analyze consumer response to m-coupons for a two-year trial at a large shopping mall. Approximately 8,500 people were recruited to a panel and received three text-message m-coupons whenever they “swiped” their mobile phone at the mall entrances, with downstream redemption recorded. Almost 144,000 m-coupons were delivered during the trial, representing 38 stores that supplied 134 different coupons. The authors find that an important feature of m-coupons is where and when they are delivered, with location and time of delivery significantly influencing redemption. How long the m-coupons are valid (expiry length) is also important because redemption times for m-coupons are much shorter than for traditional coupons. This finding suggests that their expiration length should be shortened to help signal time urgency. Nevertheless, traditional coupon features, such as face value, still dominate m-coupon effectiveness, as does the product type, with snack food coupons being particularly effective.

Modelling dependence using skew t copulas: Bayesian inference and applications

We construct a copula from the skew t distribution of Sahu, Dey & Branco (2003). This copula can capture asymmetric and extreme dependence between variables, and is one of the few copulas that can do so and still be used in high dimensions effectively. However, it is difficult to estimate the copula model by maximum likelihood when the multivariate dimension is high, or when some or all of the marginal distributions are discrete-valued, or when the parameters in the marginal distributions and copula are estimated jointly. We therefore propose a Bayesian approach that overcomes all these problems. The computations are undertaken using a Markov chain Monte Carlo simulation method which exploits the conditionally Gaussian representation of the skew t distribution. We employ the approach in two contemporary econometric studies. The first is the modeling of regional spot prices in the Australian electricity market. Here, we observe complex non-Gaussian margins and nonlinear inter-regional dependence. Accurate characterization of this dependence is important for the study of market integration and risk management purposes. The second is the modeling of ordinal exposure measures for 15 major websites. Dependence between websites is important when measuring the impact of multi-site advertising campaigns. In both cases the skew t copula substantially out-performs symmetric elliptical copula alternatives, demonstrating that the skew t copula is a powerful modeling tool when coupled with Bayesian inference.

Modeling and Short-Term Forecasting of New South Wales Electricity System Load

This article employs Bayesian semiparametric regression methodology to model intraday electricity load data and obtain short-term load forecasts. The role of such forecasts in the New South Wales wholesale electricity market is discussed and the method applied to New South Wales system load data. The semiparametric regression model used identifies daily periodic, weekly periodic, and temperature-sensitive components of load. Each component is decomposed as a linear combination of basis functions, with a nonzero probability mass that the corresponding coefficients are exactly zero. Three possible models for the errors are also considered, including independent, autoregressive, and first-differenced autoregressive models. A moving window of data is used to overcome the slow time-varying nature of the temperature and periodic effects. The entire model is estimated using a Bayesian Markov chain Monte Carlo approach, and forecasts are obtained using a Monte Carlo sample from the joint predictive distribution of future system load. It is demonstrated how accurate temperature forecasts can result in accurate intraday system load forecasts for even quite long forecast horizons.