Elías Moreno

An overview of robust Bayesian analysis

SummaryRobust Bayesian analysis is the study of the sensitivity of Bayesian answers to uncertain inputs. This paper seeks to provide an overview of the subject, one that is accessible to statisticians outside the field. Recent developments in the area are also reviewed, though with very uneven emphasis.

Objective Bayesian Variable Selection

A novel fully automatic Bayesian procedure for variable selection in normal regression models is proposed. The procedure uses the posterior probabilities of the models to drive a stochastic search. The posterior probabilities are computed using intrinsic priors, which can be considered default priors for model selection problems; that is, they are derived from the model structure and are free from tuning parameters. Thus they can be seen as objective priors for variable selection. The stochastic search is based on a Metropolis–Hastings algorithm with a stationary distribution proportional to the model posterior probabilities. The procedure is illustrated on both simulated and real examples.

CONSISTENCY OF BAYESIAN PROCEDURES FOR VARIABLE SELECTION

It has long been known that for the comparison of pairwise nested models, a decision based on the Bayes factor produces a consistent model selector (in the frequentist sense). Here we go beyond the usual consistency for nested pairwise models, and show that for a wide class of prior distributions, including intrinsic priors, the corresponding Bayesian procedure for variable selection in normal regression is consistent in the entire class of normal linear models. We find that the asymptotics of the Bayes factors for intrinsic priors are equivalent to those of the Schwarz (BIC) criterion. Also, recall that the Jeffreys–Lindley paradox refers to the well-known fact that a point null hypothesis on the normal mean parameter is always accepted when the variance of the conjugate prior goes to infinity. This implies that some limiting forms of proper prior distributions are not necessarily suitable for testing problems. Intrinsic priors are limits of proper prior distributions, and for finite sample sizes they have been proved to behave extremely well for variable selection in regression; a consequence of our results is that for intrinsic priors Lindley’s paradox does not arise.

Default Bayesian analysis of the Behrens–Fisher problem

Abstract In the Bayesian approach, the Behrens–Fisher problem has been posed as one of estimation for the difference of two means. No Bayesian solution to the Behrens–Fisher testing problem has yet been given due, perhaps, to the fact that the conventional priors used are improper. While default Bayesian analysis can be carried out for estimation purposes, it poses difficulties for testing problems. This paper generates sensible intrinsic and fractional prior distributions for the Behrens–Fisher testing problem from the improper priors commonly used for estimation. It allows us to compute the Bayes factor to compare the null and the alternative hypotheses. This default procedure of model selection is compared with a frequentist test and the Bayesian information criterion. We find discrepancy in the sense that frequentist and Bayesian information criterion reject the null hypothesis for data, that the Bayes factor for intrinsic or fractional priors do not.

Bayesian model selection approach to analysis of variance under heteroscedasticity

The classical Bayesian approach to analysis of variance assumes the homoscedastic condition and uses conventional uniform priors on the location parameters and on the logarithm of the common scale. The problem has been developed as one of estimation of location parameters. We argue that this does not lead to an appropriate Bayesian solution. A solution based on a Bayesian model selection procedure is proposed. Our development is in the general heteroscedastic setting in which a frequentist exact test does not exist. The Bayes factor involved uses intrinsic and fractional priors which are used instead of the usual default prior distributions for which the Bayes factor is not well defined. The behaviour of these Bayes factors is compared with the Bayesian information criterion of Schwarz and the frequentist asymptotic approximations of Welch and Brown and Forsythe.

Bayesian robustness in bidimensional models: Prior independence

Abstract When θ is a multidimensional parameter, the issue of prior dependence or independence of coordinates is a serious concern. This is especially true in robust Bayesian analysis; Lavine et al. (J. Amer. Statist. Assoc.86, 964–971 (1991)) show that allowing a wide range of prior dependencies among coordinates can result in near vacuous conclusions. It is sometimes possible, however, to make confidently the judgement that the coordinates of θ are independent a priori and, when this can be done, robust Bayesian conclusions improve dramatically. In this paper, it is shown how to incorporate the independence assumption into robust Bayesian analysis involving ϵ-contamination and density band classes of priors. Attention is restricted to the case θ = (θ1, θ2) for clarity, although the ideas generalize.

Estimating with incomplete count data A Bayesian approach

Abstract Let xt be a random process which is distributed as an homogeneous Poisson process P(x t ¦λt) . We assume that λ is unknown and xt is unobservable, but instead we are able to observe another process yt which is an unknown proportion, say θ, of xt for every t. The goal is to make inferences on xt, conditional on the observed yt. Assuming that the distribution of yt given xt and θ is Binomial, Bi (y t ¦x t , θ) , then in order to make inferences on xt, a distribution for the pair λ, θ has, a priori, to be elicited. The relationship between the hierarchical formulation of the problem and the issues of prior independence of the unknown parameters is analysed. Robustness with respect to the prior is also considered. This problem is of interest in criminology where xt might represent the number of crimes committed in a given place during a period of timelength t, and yt the reported number of crimes in that period. The resulting inferencial procedures are illustrated with real data on reported assaults in Malaga in 1993 and in Stockholm for the years 1980–1986.

A default Bayesian test for the number of components in a mixture

Abstract In the last few years, there has been an increasing interest for default Bayes methods for hypothesis testing and model selection. The availability of such methods is potentially very useful in mixture models, where the elicitation process on the (unknown number of) parameters is usually rather difficult. Two recent yet already popular approaches, namely intrinsic Bayes factor (J. Amer. Statist. Assoc. 91 (1996) 109) and fractional Bayes factor (J. Roy. Statist. Soc. Ser. B 57 (1995) 99), have been proven quite successful in generating sensible prior distributions, to compute actual Bayes factors. From a theoretical viewpoint, the application of these methods to a mixture model selection problem involves two difficulties. The first is the choice of a “good” default prior for the mixture. The second problem is related to the fact that, for improper default priors, the prior predictive distribution of the data need not exist; In this paper, we argue that the problem of choosing among mixture models can be reduced to the problem of comparing models with simpler structures. It is shown that these simpler models can be compared via standard default Bayesian methods.

Objective Bayes Model Selection in Probit Models

We describe a new variable selection procedure for categorical responses where the candidate models are all probit regression models. The procedure uses objective intrinsic priors for the model parameters, which do not depend on tuning parameters, and ranks the models for the different subsets of covariates according to their model posterior probabilities. When the number of covariates is moderate or large, the number of potential models can be very large, and for those cases, we derive a new stochastic search algorithm that explores the potential sets of models driven by their model posterior probabilities. The algorithm allows the user to control the dimension of the candidate models and thus can handle situations when the number of covariates exceed the number of observations. We assess, through simulations, the performance of the procedure and apply the variable selector to a gene expression data set, where the response is whether a patient exhibits pneumonia. Software needed to run the procedures is available in the R package varselectIP.

Optimal healthcare decisions: Comparing medical treatments on a cost-effectiveness basis

This paper deals with medical treatments comparison from the cost-effectiveness viewpoint. A decision theory scheme is considered, where the decision space is the set of treatments involved, the space of states of nature consists of the respective net benefits of the treatments, and the utility function is one of two possible candidates. A first candidate is the one typically used in the literature on cost-effectiveness analysis, for which the utility of a decision is proportional to the net benefit gain, and a second one is of the type 0-1, which penalizes the wrong decisions with a fixed quantity. Their associated optimal decision rules, both frequentist and Bayesian, are analyzed and compared via frequentist evaluation of their performance, and the conclusion is that the latter beats the former in the sense of choosing the optimal treatment more often than any other, thus minimizing the proportion of wrong decisions. Illustrations with simulated and real data are provided.