Eduardo Gutiérrez-Peña

Exponential and bayesian conjugate families: Review and extensions

SummaryThe notion of a conjugate family of distributions plays a very important role in the Bayesian approach to parametric inference. One of the main features of such a family is that it is closed under sampling, but a conjugate family often provides prior distributions which are tractable in various other respects. This paper is concerned with the properties of conjugate families for exponential family models. Special attention is given to the class of natural exponential families having a quadratic variance function, for which the theory is particularly fruitful. Several classes of conjugate families have been considered in the literature and here we describe some of their most interesting features. Relationships between such classes are also discussed. Our aim is to provide a unified approach to the theory of conjugate families for exponential family likelihoods. An important aspect of the theory concerns reparameterisations of the exponential family under consideration. We briefly review the concept of a conjugate parameterisation, which provides further insight into many of the properties discussed throughout the paper. Finally, further implications of these results for Bayesian conjugate analysis of exponential families are investigated.

A Bayesian Analysis of Directional Data Using the von Mises–Fisher Distribution

ABSTRACT This article presents a Bayesian analysis of the von Mises–Fisher distribution, which is the most important distribution in the analysis of directional data. We obtain samples from the posterior distribution using a sampling-importance-resampling method. The procedure is illustrated using simulated data as well as real data sets previously analyzed in the literature.

A Bayesian predictive approach to model selection

Abstract This paper proposes a predictive approach to Bayesian model selection based on independent and identically distributed observations. In particular, we generalise the criterion of San Martini and Spezzaferri (J. Roy. Statist. Soc. B 46 (1984) 296–303) to take into account more realistic views as discussed by Bernardo and Smith (Bayesian Theory. Wiley, Chichester, 1994). The former authors only consider what the latter authors name the M -closed view; that is, the assumption that one of the competing models is the true model. More realistic is the M -open view in which it is believed that none of the competing models is the true model. Our new approach can encompass both of these views and moreover we introduce the M -mixture view where the experimenter can express prior opinion concerning his/her belief as to whether one of the competing models is the true model or not. Essentially, we embed the M -open view in a larger (nonparametric) M -closed view.

A Bayesian analysis of directional data using the projected normal distribution

Abstract This paper presents a Bayesian analysis of the projected normal distribution, which is a flexible and useful distribution for the analysis of directional data. We obtain samples from the posterior distribution using the Gibbs sampler after the introduction of suitably chosen latent variables. The procedure is illustrated using simulated data as well as a real data set previously analysed in the literature.

A Bayesian regression model for circular data based on the projected normal distribution

Inferences based on regression models for a directional response are usually problematic. This paper presents a Bayesian analysis of a regression model for circular data using the projected normal distribution. Inferences about the model are based on samples from the posterior densities which are obtained using the Gibbs sampler after the introduction of suitable latent variables. The problem of missing data in the response variable is also addressed in this context as is the use of a predictive criterion for model selection. The procedures are illustrated using two simulated datasets a dataset previously analysed in the literature and a real dataset concerning wind directions.

Bayesian conjugate analysis of the Galton-Watson process

In this article we consider the Bayesian statistical analysis of a simple Galton-Watson process. Problems of interest include estimation of the offspring distribution, classification of the process, and prediction. We propose two simple analytic approximations to the posterior marginal distribution of the reproduction mean. This posterior distribution suffices to classify the process. In order to assess the accuracy of these approximations, a comparison is provided with a computationally more expensive approximation obtained via standard Monte Carlo techniques. Similarly, a fully analytic approximation to the predictive distribution of the future size of the population is discussed. Sampling-based and hybrid approximations to this distribution are also considered. Finally, we present some illustrative examples.

Reference priors for exponential families with simple quadratic variance function

Reference analysis is one of the most successful general methods to derive noninformative prior distributions. In practice, however, reference priors are often difficult to obtain. Recently developed theory for conditionally reducible natural exponential families identifies an attractive reparameterization which allows one, among other things, to construct an enriched conjugate prior. In this paper, under the assumption that the variance function is simple quadratic, the order-invariant group reference prior for the above parameter is found. Furthermore, group reference priors for the mean- and natural parameter of the families are obtained. A brief discussion of the frequentist coverage properties is also presented. The theory is illustrated for the multinomial and negative-multinomial family. Posterior computations are especially straightforward due to the fact that the resulting reference distributions belong to the corresponding enriched conjugate family. A substantive application of the theory relates to the construction of reference priors for the Bayesian analysis of two-way contingency tables with respect to two alternative parameterizations.

Reference priors for exponential families

Abstract Reference analysis, introduced by Bernardo (J. Roy. Statist. Soc. 41 (1979) 113) and further developed by Berger and Bernardo (On the development of reference priors (with discussion). In: J.M. Bernardo, J.O. Berger, A.P. Dawid, A.F.M. Smith (Eds.), Bayesian Statistics, Vol. 4, Clarendon Press, Oxford, pp. 35–60), has proved to be one of the most successful general methods to derive noninformative prior distributions. In practice, however, reference priors are typically difficult to obtain. In this paper we show how to find reference priors for a wide class of exponential family likelihoods.

A Note on Whittle's Likelihood

The approximate likelihood function introduced by Whittle has been used to estimate the spectral density and certain parameters of a variety of time series models. In this note we attempt to empirically quantify the loss of efficiency of Whittles method in nonstandard settings. A recently developed representation of some first-order non-Gaussian stationary autoregressive process allows a direct comparison of the true likelihood function with that of Whittle. The conclusion is that Whittles likelihood can produce unreliable estimates in the non-Gaussian case, even for moderate sample sizes. Moreover, for small samples, and if the autocorrelation of the process is high, Whittles approximation is not efficient even in the Gaussian case. While these facts are known to some extent, the present study sheds more light on the degree of efficiency loss incurred by using Whittles likelihood, in both Gaussian and non-Gaussian cases.

Laplace Approximations for Natural Exponential Families with Cuts

Standard and fully exponential form Laplace approximations to marginal densities are described and conditions under which these give exact answers are investigated. A general result is obtained and is subsequently applied in the case of natural exponential families with cuts, in order to derive the marginal posterior density of the mean parameter corresponding to the cut, the canonical parameter corresponding to the complement of the cut and transformations of these. Important cases of families for which a cut exists and the approximations are exact are presented as examples.