Bertrand Clarke

Information-theoretic asymptotics of Bayes methods

In the absence of knowledge of the true density function, Bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior. The authors examine the relative entropy distance D/sub n/ between the true density and the Bayesian density and show that the asymptotic distance is (d/2)(log n)+c, where d is the dimension of the parameter vector. Therefore, the relative entropy rate D/sub n//n converges to zero at rate (log n)/n. The constant c, which the authors explicitly identify, depends only on the prior density function and the Fisher information matrix evaluated at the true parameter value. Consequences are given for density estimation, universal data compression, composite hypothesis testing, and stock-market portfolio selection. >

Jeffreys prior is asymptotically least favorable under entropy risk

We provide a rigorous proof that Jeffreys’ prior asymptotically maximizes Shannon’s mutual information between a sample of size n and the parameter. This was conjectured by Bernard0 (1979) and, despite the absence of a proof, forms the basis of the reference prior method in Bayesian statistical analysis. Our proof rests on an examination of large sample decision theoretic properties associated with the relative entropy or the Kullback-Leibler distance between probability density functions for independent and identically distributed random variables. For smooth finite-dimensional parametric families we derive an asymptotic expression for the minimax risk and for the related maximin risk. As a result, we show that, among continuous positive priors, Jeffreys’ prior uniquely achieves the asymptotic maximin value. In the discrete parameter case we show that, asymptotically, the Bayes risk reduces to the entropy of the prior so that the reference prior is seen to be the maximum entropy prior. We identify the physical significance of the risks by giving two information-theoretic interpretations in terms of probabilistic coding. AMS Subject Class

Statistical expression deconvolution from mixed tissue samples

caGon: Primary 62C10, 62C20; secondary 62F12, 62F15.

Implications of Reference Priors for Prior Information and for Sample Size

Motivation: Global expression patterns within cells are used for purposes ranging from the identification of disease biomarkers to basic understanding of cellular processes. Unfortunately, tissue samples used in cancer studies are usually composed of multiple cell types and the non-cancerous portions can significantly affect expression profiles. This severely limits the conclusions that can be made about the specificity of gene expression in the cell-type of interest. However, statistical analysis can be used to identify differentially expressed genes that are related to the biological question being studied. Results: We propose a statistical approach to expression deconvolution from mixed tissue samples in which the proportion of each component cell type is unknown. Our method estimates the proportion of each component in a mixed tissue sample; this estimate can be used to provide estimates of gene expression from each component. We demonstrate our technique on xenograft samples from breast cancer research and publicly available experimental datasets found in the National Center for Biotechnology Information Gene Expression Omnibus repository. Availability: R code (http://www.r-project.org/) for estimating sample proportions is freely available to non-commercial users and available at http://www.med.miami.edu/medicine/x2691.xml Contact: jclarke@med.miami.edu

Partial information reference priors: derivation and interpretations

Abstract Here we use posterior densities based on relative entropy reference priors for two purposes. The first purpose is to identify data implicit in the use of informative priors. We represent an informative prior as the posterior from an experiment with a known likelihood and a reference prior. Minimizing the relative entropy distance between this posterior and the informative prior over choices of data results in a data set that can be regarded as representative of the information in the informative prior. The second implication from reference priors is obtained by replacing the informative prior with a class of densities from which one might wish to make inferences. For each density in this class, one can obtain a data set that minimizes a relative entropy. The maximum of these sample sizes as the inferential density varies over its class can be used as a guess as to how much data is required for the desired inferences. We bound this sample size above and below by other techniques that permit it to ...

On the overall sensitivity of the posterior distribution to its inputs

Abstract Suppose X1,…,Xn are IID p(·|θ,ψ) where (θ,ψ)∈ R d is distributed according to the prior density w(·). For estimators S n =S( X ) and T n =T( X ) assumed to be consistent for some function of θ and asymptotically normal, we examine the conditional Shannon mutual information (CSMI) between Θ and Tn given Ψ and Sn, I(Θ,Tn|Ψ,Sn). It is seen there are several important special cases of this CSMI. We establish asymptotic formulas for various cases and identify the resulting noninformative reference priors. As a consequence, we develop the notion of data-dependent priors and a calibration for how close an estimator is to sufficiency.

Asymptotic normality of the posterior in relative entropy

In a parametric Bayesian analysis, the posterior distribution of the parameter is determined by three inputs: the prior distribution of the parameter, the model distribution of the data given the parameter, and the data themselves. Working in the framework of two particular families of parametric models with conjugate priors, we develop a method for quantifying the local sensitivity of the posterior to simultaneous perturbations of all three inputs. The method uses relative entropy to measure discrepancies between pairs of posterior distributions, model distributions, and prior distributions. It also requires a measure of discrepancy between pairs of data sets. The fundamental sensitivity measure is taken to be the maximum discrepancy between a baseline posterior and a perturbed posterior, given a constraint on the size of the discrepancy between the baseline set of inputs and the perturbed inputs. We also examine the perturbed inputs which attain this maximum sensitivity, to see how influential the prior, model, and data are relative to one another. An empirical study highlights some interesting connections between sensitivity and the extent to which the data conflict with both the prior and the model.

A Bayesian test for excess zeros in a zero-inflated power series distribution

We show that the relative entropy between a posterior density formed from a smooth likelihood and prior and a limiting normal form tends to zero in the independent and identically distributed case. The mode of convergence is in probability and in mean. Applications to code lengths in stochastic complexity and to sample size selection are discussed.

Closed form expressions for Bayesian sample size

Power series distributions form a useful subclass of one-parameter discrete exponential families suitable for modeling count data. A zero-inflated power series distribution is a mixture of a power series distribution and a degenerate distribution at zero, with a mixing probability

Noninformative priors and nuisance parameters

p