George Casella

Explaining the Gibbs Sampler

Abstract Computer-intensive algorithms, such as the Gibbs sampler, have become increasingly popular statistical tools, both in applied and theoretical work. The properties of such algorithms, however, may sometimes not be obvious. Here we give a simple explanation of how and why the Gibbs sampler works. We analytically establish its properties in a simple case and provide insight for more complicated cases. There are also a number of examples.

The Bayesian Lasso

The Lasso estimate for linear regression parameters can be interpreted as a Bayesian posterior mode estimate when the regression parameters have independent Laplace (i.e., double-exponential) priors. Gibbs sampling from this posterior is possible using an expanded hierarchy with conjugate normal priors for the regression parameters and independent exponential priors on their variances. A connection with the inverse-Gaussian distribution provides tractable full conditional distributions. The Bayesian Lasso provides interval estimates (Bayesian credible intervals) that can guide variable selection. Moreover, the structure of the hierarchical model provides both Bayesian and likelihood methods for selecting the Lasso parameter. Slight modifications lead to Bayesian versions of other Lasso-related estimation methods, including bridge regression and a robust variant.

Pyrosequencing enumerates and contrasts soil microbial diversity.

Estimates of the number of species of bacteria per gram of soil vary between 2000 and 8.3 million (Gans et al., 2005; Schloss and Handelsman, 2006). The highest estimate suggests that the number may be so large as to be impractical to test by amplification and sequencing of the highly conserved 16S rRNA gene from soil DNA (Gans et al., 2005). Here we present the use of high throughput DNA pyrosequencing and statistical inference to assess bacterial diversity in four soils across a large transect of the western hemisphere. The number of bacterial 16S rRNA sequences obtained from each site varied from 26 140 to 53 533. The most abundant bacterial groups in all four soils were the Bacteroidetes, Betaproteobacteria and Alphaproteobacteria. Using three estimators of diversity, the maximum number of unique sequences (operational taxonomic units roughly corresponding to the species level) never exceeded 52 000 in these soils at the lowest level of dissimilarity. Furthermore, the bacterial diversity of the forest soil was phylum rich compared to the agricultural soils, which are species rich but phylum poor. The forest site also showed far less diversity of the Archaea with only 0.009% of all sequences from that site being from this group as opposed to 4%–12% of the sequences from the three agricultural sites. This work is the most comprehensive examination to date of bacterial diversity in soil and suggests that agricultural management of soil may significantly influence the diversity of bacteria and archaea.

Toward defining the autoimmune microbiome for type 1 diabetes

Several studies have shown that gut bacteria have a role in diabetes in murine models. Specific bacteria have been correlated with the onset of diabetes in a rat model. However, it is unknown whether human intestinal microbes have a role in the development of autoimmunity that often leads to type 1 diabetes (T1D), an autoimmune disorder in which insulin-secreting pancreatic islet cells are destroyed. High-throughput, culture-independent approaches identified bacteria that correlate with the development of T1D-associated autoimmunity in young children who are at high genetic risk for this disorder. The level of bacterial diversity diminishes overtime in these autoimmune subjects relative to that of age-matched, genotype-matched, nonautoimmune individuals. A single species, Bacteroides ovatus, comprised nearly 24% of the total increase in the phylum Bacteroidetes in cases compared with controls. Conversely, another species in controls, represented by the human firmicute strain CO19, represented nearly 20% of the increase in Firmicutes compared with cases overtime. Three lines of evidence are presented that support the notion that, as healthy infants approach the toddler stage, their microbiomes become healthier and more stable, whereas, children who are destined for autoimmunity develop a microbiome that is less diverse and stable. Hence, the autoimmune microbiome for T1D may be distinctly different from that found in healthy children. These data also suggest bacterial markers for the early diagnosis of T1D. In addition, bacteria that negatively correlated with the autoimmune state may prove to be useful in the prevention of autoimmunity development in high-risk children.

The Effect of Improper Priors on Gibbs Sampling in Hierarchical Linear Mixed Models

Abstract Often, either from a lack of prior information or simply for convenience, variance components are modeled with improper priors in hierarchical linear mixed models. Although the posterior distributions for these models are rarely available in closed form, the usual conjugate structure of the prior specification allows for painless calculation of the Gibbs conditionals. Thus the Gibbs sampler may be used to explore the posterior distribution without ever having established propriety of the posterior. An example is given showing that the output from a Gibbs chain corresponding to an improper posterior may appear perfectly reasonable. Thus one cannot expect the Gibbs output to provide a “red flag,” informing the user that the posterior is improper. The user must demonstrate propriety before a Markov chain Monte Carlo technique is used. A theorem is given that classifies improper priors according to the propriety of the resulting posteriors. Applications concerning Bayesian analysis of animal breeding...

Gut Microbiome Metagenomics Analysis Suggests a Functional Model for the Development of Autoimmunity for Type 1 Diabetes

Recent studies have suggested a bacterial role in the development of autoimmune disorders including type 1 diabetes (T1D). Over 30 billion nucleotide bases of Illumina shotgun metagenomic data were analyzed from stool samples collected from four pairs of matched T1D case-control subjects collected at the time of the development of T1D associated autoimmunity (i.e., autoantibodies). From these, approximately one million open reading frames were predicted and compared to the SEED protein database. Of the 3,849 functions identified in these samples, 144 and 797 were statistically more prevalent in cases and controls, respectively. Genes involved in carbohydrate metabolism, adhesions, motility, phages, prophages, sulfur metabolism, and stress responses were more abundant in cases while genes with roles in DNA and protein metabolism, aerobic respiration, and amino acid synthesis were more common in controls. These data suggest that increased adhesion and flagella synthesis in autoimmune subjects may be involved in triggering a T1D associated autoimmune response. Extensive differences in metabolic potential indicate that autoimmune subjects have a functionally aberrant microbiome. Mining 16S rRNA data from these datasets showed a higher proportion of butyrate-producing and mucin-degrading bacteria in controls compared to cases, while those bacteria that produce short chain fatty acids other than butyrate were higher in cases. Thus, a key rate-limiting step in butyrate synthesis is more abundant in controls. These data suggest that a consortium of lactate- and butyrate-producing bacteria in a healthy gut induce a sufficient amount of mucin synthesis to maintain gut integrity. In contrast, non-butyrate-producing lactate-utilizing bacteria prevent optimal mucin synthesis, as identified in autoimmune subjects.

Fecal microbiota in premature infants prior to necrotizing enterocolitis.

Intestinal luminal microbiota likely contribute to the etiology of necrotizing enterocolitis (NEC), a common disease in preterm infants. Microbiota development, a cascade of initial colonization events leading to the establishment of a diverse commensal microbiota, can now be studied in preterm infants using powerful molecular tools. Starting with the first stool and continuing until discharge, weekly stool specimens were collected prospectively from infants with gestational ages ≤32 completed weeks or birth weights≤1250 g. High throughput 16S rRNA sequencing was used to compare the diversity of microbiota and the prevalence of specific bacterial signatures in nine NEC infants and in nine matched controls. After removal of short and low quality reads we retained a total of 110,021 sequences. Microbiota composition differed in the matched samples collected 1 week but not <72 hours prior to NEC diagnosis. We detected a bloom (34% increase) of Proteobacteria and a decrease (32%) in Firmicutes in NEC cases between the 1 week and <72 hour samples. No significant change was identified in the controls. At both time points, molecular signatures were identified that were increased in NEC cases. One of the bacterial signatures detected more frequently in NEC cases (p<0.01) matched closest to γ-Proteobacteria. Although this sequence grouped to the well-studied Enterobacteriaceae family, it did not match any sequence in Genbank by more than 97%. Our observations suggest that abnormal patterns of microbiota and potentially a novel pathogen contribute to the etiology of NEC.

An Introduction to Empirical Bayes Data Analysis

Abstract Empirical Bayes methods have been shown to be powerful data-analysis tools in recent years. The empirical Bayes model is much richer than either the classical or the ordinary Bayes model and often provides superior estimates of parameters. An introduction to some empirical Bayes methods is given, and these methods are illustrated with two examples.

Reconciling Bayesian and Frequentist Evidence in the One-Sided Testing Problem

Abstract For the one-sided hypothesis testing problem it is shown that it is possible to reconcile Bayesian evidence against H 0, expressed in terms of the posterior probability that H 0 is true, with frequentist evidence against H 0, expressed in terms of the p value. In fact, for many classes of prior distributions it is shown that the infimum of the Bayesian posterior probability of H 0 is equal to the p value; in other cases the infimum is less than the p value. The results are in contrast to recent work of Berger and Sellke (1987) in the two-sided (point null) case, where it was found that the p value is much smaller than the Bayesian infimum. Some comments on the point null problem are also given.

Penalized regression, standard errors, and Bayesian lassos

Penalized regression methods for simultaneous variable selection and coe-cient estimation, especially those based on the lasso of Tibshirani (1996), have received a great deal of attention in recent years, mostly through frequen- tist models. Properties such as consistency have been studied, and are achieved by difierent lasso variations. Here we look at a fully Bayesian formulation of the problem, which is ∞exible enough to encompass most versions of the lasso that have been previously considered. The advantages of the hierarchical Bayesian for- mulations are many. In addition to the usual ease-of-interpretation of hierarchical models, the Bayesian formulation produces valid standard errors (which can be problematic for the frequentist lasso), and is based on a geometrically ergodic Markov chain. We compare the performance of the Bayesian lassos to their fre- quentist counterparts using simulations, data sets that previous lasso papers have used, and a di-cult modeling problem for predicting the collapse of governments around the world. In terms of prediction mean squared error, the Bayesian lasso performance is similar to and, in some cases, better than, the frequentist lasso.