Statistics

Bayesian synthetic likelihood (BSL) is a popular method for estimating the parameter posterior distribution for complex statistical models and stochastic processes that possess a computationally intractable likelihood function. Instead of evaluating the likelihood, BSL approximates the likelihood of a judiciously chosen summary statistic of the data via model simulation and density estimation. Compared to alternative methods such as approximate Bayesian computation (ABC), BSL requires little tuning and requires less model simulations than ABC when the chosen summary statistic is high-dimensional. The original synthetic likelihood relies on a multivariate normal approximation of the intractable likelihood, where the mean and covariance are estimated by simulation. An extension of BSL considers replacing the sample covariance with a penalised covariance estimator to reduce the number of required model simulations. Further, a semi-parametric approach has been developed to relax the normality assumption. In this paper, we present an R package called BSL that amalgamates the aforementioned methods and more into a single, easy-to-use and coherent piece of software. The R package also includes several examples to illustrate how to use the package and demonstrate the utility of the methods.

The Backward Simulation (BS) approach was developed to generate, simply and efficiently, sample paths of correlated multivariate Poisson process with negative correlation coefficients between their components. In this paper, we extend the BS approach to model multivariate Mixed Poisson processes which have many important applications in Insurance, Finance, Geophysics and many other areas of Applied Probability. We also extend the Forward Continuation approach, introduced in our earlier work, to multivariate Mixed Poisson processes.

The popularity of Bayesian statistical methods has increased dramatically in recent years across many research areas and industrial applications. This is the result of a variety of methodological advances with faster and cheaper hardware as well as the development of new software tools. Here we introduce an open source Python package named Bambi (BAyesian Model Building Interface) that is built on top of the PyMC3 probabilistic programming framework and the ArviZ package for exploratory analysis of Bayesian models. Bambi makes it easy to specify complex generalized linear hierarchical models using a formula notation similar to those found in the popular R packages lme4, nlme, rstanarm and brms. We demonstrate Bambi's versatility and ease of use with a few examples spanning a range of common statistical models including multiple regression, logistic regression, and mixed-effects modeling with crossed group specific effects. Additionally we discuss how automatic priors are constructed. Finally, we conclude with a discussion of our plans for the future development of Bambi.

This article surveys computational methods for posterior inference with intractable likelihoods, that is where the likelihood function is unavailable in closed form, or where evaluation of the likelihood is infeasible. We review recent developments in pseudo-marginal methods, approximate Bayesian computation (ABC), the exchange algorithm, thermodynamic integration, and composite likelihood, paying particular attention to advancements in scalability for large datasets. We also mention R and MATLAB source code for implementations of these algorithms, where they are available.

Bayesian statistics has gained great momentum since the computational developments of the 1990s. Gradually, advances in Bayesian methodology and software have made Bayesian techniques much more accessible to applied statisticians and, in turn, have potentially transformed Bayesian education at the undergraduate level. This article provides an overview on the various options for implementing Bayesian computational methods motivated to achieve particular learning outcomes. The advantages and disadvantages of each computational method are described based on the authors' experience in using these methods in the classroom. The goal is to present guidance on the choice of computation for the instructors who are introducing Bayesian methods in their undergraduate statistics curriculum.

OGTT is a common test, frequently used to diagnose insulin resistance or diabetes, in which a patient's blood sugar is measured at various times over the course of a few hours. Recent developments in the study of OGTT results have framed it as an inverse problem which has been the subject of Bayesian inference. This is a powerful new tool for analyzing the results of an OGTT test,and the question arises as to whether the test itself can be improved. It is of particular interest to discover whether the times at which a patient's glucose is measured can be changed to improve the effectiveness of the test. The purpose of this paper is to explore the possibility of finding a better experimental design, that is, a set of times to perform the test. We review the theory of Bayesian experimental design and propose an estimator for the expected utility of a design. We then study the properties of this estimator and propose a new method for quantifying the uncertainty in comparisons between designs. We implement this method to find a new design and the proposed design is compared favorably to the usual testing scheme.

We present a Bayesian data fusion method to approximate a posterior distribution from an ensemble of particle estimates that only have access to subsets of the data. Our approach relies on approximate probabilistic inference of model parameters through Monte Carlo methods, followed by an update and resample scheme related to multiple importance sampling to combine information from the initial estimates. We show the method is convergent in the particle limit and directly suited to application on multi-sensor data fusion problems by demonstrating efficacy on a multi-sensor Keplerian orbit determination problem and a bearings-only tracking problem.

Inference for doubly intractable distributions is challenging because the intractable normalizing functions of these models include parameters of interest. Previous auxiliary variable MCMC algorithms are infeasible for multi-dimensional models with large data sets because they depend on expensive auxiliary variable simulation at each iteration. We develop a fast Bayesian indirect algorithm by replacing an expensive auxiliary variable simulation from a probability model with a computationally cheap simulation from a surrogate model. We learn the relationship between the surrogate model parameters and the probability model parameters using Gaussian process approximations. We apply our methods to challenging simulated and real data examples, and illustrate that the algorithm addresses both computational and inferential challenges for doubly intractable distributions. Especially for a large social network model with 10 parameters, we show that our method can reduce computing time from about 2 weeks to 5 hours, compared to the previous method. Our method allows practitioners to carry out Bayesian inference for more complex models with larger data sets than before.

Item Response Theory (IRT) is widely applied in the human sciences to model persons' responses on a set of items measuring one or more latent constructs. While several R packages have been developed that implement IRT models, they tend to be restricted to respective prespecified classes of models. Further, most implementations are frequentist while the availability of Bayesian methods remains comparably limited. We demonstrate how to use the R package brms together with the probabilistic programming language Stan to specify and fit a wide range of Bayesian IRT models using flexible and intuitive multilevel formula syntax. Further, item and person parameters can be related in both a linear or non-linear manner. Various distributions for categorical, ordinal, and continuous responses are supported. Users may even define their own custom response distribution for use in the presented framework. Common IRT model classes that can be specified natively in the presented framework include 1PL and 2PL logistic models optionally also containing guessing parameters, graded response and partial credit ordinal models, as well as drift diffusion models of response times coupled with binary decisions. Posterior distributions of item and person parameters can be conveniently extracted and post-processed. Model fit can be evaluated and compared using Bayes factors and efficient cross-validation procedures.

The INLAMSM package for the R programming language provides a collection of multivariate spatial models for lattice data that can be used with package INLA for Bayesian inference. The multivariate spatial models include different structures to model the spatial variation of the variables and the between-variables variability. In this way, fitting multivariate spatial models becomes faster and easier. The use of the different models included in the package is illustrated using two different datasets: the well-known North Carolina SIDS data and mortality by three causes of death in Comunidad Valenciana (Spain).