Chuanhai Liu

Efficient Algorithms for Robust Estimation in Linear Mixed-Effects Models Using the Multivariate t Distribution

Linear mixed-effects models are frequently used to analyze repeated measures data, because they model flexibly the within-subject correlation often present in this type of data. The most popular linear mixed-effects model for a continuous response assumes normal distributions for the random effects and the within-subject errors, making it sensitive to outliers. Such outliers are more problematic for mixed-effects models than for fixed-effects models, because they may occur in the random effects, in the within-subject errors, or in both, making them harder to be detected in practice. Motivated by a real dataset from an orthodontic study, we propose a robust hierarchical linear mixed-effects model in which the random effects and the within-subject errors have multivariate t-distributions, with known or unknown degrees-of-freedom, which are allowed to vary with groups of subjects. By using a gamma-normal hierarchical structure, our model allows the identification and classification of both types of outliers, comparing favorably to other multivariate t models for robust estimation in mixed-effects models previously described in the literature, which use only the marginal distribution of the responses. Allowing for unknown degrees-of-freedom, which are estimated from the data, our model provides a balance between robustness and efficiency, leading to reliable results for valid inference. We describe and compare efficient EM-type algorithms, including ECM, ECME, and PX-EM, for maximum likelihood estimation in the robust multivariate t model. We compare the performance of the Gaussian and the multivariatet models under different patterns of outliers. Simulation results indicate that the multivariate t substantially outperforms the Gaussian model when outliers are present in the data, even in moderate amounts.

Stochastic Approximation in Monte Carlo Computation

The Wang–Landau (WL) algorithm is an adaptive Markov chain Monte Carlo algorithm used to calculate the spectral density for a physical system. A remarkable feature of the WL algorithm is that it is not trapped by local energy minima, which is very important for systems with rugged energy landscapes. This feature has led to many successful applications of the algorithm in statistical physics and biophysics; however, there does not exist rigorous theory to support its convergence, and the estimates produced by the algorithm can reach only a limited statistical accuracy. In this article we propose the stochastic approximation Monte Carlo (SAMC) algorithm, which overcomes the shortcomings of the WL algorithm. We establish a theorem concerning its convergence. The estimates produced by SAMC can be improved continuously as the simulation proceeds. SAMC also extends applications of the WL algorithm to continuum systems. The potential uses of SAMC in statistics are discussed through two classes of applications, importance sampling and model selection. The results show that SAMC can work as a general importance sampling algorithm and a model selection sampler when the model space is complex.

Not Asked and Not Answered: Multiple Imputation for Multiple Surveys

Abstract We present a method of analyzing a series of independent cross-sectional surveys in which some questions are not answered in some surveys and some respondents do not answer some of the questions posed. The method is also applicable to a single survey in which different questions are asked or different sampling methods are used in different strata or clusters. Our method involves multiply imputing the missing items and questions by adding to existing methods of imputation designed for single surveys a hierarchical regression model that allows covariates at the individual and survey levels. Information from survey weights is exploited by including in the analysis the variables on which the weights were based, and then reweighting individual responses (observed and imputed) to estimate population quantities. We also develop diagnostics for checking the fit of the imputation model based on comparing imputed data to nonimputed data. We illustrate with the example that motivated this project: a study o...

Bayesian Robust Multivariate Linear Regression with Incomplete Data

Abstract The multivariate t distribution and other normal/independent multivariate distributions, such as the multivariate slash distribution and the multivariate contaminated distribution, are used for robust regression with complete or incomplete data. Most previous work focused on the method of maximum likelihood estimation for linear regression using normal/independent distributions. This article considers Bayesian estimation of multivariate linear regression models using normal/independent distributions with fully observed predictor variables and possible missing values from outcome variables. A monotone data augmentation algorithm for posterior simulation of the parameters and missing data imputation is presented. The posterior distributions of functions of the parameters can be obtained using Monte Carlo methods. The monotone data augmentation algorithm can also be used for creating multiple imputations for incomplete data sets. An illustrative example of using the multivariate t is also included.

Inferential Models: A Framework for Prior-Free Posterior Probabilistic Inference

Posterior probabilistic statistical inference without priors is an important but so far elusive goal. Fisher’s fiducial inference, Dempster–Shafer theory of belief functions, and Bayesian inference with default priors are attempts to achieve this goal but, to date, none has given a completely satisfactory picture. This article presents a new framework for probabilistic inference, based on inferential models (IMs), which not only provides data-dependent probabilistic measures of uncertainty about the unknown parameter, but also does so with an automatic long-run frequency-calibration property. The key to this new approach is the identification of an unobservable auxiliary variable associated with observable data and unknown parameter, and the prediction of this auxiliary variable with a random set before conditioning on data. Here we present a three-step IM construction, and prove a frequency-calibration property of the IM’s belief function under mild conditions. A corresponding optimality theory is developed, which helps to resolve the nonuniqueness issue. Several examples are presented to illustrate this new approach.

Adaptive Thresholds: Monitoring Streams of Network Counts

This article describes a fast, statistically principled method for monitoring streams of network counts, which have long-term trends, rough cyclical patterns, outliers, and missing data. The key step is to build a reference (predictive) model for the counts that captures their complex, salient features but has just a few parameters that can be kept up-to-date as the counts flow by, without requiring access to past data. This article justifies using a negative binomial reference distribution with parameters that capture trends and patterns and method of moment estimators that can be computed quickly enough to keep up with the data flow. The reference distribution may be of interest in its own right for traffic engineering and load balancing, but a more challenging task is to use it to identify degraded network performance as quickly as possible. Here we detect changes in network performance not by monitoring quantiles of the predictive distribution directly but by applying control chart methodology to normal scores of the p values of the counts. Using p values adjusts for the lack of stationarity from one count to the next. Compared with thresholding isolated counts, control charting reduces the false-alarm rate, increases the chance of detecting ongoing low-level events and reduces the time to detection of long events. This adaptive count thresholding procedure is shown to perform well on both real and simulated data.

Dempster-Shafer Theory and Statistical Inference with Weak Beliefs

The Dempster-Shafer (DS) theory is a powerful tool for probabilistic reasoning based on a formal calculus for combining evi- dence. DS theory has been widely used in computer science and engi- neering applications, but has yet to reach the statistical mainstream, perhaps because the DS belief functions do not satisfy long-run fre- quency properties. Recently, two of the authors proposed an extension of DS, called the weak belief (WB) approach, that can incorporate de- sirable frequency properties into the DS framework by systematically enlarging the focal elements. The present paper reviews and extends this WB approach. We present a general description of WB in the context of inferential models, its interplay with the DS calculus, and the maximal belief solution. New applications of the WB method in two high-dimensional hypothesis testing problems are given. Simula- tions show that the WB procedures, suitably calibrated, perform well compared to popular classical methods. Most importantly, the WB ap- proach combines the probabilistic reasoning of DS with the desirable frequency properties of classical statistics.

Marginal Inferential Models: Prior-Free Probabilistic Inference on Interest Parameters

The inferential models (IM) framework provides prior-free, frequency-calibrated, and posterior probabilistic inference. The key is the use of random sets to predict unobservable auxiliary variables connected to the observable data and unknown parameters. When nuisance parameters are present, a marginalization step can reduce the dimension of the auxiliary variable which, in turn, leads to more efficient inference. For regular problems, exact marginalization can be achieved, and we give conditions for marginal IM validity. We show that our approach provides exact and efficient marginal inference in several challenging problems, including a many-normal-means problem. In nonregular problems, we propose a generalized marginalization technique and prove its validity. Details are given for two benchmark examples, namely, the Behrens–Fisher and gamma mean problems.

Inference about constrained parameters using the elastic belief method

Statistical inference about unknown parameter values that have known constraints is a challenging problem for both frequentist and Bayesian methods. As an alternative, inferential models created with the weak belief method can generate inferential results with desirable frequency properties for constrained parameter problems. To accomplish this, we propose an extension of weak belief called the elastic belief method. Compared to an existing rule for conditioning on constraint information, the elastic belief method produces more efficient probabilistic inference while maintaining desirable frequency properties. The application of this new method is demonstrated in two well-studied examples: inference about a nonnegative quantity measured with Gaussian error and inference about the signal rate of a Poisson count with a known background rate. Compared to several previous interval-forming methods for the constrained Poisson signal rate, the new method gives an interval with better coverage probability or a simpler construction. More importantly, the inferential model provides a post-data predictive measure of uncertainty about the unknown parameter value that is not inherent in other interval-forming methods.

Bayesian Inference for Multivariate Ordinal Data Using Parameter Expansion

Multivariate ordinal data arise in many applications. This article proposes a new, efficient method for Bayesian inference for multivariate probit models using Markov chain Monte Carlo techniques. The key idea is the novel use of parameter expansion to sample correlation matrices. A nice feature of the approach is that inference is performed using straightforward Gibbs sampling. Bayesian methods for model selection are also discussed. Our approach is motivated by a study of how women make decisions on taking medication to reduce the risk of breast cancer. Furthermore, we compare and contrast the performance of our approach with other methods.