Douglas M. Bates

Bioconductor: open software development for computational biology and bioinformatics

The Bioconductor project is an initiative for the collaborative creation of extensible software for computational biology and bioinformatics. The goals of the project include: fostering collaborative development and widespread use of innovative software, reducing barriers to entry into interdisciplinary scientific research, and promoting the achievement of remote reproducibility of research results. We describe details of our aims and methods, identify current challenges, compare Bioconductor to other open bioinformatics projects, and provide working examples.

Nonlinear Mixed Effects Models for Repeated Measures Data

We propose a general, nonlinear mixed effects model for repeated measures data and define estimators for its parameters. The proposed estimators are a natural combination of least squares estimators for nonlinear fixed effects models and maximum likelihood (or restricted maximum likelihood) estimators for linear mixed effects models. We implement Newton-Raphson estimation using previously developed computational methods for nonlinear fixed effects models and for linear mixed effects models. Two examples are presented and the connections between this work and recent work on generalized linear mixed effects models are discussed.

Approximations to the Log-Likelihood Function in the Nonlinear Mixed-Effects Model

Abstract Nonlinear mixed-effects models have received a great deal of attention in the statistical literature in recent years because of the flexibility they offer in handling the unbalanced repeated-measures data that arise in different areas of investigation, such as pharmacokinetics and economics. Several different methods for estimating the parameters in nonlinear mixed-effects model have been proposed. We concentrate here on two of them—maximum likelihood and restricted maximum likelihood. A rather complex numerical issue for (restricted) maximum likelihood estimation in nonlinear mixed-effects models is the evaluation of the log-likelihood function of the data, because it involves the evaluation of a multiple integral that, in most cases, does not have a closed-form expression. We consider here four different approximations to the log-likelihood, comparing their computational and statistical properties. We conclude that the linear mixed-effects (LME) approximation suggested by Lindstrom and Bates, t...

Newton-Raphson and EM Algorithms for Linear Mixed-Effects Models for Repeated-Measures Data

Abstract We develop an efficient and effective implementation of the Newton—Raphson (NR) algorithm for estimating the parameters in mixed-effects models for repeated-measures data. We formulate the derivatives for both maximum likelihood and restricted maximum likelihood estimation and propose improvements to the algorithm discussed by Jennrich and Schluchter (1986) to speed convergence and ensure a positive-definite covariance matrix for the random effects at each iteration. We use matrix decompositions to develop efficient and computationally stable implementations of both the NR algorithm and an EM algorithm (Laird and Ware 1982) for this model. We compare the two methods (EM vs. NR) in terms of computational order and performance on two sample data sets and conclude that in most situations a well-implemented NR algorithm is preferable to the EM algorithm or EM algorithm with Aitkens acceleration. The term repeated measures refers to experimental designs where there are several individuals and several...

Linear and Nonlinear Mixed Effects Models

Douglas M. Bates Department of Statistics University of Wisconsin Madison Jose C. Pinheiro Bell Laboratories Lucent Technologies 1 Recent developments in computational methods for maximum likelihood (ML) or restricted maximum likelihood (REML) estimation of parameters in general linear mixed-effects models have made the analysis of data in typical agricultural settings much easier. With software such as SAS PROC MIXED we are able to handle da~ from random-effects one-way classifications, from blocked designs including incomplete blocked designs, from hierarchical designs such as splitplot designs, and other types of data that may be described as repeated measures or longitudinal data or growth-curve data. It is especially helpful that the new computational methods do not depend on balance in the data so we are able to deal more easily with observational studies or with randomly missing data in a designed experiment. We describe some of the new computational approaches and how they are implemented in the nlme3.0 library for the S-PLUS language. One of the most powerful features of this language is the graphics capabilities, especially the trellis graphics facilities developed by Bill Cleveland and his coworkers at Bell Labs. Although most participants in this conference may be more familiar with SAS, and most of the models described here can be fit with PROC MIXED or the NLiNMIX macro or new P ROC N LM IXED in SAS version 7, some exposure to the combination of graphical display and model-fitting approaches from S-PLUS may be informative. 1 Annual Conference on Applied Statistics in Agriculture Kansas State University New Prairie Press http://newprairiepress.org/agstatconference/1998/proceedings/2 2 Kansas State University We show how data exploration with trellis graphics, followed by fitting and comparing mixedeffects models, followed by graphical assessment of the fitted model can be used in a variety of situations. On some occasions, such as modeling growth curves, a linear trend or polynomial trend or other types of linear statistical models for the within-subject time dependence are just not going to do an adequate job of representing the data. In those cases, a nonlinear model is more appropriate. We show how the concept of a random coefficient model can be extended to nonlinear models so as to fit nonlinear mixed-effects models.

Unconstrained parametrizations for variance-covariance matrices

The estimation of variance-covariance matrices through optimization of an objective function, such as a log-likelihood function, is usually a difficult numerical problem. Since the estimates should be positive semi-definite matrices, we must use constrained optimization, or employ a parametrization that enforces this condition. We describe here five different parametrizations for variance-covariance matrices that ensure positive definiteness, thus leaving the estimation problem unconstrained. We compare the parametrizations based on their computational efficiency and statistical interpretability. The results described here are particularly useful in maximum likelihood and restricted maximum likelihood estimation in linear and non-linear mixed-effects models, but are also applicable to other areas of statistics.

Gcvpack – routines for generalized cross validation

These Fortran-77 subroutines provide building blocks for Generalized Cross-Validation (GCV) (Craven and Wahba, 1979) calculations in data analysis and data smoothing including ridge regression (Golub, Heath, and Wahba, 1979), thin plate smoothing splines (Wahba and Wendelberger, 1980), deconvolution (Wahba, 1982d), smoothing of generalized linear models (Osullivan, Yandell and Raynor 1986, Green 1984 and Green and Yandell 1985), and ill-posed problems (Nychka et al., 1984, Osullivan and Wahba, 1985). We present some of the types of problems for which GCV is a useful method of choosing a smoothing or regularization parameter and we describe the structure of the subroutines.Ridge Regression: A familiar example of a smoothing parameter is the ridge parameter X in the ridge regression problem which we write.

Balancing Type I error and power in linear mixed models

Linear mixed-effects models have increasingly replaced mixed-model analyses of variance for statistical inference in factorial psycholinguistic experiments. Although LMMs have many advantages over ANOVA, like ANOVAs, setting them up for data analysis also requires some care. One simple option, when numerically possible, is to fit the full variance-covariance structure of random effects (the maximal model; Barr et al. 2013), presumably to keep Type I error down to the nominal alpha in the presence of random effects. Although it is true that fitting a model with only random intercepts may lead to higher Type I error, fitting a maximal model also has a cost: it can lead to a significant loss of power. We demonstrate this with simulations and suggest that for typical psychological and psycholinguistic data, higher power is achieved without inflating Type I error rate if a model selection criterion is used to select a random effect structure that is supported by the data.

The computation of generalized cross-validation functions through householder tridiagonalization with applications to the fitting of interaction spline models

An efficient algorithm for computing the GCV (generalized cross-validation) function for the general cross-validated regularization/smoothing problem is provided. This algorithm is based on the Householder tridiagonalization, similar to Elden’s [BIT, 24 (1984), pp. 467–472] bidiagonalization and is appropriate for problems where no natural structure is available, and the regularization /smoothing problem is solved (exactly) in a reproducing kernel Hilbert space. It is particularly appropriate for certain multivariate smoothing problems with irregularly spaced data, and certain remote sensing problems, such as those that occur in meteorology, where the sensors are arranged irregularly.The algorithm is applied to the fitting of interaction spline models with irregularly spaced data and two smoothing parameters, and favorable timing results are presented. The algorithm may be extended to the computation of certain GML (generalized maximum likelihood) functions. Application of the GML algorithm

A Relative Off set Orthogonality Convergence Criterion for Nonlinear least Squares

An orthogonality convergence criterion using relative offset is proposed. This criterion is compared to currently used criteria and its advantages are discussed.