Peter H. Westfall

Simultaneous Inference in General Parametric Models

Simultaneous inference is a common problem in many areas of application. If multiple null hypotheses are tested simultaneously, the probability of rejecting erroneously at least one of them increases beyond the pre-specified significance level. Simultaneous inference procedures have to be used which adjust for multiplicity and thus control the overall type I error rate. In this paper we describe simultaneous inference procedures in general parametric models, where the experimental questions are specified through a linear combination of elemental model parameters. The framework described here is quite general and extends the canonical theory of multiple comparison procedures in ANOVA models to linear regression problems, generalized linear models, linear mixed effects models, the Cox model, robust linear models, etc. Several examples using a variety of different statistical models illustrate the breadth of the results. For the analyses we use the R add-on package multcomp, which provides a convenient interface to the general approach adopted here.

Testing association of statistically inferred haplotypes with discrete and continuous traits in samples of unrelated individuals.

There have been increasing efforts to relate drug efficacy and disease predisposition with genetic polymorphisms. We present statistical tests for association of haplotype frequencies with discrete and continuous traits in samples of unrelated individuals. Haplotype frequencies are estimated through the expectation-maximization algorithm, and each individual in the sample is expanded into all possible haplotype configurations with corresponding probabilities, conditional on their genotype. A regression-based approach is then used to relate inferred haplotype probabilities to the response. The relationship of this technique to commonly used approaches developed for case-control data is discussed. We confirm the proper size of the test under H₀ and find an increase in power under the alternative by comparing test results using inferred haplotypes with single-marker tests using simulated data. More importantly, analysis of real data comprised of a dense map of single nucleotide polymorphisms spaced along a 12-cM chromosomal region allows us to confirm the utility of the haplotype approach as well as the validity and usefulness of the proposed statistical technique. The method appears to be successful in relating data from multiple, correlated markers to response.

Optimally weighted, fixed sequence and gatekeeper multiple testing procedures

Abstract We consider a class of closed multiple test procedures indexed by a fixed weight vector. The class includes the Holm weighted step-down procedure, the closed method using the weighted Fisher combination test, and the closed method using the weighted version of Simes’ test. We show how to choose weights to maximize average power, where “average power” is itself weighted by importance assigned to the various hypotheses. Numerical computations suggest that the optimal weights for the multiple test procedures tend to certain asymptotic configurations. These configurations offer numerical justification for intuitive multiple comparisons methods, such as downweighting variables found insignificant in preliminary studies, giving primary variables more emphasis, gatekeeping test strategies, pre-determined multiple testing sequences, and pre-determined sequences of families of tests. We establish that such methods fall within the envelope of weighted closed testing procedures, thus providing a unified view of fixed sequences, fixed sequences of families, and gatekeepers within the closed testing paradigm. We also establish that the limiting cases control the familywise error rate (or FWE), using well-known results about closed tests, along with the dominated convergence theorem.

p Value Adjustments for Multiple Tests in Multivariate Binomial Models

Abstract Data from rodent carcinogenicity (preclinical) and clinical studies involving new drugs may be modeled as having come from multivariate binomial distributions. In two-year rodent carcinogenicity studies, there are typically 20–50 tissues examined for occurrence of any of several possible lesions. For a particular treatment group, the number of occurrences of a particular lesion at a particular tissue may be modeled as binomial, and the vector of such frequencies may be considered multivariate binomial with unspecified dependence structure. The same model may also apply to clinical side-effects data; in this case the marginal frequencies may represent occurrences of events ranging from headaches to ingrown toenails. Frequently, the goal of such studies is to isolate site-specific significant differences between treatment and control groups. For example, in rodent carcinogenicity analyses it is generally not sufficient to claim that a new compound causes an increase in tumors at some unspecified si...

Multiple Testing of General Contrasts Using Logical Constraints and Correlations

Abstract Use of logical constraints among hypotheses and correlations among test statistics can greatly improve the power of step-down tests. An algorithm for uncovering these logically constrained subsets in a given dataset is described. The multiple testing results are summarized using adjusted p values, which incorporate the relevant dependence structures and logical constraints. These adjusted p values are computed consistently and efficiently using a generalized least squares hybrid of simple and control-variate Monte Carlo methods, and the results are compared to alternative stepwise testing procedures.

Multiple Tests with Discrete Distributions

Abstract We review special issues in multiplicity adjustment where the sampling distributions are discrete. These include (1) incorporating discreteness into the multiplicity adjustments, (2) incorporating correlations versus using Bonferroni or independence-based approximations, and (3) using discrete tails in two-sided tests. Incorporating discrete characteristics can greatly improve the power of the tests that maintain a given familywise error rate. Use of correlations also can improve the power, but it is shown that independence-based multiplicity adjustment is not necessarily a conservative procedure. Exact methods that incorporate discreteness and correlations are generally recommended.

Multiple tests for genetic effects in association studies.

1. Introduction Many common human diseases have a genetic component as measured by familial studies. Metabolic disorders such as diabetes, cardiovascular diseases such as high blood pressure, psychiatric disorders such as schizophrenia, and neurodegenerative diseases such as Alzheimers disease all are thought to have a hereditary component. In some diseases the genetic control is through a single gene, while in others, multiple genes interact in complex ways with environmental factors to produce the disease (1 – 5). Data are and will become increasingly available to attempt to link genes to disease phenotype(s). Linkage studies, although powerful for screening relatively large chromosomal regions, lack needed precision because of the constraints imposed by the number of recombination events during generations contained in the pedigree (6). Recently, researchers have attempted to develop techniques that exploit possibilities of fine mapping due to linkage disequilibrium between genetic markers and disease genes. Typing single nucleotide polymorphism markers (SNPs) inside of candidate regions provides a potential means for such analysis (7); however, the problem remains in that the complex diseases are very likely to have multiple etiologies. Consider control of essential hypertension. It has a measured heritability of 3 45%, yet the identification of specific genes remains unclear. Many candidate genes for essential hypertension have been identified and, in a particular individual, a combination of some few of these genes might lead to disease. There is a need for a statistical strategy to analyze these complex experiments, given the multiple testing implied by multiple candidate genes and the risk of false associations. In this chapter we discuss primarily methods for controlling

Using prior information to allocate significance levels for multiple endpoints

We maximize power in a replicated clinical trial involving multiple endpoints by adjusting the individual significance levels for each hypothesis, using preliminary data to obtain the optimal adjustments. The levels are constrained to control the familywise error rate. Power is defined as the expected number of significances, where expectations are taken with respect to the posterior distributions of the non-centrality parameters under non-informative priors. Sample size requirements for the replicate study are given. Intuitive principles such as downweighting insignificant variables from a preliminary study and giving primary endpoints more emphasis are justifiable within the conceptual framework.

Kurtosis as Peakedness, 1905 – 2014. R.I.P.

The incorrect notion that kurtosis somehow measures “peakedness” (flatness, pointiness, or modality) of a distribution is remarkably persistent, despite attempts by statisticians to set the record straight. This article puts the notion to rest once and for all. Kurtosis tells you virtually nothing about the shape of the peak—its only unambiguous interpretation is in terms of tail extremity, that is, either existing outliers (for the sample kurtosis) or propensity to produce outliers (for the kurtosis of a probability distribution). To clarify this point, relevant literature is reviewed, counterexample distributions are given, and it is shown that the proportion of the kurtosis that is determined by the central μ ± σ range is usually quite small.

A Comparison of Variance Component Estimates for Arbitrary Underlying Distributions

Abstract Large-sample covariance matrices for the analysis of variance (ANOVA), minimum norm quadratic unbiased estimator (MINQUE), restricted maximum likelihood (REML), and maximum likelihood (ML) estimates of variance components are presented for the unbalanced one-way model when the underlying distributions are not necessarily normal. The limiting variances depend on the design sequence, on the actual values of the variance components, and on the kurtosis parameters of the underlying distributions. (The skewness parameters and other moments do not affect the limiting distributions.) Because all estimates are consistent and asymptotically normal, it is reasonable to compare the estimates using their asymptotic variances. Thus the efficiency of one estimate relative to another is defined as the ratio of their asymptotic variances, and these efficiencies are evaluated numerically and analytically for a variety of nonnormal situations. Various authors, including Hocking and Kutner (1975), Corbeil and Searl...