Yosef Hochberg

Multiple comparison procedures

PROCEDURES BASED ON CLASSICAL APPROACHES FOR FIXED--EFFECTS LINEAR MODELS WITH NORMAL HOMOSCEDASTIC INDEPENDENT ERRORS. Some Theory of Multiple Comparisons Procedure Fixed--effects Linear Models. Single--step Procedures for Pairwise and More General Comparisons Among All Treatments. Stepwise Procedures for Pairwise and More General Comparisons Among All Treatments. Procedures for Some Other Nonhierarchical Finite Families of Comparisons. Designing Experiments for Multiple Comparisons. PROCEDURES FOR OTHER MODELS AND PROBLEMS, AND PROCEDURES BASED ON ALTERNATIVE APPROACHES. Procedures for One--way Layouts with Unequal Variances. Procedures for Some Mixed--effects Models. Distribution--free and Robust Procedures. Some Miscellaneous Multiple Comparison Problems. Optimal Procedures Using Decision--theoretic, Bayesian, and Other Approaches. Appendixes. Tables. References. Index.

On the Adaptive Control of the False Discovery Rate in Multiple Testing With Independent Statistics

A new approach to problems of multiple significance testing was presented in Benjamini and Hochberg (1995), which calls for controlling the expected ratio of the number of erroneous rejections to the number of rejections–the False Discovery Rate (FDR). The procedure given there was shown to control the FDR for independent test statistics. When some of the hypotheses are in fact false, that procedure is too conservative. We present here an adaptive procedure, where the number of true null hypotheses is estimated first as in Hochberg and Benjamini (1990), and this estimate is used in the procedure of Benjamini and Hochberg (1995). The result is still a simple stepwise procedure, to which we also give a graphical companion. The new procedure is used in several examples drawn from educational and behavioral studies, addressing problems in multi-center studies, subset analysis and meta-analysis. The examples vary in the number of hypotheses tested, and the implication of the new procedure on the conclusions. In a large simulation study of independent test statistics the adaptive procedure is shown to control the FDR and have substantially better power than the previously suggested FDR controlling method, which by itself is more powerful than the traditional family wise error-rate controlling methods. In cases where most of the tested hypotheses are far from being true there is hardly any penalty due to the simultaneous testing of many hypotheses.

Multiple Hypotheses Testing with Weights

In this paper we offer a multiplicity of approaches and procedures for multiple testing problems with weights. Some rationale for incorporating weights in multiple hypotheses testing are discussed. Various type-I error-rates and different possible formulations are considered, for both the intersection hypothesis testing and the multiple hypotheses testing problems. An optimal per family weighted error-rate controlling procedure a la Spjotvoll (1972) is obtained. This model serves as a vehicle for demonstrating the different implications of the approaches to weighting. Alternative approach es to that of Holm (1979) for family-wise error-rate control with weights are discussed, one involving an alternative procedure for family-wise error-rate control, and the other involving the control of a weighted family-wise error-rate. Extensions and modifications of the procedures based on Simes (1986) are given. These include a test of the overall intersec tion hypothesis with general weights, and weighted sequentially rejective procedures for testing the individual hypotheses. The false discovery rate controlling approach and procedure of Benjamini & Hochberg (1995) are extended to allow for different weights.

Bayesian perspectives on multiple comparisons

Abstract We discuss Bayesian attitudes towards adjusting inferences for multiplicities. In the simplest Bayesian view, there is no need for adjustments and the Bayesian perspective is similar to that of the frequentist who makes inferences on a per-comparison basis. However, as we explain, Bayesian thinking can lead to making adjustments that are in the same spirit as those made by frequentists who subscribe to preserving the familywise error rate. We describe the differences between assuming independent prior distributions and hierarchical prior distributions. As an example of the latter, we illustrate the use of a Dirichlet process prior distribution in the context of multiplicities. We also discuss some quasi-Bayesian procedures which combine Bayesian and frequentist ideas. This shows the potential of Bayesian methodology to yield procedures that can be evaluated using “objective” criteria. Finally, we comment on the role of subjectivity in Bayesian approaches to the complex realm of multiple comparisons problems, and on robust vs. informative priors.

On the Use of Double Sampling Schemes in Analyzing Categorical Data with Misclassification Errors

Abstract In order to resolve the difficulties involved in inference from a sample of categorical data obtained by using a fallible classifying mechanism (usually inexpensive), we consider, as in Tenenbein (1970, 1971, 1972), the utilization of an additional sample. The second sample is subjected to a simultaneous cross-classification of its elements by the fallible mechanism and by some true (usually expensive) classifying mechanism. The setup is general; i.e., the discussion can be applied to any multidimensional cross-classified data obtained by unrestricted random sampling. Two methodologies are presented: (i) a combined maximum likelihood (ML) and least squares (LS) approach and (ii) a complete-LS approach. Both methodologies are illustrated using real data.

Alternative Estimation Procedures for Pr(X < Y) in Categorized Data

SUMMARY Consider two independent random variables X and Y. The functional R = Pr(X < Y) [or X = Pr(X < Y) - Pr(Y < X)] is of practical importance in many situations, including clinical trials, genetics, and reliability. In this paper several approaches to estimation of X when X and Y are presented in discretized (categorical) form are analyzed and compared. Asymptotic formulas for the variances of the estimators are derived; use of the bootstrap to estimate variances is also discussed. Computer simulations indicate that the choice of the best estimator depends on the value of X, the underlying distribution, and the sparseness of the data. It is shown that the bootstrap provides a robust estimate of variance. Several examples are treated.

Closed procedures are better and often admit a shortcut

Abstract It is ‘common knowledge’ that closed multiple test procedures are typically better than others in terms of power. In the first part of this paper we prove that closed procedures constitute a complete class in the class of all coherent procedures, and, also, that α-exhaustive procedures constitute a minimal complete class there. However, closed procedures are not always ‘monotone in p-values’ and are often computationally inefficient. Thus, in the second part of this paper we first investigate conditions for a closed procedure to be ‘monotone in p-values’, and second, we shed further light on the possibility of reducing a given closed procedure to a step-down sequentially rejective procedure. We give a sufficient condition for a closure based on some generalized Simes (1986) , (Biometrika 73, 751–754) type tests to be ‘monotone in p-values’ and clarify Hochberg and Tamhane (1987) , (Multiple Comparison Procedures, Wiley, New York) general result on the existence of a shortcut in families satisfying the free combination condition. We comment on the interrelationship between ‘stepwise’ (step-down and step-up), ‘monotone in p-values’, and ‘sequentially rejective’ procedures, in families satisfying the ‘free combinations condition’ as well as in families involving ‘logically related’ hypotheses.

A Modification of the T -Method of Multiple Comparisons for a One-Way Layout with Unequal Variances

Abstract Tukeys T-method for simultaneous inference in a one-way layout is extended to situations where heterogeneity of variance prevails. The modified method is based on the distribution of the range of iid Student-t variates. The new procedure is exemplified and a table of critical values is provided for its implementation. A different procedure based on a Bonferroni upper bound is also discussed.

On the variance estimate of a wilcoxon-mann whitney statistic for group ordered data

Two consistent estimators for the non-null variance of Wil-coxon-Mann-Whitney’s statistic applied to grouped ordered data, are considered. The first is based on U-statistics and the sec-ond is obtained by the Delta method. Some examples are given to demonstrate the extent of error when using a null variance esti-mate for constructing confidence intervals. It appears that the two consistent estimates are very close, but may both be disting-uishably larger or smaller than the null variance estimate.

Photoelectric properties of the “yellow strips” in Vespa orientalis; a mathematical model

The “yellow strips” on the cuticle of the Oriental Hornet (Vespa orientalis, Hymenoptera, Vespinae), present photoelectric properties. A mathematical model for the relative changes in resistance as a photoconductive process conforms to the general model for a semiconductor with traps.