Charles W. Dunnett

A Multiple Comparison Procedure for Comparing Several Treatments with a Control

(1955). A Multiple Comparison Procedure for Comparing Several Treatments with a Control. Journal of the American Statistical Association: Vol. 50, No. 272, pp. 1096-1121.

NEW TABLES FOR MULTIPLE COMPARISONS WITH A CONTROL

Some time ago, a multiple comparison procedure for comparing several treatments simultaneously with a control or standard treatment was introduced by the present author (Dunnett [1955]). The procedure was designed to be used either to test the significance of the differences between each of the treatments and the control with a stated value 1 - P for the joint significance level, or to set confidence limits on the true values of the treatment differences from the control with a stated value P for the joint confidence coefficient. Thus the procedure has the property of controlling the experimentwise, rather than the per-comparison, error rate associated with the comparisons, in common with the multiple comparison procedures of Tukey [unpublished] and Scheffe [1953]. In the earlier paper, tables were provided enabling up to nine treatments to be compared with a control with joint confidence coefficient either .95 or .99. Tables for both one-sided and two-sided comparisons were given but, as explained in the paper, the two-sided values were inexact for the case of more than two comparisons as a result of an approximation which had to be made in the computations. The main purpose of the present paper is to give the exact tables for making two-sided comparisons. The necessary computations were done on a General Precision LGP-30 electronic computer, by a method described in section 3 below. The tables are given here as Tables II and III; these replace Tables 2a and 2b, respectively, of the previous paper. In addition to providing the exact values, a method is given for adjusting the tabulated values to cover the situation where the variance of the control mean is smaller than the variance of the treatment means, as occurs for example when a greater number of observations is allocated to the control than to any of the test treatments. Furthermore, the number of treatments which may be simultaneously compared with a control has been extended to twenty. 482

Pairwise Multiple Comparisons in the Unequal Variance Case

Abstract Four pairwise multiple comparison procedures for the case in which the variances in the groups are unequal were compared by computer simulation: the GH procedure based on the Studentized range and the Welch formula for approximate degrees of freedom (df), the C procedure based on a weighted average of two Studentized range points, the T2 procedure based on Students t, and the T3 procedure based on the Studentized maximum modulus. The results indicate that C, T2, and T3 have the desirable property of being conservative, whereas GH sometimes is not. Of the three conservative procedures, T3 always has shorter confidence interval length than T2, whereas C has shorter length than T3 for large df, but longer length for small df.

Pairwise Multiple Comparisons in the Homogeneous Variance, Unequal Sample Size Case

Abstract The effect of unequal ni on the error rates of six procedures for pair-wise multiple comparisons beteen k treatment means with homogeneous variances was studied by computer simulation. A commonly used method, attributed to Kramer (1956) but suggested also by Tukey (1953), was found to have error rates less than the nominal value α for several patterns of inequality in the sample sizes, at least when the variations in ni were moderately large. A method that substitutes the harmonic mean sample size for n in Tukeys T method had excessively high error rates. Other methods proposed more recently in the literature were conservative relative to the Tukey-Kramer method. Thus, the Tukey-Kramer method is recommended for use in the unequal sample size, homogeneous variance situation.

A Step-Up Multiple Test Procedure

Abstract We consider the problem of simultaneously testing k ≥ 2 hypotheses on parameters θ1, …, θk . In a typical application the θs may be a set of contrasts, for instance, a set of orthogonal contrasts among population means or a set of differences between k treatment means and a standard treatment mean. It is assumed that least squares estimators 1, …, k are available that are jointly normally distributed with a common variance (known up to a scalar, namely the error variance σ 2) and a common known correlation. An independent χ 2-distributed unbiased estimator of σ 2 is also assumed to be available. We propose a step-up multiple test procedure for this problem which tests the t statistics for the k hypotheses in order starting with the least significant one and continues as long as an acceptance occurs. (By contrast, the step-down approach, which is usually used, starts with the most significant and continues as long as a rejection occurs.) Critical constants required by this step-up procedure to co...

AN ALTERNATIVE TO THE USE OF TWO-SIDED TESTS IN CLINICAL TRIALS

There is a controversy in the literature concerning the use of one- and two-sided tests in clinical trials. Some contend that, when the research question relates to improved efficacy or safety, that is, the expected change is in one direction only, the hypothesis test should reflect this by being one-sided. Others insist on the use of a two-sided test in case a treatment difference in the opposite direction to that expected might turn up. We propose an alternative procedure to the two-sided test which also provides protection against overlooking a negative effect. The proposed procedure tests simultaneously for a positive difference and for equivalence. We illustrate the procedure by applying it to the results of a recent clinical trial.

Measurement of carcinoembryonic antigen in patients with bronchogenic carcinoma

Estimation of CEA levels by the Z‐gel method indicates that smokers, patients with limited lung cancer and patients with extensive lung cancer have higher values than nonsmoking controls. The CEA levels within each group are significantly different from one another. Use of CEA estimation for diagnostic purposes is limited because of the considerable overlap between normal controls and patients with cancer, the relatively low incidence of elevated values in patients with limited disease and the high incidence of false negatives (20%) even in patients with extensive disease. Elevated CEA values are associated with a poor prognosis and could be of clinical value as an addition to clinical staging to determine survival particularly for patients with extrathoracic disease. Persistently high values in patients deemed clinically disease‐free postoperatively are indicative of residual disease and a poor prognosis. If and when effective therapy for bronchogenic carcinoma becomes available, monitoring of CEA values may be useful in some patients as an early indication of relapse. Further studies are required to determine if the extraordinarily poor prognosis associated with marked elevations of CEA may be used as an additional criterion of inoperability in such patients.

STEP-UP MULTIPLE TESTING OF PARAMETERS WITH UNEQUALLY CORRELATED ESTIMATES

We consider the problem of simultaneously testing k > or = to 2 hypotheses on parameters theta(1), ..., theta(k) using test statistics t(1), ..., t(k) such that a specified familywise error rate alpha is achieved. Dunnett and Tamhane (1992a) proposed a step-up multiple test procedure, in which testing starts with the hypothesis corresponding to the least significant test statistic and proceeds towards the most significant, stopping the first time a significant test result is obtained (and rejecting the hypotheses corresponding to that and any remaining test statistics). The parameter estimates used in the t statistics were assumed to be normally distributed with a common variance, which was a known multiple of an unknown sigma(2), and known correlations which were equal. In the present article, we show how the procedure can be extended to include unequally correlated parameter estimates. Unequal correlations occur, for example, in experiments involving comparisons among treatment groups with unequal sample sizes. We also compare the step-up and step-down multiple testing approaches and discuss applications to some biopharmaceutical testing problems.

A generalized step-up-down multiple test procedure

A generalization of step-up and step-down multiple test procedures is proposed. This step-updown procedure is useful when the objective is to reject a specified minimum number, q, out of a family of k hypotheses. If this basic objective is met at the first step, then it proceeds in a step-down manner to see if more than q hypotheses can be rejected. Otherwise it proceeds in a step-up manner to see if some number less than q hypotheses can be rejected. The usual step-down procedure is the special case where q = 1, and the usual step-up procedure is the special case where q = k. Analytical and numerical comparisons between the powers of the step-up-down procedures with different choices of q are made to see how these powers depend on the actual number of false hypotheses. Examples of application include comparing the efficacy of a treatment to a control for multiple endpoints and testing the sensitivity of a clinical trial for comparing the efficacy of a new

Multiple Comparisons for Orthogonal Contrasts: Examples and Tables

In many experimental situations the pertinent inferences are made on the basis of orthogonal contrasts among the treatment means (as in 2 n factorial experiments). In this setting a particularly useful form of inference is one involving multiple comparisons. The present article describes situations in which such inferences are meaningful, gives examples of their use, and provides a table of constants needed to implement such multiple comparison procedures. The procedures can also be used for statistically legitimate “data snooping” (in the sense ofScheffe 1959, p. 80) to help decide which contrasts within a specified set warrant further study.