Frank Bretz

Simultaneous Inference in General Parametric Models

Simultaneous inference is a common problem in many areas of application. If multiple null hypotheses are tested simultaneously, the probability of rejecting erroneously at least one of them increases beyond the pre-specified significance level. Simultaneous inference procedures have to be used which adjust for multiplicity and thus control the overall type I error rate. In this paper we describe simultaneous inference procedures in general parametric models, where the experimental questions are specified through a linear combination of elemental model parameters. The framework described here is quite general and extends the canonical theory of multiple comparison procedures in ANOVA models to linear regression problems, generalized linear models, linear mixed effects models, the Cox model, robust linear models, etc. Several examples using a variety of different statistical models illustrate the breadth of the results. For the analyses we use the R add-on package multcomp, which provides a convenient interface to the general approach adopted here.

A graphical approach to sequentially rejective multiple test procedures

For clinical trials with multiple treatment arms or endpoints a variety of sequentially rejective, weighted Bonferroni-type tests have been proposed, such as gatekeeping procedures, fixed sequence tests, and fallback procedures. They allow to map the difference in importance as well as the relationship between the various research questions onto an adequate multiple test procedure. Since these procedures rely on the closed test principle, they usually require the explicit specification of a large number of intersection hypotheses tests. The underlying test strategy may therefore be difficult to communicate. We propose a simple iterative graphical approach to construct and perform such Bonferroni-type tests. The resulting multiple test procedures are represented by directed, weighted graphs, where each node corresponds to an elementary hypothesis, together with a simple algorithm to generate such graphs while sequentially testing the individual hypotheses. The approach is illustrated with the visualization of several common gatekeeping strategies. A case study is used to illustrate how the methods from this article can be used to tailor a multiple test procedure to given study objectives.

Innovative Approaches for Designing and Analyzing Adaptive Dose-Ranging Trials

Inadequate selection of the dose to bring forward in confirmatory trials has been identified as one of the key drivers of the decreasing success rates observed in drug development programs across the pharmaceutical industry. In recognition of this problem, the Pharmaceutical Research and Manufacturers of America (PhRMA), formed a working group to evaluate and develop alternative approaches to dose finding, including adaptive dose-ranging designs. This paper summarizes the work of the group, including the results and conclusions of a comprehensive simulation study, and puts forward recommendations on how to improve dose ranging in clinical development, including, but not limited to, the use of adaptive dose-ranging methods.

Adaptive designs for confirmatory clinical trials

Adaptive designs play an increasingly important role in clinical drug development. Such designs use accumulating data of an ongoing trial to decide how to modify design aspects without undermining the validity and integrity of the trial. Adaptive designs thus allow for a number of possible adaptations at midterm: Early stopping either for futility or success, sample size reassessment, change of population, etc. A particularly appealing application is the use of adaptive designs in combined phase II/III studies with treatment selection at interim. The expectation has arisen that carefully planned and conducted studies based on adaptive designs increase the efficiency of the drug development process by making better use of the observed data, thus leading to a higher information value per patient.In this paper we focus on adaptive designs for confirmatory clinical trials. We review the adaptive design methodology for a single null hypothesis and how to perform adaptive designs with multiple hypotheses using closed test procedures. We report the results of an extensive simulation study to evaluate the operational characteristics of the various methods. A case study and related numerical examples are used to illustrate the key results. In addition we provide a detailed discussion of current methods to calculate point estimates and confidence intervals for relevant parameters.

Confirmatory adaptive designs with Bayesian decision tools for a targeted therapy in oncology

The ability to select a sensitive patient population may be crucial for the development of a targeted therapy. Identifying such a population with an acceptable level of confidence may lead to an inflation in development time and cost. We present an approach that allows to decrease these costs and to increase the reliability of the population selection. It is based on an actual adaptive phase II/III design and uses Bayesian decision tools to select the population of interest at an interim analysis. The primary endpoint is assumed to be the time to some event like e.g. progression. It is shown that the use of appropriately stratified logrank tests in the adaptive test procedure guarantees overall type I error control also when using information on patients that are censored at the adaptive interim analysis. The use of Bayesian decision tools for the population selection decision making is discussed. Simulations are presented to illustrate the operating characteristics of the study design relative to a more traditional development approach. Estimation of treatment effects is considered as well.

Numerical computation of multivariate t-probabilities with application to power calculation of multiple contrasts

A new method to calculate the multivariate t-distribution is introduced. We provide a series of substitutions, which transform the starting q-variate integral into one over the (q—1)-dimensional hypercube. In this situation standard numerical integration methods can be applied. Three algorithms are discussed in detail. As an application we derive an expression to calculate the power of multiple contrast tests assuming normally distributed data.

Compatible simultaneous lower confidence bounds for the Holm procedure and other Bonferroni-based closed tests.

We consider the problem of simultaneously testing multiple one-sided null hypotheses. Single-step procedures, such as the Bonferroni test, are characterized by the fact that the rejection or non-rejection of a null hypothesis does not take the decision for any other hypothesis into account. For stepwise test procedures, such as the Holm procedure, the rejection or non-rejection of a null hypothesis may depend on the decision of other hypotheses. It is well known that stepwise test procedures are by construction more powerful than their single-step counterparts. This power advantage, however, comes only at the cost of increased difficulties in constructing compatible simultaneous confidence intervals for the parameters of interest. For example, such simultaneous confidence intervals are easily obtained for the Bonferroni method, but surprisingly hard to derive for the Holm procedure. In this paper, we discuss the inherent problems and show that ad hoc solutions used in practice typically do not control the pre-specified simultaneous confidence level. Instead, we derive simultaneous confidence intervals that are compatible with a certain class of closed test procedures using weighted Bonferroni tests for each intersection hypothesis. The class of multiple test procedures covered in this paper includes gatekeeping procedures based on Bonferroni adjustments, fixed sequence procedures, the simple weighted or unweighted Bonferroni procedure by Holm and the fallback procedure. We illustrate the results with a numerical example.

Optimal Designs for Dose-Finding Studies

Understanding and properly characterizing the dose–response relationship is a fundamental step in the investigation of a new compound, be it a herbicide or fertilizer, a molecular entity, an environmental toxin, or an industrial chemical. In this article we investigate the problem of deriving efficient designs for the estimation of target doses in the context of clinical dose finding. We propose methods to determine the appropriate number and actual levels of the doses to be administered to patients, as well as their relative sample size allocations. More specifically, we derive local optimal designs that minimize the asymptotic variance of the minimum effective dose estimate under a particular dose–response model. We investigate the small-sample properties of these designs, together with their sensitivity to a misspecification of the true parameter values and of the underlying dose–response model, through simulation. Finally, we demonstrate that the designs derived for a fixed model are rather sensitive with respect to this assumption and construct robust optimal designs that take into account a set of potential dose–response profiles within classes of models commonly used in drug development practice.

On the Numerical Availability of Multiple Comparison Procedures

In the past many multiple comparison procedure were difficult to perform. Usually, such procedures can be traced back to studentized multiple contrast tests. Numerical difficulties restricted the use of the exact procedures to simple, commonly balanced, designs. Conservative approximations or simulation based approaches have been used in the general cases. However, new efforts and results in the past few years have led to fast and efficient computations of the underlying multidimensional integrals. Inferences for any finite set of linear functions of normal means are now numerically feasible. These include all-pairwise comparisons, comparisons with a control (including dose-response contrasts), multiple comparison with the best, etc. The article applies the numerical progress on multiple comparisons procedures for common balanced and unbalanced designs within the general linear model.

Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests.

The confirmatory analysis of pre-specified multiple hypotheses has become common in pivotal clinical trials. In the recent past multiple test procedures have been developed that reflect the relative importance of different study objectives, such as fixed sequence, fallback, and gatekeeping procedures. In addition, graphical approaches have been proposed that facilitate the visualization and communication of Bonferroni-based closed test procedures for common multiple test problems, such as comparing several treatments with a control, assessing the benefit of a new drug for more than one endpoint, combined non-inferiority and superiority testing, or testing a treatment at different dose levels in an overall and a subpopulation. In this paper, we focus on extended graphical approaches by dissociating the underlying weighting strategy from the employed test procedure. This allows one to first derive suitable weighting strategies that reflect the given study objectives and subsequently apply appropriate test procedures, such as weighted Bonferroni tests, weighted parametric tests accounting for the correlation between the test statistics, or weighted Simes tests. We illustrate the extended graphical approaches with several examples. In addition, we describe briefly the gMCP package in R, which implements some of the methods described in this paper.