Willi Maurer

A graphical approach to sequentially rejective multiple test procedures

For clinical trials with multiple treatment arms or endpoints a variety of sequentially rejective, weighted Bonferroni-type tests have been proposed, such as gatekeeping procedures, fixed sequence tests, and fallback procedures. They allow to map the difference in importance as well as the relationship between the various research questions onto an adequate multiple test procedure. Since these procedures rely on the closed test principle, they usually require the explicit specification of a large number of intersection hypotheses tests. The underlying test strategy may therefore be difficult to communicate. We propose a simple iterative graphical approach to construct and perform such Bonferroni-type tests. The resulting multiple test procedures are represented by directed, weighted graphs, where each node corresponds to an elementary hypothesis, together with a simple algorithm to generate such graphs while sequentially testing the individual hypotheses. The approach is illustrated with the visualization of several common gatekeeping strategies. A case study is used to illustrate how the methods from this article can be used to tailor a multiple test procedure to given study objectives.

Testing strategies in multi-dose experiments including active control

Inferential test strategies for multi-arm trials are adapted or proposed for the special situation when more than one dose of a test treatment, placebo and active control(s) are compared. This includes between doses, dose-placebo and dose-active-control comparisons. The procedures refer to situations when detailed comparisons make sense only if the sensitivity of the trial has been shown, for example, if a dose-response relationship or a difference between active control and placebo has been established. Split strategies, hierarchical (assuming an order restriction among doses) or linked procedures are introduced. In linked procedures, equivalence to the active control will be established only if the dose is also shown to be effective as compared to placebo. All the inferential procedures control the experimentwise error rate in the strong sense for the respective sets of null hypotheses considered.

Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests.

The confirmatory analysis of pre-specified multiple hypotheses has become common in pivotal clinical trials. In the recent past multiple test procedures have been developed that reflect the relative importance of different study objectives, such as fixed sequence, fallback, and gatekeeping procedures. In addition, graphical approaches have been proposed that facilitate the visualization and communication of Bonferroni-based closed test procedures for common multiple test problems, such as comparing several treatments with a control, assessing the benefit of a new drug for more than one endpoint, combined non-inferiority and superiority testing, or testing a treatment at different dose levels in an overall and a subpopulation. In this paper, we focus on extended graphical approaches by dissociating the underlying weighting strategy from the employed test procedure. This allows one to first derive suitable weighting strategies that reflect the given study objectives and subsequently apply appropriate test procedures, such as weighted Bonferroni tests, weighted parametric tests accounting for the correlation between the test statistics, or weighted Simes tests. We illustrate the extended graphical approaches with several examples. In addition, we describe briefly the gMCP package in R, which implements some of the methods described in this paper.

Test and power considerations for multiple endpoint analyses using sequentially rejective graphical procedures

A variety of powerful test procedures are available for the analysis of clinical trials addressing multiple objectives, such as comparing several treatments with a control, assessing the benefit of a new drug for more than one endpoint, etc. However, some of these procedures have reached a level of complexity that makes it difficult to communicate the underlying test strategies to clinical teams. Graphical approaches have been proposed instead that facilitate the derivation and communication of Bonferroni-based closed test procedures. In this paper we give a coherent description of the methodology and illustrate it with a real clinical trial example. We further discuss suitable power measures for clinical trials with multiple primary and/or secondary objectives and use a generic example to illustrate our considerations.

Multiple Testing in Group Sequential Trials Using Graphical Approaches

In 1997, Robert T. O’Neill introduced the framework of structuring the experimental questions to best reflect the clinical study’s objectives. This task comprises the identification of the study’s primary, secondary, and exploratory objectives and the requirements as to when the corresponding statistical tests are considered meaningful. A topic that has been considered much less in the literature until very recently is the application of group sequential trial designs to multiple endpoints. In 2007, Hung, Wang, and O’Neill showed that borrowing testing strategies naively from trial designs without interim analyses may not maintain the familywise Type I error rate at level α. The authors gave examples in the context of testing two hierarchically ordered endpoints in two-armed group sequential trials with one interim analysis. We consider the general situation of testing multiple hypotheses repeatedly in time using recently developed graphical approaches. We focus on closed testing procedures using weighted group sequential Bonferroni tests for the intersection hypotheses. Under mild monotonicity conditions on the error spending functions, this allows the use of sequentially rejective graphical procedures in group sequential trials. The methodology is illustrated with a numerical example for a three-armed trial comparing two doses against control for two hierarchically ordered endpoints.

Adaptive Designs in Clinical Drug Development: Opportunities, Challenges, and Scope Reflections Following PhRMA's November 2006 Workshop

This paper provides reflections on the opportunities, scope and challenges of adaptive design as discussed at PhRMAs workshop held in November 2006. We also provide a status report of workstreams within PhRMAs working group on adaptive designs, which were triggered by the November workshop. Rather than providing a comprehensive review of the presentations given, we limit ourselves to a selection of key statements. The authors reflect the position of PhRMAs working group on adaptive designs.

Aspects of Modernizing Drug Development Using Clinical Scenario Planning and Evaluation

Modern drug development requires an efficient clinical development program to have a reasonable chance of successfully leading to the submission of the therapy, given that the therapy is effective, or to early stopping if this is not the case. Clinical trials and programs should be designed to effectively support this final goal. Currently, the statistical planning in drug development is based on parts of a clinical program in isolation, conditioned on one fixed setting, focusing on sample size calculation or simple design questions. There is, however, an increasing demand for a clinical program optimization and acceleration as well as an unconditional evaluation of relative program efficiency, robustness, and validity. The complexity of the development process, however, often does not allow for simple solutions, frequently requiring computer simulations to support these assessments. We propose a general framework for comparing competing options for clinical programs, trial designs, and analysis methods as a basis for decision making and to facilitate the internal and external dialogue with key stakeholders. The final decision making ultimately needs to factor in quantitative aspects as well as additional qualitative dimensions such as logistic feasibility, regulatory acceptance, and so on. A terminology is introduce that clearly describes the different aspects of such a framework, the range of underlying assumptions, the competing options, and the metrics that are used to assess and compare these options. Three specific case studies are presented that illustrate these concepts at three different levels: program planning, trial design, and analysis methods.

Type I error rate control in adaptive designs for confirmatory clinical trials with treatment selection at interim.

Interest in confirmatory adaptive combined phase II/III studies with treatment selection has increased in the past few years. These studies start comparing several treatments with a control. One (or more) treatment(s) is then selected after the first stage based on the available information at an interim analysis, including interim data from the ongoing trial, external information and expert knowledge. Recruitment continues, but now only for the selected treatment(s) and the control, possibly in combination with a sample size reassessment. The final analysis of the selected treatment(s) includes the patients from both stages and is performed such that the overall Type I error rate is strictly controlled, thus providing confirmatory evidence of efficacy at the final analysis. In this paper we describe two approaches to control the Type I error rate in adaptive designs with sample size reassessment and/or treatment selection. The first method adjusts the critical value using a simulation-based approach, which incorporates the number of patients at an interim analysis, the true response rates, the treatment selection rule, etc. We discuss the underlying assumptions of simulation-based procedures and give several examples where the Type I error rate is not controlled if some of the assumptions are violated. The second method is an adaptive Bonferroni-Holm test procedure based on conditional error rates of the individual treatment-control comparisons. We show that this procedure controls the Type I error rate, even if a deviation from a pre-planned adaptation rule or the time point of such a decision is necessary.

Multiple and Repeated Testing of Primary, Coprimary, and Secondary Hypotheses

In confirmatory clinical trials the Type I error rate must be controlled for claims forming the basis for approval and labeling of a new drug. Strong control of the familywise error rate is usually needed for hypotheses related to the primary endpoint(s). For hypotheses related to secondary endpoint(s) which are only of interest if the corresponding “parent” primary null hypotheses have been rejected, less strict error rate control might be sufficient. We review and extend procedures for families of primary and secondary hypotheses when either at least one of the primary hypotheses or all coprimary hypotheses must be rejected to claim success for the trial. Such families of hypotheses arise naturally from comparing several treatments with a control, combined noninferiority and superiority testing for primary and secondary variables, the presence of multiple primary or secondary endpoints or any combination thereof. We show that many of the procedures proposed in the literature follow a common underlying principle and in some cases can be improved. In addition we present some general results on Type I error rates for the different families and subfamilies of hypotheses and their relation to group-sequential testing of multiple hypotheses.

Memory and other properties of multiple test procedures generated by entangled graphs.

Methods for addressing multiplicity in clinical trials have attracted much attention during the past 20 years. They include the investigation of new classes of multiple test procedures, such as fixed sequence, fallback and gatekeeping procedures. More recently, sequentially rejective graphical test procedures have been introduced to construct and visualize complex multiple test strategies. These methods propagate the local significance level of a rejected null hypothesis to not-yet rejected hypotheses. In the graph defining the test procedure, hypotheses together with their local significance levels are represented by weighted vertices and the propagation rule by weighted directed edges. An algorithm provides the rules for updating the local significance levels and the transition weights after rejecting an individual hypothesis. These graphical procedures have no memory in the sense that the origin of the propagated significance level is ignored in subsequent iterations. However, in some clinical trial applications, memory is desirable to reflect the underlying dependence structure of the study objectives. In such cases, it would allow the further propagation of significance levels to be dependent on their origin and thus reflect the grouped parent-descendant structures of the hypotheses. We will give examples of such situations and show how to induce memory and other properties by convex combination of several individual graphs. The resulting entangled graphs provide an intuitive way to represent the underlying relative importance relationships between the hypotheses, are as easy to perform as the original individual graphs, remain sequentially rejective and control the familywise error rate in the strong sense.