Stephen Senn

A 4-year trial of Tiotropium in chronic obstructive pulmonary disease

BACKGROUND Previous studies showing that tiotropium improves multiple end points in patients with chronic obstructive pulmonary disease (COPD) led us to examine the long-term effects of tiotropium therapy. METHODS In this randomized, double-blind trial, we compared 4 years of therapy with either tiotropium or placebo in patients with COPD who were permitted to use all respiratory medications except inhaled anticholinergic drugs. The patients were at least 40 years of age, with a forced expiratory volume in 1 second (FEV(1)) of 70% or less after bronchodilation and a ratio of FEV(1) to forced vital capacity (FVC) of 70% or less. Coprimary end points were the rate of decline in the mean FEV(1) before and after bronchodilation beginning on day 30. Secondary end points included measures of FVC, changes in response on St. Georges Respiratory Questionnaire (SGRQ), exacerbations of COPD, and mortality. RESULTS Of a total of 5993 patients (mean age, 65+/-8 years) with a mean FEV(1) of 1.32+/-0.44 liters after bronchodilation (48% of predicted value), we randomly assigned 2987 to the tiotropium group and 3006 to the placebo group. Mean absolute improvements in FEV(1) in the tiotropium group were maintained throughout the trial (ranging from 87 to 103 ml before bronchodilation and from 47 to 65 ml after bronchodilation), as compared with the placebo group (P<0.001). After day 30, the differences between the two groups in the rate of decline in the mean FEV(1) before and after bronchodilation were not significant. The mean absolute total score on the SGRQ was improved (lower) in the tiotropium group, as compared with the placebo group, at each time point throughout the 4-year period (ranging from 2.3 to 3.3 units, P<0.001). At 4 years and 30 days, tiotropium was associated with a reduction in the risks of exacerbations, related hospitalizations, and respiratory failure. CONCLUSIONS In patients with COPD, therapy with tiotropium was associated with improvements in lung function, quality of life, and exacerbations during a 4-year period but did not significantly reduce the rate of decline in FEV(1). (ClinicalTrials.gov number, NCT00144339.)

Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations

Misinterpretation and abuse of statistical tests, confidence intervals, and statistical power have been decried for decades, yet remain rampant. A key problem is that there are no interpretations of these concepts that are at once simple, intuitive, correct, and foolproof. Instead, correct use and interpretation of these statistics requires an attention to detail which seems to tax the patience of working scientists. This high cognitive demand has led to an epidemic of shortcut definitions and interpretations that are simply wrong, sometimes disastrously so—and yet these misinterpretations dominate much of the scientific literature. In light of this problem, we provide definitions and a discussion of basic statistics that are more general and critical than typically found in traditional introductory expositions. Our goal is to provide a resource for instructors, researchers, and consumers of statistics whose knowledge of statistical theory and technique may be limited but who wish to avoid and spot misinterpretations. We emphasize how violation of often unstated analysis protocols (such as selecting analyses for presentation based on the P values they produce) can lead to small P values even if the declared test hypothesis is correct, and can lead to large P values even if that hypothesis is incorrect. We then provide an explanatory list of 25 misinterpretations of P values, confidence intervals, and power. We conclude with guidelines for improving statistical interpretation and reporting.

A Systematic Review of the Antifungal Effectiveness and Tolerability of Amphotericin B Formulations

OBJECTIVE A systematic review was performed to compare the effectiveness and tolerability of lipid-based amphotericin B (AmB) formulations and conventional AmB in the treatment of systemic fungal infections. METHODS The literature and unpublished studies were searched using MEDLINE, EMBASE, Biological Abstracts, AIDSLINE, CANCERLIT, CRD database, Cochrane Controlled Trials Register, and other databases. Search terms included: amphotericin, liposom*, lipid*, colloid*, antifungal agents, and mycoses. Studies were selected according to predetermined criteria. The outcome measures reviewed were efficacy, mortality, renal toxicity, and infusion-related reactions. Meta-analyses and number-needed-to-treat (NNT) analyses were performed. RESULTS Seven studies (8 publications) met the entry criteria. Meta-analysis showed that lipid-based formulations significantly reduced all-cause mortality risk by an estimated 28% compared with conventional AmB (odds ratio [OR], 0.72; 95% CI, 0.54 to 0.97). There was no significant difference in efficacy between the lipid-based formulations and conventional AmB (OR, 1.21; 95% CI, 0.98 to 1.49). AmB lipid complex (ABLC) and liposomal AmB (L-AmB) significantly reduced the risk of doubling serum creatinine by an estimated 58% (OR, 0.42; 95% CI, 0.33 to 0.54). There was no significant reduction in risk of infusion-related reactions with lipid-based formulations, although this was difficult to interpret given the lack of consistent control of confounding factors. Comparing the lipid-based formulations with conventional AmB, the overall NNT to prevent 1 death was 31. The NNT to prevent a doubling of serum creatinine for both ABLC and L-AmB compared with conventional AmB was 6. CONCLUSIONS This study demonstrates advantages with lipid-based formulations over conventional AmB in terms of reduced risk of mortality and renal toxicity. Future trials in patients with proven fungal infection should control for factors such as premedication, infusion rates, fluid preloading, sodium/potassium supplementation, and concomitant medication.

Some controversies in planning and analysing multi‐centre trials

It is shown that a rational approach to planning multi-centre trials will lead to an unequal distribution of patients across centres. Consequently different approaches to estimation will yield different estimates. However, some such approaches are not reasonable and it is concluded that multi-centre trials are less problematic than is commonly supposed.

Repeated measures in clinical trials: simple strategies for analysis using summary measures

The summary measures approach to analysing repeated measures is described. The circumstances under which it can be advantageous to use such measures are considered. Strategies for baseline adjustment where there are multiple baselines are examined, as is the choice of appropriate summary statistic. A compromise trend/mean measure, regression through the origin, is proposed as being useful under some circumstances. An analysis using this measure is illustrated with a suitable example.

Individual response to treatment: is it a valid assumption?

Most drug trials assume that patients respond consistently to treatment, but the assumption is rarely tested. If patients vary randomly in their response to a drug rather than some patients never responding, searches for a genetic basis for non-response are futile

The Many Modes of Meta

Many of the reservations that might attach to the use of meta-analysis generally (for example, regarding publication bias) do not apply in the specific context of drug development. A meta-analysis is, in fact, a highly natural and appropriate way to summarize the results of a drug development program, as has been recognized in the International Conference on Harmonization (ICH) E9 guideline. Since a sponsor will have access to all original data, the data from a set of clinical trials in a drug development program have a very similar (hierarchical) structure to the data from a set of centers in a single multicenter trial. Curiously, however, the controversies over analyzing multicenter trials have often been different from those in the field of meta-analysis. In this paper, the options open to the meta-analyst in drug development are examined and comparisons to approaches used in analyzing multicenter trials are made in an attempt to provide some unifying insights, in particular as regards the handling of models with interactions.

Overstating the evidence - double counting in meta-analysis and related problems

BackgroundThe problem of missing studies in meta-analysis has received much attention. Less attention has been paid to the more serious problem of double counting of evidence.MethodsVarious problems in overstating the precision of results from meta-analyses are described and illustrated with examples, including papers from leading medical journals. These problems include, but are not limited to, simple double counting of the same studies, double counting of some aspects of the studies, inappropriate imputation of results, and assigning spurious precision to individual studies.ResultsSome suggestions are made as to how the quality and reliability of meta-analysis can be improved. It is proposed that the key to quality in meta-analysis lies in the results being transparent and checkable.ConclusionExisting quality check lists for meta-analysis do little to encourage an appropriate attitude to combining evidence and to statistical analysis. Journals and other relevant organisations should encourage authors to make data available and make methods explicit. They should also act promptly to withdraw meta-analyses when mistakes are found.

Individual therapy: New dawn or false dawn?

The sequencing of the human genome brings with it the hope that greater understanding of genetic components of disease will allow the more specific targeting of therapies. It has also been suggested that it will permit sponsors to run “cleaner” clinical trials with less variability and a consequent saving in patient numbers. However, we do not know how much of the variation in response that we see from patient to patient in clinical trials is genetic, because we rarely design the sort of trials that would allow us to identify patient-by-treatment interaction. Such interaction provides an upper bound for gene-by-treatment interaction for a group of patients studied since patients differ by more than their genes. On the other hand, however, the variability seen within a clinical trial may generally be expected to be less than the total variation that would be seen within a population. There is a related statistical issue to do with the interpretation of effects from clinical trials. This arises because there is confusion between experimental and sampling models of clinical research. It is concluded that we may have to pay careful attention to certain design features of clinical trials if we wish to make progress in this field.

Consensus and controversy in pharmaceutical statistics

Drug development is a highly regulated business. Statistics plays an important part in measuring and reporting the efficacy and tolerability of pharmaceuticals and this is reflected in American, European, Japanese and international guidelines of some considerable detail which cover the way in which clinical trials are to be planned, run and analysed. The dominant framework for analysis is frequentist but Bayesian approaches are becoming more popular. Some controversial issues in drug development are considered in the light of the International Conference on Harmonisations 1999 international statistical guidelines to see whether any Bayes-requentist consensus is possible.