Trial design for ineffectiveness research: a mixed-methods survey

Abstract

High-quality research demonstrating a lack of effectiveness may facilitate the ‘de-adoption’ of ineffective health services. However, there has been little debate on the optimal design for ineffectiveness research—studies exploring the research hypothesis that an intervention is ineffective. The aim of this study was to explore investigators’ preferences for trial design for ineffectiveness research. We conducted a mixed-methods online survey with principle investigators identified from clinicaltrials.gov. A vignette described researchers planning a trial to test a widely used intervention they hypothesised was ineffective. One multiple-choice question asked whether a superiority trial or equivalence trial design was favoured, and one free-response question asked about the reasons for that choice. Free-response answers were analysed using content analysis to identify related reasons. 139 participants completed the survey (completion rate 37.5%). Overall, 56.8% favoured superiority trials, 27.3% favoured equivalence trials and 15.8% were unsure. Reasons identified for favouring superiority trials were: (1) evidence of superiority should be required to justify active treatment, (2) superiority trials are more familiar, (3) placebo should not be the comparator in equivalence trials and (4) superiority trials require smaller sample sizes. Reasons identified for favouring equivalence trials were: (1) negative superiority trials represent a lack of evidence of effectiveness, not evidence of ineffectiveness and (2) the research hypothesis should not be the same as the null hypothesis. A minority of experienced researchers favour equivalence trials for ineffectiveness research, and misconceptions and lack of familiarity with equivalence trials may be contributing factors.