Kelechi K Olu

Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications

Objective To investigate the planning of subgroup analyses in protocols of randomised controlled trials and the agreement with corresponding full journal publications. Design Cohort of protocols of randomised controlled trial and subsequent full journal publications. Setting Six research ethics committees in Switzerland, Germany, and Canada. Data sources 894 protocols of randomised controlled trial involving patients approved by participating research ethics committees between 2000 and 2003 and 515 subsequent full journal publications. Results Of 894 protocols of randomised controlled trials, 252 (28.2%) included one or more planned subgroup analyses. Of those, 17 (6.7%) provided a clear hypothesis for at least one subgroup analysis, 10 (4.0%) anticipated the direction of a subgroup effect, and 87 (34.5%) planned a statistical test for interaction. Industry sponsored trials more often planned subgroup analyses compared with investigator sponsored trials (195/551 (35.4%) v 57/343 (16.6%), P<0.001). Of 515 identified journal publications, 246 (47.8%) reported at least one subgroup analysis. In 81 (32.9%) of the 246 publications reporting subgroup analyses, authors stated that subgroup analyses were prespecified, but this was not supported by 28 (34.6%) corresponding protocols. In 86 publications, authors claimed a subgroup effect, but only 36 (41.9%) corresponding protocols reported a planned subgroup analysis. Conclusions Subgroup analyses are insufficiently described in the protocols of randomised controlled trials submitted to research ethics committees, and investigators rarely specify the anticipated direction of subgroup effects. More than one third of statements in publications of randomised controlled trials about subgroup prespecification had no documentation in the corresponding protocols. Definitive judgments regarding credibility of claimed subgroup effects are not possible without access to protocols and analysis plans of randomised controlled trials.

Completion and publication rates of randomized controlled trials in surgery: an empirical study

OBJECTIVE To investigate the prevalence of discontinuation and nonpublication of surgical versus medical randomized controlled trials (RCTs) and to explore risk factors for discontinuation and nonpublication of surgical RCTs. BACKGROUND Trial discontinuation has significant scientific, ethical, and economic implications. To date, the prevalence of discontinuation of surgical RCTs is unknown. METHODS All RCT protocols approved between 2000 and 2003 by 6 ethics committees in Canada, Germany, and Switzerland were screened. Baseline characteristics were collected and, if published, full reports retrieved. Risk factors for early discontinuation for slow recruitment and nonpublication were explored using multivariable logistic regression analyses. RESULTS In total, 863 RCT protocols involving adult patients were identified, 127 in surgery (15%) and 736 in medicine (85%). Surgical trials were discontinued for any reason more often than medical trials [43% vs 27%, risk difference 16% (95% confidence interval [CI]: 5%-26%); P = 0.001] and more often discontinued for slow recruitment [18% vs 11%, risk difference 8% (95% CI: 0.1%-16%); P = 0.020]. The percentage of trials not published as full journal article was similar in surgical and medical trials (44% vs 40%, risk difference 4% (95% CI: -5% to 14%); P = 0.373). Discontinuation of surgical trials was a strong risk factor for nonpublication (odds ratio = 4.18, 95% CI: 1.45-12.06; P = 0.008). CONCLUSIONS Discontinuation and nonpublication rates were substantial in surgical RCTs and trial discontinuation was strongly associated with nonpublication. These findings need to be taken into account when interpreting surgical literature. Surgical trialists should consider feasibility studies before embarking on full-scale trials.

A systematic review of discontinued trials suggested that most reasons for recruitment failure were preventable

OBJECTIVE To collect and classify reported reasons for recruitment failure in discontinued randomized controlled trials (RCTs) and to assess reporting quality. METHODS We systematically searched MEDLINE and EMBASE (2010-2014) and a previous cohort of RCTs for published RCTs reporting trial discontinuation due to poor recruitment. Teams of two investigators selected eligible RCTs working independently and extracted information using standardized forms. We used an iterative approach to classify reasons for poor recruitment. RESULTS We included 172 RCTs discontinued due to poor recruitment (including 26 conference abstracts and 63 industry-funded RCTs). Of those, 131 (76%) reported one or more reasons for discontinuation due to poor recruitment. We identified 28 different reasons for recruitment failure; most frequently mentioned were overestimation of prevalence of eligible participants and prejudiced views of recruiters and participants on trial interventions. Few RCTs reported relevant details about the recruitment process such as how eligible participants were identified, the number of patients assessed for eligibility, and who actually recruited participants. CONCLUSION Our classification could serve as a checklist to assist investigators in the planning of RCTs. Most reasons for recruitment failure seem preventable with a pilot study that applies the planned informed consent procedure.

Cardiovascular effects and safety of long-term colchicine treatment: Cochrane review and meta-analysis

Colchicine is an old anti-inflammatory drug that has shown substantial cardiovascular benefits in recent trials. We systematically reviewed cardiovascular benefits and harms of colchicine in any population and specifically in patients with high cardiovascular risk. We evaluated randomised controlled trials comparing colchicine over at least 6 months versus any control in any adult population. Primary outcomes were all-cause mortality, myocardial infarction and adverse events. Cardiovascular mortality was a secondary outcome. We included 39 trials with 4992 patients. The quality of evidence for mortality outcomes and myocardial infarction was moderate but lower for adverse events. Colchicine had no effect on all-cause mortality (RR 0.94, 95% CI 0.82 to 1.09; I2=27%; 30 trials). Cardiovascular mortality was reduced in some but not all meta-analytical models (random-effects RR 0.34, 0.09 to 1.21, I2=9%; Petos OR 0.24, 0.09 to 0.64, I2=15%; Mantel-Haenszel fixed-effect RR 0.20, 0.06 to 0.68, I2=0%; 7 trials). The risk for myocardial infarction was reduced (RR 0.20, 0.07 to 0.57; 2 trials). There was no effect on total adverse events (RR 1.52, 0.93 to 2.46, I2=45%; 11 trials) but gastrointestinal intolerance was increased (RR 1.83, 1.03 to 3.26, I2=74%; 11 trials). Reporting of serious adverse events was inconsistent; no event occurred over 824 patient-years (4 trials). Effects in high cardiovascular risk populations were similar (4 trials; 1230 patients). We found no evidence supporting colchicine doses above 1 mg/day. Colchicine may have substantial cardiovascular benefits; however, there is sufficient uncertainty about its benefit and harm to indicate the need for large-scale trials to further evaluate this inexpensive, promising treatment in cardiovascular disease.

Agreements between Industry and Academia on Publication Rights: A Retrospective Study of Protocols and Publications of Randomized Clinical Trials

Background Little is known about publication agreements between industry and academic investigators in trial protocols and the consistency of these agreements with corresponding statements in publications. We aimed to investigate (i) the existence and types of publication agreements in trial protocols, (ii) the completeness and consistency of the reporting of these agreements in subsequent publications, and (iii) the frequency of co-authorship by industry employees. Methods and Findings We used a retrospective cohort of randomized clinical trials (RCTs) based on archived protocols approved by six research ethics committees between 13 January 2000 and 25 November 2003. Only RCTs with industry involvement were eligible. We investigated the documentation of publication agreements in RCT protocols and statements in corresponding journal publications. Of 647 eligible RCT protocols, 456 (70.5%) mentioned an agreement regarding publication of results. Of these 456, 393 (86.2%) documented an industry partner’s right to disapprove or at least review proposed manuscripts; 39 (8.6%) agreements were without constraints of publication. The remaining 24 (5.3%) protocols referred to separate agreement documents not accessible to us. Of those 432 protocols with an accessible publication agreement, 268 (62.0%) trials were published. Most agreements documented in the protocol were not reported in the subsequent publication (197/268 [73.5%]). Of 71 agreements reported in publications, 52 (73.2%) were concordant with those documented in the protocol. In 14 of 37 (37.8%) publications in which statements suggested unrestricted publication rights, at least one co-author was an industry employee. In 25 protocol-publication pairs, author statements in publications suggested no constraints, but 18 corresponding protocols documented restricting agreements. Conclusions Publication agreements constraining academic authors’ independence are common. Journal articles seldom report on publication agreements, and, if they do, statements can be discrepant with the trial protocol.

Premature Discontinuation of Randomized Trials in Critical and Emergency Care: A Retrospective Cohort Study

Objectives:Randomized clinical trials that enroll patients in critical or emergency care (acute care) setting are challenging because of narrow time windows for recruitment and the inability of many patients to provide informed consent. To assess the extent that recruitment challenges lead to randomized clinical trial discontinuation, we compared the discontinuation of acute care and nonacute care randomized clinical trials. Design:Retrospective cohort of 894 randomized clinical trials approved by six institutional review boards in Switzerland, Germany, and Canada between 2000 and 2003. Setting:Randomized clinical trials involving patients in an acute or nonacute care setting. Subjects and Interventions:We recorded trial characteristics, self-reported trial discontinuation, and self-reported reasons for discontinuation from protocols, corresponding publications, institutional review board files, and a survey of investigators. Measurements and Main Results:Of 894 randomized clinical trials, 64 (7%) were acute care randomized clinical trials (29 critical care and 35 emergency care). Compared with the 830 nonacute care randomized clinical trials, acute care randomized clinical trials were more frequently discontinued (28 of 64, 44% vs 221 of 830, 27%; p = 0.004). Slow recruitment was the most frequent reason for discontinuation, both in acute care (13 of 64, 20%) and in nonacute care randomized clinical trials (7 of 64, 11%). Logistic regression analyses suggested the acute care setting as an independent risk factor for randomized clinical trial discontinuation specifically as a result of slow recruitment (odds ratio, 4.00; 95% CI, 1.72–9.31) after adjusting for other established risk factors, including nonindustry sponsorship and small sample size. Conclusions:Acute care randomized clinical trials are more vulnerable to premature discontinuation than nonacute care randomized clinical trials and have an approximately four-fold higher risk of discontinuation due to slow recruitment. These results highlight the need for strategies to reliably prevent and resolve slow patient recruitment in randomized clinical trials conducted in the critical and emergency care setting.

An analysis of protocols and publications suggested that most discontinuations of clinical trials were not based on preplanned interim analyses or stopping rules

OBJECTIVES To investigate the frequency of interim analyses, stopping rules, and data safety and monitoring boards (DSMBs) in protocols of randomized controlled trials (RCTs); to examine these features across different reasons for trial discontinuation; and to identify discrepancies in reporting between protocols and publications. STUDY DESIGN AND SETTING We used data from a cohort of RCT protocols approved between 2000 and 2003 by six research ethics committees in Switzerland, Germany, and Canada. RESULTS Of 894 RCT protocols, 289 prespecified interim analyses (32.3%), 153 stopping rules (17.1%), and 257 DSMBs (28.7%). Overall, 249 of 894 RCTs (27.9%) were prematurely discontinued; mostly due to reasons such as poor recruitment, administrative reasons, or unexpected harm. Forty-six of 249 RCTs (18.4%) were discontinued due to early benefit or futility; of those, 37 (80.4%) were stopped outside a formal interim analysis or stopping rule. Of 515 published RCTs, there were discrepancies between protocols and publications for interim analyses (21.1%), stopping rules (14.4%), and DSMBs (19.6%). CONCLUSION Two-thirds of RCT protocols did not consider interim analyses, stopping rules, or DSMBs. Most RCTs discontinued for early benefit or futility were stopped without a prespecified mechanism. When assessing trial manuscripts, journals should require access to the protocol.

Premature trial discontinuation often not accurately reflected in registries: comparison of registry records with publications

BACKGROUND One quarter of randomized clinical trials (RCTs) are prematurely discontinued and frequently remain unpublished. Trial registries can document whether a trial is ongoing, suspended, discontinued, or completed and therefore represent an important source for trial status information. The accuracy of this information is unclear. OBJECTIVE To examine the accuracy of completion status and reasons for discontinuation documented in trial registries as compared to corresponding publications of discontinued RCTs and to investigate potential predictors for accurate trial status information in registries. METHODS We conducted a cross-sectional study comparing information provided in publications (reference standard) to corresponding registry entries. First, we reviewed publications of RCTs providing information on both discontinuation and registration. We identified eligible publications through systematic searches of MEDLINE and EMBASE (2010-2014) and an international cohort of 1,017 RCTs initiated between 2000 and 2003. Second, pairs of investigators independently and in duplicate extracted data from publications and corresponding registry records. Third, for each discontinued RCT, we compared publication information to registry information. We used multivariable regression to examine whether accurate labeling of trials as discontinued (vs. other status) in the registry was associated with recent initiation of RCT, industry sponsorship, multicenter design, or larger sample size. RESULTS We identified 173 publications of RCTs that were discontinued due to slow recruitment (55%), harm (16%), futility (11%), benefit (5%), other reasons (3%), or multiple reasons (9%). Trials were registered with clinicaltrials.gov (77%), isrctn.com (14%), or other registries (8%). Of the 173 corresponding registry records, 77 (45%) trials were labeled as discontinued and 57 (33%) provided a reason for discontinuation (of which 53, 93%, provided the same reason as in the publication). Labeling of discontinued trials as discontinued (vs. other label) in corresponding trial registry records improved over time (adjusted odds ratio 1.16 per year, confidence interval 1.04-1.30) and was possibly associated with industry sponsorship (2.01, 0.99-4.07) but unlikely with multicenter status (0.81, 0.32-2.04) or sample size (1.07, 0.89-1.29). CONCLUSIONS Less than half of published discontinued RCTs were accurately labelled as discontinued in corresponding registry records. One-third of registry records provided a reason for discontinuation. Current trial status information in registries should be viewed with caution.

Premature Discontinuation of Pediatric Randomized Controlled Trials: A Retrospective Cohort Study

Objectives To determine the proportion of pediatric randomized controlled trials (RCTs) that are prematurely discontinued, examine the reasons for discontinuation, and compare the risk for recruitment failure in pediatric and adult RCTs. Study design A retrospective cohort study of RCTs approved by 1 of 6 Research Ethics Committees (RECs) in Switzerland, Germany, and Canada between 2000 and 2003. We recorded trial characteristics, trial discontinuation, and reasons for discontinuation from protocols, corresponding publications, REC files, and a survey of trialists. Results We included 894 RCTs, of which 86 enrolled children and 808 enrolled adults. Forty percent of the pediatric RCTs and 29% of the adult RCTs were discontinued. Slow recruitment accounted for 56% of pediatric RCT discontinuations and 43% of adult RCT discontinuations. Multivariable logistic regression analyses suggested that pediatric RCT was not an independent risk factor for recruitment failure after adjustment for other potential risk factors (aOR, 1.22; 95% CI, 0.57‐2.63). Independent risk factors were acute care setting (aOR, 4.00; 95% CI, 1.72‐9.31), nonindustry sponsorship (aOR, 4.45; 95% CI, 2.59‐7.65), and smaller planned sample size (aOR, 1.05; 95% CI 1.01‐1.09, in decrements of 100 participants). Conclusion Forty percent of pediatric RCTs were discontinued prematurely, owing predominately to slow recruitment. Enrollment of children was not an independent risk factor for recruitment failure.

Discontinuation and non-publication of randomised clinical trials supported by the main public funding body in Switzerland: a retrospective cohort study

Objective The Swiss National Science Foundation (SNSF) promotes academic excellence through competitive selection of study proposals and rigorous evaluation of feasibility, but completion status and publication history of SNSF-supported randomised clinical trials (RCTs) remain unclear. The main objectives were to review all healthcare RCTs supported by the SNSF for trial discontinuation and non-publication, to investigate potential risk factors for trial discontinuation due to poor recruitment and non-publication, and to compare findings to other Swiss RCTs not supported by the SNSF. Design We established a retrospective cohort of all SNSF-supported RCTs for which recruitment and funding had ended in 2015 or earlier. For each RCT, two investigators independently searched corresponding publications in electronic databases. In addition, we approached all principal investigators to ask for additional publications and information about trial discontinuation. Teams of two investigators independently extracted details about study design, recruitment of participants, outcomes, analysis and sample size from the original proposal and, if available, from trial registries and publications. We used multivariable regression analysis to explore potential risk factors associated with discontinuation due to poor recruitment and with non-publication, and to compare our results with data from a previous cohort of Swiss RCTs not supported by the SNSF. Results We included 101 RCTs supported by the SNSF between 1986 and 2015. Eighty-seven (86%) principal investigators responded to our survey. Overall, 69 (68%) RCTs were completed, 26 (26%) RCTs were prematurely discontinued (all due to slow recruitment) and the completion status remained unclear for 6 (6%) RCTs. For analysing publication status, we excluded 4 RCTs for which follow-up was still ongoing and 9 for which manuscripts were still in preparation. Of the remaining 88 RCTs, 53 (60%) were published as full articles in peer-reviewed journals. Multivariable regression models suggested that discontinued trials were at higher risk for non-publication than completed trials (adjusted OR 7.61; 95% CI 2.44 to 27.09). Compared with other Swiss RCTs, the risk of discontinuation for SNSF-supported RCTs was higher than in industry-initiated RCTs (adjusted OR 3.84; 95% CI 1.68 to 8.74), but not significantly different from investigator-initiated RCTs not supported by the SNSF (adjusted OR 1.05; 95% CI 0.51 to 2.11). We found no evidence that the proportion of discontinued or unpublished RCTs decreased over the last 20 years. Conclusions One out of four SNSF-supported RCTs were prematurely discontinued due to slow recruitment, 40% of all included RCTs and 70% of all discontinued RCTs were not published in peer-reviewed journals. There is a case to reconsider how public funding bodies such as the SNSF could improve their feasibility assessment and promote publication of RCTs irrespective of completion status.