Lucy Turner

ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions

Non-randomised studies of the effects of interventions are critical to many areas of healthcare evaluation, but their results may be biased. It is therefore important to understand and appraise their strengths and weaknesses. We developed ROBINS-I (“Risk Of Bias In Non-randomised Studies - of Interventions”), a new tool for evaluating risk of bias in estimates of the comparative effectiveness (harm or benefit) of interventions from studies that did not use randomisation to allocate units (individuals or clusters of individuals) to comparison groups. The tool will be particularly useful to those undertaking systematic reviews that include non-randomised studies.

Effectiveness of quality improvement strategies on the management of diabetes: a systematic review and meta-analysis

BACKGROUND The effectiveness of quality improvement (QI) strategies on diabetes care remains unclear. We aimed to assess the effects of QI strategies on glycated haemoglobin (HbA(1c)), vascular risk management, microvascular complication monitoring, and smoking cessation in patients with diabetes. METHODS We identified studies through Medline, the Cochrane Effective Practice and Organisation of Care database (from inception to July 2010), and references of included randomised clinical trials. We included trials assessing 11 predefined QI strategies or financial incentives targeting health systems, health-care professionals, or patients to improve management of adult outpatients with diabetes. Two reviewers independently abstracted data and appraised risk of bias. FINDINGS We reviewed 48 cluster randomised controlled trials, including 2538 clusters and 84,865 patients, and 94 patient randomised controlled trials, including 38,664 patients. In random effects meta-analysis, the QI strategies reduced HbA(1c) by a mean difference of 0·37% (95% CI 0·28-0·45; 120 trials), LDL cholesterol by 0·10 mmol/L (0·05-0.14; 47 trials), systolic blood pressure by 3·13 mm Hg (2·19-4·06, 65 trials), and diastolic blood pressure by 1·55 mm Hg (0·95-2·15, 61 trials) versus usual care. We noted larger effects when baseline concentrations were greater than 8·0% for HbA(1c), 2·59 mmol/L for LDL cholesterol, and 80 mm Hg for diastolic and 140 mm Hg for systolic blood pressure. The effectiveness of QI strategies varied depending on baseline HbA(1c) control. QI strategies increased the likelihood that patients received aspirin (11 trials; relative risk [RR] 1·33, 95% CI 1·21-1·45), antihypertensive drugs (ten trials; RR 1·17, 1·01-1·37), and screening for retinopathy (23 trials; RR 1·22, 1·13-1·32), renal function (14 trials; RR 128, 1·13-1·44), and foot abnormalities (22 trials; RR 1·27, 1·16-1·39). However, statin use (ten trials; RR 1·12, 0·99-1·28), hypertension control (18 trials; RR 1·01, 0·96-1·07), and smoking cessation (13 trials; RR 1·13, 0·99-1·29) were not significantly increased. INTERPRETATION Many trials of QI strategies showed improvements in diabetes care. Interventions targeting the system of chronic disease management along with patient-mediated QI strategies should be an important component of interventions aimed at improving diabetes management. Interventions solely targeting health-care professionals seem to be beneficial only if baseline HbA(1c) control is poor. FUNDING Ontario Ministry of Health and Long-term Care and the Alberta Heritage Foundation for Medical Research (now Alberta Innovates--Health Solutions).

Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review.

BackgroundThe Consolidated Standards of Reporting Trials (CONSORT) Statement is intended to facilitate better reporting of randomised clinical trials (RCTs). A systematic review recently published in the Cochrane Library assesses whether journal endorsement of CONSORT impacts the completeness of reporting of RCTs; those findings are summarised here.MethodsEvaluations assessing the completeness of reporting of RCTs based on any of 27 outcomes formulated based on the 1996 or 2001 CONSORT checklists were included; two primary comparisons were evaluated. The 27 outcomes were: the 22 items of the 2001 CONSORT checklist, four sub-items describing blinding and a ‘total summary score’ of aggregate items, as reported. Relative risks (RR) and 99% confidence intervals were calculated to determine effect estimates for each outcome across evaluations.ResultsFifty-three reports describing 50 evaluations of 16,604 RCTs were assessed for adherence to at least one of 27 outcomes. Sixty-nine of 81 meta-analyses show relative benefit from CONSORT endorsement on completeness of reporting. Between endorsing and non-endorsing journals, 25 outcomes are improved with CONSORT endorsement, five of these significantly (α = 0.01). The number of evaluations per meta-analysis was often low with substantial heterogeneity; validity was assessed as low or unclear for many evaluations.ConclusionsThe results of this review suggest that journal endorsement of CONSORT may benefit the completeness of reporting of RCTs they publish. No evidence suggests that endorsement hinders the completeness of RCT reporting. However, despite relative improvements when CONSORT is endorsed by journals, the completeness of reporting of trials remains sub-optimal. Journals are not sending a clear message about endorsement to authors submitting manuscripts for publication. As such, fidelity of endorsement as an ‘intervention’ has been weak to date. Journals need to take further action regarding their endorsement and implementation of CONSORT to facilitate accurate, transparent and complete reporting of trials.

Relation of completeness of reporting of health research to journals' endorsement of reporting guidelines: systematic review.

Objective To assess whether the completeness of reporting of health research is related to journals’ endorsement of reporting guidelines. Design Systematic review. Data sources Reporting guidelines from a published systematic review and the EQUATOR Network (October 2011). Studies assessing the completeness of reporting by using an included reporting guideline (termed “evaluations”) (1990 to October 2011; addendum searches in January 2012) from searches of either Medline, Embase, and the Cochrane Methodology Register or Scopus, depending on reporting guideline name. Study selection English language reporting guidelines that provided explicit guidance for reporting, described the guidance development process, and indicated use of a consensus development process were included. The CONSORT statement was excluded, as evaluations of adherence to CONSORT had previously been reviewed. English or French language evaluations of included reporting guidelines were eligible if they assessed the completeness of reporting of studies as a primary intent and those included studies enabled the comparisons of interest (that is, after versus before journal endorsement and/or endorsing versus non-endorsing journals). Data extraction Potentially eligible evaluations of included guidelines were screened initially by title and abstract and then as full text reports. If eligibility was unclear, authors of evaluations were contacted; journals’ websites were consulted for endorsement information where needed. The completeness of reporting of reporting guidelines was analyzed in relation to endorsement by item and, where consistent with the authors’ analysis, a mean summed score. Results 101 reporting guidelines were included. Of 15 249 records retrieved from the search for evaluations, 26 evaluations that assessed completeness of reporting in relation to endorsement for nine reporting guidelines were identified. Of those, 13 evaluations assessing seven reporting guidelines (BMJ economic checklist, CONSORT for harms, PRISMA, QUOROM, STARD, STRICTA, and STROBE) could be analyzed. Reporting guideline items were assessed by few evaluations. Conclusions The completeness of reporting of only nine of 101 health research reporting guidelines (excluding CONSORT) has been evaluated in relation to journals’ endorsement. Items from seven reporting guidelines were quantitatively analyzed, by few evaluations each. Insufficient evidence exists to determine the relation between journals’ endorsement of reporting guidelines and the completeness of reporting of published health research reports. Journal editors and researchers should consider collaborative prospectively designed, controlled studies to provide more robust evidence. Systematic review registration Not registered; no known register currently accepts protocols for methodology systematic reviews.

The quality of reporting methods and results in network meta-analyses: an overview of reviews and suggestions for improvement.

Introduction Some have suggested the quality of reporting of network meta-analyses (a technique used to synthesize information to compare multiple interventions) is sub-optimal. We sought to review information addressing this claim. Objective To conduct an overview of existing evaluations of quality of reporting in network meta-analyses and indirect treatment comparisons, and to compile a list of topics which may require detailed reporting guidance to enhance future reporting quality. Methods An electronic search of Medline and the Cochrane Registry of methodologic studies (January 2004–August 2013) was performed by an information specialist. Studies describing findings from quality of reporting assessments were sought. Screening of abstracts and full texts was performed by two team members. Descriptors related to all aspects of reporting a network meta-analysis were summarized. Results We included eight reports exploring the quality of reporting of network meta-analyses. From past reviews, authors found several aspects of network meta-analyses were inadequately reported, including primary information about literature searching, study selection, and risk of bias evaluations; statement of the underlying assumptions for network meta-analysis, as well as efforts to verify their validity; details of statistical models used for analyses (including information for both Bayesian and Frequentist approaches); completeness of reporting of findings; and approaches for summarizing probability measures as additional important considerations. Conclusions While few studies were identified, several deficiencies in the current reporting of network meta-analyses were observed. These findings reinforce the need to develop reporting guidance for network meta-analyses. Findings from this review will be used to guide next steps in the development of reporting guidance for network meta-analysis in the format of an extension of the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analysis) Statement.

Perioperative use of pregabalin for acute pain-a systematic review and meta-analysis.

Abstract Evidence supporting postoperative pain management using pregabalin as an adjunct intervention across various surgical pain models is lacking. The objective of this systematic review was to evaluate “model-specific” comparative effectiveness and harms of pregabalin following a previously published systematic review protocol. MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials were searched from inception through August 2013. Data were screened and single extraction with independent verification and dual risk of bias assessment was performed. Quality of evidence (QoE) was rated using the GRADE approach. Primary outcomes were pain relief at rest and on movement and reduction in postoperative analgesic consumption. A total of 1423 records were screened, and 43 studies were included. Perioperative pregabalin resulted in: 16% (95% confidence interval [CI], 9%-21%) reduction in analgesic consumption (moderate QoE, 24 trials) and a small reduction in the magnitude of pain in surgeries associated with pronociceptive pain. Per 1000 patients, 10 more will experience blurred vision (95% CI, 5-20 more; moderate QoE, 17 trials) and 41 more sedation (95% CI, 13-77 more, 17 trials). To prevent 1 case of perioperative nausea and vomiting, the number needed to treat is 11 (95% CI: 7-28, 25 trials). Inadequate evidence addressed outcomes of enhanced recovery and serious harms. Pregabalin analgesic effectiveness is largely restricted to surgical procedures associated with pronociceptive mechanisms. The clinical significance of observed pregabalin benefits must be weighed against the uncertainties about serious harms and enhanced recovery to inform the careful selection of surgical patients. Recommendations for future research are proposed.

Effectiveness of brief interventions as part of the screening, brief intervention and referral to treatment (SBIRT) model for reducing the non-medical use of psychoactive substances: a systematic review protocol

BackgroundThere is a significant public health burden associated with substance use in Canada. The early detection and/or treatment of risky substance use has the potential to dramatically improve outcomes for those who experience harms from the non-medical use of psychoactive substances, particularly adolescents whose brains are still undergoing development. The Screening, Brief Intervention, and Referral to Treatment model is a comprehensive, integrated approach for the delivery of early intervention and treatment services for individuals experiencing substance use-related harms, as well as those who are at risk of experiencing such harm.MethodsThis article describes the protocol for a systematic review of the effectiveness of brief interventions as part of the Screening, Brief Intervention, and Referral to Treatment model for reducing the non-medical use of psychoactive substances. Studies will be selected in which brief interventions target non-medical psychoactive substance use (excluding alcohol, nicotine, or caffeine) among those 12 years and older who are opportunistically screened and deemed at risk of harms related to psychoactive substance use. We will include one-on-one verbal interventions and exclude non-verbal brief interventions (for example, the provision of information such as a pamphlet or online interventions) and group interventions. Primary, secondary and adverse outcomes of interest are prespecified. Randomized controlled trials will be included; non-randomized controlled trials, controlled before-after studies and interrupted time series designs will be considered in the absence of randomized controlled trials. We will search several bibliographic databases (for example, MEDLINE, EMBASE, CINAHL, PsycINFO, CORK) and search sources for grey literature. We will meta-analyze studies where possible. We will conduct subgroup analyses, if possible, according to drug class and intervention setting.DiscussionThis review will provide evidence on the effectiveness of brief interventions as part of the Screening, Brief Intervention, and Referral to Treatment protocol aimed at the non-medical use of psychoactive substances and may provide guidance as to where future research might be most beneficial.

Potential predatory and legitimate biomedical journals: can you tell the difference? A cross-sectional comparison

BackgroundThe Internet has transformed scholarly publishing, most notably, by the introduction of open access publishing. Recently, there has been a rise of online journals characterized as ‘predatory’, which actively solicit manuscripts and charge publications fees without providing robust peer review and editorial services. We carried out a cross-sectional comparison of characteristics of potential predatory, legitimate open access, and legitimate subscription-based biomedical journals.MethodsOn July 10, 2014, scholarly journals from each of the following groups were identified – potential predatory journals (source: Beall’s List), presumed legitimate, fully open access journals (source: PubMed Central), and presumed legitimate subscription-based (including hybrid) journals (source: Abridged Index Medicus). MEDLINE journal inclusion criteria were used to screen and identify biomedical journals from within the potential predatory journals group. One hundred journals from each group were randomly selected. Journal characteristics (e.g., website integrity, look and feel, editors and staff, editorial/peer review process, instructions to authors, publication model, copyright and licensing, journal location, and contact) were collected by one assessor and verified by a second. Summary statistics were calculated.ResultsNinety-three predatory journals, 99 open access, and 100 subscription-based journals were analyzed; exclusions were due to website unavailability. Many more predatory journals’ homepages contained spelling errors (61/93, 66%) and distorted or potentially unauthorized images (59/93, 63%) compared to open access journals (6/99, 6% and 5/99, 5%, respectively) and subscription-based journals (3/100, 3% and 1/100, 1%, respectively). Thirty-one (33%) predatory journals promoted a bogus impact metric – the Index Copernicus Value – versus three (3%) open access journals and no subscription-based journals. Nearly three quarters (n = 66, 73%) of predatory journals had editors or editorial board members whose affiliation with the journal was unverified versus two (2%) open access journals and one (1%) subscription-based journal in which this was the case. Predatory journals charge a considerably smaller publication fee (median

The influence of CONSORT on the quality of reporting of randomised controlled trials: an updated review

100 USD, IQR

The evolution of assessing bias in Cochrane systematic reviews of interventions: celebrating methodological contributions of the Cochrane Collaboration.

63–