Jelena Savovic

The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials

Flaws in the design, conduct, analysis, and reporting of randomised trials can cause the effect of an intervention to be underestimated or overestimated. The Cochrane Collaboration’s tool for assessing risk of bias aims to make the process clearer and more accurate

ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions

Non-randomised studies of the effects of interventions are critical to many areas of healthcare evaluation, but their results may be biased. It is therefore important to understand and appraise their strengths and weaknesses. We developed ROBINS-I (“Risk Of Bias In Non-randomised Studies - of Interventions”), a new tool for evaluating risk of bias in estimates of the comparative effectiveness (harm or benefit) of interventions from studies that did not use randomisation to allocate units (individuals or clusters of individuals) to comparison groups. The tool will be particularly useful to those undertaking systematic reviews that include non-randomised studies.

Methods for causality assessment of adverse drug reactions: a systematic review.

Numerous methods for causality assessment of adverse drug reactions (ADRs) have been published. The aim of this review is to provide an overview of these methods and discuss their strengths and weaknesses. We conducted electronic searches in MEDLINE (via PubMed), EMBASE and the Cochrane databases to find all assessment methods. Thirty-four different methods were found, falling into three broad categories: expert judgement/global introspection, algorithms and probabilistic methods (Bayesian approaches). Expert judgements are individual assessments based on previous knowledge and experience in the field using no standardized tool to arrive at conclusions regarding causality. Algorithms are sets of specific questions with associated scores for calculating the likelihood of a cause-effect relationship. Bayesian approaches use specific findings in a case to transform the prior estimate of probability into a posterior estimate of probability of drug causation. The prior probability is calculated from epidemiological information and the posterior probability combines this background information with the evidence in the individual case to come up with an estimate of causation. As a result of problems of reproducibility and validity, no single method is universally accepted. Different causality categories are adopted in each method, and the categories are assessed using different criteria. Because assessment methods are also not entirely devoid of individual judgements, inter-rater reliability can be low. In conclusion, there is still no method universally accepted for causality assessment of ADRs.

ROBIS: A new tool to assess risk of bias in systematic reviews was developed

Objective To develop ROBIS, a new tool for assessing the risk of bias in systematic reviews (rather than in primary studies). Study Design and Setting We used four-stage approach to develop ROBIS: define the scope, review the evidence base, hold a face-to-face meeting, and refine the tool through piloting. Results ROBIS is currently aimed at four broad categories of reviews mainly within health care settings: interventions, diagnosis, prognosis, and etiology. The target audience of ROBIS is primarily guideline developers, authors of overviews of systematic reviews (“reviews of reviews”), and review authors who might want to assess or avoid risk of bias in their reviews. The tool is completed in three phases: (1) assess relevance (optional), (2) identify concerns with the review process, and (3) judge risk of bias. Phase 2 covers four domains through which bias may be introduced into a systematic review: study eligibility criteria; identification and selection of studies; data collection and study appraisal; and synthesis and findings. Phase 3 assesses the overall risk of bias in the interpretation of review findings and whether this considered limitations identified in any of the phase 2 domains. Signaling questions are included to help judge concerns with the review process (phase 2) and the overall risk of bias in the review (phase 3); these questions flag aspects of review design related to the potential for bias and aim to help assessors judge risk of bias in the review process, results, and conclusions. Conclusions ROBIS is the first rigorously developed tool designed specifically to assess the risk of bias in systematic reviews.

Selenium and prostate cancer: systematic review and meta-analysis

BACKGROUND Prostate cancer is a growing public health problem. Several human studies have shown a potentially protective effect of selenium, but the conclusions from published reports are inconsistent. OBJECTIVE The objective was to examine the evidence for relations between selenium intake, selenium status, and prostate cancer risk. DESIGN This was a systematic review and meta-analysis of randomized controlled trials, case-control studies, and prospective cohort studies. The World Cancer Research Fund/American Institute for Cancer Research Continuous Update Project database was searched up to September 2010. The studies included reported measurements of selenium intake or status (plasma, serum, or toenail selenium), assessments of prostate cancer cases (number of events), and the RR in the adult population. Meta-analyses were performed, and study quality, heterogeneity, and small study effects were assessed. Dose-response meta-analyses were used, with restricted cubic splines and fractional polynomials for nonlinear trends, to investigate the association between selenium status and prostate cancer risk. RESULTS Twelve studies with a total of 13,254 participants and 5007 cases of prostate cancer were included. The relation between plasma/serum selenium and prostate cancer in a nonlinear dose-response meta-analysis showed that the risk decreased with increasing plasma/serum selenium up to 170 ng/mL. Three high-quality studies included in the meta-analysis of toenail selenium and cancer risk indicated a reduction in prostate cancer risk (estimated RR: 0.29; 95% CI: 0.14, 0.61) with a toenail selenium concentration between 0.85 and 0.94 μg/g. CONCLUSION The relation between selenium status and decreased prostate cancer risk was examined over a relatively narrow range of selenium status; further studies in low-selenium populations are required.

Height and Prostate Cancer Risk: A Large Nested Case-Control Study (ProtecT) and Meta-analysis

Background: Height, a marker of childhood environmental exposures, is positively associated with prostate cancer risk, perhaps through the insulin-like growth factor system. We investigated the relationship of prostate cancer with height and its components (leg and trunk length) in a nested case-control study and with height in a dose-response meta-analysis. Methods: We nested a case-control study within a population-based randomized controlled trial evaluating treatments for localized prostate cancer in British men ages 50 to 69 years, including 1,357 cases detected through prostate-specific antigen testing and 7,990 controls (matched on age, general practice, assessment date). Nine bibliographic databases were searched systematically for studies on the height-prostate cancer association that were pooled in a meta-analysis. Results: Based on the nested case-control, the odds ratio (OR) of prostate-specific antigen–detected prostate cancer per 10 cm increase in height was 1.06 [95% confidence interval (95% CI): 0.97-1.16; ptrend = 0.2]. There was stronger evidence of an association of height with high-grade prostate cancer (OR: 1.23; 95% CI: 1.06-1.43), mainly due to the leg component, but not with low-grade disease (OR: 0.99; 95% CI: 0.90-1.10). In general, associations with leg or trunk length were similar. A meta-analysis of 58 studies found evidence that height is positively associated with prostate cancer (random-effects OR per 10 cm: 1.06; 95% CI: 1.03-1.09), with a stronger effect for prospective studies of more advanced/aggressive cancers (random-effects OR: 1.12; 95% CI: 1.05-1.19). Conclusion: These data indicate a limited role for childhood environmental exposures—as indexed by adult height—on prostate cancer incidence, while suggesting a greater role for progression, through mechanisms requiring further investigation. (Cancer Epidemiol Biomarkers Prev 2008;17(9):2325–36)

Empirical evidence of study design biases in randomized trials: Systematic review of meta-epidemiological studies

Objective To synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials. Design Systematic review of meta-epidemiological studies. Methods We retrieved eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome (“mortality” versus “other objective” versus “subjective”). Direction of effect was standardised so that ROR < 1 and dSMD < 0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic. Results We included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95% CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95% CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95% CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95% CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95% CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear. Conclusions Certain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed.

Systematic review and critical appraisal of the impact of acellular dermal matrix use on the outcomes of implant-based breast reconstruction

Acellular dermal matrix (ADM) may improve outcomes in implant‐based breast reconstruction (IBBR). The aim of this study was critically to appraise and evaluate the current evidence for ADM‐assisted IBBR.

Oral anticoagulants for primary prevention, treatment and secondary prevention of venous thromboembolic disease, and for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis and cost-effectiveness analysis

BACKGROUND Warfarin is effective for stroke prevention in atrial fibrillation (AF), but anticoagulation is underused in clinical care. The risk of venous thromboembolic disease during hospitalisation can be reduced by low-molecular-weight heparin (LMWH): warfarin is the most frequently prescribed anticoagulant for treatment and secondary prevention of venous thromboembolism (VTE). Warfarin-related bleeding is a major reason for hospitalisation for adverse drug effects. Warfarin is cheap but therapeutic monitoring increases treatment costs. Novel oral anticoagulants (NOACs) have more rapid onset and offset of action than warfarin, and more predictable dosing requirements. OBJECTIVE To determine the best oral anticoagulant/s for prevention of stroke in AF and for primary prevention, treatment and secondary prevention of VTE. DESIGN Four systematic reviews, network meta-analyses (NMAs) and cost-effectiveness analyses (CEAs) of randomised controlled trials. SETTING Hospital (VTE primary prevention and acute treatment) and primary care/anticoagulation clinics (AF and VTE secondary prevention). PARTICIPANTS Patients eligible for anticoagulation with warfarin (stroke prevention in AF, acute treatment or secondary prevention of VTE) or LMWH (primary prevention of VTE). INTERVENTIONS NOACs, warfarin and LMWH, together with other interventions (antiplatelet therapy, placebo) evaluated in the evidence network. MAIN OUTCOME MEASURES Efficacy Stroke, symptomatic VTE, symptomatic deep-vein thrombosis and symptomatic pulmonary embolism. Safety Major bleeding, clinically relevant bleeding and intracranial haemorrhage. We also considered myocardial infarction and all-cause mortality and evaluated cost-effectiveness. DATA SOURCES MEDLINE and PREMEDLINE In-Process & Other Non-Indexed Citations, EMBASE and The Cochrane Library, reference lists of published NMAs and trial registries. We searched MEDLINE and PREMEDLINE In-Process & Other Non-Indexed Citations, EMBASE and The Cochrane Library. The stroke prevention in AF review search was run on the 12 March 2014 and updated on 15 September 2014, and covered the period 2010 to September 2014. The search for the three reviews in VTE was run on the 19 March 2014, updated on 15 September 2014, and covered the period 2008 to September 2014. REVIEW METHODS Two reviewers screened search results, extracted and checked data, and assessed risk of bias. For each outcome we conducted standard meta-analysis and NMA. We evaluated cost-effectiveness using discrete-time Markov models. RESULTS Apixaban (Eliquis®, Bristol-Myers Squibb, USA; Pfizer, USA) [5 mg bd (twice daily)] was ranked as among the best interventions for stroke prevention in AF, and had the highest expected net benefit. Edoxaban (Lixiana®, Daiichi Sankyo, Japan) [60 mg od (once daily)] was ranked second for major bleeding and all-cause mortality. Neither the clinical effectiveness analysis nor the CEA provided strong evidence that NOACs should replace postoperative LMWH in primary prevention of VTE. For acute treatment and secondary prevention of VTE, we found little evidence that NOACs offer an efficacy advantage over warfarin, but the risk of bleeding complications was lower for some NOACs than for warfarin. For a willingness-to-pay threshold of > £5000, apixaban (5 mg bd) had the highest expected net benefit for acute treatment of VTE. Aspirin or no pharmacotherapy were likely to be the most cost-effective interventions for secondary prevention of VTE: our results suggest that it is not cost-effective to prescribe NOACs or warfarin for this indication. CONCLUSIONS NOACs have advantages over warfarin in patients with AF, but we found no strong evidence that they should replace warfarin or LMWH in primary prevention, treatment or secondary prevention of VTE. LIMITATIONS These relate mainly to shortfalls in the primary data: in particular, there were no head-to-head comparisons between different NOAC drugs. FUTURE WORK Calculating the expected value of sample information to clarify whether or not it would be justifiable to fund one or more head-to-head trials. STUDY REGISTRATION This study is registered as PROSPERO CRD42013005324, CRD42013005331 and CRD42013005330. FUNDING The National Institute for Health Research Health Technology Assessment programme.

Symptomatic and quality‐of‐life outcomes after treatment for clinically localised prostate cancer: a systematic review

To conduct a systematic review of the risks of short‐term outcomes after major treatments for clinically localised prostate cancer. MEDLINE, EMBASE and the Cochrane Library were searched from 2004 to January 2013. Study arms that included ≥100 men with localised prostate cancer in receipt of surgery, radiotherapy or active surveillance and reported symptomatic and quality‐of‐life (QoL) data from 6 to 60 months after treatment were eligible. Data were extracted by one reviewer and checked by another. In all, 64 studies (80 treatment cohorts) were included. Most were single treatment cohorts from the USA or Europe. Radiotherapy was the most common treatment (40 cohorts, including 31 brachytherapy cohorts) followed by prostatectomy (39 cohorts), with only one active surveillance cohort. Most frequently measured symptoms were urinary, followed by sexual, and bowel; QoL was assessed in only 17 cohorts. Most studies used validated measures, although poor data reporting and differences between studies meant that it was not possible to pool data. Data on the precise impact of short‐term symptomatic and QoL outcomes after treatment for localised prostate cancer are of insufficient quality for clear guidance to men about the risks to these aspects of their lives. It is important that future studies focus on collecting core outcomes through validated measures and comply with reporting guidelines, so that clear and accurate information can be derived for men considering screening or treatment for prostate cancer.