Mohammed T Ansari

ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions

Non-randomised studies of the effects of interventions are critical to many areas of healthcare evaluation, but their results may be biased. It is therefore important to understand and appraise their strengths and weaknesses. We developed ROBINS-I (“Risk Of Bias In Non-randomised Studies - of Interventions”), a new tool for evaluating risk of bias in estimates of the comparative effectiveness (harm or benefit) of interventions from studies that did not use randomisation to allocate units (individuals or clusters of individuals) to comparison groups. The tool will be particularly useful to those undertaking systematic reviews that include non-randomised studies.

Testing the Newcastle Ottawa Scale showed low reliability between individual reviewers

OBJECTIVES To assess inter-rater reliability and validity of the Newcastle Ottawa Scale (NOS) used for methodological quality assessment of cohort studies included in systematic reviews. STUDY DESIGN AND SETTING Two reviewers independently applied the NOS to 131 cohort studies included in eight meta-analyses. Inter-rater reliability was calculated using kappa (κ) statistics. To assess validity, within each meta-analysis, we generated a ratio of pooled estimates for each quality domain. Using a random-effects model, the ratios of odds ratios for each meta-analysis were combined to give an overall estimate of differences in effect estimates. RESULTS Inter-rater reliability varied from substantial for length of follow-up (κ = 0.68, 95% confidence interval [CI] = 0.47, 0.89) to poor for selection of the nonexposed cohort and demonstration that the outcome was not present at the outset of the study (κ = -0.03, 95% CI = -0.06, 0.00; κ = -0.06, 95% CI = -0.20, 0.07). Reliability for overall score was fair (κ = 0.29, 95% CI = 0.10, 0.47). In general, reviewers found the tool difficult to use and the decision rules vague even with additional information provided as part of this study. We found no association between individual items or overall score and effect estimates. CONCLUSION Variable agreement and lack of evidence that the NOS can identify studies with biased results underscore the need for revisions and more detailed guidance for systematic reviewers using the NOS.

Grading quality of evidence and strength of recommendations: a perspective.

Evidence-based practice requires translating research findings into clinical and policy decision making. Clinical practice guidelines (CPGs) serve this purpose by evaluating evidence and making recommendations about therapeutic and diagnostic interventions and clinical management strategies. Systematic reviews are considered the best available evidence and are often used in the development of CPGs [1,2]. Since guideline development involves an assessment of the overall quality of evidence and complex balancing of trade-offs between the important benefits and harms of any given intervention, arbitrariness, value judgements, and subjectivity ultimately come into play in the guideline development process and associated recommendations [3]. In order to minimize cognitive bias in interpreting evidence and make the inherently subjective process more transparent and consistent, CPGs have traditionally employed formal systems or frameworks to understand and grade the quality of the body of evidence and strength of recommendations [4,5]. One such framework is the grading quality of evidence and strength of recommendations (GRADE), which is commonly used by guideline panels in deriving health care recommendations. GRADE was developed to overcome some of the deficiencies of earlier efforts [6]. GRADE defines the quality of evidence as the collective level of confidence guideline developers have about the validity of estimates of benefits and harms for any given intervention, and the strength of guideline recommendation as the extent of collective confidence that adherence to the recommendation will do more good than harm [7]. It urges guideline developers to consider all important patient outcomes of benefit and harm, to systematically evaluate the quality of their estimates, and to assess the trade-offs between evidence of benefits and harms, the preferences and values placed by patients on outcomes, the opportunity cost associated with the recommendation, and the feasibility of recommendations given a clinical setting before formulating guideline recommendations. Details of the GRADE approach have been published elsewhere [8]. In a new Policy Forum published in this issue of PLoS Medicine, Kavanagh [9] questions the external consistency of the GRADE framework by comparing the Surviving Sepsis Campaign (SSC) guideline recommendations developed in 2004 and updated in 2008. Moreover, Kavanagh expresses his concerns on the processes of the GRADE development and its formal validation. Had we likened the GRADE approach to an instrument or a health profile built on discrete logic to capture evidence, we would have concurred with some of Kavanagh’s criticism of GRADE. However, we see GRADE as a framework uncovering implicit subjectivity and invoking a systematic, explicit, judicious, and transparent approach to interpreting, as opposed to ‘‘capturing’’ evidence. It reveals how values are assigned to judgments, but what values are assigned it does not dictate simply because it cannot dictate. Below we first present our concern about one aspect of the GRADE framework and then our perspective on the various criticisms of it.

The GRADE Working Group clarifies the construct of certainty of evidence

OBJECTIVE To clarify the grading of recommendations assessment, development and evaluation (GRADE) definition of certainty of evidence and suggest possible approaches to rating certainty of the evidence for systematic reviews, health technology assessments, and guidelines. STUDY DESIGN AND SETTING This work was carried out by a project group within the GRADE Working Group, through brainstorming and iterative refinement of ideas, using input from workshops, presentations, and discussions at GRADE Working Group meetings to produce this document, which constitutes official GRADE guidance. RESULTS Certainty of evidence is best considered as the certainty that a true effect lies on one side of a specified threshold or within a chosen range. We define possible approaches for choosing threshold or range. For guidelines, what we call a fully contextualized approach requires simultaneously considering all critical outcomes and their relative value. Less-contextualized approaches, more appropriate for systematic reviews and health technology assessments, include using specified ranges of magnitude of effect, for example, ranges of what we might consider no effect, trivial, small, moderate, or large effects. CONCLUSION It is desirable for systematic review authors, guideline panelists, and health technology assessors to specify the threshold or ranges they are using when rating the certainty in evidence.

A comparative assessment of three formulations of botulinum toxin A for facial rhytides: a systematic review and meta-analyses

BackgroundBotulinum toxin A is a commonly used biological medication in the field of facial plastic surgery. Currently, there are three distinct formulations of botulinum toxin A, each with their purported benefits and advantages. However, there is considerable confusion as to the relative efficacy and side-effects associated with each formulation. Therefore, the purpose of this paper is to systematically assess published studies and perform a meta-analysis to determine if there is a significant advantage of any of the individual formulations.Methods/designA systematic literature search was performed for all relevant English language randomized controlled trials using Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL), MEDLINE, World Health Organization (WHO) International Clinical Trials Registry Platform, European Union (EU) Clinical Trials Register, Cochrane Library databases of clinical trials, and ClinicalTrials.gov. Inclusion criteria included any randomized controlled trial (RCT) that assessed the use of botulinum toxin for cosmetic purposes. The included articles were also analyzed for bias using the Cochrane Collaboration’s tool for assessing the risk of bias in RCTs.DiscussionThe results of this review will provide clinicians with an unbiased, high level of evidence of the comparative efficacy of individual preparations of botulinum toxin A.Systematic review registrationPROSPERO: CRD4201200337

Pre-analytic and analytic sources of variations in thiopurine methyltransferase activity measurement in patients prescribed thiopurine-based drugs: A systematic review☆

OBJECTIVES Low thiopurine S-methyltransferase (TPMT) enzyme activity is associated with increased thiopurine drug toxicity, particularly myelotoxicity. Pre-analytic and analytic variables for TPMT genotype and phenotype (enzyme activity) testing were reviewed. DESIGN AND METHODS A systematic literature review was performed, and diagnostic laboratories were surveyed. RESULTS Thirty-five studies reported relevant data for pre-analytic variables (patient age, gender, race, hematocrit, co-morbidity, co-administered drugs and specimen stability) and thirty-three for analytic variables (accuracy, reproducibility). TPMT is stable in blood when stored for up to 7 days at room temperature, and 3 months at -30°C. Pre-analytic patient variables do not affect TPMT activity. Fifteen drugs studied to date exerted no clinically significant effects in vivo. Enzymatic assay is the preferred technique. Radiochemical and HPLC techniques had intra- and inter-assay coefficients of variation (CVs) below 10%. CONCLUSION TPMT is a stable enzyme, and its assay is not affected by age, gender, race or co-morbidity.

Interactions of commonly used dietary supplements with cardiovascular drugs: a systematic review

BackgroundThe objective of this systematic review was to examine the benefits, harms and pharmacokinetic interactions arising from the co-administration of commonly used dietary supplements with cardiovascular drugs. Many patients on cardiovascular drugs take dietary supplements for presumed benefits and may be at risk for adverse supplement-drug interactions.MethodsThe Allied and Complementary Medicine Database, the Cochrane Library, EMBASE, International Bibliographic Information on Dietary Supplements and MEDLINE were searched from the inception of the review to October 2011. Grey literature was also reviewed.Two reviewers independently screened records to identify studies comparing a supplement plus cardiovascular drug(s) with the drug(s) alone. Reviewers extracted data using standardized forms, assessed the study risk of bias, graded the strength of evidence and reported applicability.ResultsEvidence was obtained from 65 randomized clinical trials, 2 controlled clinical trials and 1 observational study. With only a few small studies available per supplement, evidence was insufficient for all predefined gradable clinical efficacy and harms outcomes, such as mortality and serious adverse events. One long-term pragmatic trial showed no benefit from co-administering vitamin E with aspirin on a composite cardiovascular outcome. Evidence for most intermediate outcomes was insufficient or of low strength, suggesting no effect. Incremental benefits were noted for triglyceridemia with omega-3 fatty acid added to statins; and there was an improvement in levels of high-density lipoprotein cholesterol with garlic supplementation when people also consumed nitratesConclusionsEvidence of low-strength indicates benefits of omega-3 fatty acids (plus statin, or calcium channel blockers and antiplatelets) and garlic (plus nitrates or warfarin) on triglycerides and HDL-C, respectively. Safety concerns, however, persist.

Perioperative Diastolic Dysfunction in Patients Undergoing Noncardiac Surgery Is an Independent Risk Factor for Cardiovascular Events: A Systematic Review and Meta-analysis.

Background:The prognostic value of perioperative diastolic dysfunction (PDD) in patients undergoing noncardiac surgery remains uncertain, and the current guidelines do not recognize PDD as a perioperative risk factor. This systematic review aimed to investigate whether existing evidence supports PDD as an independent predictor of adverse events after noncardiac surgery. Methods:Ovid MEDLINE, PubMed, EMBASE, the Cochrane Library, and Google search engine were searched for English-language citations in April 2015 investigating PDD as a risk factor for perioperative adverse events in adult patients undergoing noncardiac surgery. Two reviewers independently assessed the study risk of bias. Extracted data were verified. Random-effects model was used for meta-analysis, and reviewers’ certainty was graded. Results:Seventeen studies met eligibility criteria; however, 13 contributed to evidence synthesis. The entire body of evidence addressing the research question was based on a total of 3,876 patients. PDD was significantly associated with pulmonary edema/congestive heart failure (odds ratio [OR], 3.90; 95% CI, 2.23 to 6.83; 3 studies; 996 patients), myocardial infarction (OR, 1.74; 95% CI, 1.14 to 2.67; 3 studies; 717 patients), and the composite outcome of major adverse cardiovascular events (OR, 2.03; 95% CI, 1.24 to 3.32; 4 studies; 1,814 patients). Evidence addressing other outcomes had low statistical power, but higher long-term cardiovascular mortality was observed in patients undergoing open vascular repair (OR, 3.00; 95% CI, 1.50 to 6.00). Reviewers’ overall certainty of the evidence was moderate. Conclusion:Evidence of moderate certainty indicates that PDD is an independent risk factor for adverse cardiovascular outcomes after noncardiac surgery.

Do sugar-sweetened beverages cause adverse health outcomes in children? A systematic review protocol

BackgroundChronic diseases, such as cardiovascular disease and type 2 diabetes, impose significant burden to public health. Most chronic diseases are associated with underlying preventable risk factors, such as elevated blood pressure, blood glucose, and lipids, physical inactivity, excessive sedentary behaviours, overweight and obesity, and tobacco usage. Sugar-sweetened beverages are known to be significant sources of additional caloric intake, and given recent attention to their contribution in the development of chronic diseases, a systematic review is warranted. We will assess whether the consumption of sugar-sweetened beverages in adults is associated with adverse health outcomes and what the potential moderating factors are.Methods/DesignOf interest are studies addressing sugar-sweetened beverage consumption, taking a broad perspective. Both direct consumption studies as well as those evaluating interventions that influence consumption (e.g. school policy, educational) will be relevant. Non-specific or multi-faceted behavioural, educational, or policy interventions may also be included subject to the level of evidence that exists for the other interventions/exposures. Comparisons of interest and endpoints of interest are pre-specified. We will include randomized controlled trials, controlled clinical trials, interrupted time series studies, controlled before-after studies, prospective and retrospective comparative cohort studies, case-control studies, and nested case-control designs. The MEDLINE®, Embase, The Cochrane Library, CINAHL, ERIC, and PsycINFO® databases and grey literature sources will be searched. The processes for selecting studies, abstracting data, and resolving conflicts are described. We will assess risk of bias using design-specific tools. To determine sets of confounding variables that should be adjusted for, we have developed causal directed acyclic graphs and will use those to inform our risk of bias assessments. Meta-analysis will be conducted where appropriate; parameters for exploring statistical heterogeneity and effect modifiers are pre-specified. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach will be used for determining the quality of evidence for outcomes.Systematic review registrationPROSPEROCRD42014009638

GRADE equity guidelines 3: considering health equity in GRADE guideline development: rating the certainty of synthesized evidence

Objectives The aim of this paper is to describe a conceptual framework for how to consider health equity in the Grading Recommendations Assessment and Development Evidence (GRADE) guideline development process. Study Design and Setting Consensus-based guidance developed by the GRADE working group members and other methodologists. Results We developed consensus-based guidance to help address health equity when rating the certainty of synthesized evidence (i.e., quality of evidence). When health inequity is determined to be a concern by stakeholders, we propose five methods for explicitly assessing health equity: (1) include health equity as an outcome; (2) consider patient-important outcomes relevant to health equity; (3) assess differences in the relative effect size of the treatment; (4) assess differences in baseline risk and the differing impacts on absolute effects; and (5) assess indirectness of evidence to disadvantaged populations and/or settings. Conclusion The most important priority for research on health inequity and guidelines is to identify and document examples where health equity has been considered explicitly in guidelines. Although there is a weak scientific evidence base for assessing health equity, this should not discourage the explicit consideration of how guidelines and recommendations affect the most vulnerable members of society.