Romina Brignardello-Petersen

A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis

Network meta-analysis (NMA), combining direct and indirect comparisons, is increasingly being used to examine the comparative effectiveness of medical interventions. Minimal guidance exists on how to rate the quality of evidence supporting treatment effect estimates obtained from NMA. We present a four-step approach to rate the quality of evidence in each of the direct, indirect, and NMA estimates based on methods developed by the GRADE working group. Using an example of a published NMA, we show that the quality of evidence supporting NMA estimates varies from high to very low across comparisons, and that quality ratings given to a whole network are uninformative and likely to mislead.

Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise

Background: Although several tools to evaluate the credibility of health care guidelines exist, guidance on practical steps for developing guidelines is lacking. We systematically compiled a comprehensive checklist of items linked to relevant resources and tools that guideline developers could consider, without the expectation that every guideline would address each item. Methods: We searched data sources, including manuals of international guideline developers, literature on guidelines for guidelines (with a focus on methodology reports from international and national agencies, and professional societies) and recent articles providing systematic guidance. We reviewed these sources in duplicate, extracted items for the checklist using a sensitive approach and developed overarching topics relevant to guidelines. In an iterative process, we reviewed items for duplication and omissions and involved experts in guideline development for revisions and suggestions for items to be added. Results: We developed a checklist with 18 topics and 146 items and a webpage to facilitate its use by guideline developers. The topics and included items cover all stages of the guideline enterprise, from the planning and formulation of guidelines, to their implementation and evaluation. The final checklist includes links to training materials as well as resources with suggested methodology for applying the items. Interpretation: The checklist will serve as a resource for guideline developers. Consideration of items on the checklist will support the development, implementation and evaluation of guidelines. We will use crowdsourcing to revise the checklist and keep it up to date.

The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses

OBJECTIVE We evaluated the inter-rater reliability (IRR) of assessing the quality of evidence (QoE) using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach. STUDY DESIGN AND SETTING On completing two training exercises, participants worked independently as individual raters to assess the QoE of 16 outcomes. After recording their initial impression using a global rating, raters graded the QoE following the GRADE approach. Subsequently, randomly paired raters submitted a consensus rating. RESULTS The IRR without using the GRADE approach for two individual raters was 0.31 (95% confidence interval [95% CI] = 0.21-0.42) among Health Research Methodology students (n = 10) and 0.27 (95% CI = 0.19-0.37) among the GRADE working group members (n = 15). The corresponding IRR of the GRADE approach in assessing the QoE was significantly higher, that is, 0.66 (95% CI = 0.56-0.75) and 0.72 (95% CI = 0.61-0.79), respectively. The IRR further increased for three (0.80 [95% CI = 0.73-0.86] and 0.74 [95% CI = 0.65-0.81]) or four raters (0.84 [95% CI = 0.78-0.89] and 0.79 [95% CI = 0.71-0.85]). The IRR did not improve when QoE was assessed through a consensus rating. CONCLUSION Our findings suggest that trained individuals using the GRADE approach improves reliability in comparison to intuitive judgments about the QoE and that two individual raters can reliably assess the QoE using the GRADE system.

GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction

#### Summary points Healthcare decision making is complex. Decision-making processes and the factors (criteria) that decision makers should consider vary for different types of decisions, including clinical recommendations, coverage decisions, and health system or public health recommendations or decisions.1 2 3 4 However, some criteria are relevant for all of these decisions, including the anticipated effects of the options being considered, the certainty of the evidence for those effects (also referred to as quality of evidence or confidence in effect estimates), and the costs and feasibility of the options. Decision makers must make judgments about each relevant factor, informed by the best evidence that is available to them. Often, the processes that decision makers use, the criteria that they consider and the evidence that they …

GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines

#### Summary points Clinicians regularly face situations with two or more alternative actions. Each alternative often has different advantages and disadvantages, including differences in effectiveness, adverse effects, costs and other factors …

Matrix metalloproteinases and myeloperoxidase in gingival crevicular fluid provide site‐specific diagnostic value for chronic periodontitis

AIM To identify the diagnostic accuracy of gingival crevicular fluid (GCF) candidate biomarkers to discriminate periodontitis from the inflamed and healthy sites, and to compare the performance of two independent matrix metalloproteinase (MMP)-8 immunoassays. MATERIALS AND METHODS Cross sectional study. GCF (N = 58 sites) was collected from healthy, gingivitis and chronic periodontitis volunteers and analysed for levels of azurocidin, chemokine ligand 5, MPO, TIMP-1 MMP-13 and MMP-14 by ELISA or activity assays. MMP-8 was assayed by immunofluorometric assay (IFMA) and ELISA. Statistical analysis was performed using linear mixed-effects models and Bayesian statistics in R and Stata V11. RESULTS MMP-8, MPO, azurocidin and total MMP-13 and MMP-14 were higher in periodontitis compared to gingivitis and healthy sites (p < 0.05). A very high correlation between MPO and MMP-8 was evident in the periodontitis group (r = 0.95, p < 0.0001). MPO, azurocidin and total levels of MMP-8, MMP-13 and MMP-14 showed high diagnostic accuracy (≥0.90), but only MMP-8 and MPO were significantly higher in the periodontitis versus gingivitis sites. MMP-8 determined by IFMA correlated more strongly with periodontal status and showed higher diagnostic accuracy than ELISA. CONCLUSIONS MPO and collagenolytic MMPs are highly discriminatory biomarkers for site-specific diagnosis of periodontitis. The comparison of two quantitative MMP-8 methods demonstrated IFMA to be more accurate than ELISA.

Is Adjuvant Laser Therapy Effective for Preventing Pain, Swelling, and Trismus After Surgical Removal of Impacted Mandibular Third Molars? A Systematic Review and Meta-Analysis

PURPOSE To assess the efficacy and safety of low-level laser energy irradiation (LLEI) for decreasing pain, swelling, and trismus after surgical removal of impacted mandibular third molars (IMTMs). MATERIALS AND METHODS MEDLINE, EMBASE, and the Central Register of Controlled Trials of the Cochrane Library were searched from their inception, and conference proceedings, cross-references, and gray literature were searched for the last 5 years for randomized and quasi-randomized controlled trials that evaluated the effects of any type of LLEI, compared with active or inactive treatments, in patients undergoing surgical removal of IMTMs. Risk of bias in included studies was assessed by 2 independent evaluators using the Cochrane Risk of Bias tool. A random-effects model meta-analysis was used to estimate the mean difference of trismus between the groups. Heterogeneity was assessed using Cochran χ(2) and I(2). RESULTS Ten eligible trials were included in this systematic review. The included studies overall had a moderate risk of bias. Because of heterogeneity in the intervention and outcomes assessments, pain and swelling outcomes were only qualitatively summarized and indicated no beneficial effects of LLEI over placebo. Patients receiving LLEI had an average of 4.2 mm (95% confidence interval, 1.2 to 7.2) and 5.2 mm (95% confidence interval, 1.8 to 8.2) less trismus than patients receiving no active treatment on the second and seventh day after the surgery, respectively. CONCLUSIONS There was no benefit of LLEI on pain or swelling and a moderate benefit on trismus after removal of IMTMs. It is necessary to standardize the intervention and outcomes assessment and to conduct adequately powered, well-designed trials to evaluate the efficacy of LLEI.

GRADE Guidelines : 16. GRADE evidence to decision frameworks for tests in clinical practice and public health

OBJECTIVES To describe the grading of recommendations assessment, development and evaluation (GRADE) interactive evidence to decision (EtD) frameworks for tests and test strategies for clinical, public health, or coverage decisions. STUDY DESIGN AND SETTING As part of the GRADE Working Groups DECIDE project, we conducted workshops, user testing with systematic review authors, guideline developers and other decision makers, and piloted versions of the EtD framework. RESULTS EtD frameworks for tests share the structure, explicitness, and transparency of other EtD frameworks. They require specifying the purpose of the test, linked or related management, and the key outcomes of concern for different test results and subsequent management. The EtD criteria address test accuracy and assessments of the certainty of the additional evidence necessary for decision making. When there is no direct evidence of test effects on patient-important outcomes, formal or informal modeling is needed to estimate effects. We describe the EtD criteria based on examples developed with GRADEpro (www.gradepro.org), GRADEs software that also allows development and dissemination of interactive summary of findings tables. CONCLUSION EtD frameworks for developing recommendations and making decisions about tests lay out the sequential steps in reviewing and assessing the different types of evidence that need to be linked.

Arthroscopic surgery for degenerative knee arthritis and meniscal tears: a clinical practice guideline

#### What you need to know What is the role of arthroscopic surgery in degenerative knee disease? An expert panel produced these recommendations based on a linked systematic review triggered by a randomised trial published in The BMJ in June 2016, which found that, among patients with a degenerative medial meniscus tear, knee arthroscopy was no better than exercise therapy. The panel make a strong recommendation against arthroscopy for degenerative knee disease. Box 1 shows all of the articles and evidence linked in this Rapid Recommendation package. The infographic provides an overview of the absolute benefits and harms of arthroscopy in standard GRADE format. Table 2 below shows any evidence that has emerged since the publication of this article. #### Box 1: Linked articles in this BMJ Rapid Recommendations cluster

Oral Health Care for Patients with Epidermolysis Bullosa - Best Clinical Practice Guidelines

OBJECTIVE   To provide the users with information on the current best practices for managing the oral health care of people living with EB. METHODS   A systematic literature search, in which the main topic is dental care in patients with Epidermolysis Bullosa, was performed. Consulted sources, ranging from 1970 to 2010, included MEDLINE, EMBASE, CINAHL, The Cochrane Library, DARE, and the Cochrane controlled trials register (CENTRAL). In order to formulate the recommendations of the selected studies the SIGN system was used. The first draft was analysed and discussed by clinical experts, methodologists and patients representatives on a two days consensus meeting. The resulting document went through an external review process by a panel of experts, other health care professionals, patient representatives and lay reviewers. The final document was piloted in three different centres in United Kingdom, Czech Republic and Argentina. RESULTS   The guideline is composed of 93 recommendations divided into 3 main areas: 1) Oral Care--access issues, early referral, preventative strategies, management of microstomia, prescriptions and review appointments 2) Dental treatment: general treatment modifications, radiographs, restorations, endodontics, oral rehabilitation, periodontal treatment, oral surgery and orthodontics, and 3) Anaesthetic management of dental treatment. CONCLUSIONS   A preventive protocol is todays dental management approach of choice.