Jörg J. Meerpohl

Grading quality of evidence and strength of recommendations in clinical practice guidelines: part 2 of 3. The GRADE approach to grading quality of evidence about diagnostic tests and strategies

The GRADE approach to grading the quality of evidence and strength of recommendations provides a comprehensive and transparent approach for developing clinical recommendations about using diagnostic tests or diagnostic strategies. Although grading the quality of evidence and strength of recommendations about using tests shares the logic of grading recommendations for treatment, it presents unique challenges. Guideline panels and clinicians should be alert to these special challenges when using the evidence about the accuracy of tests as the basis for clinical decisions. In the GRADE system, valid diagnostic accuracy studies can provide high quality evidence of test accuracy. However, such studies often provide only low quality evidence for the development of recommendations about diagnostic testing, as test accuracy is a surrogate for patient‐important outcomes at best. Inferring from data on accuracy that using a test improves outcomes that are important to patients requires availability of an effective treatment, improved patients’ wellbeing through prognostic information, or – by excluding an ominous diagnosis – reduction of anxiety and the opportunity for earlier search for an alternative diagnosis for which beneficial treatment can be available. Assessing the directness of evidence supporting the use of a diagnostic test requires judgments about the relationship between test results and patient‐important consequences. Well‐designed and conducted studies of allergy tests in parallel with efforts to evaluate allergy treatments critically will encourage improved guideline development for allergic diseases.

Grading quality of evidence and strength of recommendations in clinical practice guidelines Part 3 of 3. The GRADE approach to developing recommendations

To cite this article: Brożek JL, Akl EA, Compalati E, Kreis J, Terracciano L, Fiocchi A, Ueffing E, Andrews J, Alonso‐Coello P, Meerpohl JJ, Lang DM, Jaeschke R, Williams JW Jr, Phillips B, Lethaby A, Bossuyt P, Glasziou P, Helfand M, Watine J, Afilalo M, Welch V, Montedori A, Abraha I, Horvath AR, Bousquet J, Guyatt GH, Schünemann HJ, for the GRADE Working Group. Grading quality of evidence and strength of recommendations in clinical practice guidelines. Part 3 of 3. The GRADE approach to developing recommendations. Allergy 2011; 66: 588–595.

Learning from failure - rationale and design for a study about discontinuation of randomized trials (DISCO study)

BackgroundRandomized controlled trials (RCTs) may be discontinued because of apparent harm, benefit, or futility. Other RCTs are discontinued early because of insufficient recruitment. Trial discontinuation has ethical implications, because participants consent on the premise of contributing to new medical knowledge, Research Ethics Committees (RECs) spend considerable effort reviewing study protocols, and limited resources for conducting research are wasted. Currently, little is known regarding the frequency and characteristics of discontinued RCTs.Methods/DesignOur aims are, first, to determine the prevalence of RCT discontinuation for specific reasons; second, to determine whether the risk of RCT discontinuation for specific reasons differs between investigator- and industry-initiated RCTs; third, to identify risk factors for RCT discontinuation due to insufficient recruitment; fourth, to determine at what stage RCTs are discontinued; and fifth, to examine the publication history of discontinued RCTs.We are currently assembling a multicenter cohort of RCTs based on protocols approved between 2000 and 2002/3 by 6 RECs in Switzerland, Germany, and Canada. We are extracting data on RCT characteristics and planned recruitment for all included protocols. Completion and publication status is determined using information from correspondence between investigators and RECs, publications identified through literature searches, or by contacting the investigators. We will use multivariable regression models to identify risk factors for trial discontinuation due to insufficient recruitment. We aim to include over 1000 RCTs of which an anticipated 150 will have been discontinued due to insufficient recruitment.DiscussionOur study will provide insights into the prevalence and characteristics of RCTs that were discontinued. Effective recruitment strategies and the anticipation of problems are key issues in the planning and evaluation of trials by investigators, Clinical Trial Units, RECs and funding agencies. Identification and modification of barriers to successful study completion at an early stage could help to reduce the risk of trial discontinuation, save limited resources, and enable RCTs to better meet their ethical requirements.

Do Journals Publishing in the Field of Urology Endorse Reporting Guidelines? A Survey of Author Instructions

Introduction: Reporting guidelines aim to ensure adequate and complete reporting of clinical studies and are an indispensable tool to translate scientific results into clinical practice. The extent to which reporting guidelines are incorporated into the author instructions of journals publishing in the field of urology remained unclear. Materials and Methods: We assessed the author instructions of uro-nephrological journals indexed in ‘Journal Citation Reports 2009’. Two authors independently assessed the author guidelines. We evaluated additional information including whether a journal was published by or in association with a medical association. Discrepancies were resolved by re-checking the respective author instructions and by discussion with a third author. Results: The recommendations of the International Committee of Journal Editors were endorsed by 32 journals (58.2%) but were mentioned in 12 (37.5%) only to give general advice about manuscript preparation. Fourteen journals (25.5%) mentioned at least one reporting guideline, with CONSORT the most frequently cited. Journals with high impact factors were more likely to endorse CONSORT (p < 0.009). Other reporting guidelines were mentioned by <6% of the journals. Conclusion: All key stakeholders involved in the publication process should more frequently promote the awareness and use of reporting guidelines.

GRADE Guidelines: 18. How ROBINS-I and other tools to assess risk of bias in non-randomized studies should be used to rate the certainty of a body of evidence.

OBJECTIVE To provide guidance on how systematic review authors, guideline developers, and health technology assessment practitioners should approach the use of the risk of bias in nonrandomized studies of interventions (ROBINS-I) tool as a part of GRADEs certainty rating process. STUDY DESIGN AND SETTING The study design and setting comprised iterative discussions, testing in systematic reviews, and presentation at GRADE working group meetings with feedback from the GRADE working group. RESULTS We describe where to start the initial assessment of a body of evidence with the use of ROBINS-I and where one would anticipate the final rating would end up. The GRADE accounted for issues that mitigate concerns about confounding and selection bias by introducing the upgrading domains: large effects, dose-effect relations, and when plausible residual confounders or other biases increase certainty. They will need to be considered in an assessment of a body of evidence when using ROBINS-I. CONCLUSIONS The use of ROBINS-I in GRADE assessments may allow for a better comparison of evidence from randomized controlled trials (RCTs) and nonrandomized studies (NRSs) because they are placed on a common metric for risk of bias. Challenges remain, including appropriate presentation of evidence from RCTs and NRSs for decision-making and how to optimally integrate RCTs and NRSs in an evidence assessment.

Boosting qualifies capture-recapture methods for estimating the comprehensiveness of literature searches for systematic reviews.

OBJECTIVE Capture-recapture methods were proposed to evaluate the comprehensiveness of systematic literature searches. We investigate the statistical feasibility of capture-recapture techniques with model selection for estimating the number of missing references in literature searches using two systematic reviews in gastroenterology and hematology. STUDY DESIGN AND SETTING First, we compared manually selected Poisson regression models that differ with respect to included interactions. Secondly, we performed selection via componentwise boosting, which provides automatic variable selection. The proposed boosting technique is a regularized, stepwise procedure allowing to distinguish between mandatory and optional variables. Results from all models were compared based on Akaikes Information Criterion and the Bayesian Information Criterion. RESULTS For the first example, the best manually selected model suggested a number of 82 missing articles (95% CI: 52-128), whereas the boosting technique provided 127 (95% CI: 86-186) missing articles. For the second example, 140 (95% CI: 116-168) missing articles were estimated for the manually selected and 188 (95% CI: 159-223) for the automatically selected model. CONCLUSION Capture-recapture analysis requires the selection of an appropriate model. Because of problems of variable selection and overfitting, manual model selection yielded large estimates, varying markedly, with broad confidence intervals. By contrast, boosting was robust against overfitting and automatically created an appropriate model for inference.

Genetic variants in the noncoding region of RPS19 gene in Diamond-Blackfan anemia: Potential implications for phenotypic heterogeneity†

Mutations in the RPS19 gene have been identified in 25% of individuals affected by Diamond‐Blackfan anemia (DBA), a congenital erythroblastopenia characterized by an aregenerative anemia and a variety of malformations. More than 60 mutations in the five coding exons of RPS19 have been described to date. We previously reported a mutation (c.‐1 + 26G>T) and an insertion at −631 upstream of ATG (c.‐147_‐146insGCCA) in the noncoding region. Because DBA phenotype is extremely heterogeneous from silent to severe and because haploinsufficiency seems to play a role in this process, it is likely that genetic variations in the noncoding regions affecting translation of RPS19 can modulate the phenotypic expression of DBA. However, to date, very few studies have addressed this question comprehensively. In this study, we performed detailed sequence analysis of the RPS19 gene in 239 patients with DBA and 110 of their relatives. We found that 6.2% of the patients with DBA carried allelic variations upstream of ATG: 3.3% with c.‐1 + 26G>T; 2.5% with c.‐147_‐146insGCCA; and 0.4% with c.‐174G>A. Interestingly, the c.‐147_‐146insGCCA, which has been found in a black American and French Caribbean control population, was not found in 500 Caucasian control chromosomes we studied. However, it was found in association with the same haplotype distribution of four intronic polymorphisms in our patients with DBA. Although a polymorphism, the frequency of this variant in the patients with DBA and its association with the same haplotype raises the possibility that this polymorphism and the other genetic variations in the noncoding region could play a role in DBA pathogenesis. Am. J. Hematol., 2010.

Efficacy and safety of pharmacological agents in the treatment of erythema migrans in early Lyme borreliosis—systematic review protocol

BackgroundErythema migrans represents an early cutaneous and most common manifestation of Lyme borreliosis. Recommendations regarding pharmacological agents, dose and duration of treatment are subject of intense debate. This review aims to explore differences in efficacy and safety between pharmacological treatments and control treatment.MethodsTo identify relevant studies, we will conduct a systematic literature search. We will include randomised controlled trials (RCTs) and non-RCTs. Eligible comparative studies need to (1) consider patients with a diagnosis of erythema migrans resulting from Lyme borreliosis and (2) compare different pharmacological agents against each other, against any other non-pharmacological treatment, placebo or no treatment. Two review authors will independently assess included studies for risk of bias according to the methods of the Cochrane Handbook for Systematic Reviews of Interventions and related to specific study designs. We will address patient-relevant outcomes including clinical remission of cutaneous symptoms, any treatment-related adverse events, quality of life and progressive symptoms such as neuroborreliosis or Lyme carditis and flu-like symptoms. Provided that the identified trials are comparable in terms of clinical issues, combined estimates will be provided. Estimations of treatment effects will be calculated based on a random effects model. Heterogeneity will be evaluated based on I2 and chi-square test. In case of significant heterogeneity, a pooled estimate will not be provided, but heterogeneity will be investigated on the basis of methodological and clinical study aspects. We plan subgroup analysis to reveal potential differences in the effect estimates between patient populations and treatment specifications. We will consider risk of bias using sensitivity analyses to decide whether to rely on the pooled estimates. The quality of a body of evidence for individual outcomes will be assessed using the GRADE approach.DiscussionBenefits and harms of pharmacological treatment in erythema migrans have not yet been adequately assessed. This systematic review will evaluate and summarise available evidence addressing benefits and harms of different pharmacological treatments. In addition, this summary of clinical evidence will inform decision-making between clinicians and patients and will play an important part in patient care.Systematic review registrationPROSPERO: CRD42016037932

Leitlinien für Forschungsberichte sind auch für Leser medizinischer Fachartikel hilfreich

Over the last few years reporting guidelines for research papers have had increasing attention and use. They comprise recommendations and checklists, developed by expert groups consisting of researchers, methodologists and journal editors. They aim at ensuring the transparent description of research results. Following the CONSORT statement for the reporting of randomized controlled trials (RCTs), other reporting guidelines such as STARD for diagnostic accuracy studies, STROBE for observational studies and QUOROM for systematic reviews of RCTs have been published. This article discribes the concept of reporting guidelines for research reports and discusses searches of the international literature based on them. Selected reporting guidelines of particular interest are presented in detail. Nowadays, reporting guidelines exist for different areas of research and types of studies. The EQUATOR network (www.equator-network.org) offers a comprehensive overview. Other reporting guidelines are being prepared. It has been shown for some guidelines, in particular the CONSORT statement that they contribute to an improvement in reporting medical research. Existing guidelines should be assesed regularly for their timeliness and, if necessary, be updated. Reporting guidelines contribute to the improvement of the quality of medical research publications. They offer advice how to interpret and critically appraise the medical literature for authors but also for journal editors, reviewers and critical readers.

Bewertung des Risikos für Bias in kontrollierten Studien

BACKGROUND Practicing physicians are faced with many medical decisions daily. These are mainly influenced by personal experience but should also consider patient preferences and the scientific evidence reflected by a constantly increasing number of medical publications and guidelines. With the objective of optimal medical treatment, the concept of evidence-based medicine is founded on these three aspects. It should be considered that there is a high risk of misinterpreting evidence, leading to medical errors and adverse effects without knowledge of the methodological background. OBJECTIVES This article explains the concept of systematic error (bias) and its importance. Causes and effects as well as methods to minimize bias are discussed. This information should impart a deeper understanding, leading to a better assessment of studies and implementation of its recommendations in daily medical practice. CONCLUSION Developed by the Cochrane Collaboration, the risk of bias (RoB) tool is an assessment instrument for the potential of bias in controlled trials. Good handling, short processing time, high transparency of judgements and a graphical presentation of findings that is easily comprehensible are among its strengths. Attached to this article the German translation of the RoB tool is published. This should facilitate the applicability for non-experts and moreover, support evidence-based medical decision-making.