Wojtek Wiercioch

Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise

Background: Although several tools to evaluate the credibility of health care guidelines exist, guidance on practical steps for developing guidelines is lacking. We systematically compiled a comprehensive checklist of items linked to relevant resources and tools that guideline developers could consider, without the expectation that every guideline would address each item. Methods: We searched data sources, including manuals of international guideline developers, literature on guidelines for guidelines (with a focus on methodology reports from international and national agencies, and professional societies) and recent articles providing systematic guidance. We reviewed these sources in duplicate, extracted items for the checklist using a sensitive approach and developed overarching topics relevant to guidelines. In an iterative process, we reviewed items for duplication and omissions and involved experts in guideline development for revisions and suggestions for items to be added. Results: We developed a checklist with 18 topics and 146 items and a webpage to facilitate its use by guideline developers. The topics and included items cover all stages of the guideline enterprise, from the planning and formulation of guidelines, to their implementation and evaluation. The final checklist includes links to training materials as well as resources with suggested methodology for applying the items. Interpretation: The checklist will serve as a resource for guideline developers. Consideration of items on the checklist will support the development, implementation and evaluation of guidelines. We will use crowdsourcing to revise the checklist and keep it up to date.

GRADE Guidelines : 16. GRADE evidence to decision frameworks for tests in clinical practice and public health

OBJECTIVES To describe the grading of recommendations assessment, development and evaluation (GRADE) interactive evidence to decision (EtD) frameworks for tests and test strategies for clinical, public health, or coverage decisions. STUDY DESIGN AND SETTING As part of the GRADE Working Groups DECIDE project, we conducted workshops, user testing with systematic review authors, guideline developers and other decision makers, and piloted versions of the EtD framework. RESULTS EtD frameworks for tests share the structure, explicitness, and transparency of other EtD frameworks. They require specifying the purpose of the test, linked or related management, and the key outcomes of concern for different test results and subsequent management. The EtD criteria address test accuracy and assessments of the certainty of the additional evidence necessary for decision making. When there is no direct evidence of test effects on patient-important outcomes, formal or informal modeling is needed to estimate effects. We describe the EtD criteria based on examples developed with GRADEpro (www.gradepro.org), GRADEs software that also allows development and dissemination of interactive summary of findings tables. CONCLUSION EtD frameworks for developing recommendations and making decisions about tests lay out the sequential steps in reviewing and assessing the different types of evidence that need to be linked.

Systematic reviews and meta-analyses of benefits and harms of cryotherapy, LEEP, and cold knife conization to treat cervical intraepithelial neoplasia

Cervical intraepithelial neoplasia (CIN) stage 2–3 is a premalignant lesion that can progress to cervical cancer in 10–20 years if untreated.

Systematic reviews and meta‐analyses of the accuracy of HPV tests, visual inspection with acetic acid, cytology, and colposcopy

Cervical cancer screening is offered to women to identify and treat cervical intraepithelial neoplasia (CIN).

Using patient values and preferences to inform the importance of health outcomes in practice guideline development following the GRADE approach

BackgroundThere are diverse opinions and confusion about defining and including patient values and preferences (i.e. the importance people place on the health outcomes) in the guideline development processes. This article aims to provide an overview of a process for systematically incorporating values and preferences in guideline development.MethodsIn 2013 and 2014, we followed the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to adopt, adapt and develop 226 recommendations in 22 guidelines for the Ministry of Health of the Kingdom of Saudi Arabia. To collect context-specific values and preferences for each recommendation, we performed systematic reviews, asked clinical experts to provide feedback according to their clinical experience, and consulted patient representatives.ResultsWe found several types of studies addressing the importance of outcomes, including those reporting utilities, non-utility measures of health states based on structured questionnaires or scales, and qualitative studies. Guideline panels used the relative importance of outcomes based on values and preferences to weigh the balance of desirable and undesirable consequences of alternative intervention options. However, we found few studies addressing local values and preferences.ConclusionsCurrently there are different but no firmly established processes for integrating patient values and preferences in healthcare decision-making of practice guideline development. With GRADE Evidence-to-Decision (EtD) frameworks, we provide an empirical strategy to find and incorporate values and preferences in guidelines by performing systematic reviews and eliciting information from guideline panel members and patient representatives. However, more research and practical guidance are needed on how to search for relevant studies and grey literature, assess the certainty of this evidence, and best summarize and present the findings.

Decision-Making about Healthcare Related Tests and Diagnostic Strategies: User Testing of GRADE Evidence Tables

Objective To develop guidance on what information to include and how to present it in tables summarizing the evidence from systematic reviews of test accuracy following the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. Methods To design and refine the evidence tables, we used an iterative process based on the analysis of data from four rounds of discussions, feedback and user testing. During the final round, we conducted one-on-one user testing with target end users. We presented a number of alternative formats of evidence tables to participants and obtained information about users’ understanding and preferences. Results More than 150 users participated in initial discussions and provided their formal and informal feedback. 20 users completed one-on-one user testing interviews. Almost all participants preferred summarizing the results of systematic reviews of test accuracy in tabular format rather than plain text. Users generally preferred less complex tables but found presenting sensitivity and specificity estimates only as too simplistic. Users found the presentation of test accuracy for several values of prevalence initially confusing but modifying table layout and adding sample clinical scenarios for each prevalence reduced this confusion. Providing information about clinical consequences of testing result was viewed as not feasible for authors of systematic reviews. Conclusion We present the current formats for tables presenting test accuracy following the GRADE approach. These tables can be developed using GRADEpro guidelines development tool (www.guidelinedevelopment.org or www.gradepro.org) and are being further developed into electronic interactive tables that will suit the needs of different end users. The formatting of these tables, and how they influence result interpretation and decision-making will be further evaluated in a randomized trial.

Development of rapid guidelines: 2. A qualitative study with WHO guideline developers

BackgroundSituations such as public health emergencies and outbreaks necessitate the development and publication of high-quality recommendations within a condensed timeframe. For example, WHO has produced examples of and guidance for the development of rapid guidelines (RGs). However, more information is needed to understand the experiences and perceptions of guideline developers. This is the second of a series of three articles addressing methodological issues around RGs. This study describes the perceptions and experiences of guideline developers at WHO about RGs.MethodsWe conducted interviews consisting of open- and closed-ended questions with guideline developers at WHO. Our analysis described the definition and rationale of RGs, the differences from regular guidelines with regard to timelines from topic definition until publication, barriers to identifying the evidence and the lack of a standard methodology to develop RGs.ResultsWe interviewed 10 participants, the majority of whom were comfortable with the current WHO definition of RGs. Most stated that the rationale for developing RGs should be in response to new evidence about efficacy, cost-effectiveness or safety. Respondents differed with regards to the amount of time RGs should take. While the majority of participants agreed that guidelines should be based on a systematic review, this step in the process was considered the most time and resource intensive. Challenges for developing RGs included limited personnel and financial resources as well as the lack of evidence. Facilitators, in turn, that may improve RG development include additional financial and personnel resources as well as the use of virtual meetings.ConclusionsWhile our study suggests a strong need and rationale for the development of RGs, standardisation of timelines and guidance on panel composition, peer-review process, conduct of meetings and sources of permissible evidence require further research.

A methodological survey identified eight proposed frameworks for the adaptation of health related guidelines

BACKGROUND AND OBJECTIVE Our objective was to identify and describe published frameworks for adaptation of clinical, public health, and health services guidelines. METHODS We included reports describing methods of adaptation of guidelines in sufficient detail to allow its reproducibility. We searched Medline and EMBASE databases. We also searched personal files, as well manuals and handbooks of organizations and professional societies that proposed methods of adaptation and adoption of guidelines. We followed standard systematic review methodology. RESULTS Our search captured 12,021 citations, out of which we identified eight proposed methods of guidelines adaptation: ADAPTE, Adapted ADAPTE, Alberta Ambassador Program adaptation phase, GRADE-ADOLOPMENT, MAGIC, RAPADAPTE, Royal College of Nursing (RCN), and Systematic Guideline Review (SGR). The ADAPTE framework consists of a 24-step process to adapt guidelines to a local context taking into consideration the needs, priorities, legislation, policies, and resources. The Alexandria Center for Evidence-Based Clinical Practice Guidelines updated one of ADAPTEs tools, modified three tools, and added three new ones. In addition, they proposed optionally using three other tools. The Alberta Ambassador Program adaptation phase consists of 11 steps and focused on adapting good-quality guidelines for nonspecific low back pain into local context. GRADE-ADOLOPMENT is an eight-step process based on the GRADE Working Groups Evidence to Decision frameworks and applied in 22 guidelines in the context of national guideline development program. The MAGIC research program developed a five-step adaptation process, informed by ADAPTE and the GRADE approach in the context of adapting thrombosis guidelines. The RAPADAPTE framework consists of 12 steps based on ADAPTE and using synthesized evidence databases, retrospectively derived from the experience of producing a high-quality guideline for the treatment of breast cancer with limited resources in Costa Rica. The RCN outlines five key steps strategy for adaptation of guidelines to the local context. The SGR method consists of nine steps and takes into consideration both methodological gaps and context-specific normative issues in source guidelines. We identified through searching personal files two abandoned methods. CONCLUSION We identified and described eight proposed frameworks for the adaptation of health-related guidelines. There is a need to evaluate these different frameworks to assess rigor, efficiency, and transparency of their proposed processes.

The GRADE evidence-to-decision framework: a report of its testing and application in 15 international guideline panels

BackgroundJudgments underlying guideline recommendations are seldom recorded and presented in a systematic fashion. The GRADE Evidence-to-Decision Framework (EtD) offers a transparent way to record and report guideline developers’ judgments. In this paper, we report the experiences with the EtD frameworks in 15 real guideline panels.MethodsFollowing the guideline panel meetings, we asked methodologists participating in the panel to provide feedback regarding the EtD framework. They were instructed to consider their own experience and the feedback collected from the rest of the panel. Two investigators independently summarized the responses and jointly interpreted the data using pre-specified domains as coding system. We asked methodologists to review the results and provide further input to improve the structure of the EtDs iteratively.ResultsThe EtD framework was well received, and the comments were generally positive. Methodologists felt that in a real guideline panel, the EtD framework helps structuring a complex process through relatively simple steps in an explicit and transparent way. However, some sections (e.g., “values and preferences” and “balance between benefits and harms”) required further development and clarification that were considered in the current version of the EtD framework.ConclusionsThe use of an EtD framework in guideline development offers a structured and explicit way to record and report the judgments and discussion of guideline panels during the formulation of recommendations. In addition, it facilitates the formulation of recommendations, assessment of their strength, and identifying gaps in research.

Development of rapid guidelines: 1. Systematic survey of current practices and methods

BackgroundGuidelines in the healthcare field generally should contain evidence-based recommendations to inform healthcare decisions. Guidelines often require 2 years or more to develop, but certain circumstances necessitate the development of rapid guidelines (RGs) in a short period of time. Upholding methodological rigor while meeting the reduced development timeframe presents a challenge for developing RGs. Our objective was to review current practices and standards for the development of RGs. This is the first of a series of three articles addressing methodological issues around RGs.MethodsWe conducted a systematic survey of methods manuals and published RGs to identify reasons for the development of RGs. Data sources included existing guideline manuals, published RGs, Trip Medical Database, MEDLINE, EMBASE and communication with guideline developers until February 2018.ResultsWe identified 46 guidelines that used a shortened timeframe for their development. Nomenclature describing RGs varied across organisations, wherein the United States Centers for Disease Control and Prevention produced ‘Interim Guidelines’, the National Institute for Health and Care Excellence in the United Kingdom developed ‘Short Clinical Guidelines’, and WHO provided ‘Rapid Advice’. The rationale for RGs included response to emergencies, rapid increases in cases of a condition or disease severity, or new evidence regarding treatment. In general, the methods to assess the quality of evidence, the consensus process and the management of the conflict of interest were not always clear. While we identified another 11 RGs from other institutions, there was no reference to timeframe and reasons for conducting a RG. The three organisations mentioned above provide guidance for the development of RGs.ConclusionsThere is a lack of standardised nomenclature and definitions regarding RGs and there is inconsistency in the methods described in manuals and in RG. It is therefore important that all RGs provide a detailed and transparent description of their methods in order for readers and end-users to be able to assess their quality and validate their findings.