Rebecca L. Morgan

GRADE: Assessing the quality of evidence in environmental and occupational health

There is high demand in environmental health for adoption of a structured process that evaluates and integrates evidence while making decisions and recommendations transparent. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework holds promise to address this demand. For over a decade, GRADE has been applied successfully to areas of clinical medicine, public health, and health policy, but experience with GRADE in environmental and occupational health is just beginning. Environmental and occupational health questions focus on understanding whether an exposure is a potential health hazard or risk, assessing the exposure to understand the extent and magnitude of risk, and exploring interventions to mitigate exposure or risk. Although GRADE offers many advantages, including its flexibility and methodological rigor, there are features of the different sources of evidence used in environmental and occupational health that will require further consideration to assess the need for method refinement. An issue that requires particular attention is the evaluation and integration of evidence from human, animal, in vitro, and in silico (computer modeling) studies when determining whether an environmental factor represents a potential health hazard or risk. Assessment of the hazard of exposures can produce analyses for use in the GRADE evidence-to-decision (EtD) framework to inform risk-management decisions about removing harmful exposures or mitigating risks. The EtD framework allows for grading the strength of the recommendations based on judgments of the certainty in the evidence (also known as quality of the evidence), as well as other factors that inform recommendations such as social values and preferences, resource implications, and benefits. GRADE represents an untapped opportunity for environmental and occupational health to make evidence-based recommendations in a systematic and transparent manner. The objectives of this article are to provide an overview of GRADE, discuss GRADEs applicability to environmental health, and identify priority areas for method assessment and development.

GRADE Guidelines: 18. How ROBINS-I and other tools to assess risk of bias in non-randomized studies should be used to rate the certainty of a body of evidence.

OBJECTIVE To provide guidance on how systematic review authors, guideline developers, and health technology assessment practitioners should approach the use of the risk of bias in nonrandomized studies of interventions (ROBINS-I) tool as a part of GRADEs certainty rating process. STUDY DESIGN AND SETTING The study design and setting comprised iterative discussions, testing in systematic reviews, and presentation at GRADE working group meetings with feedback from the GRADE working group. RESULTS We describe where to start the initial assessment of a body of evidence with the use of ROBINS-I and where one would anticipate the final rating would end up. The GRADE accounted for issues that mitigate concerns about confounding and selection bias by introducing the upgrading domains: large effects, dose-effect relations, and when plausible residual confounders or other biases increase certainty. They will need to be considered in an assessment of a body of evidence when using ROBINS-I. CONCLUSIONS The use of ROBINS-I in GRADE assessments may allow for a better comparison of evidence from randomized controlled trials (RCTs) and nonrandomized studies (NRSs) because they are placed on a common metric for risk of bias. Challenges remain, including appropriate presentation of evidence from RCTs and NRSs for decision-making and how to optimally integrate RCTs and NRSs in an evidence assessment.

Using patient values and preferences to inform the importance of health outcomes in practice guideline development following the GRADE approach

BackgroundThere are diverse opinions and confusion about defining and including patient values and preferences (i.e. the importance people place on the health outcomes) in the guideline development processes. This article aims to provide an overview of a process for systematically incorporating values and preferences in guideline development.MethodsIn 2013 and 2014, we followed the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to adopt, adapt and develop 226 recommendations in 22 guidelines for the Ministry of Health of the Kingdom of Saudi Arabia. To collect context-specific values and preferences for each recommendation, we performed systematic reviews, asked clinical experts to provide feedback according to their clinical experience, and consulted patient representatives.ResultsWe found several types of studies addressing the importance of outcomes, including those reporting utilities, non-utility measures of health states based on structured questionnaires or scales, and qualitative studies. Guideline panels used the relative importance of outcomes based on values and preferences to weigh the balance of desirable and undesirable consequences of alternative intervention options. However, we found few studies addressing local values and preferences.ConclusionsCurrently there are different but no firmly established processes for integrating patient values and preferences in healthcare decision-making of practice guideline development. With GRADE Evidence-to-Decision (EtD) frameworks, we provide an empirical strategy to find and incorporate values and preferences in guidelines by performing systematic reviews and eliciting information from guideline panel members and patient representatives. However, more research and practical guidance are needed on how to search for relevant studies and grey literature, assess the certainty of this evidence, and best summarize and present the findings.

Enrollment in HIV Care Two Years after HIV Diagnosis in the Kingdom of Swaziland: An Evaluation of a National Program of New Linkage Procedures

To improve early enrollment in HIV care, the Swaziland Ministry of Health implemented new linkage procedures for persons HIV diagnosed during the Soka Uncobe male circumcision campaign (SOKA, 2011–2012) and the Swaziland HIV Incidence Measurement Survey (SHIMS, 2011). Abstraction of clinical records and telephone interviews of a retrospective cohort of HIV-diagnosed SOKA and SHIMS clients were conducted in 2013–2014 to evaluate compliance with new linkage procedures and enrollment in HIV care at 92 facilities throughout Swaziland. Of 1,105 clients evaluated, within 3, 12, and 24 months of diagnosis, an estimated 14.0%, 24.3%, and 37.0% enrolled in HIV care, respectively, after adjusting for lost to follow-up and non-response. Kaplan-Meier functions indicated lower enrollment probability among clients 14–24 (P = 0.0001) and 25–29 (P = 0.001) years of age compared with clients >35 years of age. At 69 facilities to which clients were referred for HIV care, compliance with new linkage procedures was low: referral forms were located for less than half (46.8%) of the clients, and few (9.6%) were recorded in the appointment register or called either before (0.3%) or after (4.9%) their appointment. Of over one thousand clients newly HIV diagnosed in Swaziland in 2011 and 2012, few received linkage services in accordance with national procedures and most had not enrolled in HIV care two years after their diagnosis. Our findings are a call to action to improve linkage services and early enrollment in HIV care in Swaziland.

Cystic fibrosis foundation pulmonary guidelines: Use of cystic fibrosis transmembrane conductance regulator modulator therapy in patients with cystic fibrosis

Rationale: Cystic fibrosis (CF) transmembrane conductance regulator (CFTR) modulators are a new class of medications targeting the underlying defect in CF. Ivacaftor (IVA) and IVA combined with lumacaftor (LUM; IVA/LUM) have been approved by the U.S. Food and Drug Administration (FDA) for use in patients with CF. However, the FDA label for these medications encompasses patient groups that were not studied as part of the drug approval process. CF clinicians, patients, and their families have recognized a need for recommendations to guide the use of these medications. Objective: Develop evidence‐based guidelines for CFTR modulator therapy in patients with CF. Methods: A multidisciplinary committee of CF caregivers and patient representatives was assembled. A methodologist, an epidemiologist, a medical librarian, and a biostatistician were recruited to assist with the literature search, evidence grading, and generation of recommendations. The committee developed clinical questions using the Patient‐Intervention‐Comparison‐Outcome format. A systematic review was conducted to find relevant publications. The evidence was then evaluated using the GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) approach, and recommendations were made based on this analysis. Results: For adults and children aged 6 years and older with CF due to gating mutations other than G551D or R117H, the guideline panel made a conditional recommendation for treatment with IVA. For those with the R117H mutation, the guideline panel made a conditional recommendation for treatment with IVA for 1) adults aged 18 years or older, and 2) children aged 6‐17 years with a forced expiratory volume in 1 second (FEV1) less than 90% predicted. For those with the R117H mutation, the guideline panel made a conditional recommendation against treatment with IVA for 1) children aged 12‐17 years with an FEV1 greater than 90% predicted, and 2) children less than 6 years of age. Among those with two copies of F508del, the guideline panel made a strong recommendation for treatment with IVA/LUM for adults and children aged 12 years and older with an FEV1 less than 90% predicted; and made a conditional recommendation for treatment with IVA/LUM for 1) adults and children aged 12 years or older with an FEV1 greater than 90% predicted, and 2) children aged 6‐11 years. Conclusions: Using the GRADE approach, we have made recommendations for the use of CFTR modulators in patients with CF. These recommendations will be of help to CF clinicians, patients, and their families in guiding decisions regarding use of these medications.

Stakeholders apply the GRADE evidence-to-decision framework to facilitate coverage decisions

OBJECTIVES Coverage decisions are complex and require the consideration of many factors. A well-defined, transparent process could improve decision-making and facilitate decision-maker accountability. STUDY DESIGN AND SETTING We surveyed key US-based stakeholders regarding their current approaches for coverage decisions. Then, we held a workshop to test an evidence-to-decision (EtD) framework for coverage based on the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) criteria. RESULTS A total of 42 individuals (including 19 US stakeholders as well as international health policymakers and GRADE working group members) attended the workshop. Of the 19 stakeholders, 14 (74%) completed the survey before the workshop. Almost all of their organizations (13 of 14; 93%) used systematic reviews for coverage decision-making; few (2 of 14; 14%) developed their own evidence synthesis; a majority (9 of 14; 64%) rated the certainty of evidence (using various systems); almost all (13 of 14; 93%) denied formal consideration of resource use; and half (7 of 14; 50%) reported explicit criteria for decision-making. At the workshop, stakeholders successfully applied the EtD framework to four case studies and provided narrative feedback, which centered on contextual factors affecting coverage decisions in the United States, the need for reliable data on subgroups of patients, and the challenge of decision-making without formal consideration of resource use. CONCLUSION Stakeholders successfully applied the EtD framework to four case studies and highlighted contextual factors affecting coverage decisions and affirmed its value. Their input informed the further development of a revised EtD framework, now publicly available (http://gradepro.org/).

Development of rapid guidelines: 2. A qualitative study with WHO guideline developers

BackgroundSituations such as public health emergencies and outbreaks necessitate the development and publication of high-quality recommendations within a condensed timeframe. For example, WHO has produced examples of and guidance for the development of rapid guidelines (RGs). However, more information is needed to understand the experiences and perceptions of guideline developers. This is the second of a series of three articles addressing methodological issues around RGs. This study describes the perceptions and experiences of guideline developers at WHO about RGs.MethodsWe conducted interviews consisting of open- and closed-ended questions with guideline developers at WHO. Our analysis described the definition and rationale of RGs, the differences from regular guidelines with regard to timelines from topic definition until publication, barriers to identifying the evidence and the lack of a standard methodology to develop RGs.ResultsWe interviewed 10 participants, the majority of whom were comfortable with the current WHO definition of RGs. Most stated that the rationale for developing RGs should be in response to new evidence about efficacy, cost-effectiveness or safety. Respondents differed with regards to the amount of time RGs should take. While the majority of participants agreed that guidelines should be based on a systematic review, this step in the process was considered the most time and resource intensive. Challenges for developing RGs included limited personnel and financial resources as well as the lack of evidence. Facilitators, in turn, that may improve RG development include additional financial and personnel resources as well as the use of virtual meetings.ConclusionsWhile our study suggests a strong need and rationale for the development of RGs, standardisation of timelines and guidance on panel composition, peer-review process, conduct of meetings and sources of permissible evidence require further research.

National Guidelines for Colorectal Cancer Screening in Saudi Arabia with strength of recommendations and quality of evidence.

BACKGROUND Colorectal cancer is the most common cancer among Saudi men and the third commonest among Saudi women. Given the predominance of colorectal cancer compared with other cancers in Saudi Arabia, context-specific guidelines are needed for screening. METHODS The Saudi Centre for Evidence-Based Healthcare assembled a panel of experts from the Saudi Society of Colon and Rectal Surgery, Saudi Gastroenterology Association, the Saudi Oncology Society, the Saudi Chapter of Enterostomal Therapy, the Family Medicine and Department of Public Health at the Saudi Arabian Ministry of Health and a patient advocate. The panel collaborated with a methodological team from McMaster University, Canada to develop national guidelines for colorectal cancer screening. After identifying key questions, the panel conducted a systematic review of all reports on the utility of screening, the cost of screening for colorectal cancer in Saudi Arabia and on the values and preferences of Saudi patients. Meta-analyses, when appropriate, were performed to generate pooled estimates of effect. Using the GRADE approach, the panel used the evidence-to-decision (EtD) framework to assess all domains important in determining the strength and direction of the recommendations (benefits and harms, values and preferences, resource implications, equity, acceptability, and feasibility). Judgments related to the EtD domains were resolved through consensus or voting, if consensus was not reached. The final recommendations were developed during a two-day meeting held in Riyadh, Saudi Arabia in March 2015. Conflicts of interests among the panel members were handled according to the World Health Organization rules. LIMITATIONS There is lack of national data on the incidence of adenomatous polyps or the age groups in which the incidence surges. There were no national clinical trials assessing the effectiveness of the different modalities of screening for colorectal cancer and their impact on mortality. CONCLUSION The panel recommends screening for colorectal cancer in Saudi Arabia in asymptomatic Saudi patients at average risk of colorectal cancer. An infrastructure should be built to achieve that goal.

Development of rapid guidelines: 3. GIN-McMaster Guideline Development Checklist extension for rapid recommendations

BackgroundPractice guidelines require a substantial investment of resources and time, often taking between 1 and 3 years from conceptualisation to publication. However, urgent situations require the development of recommendations in a shorter timeframe. In this third and final article in the series exploring challenges and solutions in developing rapid guidelines (RGs), we propose guiding principles for the development of RGs.MethodsWe utilised the Guideline International Network-McMaster Guideline Development Checklist (GDC) as a starting point for elements to consider during RG development. We built on those elements using the findings from a systematic review of guideline manuals, a survey of international organisations conducting RGs, and interviews of guideline developers within WHO. We reviewed initial findings and developed an intermediate list of elements, as well as narrative guidance. We then invited experts to validate the intermediate list, reviewing for placement, brevity and redundancy. We used this iterative process and group consensus to determine the final elements for RG development guidance.ResultsOur work identified 21 principles within the topics of the Guideline International Network-McMaster GDC to guide the planning and development of RGs. Principles fell within 15 of the 18 checklist topics, highlighting strategies to streamline and expedite the guideline development process.ConclusionsWe defined principles to guide the development of RGs, while maintaining a standardised, rigorous and transparent process. These principles will serve as guidance for guideline developers responding to urgent situations such as public health urgencies. Integration of these principles within currently disseminated guideline development standards will facilitate the use of those tools in situations necessitating RG recommendations.

Development of rapid guidelines: 1. Systematic survey of current practices and methods

BackgroundGuidelines in the healthcare field generally should contain evidence-based recommendations to inform healthcare decisions. Guidelines often require 2 years or more to develop, but certain circumstances necessitate the development of rapid guidelines (RGs) in a short period of time. Upholding methodological rigor while meeting the reduced development timeframe presents a challenge for developing RGs. Our objective was to review current practices and standards for the development of RGs. This is the first of a series of three articles addressing methodological issues around RGs.MethodsWe conducted a systematic survey of methods manuals and published RGs to identify reasons for the development of RGs. Data sources included existing guideline manuals, published RGs, Trip Medical Database, MEDLINE, EMBASE and communication with guideline developers until February 2018.ResultsWe identified 46 guidelines that used a shortened timeframe for their development. Nomenclature describing RGs varied across organisations, wherein the United States Centers for Disease Control and Prevention produced ‘Interim Guidelines’, the National Institute for Health and Care Excellence in the United Kingdom developed ‘Short Clinical Guidelines’, and WHO provided ‘Rapid Advice’. The rationale for RGs included response to emergencies, rapid increases in cases of a condition or disease severity, or new evidence regarding treatment. In general, the methods to assess the quality of evidence, the consensus process and the management of the conflict of interest were not always clear. While we identified another 11 RGs from other institutions, there was no reference to timeframe and reasons for conducting a RG. The three organisations mentioned above provide guidance for the development of RGs.ConclusionsThere is a lack of standardised nomenclature and definitions regarding RGs and there is inconsistency in the methods described in manuals and in RG. It is therefore important that all RGs provide a detailed and transparent description of their methods in order for readers and end-users to be able to assess their quality and validate their findings.