Jako S. Burgers

AGREE II: advancing guideline development, reporting and evaluation in health care

Clinical practice guidelines, which are systematically developed statements aimed at helping people make clinical, policy-related and system-related decisions,[1][1],[2][2] frequently vary widely in quality.[3][3],[4][4] A strategy was needed to differentiate among guidelines and ensure that those

Why don't physicians adhere to guideline recommendations in practice? An analysis of barriers among Dutch general practitioners.

BackgroundDespite wide distribution and promotion of clinical practice guidelines, adherence among Dutch general practitioners (GPs) is not optimal. To improve adherence to guidelines, an analysis of barriers to implementation is advocated. Because different recommendations within a guideline can have different barriers, in this study we focus on key recommendations rather than guidelines as a whole, and explore the barriers to implementation perceived by Dutch GPs.MethodsA qualitative study using six focus groups was conducted, in which 30 GPs participated, with an average of seven per session. Fifty-six key recommendations were derived from twelve national guidelines. In each focus group, barriers to the implementation of the key recommendations of two clinical practice guidelines were discussed. Focus group discussions were audiotaped and transcribed verbatim. Data was analysed by using an existing framework of barriers.ResultsThe barriers varied largely within guidelines, with each key recommendation having a unique pattern of barriers. The most perceived barriers were lack of agreement with the recommendations due to lack of applicability or lack of evidence (68% of key recommendations), environmental factors such as organisational constraints (52%), lack of knowledge regarding the guideline recommendations (46%), and guideline factors such as unclear or ambiguous guideline recommendations (43%).ConclusionOur study findings suggest a broad range of barriers. As the barriers largely differ within guidelines, tailored and barrier-driven implementation strategies focusing on key recommendations are needed to improve adherence in practice. In addition, guidelines should be more transparent concerning the underlying evidence and applicability, and further efforts are needed to address complex issues such as comorbidity in guidelines. Finally, it might be useful to include focus groups in continuing medical education as an innovative medium for guideline education and implementation.

Development of the AGREE II, part 1: performance, usefulness and areas for improvement

Background: We undertook research to improve the AGREE instrument, a tool used to evaluate guidelines. We tested a new seven-point scale, evaluated the usefulness of the original items in the instrument, investigated evidence to support shorter, tailored versions of the tool, and identified areas for improvement. Method: We report on one component of a larger study that used a mixed design with four factors (user type, clinical topic, guideline and condition). For the analysis reported in this article, we asked participants to read a guideline and use the AGREE items to evaluate it based on a seven-point scale, to complete three outcome measures related to adoption of the guideline, and to provide feedback on the instrument’s usefulness and how to improve it. Results: Guideline developers gave lower-quality ratings than did clinicians or policy-makers. Five of six domains were significant predictors of participants’ outcome measures (p < 0.05). All domains and items were rated as useful by stakeholders (mean scores > 4.0) with no significant differences by user type (p > 0.05). Internal consistency ranged between 0.64 and 0.89. Inter-rater reliability was satisfactory. We received feedback on how to improve the instrument. Interpretation: Quality ratings of the AGREE domains were significant predictors of outcome measures associated with guideline adoption: guideline endorsements, overall intentions to use guidelines, and overall quality of guidelines. All AGREE items were assessed as useful in determining whether a participant would use a guideline. No clusters of items were found more useful by some users than others. The measurement properties of the seven-point scale were promising. These data contributed to the refinements and release of the AGREE II.

The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies

Background Despite the increasing number of manuals on how to develop clinical practice guidelines (CPGs) there remain concerns about their quality. The aim of this study was to review the quality of CPGs across a wide range of healthcare topics published since 1980. Methods The authors conducted a literature search in MEDLINE to identify publications assessing the quality of CPGs with the Appraisal of Guidelines, Research and Evaluation (AGREE) instrument. For the included guidelines in each study, the authors gathered data about the year of publication, institution, country, healthcare topic, AGREE score per domain and overall assessment. Results In total, 42 reviews were selected, including a total of 626 guidelines, published between 1980 and 2007, with a median of 25 CPGs. The mean scores were acceptable for the domain ‘Scope and purpose’ (64%; 95% CI 61.9 to 66.4) and ‘Clarity and presentation’ (60%; 95% CI 57.9 to 61.9), moderate for domain ‘Rigour of development’ (43%; 95% CI 41.0 to 45.2), and low for the other domains (‘Stakeholder involvement’ 35%; 95% CI 33.9 to 37.5, ‘Editorial independence’ 30%; 95% CI 27.9 to 32.3, and ‘Applicability’ 22%; 95% CI 20.4 to 23.9). From those guidelines that included an overall assessment, 62% (168/270) were recommended or recommended with provisos. There was a significant improvement over time for all domains, except for ‘Editorial independence.’ Conclusions This review shows that despite some increase in quality of CPGs over time, the quality scores as measured with the AGREE Instrument have remained moderate to low over the last two decades. This finding urges guideline developers to continue improving the quality of their products. International collaboration could help increasing the efficiency of the process.

Effects of evidence-based clinical practice guidelines on quality of care: a systematic review

Background: Evidence-based clinical guidelines aim to improve the quality of care. In The Netherlands, considerable time and effort have been invested in the development and implementation of evidence-based guidelines since the 1990s. Thus far, no reviews are available on their effectiveness. The primary aim of this article was to assess the evidence for the effectiveness of Dutch evidence-based clinical guidelines in improving the quality of care. Methods: A systematic review of studies evaluating the effects of Dutch evidence-based guidelines on both the process and structure of care and patient outcomes was conducted. The electronic databases Medline and Embase (1990–2007) and relevant scientific journals were searched. Studies were only selected if they included a controlled trial, an interrupted time series design or a before and after design. Results: A total of 20 studies were included. In 17 of 19 studies that measured the effects on the process or structure of care, significant improvements were reported. Thirteen of these studies reported improvement with respect to some of the recommendations studied. In addition, the size of the observed effects varied largely across the recommendations within guidelines. Six of nine studies that measured patient health outcomes showed significant but small improvements as a result of the use of clinical guidelines. Conclusions: This review demonstrates that Dutch evidence-based clinical guidelines can be effective in improving the process and structure of care. The effects of guidelines on patient health outcomes were studied far less and data are less convincing. The high level of variation in effects across recommendations suggests that implementation strategies tailored to individual recommendations within the guideline are needed to establish relevant improvements in healthcare. Moreover, the results highlight the need for well-designed studies focusing on the level of the recommendations to determine which factors influence guideline utilisation and improved patient outcomes.

Development of the AGREE II, part 2: assessment of validity of items and tools to support application.

Background: We established a program of research to improve the development, reporting and evaluation of practice guidelines. We assessed the construct validity of the items and user’s manual in the β version of the AGREE II. Methods: We designed guideline excerpts reflecting high-and low-quality guideline content for 21 of the 23 items in the tool. We designed two study packages so that one low-quality and one high-quality version of each item were randomly assigned to each package. We randomly assigned 30 participants to one of the two packages. Participants reviewed and rated the guideline content according to the instructions of the user’s manual and completed a survey assessing the manual. Results: In all cases, content designed to be of high quality was rated higher than low-quality content; in 18 of 21 cases, the differences were significant (p < 0.05). The manual was rated by participants as appropriate, easy to use, and helpful in differentiating guidelines of varying quality, with all scores above the mid-point of the seven-point scale. Considerable feedback was offered on how the items and manual of the β-AGREE II could be improved. Interpretation: The validity of the items was established and the user’s manual was rated as highly useful by users. We used these results and those of our study presented in part 1 to modify the items and user’s manual. We recommend AGREE II (available at www.agreetrust.org) as the revised standard for guideline development, reporting and evaluation.

Improving the quality of health care: using international collaboration to inform guideline programmes by founding the Guidelines International Network (G-I-N)

Clinical practice guidelines are regarded as powerful tools to achieve effective health care. Although many countries have built up experience in the development, appraisal, and implementation of guidelines, until recently there has been no established forum for collaboration at an international level. As a result, in different countries seeking similar goals and using similar strategies, efforts have been unnecessarily duplicated and opportunities for harmonisation lost because of the lack of a supporting organisational framework. This triggered a proposal in 2001 for an international guidelines network built on existing partnerships. A baseline survey confirmed a strong demand for such an entity. A multinational group of guideline experts initiated the development of a non-profit organisation aimed at promotion of systematic guideline development and implementation. The Guidelines International Network (G-I-N) was founded in November 2002. One year later the Network released the International Guideline Library, a searchable database which now contains more than 2000 guideline resources including published guidelines, guidelines under development, “guidelines for guidelines”, training materials, and patient information tools. By June 2004, 52 organisations from 27 countries had joined the network including institutions from Oceania, North America, and Europe, and WHO. This paper describes the process that led to the foundation of the G-I-N, its characteristics, prime activities, and ideas on future projects and collaboration.

Defining and Measuring Integrated Patient Care: Promoting the Next Frontier in Health Care Delivery

Integration of care is emerging as a central challenge of health care delivery, particularly for patients with multiple, complex chronic conditions. The authors argue that the concept of “integrated patient care” would benefit from further clarification regarding (a) the object of integration and (b) its essential components, particularly when constructing measures.To address these issues, the authors propose a definition of integrated patient care that distinguishes it from integrated delivery organizations, acknowledging that integrated organizational structures and processes may fail to produce integrated patient care. The definition emphasizes patients’ central role as active participants in managing their own health by including patient centeredness as a key element of integrated patient care. Measures based on the proposed definition will enable empirical assessment of the potential relationships between the integration of organizations, the integration of patient care, and patient outcomes, providing valuable guidance to health systems reformers.

Characteristics of high-quality guidelines: Evaluation of 86 clinical guidelines developed in ten european countries and Canada

OBJECTIVES To identify predictors of high-quality clinical practice guidelines. METHODS A total of 86 guidelines from 11 countries were assessed by four independent appraisers per guideline using the AGREE instrument (23 items). Six aspects of guideline development were considered to explain the variation in quality scores: care level (primary/secondary care), scope (diagnosis/treatment), type of guideline (new/update), year of publication, type of agency (governmental/professional), and whether the guideline was produced within a structured and coordinated program. RESULTS Guidelines produced within a guideline program and by governmental agencies had higher scores than their counterparts. Differences in the applicability of the guidelines could not be explained by the variables studied. CONCLUSION To ensure high quality, clinical guidelines should be produced within a structured and coordinated program. Professional organizations or specialist societies that aim to develop guidelines may adopt quality criteria from leading guideline agencies.

Current Guidelines Have Limited Applicability to Patients with Comorbid Conditions: A Systematic Analysis of Evidence-Based Guidelines

Background Guidelines traditionally focus on the diagnosis and treatment of single diseases. As almost half of the patients with a chronic disease have more than one disease, the applicability of guidelines may be limited. The aim of this study was to assess the extent that guidelines address comorbidity and to assess the supporting evidence of recommendations related to comorbidity. Methodology/Principal Findings We conducted a systematic analysis of evidence-based guidelines focusing on four highly prevalent chronic conditions with a high impact on quality of life: chronic obstructive pulmonary disease, depressive disorder, diabetes mellitus type 2, and osteoarthritis. Data were abstracted from each guideline on the extent that comorbidity was addressed (general comments, specific recommendations), the type of comorbidity discussed (concordant, discordant), and the supporting evidence of the comorbidity-related recommendations (level of evidence, translation of evidence). Of the 20 guidelines, 17 (85%) addressed the issue of comorbidity and 14 (70%) provided specific recommendations on comorbidity. In general, the guidelines included few recommendations on patients with comorbidity (mean 3 recommendations per guideline, range 0 to 26). Of the 59 comorbidity-related recommendations provided, 46 (78%) addressed concordant comorbidities, 8 (14%) discordant comorbidities, and for 5 (8%) the type of comorbidity was not specified. The strength of the supporting evidence was moderate for 25% (15/59) and low for 37% (22/59) of the recommendations. In addition, for 73% (43/59) of the recommendations the evidence was not adequately translated into the guidelines. Conclusions/Significance Our study showed that the applicability of current evidence-based guidelines to patients with comorbid conditions is limited. Most guidelines do not provide explicit guidance on treatment of patients with comorbidity, particularly for discordant combinations. Guidelines should be more explicit about the applicability of their recommendations to patients with comorbidity. Future clinical trials should also include patients with the most prevalent combinations of chronic conditions.