Francoise Cluzeau

AGREE II: advancing guideline development, reporting and evaluation in health care

Clinical practice guidelines, which are systematically developed statements aimed at helping people make clinical, policy-related and system-related decisions,[1][1],[2][2] frequently vary widely in quality.[3][3],[4][4] A strategy was needed to differentiate among guidelines and ensure that those

Development of the AGREE II, part 1: performance, usefulness and areas for improvement

Background: We undertook research to improve the AGREE instrument, a tool used to evaluate guidelines. We tested a new seven-point scale, evaluated the usefulness of the original items in the instrument, investigated evidence to support shorter, tailored versions of the tool, and identified areas for improvement. Method: We report on one component of a larger study that used a mixed design with four factors (user type, clinical topic, guideline and condition). For the analysis reported in this article, we asked participants to read a guideline and use the AGREE items to evaluate it based on a seven-point scale, to complete three outcome measures related to adoption of the guideline, and to provide feedback on the instrument’s usefulness and how to improve it. Results: Guideline developers gave lower-quality ratings than did clinicians or policy-makers. Five of six domains were significant predictors of participants’ outcome measures (p < 0.05). All domains and items were rated as useful by stakeholders (mean scores > 4.0) with no significant differences by user type (p > 0.05). Internal consistency ranged between 0.64 and 0.89. Inter-rater reliability was satisfactory. We received feedback on how to improve the instrument. Interpretation: Quality ratings of the AGREE domains were significant predictors of outcome measures associated with guideline adoption: guideline endorsements, overall intentions to use guidelines, and overall quality of guidelines. All AGREE items were assessed as useful in determining whether a participant would use a guideline. No clusters of items were found more useful by some users than others. The measurement properties of the seven-point scale were promising. These data contributed to the refinements and release of the AGREE II.

Development of the AGREE II, part 2: assessment of validity of items and tools to support application.

Background: We established a program of research to improve the development, reporting and evaluation of practice guidelines. We assessed the construct validity of the items and user’s manual in the β version of the AGREE II. Methods: We designed guideline excerpts reflecting high-and low-quality guideline content for 21 of the 23 items in the tool. We designed two study packages so that one low-quality and one high-quality version of each item were randomly assigned to each package. We randomly assigned 30 participants to one of the two packages. Participants reviewed and rated the guideline content according to the instructions of the user’s manual and completed a survey assessing the manual. Results: In all cases, content designed to be of high quality was rated higher than low-quality content; in 18 of 21 cases, the differences were significant (p < 0.05). The manual was rated by participants as appropriate, easy to use, and helpful in differentiating guidelines of varying quality, with all scores above the mid-point of the seven-point scale. Considerable feedback was offered on how the items and manual of the β-AGREE II could be improved. Interpretation: The validity of the items was established and the user’s manual was rated as highly useful by users. We used these results and those of our study presented in part 1 to modify the items and user’s manual. We recommend AGREE II (available at www.agreetrust.org) as the revised standard for guideline development, reporting and evaluation.

Evaluating “payback” on biomedical research from papers cited in clinical guidelines: applied bibliometric study

Abstract Objectives: To develop a methodology for evaluating the impact of research on health care, and to characterise the papers cited on clinical guidelines. Design: The bibliographic details of the papers cited in 15 clinical guidelines, developed in and for the United Kingdom, were collated and analysed with applied bibliometric techniques. Results: The median age of papers cited in clinical guidelines was eight years; most papers were published by authors living in either the United States (36%) or the United Kingdom (25%)—this is two and a half times more than expected as about 10% of all biomedical outputs are published in the United Kingdom; and clinical guidelines do not cite basic research papers. Conclusion: Analysis of the evidence base of clinical guidelines may be one way of tracking the flow of knowledge from the laboratory to the clinic. Moreover, such analysis provides a useful, clinically relevant method for evaluating research outcomes and different strategies in research and development.

AGREE II: advancing guideline development, reporting and evaluation in health care.

From McMaster University (Brouwers, Kho, Hanna, Makarski); the Program in Evidence-based Care, Cancer Care Ontario (Brouwers), Hamilton, Ontario; British Columbia Cancer Agency (Browman), Victoria, BC; the Dutch Institute for Healthcare Improvement CBO and IQ Healthcare (Burgers), Radboud University Nijmegen Medical Centre, the Netherlands; St. George’s University of London (Cluzeau), London, UK; the University of Bristol (Feder), Bristol, UK; Unité Cancer et Environement (Fervers), Université de Lyon e Centre Léon Bérard, Université Lyon 1, EA 4129, Lyon, France; the Canadian Institutes of Health Research (Graham), Ottawa, Ontario; the Ottawa Hospital Research Institute (Grimshaw), Ottawa, Ontario; the National Institute for Health and Clinical Excellence (Littlejohns), London, UK; and the Canadian Partnership Against Cancer (Zitzelsberger), Ottawa, Ontario

Clinical practice guidelines: towards better quality guidelines and increased international collaboration

Over recent decades, the number of available clinical practice guidelines has grown enormously. These guidelines are increasingly used in health-care systems throughout the world to improve the quality of patient care, and this also applies to cancer care. Evidence-based guidelines are seen by professionals, authorities, managers and policy makers as powerful tools for achieving effective and efficient care (Woolf et al, 1999). They are considered to be the ideal mediator for bridging the gap between the growing stream of research findings and actual clinical practice. Guidelines should meet specific quality criteria to ensure good quality. Users should be confident that potential biases inherent in guideline development have been addressed appropriately and that the recommendations for practice are both internally and externally valid, as well as feasible for practice (AGREE (Appraisal of Guidelines Research and Evaluation) Collaborative Group, 2000). However, recent studies have reported that the methodological quality of many guidelines is modest and is heterogeneous between the different guidelines and different guideline programmes (Shaneyfelt et al, 1999; Grilli et al, 2000; Lacasse et al, 2001; Burgers et al, 2003a). Although clinical guidelines can provide a solution to some of the important problems in patient care, there are issues that need to be tackled before guidelines can achieve their full potential (Grol, 2001a). We will start by outlining these problems, and then we will present a set of criteria for high-quality guidelines developed and validated by an international group of researchers and guideline developers (the AGREE collaboration). Some cancer guidelines (including those produced by the French National Federation of Cancer Centres–FNCLCC the SOR) were used in the validation process for these criteria. We will then provide some recommendations for guideline developers with the aim that this will help researchers and practitioners in cancer care to develop high-quality guidelines for the management of their patients.

Appraising Clinical Practice Guidelines in England and Wales: The Development of a Methodologic Framework and Its Application to Policy

BACKGROUND Clinical guidelines are pervading all aspects of health care. Their potential benefits are vast--from decreasing unjustified variation in treatment, to improving outcomes for patient, to containing escalating costs. However, there is increasing concern that many of the guidelines produced may be of low quality and recommend ineffective treatment. A framework to appraise the methodologic quality of clinical guidelines, commissioned by the NHS in 1997, was developed and validated in the United Kingdom (UK) under the auspices of the National Research and Development Programme. THE INDEPENDENT APPRAISAL SERVICE This framework is now used to assess all national guidelines funded by the National Health Service (NHS) in the UK as part of an independent appraisal service. The appraisal provides a basis for policymakers to decide which guidelines should be commended for use in the NHS. Each guideline is appraised by a minimum of six appraisers. Twenty-one guidelines had been appraised as of July 1999. The mean time for completion of the appraisals, from receipt of the guidelines to dispatch of the reports, was just over ten weeks. There has been a marked improvement in the quality of documentation for national guidelines--including the search strategy and synthesis of evidence--in the past three years, although some areas of development remain inadequately reported. POLICY IMPLICATIONS Ensuring that the clinical guidelines are sound before recommending their use is essential to policymakers responsible for guideline programs, and a formal appraisal should be an integral part of those programs.

Improving health system quality in low- and middle-income countries that are expanding health coverage: a framework for insurance.

PURPOSE Low- and middle-income countries are increasingly pursuing health financing reforms aimed at achieving universal health coverage. As these countries rapidly expand access to care, overburdened health systems may fail to deliver high-quality care, resulting in poor health outcomes. Public insurers responsible for financing coverage expansions have the financial leverage to influence the quality of care and can benefit from guidance to execute a cohesive health-care quality strategy. DATA SOURCES and selection Following a literature review, we used a cascading expert consultation and validation process to develop a conceptual framework for insurance-driven quality improvements in health care. RESULTS OF DATA SYNTHESIS The framework presents the strategies available to insurers to influence the quality of care within three domains: ensuring a basic standard of quality, motivating providers and professionals to improve, and activating patient and public demand for quality. By being sensitive to the local context, building will among key stakeholders and selecting context-appropriate ideas for improvement, insurers can influence the quality through four possible mechanisms: selective contracting; provider payment systems; benefit package design and investments in systems, patients and providers. CONCLUSION This framework is a resource for public insurers that are responsible for rapidly expanding access to care, as it places the mechanisms that insurers directly control within the context of broader strategies of improving health-care quality. The framework bridges the existing gap in the literature between broad frameworks for strategy design for system improvement and narrower discussions of the technical methods by which payers directly influence the quality.

Improving quality for maternal care - a case study from Kerala, India

Background: The implementation of maternal health guidelines remains unsatisfactory, even for simple, well established interventions. In settings where most births occur in health facilities, as is the case in Kerala, India, preventing maternal mortality is linked to quality of care improvements. Context: Evidence-informed quality standards (QS), including quality statements and measurable structure and process indicators, are one innovative way of tackling the guideline implementation gap. Having adopted a zero tolerance policy to maternal deaths, the Government of Kerala worked in partnership with the Kerala Federation of Obstetricians & Gynaecologists (KFOG) and NICE International to select the clinical topic, develop and initiate implementation of the first clinical QS for reducing maternal mortality in the state. Description of practice: The NICE QS development framework was adapted to the Kerala context, with local ownership being a key principle. Locally generated evidence identified post-partum haemorrhage as the leading cause of maternal death, and as the key priority for the QS. A multidisciplinary group (including policy-makers, gynaecologists and obstetricians, nurses and administrators) was established. Multi-stakeholder workshops convened by the group ensured that the statements, derived from global and local guidelines, and their corresponding indicators were relevant and acceptable to clinicians and policy-makers in Kerala. Furthermore, it helped identify practical methods for implementing the standards and monitoring outcomes. Lessons learned: An independent evaluation of the project highlighted the equal importance of a strong evidence-base and an inclusive development process. There is no one-size-fits-all process for QS development; a principle-based approach might be a better guide for countries to adapt global evidence to their local context.

Let’s not forget AGREE

Clinicians should have criteria to determine whether a guideline has been developed systematically and transparently, and we are surprised that Oxman and colleagues did not mention the appraisal of guidelines research and …