Atle Fretheim

Improving the use of research evidence in guideline development: 10. Integrating values and consumer involvement

AbstractBackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the 10th of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on integrating values and consumers in guideline development.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We reviewed the titles of all citations and retrieved abstracts and full text articles if the citations appeared relevant to the topic. We checked the reference lists of articles relevant to the questions and used snowballing as a technique to obtain additional information. We did not conduct a full systematic review ourselves. Our conclusions based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersWe did not find a systematic review of methods for integrating values in guidelines, but we found several systematic reviews that dealt with related topics. Whose values should WHO use when making recommendations? • Values, the relative importance or worth of a state or consequences of a decision (outcomes relating to benefits, harms, burden and costs), play a role in every recommendation. Ethical considerations, concepts that determine what is right, also play a role.• The values used in making recommendations should reflect those of the people affected. Judgements should be explicit and should be informed by input from those affected (including citizens, patients, clinicians and policy makers).• When differences in values may lead to different decisions or there is uncertainty about values, this should also be explicit. If differences in values are likely to affect a decision, such that people in different setting would likely make different choices about interventions or actions based on differences in their values, global recommendations should be explicit in terms of which values were applied and allow for adaptation after incorporating local values. How should WHO ensure that appropriate values are integrated in recommendations? • All WHO guideline groups should uniformly apply explicit, transparent and clearly described methods for integrating values.• WHO should consider involving relevant stakeholders if this is feasible and efficient.• WHO should develop a checklist for guidelines panels to help them to ensure that ethical considerations relevant to recommendations are addressed explicitly and transparently. How should users and consumers be involved in generating recommendations? • Including consumers in groups that are making global recommendations presents major challenges with respect to the impossibility of including a representative spectrum of consumers from a variety of cultures and settings. Nonetheless, consideration should be given to including consumers in groups who are able to challenge assumptions that are made about the values used for making recommendations, rather than represent the values of consumers around the world.• WHO should establish a network to facilitate involvement of users.• Draft recommendations should be reviewed by consumers, who should be asked explicitly to consider the values that were used. How should values be presented in recommendations? • Recommendations should include a description of how decisions were made about the relative importance of the consequences (benefits, harms and costs) of a decision.• Values that influence recommendations should be reported along with the research evidence underlying recommendations.• When differences in values would lead to different decisions or there is important uncertainty about values that are critical to a decision, this should be flagged and reflected in the strength of the recommendation.• Adaptable guideline templates that allow for integration of different values should be developed and used when differences in values are likely to be critical to a decision.

Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence

AbstractBackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the eighth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on the synthesis and presentation of research evidence, focusing on four key questions.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersWe found two reviews of instruments for critically appraising systematic reviews, several studies of the importance of using extensive searches for reviews and determining when it is important to update reviews, and consensus statements about the reporting of reviews that informed our answers to the following questions. How should existing systematic reviews be critically appraised? • Because preparing systematic reviews can take over a year and require capacity and resources, existing reviews should be used when possible and updated, if needed.• Standard criteria, such as A MeaSurement Tool to Assess Reviews (AMSTAR), should be used to critically appraise existing systematic reviews, together with an assessment of the relevance of the review to the questions being asked. When and how should WHO undertake or commission new reviews? • Consideration should be given to undertaking or commissioning a new review whenever a relevant, up-to-date review of good quality is not available.• When time or resources are limited it may be necessary to undertake rapid assessments. The methods that are used to do these assessments should be reported, including important limitations and uncertainties and explicit consideration of the need and urgency of undertaking a full systematic review.• Because WHO has limited capacity for undertaking systematic reviews, reviews will often need to be commissioned when a new review is needed. Consideration should be given to establishing collaborating centres to undertake or support this work, similar to what some national organisations have done. How should the findings of systematic reviews be summarised and presented to committees responsible for making recommendations? • Concise summaries (evidence tables) of the best available evidence for each important outcome, including benefits, harms and costs, should be presented to the groups responsible for making recommendations. These should include an assessment of the quality of the evidence and a summary of the findings for each outcome.• The full systematic reviews, on which the summaries are based, should also be available to both those making recommendations and users of the recommendations. What additional information is needed to inform recommendations and how should this information be synthesised with information about effects and presented to committees? • Additional information that is needed to inform recommendations includes factors that might modify the expected effects, need (prevalence, baseline risk or status), values (the relative importance of key outcomes), costs and the availability of resources.• Any assumptions that are made about values or other factors that may vary from setting to setting should be made explicit.• For global guidelines that are intended to inform decisions in different settings, consideration should be given to using a template to assist the synthesis of information specific to a setting with the global evidence of the effects of the relevant interventions.

Improving the use of research evidence in guideline development: 3. Group composition and consultation process

AbstractBackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the third of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectiveIn this review we address the composition of guideline development groups and consultation processes during guideline development.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersWhat should be the composition of a WHO-panel that is set up to develop recommendations? The existing empirical evidence suggests that panel composition has an impact on the content of the recommendations that are made. There is limited research evidence to guide the exact composition of a panel. Based on logical arguments and the experience of other organisations we recommend the following:• Groups that develop guidelines or recommendations should be broadly composed and include important stakeholders such as consumers, health professionals that work within the relevant area, and managers or policy makers.• Groups should include or have access to individuals with the necessary technical skills, including information retrieval, systematic reviewing, health economics, group facilitation, project management, writing and editing.• Groups should include or have access to content experts.• To work well a group needs an effective leader, capable of guiding the group in terms of the task and process, and capable of facilitating collaboration and balanced contribution from all of the group members.• Because many group members will not be familiar with the methods and processes that are used in developing recommendations, groups should be offered training and support to help ensure understanding and facilitate active participation. What groups should be consulted when a panel is being set up? We did not identify methodological research that addressed this question, but based on logical arguments and the experience of other organisations we recommend that as many relevant stakeholder groups as practical should be consulted to identify suitable candidates with an appropriate mix of perspectives, technical skills and expertise, as well as to obtain a balanced representation with respect to regions and gender. What methods should WHO use to ensure appropriate consultations? We did not find any references that addressed issues related to this question. Based on logical arguments and the experience of other organisations we believe that consultations may be desirable at several stages in the process of developing guidelines or recommendations, including:• Identifying and setting priorities for guidelines and recommendations• Commenting on the scope of the guidelines or recommendations• Commenting on the evidence that is used to inform guidelines or recommendations• Commenting on drafts of the guidelines or recommendations• Commenting on plans for disseminating and supporting the adaptation and implementation of the guidelines or recommendations.• Key stakeholder organisations should be contacted directly whenever possible.• Consultation processes should be transparent and should encourage feedback from interested parties.

Improving the use of research evidence in guideline development: 1. Guidelines for guidelines

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the first of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on guidelines for the development of guidelines.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersWe found no experimental research that compared different formats of guidelines for guidelines or studies that compared different components of guidelines for guidelines. However, there are many examples, surveys and other observational studies that compared the impact of different guideline development documents on guideline quality.What have other organizations done to develop guidelines for guidelines from which WHO can learn?Establish a credible, independent committee that evaluates existing methods for developing guidelines or that updates existing ones.Obtain feedback and approval from various stakeholders during the development process of guidelines for guidelines.Develop a detailed source document (manual) that guideline developers can use as reference material.What should be the key components of WHO guidelines for guidelines?Guidelines for guidelines should include information and instructions about the following components: 1) Priority setting; 2) Group composition and consultations; 3) Declaration and avoidance of conflicts of interest; 4) Group processes; 5) Identification of important outcomes; 6) Explicit definition of the questions and eligibility criteria ; 7) Type of study designs for different questions; 8) Identification of evidence; 9) Synthesis and presentation of evidence; 10) Specification and integration of values; 11) Making judgments about desirable and undesirable effects; 12) Taking account of equity; 13) Grading evidence and recommendations; 14) Taking account of costs; 15) Adaptation, applicability, transferability of guidelines; 16) Structure of reports; 17) Methods of peer review; 18) Planned methods of dissemination & implementation; 19) Evaluation of the guidelines.What have other organizations done to implement guidelines for guidelines from which WHO can learn?Obtain buy-in from regions and country level representatives for guidelines for guidelines before dissemination of a revised version.Disseminate the guidelines for guidelines widely and make them available (e.g. on the Internet).Develop examples of guidelines that guideline developers can use as models when applying the guidelines for guidelines.Ensure training sessions for those responsible for developing guidelines.Continue to monitor the methodological literature on guideline development.

Improving the use of research evidence in guideline development: 5. Group processes.

AbstractBackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the fifth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectiveIn this review we address approaches to facilitate sound processes within groups that develop recommendations for health care.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key question and answerWhat should WHO do to ensure appropriate group processes? Various strategies can be adopted to ensure that the group processes in play when panels are developing recommendations are inclusive, so that all voices can be heard and all arguments given fair weight, including• the use of formal consensus development methods, such at the Nominal Group Technique or the Delphi method• the selection of a group leader who is qualified and responsible for facilitating an appropriate group process.

Improving the use of research evidence in guideline development: 14. Reporting guidelines

AbstractBackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the 14th of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on reporting guidelines and recommendations.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersThere is little empirical evidence that addresses these questions. Our answers are based on logical arguments and standards put forward by other groups. What standard types of recommendations or reports should WHO use?WHO should develop standard formats for reporting recommendations to facilitate recognition and use by decision makers for whom the recommendations are intended, and to ensure that all the information needed to judge the quality of a guideline, determine its applicability and, if needed, adapt it, is reported.WHO should develop standard formats for full systematically developed guidelines that are sponsored by WHO, rapid assessments, and guidelines that are endorsed by WHO.All three formats should include the same information as full guidelines, indicating explicitly what the group preparing the guideline did not do, as well as the methods that were used.These formats should be used across clinical, public health and health systems recommendations.How should recommendations be formulated and reported?Reports should be structured, using headings that correspond to those suggested by the Conference on Guideline Standardization or similar headings.The quality of evidence and strength of recommendations should be reported explicitly using a standard approach.The way in which recommendations are formulated should be adapted to the specific characteristics of a specific guideline.Urgent attention should be given to developing a template that provides decision makers with the relevant global evidence that is needed to inform a decision and offers practical methods for incorporating the context specific evidence and judgements that are needed.

Improving the use of research evidence in guideline development: 15. Disseminating and implementing guidelines

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the 15th of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesIn this review we address strategies for the implementation of recommendations in health care.MethodsWe examined overviews of systematic reviews of interventions to improve health care delivery and health care systems prepared by the Cochrane Effective Practice and Organisation of Care (EPOC) group. We also conducted searches using PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersWhat should WHO do to disseminate and facilitate the uptake of recommendations?WHO should choose strategies to implement their guidelines from among those which have been evaluated positively in the published literature on implementation researchBecause the evidence base is weak and modest to moderate effects, at best, can be anticipated, WHO should promote rigorous evaluations of implementation strategies.What should be done at headquarters, by regional offices and in countries?Adaptation and implementation of WHO guidelines should be done locally, at the national or sub-national level.WHO headquarters and regional offices should support the development and evaluation of implementation strategies by local authorities.

Improving the use of research evidence in guideline development: 16. Evaluation

AbstractBackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the last of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on evaluating guidelines and recommendations, including their quality, whether they are likely to be up-to-date, and their implementation. We also considered the role of guideline developers in undertaking evaluations that are needed to inform recommendations.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersOur answers to these questions were informed by a review of instruments for evaluating guidelines, several studies of the need for updating guidelines, discussions of the pros and cons of different research designs for evaluating the implementation of guidelines, and consideration of the use of uncertainties identified in systematic reviews to set research priorities. How should the quality of guidelines or recommendations be appraised? • WHO should put into place processes to ensure that both internal and external review of guidelines is undertaken routinely.• A checklist, such as the AGREE instrument, should be used.• The checklist should be adapted and tested to ensure that it is suitable to the broad range of recommendations that WHO produces, including public health and health policy recommendations, and that it includes questions about equity and other items that are particularly important for WHO guidelines. When should guidelines or recommendations be updated? • Processes should be put into place to ensure that guidelines are monitored routinely to determine if they are in need of updating.• People who are familiar with the topic, such as Cochrane review groups, should do focused, routine searches for new research that would require revision of the guideline.• Periodic review of guidelines by experts not involved in developing the guidelines should also be considered.• Consideration should be given to establishing guideline panels that are ongoing, to facilitate routine updating, with members serving fixed periods with a rotating membership. How should the impact of guidelines or recommendations be evaluated? • WHO headquarters and regional offices should support member states and those responsible for policy decisions and implementation to evaluate the impact of their decisions and actions by providing advice regarding impact assessment, practical support and coordination of efforts.• Before-after evaluations should be used cautiously and when there are important uncertainties regarding the effects of a policy or its implementation, randomised evaluations should be used when possible. What responsibility should WHO take for ensuring that important uncertainties are addressed by future research when the evidence needed to inform recommendations is lacking? • Guideline panels should routinely identify important uncertainties and research priorities. This source of potential priorities for research should be used systematically to inform priority-setting processes for global research.

The potential savings of using thiazides as the first choice antihypertensive drug: cost-minimisation analysis

BackgroundAll clinical practice guidelines recommend thiazides as a first-choice drug for the management of uncomplicated hypertension. Thiazides are also the lowest priced antihypertensive drugs. Despite this, the use of thiazides is much lower than that of other drug-classes. We wanted to estimate the potential for savings if thiazides were used as the first choice drug for the management of uncomplicated hypertension.MethodsFor six countries (Canada, France, Germany, Norway, the UK and the US) we estimated the number of people that are being treated for hypertension, and the proportion of them that are suitable candidates for thiazide-therapy. By comparing this estimate with thiazide prescribing, we calculated the number of people that could switch from more expensive medication to thiazides. This enabled us to estimate the potential drug-cost savings. The analysis was based on findings from epidemiological studies and drug trials, and data on sales and prescribing provided by IMS for the year 2000.ResultsFor Canada, France, Germany, Norway, the UK and the US the estimated potential annual savings were US

Improving the use of research evidence in guideline development: 6. Determining which outcomes are important

13.8 million, US