Kaelan A. Moat

Capturing lessons learned from evidence-to-policy initiatives through structured reflection

BackgroundKnowledge translation platforms (KTPs), which are partnerships between policymakers, stakeholders, and researchers, are being established in low- and middle-income countries (LMICs) to enhance evidence-informed health policymaking (EIHP). This study aims to gain a better understanding of the i) activities conducted by KTPs, ii) the way in which KTP leaders, policymakers, and stakeholders perceive these activities and their outputs, iii) facilitators that support KTP work and challenges, and the lessons learned for overcoming such challenges, and iv) factors that can help to ensure the sustainability of KTPs.MethodsThis paper triangulated qualitative data from: i) 17 semi-structured interviews with 47 key informants including KTP leaders, policymakers, and stakeholders from 10 KTPs; ii) document reviews, and iii) observation of deliberations at the International Forum on EIHP in LMICs held in Addis Ababa in August 2012. Purposive sampling was used and data were analyzed using thematic analysis.ResultsDeliberative dialogues informed by evidence briefs were identified as the most commendable tools by interviewees for enhancing EIHP. KTPs reported that they have contributed to increased awareness of the importance of EIHP and strengthened relationships among policymakers, stakeholders, and researchers. Support from policymakers and international funders facilitated KTP activities, while the lack of skilled human resources to conduct EIHP activities impeded KTPs. Ensuring the sustainability of EIHP initiatives after the end of funding was a major challenge for KTPs. KTPs reported that institutionalization within the government has helped to retain human resources and secure funding, whereas KTPs hosted by universities highlighted the advantage of autonomy from political interests.ConclusionsThe establishment of KTPs is a promising development in supporting EIHP. Real-time lesson drawing from the experiences of KTPs can support improvements in the functioning of KTPs in the short term, while making the case for sustaining their work in the long term. Lessons learned can help to promote similar EIHP initiatives in other countries.

Developing and refining the methods for a ‘one-stop shop’ for research evidence about health systems

BackgroundPolicymakers, stakeholders and researchers have not been able to find research evidence about health systems using an easily understood taxonomy of topics, know when they have conducted a comprehensive search of the many types of research evidence relevant to them, or rapidly identify decision-relevant information in their search results.MethodsTo address these gaps, we developed an approach to building a ‘one-stop shop’ for research evidence about health systems. We developed a taxonomy of health system topics and iteratively refined it by drawing on existing categorization schemes and by using it to categorize progressively larger bundles of research evidence. We identified systematic reviews, systematic review protocols, and review-derived products through searches of Medline, hand searches of several databases indexing systematic reviews, hand searches of journals, and continuous scanning of listservs and websites. We developed an approach to providing ‘added value’ to existing content (e.g., coding systematic reviews according to the countries in which included studies were conducted) and to expanding the types of evidence eligible for inclusion (e.g., economic evaluations and health system descriptions). Lastly, we developed an approach to continuously updating the online one-stop shop in seven supported languages.ResultsThe taxonomy is organized by governance, financial, and delivery arrangements and by implementation strategies. The ‘one-stop shop’, called Health Systems Evidence, contains a comprehensive inventory of evidence briefs, overviews of systematic reviews, systematic reviews, systematic review protocols, registered systematic review titles, economic evaluations and costing studies, health reform descriptions and health system descriptions, and many types of added-value coding. It is continuously updated and new content is regularly translated into Arabic, Chinese, English, French, Portuguese, Russian, and Spanish.ConclusionsPolicymakers and stakeholders can now easily access and use a wide variety of types of research evidence about health systems to inform decision-making and advocacy. Researchers and research funding agencies can use Health Systems Evidence to identify gaps in the current stock of research evidence and domains that could benefit from primary research, systematic reviews, and review overviews.

Twelve myths about systematic reviews for health system policymaking rebutted.

Systematic reviews are increasingly being viewed as important sources of information for policymakers who need to make decisions on different aspects of the health system, often under tight time constraints and with many factors competing for their attention. Unfortunately, a number of misconceptions, or ‘myths’, stand in the way of promoting their use. The belief that systematic review topics are not relevant to health systems policymaking, that they cannot be found quickly, and that they are not available in formats that are useful for policymakers are but three examples of such myths. This paper uses evidence drawn mainly from Health Systems Evidence, a continuously updated repository of syntheses of health systems research, to counter these and nine other common myths, with the aim of changing the constraining beliefs associated with them, while improving the prospects for the use of systematic reviews in health system policymaking.

The global stock of research evidence relevant to health systems policymaking

BackgroundPolicymakers and stakeholders need immediate access to many types of research evidence to make informed decisions about the full range of questions that may arise regarding health systems.MethodsWe examined all types of research evidence about governance, financial and delivery arrangements, and implementation strategies within health systems contained in Health Systems Evidence (HSE) (http://www.healthsystemsevidence.org). The research evidence types include evidence briefs for policy, overviews of systematic reviews, systematic reviews of effects, systematic reviews addressing other questions, systematic reviews in progress, systematic reviews being planned, economic evaluations, and health reform and health system descriptions. Specifically, we describe their distribution across health system topics and domains, trends in their production over time, availability of supplemental content in various languages, and the extent to which they focus on low- and middle-income countries (LMICs), as well as (for systematic reviews) their methodological quality and the availability of user-friendly summaries.ResultsAs of July 2013, HSE contained 2,629 systematic reviews of effects (of which 501 are Cochrane reviews), 614 systematic reviews addressing other questions, 283 systematic reviews in progress, 186 systematic reviews being planned, 140 review-derived products (evidence briefs and overviews of systematic reviews), 1,669 economic evaluations, 1,092 health reform descriptions, and 209 health system descriptions. Most systematic reviews address topics related to delivery arrangements (n = 2,663) or implementation strategies (n = 1,653) with far fewer addressing financial (n = 241) or governance arrangements (n = 231). In addition, 2,928 systematic reviews have been quality appraised with moderate AMSTAR ratings found for reviews addressing governance (5.6/11), financial (5.9/11), and delivery (6.3/11) arrangements and implementation strategies (6.5/11); 1,075 systematic reviews have no independently produced user-friendly summary and only 737 systematic reviews have an LMIC focus. Literature searches for half of the systematic reviews (n = 1,584, 49%) were conducted within the last five years.ConclusionsGreater effort needs to focus on assessing whether the current distribution of systematic reviews corresponds to policymakers’ and stakeholders’ priorities, updating systematic reviews, increasing the quality of systematic reviews, and focusing on LMICs.

10 best resources for … evidence-informed health policy making

Health Policy PhD Program, McMaster University, Hamilton, Canada, Program in Policy Decision-making, McMaster University, Hamilton, Canada, Centre for Health Economics and Policy Analysis, McMaster University, Hamilton, Canada, McMaster Health Forum, McMaster University, Hamilton, Canada, Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada and Department of Political Science, McMaster University, Hamilton, Canada *Corresponding author. McMaster University, 1280 Main St. West, CRL-209, Hamilton, ON, Canada L8S 4K1. Tel: þ1 905 525 9140 ext. 22521. Fax: þ 1 905 546 5211. E-mail: moatka@mcmaster.ca

Processes, contexts, and rationale for disinvestment: a protocol for a critical interpretive synthesis

BackgroundPractical solutions are needed to support the appropriate use of available health system resources as countries are continually pressured to ‘do more with less’ in health care. Increasingly, health systems and organizations are exploring the reassessment of possibly obsolete, inefficient, or ineffective health system resources and potentially redirecting funds to those that are more effective and efficient. Such processes are often referred to as ‘disinvestment’. Our objective is to gain further understanding about: 1) whether how and under what conditions health systems decide to pursue disinvestment; 2) how health systems have chosen to undertake disinvestment; and 3) how health systems have implemented their disinvestment approach.Methods/DesignWe will use a critical interpretive synthesis (CIS) approach, to develop a theoretical framework based on insights drawn from a range of relevant sources. We will conduct systematic searches of databases as well as purposive searches to identify literature to fill conceptual gaps that may emerge during our inductive process of synthesis and analysis. Two independent reviewers will assess search results for relevance and conceptually map included references. We will include all empirical and non-empirical articles that focus on disinvestment at a system level. We will then extract key findings from a purposive sample of articles using frameworks related to government agendas, policy development and implementation, and health system contextual factors and then synthesize and integrate the findings to develop a framework about our core areas of interest. Lastly, we will convene a stakeholder dialogue with Canadian and international policymakers and other stakeholders to solicit targeted feedback about the framework (e.g., by identifying any gaps in the literature that we may want to revisit before finalizing it) and deliberating about barriers for developing and implementing approaches to disinvestment, strategies to address these barriers and about next steps that could be taken by different constituencies.DiscussionDisinvestment is an emerging field and there is a need for evidence to inform the prioritization, development, and implementation of strategies in different contexts. Our CIS and the framework developed through it will support the actions of those involved in the prioritization, development, and implementation of disinvestment initiatives.Systematic review registrationPROSPEROCRD42014013204

Analysis of the policymaking process in Burkina Faso’s health sector: case studies of the creation of two health system support units

BackgroundBurkina Faso has made a number of health system policy decisions to improve performance on health indicators and strengthen responsiveness to health-related challenges. These included the creation of a General Directorate of Health Information and Statistics (DGISS) and a technical unit to coordinate performance-based financing (CT-FBR). We analysed the policymaking processes associated with the establishment of these units, and documented the factors that influenced this process.MethodWe used a multiple-case study design based on Kingdon’s agenda-setting model to investigate the DGISS and CT-FBR policymaking processes. Data were collected from interviews with key informants (n = 28), published literature, policy documents (including two strategic and 230 action plans), and 55 legal/regulatory texts. Interviews were analysed using thematic qualitative analysis. Data from the documentary analysis were triangulated with the qualitative interview data.ResultsKey factors influencing the policymaking processes associated with the two units involved the ‘problem’ (problem identification), ‘policy’ (formation of policy proposals), and ‘politics’ (political climate/change) streams, which came together in a way that resulted in proposals being placed on the decision agenda. A number of problems with Burkina Faso’s health information and financing systems were identified. Policy proposals for the DGISS and CT-FBR units were developed in response to these problems, emerging from several sources including development partners. Changes in political and public service administrations (specifically the 2008 appointment of a new Minister of Health and the establishment of a new budget allocation system), with corresponding changes in the actors and interests involved, appeared key in elevating the proposals to the decision agenda.ConclusionsEfforts to improve performance on health indicators and strengthen responsiveness to health-related challenges need focus on the need for a compelling problem, a viable policy, and conducive politics in order to make it to the decision agenda.

Towards a better understanding of the nomenclature used in information-packaging efforts to support evidence-informed policymaking in low- and middle-income countries

BackgroundThe growing recognition of the importance of concisely communicating research evidence and other policy-relevant information to policymakers has underpinned the development of several information-packaging efforts over the past decade. This has led to a wide variability in the types of documents produced, which is at best confusing and at worst discouraging for those they intend to reach. This paper has two main objectives: to develop a better understanding of the range of documents and document names used by the organizations preparing them; and to assess whether there are any consistencies in the characteristics of sampled documents across the names employed to label (in the title) or describe (in the document or website) them.MethodsWe undertook a documentary analysis of web-published document series that are prepared by a variety of organizations with the primary intention of providing information to health systems policymakers and stakeholders, and addressing questions related to health policy and health systems with a focus on low- and middle-income countries. No time limit was set.ResultsIn total, 109 individual documents from 24 series produced by 16 different organizations were included. The name ‘policy brief/briefing’ was the most frequently used (39%) to label or describe a document, and was used in all eight broad content areas that we identified, even though they did not have obviously common traits among them. In terms of document characteristics, most documents (90%) used skimmable formats that are easy to read, with understandable, jargon-free, language (80%). Availability of information on the methods (47%) or the quality of the presented evidence (27%) was less common. One-third (32%) chose the topic based on an explicit process to assess the demand for information from policy makers and even fewer (19%) engaged with policymakers to discuss the content of these documents such as through merit review.ConclusionsThis study highlights the need for organizations embarking on future information-packaging efforts to be more thoughtful when deciding how to name these documents and the need for greater transparency in describing their content, purpose and intended audience.

The factors affecting the institutionalisation of two policy units in Burkina Faso’s health system: a case study

BackgroundThis paper is one of three linked studies that attempts to understand the process of institutionalisation of policy units within Burkina Faso’s health system. It examines the relationships between the existence of an institutional framework, data production capacity and other resource availability in the institutionalisation of policy units in health systems. It therefore contributes to our understanding of the dynamics linking the key drivers and indicators of institutionalisation. Additionally, it examines how factors within the managerial setting, including workplace environment, and budgetary and human resource availability, may influence the institutionalisation process.MethodsThe study used an explanatory qualitative case study approach, examining two policy units in Burkina Faso’s Ministry of Health, the first of which had been institutionalised successfully and the other less so. Data were collected from key policymakers, including 13 connected with the first policy unit and 10 with the second, plus two funders. We also conducted a documentary analysis of the National Program for Health Development, two mid-term strategic plans, 230 action plans, eight Ministry of Health state budgets, eight Ministry of Health annual statistics reports, 16 policy unit budgets and published literature.ResultsThe framework within which the government gave the policy unit its mandate and policy focus had the strongest effect on the institutionalisation process. Institutionalisation depended on political will, in both the host government and any donors, and the priority given to the policy unit’s focus. It was also affected by the leadership of the policy unit managers. These factors were influenced by human resource capacity, and our findings suggest that, for successful institutionalisation in Burkina Faso’s health system, policy units need to be given sufficient human resources to achieve their objectives.ConclusionPolicy units’ institutionalisation in Burkina Faso’s health system depend on the leadership of the unit managers to implement relevant activities, mobilise funding, and recruit and maintain enough human resources, as well as the mandate given by the government.

Addressing overuse of health services in health systems: a critical interpretive synthesis

BackgroundHealth systems are increasingly focusing on the issue of ‘overuse’ of health services and how to address it. We developed a framework focused on (1) the rationale and context for health systems prioritising addressing overuse, (2) elements of a comprehensive process and approach to reduce overuse and (3) implementation considerations for addressing overuse.MethodsWe conducted a critical interpretive synthesis informed by a stakeholder-engagement process. The synthesis identified relevant empirical and non-empirical articles about system-level overuse. Two reviewers independently screened records, assessed for inclusion and conceptually mapped included articles. From these, we selected a purposive sample, created structured summaries of key findings and thematically synthesised the results.ResultsOur search identified 3545 references, from which we included 251. Most articles (76%; n = 192) were published within 5 years of conducting the review and addressed processes for addressing overuse (63%; n = 158) or political and health system context (60%; n = 151). Besides negative outcomes at the patient, system and global level, there were various contextual factors to addressing service overuse that seem to be key issue drivers. Processes for addressing overuse can be grouped into three elements comprising a comprehensive approach, including (1) approaches to identify overused health services, (2) stakeholder- or patient-led approaches and (3) government-led initiatives. Key implementation considerations include the need to develop ‘buy in’ from stakeholders and citizens.ConclusionsHealth systems want to ensure the use of high-value services to keep citizens healthy and avoid harm. Our synthesis can be used by policy-makers, stakeholders and researchers to understand how the issue has been prioritised, what approaches have been used to address it and implementation considerations.Systematic review registrationPROSPERO CRD42014013204.