Etienne V. Langlois

Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series

The GRADE-CERQual (‘Confidence in the Evidence from Reviews of Qualitative research’) approach provides guidance for assessing how much confidence to place in findings from systematic reviews of qualitative research (or qualitative evidence syntheses). The approach has been developed to support the use of findings from qualitative evidence syntheses in decision-making, including guideline development and policy formulation. Confidence in the evidence from qualitative evidence syntheses is an assessment of the extent to which a review finding is a reasonable representation of the phenomenon of interest. CERQual provides a systematic and transparent framework for assessing confidence in individual review findings, based on consideration of four components: (1) methodological limitations, (2) coherence, (3) adequacy of data, and (4) relevance. A fifth component, dissemination (or publication) bias, may also be important and is being explored. As with the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach for effectiveness evidence, CERQual suggests summarising evidence in succinct, transparent, and informative Summary of Qualitative Findings tables. These tables are designed to communicate the review findings and the CERQual assessment of confidence in each finding. This article is the first of a seven-part series providing guidance on how to apply the CERQual approach. In this paper, we describe the rationale and conceptual basis for CERQual, the aims of the approach, how the approach was developed, and its main components. We also outline the purpose and structure of this series and discuss the growing role for qualitative evidence in decision-making. Papers 3, 4, 5, 6, and 7 in this series discuss each CERQual component, including the rationale for including the component in the approach, how the component is conceptualised, and how it should be assessed. Paper 2 discusses how to make an overall assessment of confidence in a review finding and how to create a Summary of Qualitative Findings table. The series is intended primarily for those undertaking qualitative evidence syntheses or using their findings in decision-making processes but is also relevant to guideline development agencies, primary qualitative researchers, and implementation scientists and practitioners.

Enhancing evidence informed policymaking in complex health systems: lessons from multi-site collaborative approaches

BackgroundThere is an increasing interest worldwide to ensure evidence-informed health policymaking as a means to improve health systems performance. There is a need to engage policymakers in collaborative approaches to generate and use knowledge in real world settings. To address this gap, we implemented two interventions based on iterative exchanges between researchers and policymakers/implementers. This article aims to reflect on the implementation and impact of these multi-site evidence-to-policy approaches implemented in low-resource settings.MethodsThe first approach was implemented in Mexico and Nicaragua and focused on implementation research facilitated by communities of practice (CoP) among maternal health stakeholders. We conducted a process evaluation of the CoPs and assessed the professionals’ abilities to acquire, analyse, adapt and apply research. The second approach, called the Policy BUilding Demand for evidence in Decision making through Interaction and Enhancing Skills (Policy BUDDIES), was implemented in South Africa and Cameroon. The intervention put forth a ‘buddying’ process to enhance demand and use of systematic reviews by sub-national policymakers. The Policy BUDDIES initiative was assessed using a mixed-methods realist evaluation design.ResultsIn Mexico, the implementation research supported by CoPs triggered monitoring by local health organizations of the quality of maternal healthcare programs. Health programme personnel involved in CoPs in Mexico and Nicaragua reported improved capacities to identify and use evidence in solving implementation problems. In South Africa, Policy BUDDIES informed a policy framework for medication adherence for chronic diseases, including both HIV and non-communicable diseases. Policymakers engaged in the buddying process reported an enhanced recognition of the value of research, and greater demand for policy-relevant knowledge.ConclusionsThe collaborative evidence-to-policy approaches underline the importance of iterations and continuity in the engagement of researchers and policymakers/programme managers, in order to account for swift evolutions in health policy planning and implementation. In developing and supporting evidence-to-policy interventions, due consideration should be given to fit-for-purpose approaches, as different needs in policymaking cycles require adapted processes and knowledge. Greater consideration should be provided to approaches embedding the use of research in real-world policymaking, better suited to the complex adaptive nature of health systems.

Advancing the field of health systems research synthesis

Those planning, managing and working in health systems worldwide routinely need to make decisions regarding strategies to improve health care and promote equity. Systematic reviews of different kinds can be of great help to these decision-makers, providing actionable evidence at every step in the decision-making process. Although there is growing recognition of the importance of systematic reviews to inform both policy decisions and produce guidance for health systems, a number of important methodological and evidence uptake challenges remain and better coordination of existing initiatives is needed. The Alliance for Health Policy and Systems Research, housed within the World Health Organization, convened an Advisory Group on Health Systems Research (HSR) Synthesis to bring together different stakeholders interested in HSR synthesis and its use in decision-making processes. We describe the rationale of the Advisory Group and the six areas of its work and reflects on its role in advancing the field of HSR synthesis. We argue in favour of greater cross-institutional collaborations, as well as capacity strengthening in low- and middle-income countries, to advance the science and practice of health systems research synthesis. We advocate for the integration of quasi-experimental study designs in reviews of effectiveness of health systems intervention and reforms. The Advisory Group also recommends adopting priority-setting approaches for HSR synthesis and increasing the use of findings from systematic reviews in health policy and decision-making.

Postnatal care: increasing coverage, equity, and quality

As the Millennium Development Goals came to a close last year and we entered the new Sustainable Development Goals (SDGs) era the global health community took stock of accomplishments over the past decades and continuing challenges for the future. Despite impressive reductions in maternal and under-5 mortality rates neonatal mortality reduction continues to lag behind. Neonates account for an increasing share of child deaths now reaching almost half (45%) of the burden of under-5 mortality. It took a long time for the global health community to seriously address the quality of the content of antenatal care and intrapartum care beyond coverage indicators. We need to learn from this experience and ensure that quality and integration of postnatal care for mothers and neonates gets political attention and investment sooner. It is incumbent on us to accelerate the trajectory and talk about content and quality for postnatal care within continuing efforts to increase coverage and equity. The time to unpack the “black box” of postnatal care services delivery is now and greater attention should be provided to quality postnatal care in research policy and practice. The Global Strategy for Women’s Children’s and Adolescents’ Health 2016-2030 aims to help people survive thrive and transform; thus the new era cannot only be about survival but efforts must be made to improve and transform health systems. Quality needs to be understood and addressed if we are serious about reducing neonatal maternal and child mortality; progressing toward universal health coverage; and achieving the SDGs. (Excerpts)

The SPARK Tool to prioritise questions for systematic reviews in health policy and systems research: development and initial validation

BackgroundGroups or institutions funding or conducting systematic reviews in health policy and systems research (HPSR) should prioritise topics according to the needs of policymakers and stakeholders. The aim of this study was to develop and validate a tool to prioritise questions for systematic reviews in HPSR.MethodsWe developed the tool following a four-step approach consisting of (1) the definition of the purpose and scope of tool, (2) item generation and reduction, (3) testing for content and face validity, (4) and pilot testing of the tool. The research team involved international experts in HPSR, systematic review methodology and tool development, led by the Center for Systematic Reviews on Health Policy and Systems Research (SPARK). We followed an inclusive approach in determining the final selection of items to allow customisation to the user’s needs.ResultsThe purpose of the SPARK tool was to prioritise questions in HPSR in order to address them in systematic reviews. In the item generation and reduction phase, an extensive literature search yielded 40 relevant articles, which were reviewed by the research team to create a preliminary list of 19 candidate items for inclusion in the tool. As part of testing for content and face validity, input from international experts led to the refining, changing, merging and addition of new items, and to organisation of the tool into two modules. Following pilot testing, we finalised the tool, with 22 items organised in two modules – the first module including 13 items to be rated by policymakers and stakeholders, and the second including 9 items to be rated by systematic review teams. Users can customise the tool to their needs, by omitting items that may not be applicable to their settings. We also developed a user manual that provides guidance on how to use the SPARK tool, along with signaling questions.ConclusionWe have developed and conducted initial validation of the SPARK tool to prioritise questions for systematic reviews in HPSR, along with a user manual. By aligning systematic review production to policy priorities, the tool will help support evidence-informed policymaking and reduce research waste. We invite others to contribute with additional real-life implementation of the tool.

Effect of a policy to reduce user fees on the rate of skilled birth attendance across socioeconomic strata in Burkina Faso

Background. In Sub-Saharan Africa, maternal and neonatal morbidity and mortality rates are associated with underutilization of skilled birth attendance (SBA). In 2007, Burkina Faso introduced a subsidy scheme for SBA fees. The objective of this study was to evaluate the effect of Burkina Faso’s subsidy policy on SBA rate across socioeconomic status (SES) strata. Methods. We used a quasi-experimental design. The data sources were two representative surveys (n = 1408 and n = 1403) of women from Houndé and Ziniaré health districts of Burkina Faso, and a survey of health centres assessing structural quality of care. Multilevel Poisson regression models were used with robust variance estimators. We estimated adjusted rate ratios (RR) and rate differences (RD) as a function of time and SES. Results. For lowest-SES women, immediately upon the introduction of the subsidy policy, the rate of SBA was 45% higher (RR = 1.45, 95% confidence interval (CI): 1.19–1.77) than expected in the absence of subsidy introduction. The results indicated a sustained effect after introduction of the subsidy policy, based on RR estimate (95% CI) of 1.48 (1.21–1.81) at 2 years. For middle-SES women, the RR estimates were 1.28 (1.09–1.49) immediately after introduction of the subsidy policy and 1.30 (1.11–1.51) at 2 years, respectively. For highest-SES women, the RR estimates were 1.19 (1.02–1.38) immediately after subsidy introduction and 1.21 (1.06–1.38) at 2 years, respectively. The RD (95% CI) was 14% (3–24%) for lowest-SES women immediately after introduction of the policy, and the effect was sustained at 14% (4–25%) at 2 years. Conclusion. Our study suggests that the introduction of a user-fee subsidy in Burkina Faso resulted in increased rates of SBA across all SES strata. The increase was sustained over time and strongest among the poorest women. These findings have important implications for evidence-informed policymaking in Burkina Faso and other countries in Sub-Saharan Africa.

Using media to impact health policy-making: an integrative systematic review

IntroductionMedia interventions can potentially play a major role in influencing health policies. This integrative systematic review aimed to assess the effects of planned media interventions—including social media—on the health policy-making process.MethodsEligible study designs included randomized and non-randomized designs, economic studies, process evaluation studies, stakeholder analyses, qualitative methods, and case studies. We electronically searched Medline, EMBASE, Communication and Mass Media Complete, Cochrane Central Register of Controlled Trials, and the WHO Global Health Library. We followed standard systematic review methodology for study selection, data abstraction, and risk of bias assessment.ResultsTwenty-one studies met our eligibility criteria: 10 evaluation studies using either quantitative (n = 7) or qualitative (n = 3) designs and 11 case studies. None of the evaluation studies were on social media. The findings of the evaluation studies suggest that media interventions may have a positive impact when used as accountability tools leading to prioritizing and initiating policy discussions, as tools to increase policymakers’ awareness, as tools to influence policy formulation, as awareness tools leading to policy adoption, and as awareness tools to improve compliance with laws and regulations. In one study, media-generated attention had a negative effect on policy advocacy as it mobilized opponents who defeated the passage of the bills that the media intervention advocated for. We judged the confidence in the available evidence as limited due to the risk of bias in the included studies and the indirectness of the evidence.ConclusionThere is currently a lack of reliable evidence to guide decisions on the use of media interventions to influence health policy-making. Additional and better-designed, conducted, and reported primary research is needed to better understand the effects of media interventions, particularly social media, on health policy-making processes, and the circumstances under which media interventions are successful.Trial registrationPROSPERO 2015:CRD42015020243

Correction to: Engaging policy-makers, health system managers, and policy analysts in the knowledge synthesis process: a scoping review

Following the publication of the original article [1], it was brought to our attention that the letter ‘l’ was unfortunately omitted from the word ‘health’ in the article’s title.

Evaluation of Communities of Practice performance developing implementation research to enhance maternal health decision-making in Mexico and Nicaragua

BackgroundDespite the existence of proven solutions to improve maternal health, innovative approaches are still necessary to support the implementation of programs and policies in real-world settings. To address this challenge, we developed a participatory capacity strengthening model based on Communities of Practice (CoPs) of maternal health frontline personnel and decision-makers involved in implementing maternal health programs in Latin America. The activities focused on the conduct and use of implementation research to enhance maternal health decision-making. Our objective is to evaluate the performance of the Communities of Practice in using implementation research to support decision-making in maternal health programs in Mexico and Nicaragua.MethodsWe evaluated the CoPs’ performance using a mixed methods approach. We appraised the performance of CoPs using five criteria: (i) integration of the CoP, (ii) ownership of the methodology, (iii) timely delivery of products, (iv) feedback to decision-makers, and (v) influence on program changes. We also included an assessment of the barriers and facilitators to the conduct and uptake of implementation research findings in maternal health decision-making.ResultsTwo CoPs showed “sub-optimal” performance, one was signaled as “needing strengthening,” and three reached “optimal” performance in the use of implementation research to enhance maternal health programs. The relationships between champions, facilitators, and research team were the main internal enabling factor for success. Externally, political and epidemiological environments acted as the main barriers to the performance of CoPs.ConclusionsOur study highlights the value of involving decision-makers in CoPs, ensuring varied skill sets and profiles of health professionals, as well as maintaining strong and continuous collaborations with researchers. Collaborative approaches and meaningful engagement of decision-makers and researchers are useful in conducting implementation research and promoting the use of evidence to improve maternal health programs in resource-strained settings.

Intersectoral approaches and integrated services in achieving the right to health for refugees upon resettlement: a scoping review protocol

Introduction Global insecurity and climate change are exacerbating the need for improved management of refugee resettlement services. International standards hold states responsible for the protection of the right of non-citizens to an adequate standard of physical and mental health while recognising the importance of social determinants of health. However, programmes to protect refugees’ right to health often lack coordination and monitoring. This paper describes the protocol for a scoping review to explore barriers and facilitators to the integration of health services for refugees; the content, process and actors involved in protecting refugee health; and the extent to which intersectoral approaches are leveraged to protect refugees’ right to health on resettlement, especially for vulnerable groups such as women and children. Methods and analysis Peer-reviewed (through four databases including MEDLINE, Web of Science, Global Health and PsycINFO) and grey literature were searched to identify programmes and interventions designed to promote refugee health in receiving countries. Two reviewers will screen articles and abstract data. Two frameworks for integration and intersectoral action will be applied to understand how and why certain approaches work while others do not and to identify the actors involved in achieving success at different levels of integration as defined by these frameworks. Ethics and dissemination Findings from the scoping review will be shared in relevant conferences and meetings. A brief will be created with lessons learnt from successful programmes to inform decision making in design of refugee programmes and services. Ethical approval is not required as human subjects are not involved. Trial registration number Registered on Open Science Framework at https://osf.io/gt9ck/.