Julia Lohmann

Protocol for the process evaluation of interventions combining performance-based financing with health equity in Burkina Faso

BackgroundThe low quality of healthcare and the presence of user fees in Burkina Faso contribute to low utilization of healthcare and elevated levels of mortality. To improve access to high-quality healthcare and equity, national authorities are testing different intervention arms that combine performance-based financing with community-based health insurance and pro-poor targeting. There is a need to evaluate the implementation of these unique approaches. We developed a research protocol to analyze the conditions that led to the emergence of these intervention arms, the fidelity between the activities initially planned and those conducted, the implementation and adaptation processes, the sustainability of the interventions, the possibilities for scaling them up, and their ethical implications.Methods/DesignThe study adopts a longitudinal multiple case study design with several embedded levels of analyses. To represent the diversity of contexts where the intervention arms are carried out, we will select three districts. Within districts, we will select both primary healthcare centers (n =18) representing different intervention arms and the district or regional hospital (n =3). We will select contrasted cases in relation to their initial performance (good, fair, poor). Over a period of 18 months, we will use quantitative and qualitative data collection and analytical tools to study these cases including in-depth interviews, participatory observation, research diaries, and questionnaires. We will give more weight to qualitative methods compared to quantitative methods.DiscussionPerformance-based financing is expanding rapidly across low- and middle-income countries. The results of this study will enable researchers and decision makers to gain a better understanding of the factors that can influence the implementation and the sustainability of complex interventions aiming to increase healthcare quality as well as equity.

Factors associated with delivery outside a health facility: Cross-sectional study in rural Malawi

To identify factors associated with delivery outside a health facility in rural Malawi.

Measuring health workers’ motivation composition: validation of a scale based on Self-Determination Theory in Burkina Faso

BackgroundAlthough motivation of health workers in low- and middle-income countries (LMICs) has become a topic of increasing interest by policy makers and researchers in recent years, many aspects are not well understood to date. This is partly due to a lack of appropriate measurement instruments. This article presents evidence on the construct validity of a psychometric scale developed to measure motivation composition, i.e., the extent to which motivation of different origin within and outside of a person contributes to their overall work motivation. It is theoretically grounded in Self-Determination Theory (SDT).MethodsWe conducted a cross-sectional survey of 1142 nurses in 522 government health facilities in 24 districts of Burkina Faso. We assessed the scale’s validity in a confirmatory factor analysis framework, investigating whether the scale measures what it was intended to measure (content, structural, and convergent/discriminant validity) and whether it does so equally well across health worker subgroups (measurement invariance).ResultsOur results show that the scale measures a slightly modified version of the SDT continuum of motivation well. Measurements were overall comparable between subgroups, but results indicate that caution is warranted if a comparison of motivation scores between groups is the focus of analysis.ConclusionsThe scale is a valuable addition to the repository of measurement tools for health worker motivation in LMICs. We expect it to prove useful in the quest for a more comprehensive understanding of motivation as well as of the effects and potential side effects of interventions intended to enhance motivation.Abstract FrenchContexteLa motivation des agents de santé dans les pays à revenu faible et intermédiaire est devenue un sujet de grand intérêt pour les décideurs et les chercheurs au cours des dernières années. Pourtant, de nombreux aspects de la motivation des agents de santé ne sont pas encore bien compris. Ceci est dû en particulier à l’absence d’outils de mesure appropriés. Cet article présente une preuve de la validité conceptuelle d’une échelle psychométrique développée pour mesurer la composition de la motivation, c’est-à-dire le degré auquel des types de motivation d’origine différente à l’intérieur et à l’extérieur d’une personne contribuent à leur motivation globale au travail. L’échelle est fondée sur la théorie de l’auto-détermination (Self-Determination Theory).MéthodesUne enquête transversale a été mise en place auprès de 1 142 infirmières dans 522 formations sanitaires gouvernementales de 24 districts du Burkina Faso. Par analyse factorielle confirmatoire, nous avons examiné si l’échelle mesure ce qu’elle était supposée mesurer (validité structurelle et convergente/discriminante) et si ses propriétés de mesures sont comparables dans différentes sous-groupes d’agents de santé (invariance de la mesure).RésultatsLes résultats montrent que l’échelle mesure une version légèrement modifiée du continuum de motivation proposée par la théorie de l’auto-détermination. Les propriétés de mesure étaient globalement comparables entre les sous-groupes, mais une certaine prudence est indiquée si une comparaison des moyennes entre les groupes est l’objectif principal de l’analyse.ConclusionL’échelle est une un apport important au référentiel des outils de mesure de la motivation des agents de santé dans les pays à revenu faible et modérés. Elle sera utile pour une meilleure compréhension de la motivation des prestataires, ainsi que des effets positifs et potentiellement secondaires des interventions visant à renforcer la motivation.

Implementation Research to Improve Quality of Maternal and Newborn Health Care, Malawi/Recherche Sur la Mise En Oeuvre, Dans Une Optique D'amelioration De la Qualite Des Soins De Sante Maternelle et Neonatale Au Malawi/Investigaciones Sobre la Ejecucion Para Mejorar la Calidad De la Atencion Sanitaria Materna Y Obstetrica, Malawi

Abstract Objective To evaluate the impact of a performance-based financing scheme on maternal and neonatal health service quality in Malawi. Methods We conducted a non-randomized controlled before and after study to evaluate the effects of district- and facility-level performance incentives for health workers and management teams. We assessed changes in the facilities’ essential drug stocks, equipment maintenance and clinical obstetric care processes. Difference-in-difference regression models were used to analyse effects of the scheme on adherence to obstetric care treatment protocols and provision of essential drugs, supplies and equipment. Findings We observed 33 health facilities, 23 intervention facilities and 10 control facilities and 401 pregnant women across four districts. The scheme improved the availability of both functional equipment and essential drug stocks in the intervention facilities. We observed positive effects in respect to drug procurement and clinical care activities at non-intervention facilities, likely in response to improved district management performance. Birth assistants’ adherence to clinical protocols improved across all studied facilities as district health managers supervised and coached clinical staff more actively. Conclusion Despite nation-wide stock-outs and extreme health worker shortages, facilities in the study districts managed to improve maternal and neonatal health service quality by overcoming bottlenecks related to supply procurement, equipment maintenance and clinical performance. To strengthen and reform health management structures, performance-based financing may be a promising approach to sustainable improvements in quality of health care.

How do Malawian women rate the quality of maternal and newborn care? Experiences and perceptions of women in the central and southern regions

BackgroundWhile perceived quality of care is now widely recognized to influence health service utilization, limited research has been conducted to explore and measure perceived quality of care using quantitative tools. Our objective was to measure women’s perceived quality of maternal and newborn care using a composite scale and to identify individual and service delivery factors associated with such perceptions in Malawi.MethodsWe conducted a cross-sectional survey in selected health facilities from March to May 2013. Exit interviews were conducted with 821 women convenience sampled at antenatal, delivery, and postnatal clinics using structured questionnaires. Experiences and the corresponding perceived quality of care were measured using a composite perception scale based on 27 items, clustered around three dimensions of care: interpersonal relations, conditions of the consultation and delivery rooms, and nursing care services. Statements reflecting the 27 items were read aloud and the women were asked to rate the quality of care received on a visual scale of 1 to 10 (10 being the highest score). For each dimension, an aggregate score was calculated using the un-weighted item means, representing three outcome variables. Descriptive statistics were used to display distribution of explanatory variables and one-way analysis of variance was used to analyse bivariate associations between the explanatory and the outcome variables.ResultsA high perceived quality of care rating was observed on interpersonal relations, conditions of the examination rooms and nursing care services with an overall mean score of 9/10. Self-introduction by the health worker, explanation of examination procedures, consent seeking, encouragement to ask questions, confidentiality protection and being offered to have a guardian during delivery were associated with a high quality rating of interpersonal relations for antenatal and delivery care services. Being literate, never experienced a still birth and, first ANC visit were associated with a high quality rating of room conditions for antenatal care service.ConclusionsThe study highlights some of the multiple factors associated with perceived quality of care. We conclude that proper interventions or practices and policies should consider these factors when making quality improvements.

How to do (or not to do)… Measuring health worker motivation in surveys in low- and middle-income countries.

Abstract A health system’s ability to deliver quality health care depends on the availability of motivated health workers, which are insufficient in many low income settings. Increasing policy and researcher attention is directed towards understanding what drives health worker motivation and how different policy interventions affect motivation, as motivation is key to performance and quality of care outcomes. As a result, there is growing interest among researchers in measuring motivation within health worker surveys. However, there is currently limited guidance on how to conceptualize and approach measurement and how to validate or analyse motivation data collected from health worker surveys, resulting in inconsistent and sometimes poor quality measures. This paper begins by discussing how motivation can be conceptualized, then sets out the steps in developing questions to measure motivation within health worker surveys and in ensuring data quality through validity and reliability tests. The paper also discusses analysis of the resulting motivation measure/s. This paper aims to promote high quality research that will generate policy relevant and useful evidence.

Evaluating complex health financing interventions: using mixed methods to inform further implementation of a novel PBI intervention in rural Malawi

BackgroundGaps remain in understanding how performance-based incentive (PBI) programs affect quality of care and service quantity, whether programs are cost effective and how programs could be tailored to meet client and provider needs while remaining operationally viable. In 2014, Malawi’s Ministry of Health launched the Service Delivery Integration-PBI (SSDI-PBI) program. The program is unique in that no portion of performance bonuses are paid to individual health workers, and it shifts responsibility for infrastructure and equipment procurement from facility staff to implementing partners. This protocol outlines an approach that analyzes processes and outcomes, considers expected and unexpected consequences of the program and frames the program’s outputs relative to its costs. Findings from this evaluation will inform the intended future scale-up of PBI in Malawi.Methods/designThis study employs a prospective controlled before-and-after triangulation design to assess effects of the PBI program by analyzing quantitative and qualitative data from intervention and control facilities. Guided by a theoretical framework, the evaluation consists of four main components: service provision, health worker motivation, implementation processes and costing. Quality and access outcomes are assessed along four dimensions: (1) structural elements (related to equipment, drugs, staff); (2) process elements (providers’ compliance with standards); (3) outputs (service utilization); (4) experiential elements (experiences of service delivery). The costing component includes costs related to start-up, ongoing management, and the cost of incentives themselves. The cost analysis considers costs incurred within the Ministry of Health, funders, and the implementing agency. The evaluation relies on primary data (including interviews and surveys) and secondary data (including costing and health management information system data).DiscussionThrough the lens of a PBI program, we illustrate how complex interventions can be evaluated via not only primary, mixed-methods data collection, but also through a wealth of secondary data from program implementers (including monitoring, evaluation and financial data), and the health system (including service utilization and service readiness data). We also highlight the importance of crafting a theory and using theory to inform the nature of data collected. Finally, we highlight the need to be responsive to stakeholders in order to enhance a study’s relevance.

Perceptions of quality across the maternal care continuum in the context of a health financing intervention: Evidence from a mixed methods study in rural Malawi

BackgroundIn 2013, Malawi with its development partners introduced a Results-Based Financing for Maternal and Newborn Health (RBF4MNH) intervention to improve the quality of maternal and newborn health-care services. Financial incentives are awarded to health facilities conditional on their performance and to women for delivering in the health facility. We assessed the effect of the RBF4MNH on quality of care from women’s perspectives.MethodsWe used a mixed-method prospective sequential controlled pre- and post-test design. We conducted 3060 structured client exit interviews, 36 in-depth interviews and 29 focus group discussions (FGDs) with women and 24 in-depth interviews with health service providers between 2013 and 2015. We used difference-in-differences regression models to measure the effect of the RBF4MNH on experiences and perceived quality of care. We used qualitative data to explore the matter more in depth.ResultsWe did not observe a statistically significant effect of the intervention on women’s perceptions of technical care, quality of amenities and interpersonal relations. However, in the qualitative interviews, most women reported improved health service provision as a result of the intervention. RBF4MNH increased the proportion of women reporting to have received medications/treatment during childbirth. Participants in interviews expressed that drugs, equipment and supplies were readily available due to the RBF4MNH. However, women also reported instances of neglect, disrespect and verbal abuse during the process of care. Providers attributed these negative instances to an increased workload resulting from an increased number of women seeking services at RBF4MNH facilities.ConclusionOur qualitative findings suggest improvements in the availability of drugs and supplies due to RBF4MNH. Despite the intervention, challenges in the provision of quality care persisted, especially with regard to interpersonal relations. RBF interventions may need to consider including indicators that specifically target the provision of respectful maternity care as a means to foster providers’ positive attitudes towards women in labour. In parallel, governments should consider enhancing staff and infrastructural capacity before implementing RBF.

Performance-based financing kick-starts motivational “feedback loop”: findings from a process evaluation in Mozambique

BackgroundPerformance-based financing (PBF) reforms aim to directly influence health worker behavior through changes to institutional arrangements, accountability structures, and financial incentives based on performance. While there is still some debate about whether PBF influences extrinsic or intrinsic motivators, recent research finds that PBF affects both. Against this backdrop, our study presents findings from a process evaluation of a PBF program in Mozambique, exploring the perceived changes to both internal and external drivers of health worker motivation associated with PBF.MethodsWe used a qualitative research design with in-depth, semi-structured interviews with health workers, which included a rank order exercise and focus group discussions. Interviews were analyzed by two researchers using thematic analysis techniques. Rank order frequency was calculated using weighted average methodology.ResultsHealth workers reported that PBF, overall, positively influenced their motivation by introducing or reinforcing both internal and external motivational drivers. Internal drivers included enhanced self-efficacy driven by goal orientation, healthy competition among colleagues, and job satisfaction. External drivers included an organized work environment, enhanced access to equipment and supplies, financial incentives, teamwork, and regular consultations with verifiers (a type of supervision). PBF stimulates an interactive relationship between internal and external motivational drivers, creating a feedback loop involving responsibility, achievement, and recognition, which increased perceived motivation.ConclusionsThe PBF program helped workers feel that they had well-defined and achievable goals and that they received recognition from verification teams, management committees, and colleagues due to enhanced accountability and governance. Our paper shows that financial incentives could serve as the “driver” to kick-start the feedback loop, of responsibility, achievement, and recognition, in environments that lack other drivers. Understanding how PBF programs can be designed and refined to reinforce this feedback loop could be a powerful tool to further enhance and track positive motivational changes. For countries thinking about PBF, we recommend that policymakers assess the loop in their contexts, identify drivers, determine whether these drivers are sufficient, and consider PBF if they are not.Trial registrationWe obtained ethical approval for the study protocol, data collection instruments, and informed consent forms from the Ethics Review Committee of the Centers for Disease Control and Prevention (CDC) [IRB 2015–190] and the Ethics Review Committee of the Mozambique Ministry of Health.

Impact of results-based financing on effective obstetric care coverage: evidence from a quasi-experimental study in Malawi

BackgroundResults-based financing (RBF) describes health system approaches addressing both service quality and use. Effective coverage is a metric measuring progress towards universal health coverage (UHC). Although considered a means towards achieving UHC in settings with weak health financing modalities, the impact of RBF on effective coverage has not been explicitly studied.MethodsMalawi introduced the Results-Based Financing For Maternal and Neonatal Health (RBF4MNH) Initiative in 2013 to improve quality of maternal and newborn health services at emergency obstetric care facilities. Using a quasi-experimental design, we examined the impact of the RBF4MNH on both crude and effective coverage of pregnant women across four districts during the two years following implementation.ResultsThere was no effect on crude coverage. With a larger proportion of women in intervention areas receiving more effective care over time, the overall net increase in effective coverage was 7.1%-points (p = 0.07). The strongest impact on effective coverage (31.0%-point increase, p = 0.02) occurred only at lower cut-off level (60% of maximum score) of obstetric care effectiveness. Design-specific and wider health system factors likely limited the program’s potential to produce stronger effects.ConclusionThe RBF4MNH improved effective coverage of pregnant women and seems to be a promising reform approach towards reaching UHC. Given the short study period, the full potential of the current RBF scheme has likely not yet been reached.