Bruno Marchal

Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research

This overview aims to stimulate conceptual and practical discussions to help unlock the full potential of realist evaluation in health systems research. Based on a structured literature search, this review maps how the concepts of realist evaluation are applied in health systems research and which methodological problems are encountered. We found a great diversity in the depth of application of the philosophical concepts, use of terminology and scope of application in the research process. Terms of theory-driven evaluation, theories of change and realist evaluation are often used interchangeably. Diverging views exist regarding the nature of ‘mechanism’ and the difference between mechanism and essential context condition. A lack of methodological guidance was highlighted by many authors. Realist evaluation is slowly gaining traction in health systems research, but more clarity is needed concerning the definitions of mechanisms and context and how the configuration of context, mechanism and intervention can be described and assessed.

Global Health Actors Claim To Support Health System Strengthening—Is This Reality or Rhetoric?

Bruno Marchal and colleagues argue that most current strategies aimed at health systems strengthening remain selectively targeted at specific diseases.

Viewpoint: HIV/AIDS and the health workforce crisis: What are the next steps?

In scaling up antiretroviral treatment (ART), financing is fast becoming less of a constraint than the human resources to ensure the implementation of the programmes. In the countries hardest affected by the acquired immunodeficiency syndrome (AIDS) pandemic, AIDS increases workloads, professional frustration and burn‐out. It affects health workers also directly, contributing to rising sick leave and attrition rates. This burden is shouldered by a health workforce weakened already by chronic deficiencies in training, distribution and retention. In these countries, health workforce issues can no longer be analysed from the traditional perspective of human resource development, but should start from the position that entire societies are in a process of social involution of a scale unprecedented in human history. Strategies that proved to be effective and correct in past conditions need be reviewed, particularly in the domains of human resource management and policy‐making, education and international aid. True paradigm shifts are thus required, without which the fundamental changes required to effectively strengthen the health workforce are unlikely to be initiated.

Health systems frameworks in their political context: framing divergent agendas

BackgroundDespite the mounting attention for health systems and health systems theories, there is a persisting lack of consensus on their conceptualisation and strengthening. This paper contributes to structuring the debate, presenting landmarks in the development of health systems thinking against the backdrop of the policy context and its dominant actors. We argue that frameworks on health systems are products of their time, emerging from specific discourses. They are purposive, not neutrally descriptive, and are shaped by the agendas of their authors.DiscussionThe evolution of thinking over time does not reflect a progressive accumulation of insights. Instead, theories and frameworks seem to develop in reaction to one another, partly in line with prevailing paradigms and partly as a response to the very different needs of their developers. The reform perspective considering health systems as projects to be engineered is fundamentally different from the organic view that considers a health system as a mirror of society. The co-existence of health systems and disease-focused approaches indicates that different frameworks are complementary but not synthetic.The contestation of theories and methods for health systems relates almost exclusively to low income countries. At the global level, health system strengthening is largely narrowed down to its instrumental dimension, whereby well-targeted and specific interventions are supposed to strengthen health services and systems or, more selectively, specific core functions essential to programmes. This is in contrast to a broader conceptualization of health systems as social institutions.SummaryHealth systems theories and frameworks frame health, health systems and policies in particular political and public health paradigms. While there is a clear trend to try to understand the complexity of and dynamic relationships between elements of health systems, there is also a demand to provide frameworks that distinguish between health system interventions, and that allow mapping with a view of analysing their returns. The choice for a particular health system model to guide discussions and work should fit the purpose. The understanding of the underlying rationale of a chosen model facilitates an open dialogue about purpose and strategy.

How to develop a theory-driven evaluation design? Lessons learned from an adolescent sexual and reproductive health programme in West Africa.

BackgroundThis paper presents the development of a study design built on the principles of theory-driven evaluation. The theory-driven evaluation approach was used to evaluate an adolescent sexual and reproductive health intervention in Mali, Burkina Faso and Cameroon to improve continuity of care through the creation of networks of social and health care providers.Methods/designBased on our experience and the existing literature, we developed a six-step framework for the design of theory-driven evaluations, which we applied in the ex-post evaluation of the networking component of the intervention. The protocol was drafted with the input of the intervention designer. The programme theory, the central element of theory-driven evaluation, was constructed on the basis of semi-structured interviews with designers, implementers and beneficiaries and an analysis of the interventions logical framework.DiscussionThe six-step framework proved useful as it allowed for a systematic development of the protocol. We describe the challenges at each step. We found that there is little practical guidance in the existing literature, and also a mix up of terminology of theory-driven evaluation approaches. There is a need for empirical methodological development in order to refine the tools to be used in theory driven evaluation. We conclude that ex-post evaluations of programmes can be based on such an approach if the required information on context and mechanisms is collected during the programme.

Research, evidence and policymaking: the perspectives of policy actors on improving uptake of evidence in health policy development and implementation in Uganda

BackgroundUse of evidence in health policymaking plays an important role, especially in resource-constrained settings where informed decisions on resource allocation are paramount. Several knowledge translation (KT) models have been developed, but few have been applied to health policymaking in low income countries. If KT models are expected to explain evidence uptake and implementation, or lack of it, they must be contextualized and take into account the specificity of low income countries for example, the strong influence of donors. The main objective of this research is to elaborate a Middle Range Theory (MRT) of KT in Uganda that can also serve as a reference for other low- and middle income countries.MethodsThis two-step study employed qualitative approaches to examine the principal barriers and facilitating factors to KT. Step 1 involved a literature review and identification of common themes. The results informed the development of the initial MRT, which details the facilitating factors and barriers to KT at the different stages of research and policy development. In Step 2, these were further refined through key informant interviews with policymakers and researchers in Uganda. Deductive content and thematic analysis was carried out to assess the degree of convergence with the elements of the initial MRT and to identify other emerging issues.ResultsReview of the literature revealed that the most common emerging facilitating factors could be grouped under institutional strengthening for KT, research characteristics, dissemination, partnerships and political context. The analysis of interviews, however, showed that policymakers and researchers ranked institutional strengthening for KT, research characteristics and partnerships as the most important. New factors emphasized by respondents were the use of mainstreamed structures within MoH to coordinate and disseminate research, the separation of roles between researchers and policymakers, and the role of the community and civil society in KT.ConclusionsThis study refined an initial MRT on KT in policymaking in the health sector in Uganda that was based on a literature review. It provides a framework that can be used in empirical research of the process of KT on specific policy issues.

The contribution of international health volunteers to the health workforce in sub-Saharan Africa

BackgroundIn this paper, we aim to quantify the contribution of international health volunteers to the health workforce in sub-Saharan Africa and to explore the perceptions of health service managers regarding these volunteers.MethodsRapid survey among organizations sending international health volunteers and group discussions with experienced medical officers from sub-Saharan African countries.ResultsWe contacted 13 volunteer organizations having more than 10 full-time equivalent international health volunteers in sub-Saharan Africa and estimated that they employed together 2072 full-time equivalent international health volunteers in 2005. The numbers sent by secular non-governmental organizations (NGOs) is growing, while the number sent by development NGOs, including faith-based organizations, is mostly decreasing. The cost is estimated at between US

Neglected tropical disease (NTD) control in health systems: the interface between programmes and general health services.

36 000 and US

Advancing the application of systems thinking in health: a realist evaluation of a capacity building programme for district managers in Tumkur, India

50 000 per expatriate volunteer per year. There are trends towards more employment of international health volunteers from low-income countries and of national medical staff.Country experts express more negative views about international health volunteers than positive ones. They see them as increasingly paradoxical in view of the existence of urban unemployed doctors and nurses in most countries. Creating conditions for employment and training of national staff is strongly favoured as an alternative. Only in exceptional circumstances is sending international health volunteers viewed as a defendable temporary measure.ConclusionWe estimate that not more than 5000 full-time equivalent international health volunteers were working in sub-Saharan Africa in 2005, of which not more than 1500 were doctors. A distinction should be made between (1) secular medical humanitarian NGOs, (2)development NGOs, and (3) volunteer organizations, as Voluntary Service Overseas (VSO) and United Nations volunteers (UNV). They have different views, undergo different trends and are differently appreciated by government officials.International health volunteers contribute relatively small numbers to the health workforce in sub-Saharan Africa, and it seems unlikely that they will do more in the future. In areas where they play a role, their contribution to service delivery is sometimes very significant.

Protocol: a realist review of user fee exemption policies for health services in Africa

Disease control programmes are an intrinsic part of health systems. Neglected tropical disease (NTD) control is a clear case in point. While there is a growing consensus that NTD control and health services are linked, with important mutual impacts, little is known of what actually happens at the interface between the two entities. Here, we review the rationale, viewpoints and experiences of NTD control programmes regarding general health services, and vice versa, and compare their respective arguments. We discuss the interactions and interface between disease control and health systems, and present possible scenarios for health system strengthening by NTD- and other disease-specific programmes. Focusing on countries in sub-Saharan Africa, we suggest a number of principles that could pave the way for fruitful discussions and development of synergies.