Jean-Christophe Chiêm

Looking at the effects of performance-based financing through a complex adaptive systems lens

The debate on PBF is misdirected. As is too often the case in international aid financing, agencies try to prove the effectiveness of their contribution by isolating it as the main reason for success.1 In reaction, opponents will often use the same approach in an attempt to prove that another factor is actually the cause of an observed change. We argue that this endless and futile debate, often present among experts in health systems strengthening, will not contribute to improving public health in low-income countries. Rather than searching for the impossible proof of whether PBF works or not, we should instead try to learn useful lessons from experiences. We agree with Ireland et al. that the focus of PBF assessment should be on “why” and “how” the intervention works.2 Comprehensive evaluation of PBF is needed as part of complete health system reform. We think that, to respond to some of these key questions, health systems should be analysed using a complex adaptive systems lens, as others have advocated in the past.3,4 A complex adaptive system is a collection of interacting components, each of which has its own rules and responsibilities. The behaviour of this kind of system is different to the sum of the behaviour of each of its components. Examples of complex adaptive systems include the human brain, ecosystems and manufacturing businesses. Health system “behaviour” and particularly counterintuitive behaviour (unexpected changes or lack of change) can be analysed using a complex adaptive systems lens when PBF is introduced, often with a mix of other interventions such as in a context of system reform. The purpose of this analysis is not to isolate causal factors but rather to identify “macro” characteristics of the system that may explain behaviour change. Although it has often been ignored in health system evaluation, social simulation can be useful for this approach. The most frequently used technique, agent-based modelling, uses computer simulation centred on a collection of autonomous agents whose interactions are based on a set of rules. These simulations can integrate empirical data or existing knowledge or opinions.5 One of the powerful features of agent-based modelling lies in its capacity to study complex phenomena in a simple and flexible way. Indeed, this approach does not require a high level of mathematical or programming skills, making it accessible to many researchers. Furthermore, it allows for an iterative learning process that is easy to set up compared to long and costly data collection processes. While this methodological approach may not “prove” the effectiveness of an intervention, it could provide insight into the reason a health system behaves in a given way (whether it changes or remains in a steady-state) when PBF is introduced. We believe that this type of information, although maybe less appealing to the usual stakeholders in development aid debates, is much more useful in evaluating PBF.

Agent-Based Modelling to Inform Health Intervention Strategies: The Case of Severe Acute Malnutrition in Children in High-Burden Low-Income Countries

Health interventions improve the management of severe acute malnutrition (SAM) in children under 5 in high-burden low-income countries. However, evaluation of their implementation faces a paucity of information and could benefit from a system perspective derived from the knowledge of implementers and experts. These challenges could be addressed using simulation modelling. We compared Markov and agent-based models of interventions for improving the management of SAM and assessed benefits and limitations in informing complex health intervention strategy designs. Based on a conceptual framework developed with existing evidence and expert advice, the agent-based model generated simulated data representing the complex evolution of the system. Multiple scenarios were investigated by varying parameters and mimicking rules of interventions. This study pointed out possible synergies between interventions enhancing early start of treatment and increasing recovery from SAM. When these interventions were adequately combined, outcomes of coverage, recovery and overall survival improved. Benefits of agent-based modelling were use of history, if-then rules to uncover mechanisms behind probabilities, and modifiable transition rates. Limitations related to model validation, choices of assumptions, and simplification. Agent-based modelling could be used to adapt intervention strategies to local contexts and support scale-up. As such, modelling could complement the methodological toolkit of health intervention strategy designs for improved policy decision.