Enora Robin

Evaluating a complex research capacity-buildingintervention: reflections on an evaluation of the AfricanInstitutions Initiative

There is increasing policy demand for real-time evaluations of research and capacity-building programmes reflecting a recognition of the management, governance and impact gains that can result. However, the evidence base on how to successfully implement real-time evaluations of complex interventions in international development efforts is scarce. There is therefore a need for reflective work that considers methodologies in context. This article shares learning from the experience of conducting a participatory, real-time, ‘theory driven’ evaluation of the African Institutions Initiative, a Wellcome Trust-funded programme that aimed to build sustainable health research capacity in Africa at institutional and network levels, across seven research consortia. We reflect on the key challenges experienced and ways of managing them, highlight opportunities and critical success factors associated with this evaluation approach, compared with alternative evaluation approaches.

058: EVALUATING A COMPLEX CAPACITY-BUILDING INTERVENTION IN AN INTERNATIONAL DEVELOPMENT CONTEXT: REFLECTING ON THE AFRICAN INSTITUTIONS INITIATIVE EVALUATION

Background In the past decade, global health funders, non-government organisations and policymakers have increased their efforts to support health research capacity in developing low and middle income countries (LMICs). These efforts are aimed at securing the provision of—and eventually access to—high-quality health services. The Wellcome Trusts African Institutions Initiative (AII), launched in 2009 aimed to broaden the research base for scientific endeavour in under-resourced environments; to support areas of science with the potential to contribute to health benefits for people and livestock; and to support international networks and partnerships focused on health problems of resource-poor countries. Objectives We present findings from an independent real-time evaluation of the first four years of the Initiative, from 2009–2013. The evaluation project aims to address the gaps in our understanding of effective capacity building initiatives and to help inform the next phase of the Initiative. Methods Evaluation in real time – during a programmes life as opposed to at the end of it – is particularly suitable for the kinds of complex interventions and uncertain contexts reflected in this Initiative. With real-time evaluation, on-going learning can be optimised to inform programme implementation. This evaluation adopted an approach which mirrored the participatory, African-led ethos of the overall Initiative. It worked with the funding recipients to evaluate them against their own articulated and documented aims and objectives. Result The AII was not designed to deliver “quick wins”. Rather it plans to lay the foundations for increased research capacity and the emergence of locally relevant health research agendas over time, which is reflected in its direct engagement with African universities and research institutes to develop Africa-led research programmes. This being said, programme-wide achievements during the first phase of funding are evident.The AII has successfully supported active networks amongst African institutions and between African and Northern partners. Networks present a cost-effective approach to supporting many institutions with limited resources. A fundamental component of the Initiatives intervention logic is the notion that collaborative networks, through a consortium-based model, can maximise potential impacts of investments in capacity-building activities. Conclusion Overall, the objectives of the AII represent an ambitious departure from traditional modes of strengthening research capacity in LMICs. While the Initiative is still at an early phase, evidence gathered in this evaluation show that consortia are contributing in multiple ways to developing sustainable research capacity. The unique features of the Initiative—an African-led, networked approach—have facilitated many of these achievements. The diversity of networking models for capacity building developed by consortia present a range of models with different strengths and weaknesses, enabling funders to reflect on future models.