Ranjeeta Thomas

Evaluating innovative health programs

Identifying innovative interventions that meet critical health policy needs in local settings is likely to benefit from local knowledge. The Global Development Network’s (GDN) project ‘Evaluating Innovative Health Programs’ (EIHP) project is built on the ability of local researchers to identify such solutions. It evaluates the impact of 19 programs from across developing and transition countries that focus on the health-related Millennium Development Goals (MDGs) of reducing child and maternal mortality, and halting and reversing the trend of communicable diseases such as HIV/AIDS, malaria, and other diseases (United Nations, 2008). Local researchers are often best placed to identify innovative solutions to the problem faced in their countries. That said, identifying the impact of these interventions requires rigorous evaluations. The credibility of the findings depends on the quality of the evaluations. The research skills required to carry out these evaluations are not always available in these countries. Building long-term evaluation capacity is a key component of this project and has advantages both in terms of increasing the quality of the evaluations carried out under this project and also in encouraging future evaluations. To promote capacity building, a panel of international experts in impact evaluation was identified to act as mentors. Each research team was paired with a mentor through the duration of the evaluation. The panel of experts was drawn from leading academic institutions. The mentors provided active guidance in the design of the evaluation and the methods used. They facilitated transfer of knowledge as well as ensuring evaluations met rigorous methodological standards. African and Asian policymakers from health and related ministries and practitioners from NGOs were involved in the project from its inception. A policymakers/practitioners panel, comprising African and Asian policymakers and practitioners, is a key element of the project that aims at ensuring relevance of the interventions and outcomes being evaluated.

Departures from Cost-Effectiveness Recommendations: The Impact of Health System Constraints on Priority Setting

Abstract—The methods and application of cost-effectiveness analysis have reached an advanced stage of development. Many decision makers consider cost-effectiveness analysis (CEA) to be a valid and feasible approach toward setting health priorities, and it has been extensively applied in evaluating interventions and developing evidence-based clinical guidelines. However, the recommendations arising from cost-effectiveness analysis are often not implemented as intended. A fundamental reason for the failure to implement is that CEA assumes a single constraint, in the form of the budget constraint, whereas in reality decision makers may be faced with numerous other constraints. The objective of this article is to develop a typology of constraints that may act as barriers to implementation of cost-effectiveness recommendations. Six categories of constraints are considered: the design of the health system; costs of implementing change; system interactions between interventions; uncertainty in estimates of costs and benefits; weak governance; and political constraints. Where possible—and if applicable—for each class of constraint, the article discusses ways in which these constraints can be taken into account by a decision maker wishing to pursue the principles of cost-effectiveness.

Decision rules for allocation of finances to health systems strengthening.

A key dilemma in global health is how to allocate funds between disease-specific “vertical projects” on the one hand and “horizontal programmes” which aim to strengthen the entire health system on the other. While economic evaluation provides a way of approaching the prioritisation of vertical projects, it provides less guidance on how to prioritise between horizontal and vertical spending. We approach this problem by formulating a mathematical program which captures the complementary benefits of funding both vertical projects and horizontal programmes. We show that our solution to this math program has an appealing intuitive structure. We illustrate our model by computationally solving two specialised versions of this problem, with illustrations based on the problem of allocating funding for infectious diseases in sub-Saharan Africa. We conclude by reflecting on how such a model may be developed in the future and used to guide empirical data collection and theory development.

Impact and Cost-Effectiveness of Point-Of-Care CD4 Testing on the HIV Epidemic in South Africa

Rapid diagnostic tools have been shown to improve linkage of patients to care. In the context of infectious diseases, assessing the impact and cost-effectiveness of such tools at the population level, accounting for both direct and indirect effects, is key to informing adoption of these tools. Point-of-care (POC) CD4 testing has been shown to be highly effective in increasing the proportion of HIV positive patients who initiate ART. We assess the impact and cost-effectiveness of introducing POC CD4 testing at the population level in South Africa in a range of care contexts, using a dynamic compartmental model of HIV transmission, calibrated to the South African HIV epidemic. We performed a meta-analysis to quantify the differences between POC and laboratory CD4 testing on the proportion linking to care following CD4 testing. Cumulative infections averted and incremental cost-effectiveness ratios (ICERs) were estimated over one and three years. We estimated that POC CD4 testing introduced in the current South African care context can prevent 1.7% (95% CI: 0.4% - 4.3%) of new HIV infections over 1 year. In that context, POC CD4 testing was cost-effective 99.8% of the time after 1 year with a median estimated ICER of US

Methods for Evaluating Innovative Health Programs: a multi-country study

4,468/DALY averted. In healthcare contexts with expanded HIV testing and improved retention in care, POC CD4 testing only became cost-effective after 3 years. The results were similar when, in addition, ART was offered irrespective of CD4 count, and CD4 testing was used for clinical assessment. Our findings suggest that even if ART is expanded to all HIV positive individuals and HIV testing efforts are increased in the near future, POC CD4 testing is a cost-effective tool, even within a short time horizon. Our study also illustrates the importance of evaluating the potential impact of such diagnostic technologies at the population level, so that indirect benefits and costs can be incorporated into estimations of cost-effectiveness.

A health intervention or a kitchen appliance? Household costs and benefits of a cleaner burning biomass-fuelled cookstove in Malawi

Designed as a global research initiative, the Evaluating Innovative Health Programs project aims at adding to the evidence base of health interventions that have the potential to improve health outcomes in Africa and Asia. The project focuses on rigorous, quantitative evaluations of innovative local initiatives that address the Millennium Development Goals for health: reductions in child and maternal mortality and communicable diseases. This overview brings together the outcomes and lessons from the project for evaluation methods. It draws together the methodological implications of carrying out impact evaluations under very different settings and emphasises the need to build evaluations into project designs.

CONDITIONAL CASH TRANSFERS TO IMPROVE EDUCATION AND HEALTH: AN EX ANTE EVALUATION OF RED DE PROTECCIÓN SOCIAL, NICARAGUA

Pneumonia is the leading cause of mortality for children under five years in sub-Saharan Africa. Household air pollution has been found to increase risk of pneumonia, especially due to exposure from dirty burning biomass fuels. It has been suggested that advanced stoves, which burn fuel more efficiently and reduce smoke emissions, may help to reduce household air pollution in poor, rural settings. This qualitative study aims to provide an insight into the household costs and perceived benefits from use of the stove in Malawi. It was conducted alongside The Cooking and Pneumonia Study (CAPS), the largest village cluster-level randomised controlled trial of an advanced combustion cookstove intervention to prevent pneumonia in children under five to date. In 2015, using 100 semi-structured interviews this study assessed household time use and perceptions of the stove from both control and intervention participants taking part in the CAPS trial in Chilumba. Household direct and indirect costs associated with the intervention were calculated. Users overwhelming liked using the stove. The main reported benefits were reduced cooking times and reduced fuel consumption. In most interviews, the health benefits were not initially identified as advantages of the stove, although when prompted, respondents stated that reduced smoke emissions contributed to a reduction in respiratory symptoms. The cost of the stove was much higher than most respondents said they would be willing to pay. The stoves were not primarily seen as health products. Perceptions of limited impact on health was subsequently supported by the CAPS trial data which showed no significant effect on pneumonia. While the findings are encouraging from the perspective of acceptability, without innovative financing mechanisms, general uptake and sustained use of the stove may not be possible in this setting. The findings also raise the question of whether the stoves should be marketed and championed as ‘health interventions’.

Differences in health-related quality of life between HIV-positive and HIV-negative people in Zambia and South Africa: a cross-sectional baseline survey of the HPTN 071 (PopART) trial

Using baseline data from a randomized experiment, this article extends and tests in the context of health, the feasibility of a recently proposed reduced form approach to ex ante evaluations of social programs with an application to a conditional cash transfer program in Nicaragua. It uses a behavioural model to estimate the impact on preventive care utilization outcomes for children younger than 3 years. It validates the model with the results of the experiment and then simulates two alternate policy scenarios. The model performs well in predicting the health related outcomes and shows different results for the two sets of policy scenarios. In addition, simulations are also carried out for the school component of the cash transfer program.

Wealth Differentials in the Impact of Conditional and Unconditional Cash Transfers on Education: Findings from a Community-Randomised Controlled Trial in Zimbabwe

Summary Background The life expectancy of HIV-positive individuals receiving antiretroviral therapy (ART) is approaching that of HIV-negative people. However, little is known about how these populations compare in terms of health-related quality of life (HRQoL). We aimed to compare HRQoL between HIV-positive and HIV-negative people in Zambia and South Africa. Methods As part of the HPTN 071 (PopART) study, data from adults aged 18–44 years were gathered between Nov 28, 2013, and March 31, 2015, in large cross-sectional surveys of random samples of the general population in 21 communities in Zambia and South Africa. HRQoL data were collected with a standardised generic measure of health across five domains. We used β-distributed multivariable models to analyse differences in HRQoL scores between HIV-negative and HIV-positive individuals who were unaware of their status; aware, but not in HIV care; in HIV care, but who had not initiated ART; on ART for less than 5 years; and on ART for 5 years or more. We included controls for sociodemographic variables, herpes simplex virus type-2 status, and recreational drug use. Findings We obtained data for 19 750 respondents in Zambia and 18 941 respondents in South Africa. Laboratory-confirmed HIV status was available for 19 330 respondents in Zambia and 18 004 respondents in South Africa; 4128 (21%) of these 19 330 respondents in Zambia and 4012 (22%) of 18 004 respondents in South Africa had laboratory-confirmed HIV. We obtained complete HRQoL information for 19 637 respondents in Zambia and 18 429 respondents in South Africa. HRQoL scores did not differ significantly between individuals who had initiated ART more than 5 years previously and HIV-negative individuals, neither in Zambia (change in mean score −0·002, 95% CI −0·01 to 0·001; p=0·219) nor in South Africa (0·000, −0·002 to 0·003; p=0·939). However, scores did differ between HIV-positive individuals who had initiated ART less than 5 years previously and HIV-negative individuals in Zambia (−0·006, 95% CI −0·008 to −0·003; p<0·0001). A large proportion of people with clinically confirmed HIV were unaware of being HIV-positive (1768 [43%] of 4128 people in Zambia and 2026 [50%] of 4012 people in South Africa) and reported good HRQoL, with no significant differences from that of HIV-negative people (change in mean HRQoL score −0·001, 95% CI −0·003 to 0·001, p=0·216; and 0·001, −0·001 to 0·001, p=0·997, respectively). In South Africa, HRQoL scores were lower in HIV-positive individuals who were aware of their status but not enrolled in HIV care (change in mean HRQoL −0·004, 95% CI −0·01 to −0·001; p=0·010) and those in HIV care but not on ART (−0·008, −0·01 to −0·004; p=0·001) than in HIV-negative people, but the magnitudes of difference were small. Interpretation ART is successful in helping to reduce inequalities in HRQoL between HIV-positive and HIV-negative individuals in this general population sample. These findings highlight the importance of improving awareness of HIV status and expanding ART to prevent losses in HRQoL that occur with untreated HIV progression. The gains in HRQoL after individuals initiate ART could be substantial when scaled up to the population level. Funding National Institute of Allergy and Infectious Diseases, National Institute on Drug Abuse, National Institute of Mental Health, Presidents Emergency Plan for AIDS Relief, International Initiative for Impact Evaluation, the Bill & Melinda Gates Foundation.

Allocation rules for global donors

Abstract We investigated (1) how household wealth affected the relationship between conditional cash transfers (CCT) and unconditional cash transfers (UCT) and school attendance, (2) whether CCT and UCT affected educational outcomes (repeating a year of school), (3) if baseline school attendance and transfer conditions affected how much of the transfers participants spent on education and (4) if CCT or UCT reduced child labour in recipient households. Data were analysed from a cluster-randomized controlled trial of CCT and UCT in 4043 households from 2009 to 2010. Recipient households received