Annette N. Brown

A Review of the Evidence of Harm from Self-Tests

Although HIV self-testing may overcome some barriers to HIV testing, various stakeholders have expressed concerns that HIV self-testing may lead to unintended harm, including psychological, social and medical harm. Recognizing that similar concerns were raised in the past for some other self-tests, we conduct a review of the literature on a set of self-tests that share some characteristics with HIV self-tests to determine whether there is any evidence of harm. We find that although the potential for harm is discussed in the literature on self-tests, there is very little evidence that such harm occurs.

The growth of impact evaluation for international development: how much have we learned?

This article examines the content of a web-based repository of published impact evaluations of international development interventions. To populate this repository, we conducted a systematic search and screening process. We find that of the 2259 studies published from 1981 to 2012, annual publication increased dramatically after 2008. Most studies are on health, education, social protection and agriculture and are concentrated in South Asia, East Africa, South and Central America and Southeast Asia. There are statistically significant differences in time between end line data collection and publishing by the publication type, and institutional affiliation of authors has shifted towards countries in North America and Europe.

Quasi-experimental study designs series-paper 4: uses and value.

Quasi-experimental studies are increasingly used to establish causal relationships in epidemiology and health systems research. Quasi-experimental studies offer important opportunities to increase and improve evidence on causal effects: (1) they can generate causal evidence when randomized controlled trials are impossible; (2) they typically generate causal evidence with a high degree of external validity; (3) they avoid the threats to internal validity that arise when participants in nonblinded experiments change their behavior in response to the experimental assignment to either intervention or control arm (such as compensatory rivalry or resentful demoralization); (4) they are often well suited to generate causal evidence on long-term health outcomes of an intervention, as well as nonhealth outcomes such as economic and social consequences; and (5) they can often generate evidence faster and at lower cost than experiments and other intervention studies.

Quality evidence for policymaking: I'll believe it when I see the replication

In this paper, we make the case for replication as a crucial methodology for validating research used for evidence-based policymaking, especially in low- and middle-income countries. We focus on internal replication or the reanalysis of original data to address an original evaluation or research question. We review the current state of replication in the social sciences and present data on the trends among academic publications. We then discuss four challenges facing empirical research that internal replication can help to address. We offer a new typology of replication approaches for addressing these challenges. The types – pure replication, measurement and estimation analysis, and theory of change analysis – highlight that internal replication can test for consistency and statistical robustness but can and should also be used to ensure that a study fully explores possible theories of change in order to draw appropriate conclusions and recommendations for policymaking and programme design.

Strength of Evidence on Demand Creation for Voluntary Medical Male Circumcision From 7 Impact Evaluations in Southern and Eastern Africa.

Background:Seven new impact evaluations of pilot programs for increasing the demand for voluntary medical male circumcision (VMMC) provide evidence of what works and what does not. The study findings suggest that financial compensation designed to relieve the opportunity or transportation costs from undergoing the procedure can increase the uptake of VMMC. There is also evidence that programs using peer influence can be effective, although so far only sports-based programs demonstrate a strong effect. We explore the strength of evidence in each of these 7 studies to better interpret the findings for policy making. Methods:We perform a risk of bias assessment and conduct power calculations using actual values for each of the 7 studies. Results:Three of the 7 studies have a medium risk of bias, whereas the other 4 have a low risk of bias. All but 2 of the studies have adequate power to detect meaningful effects. In the 2 with insufficient power, the estimated effects are large but statistically insignificant. Conclusion:The positive evidence that financial incentives presented as compensation for opportunity costs to men seeking and obtaining VMMC can increase uptake comes from strong studies, which have high power and low to medium risk of bias. The positive evidence that a comprehensive sports-based program for young men can increase uptake also comes from a strong study. The strength of the studies further validates these findings.

Replication Studies of Development Impact Evaluations

Abstract Six years ago, the International Initiative for Impact Evaluation (3ie) launched a programme to promote and fund replication studies of impact evaluations in international development. We designed the programme with the objective of improving the quality of evidence for development policy-making, using replication research to both validate the results of published impact evaluations and build the incentives for more transparent and high quality research going forward. The programme’s focus is internal replication, which uses the original data from a study to address the same question as that study. This Journal of Development Studies special issue compiles the majority of completed 3ie-funded replication studies initiated in the first years of the programme. In all cases the pure replication components of these studies are generally able to reproduce the results published in the original article. Most of the measurement and estimation analyses confirm the robustness of the original articles or call into question just a subset of the original findings. These replication studies mostly focus on providing additional information about the impacts of the interventions – especially additional information that can be important for interpreting the articles for the purpose of policy-making.

Is impact evaluation still on the rise? The new trends in 2010–2015: For submission to the Journal of Development Effectiveness

ABSTRACT This article examines the content of a web-based repository of published international development impact evaluations created by the International Initiative for Impact Evaluation (3ie) in 2012. In 2015, the authors of this article along with a team of reviewers at 3ie used an updated search and screening protocol to identify new impact evaluation studies published until September 2015. By using systematic search and screening techniques to populate the repository, which contains 4,205 development impact evaluations published between 1981 and September 2015, we can use the data to analyse the trends in impact evaluation research. Though we find early evidence of a plateau in the growth rate of development impact evaluations, the number of studies published between January 2010 and September 2015 account for almost two thirds of the total evidence base. Over half of all studies fall under health and education sectors, though we see in the current decade an emergence of studies in formerly unrepresented sectors. While development impact evaluations are concentrated in Sub-Saharan Africa and Latin America (60%), studies are increasingly conducted in underrepresented regions such as the Middle East and North Africa.