Peter C. Rockers

Inclusion of quasi-experimental studies in systematic reviews of health systems research

Systematic reviews of health systems research commonly limit studies for evidence synthesis to randomized controlled trials. However, well-conducted quasi-experimental studies can provide strong evidence for causal inference. With this article, we aim to stimulate and inform discussions on including quasi-experiments in systematic reviews of health systems research. We define quasi-experimental studies as those that estimate causal effect sizes using exogenous variation in the exposure of interest that is not directly controlled by the researcher. We incorporate this definition into a non-hierarchical three-class taxonomy of study designs - experiments, quasi-experiments, and non-experiments. Based on a review of practice in three disciplines related to health systems research (epidemiology, economics, and political science), we discuss five commonly used study designs that fit our definition of quasi-experiments: natural experiments, instrumental variable analyses, regression discontinuity analyses, interrupted times series studies, and difference studies including controlled before-and-after designs, difference-in-difference designs and fixed effects analyses of panel data. We further review current practices regarding quasi-experimental studies in three non-health fields that utilize systematic reviews (education, development, and environment studies) to inform the design of approaches for synthesizing quasi-experimental evidence in health systems research. Ultimately, the aim of any review is practical: to provide useful information for policymakers, practitioners, and researchers. Future work should focus on building a consensus among users and producers of systematic reviews regarding the inclusion of quasi-experiments.

Quasi-experimental study designs series-paper 4: uses and value.

Quasi-experimental studies are increasingly used to establish causal relationships in epidemiology and health systems research. Quasi-experimental studies offer important opportunities to increase and improve evidence on causal effects: (1) they can generate causal evidence when randomized controlled trials are impossible; (2) they typically generate causal evidence with a high degree of external validity; (3) they avoid the threats to internal validity that arise when participants in nonblinded experiments change their behavior in response to the experimental assignment to either intervention or control arm (such as compensatory rivalry or resentful demoralization); (4) they are often well suited to generate causal evidence on long-term health outcomes of an intervention, as well as nonhealth outcomes such as economic and social consequences; and (5) they can often generate evidence faster and at lower cost than experiments and other intervention studies.

Quasi-experimental study designs series – Paper 12: Strengthening global capacity for evidence synthesis of quasi-experimental health systems research

Evidence from quasi-experimental studies is often excluded from systematic reviews of health systems research despite the fact that such studies can provide strong causal evidence when well conducted. This article discusses global coordination of efforts to institutionalize the inclusion of causal evidence from quasi-experiments in systematic reviews of health systems research. In particular, we are concerned with identifying opportunities for strengthening capacity at the global and local level for implementing protocols necessary to ensure that reviews that include quasi-experiments are consistently of the highest quality. We first describe the current state of the global infrastructure that facilitates the production of systematic reviews of health systems research. We identify five important types of actors operating within this infrastructure: review authors; synthesis collaborations that facilitate the review process; synthesis interest groups that supplement the work of the larger collaborations; review funders; and end users, including policymakers. Then, we examine opportunities for intervening to build the capacity of each type of actors to support the inclusion of quasi-experiments in reviews. Finally, we suggest practical next steps for proceeding with capacity building efforts. Because of the complexity and relative nascence of the field, we recommend a carefully planned and executed approach to strengthening global capacity for the inclusion of quasi-experimental studies in systematic reviews.

Quasi-experimental study designs series—paper 13: realizing the full potential of quasi-experiments for health research

Although the number of quasi-experiments conducted by health researchers has increased in recent years, there clearly remains unrealized potential for using these methods for causal evaluation of health policies and programs globally. This article proposes five prescriptions for capturing the full value of quasi-experiments for health research. First, new funding opportunities targeting proposals that use quasi-experimental methods should be made available to a broad pool of health researchers. Second, administrative data from health programs, often amenable to quasi-experimental analysis, should be made more accessible to researchers. Third, training in quasi-experimental methods should be integrated into existing health science graduate programs to increase global capacity to use these methods. Fourth, clear guidelines for primary research and synthesis of evidence from quasi-experiments should be developed. Fifth, strategic investments should be made to continue to develop new innovations in quasi-experimental methodologies. Tremendous opportunities exist to expand the use of quasi-experimental methods to increase our understanding of which health programs and policies work and which do not. Health researchers should continue to expand their commitment to rigorous causal evaluation with quasi-experimental methods, and international institutions should increase their support for these efforts.

Screening HIV-Infected Patients with Low CD4 Counts for Cryptococcal Antigenemia prior to Initiation of Antiretroviral Therapy: Cost Effectiveness of Alternative Screening Strategies in South Africa

Background In 2015 South Africa established a national cryptococcal antigenemia (CrAg) screening policy targeted at HIV-infected patients with CD4+ T-lymphocyte (CD4) counts <100 cells/ μl who are not yet on antiretroviral treatment (ART). Two screening strategies are included in national guidelines: reflex screening, where a CrAg test is performed on remnant blood samples from CD4 testing; and provider-initiated screening, where providers order a CrAg test after a patient returns for CD4 test results. The objective of this study was to compare costs and effectiveness of these two screening strategies. Methods We developed a decision analytic model to compare reflex and provider-initiated screening in terms of programmatic and health outcomes (number screened, number identified for preemptive treatment, lives saved, and discounted years of life saved) and screening and treatment costs (2015 USD). We estimated a base case with prevalence and other parameters based on data collected during CrAg screening pilot projects integrated into routine HIV care in Gauteng, Free State, and Western Cape Provinces. We conducted sensitivity analyses to explore how results change with underlying parameter assumptions. Results In the base case, for each 100,000 CD4 tests, the reflex strategy compared to the provider-initiated strategy has higher screening costs (

Study protocol for a cluster-randomised controlled trial of an NCD access to medicines initiative: evaluation of Novartis Access in Kenya

37,536 higher) but lower treatment costs (

Impact of a community-based package of interventions on child development in Zambia: a cluster-randomised controlled trial

55,165 lower), so overall costs of screening and treatment are

Prevalence and unmet need for diabetes care across the care continuum in a national sample of South African adults: evidence from the SANHANES-1, 2011-2012

17,629 less with the reflex strategy. The reflex strategy saves more lives (30 lives, 647 additional years of life saved). Sensitivity analyses suggest that reflex screening dominates provider-initiated screening (lower total costs and more lives saved) or saves additional lives for small additional costs (<

Child Gender and Parental Reporting of Illness Symptoms in Sub-Saharan Africa

125 per life year) across a wide range of conditions (CrAg prevalence, patient and provider behavior, patient survival without treatment, and effectiveness of preemptive fluconazole treatment). Conclusions In countries with substantial numbers of people with untreated, advanced HIV disease such as South Africa, CrAg screening before initiation of ART has the potential to reduce cryptococcal meningitis and save lives. Reflex screening compared to provider-initiated screening saves more lives and is likely to be cost saving or have low additional costs per additional year of life saved.

Equity dimensions of the availability and quality of reproductive, maternal and neonatal health services in Zambia

Introduction Novartis recently launched Novartis Access, an initiative to provide a basket of reduced price medicines for non-communicable diseases (NCDs) to be sold through the public and private non-profit sectors in programme countries. This study will evaluate the impact of Novartis Access on the availability and price of NCD medicines at health facilities and households in Kenya, the first country to receive the programme. Methods and analysis This study will be a cluster-randomised controlled trial. 8 counties in Kenya will be randomly assigned to the intervention or control group using a covariate constrained randomisation method to maximise balance on demographic and health characteristics. In intervention counties, public and private non-profit health facilities will be able to order Novartis Access NCD medicines from the Mission for Essential Drugs and Supplies (MEDS). Data will be collected from a random sample of 384 health facilities and 800 households at baseline, midline after 1-year of intervention, and end-line after 2 years. Quarterly surveillance data will also be collected from health facilities and a subsample of households through phone-based interviews. Households will be eligible if at least one resident has been previously diagnosed and prescribed a medicine for an NCD addressed by Novartis Access, including hypertension and diabetes. The primary outcomes will be availability and price of NCD medicines at health facilities, and availability, price, and expenditures on NCD medicines at households. Impacts will be estimated using intention-to-treat analysis. Ethics and dissemination This protocol was approved by the Institutional Review Boards at Strathmore University and at Boston University. Informed consent will be obtained from all participants at the start of the trial. The findings of the trial will be disseminated through peer-reviewed journals, international conferences, and meetings and events organised with local stakeholders. Trial registration number NCT02773095.