Mary Ellen Slaughter

A Tutorial on Propensity Score Estimation for Multiple Treatments Using Generalized Boosted Models

The use of propensity scores to control for pretreatment imbalances on observed variables in non-randomized or observational studies examining the causal effects of treatments or interventions has become widespread over the past decade. For settings with two conditions of interest such as a treatment and a control, inverse probability of treatment weighted estimation with propensity scores estimated via boosted models has been shown in simulation studies to yield causal effect estimates with desirable properties. There are tools (e.g., the twang package in R) and guidance for implementing this method with two treatments. However, there is not such guidance for analyses of three or more treatments. The goals of this paper are twofold: (1) to provide step-by-step guidance for researchers who want to implement propensity score weighting for multiple treatments and (2) to propose the use of generalized boosted models (GBM) for estimation of the necessary propensity score weights. We define the causal quantities that may be of interest to studies of multiple treatments and derive weighted estimators of those quantities. We present a detailed plan for using GBM to estimate propensity scores and using those scores to estimate weights and causal effects. We also provide tools for assessing balance and overlap of pretreatment variables among treatment groups in the context of multiple treatments. A case study examining the effects of three treatment programs for adolescent substance abuse demonstrates the methods.

The Women's Health Initiative: The food environment, neighborhood socioeconomic status, BMI, and blood pressure.

Using data (n = 60,775 women) from the Womens Health Initiative Clinical Trial (WHI CT)—a national study of postmenopausal women aged 50–79 years—we analyzed cross‐sectional associations between the availability of different types of food outlets in the 1.5 miles surrounding a womans residence, census tract neighborhood socioeconomic status (NSES), BMI, and blood pressure (BP). We simultaneously modeled NSES and food outlets using linear and logistic regression models, adjusting for multiple sociodemographic factors, population density and random effects at the tract and metropolitan statistical area (MSA) level. We found significant associations between NSES, availability of food outlets and individual‐level measurements of BMI and BP. As grocery store/supermarket availability increased from the 10th to the 90th percentile of its distribution, controlling for confounders, BMI was lower by 0.30 kg/m2. Conversely, as fast‐food outlet availability increased from the 10th to the 90th percentile, BMI was higher by 0.28 kg/m2. When NSES increased from the 10th to the 90th percentile of its distribution, BMI was lower by 1.26 kg/m2. As NSES increased from the 10th to the 90th percentile, systolic and diastolic BP were lower by 1.11 mm Hg and 0.40 mm Hg, respectively. As grocery store/supermarket outlet availability increased from the 10th and 90th percentiles, diastolic BP was lower by 0.31 mm Hg. In this national sample of postmenopausal women, we found important independent associations between the food and socioeconomic environments and BMI and BP. These findings suggest that changes in the neighborhood environment may contribute to efforts to control obesity and hypertension.

Neighborhood Socioeconomic Status and Cognitive Function in Women

OBJECTIVES We examined whether neighborhood socioeconomic status (NSES) is associated with cognitive functioning in older US women and whether this relationship is explained by associations between NSES and vascular, health behavior, and psychosocial factors. METHODS We assessed women aged 65 to 81 years (n = 7479) who were free of dementia and took part in the Womens Health Initiative Memory Study. Linear mixed models examined the cross-sectional association between an NSES index and cognitive functioning scores. A base model adjusted for age, race/ethnicity, education, income, marital status, and hysterectomy. Three groups of potential confounders were examined in separate models: vascular, health behavior, and psychosocial factors. RESULTS Living in a neighborhood with a 1-unit higher NSES value was associated with a level of cognitive functioning that was 0.022 standard deviations higher (P = .02). The association was attenuated but still marginally significant (P < .1) after adjustment for confounders and, according to interaction tests, stronger among younger and non-White women. CONCLUSIONS The socioeconomic status of a womans neighborhood may influence her cognitive functioning. This relationship is only partially explained by vascular, health behavior, or psychosocial factors. Future research is needed on the longitudinal relationships between NSES, cognitive impairment, and cognitive decline.

The relationship between urban sprawl and coronary heart disease in women

Studies have reported relationships between urban sprawl, physical activity, and obesity, but - to date - no studies have considered the relationship between sprawl and coronary heart disease (CHD) endpoints. In this analysis, we use longitudinal data on post-menopausal women from the Womens Health Initiative (WHI) Clinical Trial to analyze the relationship between metropolitan statistical area (MSA)-level urban compactness (the opposite of sprawl) and CHD endpoints including death, any CHD event, and myocardial infarction. Models control for individual and neighborhood socio-demographic characteristics. Women who lived in more compact communities at baseline had a lower probability of experiencing a CHD event and CHD death or MI during the study follow-up period. One component of compactness, high residential density, had a particularly noteworthy effect on outcomes. Finally, exploratory analyses showed evidence that the effects of compactness were moderated by race and region.

An intervention to improve program implementation: findings from a two-year cluster randomized trial of Assets-Getting To Outcomes

BackgroundStudies have shown that communities have not always been able to implement evidence-based prevention programs with quality and achieve outcomes demonstrated by prevention science. Implementation support interventions are needed to bridge this gap between science and practice. The purpose of this article is to present two-year outcomes from an evaluation of the Assets Getting To Outcomes (AGTO) intervention in 12 Maine communities engaged in promoting Developmental Assets, a positive youth development approach to prevention. AGTO is an implementation support intervention that consists of: a manual of text and tools; face-to-face training, and onsite technical assistance, focused on activities shown to be associated with obtaining positive results across any prevention program.MethodsThis study uses a nested and cross-sectional, cluster randomized controlled design. Participants were coalition members and program staff from 12 communities in Maine. Each coalition nominated up to five prevention programs to participate. At random, six coalitions and their respective 30 programs received the two-year AGTO intervention and the other six maintained routine operations. The study assessed prevention practitioner capacity (efficacy and behaviors), practitioner exposure to and use of AGTO, practitioner perceptions of AGTO, and prevention program performance. Capacity of coalition members and performance of their programs were compared between the two groups across the baseline, one-, and two-year time points.ResultsWe found no significant differences between AGTO and control group’s prevention capacity. However, within the AGTO group, significant differences were found between those with greater exposure to and use of AGTO. Programs that received the highest number of technical assistance hours showed the most program improvement.ConclusionsThis study is the first of its kind to show that use of an implementation support intervention-AGTO -yielded improvements in practitioner capacity and consequently in program performance on a large sample of practitioners and programs using a randomized controlled design.ClinicalTrials.gov identifierNCT00780338

An Experiment to Evaluate the Efficacy of Cognitive Tutor Geometry

Abstract This randomized, controlled field trial estimated the causal impact of a technology-based geometry curriculum on students’ geometry achievement, as well as their attitudes toward mathematics and technology. The curriculum combines learner-centered classroom pedagogy with individualized, computer-based student instruction. Conducted over a 3-year period in eight high schools within an urban fringe district, the study found that students assigned to the treatment curriculum scored 19% of a standard deviation lower on the geometry posttest than their counterparts assigned to the districts standard curriculum, but found no statistically significant impact on students’ attitudes toward mathematics and technology. Researchers also collected observation and interview data on teachers’ instructional practices. These data suggest that many teachers had difficulty implementing the treatment curriculums learner-centered pedagogy. In fact, observed levels of learner-centered practices were only modestly higher in treatment classes than in control classes. In both treatment and control classes, however, higher levels of learner-centered pedagogy were associated with higher student achievement in geometry.

Neighborhood socioeconomic status and coronary heart disease risk prediction in a nationally representative sample

OBJECTIVES Test the association between coronary heart disease (CHD) risk scores and neighborhood socioeconomic status (NSES) in a US nationally-representative sample and describe whether the association varies by gender and race/ethnicity. STUDY DESIGN Cross-sectional study. METHODS We use Health and Nutrition Examination Survey (NHANES) data from 1999 to 2004 linked with Census tract data. Multivariable regression models and propensity score adjusted models are employed to test the association between NSES and 10-year risk of CHD based on the Framingham Risk Score (FRS), adjusting for individual-level characteristics. RESULTS An individual living in a neighborhood at the 75th percentile of NSES (high NSES) has, on average, a 10-year CHD risk that is 0.16 percentage points lower (95% Confidence Interval 0.16, 0.17) than a similar person residing in a neighborhood at the 25th percentile of NSES (low NSES). Race/ethnicity and gender were found to significantly modify the association between NSES and CHD risk: the association is larger in men than women and in whites than minorities. Propensity score models showed that findings on the main effects of NSES were robust to self-selection into neighborhoods. Similar results were observed between NSES and risk of cardiovascular disease events. CONCLUSIONS NSES is significantly associated with CHD risk, and the relationship varies by gender and race/ethnicity.

Understanding Variations in Medicare Consumer Assessment of Health Care Providers and Systems Scores: California as an Example

OBJECTIVE To understand reasons why California has lower Consumer Assessment of Healthcare Providers and Systems (CAHPS) scores than the rest of the country, including differing patterns of CAHPS scores between Medicare Advantage (MA) and fee-for-service, effects of additional demographic characteristics of beneficiaries, and variation across MA plans within California. STUDY DESIGN/DATA COLLECTION Using 2008 CAHPS survey data for fee-for-service Medicare beneficiaries and MA members, we compared mean case mix adjusted Medicare CAHPS scores for California and the remainder of the nation. PRINCIPAL FINDINGS California fee-for-service Medicare had lower scores than non-California fee-for-service on 11 of 14 CAHPS measures; California MA had lower scores only for physician services measures and higher scores for other measures. Adding race/ethnicity and urbanity to risk adjustment improved California standing for all measures in both MA and fee-for-service. Within the MA plans, one large plan accounted for the positive performance in California MA; other California plans performed below national averages. CONCLUSIONS This study shows that the mix of fee-for-service and MA enrollees, demographic characteristics of populations, and plan-specific factors can all play a role in observed regional variations. Anticipating value-based payments, further study of successful MA plans could generate lessons for enhancing patient experience for the Medicare population.

A novel test of the GTO implementation support intervention in low resource settings: Year 1 findings and challenges

Implementation research is expanding because Evidence-Based Programs (EBPs) are not adopted in many medical domains. However, rigorous implementation research is needed in nonclinical, community-based settings, which often have low capacity that undermines implementation quality and outcomes. This presentation describes Enhancing Quality Interventions Promoting Healthy Sexuality, a 5-year, cluster Hybrid Implementation RCT (Type II) testing how well a community-based setting (Boys & Girls Clubs, BGCs) conducts an EBP called Making Proud Choices (MPC) that aims to prevent teen pregnancy and sexually transmitted infections, with and without an implementation support intervention called Getting To Outcomes (GTO). GTO is a 10-step model of program operation grounded in implementation theory and an intervention of written tools, training and bi-weekly onsite technical assistance.

Estimating the Causal Effects of Cumulative Treatment Episodes for Adolescents Using Marginal Structural Models and Inverse Probability of Treatment Weighting

BACKGROUND Substance use treatment is rarely a one-time event for individuals with substance use disorders. Sustained reductions in substance use and its related symptoms may result from multiple treatment episodes. METHODS We use a marginal structural model with inverse-probability-of-treatment weighting to estimate the causal effects of cumulative treatment experiences over a period of 9 months on drug use at the end of 1-year among 2870 adolescents receiving care in community-based treatment settings. During the 9 months, adolescents move in and out of outpatient and residential treatment with periods where they only receive biological drug screening (BDS) or no treatment at all. The use of inverse-probability-of-treatment weighting reduces confounding bias due to observed baseline and time-varying measures over the course of follow-up; weights were estimated using generalized boosted models. RESULTS Each additional period of treatment (representing at least one day, 1 session, or 1 BDS during the 90 day period between follow-up visits) yielded reductions in average substance use frequency at 1-year relative to no treatment during the 90-day period. For residential treatment it was a 16% decrease (95% CI=-27%, -7%), for outpatient treatment it was a 9% decrease (95% CI=-18%, -0%), and for BDS (with no additional outpatient or residential treatment) it was an 11% decrease (95% CI=-20%, -3%). CONCLUSIONS Using robust statistical methods, we find promising (albeit preliminary) evidence that additional periods of outpatient and residential treatment, as well as biological drug screening, lead to reductions in substance use outcomes at one year.