Megan S. Schuler

Generalizing observational study results: Applying propensity score methods to complex surveys

OBJECTIVE To provide a tutorial for using propensity score methods with complex survey data. DATA SOURCES Simulated data and the 2008 Medical Expenditure Panel Survey. STUDY DESIGN Using simulation, we compared the following methods for estimating the treatment effect: a naïve estimate (ignoring both survey weights and propensity scores), survey weighting, propensity score methods (nearest neighbor matching, weighting, and subclassification), and propensity score methods in combination with survey weighting. Methods are compared in terms of bias and 95 percent confidence interval coverage. In Example 2, we used these methods to estimate the effect on health care spending of having a generalist versus a specialist as a usual source of care. PRINCIPAL FINDINGS In general, combining a propensity score method and survey weighting is necessary to achieve unbiased treatment effect estimates that are generalizable to the original survey target population. CONCLUSIONS Propensity score methods are an essential tool for addressing confounding in observational studies. Ignoring survey weights may lead to results that are not generalizable to the survey target population. This paper clarifies the appropriate inferences for different propensity score methods and suggests guidelines for selecting an appropriate propensity score method based on a researchers goal.

Temporal and gender trends in concordance of urine drug screens and self-reported use in cocaine treatment studies

Objectives:To describe temporal trends in concordance, sensitivity, and specificity and to explore demographic trends in concordance in 2 outpatient treatment studies for cocaine dependence. Methods:We obtained 2229 urine drug screens (UDS) from 129 individuals, along with accompanying self-use reports (SUR). Paired SUR and UDS were considered concordant if the 2 measures of cocaine use were in agreement. The sensitivity and specificity of the SUR in predicting the UDS was also estimated. To model concordance, sensitivity, and specificity as a function of time, generalized estimating equations were used. Demographic effects on concordance among subjects who achieved 100% concordance and subjects who achieved a recently proposed 70% concordance threshold were tested. Results:During the course of our studies, both sensitivity and concordance statistically decreased; however, specificity remained relatively constant. Median concordance for all subjects was 88%. Among all subjects, concordance varied significantly by gender, with women achieving significantly higher concordance than men (96% vs 86%). Similarly, women were almost twice as likely to achieve 100% concordance as men (42% vs 22%). Finally, 80% of participants achieved the 70% concordance threshold, and no differences among demographic groups with regards to the 70% concordance threshold were observed. Conclusions:Temporal effects of concordance and sensitivity may have profound repercussions when using SUR to gauge efficacy of an experimental intervention. Furthermore, gender may differentially affect concordance. Finally, a substance abuse outcome measure that reliably combines objective and self-report data is promising, but further research is needed.

Age-varying associations between substance use behaviors and depressive symptoms during adolescence and young adulthood.

BACKGROUND Substance use and depression often co-occur, complicating treatment of both substance use and depression. Despite research documenting age-related trends in both substance use and depression, little research has examined how the associations between substance use behaviors and depression changes across the lifespan. METHODS This study examines how the associations between substance use behaviors (daily smoking, regular heavy episodic drinking (HED), and marijuana use) and depressive symptoms vary from adolescence into young adulthood (ages 12-31), and how these associations differ by gender. Using data from the National Longitudinal Study of Adolescent to Adult Health (Add Health), we implemented time-varying effect models (TVEM), an analytic approach that estimates how the associations between predictors (e.g., substance use measures) and an outcome (e.g., depressive symptoms) vary across age. RESULTS Marijuana use and daily smoking were significantly associated with depressive symptoms at most ages from 12 to 31. Regular HED was significantly associated with depressive symptoms during adolescence only. In bivariate analyses, the association with depressive symptoms for each substance use behavior was significantly stronger for females at certain ages; when adjusting for concurrent substance use in a multivariate analysis, no gender differences were observed. CONCLUSIONS While the associations between depressive symptoms and both marijuana and daily smoking were relatively stable across ages 12-31, regular HED was only significantly associated with depressive symptoms during adolescence. Understanding age and gender trends in these associations can help tailor prevention efforts and joint treatment methods in order to maximize public health benefit.

Targeted Maximum Likelihood Estimation for Causal Inference in Observational Studies

Estimation of causal effects using observational data continues to grow in popularity in the epidemiologic literature. While many applications of causal effect estimation use propensity score methods or G-computation, targeted maximum likelihood estimation (TMLE) is a well-established alternative method with desirable statistical properties. TMLE is a doubly robust maximum-likelihood–based approach that includes a secondary “targeting” step that optimizes the bias-variance tradeoff for the target parameter. Under standard causal assumptions, estimates can be interpreted as causal effects. Because TMLE has not been as widely implemented in epidemiologic research, we aim to provide an accessible presentation of TMLE for applied researchers. We give step-by-step instructions for using TMLE to estimate the average treatment effect in the context of an observational study. We discuss conceptual similarities and differences between TMLE and 2 common estimation approaches (G-computation and inverse probability weighting) and present findings on their relative performance using simulated data. Our simulation study compares methods under parametric regression misspecification; our results highlight TMLEs property of double robustness. Additionally, we discuss best practices for TMLE implementation, particularly the use of ensembled machine learning algorithms. Our simulation study demonstrates all methods using super learning, highlighting that incorporation of machine learning may outperform parametric regression in observational data settings.

Propensity score weighting for a continuous exposure with multilevel data

Propensity score methods (e.g., matching, weighting, subclassification) provide a statistical approach for balancing dissimilar exposure groups on baseline covariates. These methods were developed in the context of data with no hierarchical structure or clustering. Yet in many applications the data have a clustered structure that is of substantive importance, such as when individuals are nested within healthcare providers or within schools. Recent work has extended propensity score methods to a multilevel setting, primarily focusing on binary exposures. In this paper, we focus on propensity score weighting for a continuous, rather than binary, exposure in a multilevel setting. Using simulations, we compare several specifications of the propensity score: a random effects model, a fixed effects model, and a single-level model. Additionally, our simulations compare the performance of marginal versus cluster-mean stabilized propensity score weights. In our results, regression specifications that accounted for the multilevel structure reduced bias, particularly when cluster-level confounders were omitted. Furthermore, cluster mean weights outperformed marginal weights.

Disparities in substance use behaviors and disorders among adult sexual minorities by age, gender, and sexual identity

BACKGROUND Sexual minorities (SMs) experience elevated rates of substance use behaviors and disorders relative to heterosexuals; minority stress is theorized to contribute to these disparities. As SMs are not a homogenous group, analyses that aggregate SMs across sexual identity, age, or gender obscure important variation among this population. To date, age- and gender-specific disparities have not been rigorously examined using a large national sample. METHODS Using data on 67,354 adults (ages 18-49) from the 2015 and 2016 National Survey of Drug Use and Health we examined age- and gender-specific disparities in smoking, heavy episodic drinking, marijuana use, illicit drug use, and alcohol/substance use disorder. Age groups were ages 18-25, 26-34, and 35-49. Using logistic regression, we estimated age- and gender-specific odds ratios for gay/lesbian and bisexual individuals, relative to heterosexuals; analyses adjusted for demographic characteristics. RESULTS Bisexual women had significantly elevated odds of all outcomes at all ages, relative to heterosexual women. Gay/lesbian individuals had significantly elevated odds for nearly all outcomes compared to same-gender heterosexuals at ages 18-25, but not consistently at older ages. For bisexual men, significant disparities compared to heterosexual men were only observed at ages 35-49 for marijuana use and alcohol/substance use disorder. CONCLUSIONS We found notable within-group differences regarding SM disparities. While disparities were most pronounced in young adulthood for gay/lesbian individuals and mid-adulthood for bisexual men, bisexual women uniquely experienced disparities across all ages. Minority stress experiences may vary with respect to gender, age/cohort, and sexual identity, resulting in differential risk for substance use.

Human Papillomavirus Vaccine Effectiveness Against Incident Genital Warts Among Female Health-Plan Enrollees, United States

We examined the effectiveness of human papillomavirus vaccination by dose number and spacing against incident genital warts in a cohort of 64,517 female health-plan enrollees in the United States during 2006-2012. Eligible recipients were classified into groups by regimen: 0, 1, 2 (<6 months apart), 2 (≥6 months apart), or 3 doses. They were followed until a genital wart diagnosis, loss to follow-up, or the end of study. Propensity score weights were used to balance baseline differences across groups. To account for latent genital warts before vaccination, we applied 6- and 12-month buffer periods from last and first vaccine dose, respectively. Incidence rates and hazard ratios were calculated using Poisson regression and Cox models. The propensity score-weighted incidence rate per 100,000 person-years was 762 among unvaccinated participants. Using 6- and 12-month buffer periods, respectively, incidence rates were 641 and 257 for 1 dose, 760 and 577 for the 2-dose (<6-month interval) regimen, 313 and 194 for the 2-dose (≥6-month interval) regimen, and 199 and 162 among 3-dose vaccinees; vaccine effectiveness was 68% and 76% for the 2-dose (≥6-month interval) regimen and 77% and 80% in 3-dose vaccinees compared with unvaccinated participants. Vaccine effectiveness was not significant among vaccinees receiving 1-dose and 2-dose (<6-month interval) regimens compared with unvaccinated participants. Our findings contribute to a better understanding of the real-world effectiveness of HPV vaccination.

Substance Use Behaviors and the Timing of Family Formation During Young Adulthood

The impact of substance use on the life course of young adults can be substantial, yet few studies have examined to what extent early adult substance use behaviors are related to the timing of family formation, independent of confounding factors from adolescence. Using panel data from the Monitoring the Future study (N~20,000), the current study examined the associations between three substance use behaviors (i.e., cigarette use, binge drinking, and marijuana use) and the timing of family formation events in young adulthood. Survival analysis and propensity score weighting addressed preexisting differences between substance users and nonusers in the estimation of the timing of union formation (i.e., marriage, cohabitation) and parenthood. Results for young adult substance users showed general patterns of reduced rates of marriage and parenthood and increased cohabitation during young adulthood. Variations were evident by substance and sex.

Relative influence of perceived peer and family substance use on adolescent alcohol, cigarette, and marijuana use across middle and high school

PURPOSE Substance use by peers and family may affect adolescent substance use, yet the relative influence may shift during adolescence as youth differentiate themselves from family and more closely affiliate with peers. This study examined trends in concordance of adolescent cigarette, alcohol and marijuana use and corresponding perceived use by friends and family members during middle and high school. METHODS Data are from a longitudinal cohort of 12,038 youth who completed up to five surveys during grades 6-12. At each wave, adolescents reported past month use of cigarettes, alcohol and marijuana, as well as perceived use by their best friend, older sibling and most important adult figure. For each substance, we used time-varying effect models to estimate how associations between adolescent use and perceived use varied across grade. RESULTS For all substances, concordance with best friend use was positive and stronger than concordance with older sibling or adult use at all grades. Concordance with both best friend and older sibling use of all substances was pronounced in 6th grade. Concordance peaked again during mid-high school for smoking (best friend, older sibling) and marijuana (best friend). Concordance with adult marijuana use peaked in middle school, yet associations with adult alcohol and cigarette use were relatively stable. CONCLUSIONS Substance use prevention efforts that seek to counter peer normative pressures should begin prior to middle school and span high school. Such efforts should address the role of peer and family environments, as both were found to be relevant during middle and high school.

Free Tax Services in Pediatric Clinics

With this study, we provide initial evidence that free tax services in pediatric clinics may maximize tax refunds, particularly the EITC, which impacts child health. OBJECTIVES: The earned income tax credit (EITC), refundable monies for America’s working poor, is associated with improved child health. Yet, 20% of eligible families do not receive it. We provided free tax preparation services in clinics serving low-income families and assessed use, financial impact, and accuracy. METHODS: Free tax preparation services (“StreetCred”) were available at 4 clinics in Boston in 2016 and 2017. We surveyed a convenience sample of clients (n = 244) about experiences with StreetCred and previous tax services and of nonparticipants (n = 100; 69% response rate) and clinic staff (n = 41; 48% response rate) about acceptability and feasibility. RESULTS: A total of 753 clients received