Richard Wyss

Propensity Score Methods for Confounding Control in Nonexperimental Research

Nonexperimental studies are increasingly used to investigate the safety and effectiveness of medical products as they are used in routine care. One of the primary challenges of such studies is confounding, systematic differences in prognosis between patients exposed to an intervention of interest and the selected comparator group. In the presence of uncontrolled confounding, any observed difference in outcome risk between the groups cannot be attributed solely to a causal effect of the exposure on the outcome. Confounding in studies of medical products can arise from a variety of different sociomedical processes.1 The most common form of confounding arises from good medical practice, physicians prescribing medications and performing procedures on patients who are most likely to benefit from them. This leads to a bias known as confounding by indication, which can cause medical interventions to appear to cause events that they prevent.2,3 Conversely, patients who are perceived by a physician to be near the end of life may be less likely to receive preventive medications, leading to confounding by frailty or comorbidity.4–6 Additional sources of confounding bias can result from patients’ health-related behaviors. For example, patients who initiate a preventive medication may be more likely than other patients to engage in other healthy, prevention-oriented behaviors leading to bias known as the healthy user/adherer effect.7–9 Many statistical approaches can be used to remove the confounding effects of such factors if they are captured in the data. The most common statistical approaches for confounding control are based on multivariable regression models of the outcome. To yield unbiased estimates of treatment effects, these approaches require that the researcher correctly models the effect of the treatment and covariates on the outcome. However, correct specification of an outcome model can be challenging, particularly in studies …

Propensity scores for confounder adjustment when assessing the effects of medical interventions using nonexperimental study designs.

Treatment effects, especially when comparing two or more therapeutic alternatives as in comparative effectiveness research, are likely to be heterogeneous across age, gender, co‐morbidities and co‐medications. Propensity scores (PSs), an alternative to multivariable outcome models to control for measured confounding, have specific advantages in the presence of heterogeneous treatment effects. Implementing PSs using matching or weighting allows us to estimate different overall treatment effects in differently defined populations. Heterogeneous treatment effects can also be due to unmeasured confounding concentrated in those treated contrary to prediction. Sensitivity analyses based on PSs can help to assess such unmeasured confounding. PSs should be considered a primary or secondary analytic strategy in nonexperimental medical research, including pharmacoepidemiology and nonexperimental comparative effectiveness research.

The Role of Prediction Modeling in Propensity Score Estimation: An Evaluation of Logistic Regression, bCART, and the Covariate-Balancing Propensity Score

The covariate-balancing propensity score (CBPS) extends logistic regression to simultaneously optimize covariate balance and treatment prediction. Although the CBPS has been shown to perform well in certain settings, its performance has not been evaluated in settings specific to pharmacoepidemiology and large database research. In this study, we use both simulations and empirical data to compare the performance of the CBPS with logistic regression and boosted classification and regression trees. We simulated various degrees of model misspecification to evaluate the robustness of each propensity score (PS) estimation method. We then applied these methods to compare the effect of initiating glucagonlike peptide-1 agonists versus sulfonylureas on cardiovascular events and all-cause mortality in the US Medicare population in 2007-2009. In simulations, the CBPS was generally more robust in terms of balancing covariates and reducing bias compared with misspecified logistic PS models and boosted classification and regression trees. All PS estimation methods performed similarly in the empirical example. For settings common to pharmacoepidemiology, logistic regression with balance checks to assess model specification is a valid method for PS estimation, but it can require refitting multiple models until covariate balance is achieved. The CBPS is a promising method to improve the robustness of PS models.

Variable Selection for Propensity Score Models When Estimating Treatment Effects on Multiple Outcomes: a Simulation Study

It is often preferable to simplify the estimation of treatment effects on multiple outcomes by using a single propensity score (PS) model. Variable selection in PS models impacts the efficiency and validity of treatment effects. However, the impact of different variable selection strategies on the estimated treatment effects in settings involving multiple outcomes is not well understood. The authors use simulations to evaluate the impact of different variable selection strategies on the bias and precision of effect estimates to provide insight into the performance of various PS models in settings with multiple outcomes.

Matching on the disease risk score in comparative effectiveness research of new treatments.

We use simulations and an empirical example to evaluate the performance of disease risk score (DRS) matching compared with propensity score (PS) matching when controlling large numbers of covariates in settings involving newly introduced treatments.

Reducing Bias Amplification in the Presence of Unmeasured Confounding Through Out-of-Sample Estimation Strategies for the Disease Risk Score.

Abstract The prognostic score, or disease risk score (DRS), is a summary score that is used to control for confounding in non-experimental studies. While the DRS has been shown to effectively control for measured confounders, unmeasured confounding continues to be a fundamental obstacle in non-experimental research. Both theory and simulations have shown that in the presence of unmeasured confounding, controlling for variables that affect treatment (both instrumental variables and measured confounders) amplifies the bias caused by unmeasured confounders. In this paper, we use causal diagrams and path analysis to review and illustrate the process of bias amplification. We show that traditional estimation strategies for the DRS do not avoid bias amplification when controlling for predictors of treatment. We then discuss estimation strategies for the DRS that can potentially reduce bias amplification that is caused by controlling both instrumental variables and measured confounders. We show that under certain assumptions, estimating the DRS in populations outside the defined study cohort where treatment has not been introduced, or in outside populations with reduced treatment prevalence, can control for the confounding effects of measured confounders while at the same time reduce bias amplification.

Commentary: Balancing automated procedures for confounding control with background knowledge.

In the publication by Patorno et al.1 found in this issue of Epidemiology, the authors illustrate the importance of using subject matter knowledge to complement the automated high-dimensional propensity score (hdPS) algorithm when controlling for confounding in studies based on claims data with few exposed outcomes. The topic of variable selection for PS models in settings involving large numbers of potential confounders has received considerable attention in recent years. This interest is in part due to the uncertainty in determining what role automated procedures should play in the variable selection process. With large healthcare databases becoming increasingly used in epidemiology,2–4 automated procedures can be beneficial in selecting potential confounders that are unknown to the investigator.5–7 Further, the application of automated procedures is likely to expand as safety surveillance receives more attention as part of the Food and Drug Administration’s Sentinel Initiative.8 In these settings, automated procedures, such as the hdPS, can increase the speed and efficiency of active surveillance.7 With an increasing need for automated methods for confounding control in these areas of epidemiologic research, the question becomes: how should investigators balance automated procedures with the use of subject matter knowledge?

Assessing the impact of propensity score estimation and implementation on covariate balance and confounding control within and across important subgroups in comparative effectiveness research.

Purpose:Researchers are often interested in estimating treatment effects in subgroups controlling for confounding based on a propensity score (PS) estimated in the overall study population. Objective:To evaluate covariate balance and confounding control in sulfonylurea versus metformin initiators within subgroups defined by cardiovascular disease (CVD) history comparing an overall PS with subgroup-specific PSs implemented by 1:1 matching and stratification. Methods:We analyzed younger patients from a US insurance claims database and older patients from 2 Medicare (Humana Medicare Advantage, fee-for-service Medicare Parts A, B, and D) datasets. Confounders and risk factors for acute myocardial infarction were included in an overall PS and subgroup PSs with and without CVD. Covariate balance was assessed using the average standardized absolute mean difference (ASAMD). Results:Compared with crude estimates, ASAMD across covariates was improved 70%–94% for stratification for Medicare cohorts and 44%–99% for the younger cohort, with minimal differences between overall and subgroup-specific PSs. With matching, 75%–99% balance improvement was achieved regardless of cohort and PS, but with smaller sample size. Hazard ratios within each CVD subgroup differed minimally among PS and cohorts. Conclusions:Both overall PSs and CVD subgroup-specific PSs achieved good balance on measured covariates when assessing the relative association of diabetes monotherapy with nonfatal myocardial infarction. PS matching generally led to better balance than stratification, but with smaller sample size. Our study is limited insofar as crude differences were minimal, suggesting that the new user, active comparator design identified patients with some equipoise between treatments.

A review of the performance of different methods for propensity score matched subgroup analyses and a summary of their application in peer-reviewed research studies

When evaluating safety signals, there is often interest in understanding safety in all patients for whom compared treatments are reasonable alternatives, as well as in specific subgroups of interest. There are numerous ways that propensity score (PS) matching can be implemented for subgroup analyses.

A Review of Disease Risk Scores and Their Application in Pharmacoepidemiology

Summary scores that reduce baseline covariate information to a single value have become standard tools for confounding control in pharmacoepidemiologic studies. The propensity score (PS) summarizes covariate associations with treatment assignment and has been the most widely used summary score for confounding control. An alternative to the PS is the prognostic score, often referred to as the disease risk score (DRS). Instead of summarizing covariate associations with treatment, the DRS summarizes covariate associations with potential outcomes. Adjustment based on the DRS has unique challenges and limitations compared to PS adjustment, but the DRS also has advantages over the PS in certain settings. In this paper, we review the recent developments and applications of DRSs. We discuss differences between the PS and the DRS as well as the benefits and challenges of using the DRS for confounding control. Finally, we discuss areas for future research and development for the application of risk scores in pharmacoepidemiology and nonexperimental medical studies.