Matthew Cefalu

Model averaged double robust estimation.

Researchers estimating causal effects are increasingly challenged with decisions on how to best control for a potentially high-dimensional set of confounders. Typically, a single propensity score model is chosen and used to adjust for confounding, while the uncertainty surrounding which covariates to include into the propensity score model is often ignored, and failure to include even one important confounder will results in bias. We propose a practical and generalizable approach that overcomes the limitations described above through the use of model averaging. We develop and evaluate this approach in the context of double robust estimation. More specifically, we introduce the model averaged double robust (MA-DR) estimators, which account for model uncertainty in both the propensity score and outcome model through the use of model averaging. The MA-DR estimators are defined as weighted averages of double robust estimators, where each double robust estimator corresponds to a specific choice of the outcome model and the propensity score model. The MA-DR estimators extend the desirable double robustness property by achieving consistency under the much weaker assumption that either the true propensity score model or the true outcome model be within a specified, possibly large, class of models. Using simulation studies, we also assessed small sample properties, and found that MA-DR estimators can reduce mean squared error substantially, particularly when the set of potential confounders is large relative to the sample size. We apply the methodology to estimate the average causal effect of temozolomide plus radiotherapy versus radiotherapy alone on one-year survival in a cohort of 1887 Medicare enrollees who were diagnosed with glioblastoma between June 2005 and December 2009.

Does exposure prediction bias health-effect estimation?: The relationship between confounding adjustment and exposure prediction.

In environmental epidemiology, we are often faced with 2 challenges. First, an exposure prediction model is needed to estimate the exposure to an agent of interest, ideally at the individual level. Second, when estimating the health effect associated with the exposure, confounding adjustment is needed in the health-effects regression model. The current literature addresses these 2 challenges separately. That is, methods that account for measurement error in the predicted exposure often fail to acknowledge the possibility of confounding, whereas methods designed to control confounding often fail to acknowledge that the exposure has been predicted. In this article, we consider exposure prediction and confounding adjustment in a health-effects regression model simultaneously. Using theoretical arguments and simulation studies, we show that the bias of a health-effect estimate is influenced by the exposure prediction model, the type of confounding adjustment used in the health-effects regression model, and the relationship between these 2. Moreover, we argue that even with a health-effects regression model that properly adjusts for confounding, the use of a predicted exposure can bias the health-effect estimate unless all confounders included in the health-effects regression model are also included in the exposure prediction model. While these results of this article were motivated by studies of environmental contaminants, they apply more broadly to any context where an exposure needs to be predicted.

Comparative effectiveness of radiotherapy with vs. without temozolomide in older patients with glioblastoma

It is unknown whether the addition of temozolomide (TMZ) to radiotherapy (RT) is associated with improved overall survival (OS) among older glioblastoma patients. We performed a retrospective cohort SEER-Medicare analysis of 1652 patients aged ≥65 years with glioblastoma who received ≥10 fractions of RT from 2005 to 2009, or from 1995 to 1999 before TMZ was available. Three cohorts were assembled based on diagnosis year and treatment initiated within 60 days of diagnosis: (1) 2005–2009 and TMZ/RT, (2) 2005–2009 and RT only, or (3) 1995–1999 and RT only. Associations with OS were estimated using Cox proportional hazards models and propensity score analyses; OS was calculated starting 60 days after diagnosis. Pre-specified sensitivity analyses were performed among patients who received long-course RT (≥27 fractions). Median survival estimates were 7.4 (IQR, 3.3–14.7) months for TMZ/RT, 5.9 (IQR, 2.6–12.1) months for RT alone in 2005–2009, and 5.6 (IQR, 2.7–9.6) months for RT alone in 1995–1999. OS at 2 years was 10.1 % for TMZ/RT, 7.1 % for RT in 2005–2009, and 4.7 % for RT in 1995–1999. Adjusted models suggested decreased mortality risk for TMZ/RT compared to RT in 2005–2009 (AHR, 0.86; 95 % CI, 0.76–0.98) and RT in 1995–1999 (AHR, 0.71; 95 % CI, 0.57–0.90). Among patients from 2005 to 2009 who received long-course RT, however, the addition of TMZ did not significantly improve survival (AHR, 0.91; 95 % CI, 0.80–1.04). In summary, among a large cohort of older glioblastoma patients treated in a real-world setting, the addition of TMZ to RT was associated with a small survival gain.

Comparing Quality of Care in Veterans Affairs and Non-Veterans Affairs Settings

BackgroundCongress, veterans’ groups, and the press have expressed concerns that access to care and quality of care in Department of Veterans Affairs (VA) settings are inferior to access and quality in non-VA settings.ObjectiveTo assess quality of outpatient and inpatient care in VA at the national level and facility level and to compare performance between VA and non-VA settings using recent performance measure data.Main MeasuresWe assessed Patient Safety Indicators (PSIs), 30-day risk-standardized mortality and readmission measures, and ORYX measures for inpatient safety and effectiveness; Healthcare Effectiveness Data and Information Set (HEDIS®) measures for outpatient effectiveness; and Consumer Assessment of Healthcare Providers and Systems Hospital Survey (HCAHPS) and Survey of Healthcare Experiences of Patients (SHEP) survey measures for inpatient patient-centeredness. For inpatient care, we used propensity score matching to identify a subset of non-VA hospitals that were comparable to VA hospitals.Key ResultsVA hospitals performed on average the same as or significantly better than non-VA hospitals on all six measures of inpatient safety, all three inpatient mortality measures, and 12 inpatient effectiveness measures, but significantly worse than non-VA hospitals on three readmission measures and two effectiveness measures. The performance of VA facilities was significantly better than commercial HMOs and Medicaid HMOs for all 16 outpatient effectiveness measures and for Medicare HMOs, it was significantly better for 14 measures and did not differ for two measures. High variation across VA facilities in the performance of some quality measures was observed, although variation was even greater among non-VA facilities.ConclusionsThe VA system performed similarly or better than the non-VA system on most of the nationally recognized measures of inpatient and outpatient care quality, but high variation across VA facilities indicates a need for targeted quality improvement.

Collaborative Targeted Learning Using Regression Shrinkage

Causal inference practitioners are routinely presented with the challenge of model selection and, in particular, reducing the size of the covariate set with the goal of improving estimation efficiency. Collaborative targeted minimum loss-based estimation (CTMLE) is a general framework for constructing doubly robust semiparametric causal estimators that data-adaptively limit model complexity in the propensity score to optimize a preferred loss function. This stepwise complexity reduction is based on a loss function placed on a strategically updated model for the outcome variable through which the error is assessed using cross-validation. We demonstrate how the existing stepwise variable selection CTMLE can be generalized using regression shrinkage of the propensity score. We present 2 new algorithms that involve stepwise selection of the penalization parameter(s) in the regression shrinkage. Simulation studies demonstrate that, under a misspecified outcome model, mean squared error and bias can be reduced by a CTMLE procedure that separately penalizes individual covariates in the propensity score. We demonstrate these approaches in an example using electronic medical data with sparse indicator covariates to evaluate the relative safety of 2 similarly indicated asthma therapies for pregnant women with moderate asthma.

Confounding adjustment and exposure prediction in environmental epidemiology: additional insights.

e28 | www.epidem.com

Completing the results of the 2013 Boston marathon.

The 2013 Boston marathon was disrupted by two bombs placed near the finish line. The bombs resulted in three deaths and several hundred injuries. Of lesser concern, in the immediate aftermath, was the fact that nearly 6,000 runners failed to finish the race. We were approached by the marathons organizers, the Boston Athletic Association (BAA), and asked to recommend a procedure for projecting finish times for the runners who could not complete the race. With assistance from the BAA, we created a dataset consisting of all the runners in the 2013 race who reached the halfway point but failed to finish, as well as all runners from the 2010 and 2011 Boston marathons. The data consist of split times from each of the 5 km sections of the course, as well as the final 2.2 km (from 40 km to the finish). The statistical objective is to predict the missing split times for the runners who failed to finish in 2013. We set this problem in the context of the matrix completion problem, examples of which include imputing missing data in DNA microarray experiments, and the Netflix prize problem. We propose five prediction methods and create a validation dataset to measure their performance by mean squared error and other measures. The best method used local regression based on a K-nearest-neighbors algorithm (KNN method), though several other methods produced results of similar quality. We show how the results were used to create projected times for the 2013 runners and discuss potential for future application of the same methodology. We present the whole project as an example of reproducible research, in that we are able to make the full data and all the algorithms we have used publicly available, which may facilitate future research extending the methods or proposing completely different approaches.

2015 Health Related Behaviors Survey: Physical Health and Functional Limitations Among U.S. Active-Duty Service Members

The Health Related Behaviors Survey (HRBS) is the U.S. Department of Defense’s flagship survey for understanding the health, health-related behaviors, and well-being of service members. Fielded periodically for more than 30 years, the HRBS includes content areas—such as substance use, mental and physical health, sexual behavior, and postdeployment problems—that may affect force readiness or the ability to meet the demands of military life. The Defense Health Agency asked the RAND Corporation to revise and field the 2015 HRBS. In this brief, we review results for physical health and functional limitations. Specifically, we consider the prevalence of chronic medical conditions, such as high blood pressure and high cholesterol levels; physical symptoms, such as back or joint pain; and health-related functional limitations at work or at home. We note possible limitations to the data and implications of the findings. We make several comparisons to the overall U.S. population, including progress toward Healthy People 2020 (HP2020) objectives established by the U.S. Department of Health and Human Services. Because the military differs notably from the general population (e.g., service members are more likely to be young and male) and service members must be in good health to join the military, these comparisons are offered only as a benchmark of interest.

2015 Department of Defense Health Related Behaviors Survey (HRBS)

The Health Related Behaviors Survey (HRBS) is the U.S. Department of Defenses flagship survey for understanding the health, health-related behaviors, and well-being of service members. In 2014, the Defense Health Agency asked the RAND Corporation to review previous iterations of the HRBS, update survey content, administer a revised version of the survey, and analyze data from the resulting 2015 HRBS of active-duty personnel, including those in the U.S. Air Force, Army, Marine Corps, Navy, and Coast Guard. This study details the methodology, sample demographics, and results from that survey in the following domains: health promotion and disease prevention; substance use; mental and emotional health; physical health and functional limitations; sexual behavior and health; sexual orientation, transgender identity, and health; and deployment experiences and health. The results presented here are intended to supplement data already collected by the Department of Defense and to inform policy initiatives to help improve the readiness, health, and well-being of the force.

2015 Health Releated Behaviors Survey: Deployment Experiences and Health Among U.S. Active-Duty Service Members

The Health Related Behaviors Survey (HRBS) is the U.S. Department of Defense (DoD)’s flagship survey for understanding the health, health-related behaviors, and well-being of service members. Fielded periodically for more than 30 years, the HRBS includes content areas—such as substance use, mental and physical health, sexual behavior, and postdeployment problems—that may affect force readiness or the ability to meet the demands of military life. The Defense Health Agency asked the RAND Corporation to revise and field the 2015 HRBS. In this brief, we review results for recent deployment experiences and health, including the frequency and duration of deployments, levels of exposure to combat-related experiences, the prevalence of deployment-related injuries, the prevalence of deployment-related substance use, and deployment-related mental and physical health.