Andrew M. Ryan

Methods for Evaluating Changes in Health Care Policy: The Difference-in-Differences Approach

Observational studies are commonly used to evaluate the changes in outcomes associated with health care policy implementation. An important limitation in using observational studies in this context is the need to control for background changes in outcomes that occur with time (eg, secular trends affecting outcomes). The difference-in-differences approach is increasingly applied to address this problem.1 In this issue of JAMA, studies by Rajaram and colleagues2 and Patel and colleagues3 used the difference-in-differences approach to evaluate the changes that occurred following the 2011 Accreditation Council for Graduate Medical Education (ACGME) duty hour reforms. The 2 studies were conducted with different data sources and study populations but used similar methods.

The Association between Self-Reported Discrimination, Physical Health and Blood Pressure: Findings from African Americans, Black Immigrants, and Latino Immigrants in New Hampshire

The relationship between perceived racial discrimination and both blood pressure and perceived physical health has been documented among African Americans. However, this association has not been well-studied for Black or Latino immigrants. We used multiple regression analysis with a cross-sectional sample of 666 African Americans, Black immigrants, and Latino immigrants from the New Hampshire Racial and Ethnic Approaches to Community Health 2010 Initiative to assess the relationship between discrimination and measures of physical health and blood pressure. The study found evidence of a significant U-shaped relationship between discrimination and systolic blood pressure for all three cohorts. Evidence was also found supporting a negative linear relationship between discrimination and physical health. In addition, the association between discrimination and physical health was attenuated for Latinos compared with the other groups. Future research should evaluate how factors associated with acculturation or cumulative exposure to discriminatory stressors may affect the protective resources of immigrants.

Bariatric surgery complications before vs after implementation of a national policy restricting coverage to centers of excellence.

IMPORTANCE Starting in 2006, the Centers for Medicare & Medicaid Services (CMS) has restricted coverage of bariatric surgery to hospitals designated as centers of excellence (COE) by 2 major professional organizations. OBJECTIVE To evaluate whether the implementation of the COE component of the national coverage decision was associated with improved bariatric surgery outcomes in Medicare patients. DESIGN, SETTING, AND PATIENTS Retrospective, longitudinal study using 2004-2009 hospital discharge data from 12 states (n = 321,464 patients) of changes in outcomes in Medicare patients undergoing bariatric surgery (n = 6723 before and n = 15,854 after implementation of the policy). A difference-in-differences analytic approach was used to evaluate whether the national coverage decision was associated with improved outcomes in Medicare patients above and beyond existing time trends in non-Medicare patients (n = 95,558 before and n = 155,117 after implementation of the policy). MAIN OUTCOME MEASURES Risk-adjusted rates of any complication, serious complications, and reoperation. RESULTS Bariatric surgery outcomes improved during the study period in both Medicare and non-Medicare patients; however, this change was already under way prior to the CMS coverage decision. After accounting for patient factors, changes in procedure type, and preexisting time trends toward improved outcomes, there were no statistically significant improvements in outcomes after (vs before) implementation of the CMS national coverage decision for any complication (8.0% after vs 7.0% before; relative risk [RR], 1.14 [95% CI, 0.95-1.33]), serious complications (3.3% vs 3.6%, respectively; RR, 0.92 [95% CI, 0.62-1.22]), and reoperation (1.0% vs 1.1%; RR, 0.90 [95% CI, 0.64-1.17]). In a direct assessment comparing outcomes at hospitals designated as COEs (n = 179) vs hospitals without the COE designation (n = 519), no significant differences were found for any complication (5.5% vs 6.0%, respectively; RR, 0.98 [95% CI, 0.90-1.06]), serious complications (2.2% vs 2.5%; RR, 0.92 [95% CI, 0.84-1.00]), and reoperation (0.83% vs 0.96%; RR, 1.00 [95% CI, 0.86-1.17]). CONCLUSIONS AND RELEVANCE Among Medicare patients undergoing bariatric surgery, there was no significant difference in the rates of complications and reoperation before vs after the CMS policy of restricting coverage to COEs. Combined with prior studies showing no association of COE designation and outcomes, these results suggest that Medicare should reconsider this policy.

Association of Hospital Participation in a Quality Reporting Program With Surgical Outcomes and Expenditures for Medicare Beneficiaries

IMPORTANCE The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) provides feedback to hospitals on risk-adjusted outcomes. It is not known if participation in the program improves outcomes and reduces costs relative to nonparticipating hospitals. OBJECTIVE To evaluate the association of enrollment and participation in the ACS NSQIP with outcomes and Medicare payments compared with control hospitals that did not participate in the program. DESIGN, SETTING, AND PARTICIPANTS Quasi-experimental study using national Medicare data (2003-2012) for a total of 1,226,479 patients undergoing general and vascular surgery at 263 hospitals participating in ACS NSQIP and 526 nonparticipating hospitals. A difference-in-differences analytic approach was used to evaluate whether participation in ACS NSQIP was associated with improved outcomes and reduced Medicare payments compared with nonparticipating hospitals that were otherwise similar. Control hospitals were selected using propensity score matching (2 control hospitals for each ACS NSQIP hospital). MAIN OUTCOMES AND MEASURES Thirty-day mortality, serious complications (eg, pneumonia, myocardial infarction, or acute renal failure and a length of stay >75th percentile), reoperation, and readmission within 30 days. Hospital costs were assessed using price-standardized Medicare payments during hospitalization and 30 days after discharge. RESULTS After accounting for patient factors and preexisting time trends toward improved outcomes, there were no statistically significant improvements in outcomes at 1, 2, or 3 years after (vs before) enrollment in ACS NSQIP. For example, in analyses comparing outcomes at 3 years after (vs before) enrollment, there were no statistically significant differences in risk-adjusted 30-day mortality (4.3% after enrollment vs 4.5% before enrollment; relative risk [RR], 0.96 [95% CI, 0.89 to 1.03]), serious complications (11.1% after enrollment vs 11.0% before enrollment; RR, 0.96 [95% CI, 0.91 to 1.00]), reoperations (0.49% after enrollment vs 0.45% before enrollment; RR, 0.97 [95% CI, 0.77 to 1.16]), or readmissions (13.3% after enrollment vs 12.8% before enrollment; RR, 0.99 [95% CI, 0.96 to 1.03]). There were also no differences at 3 years after (vs before) enrollment in mean total Medicare payments (

Effects of the Premier Hospital Quality Incentive Demonstration on Medicare Patient Mortality and Cost

40 [95% CI, -

Association of caesarean delivery with child adiposity from age 6 weeks to 15 years

268 to

Paid Malpractice Claims for Adverse Events in Inpatient and Outpatient Settings

348]), or payments for the index admission (-

Recruitment to multicentre trials—lessons from UKCTOCS: descriptive study

11 [95% CI, -

Medicare’s Public Reporting Initiative On Hospital Quality Had Modest Or No Impact On Mortality From Three Key Conditions

278 to

Sensitivity of transvaginal ultrasound screening for endometrial cancer in postmenopausal women: a case-control study within the UKCTOCS cohort

257]), hospital readmission (