Deborah Peikes

Propensity Score Matching: A Note of Caution for Evaluators of Social Programs

Over the past 25 years, evaluators of social programs have searched for nonexperimental methods that can substitute effectively for experimental ones. Recently, the spotlight has focused on one method, propensity score matching (PSM), as the suggested approach for evaluating employment and education programs. We present a case study of our experience using PSM, under seemingly ideal circumstances, for the evaluation of the State Partnership Initiative employment promotion program. Despite ideal conditions and the passing of statistical tests suggesting that the matching procedure had worked, we find that PSM produced incorrect impact estimates when compared with a randomized design. Based on this experience, we caution practitioners about the risks of implementing PSM-based designs.

Enhancing the Primary Care Team to Provide Redesigned Care: The Roles of Practice Facilitators and Care Managers

ABSTRACT Efforts to redesign primary care require multiple supports. Two potential members of the primary care team—practice facilitator and care manager—can play important but distinct roles in redesigning and improving care delivery. Facilitators, also known as quality improvement coaches, assist practices with coordinating their quality improvement activities and help build capacity for those activities—reflecting a systems-level approach to improving quality, safety, and implementation of evidence-based practices. Care managers provide direct patient care by coordinating care and helping patients navigate the system, improving access for patients, and communicating across the care team. These complementary roles aim to help primary care practices deliver coordinated, accessible, comprehensive, and patient-centered care.

Two-Year Costs and Quality in the Comprehensive Primary Care Initiative

BACKGROUND The 4-year, multipayer Comprehensive Primary Care Initiative was started in October 2012 to determine whether several forms of support would produce changes in care delivery that would improve the quality and reduce the costs of care at 497 primary care practices in seven regions across the United States. Support included the provision of care-management fees, the opportunity to earn shared savings, and the provision of data feedback and learning support. METHODS We tracked changes in the delivery of care by practices participating in the initiative and used difference-in-differences regressions to compare changes over the first 2 years of the initiative in Medicare expenditures, health care utilization, claims-based measures of quality, and patient experience for Medicare fee-for-service beneficiaries attributed to initiative practices and a group of matched comparison practices. RESULTS During the first 2 years, initiative practices received a median of

Staffing patterns of primary care practices in the comprehensive primary care initiative.

115,000 per clinician in care-management fees. The practices reported improvements in approaches to the delivery of primary care in areas such as management of the care of high-risk patients and enhanced access to care. Changes in average monthly Medicare expenditures per beneficiary did not differ significantly between initiative and comparison practices when care-management fees were not taken into account (-

How Changes In Washington University’s Medicare Coordinated Care Demonstration Pilot Ultimately Achieved Savings

11; 95% confidence interval [CI], -

Organizing Care for Complex Patients in the Patient-Centered Medical Home

23 to

Early Experiences Engaging Patients Through Patient and Family Advisory Councils.

1; P=0.07; negative values indicate less growth in spending at initiative practices) or when these fees were taken into account (

Effects of Care Coordination on Hospitalization, Quality of Care, and Health Care Expenditures Among Medicare Beneficiaries: 15 Randomized Trials

7; 95% CI, -

Six Features Of Medicare Coordinated Care Demonstration Programs That Cut Hospital Admissions Of High-Risk Patients

5 to

Effects of Care Coordination on Hospitalization, Quality of Care, and Health Care Expenditures Among Medicare Beneficiaries

19; P=0.27). The only significant differences in other measures were a 3% reduction in primary care visits for initiative practices relative to comparison practices (P<0.001) and changes in two of the six domains of patient experience--discussion of decisions regarding medication with patients and the provision of support for patients taking care of their own health--both of which showed a small improvement in initiative practices relative to comparison practices (P=0.006 and P<0.001, respectively). CONCLUSIONS Midway through this 4-year intervention, practices participating in the initiative have reported progress in transforming the delivery of primary care. However, at this point these practices have not yet shown savings in expenditures for Medicare Parts A and B after accounting for care-management fees, nor have they shown an appreciable improvement in the quality of care or patient experience. (Funded by the Department of Health and Human Services, Centers for Medicare and Medicaid Services; ClinicalTrials.gov number, NCT02320591.).