Aaron Mendelson

The Effects of Pay-for-Performance Programs on Health, Health Care Use, and Processes of Care: A Systematic Review

Pay-for-performance (P4P) programs provide financial rewards or penalties to individual health care providers, groups of providers, or institutions according to their performance on measures of quality. In theory, if properly targeted and designed, P4P programs would help drive the behavior of providers and health care systems to improve the quality of care delivered, reduce unnecessary use of expensive health care services, and improve patient health outcomes (1). The idea is particularly relevant in the United States, where serious and broad gaps in health care quality have been tied in part to the long-standing fee-for-service system, which may provide incentives for service volume rather than quality (2). Despite their intuitive appeal, the promise of P4P programs in improving outcomes has not been empirically realized in past studies (36). The most recent systematic review examining the effectiveness of P4P programs in the United States found mixed evidence that P4P was associated with modest improvements in process-of-care outcomes but had little effect on patient outcomes (7). However, the literature has grown considerably since this review (which searched through 2012), and other countries, such as the United Kingdom, have gained considerable experience with large P4P initiatives that may provide information relevant to the United States. The purpose of the current review is to update and expand the prior systematic review in order to summarize current understanding of the effects of P4P programs targeted at physicians, groups, and institutions on process-of-care and patient outcomes in ambulatory and outpatient settings in and outside the United States. Methods This review was conducted according to a protocol that was developed using established reporting standards and posted to a public Web site (8) before the study was initiated (Appendix 1 of the Supplement). We used an analytic framework based on work by Damberg and colleagues (7) (Appendix 2 of the Supplement). Supplement. Supplemental Materials Data Sources and Searches We searched the following databases for studies that evaluated P4P programs: PubMed (1 June 2007 to 6 October 2016), MEDLINE, PsycINFO, CINAHL, Business Economics and Theory, Business Source Elite, Scopus, Faculty of 1000, and Gartner Research (1 June 2007 to 29 February 2016). We also performed targeted Google and PubMed searches aimed at well-known P4P demonstrations. We obtained additional articles from reference lists of pertinent studies, reviews, editorials, and expert recommendations. The search strategies are detailed in Appendix 3 of the Supplement. Study Selection Investigators reviewed titles and abstracts identified from literature searches. Two investigators independently assessed each potentially relevant article for inclusion using preestablished criteria (Appendices 4 and 5 of the Supplement). We included English-language studies of adult patients that evaluated ambulatory care or hospital-based P4P programs targeting health care providers at the individual, group, managerial, or institutional level and that reported any process-of-care, utilization, health, or intermediate health (clinical measures, such as a laboratory value or blood pressure) outcome. We included studies from other countries that have health systems similar to portions of the U.S. health care system. Studies examining only patient-targeted financial incentives, as well as payment models other than direct P4P, such as managed care, capitation, bundled payments, and accountable care organizations, were excluded. We also excluded studies that were not conducted in hospital or ambulatory settings, such as studies in long-term care facilities or nursing homes. We included clinical or cluster randomized, controlled trials (RCTs) of any size. We used a best-evidence approach, which is a method of specifying minimum inclusion criteria for nonrandomized studies (9). Inclusion of observational studies was limited to those with a comparison group, interrupted time series (ITS) studies, or large (n> 10000) cross-sectional or uncontrolled beforeafter studies. We excluded smaller uncontrolled studies because we had identified a large number of potentially relevant studies during a preliminary search and because the smaller uncontrolled studies were less likely to provide broadly applicable information given their limited scope and inherent methodological deficiencies. Data Extraction and Quality Assessment One investigator abstracted data elements from each included study, which were reviewed for accuracy by at least 1 additional investigator. We abstracted information on study design, sample size, country, program description, incentive structure (size and timing), target of the incentive, comparator, and outcomes (grouped as health, intermediate health, process-of-care, and utilization measures). Appendices 6 and 7 of the Supplement report these data. We classified studies according to 4 broad groupings: RCTs, ITS studies, controlled beforeafter studies, and uncontrolled beforeafter studies. Two investigators independently assessed study quality using the Newcastle-Ottawa Scale (10) for observational studies and the Cochrane Risk-of-Bias tool (11) for RCTs (Appendix 8 of the Supplement). Disagreements were resolved by consensus. Data Synthesis and Analysis We qualitatively synthesized the results of ambulatory and hospital studies separately and report process-of-care and patient outcomes for each setting. We synthesized results for specific P4P programs whenever possible. The review team evaluated the strength of the evidence according to guidance from the Agency for Healthcare Research and Quality (12). We did not perform meta-analysis because of the marked clinical heterogeneity across studies and the large number of observational studies. Role of the Funding Source The U.S. Department of Veterans Affairs Quality Enhancement Research Initiative supported this review but had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication. Results Search Results We reviewed 3418 titles and abstracts, identified 586 potentially eligible full-text articles, and ultimately included 69 studies (Figure). Fifty-eight studies were in ambulatory settings (Table 1 and Appendix 6 of the Supplement), 11 were in hospital settings (Table 2 and Appendix 7 of the Supplement), 52 reported process-of-care outcomes, and 38 assessed patient outcomes. The studies examined a wide range of P4P programs with varying incentive structures, goals, and contexts. The programs also differed in their purposes and targets, but the largest number of studies focused on managing chronic conditions in the primary care setting. Studies were conducted in a wide range of countries, including the United Kingdom (27 studies), the United States (17 studies), Taiwan (13 studies), France (3 studies), the Netherlands (3 studies), Canada (3 studies), Australia (1 study), South Korea (1 study), and Italy (1 study). There were 2 RCTs and 67 observational studies (10 ITS studies, 37 controlled beforeafter studies, and 20 large uncontrolled beforeafter studies). Figure. Literature flow diagram. P4P= pay-for-performance. * The current systematic review updates and expands on the review by Damberg and colleagues (7). Table 1. Findings From Studies of Ambulatory-Based Pay-for-Performance Programs Table 2. Findings From Studies of Hospital-Based Pay-for-Performance Programs A large number of studies evaluated different aspects of 2 large-scale national programs: the United Kingdoms Quality and Outcomes Framework (QOF) (24 studies) and Taiwans diabetes mellitus (DM-P4P) program (9 studies). The QOF is a nationwide program that began in 2004. It incentivizes primary care practices to achieve quality indicators that support clinical care and public health goals. Incentive payments can comprise up to approximately 30% of total income. Practices are aided by integrated health information technology that delivers automated prompts and decision support (36, 83). Taiwans DM-P4P program, implemented in 2001, allows physicians to voluntarily enroll in the program, and they in turn are given freedom to choose which patients to enroll (51). From 2001 to 2006, incentives targeted process-of-care outcomes, which were augmented with intermediate health outcome measures after 2006. Ambulatory CareBased Programs Process-of-Care Outcomes We found 9 studies from the United States evaluating the effects of P4P on process-of-care outcomes (14, 1620, 2224). Most of these studies examined outcomes over 4 years and had an average follow-up of 2.5 years; very few studies reported longer-term data. One RCT found that individual incentives increased appropriate response to high blood pressure but not use of guideline-recommended antihypertensive medication (14). Of the 6 studies that reported positive results (16, 18, 19, 2224), 1 did not have a control group (24), and selection bias was a serious concern in 3 others because of the way the control group was chosen (18, 22, 23). Two methodologically sound controlled beforeafter studies found no improvements in processes of care (17, 20). In general, there was evidence across 17 studies in the United Kingdom (2631, 33, 3638, 4147) that the QOF was associated with improvements in process-of-care measures, although the evidence was mixed among the more methodologically rigorous studies. There were 6 ITS studies. One showed substantial improvements in the prescription of long-acting reversible contraceptives (26), and another showed modest improvement in the initiation of diabetes medications (27). Another study found increased rates of depression screening and diagnoses, but antidepressant prescribing remained unchanged (31). In the other 3 studies, improvements had begun well bef

Implementation Processes and Pay for Performance in Healthcare: A Systematic Review

Over the last decade, various pay-for-performance (P4P) programs have been implemented to improve quality in health systems, including the VHA. P4P programs are complex, and their effects may vary by design, context, and other implementation processes. We conducted a systematic review and key informant (KI) interviews to better understand the implementation factors that modify the effectiveness of P4P. We searched PubMed, PsycINFO, and CINAHL through April 2014, and reviewed reference lists. We included trials and observational studies of P4P implementation. Two investigators abstracted data and assessed study quality. We interviewed P4P researchers to gain further insight. Among 1363 titles and abstracts, we selected 509 for full-text review, and included 41 primary studies. Of these 41 studies, 33 examined P4P programs in ambulatory settings, 7 targeted hospitals, and 1 study applied to nursing homes. Related to implementation, 13 studies examined program design, 8 examined implementation processes, 6 the outer setting, 18 the inner setting, and 5 provider characteristics. Results suggest the importance of considering underlying payment models and using statistically stringent methods of composite measure development, and ensuring that high-quality care will be maintained after incentive removal. We found no conclusive evidence that provider or practice characteristics relate to P4P effectiveness. Interviews with 14 KIs supported limited evidence that effective P4P program measures should be aligned with organizational goals, that incentive structures should be carefully considered, and that factors such as a strong infrastructure and public reporting may have a large influence. There is limited evidence from which to draw firm conclusions related to P4P implementation. Findings from studies and KI interviews suggest that P4P programs should undergo regular evaluation and should target areas of poor performance. Additionally, measures and incentives should align with organizational priorities, and programs should allow for changes over time in response to data and provider input.BackgroundOver the last decade, various pay-for-performance (P4P) programs have been implemented to improve quality in health systems, including the VHA. P4P programs are complex, and their effects may vary by design, context, and other implementation processes. We conducted a systematic review and key informant (KI) interviews to better understand the implementation factors that modify the effectiveness of P4P.MethodsWe searched PubMed, PsycINFO, and CINAHL through April 2014, and reviewed reference lists. We included trials and observational studies of P4P implementation. Two investigators abstracted data and assessed study quality. We interviewed P4P researchers to gain further insight.ResultsAmong 1363 titles and abstracts, we selected 509 for full-text review, and included 41 primary studies. Of these 41 studies, 33 examined P4P programs in ambulatory settings, 7 targeted hospitals, and 1 study applied to nursing homes. Related to implementation, 13 studies examined program design, 8 examined implementation processes, 6 the outer setting, 18 the inner setting, and 5 provider characteristics. Results suggest the importance of considering underlying payment models and using statistically stringent methods of composite measure development, and ensuring that high-quality care will be maintained after incentive removal. We found no conclusive evidence that provider or practice characteristics relate to P4P effectiveness. Interviews with 14 KIs supported limited evidence that effective P4P program measures should be aligned with organizational goals, that incentive structures should be carefully considered, and that factors such as a strong infrastructure and public reporting may have a large influence.DiscussionThere is limited evidence from which to draw firm conclusions related to P4P implementation. Findings from studies and KI interviews suggest that P4P programs should undergo regular evaluation and should target areas of poor performance. Additionally, measures and incentives should align with organizational priorities, and programs should allow for changes over time in response to data and provider input.

Early Performance in Medicaid Accountable Care Organizations: A Comparison of Oregon and Colorado

Importance Several state Medicaid reforms are under way, but the relative performance of different approaches is unclear. Objective To compare the performance of Oregon’s and Colorado’s Medicaid Accountable Care Organization (ACO) models. Design, Setting, and Participants Oregon initiated its Medicaid transformation in 2012, supported by a

Health Disparities in Veterans: A Map of the Evidence

1.9 billion investment from the federal government, moving most Medicaid enrollees into 16 Coordinated Care Organizations, which managed care within a global budget. Colorado initiated its Medicaid Accountable Care Collaborative in 2011, creating 7 Regional Care Collaborative Organizations that received funding to coordinate care with providers and connect Medicaid enrollees with community services. Data spanning July 1, 2010, through December 31, 2014 (18 months before intervention and 24 months after intervention, treating 2012 as a transition year) were analyzed for 452 371 Oregon and 330 511 Colorado Medicaid enrollees, assessing changes in outcomes using difference-in-differences analyses of regional focus, primary care homes, and care coordination. Oregon’s Coordinated Care Organization model was more comprehensive in its reform goals and in the imposition of downside financial risk. Exposures Regional focus, primary care homes, and care coordination in Medicaid ACOs. Main Outcomes and Measures Performance on claims-based measures of standardized expenditures and utilization for selected services, access, preventable hospitalizations, and appropriateness of care. Results In a total of 782 882 Medicaid enrollees, 45.0% were male, with mean (SD) age 16.74 (14.41) years. Standardized expenditures for selected services declined in both states during the 2010-2014 period, but these decreases were not significantly different between the 2 states. Oregon’s model was associated with reductions in emergency department visits (−6.28 per 1000 beneficiary-months; 95% CI, −10.51 to −2.05) and primary care visits (−15.09 visits per 1000 beneficiary-months; 95% CI, −26.57 to −3.61), improvements in acute preventable hospital admissions (−1.01 admissions per 1000 beneficiary-months; 95% CI, −1.61 to −0.42), 3 of 4 measures of access (well-child visits, ages 3-6 years, 2.69%; 95% CI, 1.20% to 4.19%; adolescent well-care visits, 6.77%; 95% CI, 5.22% to 8.32%; and adult access to preventive ambulatory care, 1.26%; 95% CI, 0.28% to 2.25%), and 1 of 4 measures of appropriateness of care (avoidance of head imaging for uncomplicated headache, 2.59%; 95% CI, 1.35% to 3.83%). Conclusions and Relevance Two years into implementation, Oregon’s and Colorado’s Medicaid ACO models exhibited similar performance on standardized expenditures for selected services. Oregon’s model, marked by a large federal investment and movement to global budgets, was associated with improvements in some measures of utilization, access, and quality, but Colorado’s model paralleled Oregon’s on several other metrics.

Pay-for-Performance and Veteran Care in the VHA and the Community: a Systematic Review

Background: Goals for improving the quality of care for all Veterans and eliminating health disparities are outlined in the Veterans Health Administration Blueprint for Excellence, but the degree to which disparities in utilization, health outcomes, and quality of care affect Veterans is not well understood. Objectives: To characterize the research on health care disparities in the Veterans Health Administration by means of a map of the evidence. Research Design: We conducted a systematic search for research studies published from 2006 to February 2016 in MEDLINE and other data sources. We included studies of Veteran populations that examined disparities in 3 outcome categories: utilization, quality of health care, and patient health. Measures: We abstracted data on study design, setting, population, clinical area, outcomes, mediators, and presence of disparity for each outcome category. We grouped the data by population characteristics including race, disability status, mental illness, demographics (age, era of service, rural location, and distance from care), sex identity, socioeconomic status, and homelessness, and created maps illustrating the evidence. Results: We reviewed 4249 citations and abstracted data from 351 studies which met inclusion criteria. Studies examining disparities by race/ethnicity comprised by far the vast majority of the literature, followed by studies examining disparities by sex, and mental health condition. Very few studies examined disparities related to lesbian, gay, bisexual, or transgender identity or homelessness. Disparities findings vary widely by population and outcome. Conclusions: Our evidence maps provide a “lay of the land” and identify important gaps in knowledge about health disparities experienced by different Veteran populations.

Interventions to Improve Pharmacological Adherence Among Adults With Psychotic Spectrum Disorders and Bipolar Disorder: A Systematic Review

BackgroundAlthough pay-for-performance (P4P) strategies have been used by the Veterans Health Administration (VHA) for over a decade, the long-term benefits of P4P are unclear. The use of P4P is further complicated by the increased use of non-VHA healthcare providers as part of the Veterans Choice Program. We conducted a systematic review and key informant interviews to better understand the effectiveness and potential unintended consequences of P4P, as well as the implementation factors and design features important in both VHA and non-VHA/community settings.MethodsWe searched PubMed, PsycINFO, and CINAHL through March 2017 and reviewed reference lists. We included trials and observational studies of P4P targeting Veteran health. Two investigators abstracted data and assessed study quality. We interviewed VHA stakeholders to gain further insight.ResultsThe literature search yielded 1031 titles and abstracts, of which 30 studies met pre-specified inclusion criteria. Twenty-five examined P4P in VHA settings and 5 in community settings. There was no strong evidence supporting the effectiveness of P4P in VHA settings. Interviews with 17 key informants were consistent with studies that identified the potential for overtreatment associated with performance metrics in the VHA. Key informants’ views on P4P in community settings included the need to develop relationships with providers and health systems with records of strong performance, to improve coordination by targeting documentation and data sharing processes, and to troubleshoot the limited impact of P4P among practices where Veterans make up a small fraction of the patient population.DiscussionThe evidence to support the effectiveness of P4P on Veteran health is limited. Key informants recognize the potential for unintended consequences, such as overtreatment in VHA settings, and suggest that implementation of P4P in the community focus on relationship building and target areas such as documentation and coordination of care.

The Effects of Pay-for-Performance Programs on Health, Health Care Use, and Processes of Care

BACKGROUND It is unclear as to which interventions are effective at improving medication adherence in individuals with serious and persistent mental illness. The goal of this systematic review is to synthesize evidence examining the effectiveness, harms, and costs of interventions to improve medication adherence in patients with psychotic spectrum disorders and bipolar disorder. METHODS We conducted a systematic search of several electronic databases through January 2015 using a structured search strategy. Studies were included if they involved adult patients in general mental health settings, reported both measures of medication adherence and clinical outcomes, and were of sufficient methodological rigor. Studies were quality assessed and synthesized using established methods. RESULTS We identified 24 studies that met inclusion criteria. Overall, 20 studies addressed interventions in patients with psychotic spectrum disorders. These interventions varied widely, with generally mixed findings contributing to low or insufficient strength of evidence; studies involving family members and technology interventions were the most consistently associated with a positive effect; however, the strength of the evidence was low because of intervention heterogeneity. The evidence was insufficient to determine the effectiveness of interventions in patients with bipolar disorder. CONCLUSIONS In individuals with psychotic spectrum disorders, interventions with family members or technology had the most consistent positive effect on adherence, although replication with objective adherence measures along with evaluation of harms and costs is needed. There was insufficient evidence to draw conclusions about interventions in individuals with bipolar disorder.