Frank Eijkenaar

Key issues in the design of pay for performance programs

Pay for performance (P4P) is increasingly being used to stimulate healthcare providers to improve their performance. However, evidence on P4P effectiveness remains inconclusive. Flaws in program design may have contributed to this limited success. Based on a synthesis of relevant theoretical and empirical literature, this paper discusses key issues in P4P-program design. The analysis reveals that designing a fair and effective program is a complex undertaking. The following tentative conclusions are made: (1) performance is ideally defined broadly, provided that the set of measures remains comprehensible, (2) concerns that P4P encourages “selection” and “teaching to the test” should not be dismissed, (3) sophisticated risk adjustment is important, especially in outcome and resource use measures, (4) involving providers in program design is vital, (5) on balance, group incentives are preferred over individual incentives, (6) whether to use rewards or penalties is context-dependent, (7) payouts should be frequent and low-powered, (8) absolute targets are generally preferred over relative targets, (9) multiple targets are preferred over single targets, and (10) P4P should be a permanent component of provider compensation and is ideally “decoupled” form base payments. However, the design of P4P programs should be tailored to the specific setting of implementation, and empirical research is needed to confirm the conclusions.

Economic evaluation of pay-for-performance in health care: a systematic review.

BackgroundPay-for-performance (P4P) intents to stimulate both more effective and more efficient health care delivery. To date, evidence on whether P4P itself is an efficient method has not been systematically analyzed.ObjectiveTo identify and analyze the existing literature regarding economic evaluation of P4P.Data sourcesEnglish, German, Spanish, and Turkish language literature were searched in the following databases: Business Source Complete, the Cochrane Library, Econlit, ISI web of knowledge, Medline (via PubMed), and PsycInfo (January 2000–April 2010).Study selectionArticles published in peer-reviewed journals and describing economic evaluations of P4P initiatives. Full economic evaluations, considering costs and consequences of the P4P intervention simultaneously, were the prime focus. Additionally, comparative partial evaluations were included if costs were described and the study allows for an assessment of consequences. Both experimental and observational studies were considered.ResultsIn total, nine studies could be identified. Three studies could be regarded as full economic evaluations, and six studies were classified as partial economic evaluations. Based on the full economic evaluations, P4P efficiency could not be demonstrated. Partial economic evaluations showed mixed results, but several flaws limit their significance. Ranges of costs and consequences were typically narrow, and programs differed considerably in design. Methodological quality assessment showed scores between 32% and 65%.ConclusionThe results show that evidence on the efficiency of P4P is scarce and inconclusive. P4P efficiency could not be demonstrated. The small number and variability of included studies limit the strength of our conclusions. More research addressing P4P efficiency is needed.

Performance profiling in primary care: does the choice of statistical model matter?

Background. Profiling is increasingly being used to generate input for improvement efforts in health care. For these efforts to be successful, profiles must reflect true provider performance, requiring an appropriate statistical model. Sophisticated models are available to account for the specific features of performance data, but they may be difficult to use and explain to providers. Objective. To assess the influence of the statistical model on the performance profiles of primary care providers. Data Source. Administrative data (2006–2008) on 2.8 million members of a Dutch health insurer who were registered with 1 of 4396 general practitioners. Methods. Profiles are constructed for 6 quality measures and 5 resource use measures, controlling for differences in case mix. Models include ordinary least squares, generalized linear models, and multilevel models. Separately for each model, providers are ranked on z scores and classified as outlier if belonging to the 10% with the worst or best performance. The impact of the model is evaluated using the weighted kappa for rankings overall, percentage agreement on outlier designation, and changes in rankings over time. Results. Agreement among models was relatively high overall (kappa typically >0.85). Agreement on outlier designation was more variable and often below 80%, especially for high outliers. Rankings were more similar for processes than for outcomes and expenses. Agreement among annual rankings per model was low for all models. Conclusions. Differences among models were relatively small, but the choice of statistical model did affect the rankings. In addition, most measures appear to be driven largely by chance, regardless of the model that is used. Profilers should pay careful attention to the choice of both the statistical model and the performance measures.

Profiling individual physicians using administrative data from a single insurer: variance components, reliability, and implications for performance improvement efforts.

Background:Individual physicians are increasingly being subjected to comparative performance assessments. When single-insurer data are used to profile individual physicians’ performance, reliable measurements are uncertain because of small sample sizes. Methods:Administrative data (2006–2008) from a Dutch insurer are used to examine variation in general practitioners’ (GPs) performance on expenses (5 measures), utilization of hospital care (2 measures), and clinical quality for diabetes and chronic obstructive pulmonary disease (6 measures). Unadjusted and adjusted multilevel models are used to separate total variance in between-GP and within-GP components. The components are used to calculate intraclass correlation coefficients (ICCs), reliability, and sample size requirements at common reliability thresholds. Results:Average ICCs varied between 0.07% (hospital admissions) and 8.34% (physiotherapy for chronic obstructive pulmonary disease patients). Risk-adjustment often greatly changed the relative size of variance components and often led to lower ICCs. In addition, ICCs and thus reliability generally decreased over time. Eight measures had reliabilities > 0.70, and 3 of these (all GP-related expenses) > 0.90. Measures related to utilization of hospital care had reliabilities < 0.60 or even 0.50. For 5 measures, the vast majority of GPs had sufficient patients to reach 0.70 reliability. At a reliability of 0.90, however, there were no measures for which all GPs met the sample size requirements. Conclusions:Reliable measurement of individual physicians’ performance using single-purchaser data is challenging. For most measures reliability was insufficient to allow for high-stakes applications or even any application of profiling. Future research should continue to explore methods for enhancing the reliability of individual physicians’ profiles.

Health Plan Payment in the Netherlands

Abstract This chapter describes the institutional setting, design, and performance of health plan payment in the Netherlands. The primary focus is on the Health Insurance Act (2006), which is based on principles of regulated competition and strongly relies on health plan payment as a tool for achieving public objectives. In anticipation of the introduction of regulated competition, the construction of the current health plan payment system started in the early 1990s. More than two decades later we conclude that important progress has been made. We also conclude, however, that further improvements are necessary. Two main issues of the current system are the remaining incentives for risk selection and the strong dependence on endogenous variables in the risk equalization model. Incentives for risk selection threaten the solidarity, efficiency, and quality of care, while endogenous risk adjustor variables threaten efficiency in the delivery of care. Further improvements to health plan payment are crucial to overcome these potential failures.

Diagnosis-based Cost Groups in the Dutch Risk-equalization Model: Effects of Clustering Diagnoses and of Allowing Patients to be Classified into Multiple Risk-classes

Background: The risk-equalization (RE) model in the Dutch health insurance market has evolved to a sophisticated model containing direct proxies for health. However, it still has important imperfections, leaving incentives for risk selection. This paper focuses on refining an important health-based risk-adjuster in this model: the diagnosis-based costs groups (DCGs). The current (2017) DCGs are calibrated on “old” data of 2011/2012, are mutually exclusive, and are essentially clusters of about 200 diagnosis-groups (“dxgroups”). Methods: Hospital claims data (2013), administrative data (2014) on costs and risk-characteristics for the entire Dutch population (N≈16.9 million), and health survey data (2012, N≈387,000) are used. The survey data are used to identify subgroups of individuals in poor or in good health. The claims and administrative data are used to develop alternative DCG-modalities to examine the impact on individual-level and group-level fit of recalibrating the DCGs based on new data, of allowing patients to be classified in multiple DCGs, and of refraining from clustering. Results: Recalibrating the DCGs and allowing enrolees to be classified into multiple DCGs lead to nontrivial improvements in individual-level and group-level fit (especially for cancer patients and people with comorbid conditions). The improvement resulting from refraining from clustering does not seem to justify the increase in model complexity this would entail. Conclusions: The performance of the sophisticated Dutch RE-model can be improved by allowing classification in multiple (clustered) DCGs and using new data. Irrespective of the modality used, however, various subgroups remain significantly undercompensated. Further improvement of the RE-model merits high priority.

Examining unpriced risk heterogeneity in the Dutch health insurance market

A major challenge in regulated health insurance markets is to mitigate risk selection potential. Risk selection can occur in the presence of unpriced risk heterogeneity, which refers to predictable variation in health care spending not reflected in either premiums by insurers or risk equalization payments. This paper examines unpriced risk heterogeneity within risk groups distinguished by the sophisticated Dutch risk equalization model of 2016. Our strategy is to combine the administrative dataset used for estimation of the risk equalization model (n = 16.9 million) with information derived from a large health survey (n = 387k). The survey information allows for explaining and predicting residual spending of the risk equalization model. Based on the predicted residual spending, two metrics are used to indicate unpriced risk heterogeneity at the individual level and at the level of certain (risk) groups: the correlation coefficient between residual spending and predicted residual spending, and the mean absolute value of predicted residual spending. The analyses yield three main findings: (1) the health survey information is able to explain some residual spending of the risk equalization model, (2) unpriced risk heterogeneity exists both in morbidity and in non-morbidity groups, and (3) unpriced risk heterogeneity increases with predicted spending by the risk equalization model. These findings imply that the sophisticated Dutch risk equalization model does not completely remove unpriced risk heterogeneity. Further improvement of the model should focus on broadening and refining the current set of morbidity-based risk adjusters.

Risk equalization in competitive health insurance markets: Identifying healthy individuals on the basis of multiple-year low spending

Objective To study the extent to which risk equalization (RE) in competitive health insurance markets can be improved by including an indicator for being healthy. Study Setting/Data Sources This study is conducted in the context of the Dutch individual health insurance market. Administrative data on spending and risk characteristics (2011‐2014) for the entire population (N = 16.6 m) as well as health survey data from a large sample (N = 387 k) are used. Study Design The indicator for being healthy is low spending in three consecutive prior years. “Low spending” is defined in three ways: belonging to the bottom 60%, 70%, or 80% of the annual spending distribution. Versions of the Dutch RE model 2017 with and without the indicator are compared on individual‐level payment fit and, using the survey data, group‐level payment fit. Principal Findings All three alternative models outperform the Dutch RE model 2017. However, significant unpriced risk heterogeneity remains. Compared with the 60% threshold, the 80% threshold comes with a larger improvement in fit but identifies a less selective group. Conclusions The performance of the RE model can be improved by adding an indicator for being healthy based on multiple‐year low spending. However, risk‐selection potential remains, warranting high priority to further improvement of RE.

Design and effects of outcome-based payment models in healthcare: a systematic review

IntroductionOutcome-based payment models (OBPMs) might solve the shortcomings of fee-for-service or diagnostic-related group (DRG) models using financial incentives based on outcome indicators of the provided care. This review provides an analysis of the characteristics and effectiveness of OBPMs, to determine which models lead to favourable effects.MethodsWe first developed a definition for OBPMs. Next, we searched four data sources to identify the models: (1) scientific literature databases; (2) websites of relevant governmental and scientific agencies; (3) the reference lists of included articles; (4) experts in the field. We only selected studies that examined the impact of the payment model on quality and/or costs. A narrative evidence synthesis was used to link specific design features to effects on quality of care or healthcare costs.ResultsWe included 88 articles, describing 12 OBPMs. We identified two groups of models based on differences in design features: narrow OBPMs (financial incentives based on quality indicators) and broad OBPMs (combination of global budgets, risk sharing, and financial incentives based on quality indicators). Most (5 out of 9) of the narrow OBPMs showed positive effects on quality; the others had mixed (2) or negative (2) effects. The effects of narrow OBPMs on healthcare utilization or costs, however, were unfavourable (3) or unknown (6). All broad OBPMs (3) showed positive effects on quality of care, while reducing healthcare cost growth.DiscussionAlthough strong empirical evidence on the effects of OBPMs on healthcare quality, utilization, and costs is limited, our findings suggest that broad OBPMs may be preferred over narrow OBPMs.