Jason J. Wang

Validity of electronic health record-derived quality measurement for performance monitoring

Background Since 2007, New York Citys primary care information project has assisted over 3000 providers to adopt and use a prevention-oriented electronic health record (EHR). Participating practices were taught to re-adjust their workflows to use the EHR built-in population health monitoring tools, including automated quality measures, patient registries and a clinical decision support system. Practices received a comprehensive suite of technical assistance, which included quality improvement, EHR customization and configuration, privacy and security training, and revenue cycle optimization. These services were aimed at helping providers understand how to use their EHR to track and improve the quality of care delivered to patients. Materials and Methods Retrospective electronic chart reviews of 4081 patient records across 57 practices were analyzed to determine the validity of EHR-derived quality measures and documented preventive services. Results Results from this study show that workflow and documentation habits have a profound impact on EHR-derived quality measures. Compared with the manual review of electronic charts, EHR-derived measures can undercount practice performance, with a disproportionately negative impact on the number of patients captured as receiving a clinical preventive service or meeting a recommended treatment goal. Conclusion This study provides a cautionary note in using EHR-derived measurement for public reporting of provider performance or use for payment.

Effect of pay-for-performance incentives on quality of care in small practices with electronic health records: A randomized trial

IMPORTANCE Most evaluations of pay-for-performance (P4P) incentives have focused on large-group practices. Thus, the effect of P4P in small practices, where many US residents receive care, is largely unknown. Furthermore, whether electronic health records (EHRs) with chronic disease management capabilities support small-practice response to P4P has not been studied. OBJECTIVE To assess the effect of P4P incentives on quality in EHR-enabled small practices in the context of an established quality improvement initiative. DESIGN, SETTING, AND PARTICIPANTS A cluster-randomized trial of small (<10 clinicians) primary care clinics in New York City from April 2009 through March 2010. A city program provided all participating clinics with the same EHR software with decision support and patient registry functionalities and quality improvement specialists offering technical assistance. INTERVENTIONS Incentivized clinics were paid for each patient whose care met the performance criteria, but they received higher payments for patients with comorbidities, who had Medicaid insurance, or who were uninsured (maximum payments:

The intended and unintended consequences of quality improvement interventions for small practices in a community-based electronic health record implementation project.

200/patient;

Sustained Improvement in Clinical Preventive Service Delivery Among Independent Primary Care Practices After Implementing Electronic Health Record Systems

100,000/clinic). Quality reports were given quarterly to both the intervention and control groups. MAIN OUTCOMES AND MEASURES Comparison of differences in performance improvement, from the beginning to the end of the study, between control and intervention clinics for aspirin or antithrombotic prescription, blood pressure control, cholesterol control, and smoking cessation interventions. Mixed-effects logistic regression was used to account for clustering of patients within clinics, with a treatment by time interaction term assessing the statistical significance of the effect of the intervention. RESULTS Participating clinics (n = 42 for each group) had similar baseline characteristics, with a mean of 4592 (median, 2500) patients at the intervention group clinics and 3042 (median, 2000) at the control group clinics. Intervention clinics had greater adjusted absolute improvement in rates of appropriate antithrombotic prescription (12.0% vs 6.1%, difference: 6.0% [95% CI, 2.2% to 9.7%], P = .001 for interaction term), blood pressure control (no comorbidities: 9.7% vs 4.3%, difference: 5.5% [95% CI, 1.6% to 9.3%], P = .01 for interaction term; with diabetes mellitus: 9.0% vs 1.2%, difference: 7.8% [95% CI, 3.2% to 12.4%], P = .007 for interaction term; with diabetes mellitus or ischemic vascular disease: 9.5% vs 1.7%, difference: 7.8% [95% CI, 3.0% to 12.6%], P = .01 for interaction term), and in smoking cessation interventions (12.4% vs 7.7%, difference: 4.7% [95% CI, -0.3% to 9.6%], P = .02 for interaction term). Intervention clinics performed better on all measures for Medicaid and uninsured patients except cholesterol control, but no differences were statistically significant. CONCLUSIONS AND RELEVANCE Among small EHR-enabled clinics, a P4P incentive program compared with usual care resulted in modest improvements in cardiovascular care processes and outcomes. Because most proposed P4P programs are intended to remain in place more than a year, further research is needed to determine whether this effect increases or decreases over time. TRIAL REGISTRATION clinicaltrials.gov Identifier: NCT00884013.

Patient Characteristics Associated With Smoking Cessation Interventions and Quit Attempt Rates Across 10 Community Health Centers With Electronic Health Records

Background:Despite the rapid rise in the implementation of electronic health records (EHR), commensurate improvements in health care quality have not been consistently observed. Objectives:To evaluate whether the implementation of EHRs and complementary interventions—including clinical decision support, technical assistance, and financial incentives—improved quality of care. Research Design:The study included 143 practices that implemented EHRs as part of the Primary Care Information Project—a long-standing community-based EHR implementation initiative. A total of 71 practices were randomized to receive financial incentives and quality feedback and 72 were randomized to feedback alone. All practices received technical assistance and had clinical decision support in their EHR. Using data from 2009 to 2011, we estimated measure-level fixed effects models to evaluate the association between exposure to clinical decision support, technical assistance, financial incentives, and quality of care. Associations were estimated separately for 4 cardiovascular measures that were rewarded by the financial incentive program and 4 measures that were not rewarded by incentives. Results:Financial incentives for quality were consistently associated with higher performance for the incentivized measures [+10.1 percentage points at 18 mo of exposure (approximately +22%), P<0.05] and lower performance for the unincentivized measures [−8.3 percentage points at 12 mo of exposure (approximately −20%), P<0.05]. Technical assistance was associated with higher quality for the unincentivized measures, but not for the incentivized measures. Conclusions:Technical assistance and financial incentives—alongside EHR implementation—can improve quality of care. Financial incentives for quality may not result in similar improvements for incentivized and unincentivized measures.

Assessing Capacity and Disease Burden in a Virtual Network of New York City Primary Care Providers Following Hurricane Sandy

Introduction Studies showing sustained improvements in the delivery of clinical preventive services are limited. Fewer studies demonstrate sustained improvements among independent practices that are not affiliated with hospitals or integrated health systems. This study examines the continued improvement in clinical quality measures for a group of independent primary care practices using electronic health records (EHRs) and receiving technical support from a local public health agency. Methods We analyzed clinical quality measure performance data from a cohort of primary care practices that implemented an EHR at least 3 months before October 2009, the study baseline. We assessed trends for 4 key quality measures: antithrombotic therapy, blood pressure control, smoking cessation intervention, and hemoglobin A1c (HbA1c) testing based on monthly summary data transmitted by the practices. Results Of the 151 practices, 140 were small practices and 11 were community health centers; average time using an EHR was 13.7 months at baseline. From October 2009 through October 2011, average rates increased for antithrombotic therapy (from 58.4% to 74.8%), blood pressure control (from 55.3% to 64.1%), HbA1c testing (from 46.4% to 57.7%), and smoking cessation intervention (from 29.3% to 46.2%). All improvements were significant. Conclusion During 2 years, practices showed significant improvement in the delivery of several key clinical preventive services after implementing EHRs and receiving support services from a public health agency.

Factors Related to Clinical Quality Improvement for Small Practices Using an EHR

OBJECTIVES We used electronic health record (EHR) data to determine rates and patient characteristics in offering cessation interventions (counseling, medications, or referral) and initiating quit attempts. METHODS Ten community health centers in New York City contributed 30 months of de-identified patient data from their EHRs. RESULTS Of 302 940 patients, 40% had smoking status recorded and only 34% of documented current smokers received an intervention. Women and younger patients were less likely to have their smoking status documented or to receive an intervention. Patients with comorbidities that are exacerbated by smoking were more likely to have status documented (82.2%) and to receive an intervention (52.1%), especially medication (10.8%). Medication, either alone (odds ratio [OR] = 1.9; 95% confidence interval [CI] = 1.5, 2.3) or combined with counseling (OR = 1.8; 95% CI = 1.5, 2.3), was associated with higher quit attempts compared with no intervention. CONCLUSIONS Data from EHRs demonstrated underdocumentation of smoking status and missed opportunities for cessation interventions. Use of data from EHRs can facilitate quality improvement efforts to increase screening and intervention delivery, with the potential to improve smoking cessation rates.

Quality Measure Performance in Small Practices Before and After Electronic Health Record Adoption

Urban contexts introduce unique challenges that must be addressed to ensure that areas of high population density can function when disasters occur. The ability to generate useful data to guide decision-making is critical in this context. Widespread adoption of electronic health record (EHR) systems in recent years has created electronic data sources and networks that may play an important role in public health surveillance efforts, including in post-disaster situations. The Primary Care Information Project (PCIP) at the New York City Department of Health and Mental Hygiene has partnered with local clinicians to establish an electronic data system, and this network provides infrastructure to support primary care surveillance activities in New York City. After Hurricane Sandy, PCIP generated several sets of data to contribute to the city’s efforts to assess the impact of the storm, including daily connectivity data to establish practice operations, data to examine patterns of primary care utilization in severely affected and less affected areas, and data on the frequency of respiratory infection diagnosis in the primary care setting. Daily patient visit data from three heavily affected neighborhoods showed the health department where primary care capacity was most affected in the weeks following Sandy. Overall transmission data showed that practices in less affected areas were quicker to return to normal reporting patterns, while those in more affected areas did not resume normal data transmissions for a few months. Rates of bronchitis increased after Sandy compared to the two prior years; while this was most likely attributable to a more severe flu season, it demonstrates the capacity of primary care networks to pick up on these types of post-emergency trends. Hurricane Sandy was the first disaster situation where PCIP was asked to assess public health impact, generating information that could contribute to aid and recovery efforts. This experience allowed us to explore the strengths and weaknesses of ambulatory EHR data in post-disaster settings. Data from ambulatory EHR networks can augment existing surveillance streams by providing sentinel population snapshots on clinically available indicators in near real time.

Applied Use of Composite Quality Measures for EHR-enabled Practices

OBJECTIVE To analyze the impact of three primary care practice transformation program models on performance: Meaningful Use (MU), Patient-Centered Medical Home (PCMH), and a pay-for-performance program (eHearts). DATA SOURCES/STUDY SETTING Data for seven quality measures (QM) were retrospectively collected from 192 small primary care practices between October 2009 and October 2012; practice demographics and program participation status were extracted from in-house data. STUDY DESIGN Bivariate analyses were conducted to measure the impact of individual programs, and a Generalized Estimating Equation model was built to test the impact of each program alongside the others. DATA COLLECTION/EXTRACTION METHODS Monthly data were extracted via a structured query data network and were compared to program participation status, adjusting for variables including practice size and patient volume. Seven QMs were analyzed related to smoking prevention, blood pressure control, BMI, diabetes, and antithrombotic therapy. PRINCIPAL FINDINGS In bivariate analysis, MU practices tended to perform better on process measures, PCMH practices on more complex process measures, and eHearts practices on measures for which they were incentivized; in multivariate analysis, PCMH recognition was associated with better performance on more QMs than any other program. CONCLUSIONS Results suggest each of the programs can positively impact performance. In our data, PCMH appears to have the most positive impact.

Evaluating the 2010 New York State HIV testing law in NYC ambulatory practices using electronic health records.

Introduction: To date, little research has been published on the impact that the transition from paper-based record keeping to the use of electronic health records (EHR) has on performance on clinical quality measures. This study examines whether small, independent medical practices improved in their performance on nine clinical quality measures soon after adopting EHRs. Methods: Data abstracted by manual review of paper and electronic charts for 6,007 patients across 35 small, primary care practices were used to calculate rates of nine clinical quality measures two years before and up to two years after EHR adoption. Results: For seven measures, population-level performance rates did not change before EHR adoption. Rates of antithrombotic therapy and smoking status recorded increased soon after EHR adoption; increases in blood pressure control occurred later. Rates of hemoglobin A1c testing, BMI recorded, and cholesterol testing decreased before rebounding; smoking cessation intervention, hemoglobin A1c control and cholesterol control did not significantly change. Discussion: The effect of EHR adoption on performance on clinical quality measures is mixed. To improve performance, practices may need to develop new workflows and adapt to different documentation methods after EHR adoption. Conclusions: In the short term, EHRs may facilitate documentation of information needed for improving the delivery of clinical preventive services. Policies and incentive programs intended to drive improvement should include in their timelines consideration of the complexity of clinical tasks and documentation needed to capture performance on measures when developing timelines, and should also include assistance with workflow redesign to fully integrate EHRs into medical practice.