Colleen M. McCullough

Validity of electronic health record-derived quality measurement for performance monitoring

Background Since 2007, New York Citys primary care information project has assisted over 3000 providers to adopt and use a prevention-oriented electronic health record (EHR). Participating practices were taught to re-adjust their workflows to use the EHR built-in population health monitoring tools, including automated quality measures, patient registries and a clinical decision support system. Practices received a comprehensive suite of technical assistance, which included quality improvement, EHR customization and configuration, privacy and security training, and revenue cycle optimization. These services were aimed at helping providers understand how to use their EHR to track and improve the quality of care delivered to patients. Materials and Methods Retrospective electronic chart reviews of 4081 patient records across 57 practices were analyzed to determine the validity of EHR-derived quality measures and documented preventive services. Results Results from this study show that workflow and documentation habits have a profound impact on EHR-derived quality measures. Compared with the manual review of electronic charts, EHR-derived measures can undercount practice performance, with a disproportionately negative impact on the number of patients captured as receiving a clinical preventive service or meeting a recommended treatment goal. Conclusion This study provides a cautionary note in using EHR-derived measurement for public reporting of provider performance or use for payment.

Patient-Centered Medical Home Among Small Urban Practices Serving Low-Income and Disadvantaged Patients

PURPOSE Research on the patient-centered medical home (PCMH) model and practice redesign has not focused on the unique challenges and strengths of very small primary care practices serving disadvantaged patient populations. We analyzed the practice characteristics, prior experiences, and dimensions of the PCMH model that exist in such practices participating in the Primary Care Information Project (PCIP) of the New York City Department of Mental Health and Hygiene. METHODS We obtained descriptive data, focusing on PCMH, for 94 primary care practices with 5 or fewer clinicians serving high volumes of Medicaid and minority patient populations in New York City. Data included information extracted from PCIP administrative data and survey data collected specifically for this study. RESULTS Survey results indicated substantial implementation of key aspects of the PCMH among small practices serving disadvantaged patient populations, despite considerable potential challenges to achieving PCMH implementation. Practices tended to use few formal mechanisms, such as formal care teams and designated care or case managers, but there was considerable evidence of use of informal team-based care and care coordination nonetheless. It appears that many of these practices achieved the spirit, if not the letter, of the law in terms of key dimensions of PCMH. CONCLUSIONS Small practices can achieve important aspects of the PCMH model of primary care, often with informal rather than formal mechanisms and strategies. The use of flexible, less formal strategies is important to keep in mind when considering implementation and assessment of PCMH-like initiatives in small practices.

The intended and unintended consequences of quality improvement interventions for small practices in a community-based electronic health record implementation project.

Background:Despite the rapid rise in the implementation of electronic health records (EHR), commensurate improvements in health care quality have not been consistently observed. Objectives:To evaluate whether the implementation of EHRs and complementary interventions—including clinical decision support, technical assistance, and financial incentives—improved quality of care. Research Design:The study included 143 practices that implemented EHRs as part of the Primary Care Information Project—a long-standing community-based EHR implementation initiative. A total of 71 practices were randomized to receive financial incentives and quality feedback and 72 were randomized to feedback alone. All practices received technical assistance and had clinical decision support in their EHR. Using data from 2009 to 2011, we estimated measure-level fixed effects models to evaluate the association between exposure to clinical decision support, technical assistance, financial incentives, and quality of care. Associations were estimated separately for 4 cardiovascular measures that were rewarded by the financial incentive program and 4 measures that were not rewarded by incentives. Results:Financial incentives for quality were consistently associated with higher performance for the incentivized measures [+10.1 percentage points at 18 mo of exposure (approximately +22%), P<0.05] and lower performance for the unincentivized measures [−8.3 percentage points at 12 mo of exposure (approximately −20%), P<0.05]. Technical assistance was associated with higher quality for the unincentivized measures, but not for the incentivized measures. Conclusions:Technical assistance and financial incentives—alongside EHR implementation—can improve quality of care. Financial incentives for quality may not result in similar improvements for incentivized and unincentivized measures.

Syndromic Surveillance during Pandemic (H1N1) 2009 Outbreak, New York, New York, USA

We compared emergency department and ambulatory care syndromic surveillance systems during the pandemic (H1N1) 2009 outbreak in New York City. Emergency departments likely experienced increases in influenza-like-illness significantly earlier than ambulatory care facilities because more patients sought care at emergency departments, differences in case definitions existed, or a combination thereof.

Sustained Improvement in Clinical Preventive Service Delivery Among Independent Primary Care Practices After Implementing Electronic Health Record Systems

Introduction Studies showing sustained improvements in the delivery of clinical preventive services are limited. Fewer studies demonstrate sustained improvements among independent practices that are not affiliated with hospitals or integrated health systems. This study examines the continued improvement in clinical quality measures for a group of independent primary care practices using electronic health records (EHRs) and receiving technical support from a local public health agency. Methods We analyzed clinical quality measure performance data from a cohort of primary care practices that implemented an EHR at least 3 months before October 2009, the study baseline. We assessed trends for 4 key quality measures: antithrombotic therapy, blood pressure control, smoking cessation intervention, and hemoglobin A1c (HbA1c) testing based on monthly summary data transmitted by the practices. Results Of the 151 practices, 140 were small practices and 11 were community health centers; average time using an EHR was 13.7 months at baseline. From October 2009 through October 2011, average rates increased for antithrombotic therapy (from 58.4% to 74.8%), blood pressure control (from 55.3% to 64.1%), HbA1c testing (from 46.4% to 57.7%), and smoking cessation intervention (from 29.3% to 46.2%). All improvements were significant. Conclusion During 2 years, practices showed significant improvement in the delivery of several key clinical preventive services after implementing EHRs and receiving support services from a public health agency.

Factors Related to Clinical Quality Improvement for Small Practices Using an EHR

OBJECTIVE To analyze the impact of three primary care practice transformation program models on performance: Meaningful Use (MU), Patient-Centered Medical Home (PCMH), and a pay-for-performance program (eHearts). DATA SOURCES/STUDY SETTING Data for seven quality measures (QM) were retrospectively collected from 192 small primary care practices between October 2009 and October 2012; practice demographics and program participation status were extracted from in-house data. STUDY DESIGN Bivariate analyses were conducted to measure the impact of individual programs, and a Generalized Estimating Equation model was built to test the impact of each program alongside the others. DATA COLLECTION/EXTRACTION METHODS Monthly data were extracted via a structured query data network and were compared to program participation status, adjusting for variables including practice size and patient volume. Seven QMs were analyzed related to smoking prevention, blood pressure control, BMI, diabetes, and antithrombotic therapy. PRINCIPAL FINDINGS In bivariate analysis, MU practices tended to perform better on process measures, PCMH practices on more complex process measures, and eHearts practices on measures for which they were incentivized; in multivariate analysis, PCMH recognition was associated with better performance on more QMs than any other program. CONCLUSIONS Results suggest each of the programs can positively impact performance. In our data, PCMH appears to have the most positive impact.

Quality Measure Performance in Small Practices Before and After Electronic Health Record Adoption

Introduction: To date, little research has been published on the impact that the transition from paper-based record keeping to the use of electronic health records (EHR) has on performance on clinical quality measures. This study examines whether small, independent medical practices improved in their performance on nine clinical quality measures soon after adopting EHRs. Methods: Data abstracted by manual review of paper and electronic charts for 6,007 patients across 35 small, primary care practices were used to calculate rates of nine clinical quality measures two years before and up to two years after EHR adoption. Results: For seven measures, population-level performance rates did not change before EHR adoption. Rates of antithrombotic therapy and smoking status recorded increased soon after EHR adoption; increases in blood pressure control occurred later. Rates of hemoglobin A1c testing, BMI recorded, and cholesterol testing decreased before rebounding; smoking cessation intervention, hemoglobin A1c control and cholesterol control did not significantly change. Discussion: The effect of EHR adoption on performance on clinical quality measures is mixed. To improve performance, practices may need to develop new workflows and adapt to different documentation methods after EHR adoption. Conclusions: In the short term, EHRs may facilitate documentation of information needed for improving the delivery of clinical preventive services. Policies and incentive programs intended to drive improvement should include in their timelines consideration of the complexity of clinical tasks and documentation needed to capture performance on measures when developing timelines, and should also include assistance with workflow redesign to fully integrate EHRs into medical practice.