Gabrielle Gundersen

Construction of a Multisite DataLink Using Electronic Health Records for the Identification, Surveillance, Prevention, and Management of Diabetes Mellitus: The SUPREME-DM Project

Introduction Electronic health record (EHR) data enhance opportunities for conducting surveillance of diabetes. The objective of this study was to identify the number of people with diabetes from a diabetes DataLink developed as part of the SUPREME-DM (SUrveillance, PREvention, and ManagEment of Diabetes Mellitus) project, a consortium of 11 integrated health systems that use comprehensive EHR data for research. Methods We identified all members of 11 health care systems who had any enrollment from January 2005 through December 2009. For these members, we searched inpatient and outpatient diagnosis codes, laboratory test results, and pharmaceutical dispensings from January 2000 through December 2009 to create indicator variables that could potentially identify a person with diabetes. Using this information, we estimated the number of people with diabetes and among them, the number of incident cases, defined as indication of diabetes after at least 2 years of continuous health system enrollment. Results The 11 health systems contributed 15,765,529 unique members, of whom 1,085,947 (6.9%) met 1 or more study criteria for diabetes. The nonstandardized proportion meeting study criteria for diabetes ranged from 4.2% to 12.4% across sites. Most members with diabetes (88%) met multiple criteria. Of the members with diabetes, 428,349 (39.4%) were incident cases. Conclusion The SUPREME-DM DataLink is a unique resource that provides an opportunity to conduct comparative effectiveness research, epidemiologic surveillance including longitudinal analyses, and population-based care management studies of people with diabetes. It also provides a useful data source for pragmatic clinical trials of prevention or treatment interventions.

Physician Service Attribution Methods for Examining Provision of Low-Value Care

Objectives: There has been significant research on provider attribution for quality and cost. Low-value care is an area of heightened focus, with little of the focus being on measurement; a key methodological decision is how to attribute delivered services and procedures. We illustrate the difference in relative and absolute physician- and panel-attributed services and procedures using overuse in cervical cancer screening. Study Design: A retrospective, cross-sectional study in an integrated health care system. Methods: We used 2013 physician-level data from Group Health Cooperative to calculate two utilization attributions: (1) panel attribution with the procedure assigned to the physician’s predetermined panel, regardless of who performed the procedure; and (2) physician attribution with the procedure assigned to the performing physician. We calculated the percentage of low-value cervical cancer screening tests and ranked physicians within the clinic using the two utilization attribution methods. Results: The percentage of low-value cervical cancer screening varied substantially between physician and panel attributions. Across the whole delivery system, median panel- and physician-attributed percentages were 15 percent and 10 percent, respectively. Among sampled clinics, panel-attributed percentages ranged between 10 percent and 17 percent, and physician-attributed percentages ranged between 9 percent and 13 percent. Within a clinic, median panel-attributed screening percentage was 17 percent (range 0 percent–27 percent) and physician-attributed percentage was 11 percent (range 0 percent–24 percent); physician rank varied by attribution method. Conclusions: The attribution method is an important methodological decision when developing low-value care measures since measures may ultimately have an impact on national benchmarking and quality scores. Cross-organizational dialogue and transparency in low-value care measurement will become increasingly important for all stakeholders.

PS2-21: Extracting Signs and Symptoms from Colonoscopy Reports Using NLP

Background/Aims The overarching goal of the Studying Colorectal Cancer: Effectiveness of Screening Strategies (SuCCESS) project at Group Health (GH) is to develop evidence to inform personalized colorectal cancer (CRC) screening recommendations. Specifically, we aim to study the comparative effectiveness of screening as practiced, evaluate the potential for personalizing screening and surveillance recommendations, and model the long-term comparative effectiveness of screening in a cohort of GH members enrolled between 1993 and 2015. To accomplish these goals, we used Natural Language Processing (NLP) to collect detailed information from colonoscopy reports in GH’s electronic medical record (EMR). Specifically, we extended an existing NLP system to identify whether signs or symptoms related to CRC were reported at the time of colonoscopy. Methods To prepare the NLP system to process all colonoscopy reports available in the EMR during the study period, we used a development set of 248 documents. The reports were randomly selected from colonoscopies performed in 2011 for which there was a corresponding pathology report on the same day. Trained medical record reviewers created the development set gold standard. The NLP system, developed in GATE (an open source text processing architecture), was an extension of a system created by Harkema et al. that used MetaMap as a resource to process all documents before sending the appropriate reports through a set of colonoscopy extraction rules. Colonoscopy results were consolidated by a post-processing and evaluation tool written in Python by the authors. Results The system performed admirably on CRC signs and symptoms. Aggregate sensitivity and specificity were 0.885 and 0.980, respectively, and positive predictive value (PPV), or precision, was 0.826, resulting in an F-score of 0.855. Conclusions In order to execute the SuCCESS project’s ambitious research aims we need high-quality data from tens of thousands of colonoscopy procedures. This information is not captured in a structured way in GH’s EMR, and manual abstraction of this information is not feasible, but our results show that NLP can reliably extract detailed information from the text reports. Future work includes improving the system’s precision, extracting patient and family history, and extracting results from the associated pathology reports.

PS2-03: Improving Quality Of Breast Cancer Surgery Through Development of a National Breast Cancer Surgical Outcomes Database

Background/Aims Surgical quality is typically measured using 30-day morbidity and mortality statistics – measures that are not very meaningful for procedures such as lumpectomies and mastectomies that have extremely high survival rates. The University of Vermont previously developed a single-site Breast Cancer Surgical Outcomes (BRCASO) database to capture meaningful quality measures such as breast conservation rates, positive margins rates, and number of procedures to complete breast cancer surgery. We extended the BRCASO database to three Cancer Research Network (CRN) institutions to study variation in breast cancer surgical quality across providers, facilities, and health plans. Methods The University of Vermont BRCASO data were collected on women diagnosed with breast cancer between 2003–2008 via medical record abstraction. In order to efficiently extend this work to CRN institutions, we collected electronic administrative data from each health plan (Group Health, Kaiser Permanente Colorado, and Marshfield Clinic) on women diagnosed with breast cancer between 2003–2008. Electronic administrative data included tumor registry information, Current Procedure Terminology codes for all breast cancer surgeries, study IDs for surgeons and surgical facilities, and demographic information including geocoded data. We supplemented the electronic administrative data with medical record abstraction to collect detailed information on surgical margins and lymph nodes. All medical record data were entered into a secure, online database developed using Silverlight. Results Using electronic administrative data, we determined 5,673 women met the study inclusion criteria. The CRN institutions pre-filled 30% (22 out of 72) of elements using electronic data. The remaining 50 elements required chart abstraction, which took approximately 45–60 minutes per record. Conclusions Electronic administrative data, while useful for research, have limitations and may not suit the needs of all studies. In our study, using electronic administrative data substantially decreased the amount of chart abstraction required at the CRN institutions; however, was not exclusively sufficient for all abstraction. Although manual abstraction was necessary for high quality data, the use of an electronic system greatly facilitated this effort and helped expand the BRCASO database to become the only multi-site source of detailed surgical quality information in the country.