Roy Pardee

Validity of Eight Integrated Healthcare Delivery Organizations' Administrative Clinical Data to Capture Breast Cancer Chemotherapy Exposure

Background: Cancer Research Network (CRN) sites use administrative data to populate their Virtual Data Warehouse (VDW). However, information on VDW chemotherapy data validity is limited. The purpose of this study was to assess the validity of VDW chemotherapy data. Methods: This was a retrospective cohort study of women ≥18 years with incident, invasive breast cancer diagnosed between January 1999 and December 2007. Pharmacy and procedure chemotherapy data were extracted from each sites VDW. Random samples of 50 patients stratified on trastuzumab, anthracyclines, and no chemotherapy exposure was selected from each site for detailed chart abstraction. Weighted sensitivities and specificities of VDW compared with abstracted data were calculated. Cumulative doses calculated from VDW data were compared with doses obtained from the medical chart review. Results: The cohort included 13,497 patients with 6,456 (48%) chart review eligible. Patients in the sample (N = 400) had a mean age of 65 years. Trastuzumab, anthracycline, and other chemotherapy weighted sensitivities were 95%, 97%, and 100%, respectively; specificities were 99%, 99%, and 93%, respectively; positive predictive values were 96%, 99%, and 55%, respectively; and negative predictive values were 99%, 96%, and 100%. Trastuzumab and anthracyclines VDW mean doses were 873 and 386 mg, respectively, whereas abstracted mean doses were 1,734 and 369 mgs, respectively (R2 = 0.14, P < 0.01 and R2 = 0.05, P = 0.03, respectively). Conclusions: Sensitivities and specificities for CRN chemotherapy VDW data were high and dosages were correlated with chart information. Impact: The findings support the use of CRN data in evaluating chemotherapy exposures and related outcomes. Cancer Epidemiol Biomarkers Prev; 21(4); 673–80. ©2012 AACR.

Performance of Claims-Based Algorithms for Identifying Heart Failure and Cardiomyopathy Among Patients Diagnosed with Breast Cancer

Background:Cardiotoxicity is a known complication of certain breast cancer therapies, but rates come from clinical trials with design features that limit external validity. The ability to accurately identify cardiotoxicity from administrative data would enhance safety information. Objective:To characterize the performance of clinical coding algorithms for identification of cardiac dysfunction in a cancer population. Research Design:We sampled 400 charts among 6460 women diagnosed with incident breast cancer, tumor size ≥2 cm or node positivity, treated within 8 US health care systems between 1999 and 2007. We abstracted medical records for clinical diagnoses of heart failure (HF) and cardiomyopathy (CM) or evidence of reduced left ventricular ejection fraction. We then assessed the performance of 3 different International Classification of Diseases, 9th Edition (ICD-9)-based algorithms. Results:The HF/CM coding algorithm designed a priori to balance performance characteristics provided a sensitivity of 62% (95% confidence interval, 40%–80%), specificity of 99% (range, 97% to 99%), positive predictive value (PPV) of 69% (range, 45% to 85%), and negative predictive value (NPV) of 98% (range, 96% to 99%). When applied only to incident HF/CM (ICD-9 codes and gold standard diagnosis both occurring after breast cancer diagnosis) in patients exposed to anthracycline and/or trastuzumab therapy, the PPV was 42% (range, 14% to 76%). Conclusions:Claims-based algorithms have moderate sensitivity and high specificity for identifying HF/CM among patients with invasive breast cancer. As the prevalence of HF/CM among the breast cancer population is low, ICD-9 codes have high NPV but only moderate PPV. These findings suggest a significant degree of misclassification due to HF/CM overcoding versus incomplete clinical documentation of HF/CM in the medical record.

Physician Service Attribution Methods for Examining Provision of Low-Value Care

Objectives: There has been significant research on provider attribution for quality and cost. Low-value care is an area of heightened focus, with little of the focus being on measurement; a key methodological decision is how to attribute delivered services and procedures. We illustrate the difference in relative and absolute physician- and panel-attributed services and procedures using overuse in cervical cancer screening. Study Design: A retrospective, cross-sectional study in an integrated health care system. Methods: We used 2013 physician-level data from Group Health Cooperative to calculate two utilization attributions: (1) panel attribution with the procedure assigned to the physician’s predetermined panel, regardless of who performed the procedure; and (2) physician attribution with the procedure assigned to the performing physician. We calculated the percentage of low-value cervical cancer screening tests and ranked physicians within the clinic using the two utilization attribution methods. Results: The percentage of low-value cervical cancer screening varied substantially between physician and panel attributions. Across the whole delivery system, median panel- and physician-attributed percentages were 15 percent and 10 percent, respectively. Among sampled clinics, panel-attributed percentages ranged between 10 percent and 17 percent, and physician-attributed percentages ranged between 9 percent and 13 percent. Within a clinic, median panel-attributed screening percentage was 17 percent (range 0 percent–27 percent) and physician-attributed percentage was 11 percent (range 0 percent–24 percent); physician rank varied by attribution method. Conclusions: The attribution method is an important methodological decision when developing low-value care measures since measures may ultimately have an impact on national benchmarking and quality scores. Cross-organizational dialogue and transparency in low-value care measurement will become increasingly important for all stakeholders.

PS2-43: VDW Data Sources: Group Health

Background The Virtual Data Warehouse (VDW) was created as a mechanism for producing comparable data across sites for purposes of proposing and conducting research. It is “virtual” in the sense that the data remain at the local sites; there is no multi-site physical database at a centralized data coordinating center. At the core of the VDW are a series of standardized file definitions. Content areas and data elements that are commonly required for research studies are identified, and data dictionaries are created for each of the content areas, specifying a common format for each of the elements—variable name, label, description, code values, and value labels. Local site programmers have mapped the data elements from their HMO’s data systems into this standardized set of variable definitions, names, and codes, as well as onto standardized SAS file formats. This common structure of the VDW files enables a SAS analyst at one site to write one program to extract and/or analyze data at all participating sites. Methods This poster demonstrates the wide range of data sources used at Group Health to feed information into our local implementation of the VDW datasets. Results The Group Health local implementation of the VDW contains detailed medical information on Group Health members. These files contain details on 143 million pharmacy dispensings (1977–2011), nearly 98 million unique medical encounters (1993–2011), including .75 million hospitalizations, 68 million ambulatory visits, 141 million diagnoses, and 224 million procedures. We have some 11 million Vital Signs observations, and 82 million lab results. The VDW Enrollment and Demographic files are derived from several historical and current membership files; the VDW Pharmacy and utilization files are derived from internal Group Health systems plus claims files; the VDW tumor data is derived from the Puget Sound SEER Registry. Conclusions The VDW at Group Health provides an easily employed unified central repository of data from all available source files. This resource enables the sharing of compatible data in multi-site studies, and also improves programming efficiency, accuracy, and completeness for local single site studies by expending resources to link these legacy systems only once.

PS2-01: A System for Computer-Assisted Rapid Case Ascertainment of Breast, Lung and Colorectal Cancers

Background and Aims: An intervention trial requiring recruitment of new cancer cases prompted this investigation into text classifier programs to identify pathology reports describing malignancies. Text classifiers are among the simplest Natural Language Processing methods, and are particularly useful for filtering large streams of documents. Our classifiers allowed study staff to concentrate their time on the reports most likely to result in recruitable subjects. The proposed presentation will describe our use of text classifiers as part of an efficient computer-assisted recruitment pipeline. Methods: We gathered corpora of pathology reports for breast, lung and colorectal tissue, each with a ‘gold standard’ assessment of whether they discussed a malignancy. We then randomly divided each into 75% training and 25% evaluation subsamples, trained the classifier on the training subsample and compared its categorizations to the gold standard in the evaluation subsample. We performed this training/evaluation cycle repeatedly, generating a distribution of predictive value statistics, thereby getting insight into the classifiers’ sensitivity to the particular random division of corpus reports. More importantly, it allowed us to experiment with tweaks to the classifiers and to the basic text processing that preceded them, and quickly evaluate whether they contributed to the predictive power of the classifier. Results: Several enhancements contributed to the accuracy of the system. In particular, taking multi-word phrases in addition to individual words as features increased the classifier’s predictive power. To our surprise, ‘stemming’ individual words (that is, reducing them to their root forms) did not have a perceptible effect. Once the classifiers were optimized, we were able to cover approximately 1600 path reports/week in approximately 30 minutes of a Research Specialist’s time. As an added benefit, when the RS found that a report was flagged in error, the classifier was trained on that report, thereby improving its future performance. Conclusions: Text classifiers are an effective tool for optimizing the use of staff time in rapidly ascertaining cancer cases for recruitment. Furthermore, because the classifiers’ training (including the corrections made during the study) is easily reduced to a file on disk, it becomes an independent asset, useful for future studies needing to do rapid ascertainment of cancer.

PS1-41: Just Add Data: Implementing an Event-Based Data Model for Clinical Trial Tracking

Background/Aims Clinical research trials often have similar fundamental tracking needs, despite being quite variable in their specific logic and activities. A model tracking database that can be quickly adapted by a variety of studies has the potential to achieve significant efficiencies in database development and maintenance. Methods Over the course of several different clinical trials, we have developed a database model that is highly adaptable to a variety of projects. Rather than hard-coding each specific event that might occur in a trial, along with its logical consequences, this model considers each event and its parameters to be a data record in its own right. Each event may have related variables (metadata) describing its prerequisites, subsequent events due, associated mailings, or events that it overrides. The metadata for each event is stored in the same record with the event name. When changes are made to the study protocol, no structural changes to the database are needed. One has only to add or edit events and their metadata. Changes in the event metadata automatically determine any related logic changes. In addition to streamlining application code, this model simplifies communication between the programmer and other team members. Database requirements can be phrased as changes to the underlying data, rather than to the application code. The project team can review a single report of events and metadata and easily see where changes might be needed. In addition to benefitting from streamlined code, the front end database application can also implement useful standard features such as automated mail merges and to do lists. Results The event-based data model has proven itself to be robust, adaptable and user-friendly in a variety of study contexts. We have chosen to implement it as a SQL Server back end and distributed Access front end. Interested readers may request a copy of the Access front end and scripts for creating the back end database. Discussion An event-based database with a consistent, robust set of features has the potential to significantly reduce development time and maintenance expense for clinical trial tracking databases.

C-C1-02: Validity of HMO Administrative Data for Breast Cancer Chemotherapy Exposure

Background/Aims Chemotherapy-related toxicity, while a major focus of randomized clinical trials, has been understudied in population-based research. Eight Cancer Research Network (CRN) sites used Virtual Data Warehouse (VDW) data to examine the risk of cardiotoxicity following chemotherapy for invasive breast cancer. However, information on VDW chemotherapy data validity is limited. This study assessed the validity of VDW data related to the reception and dosages of cardiotoxic chemotherapeutic agents among breast cancer (BC) patients. Methods This was a retrospective, cohort study of women =18 years diagnosed with incident, invasive BC between January 1999 and December 2007. Pharmacy and procedure chemotherapy data were extracted from each site’s administrative databases for up to one year following cancer diagnosis date. Random samples of 50 patients stratified on trastuzumab, anthracyclines, and no chemotherapy exposure were selected from each CRN site for detailed medical chart abstraction. We calculated weighted sensitivities and specificities (and 95% confidence intervals [CI]) of using administrative data to accurately capture chemotherapy treatment compared to medical records. Median cumulative doses calculated from administrative data were compared to median doses obtained from the medical chart using the Spearman’s correlation coefficient. Results The total cohort included 13,497 BC patients. Patients in the random sample (n=400) had a mean age of 65 (±14) years and were primarily white with most tumors diagnosed at AJCC stage 1 or 2. From the sample, 20% (80/400), 38% (152/400), and 40% (158/400) of patients had VDW exposure to trastuzumab, an anthracycline, and any type chemotherapy, respectively. Trastuzumab, anthracycline, and any chemotherapy sensitivities were 97% (CI=94%–98%), 97% (CI=93%–98%), and 99% (CI=97%–99%), respectively; and specificities were 99% (CI=98%–100%), 99% (CI=96%–99%), and 86% (CI=79%–91%), respectively. Median doses for trastuzumab and anthracyclines from administrative data were 227 mgs and 420 mgs, respectively, while median doses from the medical chart were 2370 mgs and 416 mgs, respectively (r=0.60, p<0.001 and r=0.23, p=0.030, respectively). Conclusions Sensitivities and specificities for CRN chemotherapy administrative data were high and dosages were correlated moderately with chart information. These findings support the use of CRN data in evaluating chemotherapy exposures and related outcomes.

PS2-49: The VDW Census File: Strengths, Issues, and Recommendations for the Future

Background/Aims The VDW Census file contains information about our members such as income and education. The information is estimated based on data collected by the decennial census for the geographic areas around our members’ addresses (identified by geocodes).The VDW Census file was initially developed based on 2000 census data. Data from the 2010 census is now being released, and in many cases, the 2010 census data structure looks different than the 2000 census. For example, income and education were no longer collected in the 2010 census, but have been moved to a supplementary survey (American Community Survey, or ACS) collected over a different time period. Methods The VDW Census workgroup reviewed the changes in the 2010 census data and compared the available information to the data in the existing VDW Census file. Many of the fields from the ‘long-form’ in the 2000 census were moved to the ACS. Where possible, the workgroup mapped the existing VDW Census fields to fields with similar, if not equivalent, data in the new census data structure. The workgroup also considered whether new data available from the ACS might be of interest to HMORN investigators. Where major changes in information availability were anticipated, the workgroup used online surveys to solicit feedback from a larger group of HMORN analysts and investigators. Results Our primary goal was to identify changes to the specification that would allow updating the Census file to 2010 data. Our secondary goal was to identify that would allow sites to maintain multiple sets of census data to accommodate projects with different time periods. To accommodate multiple years of census data and reduce file sizes, we recommend splitting the VDW Census file into two files: one tracking members’ geocoded addresses, and another containing the census data for all available geocodes. Conclusions We found significant changes in the 2010 census data that require considerable changes to the VDW Census file. Once agreement has been reached on the new Census specification, a subcommittee will develop code to build a new VDW Census data file and recommendations on building a file containing members’ geocodes over time.

C-A5-04: A Simple, Accurate SAS Algorithm for Electronic Abstraction of Race from Digitized Progress Notes

Background and Aims: Individual-level race/ethnicity is important for research into causes and consequences of health disparities. For various non-research reasons, it has rarely been collected on enrollees in integrated delivery systems. Individual-level race/ethnicity can be found in medical record documentation. Manual abstraction on large numbers of medical records is costly. We developed a simple SAS algorithm for electronic abstraction of white and African American race from digitized progress notes and evaluated its accuracy by comparing electronically abstracted race with other data sources. Methods: A simple SAS algorithm, based on text search strings (e.g. white male, African American woman), scanned digitized progress notes for provider face-to-face visits from 2005 through July 2009 in Kaiser Permanente Georgia’s (KPG) and Group Health Cooperative’s (GHC) electronic medical record systems. White and African American race was abstracted. If the patient had more than 1 visit with abstracted race, the patient was classified using the earliest visit. Abstracted race was linked at the individual-level to survey datasets with self-reported race (2005 survey of working age adults, 2007 survey of adults with hypertension, 2000–2005 Medicare surveys) and mother’s race on 2000–2006 birth certificates. White and African American race was abstracted from GHC progress notes from 2005 through July 2009 using the same algorithm and compared to self-reported race on health risk appraisals. Accuracy of the SAS algorithm was assessed by overall proportion matching race from the other datasets, Cohen’s kappa, and McNemar’s test. Results: White or African American race was electronically abstracted for 56,261 KPG and 6,427 GHC enrollees. Abstracted race matched race from the other datasets in 97–99% of enrollees. Cohen’s kappas were highly significant (p<0.05), ranging from 0.939 ± 0.013 (N=657 matches with hypertension survey records) to 0.994 ± 0.006 (N=518 matches with Medicare surveys). McNemar’s tests were marginally significant for several datasets; and, misclassification was not systematically biased toward white or African American race. Conclusions: The SAS algorithm was highly accurate in electronically abstracting white and African American race from digitized progress notes of provider visits at KPG and GHC. We are expanding the evaluation to include additional sites and additional race/ ethnic categories (e.g. Asian, Hispanic).