Michele Morris

Challenges in adapting existing clinical natural language processing systems to multiple, diverse health care settings

Objective Widespread application of clinical natural language processing (NLP) systems requires taking existing NLP systems and adapting them to diverse and heterogeneous settings. We describe the challenges faced and lessons learned in adapting an existing NLP system for measuring colonoscopy quality. Materials and Methods Colonoscopy and pathology reports from 4 settings during 2013-2015, varying by geographic location, practice type, compensation structure, and electronic health record. Results Though successful, adaptation required considerably more time and effort than anticipated. Typical NLP challenges in assembling corpora, diverse report structures, and idiosyncratic linguistic content were greatly magnified. Discussion Strategies for addressing adaptation challenges include assessing site-specific diversity, setting realistic timelines, leveraging local electronic health record expertise, and undertaking extensive iterative development. More research is needed on how to make it easier to adapt NLP systems to new clinical settings. Conclusions A key challenge in widespread application of NLP is adapting existing systems to new clinical settings.

Public reporting of colonoscopy quality is associated with an increase in endoscopist adenoma detection rate

BACKGROUND Colonoscopy is the predominant method for colorectal cancer screening in the United States. Previous studies have documented variation across physicians in colonoscopy quality as measured by the adenoma detection rate (ADR). ADR is the primary quality measure of colonoscopy examinations and an indicator of the likelihood of subsequent colorectal cancer. There is interest in mechanisms to improve the ADR. In Central Illinois, a local employer and a quality improvement organization partnered to publically report physician colonoscopy quality. OBJECTIVE We assessed whether this initiative was associated with an improvement in the ADR. DESIGN We compared ADRs before and after public reporting at a private practice endoscopy center with 11 gastroenterologists in Peoria, Illinois, who participated in the initiative. To generate the ADR, colonoscopy and pathology reports from examinations performed over 4 years at the endoscopy center were analyzed by using previously validated natural language processing software. SETTING A central Illinois endoscopy center. RESULTS The ADR in the pre-public reporting period was 34.3% and 39.2% in the post-public reporting period (an increase of 4.9%, P < .001). The increase in the right-sided ADR was 5.1% (P < .01), whereas the increase in the left-sided ADR was 2.1% (P < .05). The increase in the ADR was 7.8% for screening colonoscopies (P < 0.05) and 3.5% for nonscreening colonoscopies (P < .05). All but 1 physicians ADR increased (range -2.7% to 10.5%). There was no statistically significant change in the advanced ADR (increase of 0.8%, P = .22). LIMITATIONS There was no concurrent control group to assess whether the increased ADR was due to a secular trend. CONCLUSION A public reporting initiative on colonoscopy quality was associated with an increase in ADR.

Variation in Pathologist Classification of Colorectal Adenomas and Serrated Polyps

Objectives:Endoscopist quality measures such as adenoma detection rate (ADR) and serrated polyp detection rates (SPDRs) depend on pathologist classification of histology. Although variation in pathologic interpretation is recognized, we add to the literature by quantifying the impact of pathologic variability on endoscopist performance.Methods:We used natural language processing to abstract relevant data from colonoscopy and related pathology reports performed over 2 years at four clinical sites. We quantified each pathologist’s likelihood of classifying polyp specimens as adenomas or serrated polyps. We estimated the impact on endoscopists’ ADR and SPDR of sending their specimens to pathologists with higher or lower classification rates.Results:We observed 85,526 colonoscopies performed by 119 endoscopists; 50,453 had a polyp specimen, which were analyzed by 48 pathologists. There was greater variation across pathologists in classification of serrated polyps than in classification of adenomas. We estimate the endoscopist’s average SPDR would be 0.5% if all their specimens were analyzed by the pathologist in our sample with the lowest classification rate and 12.0% if all their specimens were analyzed by the pathologist with the highest classification rate. In contrast, the endoscopist’s average ADR would be 28.5% and 42.4% if their specimens were analyzed by the pathologist with lowest and highest classification rate, respectively.Conclusions:There is significant variation in pathologic interpretation, which more substantially affects endoscopist SPDR than ADR.

Incidence of Interval Colorectal Cancer Attributable to an Endoscopist in Clinical Practice

BACKGROUND AND AIMS Endoscopists who encounter an interval colorectal cancer (I-CRC) may be concerned about the implications because I-CRCs may represent a lapse in colonoscopy quality and a missed opportunity for prevention. We wanted to determine the I-CRC rate per colonoscopy examination and to examine the effect of colonoscopy volume and adenoma detection rate (ADR) on the number of I-CRCs attributable to an endoscopist. METHODS We determined the rate of I-CRC diagnosis per outpatient colonoscopy examination by measuring the incidence of CRC diagnosis in practice and by assessing, via literature review, the percentage of cancers that are interval. We also estimated the number of attributable I-CRCs as a function of ADR and colonoscopy volume. RESULTS Among 93,562 colonoscopies performed in 2013 to 2015 by 120 physicians in 4 diverse U.S. medical centers, 526 CRCs were diagnosed (.6%). Of 149,556 CRCs in the published literature, 7958 were I-CRCs (5.25% ± .94%). With rates of .6% (CRC per colonoscopy) and 5.25% (I-CRC per CRC), the rate of I-CRC is 1 per 3174 colonoscopies (95% confidence interval, 1 per 2710 to 1 per 3875). An endoscopist at the median of outpatient colonoscopy volume (316/year) in the lowest ADR quintile of detection (7%-19%) would have an I-CRC attributed every 8.2 years, or 4.2 I-CRCs in a 35-year career, versus every 16.7 years, or 2.0 I-CRCs in a 35-year career, for an endoscopist in the highest ADR quintile (33%-52%). CONCLUSIONS An average-volume endoscopist will have 2 to 4 attributable I-CRCs in a 35-year career, but the frequency will vary depending on colonoscopy volume and ADR.

Endoscopist factors that influence serrated polyp detection: a multicenter study

BACKGROUND Serrated polyps are important colorectal cancer precursors that are variably detected during colonoscopy. We measured serrated polyp detection rate (SPDR) in a large, multicenter, cross-sectional study of colonoscopy quality to identify drivers of SPDR variation. METHODS Colonoscopy and pathology reports were collected for a 2-year period (10/2013-9/2015) from four sites across the United States. Data from reports, including size, location, and histology of polyps, were abstracted using a validated natural language processing algorithm. SPDR was defined as the proportion of colonoscopies with ≥ 1 serrated polyp (not including hyperplastic polyps). Multivariable logistic regression was performed to determine endoscopist characteristics associated with serrated polyp detection. RESULTS A total of 104 618 colonoscopies were performed by 201 endoscopists who varied with respect to specialty (86 % were gastroenterologists), sex (18 % female), years in practice (range 1 - 51), and number of colonoscopies performed during the study period (range 30 - 2654). The overall mean SPDR was 5.1 % (SD 3.8 %, range 0 - 18.8 %). In multivariable analysis, gastroenterology specialty training (odds ratio [OR] 1.89, 95 % confidence interval [CI] 1.33 - 2.70), fewer years in practice (≤ 9 years vs. ≥ 27 years: OR 1.52, 95 %CI 1.14 - 2.04)], and higher procedure volumes (highest vs. lowest quartile: OR 1.77, 95 %CI 1.27 - 2.46)] were independently associated with serrated polyp detection. CONCLUSIONS Gastroenterology specialization, more recent completion of training, and greater procedure volume are associated with serrated polyp detection. These findings imply that both repetition and training are likely to be important contributors to adequate detection of these important cancer precursors. Additional efforts to improve SPDR are needed.