Ronilda Lacson

Factors Associated With Radiologists' Adherence to Fleischner Society Guidelines for Management of Pulmonary Nodules

PURPOSE In 2005, the Fleischner Society guidelines (FSG) for managing pulmonary nodules detected on CT scans were published. The aim of this study was to evaluate adherence to the FSG, adjusting for demographic and clinical variables that may contribute to adherence. METHODS Radiology reports were randomly obtained for 1,100 chest and abdominal CT scans performed between January and June 2010 in a tertiary hospitals emergency department and outpatient clinics. An automated document retrieval system using natural language processing was used to identify patients with pulmonary nodules from the data set. Features relevant to evaluating variation in adherence to the FSG, including age, sex, race, nodule size, and scan site (eg, the emergency department) and type, were extracted by manual review from reports retrieved using natural language processing. All variables were entered into a logistic regression model. RESULTS Three hundred fifteen reports were identified to have pulmonary nodules, 75 of which were for patients with concurrent malignancies or aged < 35 years. Of the remaining 240 reports, 34% of recommendations for pulmonary nodules were adherent to the FSG. Nodule size demonstrated an association with guideline adherence, with adherence highest in the >4-mm to 6-mm nodule group (P = .04) and progressively diminishing for smaller and bigger nodules. CONCLUSIONS Pulmonary nodules are prevalent findings on chest and abdominal CT scans. Although most radiologists recommend follow-up imaging for these findings, recommendations for pulmonary nodules were consistent with the FSG in 34% of radiology reports. Nodule size demonstrated an association with guideline adherence, after adjusting for key variables.

Information from Searching Content with an Ontology-Utilizing Toolkit (iSCOUT).

Radiology reports are permanent legal documents that serve as official interpretation of imaging tests. Manual analysis of textual information contained in these reports requires significant time and effort. This study describes the development and initial evaluation of a toolkit that enables automated identification of relevant information from within these largely unstructured text reports. We developed and made publicly available a natural language processing toolkit, Information from Searching Content with an Ontology-Utilizing Toolkit (iSCOUT). Core functions are included in the following modules: the Data Loader, Header Extractor, Terminology Interface, Reviewer, and Analyzer. The toolkit enables search for specific terms and retrieval of (radiology) reports containing exact term matches as well as similar or synonymous term matches within the text of the report. The Terminology Interface is the main component of the toolkit. It allows query expansion based on synonyms from a controlled terminology (e.g., RadLex or National Cancer Institute Thesaurus [NCIT]). We evaluated iSCOUT document retrieval of radiology reports that contained liver cysts, and compared precision and recall with and without using NCIT synonyms for query expansion. iSCOUT retrieved radiology reports with documented liver cysts with a precision of 0.92 and recall of 0.96, utilizing NCIT. This recall (i.e., utilizing the Terminology Interface) is significantly better than using each of two search terms alone (0.72, p = 0.03 for liver cyst and 0.52, p = 0.0002 for hepatic cyst). iSCOUT reliably assembled relevant radiology reports for a cohort of patients with liver cysts with significant improvement in document retrieval when utilizing controlled lexicons.

Towards large-scale sample annotation in gene expression repositories

BackgroundLarge repositories of biomedical research data are most useful to translational researchers if their data can be aggregated for efficient queries and analyses. However, inconsistent or non-existent annotations describing important sample details such as name of tissue or cell line, histopathological type, and subject characteristics like demographics, treatment, and survival are seldom present in data repositories, making it difficult to aggregate data.ResultsWe created a flexible software tool that allows efficient annotation of samples using a controlled vocabulary, and report on its use for the annotation of over 12,500 samples.ConclusionWhile the amount of data is very large and seemingly poorly annotated, a lot of information is still within reach. Consistent tool-based re-annotation enables many new possibilities for large scale interpretation and analyses that would otherwise be impossible.

Four-year impact of an alert notification system on closed-loop communication of critical test results.

OBJECTIVE One of the patient safety goals proposed by the Joint Commission urges hospitals to develop a policy for communicating critical test results and to measure adherence to that policy. We evaluated the impact of an alert notification system on policy adherence for communicating critical imaging test results to referring providers and assessed system adoption over the first 4 years after implementation. MATERIALS AND METHODS This study was performed in a 753-bed academic medical center. The intervention, an automated alert notification system for critical results, was implemented in January 2010. The primary outcome was adherence to institutional policy for timely closed-loop communication of critical imaging results, and the secondary outcome was system adoption. Policy adherence was determined through manual review of a random sample of radiology reports from the first 4 years after the intervention (n = 37,604) compared with baseline outcomes 1 year before the intervention (n = 9430). Adoption was evaluated by quantifying the use of the system overall and the proportion of alerts that used noninterruptive communication as a percentage of all reports generated by 320 radiologists (n = 1,538,059). A statistical analysis of the trend at 6-month intervals over 4 years was performed using a chi-square trend test. RESULTS Adherence to the policy increased from 91.3% before the intervention to 95.0% after the intervention (p < 0.0001). There was a ninefold increase in the critical results communicated via the system (chi-square trend test, p < 0.0001). During the first 4 years after the intervention, 41,445 alerts (41% of the total number of alerts) used the systems noninterruptive process for communicating less urgent critical results, which was substantially unchanged over the 4 years postintervention, thus reducing unnecessary paging interruptions. CONCLUSION An automated alert notification system for communicating critical imaging results was successfully adopted and was associated with increased adherence to institutional policy for communicating critical test results and with reduced workflow interruptions.

DSGeo: Software tools for cross-platform analysis of gene expression data in GEO

The Gene Expression Omnibus (GEO) is the largest resource of public gene expression data. While GEO enables data browsing, query and retrieval, additional tools can help realize its potential for aggregating and comparing data across multiple studies and platforms. This paper describes DSGeo-a collection of valuable tools that were developed for annotating, aggregating, integrating, and analyzing data deposited in GEO. The core set of tools include a Relational Database, a Data Loader, a Data Browser, and an Expression Combiner and Analyzer. The application enables querying for specific sample characteristics and identifying studies containing samples that match the query. The Expression Combiner application enables normalization and aggregation of data from these samples and returns these data to the user after filtering, according to the users preferences. The Expression Analyzer allows simple statistical comparisons between groups of data. This seamless integration makes annotated cross-platform data directly available for analysis.

Evaluation of a large-scale biomedical data annotation initiative.

BackgroundThis study describes a large-scale manual re-annotation of data samples in the Gene Expression Omnibus (GEO), using variables and values derived from the National Cancer Institute thesaurus. A framework is described for creating an annotation scheme for various diseases that is flexible, comprehensive, and scalable. The annotation structure is evaluated by measuring coverage and agreement between annotators.ResultsThere were 12,500 samples annotated with approximately 30 variables, in each of six disease categories – breast cancer, colon cancer, inflammatory bowel disease (IBD), rheumatoid arthritis (RA), systemic lupus erythematosus (SLE), and Type 1 diabetes mellitus (DM). The annotators provided excellent variable coverage, with known values for over 98% of three critical variables: disease state, tissue, and sample type. There was 89% strict inter-annotator agreement and 92% agreement when using semantic and partial similarity measures.ConclusionWe show that it is possible to perform manual re-annotation of a large repository in a reliable manner.

Impact of an Information Technology–Enabled Initiative on the Quality of Prostate Multiparametric MRI Reports

RATIONALE AND OBJECTIVES Assess the impact of implementing a structured report template and a computer-aided diagnosis (CAD) tool on the quality of prostate multiparametric magnetic resonance imaging (mp-MRI) reports. MATERIALS AND METHODS Institutional Review Board approval was obtained for this Health Insurance Portability and Accountability Act-compliant study performed at an academic medical center. The study cohort included all prostate mp-MRI reports (n = 385) finalized 6 months before and after implementation of a structured report template and a CAD tool (collectively the information technology [IT] tools) integrated into the picture archiving and communication system workstation. Primary outcome measure was quality of prostate mp-MRI reports. An expert panel of our institutions subspecialty-trained abdominal radiologists defined prostate mp-MRI report quality as optimal, satisfactory, or unsatisfactory based on documentation of nine variables. Reports were reviewed to extract the predefined quality variables and determine whether the IT tools were used to create each report. Chi-square and Students t tests were used to compare report quality before and after implementation of IT tools. RESULTS The overall proportion of optimal or satisfactory reports increased from 29.8% (47/158) to 53.3% (121/227) (P < .001) after implementing the IT tools. Although the proportion of optimal or satisfactory reports increased among reports generated using at least one of the IT tools (47/158 = [29.8%] vs. 105/161 = [65.2%]; P < .001), there was no change in quality among reports generated without use of the IT tools (47/158 = [29.8%] vs. 16/66 = [24.2%]; P = .404). CONCLUSIONS The use of a structured template and CAD tool improved the quality of prostate mp-MRI reports compared to free-text report format and subjective measurement of contrast enhancement kinetic curve.

Impact of an electronic alert notification system embedded in radiologists’ workflow on closed-loop communication of critical results: a time series analysis

Introduction Optimal critical test result communication is a Joint Commission national patient safety goal and requires documentation of closed-loop communication among care providers in the medical record. Electronic alert notification systems can facilitate an auditable process for creating alerts for transmission and acknowledgement of critical test results. We evaluated the impact of a patient safety initiative with an alert notification system on reducing critical results lacking documented communication, and assessed potential overuse of the alerting system for communicating results. Methods We implemented an alert notification system—Alert Notification of Critical Results (ANCR)—in January 2010. We reviewed radiology reports finalised in 2009–2014 which lacked documented communication between the radiologist and another care provider, and assessed the impact of ANCR on the proportion of such reports with critical findings, using trend analysis over 10 semiannual time periods. To evaluate potential overuse of ANCR, we assessed the proportion of reports with non-critical results among provider-communicated reports. Results The proportion of reports with critical results among reports without documented communication decreased significantly over 4 years (2009–2014) from 0.19 to 0.05 (p<0.0001, Cochran–Armitage trend test). The proportion of provider-communicated reports with non-critical results remained unchanged over time before and after ANCR implementation (0.20 to 0.15, p=0.45, Cochran–Armitage trend test). Conclusions A patient safety initiative with an alert notification system reduced the proportion of critical results among reports lacking documented communication between care providers. We observed no change in documented communication of non-critical results, suggesting the system did not promote overuse. Future studies are needed to evaluate whether such systems prevent subsequent patient harm.

Gastrointestinal Stromal Tumor: Optimizing the Use of Cross-sectional Chest Imaging during Follow-up

PURPOSE To identify the frequency of and variables associated with thoracic metastasis in patients with gastrointestinal stromal tumor (GIST) to help optimize the use of cross-sectional chest imaging. MATERIALS AND METHODS This retrospective institutional review board-approved study included 631 patients (343 men; mean age, 55 years; range, 19-94 years) with pathologically confirmed GIST who were identified with a natural language processing algorithm in a review of radiologic reports from January 2004 through October 2012, followed by manual confirmation. The requirement for informed consent was waived. Available imaging, pathologic, and clinical records were reviewed to confirm the presence of abdominal and thoracic metastases. The association of age; sex; size, location, mitotic count, and risk stratification of the primary tumor; initial treatment; presence of abdominal metastases; and bulky abdominal metastases (more than 10 lesions larger than 1 cm, or more than five lesions with at least one larger than 5 cm) with development of thoracic metastases, the primary outcome measure, was studied by using logistic regression. RESULTS During median follow-up of 61.4 months (interquartile range, 37.8-93 months), 401 of 631 (63.5%) patients developed metastatic disease (median interval, 6.9 months; interquartile range, 0-25.6 months), all with peritoneal (n = 324) and/or hepatic metastases (n = 303). Bulky abdominal metastases were found in 218 (34.5%) patients. Although 579 (91.8%) patients underwent chest imaging, only 64 of 631 (10.1%) developed thoracic metastases (median, 51.4 months; interquartile range, 36-78.7 months); all had bulky abdominal metastases except one patient who presented with symptomatic scapular metastasis. Only bulky abdominal metastasis was significantly associated with the development of thoracic metastasis (P < .0001; odds ratio, 42.6; range, 8.6-211.5). CONCLUSION Thoracic metastases are relatively uncommon in patients with GIST and are significantly associated only with presence of bulky abdominal metastases.

Retrieval of Radiology Reports Citing Critical Findings with Disease-Specific Customization

Background: Communication of critical results from diagnostic procedures between caregivers is a Joint Commission national patient safety goal. Evaluating critical result communication often requires manual analysis of voluminous data, especially when reviewing unstructured textual results of radiologic findings. Information retrieval (IR) tools can facilitate this process by enabling automated retrieval of radiology reports that cite critical imaging findings. However, IR tools that have been developed for one disease or imaging modality often need substantial reconfiguration before they can be utilized for another disease entity. Purpose: This paper: 1) describes the process of customizing two Natural Language Processing (NLP) and Information Retrieval/Extraction applications – an open-source toolkit, A Nearly New Information Extraction system (ANNIE); and an application developed in-house, Information for Searching Content with an Ontology-Utilizing Toolkit (iSCOUT) – to illustrate the varying levels of customization required for different disease entities and; 2) evaluates each application’s performance in identifying and retrieving radiology reports citing critical imaging findings for three distinct diseases, pulmonary nodule, pneumothorax, and pulmonary embolus. Results: Both applications can be utilized for retrieval. iSCOUT and ANNIE had precision values between 0.90-0.98 and recall values between 0.79 and 0.94. ANNIE had consistently higher precision but required more customization. Conclusion: Understanding the customizations involved in utilizing NLP applications for various diseases will enable users to select the most suitable tool for specific tasks.