Lee B. Peters

From indexing the biomedical literature to coding clinical text: experience with MTI and machine learning approaches

This paper describes the application of an ensemble of indexing and classification systems, which have been shown to be successful in information retrieval and classification of medical literature, to a new task of assigning ICD-9-CM codes to the clinical history and impression sections of radiology reports. The basic methods used are: a modification of the NLM Medical Text Indexer system, SVM, k-NN and a simple pattern-matching method. The basic methods are combined using a variant of stacking. Evaluated in the context of a Medical NLP Challenge, fusion produced an F-score of 0.85 on the Challenge test set, which is considerably above the mean Challenge F-score of 0.77 for 44 participating groups.

Extracting Rx information from clinical narrative

OBJECTIVE The authors used the i2b2 Medication Extraction Challenge to evaluate their entity extraction methods, contribute to the generation of a publicly available collection of annotated clinical notes, and start developing methods for ontology-based reasoning using structured information generated from the unstructured clinical narrative. DESIGN Extraction of salient features of medication orders from the text of de-identified hospital discharge summaries was addressed with a knowledge-based approach using simple rules and lookup lists. The entity recognition tool, MetaMap, was combined with dose, frequency, and duration modules specifically developed for the Challenge as well as a prototype module for reason identification. MEASUREMENTS Evaluation metrics and corresponding results were provided by the Challenge organizers. RESULTS The results indicate that robust rule-based tools achieve satisfactory results in extraction of simple elements of medication orders, but more sophisticated methods are needed for identification of reasons for the orders and durations. LIMITATIONS Owing to the time constraints and nature of the Challenge, some obvious follow-on analysis has not been completed yet. CONCLUSIONS The authors plan to integrate the new modules with MetaMap to enhance its accuracy. This integration effort will provide guidance in retargeting existing tools for better processing of clinical text.

Evaluating drug-drug interaction information in NDF-RT and DrugBank

BackgroundThere is limited consensus among drug information sources on what constitutes drug-drug interactions (DDIs). We investigate DDI information in two publicly available sources, NDF-RT and DrugBank.MethodsWe acquire drug-drug interactions from NDF-RT and DrugBank, and normalize the drugs to RxNorm. We compare interactions between NDF-RT and DrugBank and evaluate both sources against a reference list of 360 critical interactions. We compare the interactions detected with NDF-RT and DrugBank on a large prescription dataset. Finally, we contrast NDF-RT and DrugBank against a commercial source.ResultsDrugBank drug-drug interaction information has limited overlap with NDF-RT (24-30%). The coverage of the reference set by both sources is about 60%. Applied to a prescription dataset of 35.5M pairs of co-prescribed systemic clinical drugs, NDF-RT would have identified 808,285 interactions, while DrugBank would have identified 1,170,693. Of these, 382,833 are common. The commercial source Multum provides a more systematic coverage (91%) of the reference list.ConclusionsThis investigation confirms the limited overlap of DDI information between NDF-RT and DrugBank. Additional research is required to determine which source is better, if any. Usage of any of these sources in clinical decision systems should disclose these limitations.

A graph-based approach to auditing RxNorm

OBJECTIVES RxNorm is a standardized nomenclature for clinical drug entities developed by the National Library of Medicine. In this paper, we audit relations in RxNorm for consistency and completeness through the systematic analysis of the graph of its concepts and relationships. METHODS The representation of multi-ingredient drugs is normalized in order to make it compatible with that of single-ingredient drugs. All meaningful paths between two nodes in the type graph are computed and instantiated. Alternate paths are automatically compared and manually inspected in case of inconsistency. RESULTS The 115 meaningful paths identified in the type graph can be grouped into 28 groups with respect to start and end nodes. Of the 19 groups of alternate paths (i.e., with two or more paths) between the start and end nodes, 9 (47%) exhibit inconsistencies. Overall, 28 (24%) of the 115 paths are inconsistent with other alternate paths. A total of 348 inconsistencies were identified in the April 2008 version of RxNorm and reported to the RxNorm team, of which 215 (62%) had been corrected in the January 2009 version of RxNorm. CONCLUSION The inconsistencies identified involve missing nodes (93), missing links (17), extraneous links (237) and one case of mix-up between two ingredients. Our auditing method proved effective in identifying a limited number of errors that had defeated the quality assurance mechanisms currently in place in the RxNorm production system. Some recommendations for the development of RxNorm are provided.

Comparing and evaluating terminology services application programming interfaces: RxNav, UMLSKS and LexBIG

To facilitate the integration of terminologies into applications, various terminology services application programming interfaces (API) have been developed in the recent past. In this study, three publicly available terminology services API, RxNav, UMLSKS and LexBIG, are compared and functionally evaluated with respect to the retrieval of information from one biomedical terminology, RxNorm, to which all three services provide access. A list of queries is established covering a wide spectrum of terminology services functionalities such as finding RxNorm concepts by their name, or navigating different types of relationships. Test data were generated from the RxNorm dataset to evaluate the implementation of the functionalities in the three API. The results revealed issues with various aspects of the API implementation (eg, handling of obsolete terms by LexBIG) and documentation (eg, navigational paths used in RxNav) that were subsequently addressed by the development teams of the three API investigated. Knowledge about such discrepancies helps inform the choice of an API for a given use case.