Michal Kouril

Preparing an annotated gold standard corpus to share with extramural investigators for de-identification research

OBJECTIVE The current study aims to fill the gap in available healthcare de-identification resources by creating a new sharable dataset with realistic Protected Health Information (PHI) without reducing the value of the data for de-identification research. By releasing the annotated gold standard corpus with Data Use Agreement we would like to encourage other Computational Linguists to experiment with our data and develop new machine learning models for de-identification. This paper describes: (1) the modifications required by the Institutional Review Board before sharing the de-identification gold standard corpus; (2) our efforts to keep the PHI as realistic as possible; (3) and the tests to show the effectiveness of these efforts in preserving the value of the modified data set for machine learning model development. MATERIALS AND METHODS In a previous study we built an original de-identification gold standard corpus annotated with true Protected Health Information (PHI) from 3503 randomly selected clinical notes for the 22 most frequent clinical note types of our institution. In the current study we modified the original gold standard corpus to make it suitable for external sharing by replacing HIPAA-specified PHI with newly generated realistic PHI. Finally, we evaluated the research value of this new dataset by comparing the performance of an existing published in-house de-identification system, when trained on the new de-identification gold standard corpus, with the performance of the same system, when trained on the original corpus. We assessed the potential benefits of using the new de-identification gold standard corpus to identify PHI in the i2b2 and PhysioNet datasets that were released by other groups for de-identification research. We also measured the effectiveness of the i2b2 and PhysioNet de-identification gold standard corpora in identifying PHI in our original clinical notes. RESULTS Performance of the de-identification system using the new gold standard corpus as a training set was very close to training on the original corpus (92.56 vs. 93.48 overall F-measures). Best i2b2/PhysioNet/CCHMC cross-training performances were obtained when training on the new shared CCHMC gold standard corpus, although performances were still lower than corpus-specific trainings. DISCUSSION AND CONCLUSION We successfully modified a de-identification dataset for external sharing while preserving the de-identification research value of the modified gold standard corpus with limited drop in machine learning de-identification performance.

Automated identification of antibiotic overdoses and adverse drug events via analysis of prescribing alerts and medication administration records

Objectives: Electronic trigger detection tools hold promise to reduce Adverse drug event (ADEs) through efficiencies of scale and real-time reporting. We hypothesized that such a tool could automatically detect medication dosing errors as well as manage and evaluate dosing rule modifications. Materials and Methods: We created an order and alert analysis system that identified antibiotic medication orders and evaluated user response to dosing alerts. Orders associated with overridden alerts were examined for evidence of administration and the delivered dose was compared to pharmacy-derived dosing rules to confirm true overdoses. True overdose cases were reviewed for association with known ADEs. Results: Of 55 546 orders reviewed, 539 were true overdose orders, which lead to 1965 known overdose administrations. Documentation of loose stools and diarrhea was significantly increased following drug administration in the overdose group. Dosing rule thresholds were altered to reflect clinically accurate dosing. These rule changes decreased overall alert burden and improved the salience of alerts. Discussion: Electronic algorithm-based detection systems can identify antibiotic overdoses that are clinically relevant and are associated with known ADEs. The system also serves as a platform for evaluating the effects of modifying electronic dosing rules. These modifications lead to decreased alert burden and improvements in response to decision support alerts. Conclusion: The success of this test case suggests that gains are possible in reducing medication errors and improving patient safety with automated algorithm-based detection systems. Follow-up studies will determine if the positive effects of the system persist and if these changes lead to improved safety outcomes.

GRcalculator: an online tool for calculating and mining dose–response data

BackgroundQuantifying the response of cell lines to drugs or other perturbagens is the cornerstone of pre-clinical drug development and pharmacogenomics as well as a means to study factors that contribute to sensitivity and resistance. In dividing cells, traditional metrics derived from dose–response curves such as IC50, AUC, and Emax, are confounded by the number of cell divisions taking place during the assay, which varies widely for biological and experimental reasons. Hafner et al. (Nat Meth 13:521–627, 2016) recently proposed an alternative way to quantify drug response, normalized growth rate (GR) inhibition, that is robust to such confounders. Adoption of the GR method is expected to improve the reproducibility of dose–response assays and the reliability of pharmacogenomic associations (Hafner et al. 500–502, 2017).ResultsWe describe here an interactive website (www.grcalculator.org) for calculation, analysis, and visualization of dose–response data using the GR approach and for comparison of GR and traditional metrics. Data can be user-supplied or derived from published datasets. The web tools are implemented in the form of three integrated Shiny applications (grcalculator, grbrowser, and grtutorial) deployed through a Shiny server. Intuitive graphical user interfaces (GUIs) allow for interactive analysis and visualization of data. The Shiny applications make use of two R packages (shinyLi and GRmetrics) specifically developed for this purpose. The GRmetrics R package is also available via Bioconductor and can be used for offline data analysis and visualization. Source code for the Shiny applications and associated packages (shinyLi and GRmetrics) can be accessed at www.github.com/uc-bd2k/grcalculator and www.github.com/datarail/gr_metrics.ConclusionsGRcalculator is a powerful, user-friendly, and free tool to facilitate analysis of dose–response data. It generates publication-ready figures and provides a unified platform for investigators to analyze dose–response data across diverse cell types and perturbagens (including drugs, biological ligands, RNAi, etc.). GRcalculator also provides access to data collected by the NIH LINCS Program (http://www.lincsproject.org/) and other public domain datasets. The GRmetrics Bioconductor package provides computationally trained users with a platform for offline analysis of dose–response data and facilitates inclusion of GR metrics calculations within existing R analysis pipelines. These tools are therefore well suited to users in academia as well as industry.

GREIN: An interactive web platform for re-analyzing GEO RNA-seq data

The vast amount of RNA-seq data deposited in Gene Expression Omnibus (GEO) and Sequence Read Archive (SRA) is still a grossly underutilized resource for biomedical research. To remove technical roadblocks for re-using these data, we have developed a web-application GREIN (GEO RNA-seq Experiments Interactive Navigator) which provides simple user-friendly interfaces for manipulating and analyses of GEO RNA-seq data. GREIN is powered by the back-end computational pipeline for uniform processing of RNA-seq data and the large number (>5,500) of already processed datasets. The front-end user interfaces provide a wealth of user-analytics options including sub-setting and downloading processed data, interactive visualization, statistical power analyses, construction of differential gene expression signatures and their comprehensive functional characterization, connectivity analysis with LINCS L1000 data, etc. The combination of the massive amount of back-end data and front-end analytics options driven by user-friendly interfaces makes GREIN a unique open-source resource for re-using GEO RNA-seq data. GREIN is freely accessible at: https://shiny.ilincs.org/grein, the source code is available at: https://github.com/uc-bd2k/grein, and the Docker container is available at: https://hub.docker.com/r/ucbd2k/grein. Contact: Mario.Medvedovic@uc.edu