Dezon Finch

Finding falls in ambulatory care clinical documents using statistical text mining.

OBJECTIVE To determine how well statistical text mining (STM) models can identify falls within clinical text associated with an ambulatory encounter. MATERIALS AND METHODS 2241 patients were selected with a fall-related ICD-9-CM E-code or matched injury diagnosis code while being treated as an outpatient at one of four sites within the Veterans Health Administration. All clinical documents within a 48-h window of the recorded E-code or injury diagnosis code for each patient were obtained (n=26 010; 611 distinct document titles) and annotated for falls. Logistic regression, support vector machine, and cost-sensitive support vector machine (SVM-cost) models were trained on a stratified sample of 70% of documents from one location (dataset Atrain) and then applied to the remaining unseen documents (datasets Atest-D). RESULTS All three STM models obtained area under the receiver operating characteristic curve (AUC) scores above 0.950 on the four test datasets (Atest-D). The SVM-cost model obtained the highest AUC scores, ranging from 0.953 to 0.978. The SVM-cost model also achieved F-measure values ranging from 0.745 to 0.853, sensitivity from 0.890 to 0.931, and specificity from 0.877 to 0.944. DISCUSSION The STM models performed well across a large heterogeneous collection of document titles. In addition, the models also generalized across other sites, including a traditionally bilingual site that had distinctly different grammatical patterns. CONCLUSIONS The results of this study suggest STM-based models have the potential to improve surveillance of falls. Furthermore, the encouraging evidence shown here that STM is a robust technique for mining clinical documents bodes well for other surveillance-related topics.

A Case Study of Data Quality in Text Mining Clinical Progress Notes

Text analytic methods are often aimed at extracting useful information from the vast array of unstructured, free format text documents that are created by almost all organizational processes. The success of any text mining application rests on the quality of the underlying data being analyzed, including both predictive features and outcome labels. In this case study, some focused experiments regarding data quality are used to assess the robustness of Statistical Text Mining (STM) algorithms when applied to clinical progress notes. In particular, the experiments consider the impacts of task complexity (by removing signals), training set size, and target outcome quality. While this research is conducted using a dataset drawn from the medical domain, the data quality issues explored are of more general interest.

Using ensemble models to classify the sentiment expressed in suicide notes.

In 2007, suicide was the tenth leading cause of death in the U.S. Given the significance of this problem, suicide was the focus of the 2011 Informatics for Integrating Biology and the Bedside (i2b2) Natural Language Processing (NLP) shared task competition (track two). Specifically, the challenge concentrated on sentiment analysis, predicting the presence or absence of 15 emotions (labels) simultaneously in a collection of suicide notes spanning over 70 years. Our team explored multiple approaches combining regular expression-based rules, statistical text mining (STM), and an approach that applies weights to text while accounting for multiple labels. Our best submission used an ensemble of both rules and STM models to achieve a micro-averaged F1 score of 0.5023, slightly above the mean from the 26 teams that competed (0.4875).

Improving identification of fall-related injuries in ambulatory care using statistical text mining

OBJECTIVES We determined whether statistical text mining (STM) can identify fall-related injuries in electronic health record (EHR) documents and the impact on STM models of training on documents from a single or multiple facilities. METHODS We obtained fiscal year 2007 records for Veterans Health Administration (VHA) ambulatory care clinics in the southeastern United States and Puerto Rico, resulting in a total of 26 010 documents for 1652 veterans treated for fall-related injury and 1341 matched controls. We used the results of an STM model to predict fall-related injuries at the visit and patient levels and compared them with a reference standard based on chart review. RESULTS STM models based on training data from a single facility resulted in accuracy of 87.5% and 87.1%, F-measure of 87.0% and 90.9%, sensitivity of 92.1% and 94.1%, and specificity of 83.6% and 77.8% at the visit and patient levels, respectively. Results from training data from multiple facilities were almost identical. CONCLUSIONS STM has the potential to improve identification of fall-related injuries in the VHA, providing a model for wider application in the evolving national EHR system.

Leveraging Electronic Health Care Record Information to Measure Pressure Ulcer Risk in Veterans With Spinal Cord Injury: A Longitudinal Study Protocol

Background Pressure ulcers (PrUs) are a frequent, serious, and costly complication for veterans with spinal cord injury (SCI). The health care team should periodically identify PrU risk, although there is no tool in the literature that has been found to be reliable, valid, and sensitive enough to assess risk in this vulnerable population. Objective The immediate goal is to develop a risk assessment model that validly estimates the probability of developing a PrU. The long-term goal is to assist veterans with SCI and their providers in preventing PrUs through an automated system of risk assessment integrated into the veteran’s electronic health record (EHR). Methods This 5-year longitudinal, retrospective, cohort study targets 12,344 veterans with SCI who were cared for in the Veterans Health Administration (VHA) in fiscal year (FY) 2009 and had no record of a PrU in the prior 12 months. Potential risk factors identified in the literature were reviewed by an expert panel that prioritized factors and determined if these were found in structured data or unstructured form in narrative clinical notes for FY 2009-2013. These data are from the VHA enterprise Corporate Data Warehouse that is derived from the EHR structured (ie, coded in database/table) or narrative (ie, text in clinical notes) data for FY 2009-2013. Results This study is ongoing and final results are expected in 2017. Thus far, the expert panel reviewed the initial list of risk factors extracted from the literature; the panel recommended additions and omissions and provided insights about the format in which the documentation of the risk factors might exist in the EHR. This list was then iteratively refined through review and discussed with individual experts in the field. The cohort for the study was then identified, and all structured, unstructured, and semistructured data were extracted. Annotation schemas were developed, samples of documents were extracted, and annotations are ongoing. Operational definitions of structured data elements have been created and steps to create an analytic dataset are underway. Conclusions To our knowledge, this is the largest cohort employed to identify PrU risk factors in the United States. It also represents the first time natural language processing and statistical text mining will be used to expand the number of variables available for analysis. A major strength of this quantitative study is that all VHA SCI centers were included in the analysis, reducing potential for selection bias and providing increased power for complex statistical analyses. This longitudinal study will eventually result in a risk prediction tool to assess PrU risk that is reliable and valid, and that is sensitive to this vulnerable population.

MedCat: A Framework for High Level Conceptualization of Medical Notes

In this paper we introduce a new framework called MedCat to delineate and demonstrate an approach for projecting representations of concept-derived content in clinical notes into a new categorization space to reduce dimensionality and noise in the data. Constructing MedCat framework required several steps including manual annotation, knowledge base expansion using MetaMap, concept category construction, automated annotation using NLP to generate a bag of concepts, and finally concept conversion to higher level abstracted categories. The framework was applied to Post Traumatic Stress Disorder (PTSD) clinical notes for evaluation. A random sample of PTSD clinical note content was automatically recategorized into six PTSD treatment categories using MedCat. Using existing annotations from PTSD notes that were categorized by content experts into treatment categories as the reference standard, the sensitivity of the framework in detecting the treatment categories was greater than 90%. The results suggest that representations of concept-derived content when categorized by relevance features can be used to reliably understand and summarize clinical notes.

Understanding trader heterogeneity in information markets

Purpose – The purpose of this paper is to contribute to the growing body of research in prediction markets by using trading data as a means of characterizing trader behavior in these markets. Traders are placed in homogenous groups based on common Trading habits using clustering algorithms. Several behavioral themes are used to guide the analyses.Design/methodology/approach – Several market experiments were run to collect trading data, which was then exported into a data warehouse. A secondary data analysis is performed on variables derived from the original trade data. In particular, k‐means clustering is used to form groups of traders that share common characteristics.Findings – Participants can be classified into homogenous groups based on their trading behavior. Groups tend to differ based on the overall level of participation, how much of their trading activity is spent buying or selling, and how early they enter the market.Research limitations/implications – More research should be done using a vari...