Peter J. Haug

Automatic Detection of Acute Bacterial Pneumonia from Chest X-ray Reports

OBJECTIVE To evaluate the performance of a natural language processing system in extracting pneumonia-related concepts from chest x-ray reports. METHODS DESIGN Four physicians, three lay persons, a natural language processing system, and two keyword searches (designated AAKS and KS) detected the presence or absence of three pneumonia-related concepts and inferred the presence or absence of acute bacterial pneumonia from 292 chest x-ray reports. Gold standard: Majority vote of three independent physicians. Reliability of the gold standard was measured. OUTCOME MEASURES Recall, precision, specificity, and agreement (using Finns R: statistic) with respect to the gold standard. Differences between the physicians and the other subjects were tested using the McNemar test for each pneumonia concept and for the disease inference of acute bacterial pneumonia. RESULTS Reliability of the reference standard ranged from 0.86 to 0.96. Recall, precision, specificity, and agreement (Finn R:) for the inference on acute bacterial pneumonia were, respectively, 0.94, 0.87, 0.91, and 0.84 for physicians; 0.95, 0.78, 0.85, and 0.75 for natural language processing system; 0.46, 0.89, 0.95, and 0.54 for lay persons; 0.79, 0.63, 0.71, and 0.49 for AAKS; and 0.87, 0.70, 0.77, and 0.62 for KS. The McNemar pairwise comparisons showed differences between one physician and the natural language processing system for the infiltrate concept and between another physician and the natural language processing system for the inference on acute bacterial pneumonia. The comparisons also showed that most physicians were significantly different from the other subjects in all pneumonia concepts and the disease inference. CONCLUSION In extracting pneumonia related concepts from chest x-ray reports, the performance of the natural language processing system was similar to that of physicians and better than that of lay persons and keyword searches. The encoded pneumonia information has the potential to support several pneumonia-related applications used in our institution. The applications include a decision support system called the antibiotic assistant, a computerized clinical protocol for pneumonia, and a quality assurance application in the radiology department.

Hospital Workload and Adverse Events

Context:Hospitals are under pressure to increase revenue and lower costs, and at the same time, they face dramatic variation in clinical demand. Objective:We sought to determine the relationship between peak hospital workload and rates of adverse events (AEs). Methods:A random sample of 24,676 adult patients discharged from the medical/surgical services at 4 US hospitals (2 urban and 2 suburban teaching hospitals) from October 2000 to September 2001 were screened using administrative data, leaving 6841 cases to be reviewed for the presence of AEs. Daily workload for each hospital was characterized by volume, throughput (admissions and discharges), intensity (aggregate DRG weight), and staffing (patient-to-nurse ratios). For volume, we calculated an “enhanced” occupancy rate that accounted for same-day bed occupancy by more than 1 patient. We used Poisson regressions to predict the likelihood of an AE, with control for workload and individual patient complexity, and the effects of clustering. Results:One urban teaching hospital had enhanced occupancy rates more than 100% for much of the year. At that hospital, admissions and patients per nurse were significantly related to the likelihood of an AE (P < 0.05); occupancy rate, discharges, and DRG-weighted census were significant at P < 0.10. For example, a 0.1% increase in the patient-to-nurse ratio led to a 28% increase in the AE rate. Results at the other 3 hospitals varied and were mainly non significant. Conclusions:Hospitals that operate at or over capacity may experience heightened rates of patient safety events and might consider re-engineering the structures of care to respond better during periods of high stress.

Accuracy of administrative data for identifying patients with pneumonia.

The goal of this study was to determine the accuracy and the impact of 5 different claims-based pneumonia definitions. Three International Classification of Diseases, Version 9, (ICD-9), and 2 diagnosis-related group (DRG)-based case identification algorithms were compared against an independent, clinical pneumonia reference standard. Among 10748 patients, 272 (2.5%) had pneumonia verified by the reference standard. The sensitivity of claims-based algorithms ranged from 47.8% to 66.2%. The positive predictive values ranged from 72.6% to 80.8%. Patient-related variables were not significantly different from the reference standard among the 3 ICD-9-based algorithms. DRG-based algorithms had significantly lower hospital admission rates (57% and 65% vs 73.2%), lower 30-day mortality (5.0% and 5.8% vs 10.7%), shorter length of stay (3.9 and 4.1 days vs 5.6 days), and lower costs (US

Classifying free-text triage chief complaints into syndromic categories with natural languages processing

4543 and US

Population-based family-history-specific risks for colorectal cancer: a constellation approach

5159 vs US

Forecasting daily patient volumes in the emergency department.

8585). Claims-based identification algorithms for defining pneumonia in administrative databases are imprecise. ICD-9-based algorithms did not influence patient variables in our population. Identifying pneumonia patients with DRG codes is significantly less precise.

MPLUS: a probabilistic medical language understanding system

OBJECTIVE Develop and evaluate a natural language processing application for classifying chief complaints into syndromic categories for syndromic surveillance. INTRODUCTION Much of the input data for artificial intelligence applications in the medical field are free-text patient medical records, including dictated medical reports and triage chief complaints. To be useful for automated systems, the free-text must be translated into encoded form. METHODS We implemented a biosurveillance detection system from Pennsylvania to monitor the 2002 Winter Olympic Games. Because input data was in free-text format, we used a natural language processing text classifier to automatically classify free-text triage chief complaints into syndromic categories used by the biosurveillance system. The classifier was trained on 4700 chief complaints from Pennsylvania. We evaluated the ability of the classifier to classify free-text chief complaints into syndromic categories with a test set of 800 chief complaints from Utah. RESULTS The classifier produced the following areas under the ROC curve: Constitutional = 0.95; Gastrointestinal = 0.97; Hemorrhagic = 0.99; Neurological = 0.96; Rash = 1.0; Respiratory = 0.99; Other = 0.96. Using information stored in the systems semantic model, we extracted from the Respiratory classifications lower respiratory complaints and lower respiratory complaints with fever with a precision of 0.97 and 0.96, respectively. CONCLUSION Results suggest that a trainable natural language processing text classifier can accurately extract data from free-text chief complaints for biosurveillance.

Automation of a problem list using natural language processing

BACKGROUND & AIMS Colorectal cancer (CRC) risk estimates based on family history typically include only close relatives. We report familial relative risk (FRR) in probands with various combinations, or constellations, of affected relatives, extending to third-degree. METHODS A population-based resource that includes a computerized genealogy linked to statewide cancer records was used to identify genetic relationships among CRC cases and their first-, second-, and third-degree relatives (FDRs, SDRs, and TDRs). FRRs were estimated by comparing the observed number of affected persons with a particular family history constellation to the expected number, based on cohort-specific CRC rates. RESULTS A total of 2,327,327 persons included in > or =3 generation family histories were analyzed; 10,556 had a diagnosis of CRC. The FRR for CRC in persons with > or =1 affected FDR = 2.05 (95% CI, 1.96-2.14), consistent with published estimates. In the absence of a positive first-degree family history, considering both affected SDRs and TDRs, only 1 constellation had an FRR estimate that was significantly >1.0 (0 affected FDRs, 1 affected SDR, 2 affected TDRs; FRR = 1.33; 95% CI, 1.13-1.55). The FRR for persons with 1 affected FDR, 1 affected SDR, and 0 affected TDRs was 1.88 (95% CI, 1.59-2.20), increasing to FRR = 3.28 (95% CI, 2.44-4.31) for probands with 1 affected FDR, 1 affected SDR, and > or =3 affected TDRs. CONCLUSIONS Increased numbers of affected FDRs influences risk much more than affected SDRs or TDRs. However, when combined with a positive first-degree family history, a positive second- and third-degree family history can significantly increase risk.

A Comparison of Classification Algorithms to Automatically Identify Chest X-Ray Reports That Support Pneumonia

BACKGROUND Shifts in the supply of and demand for emergency department (ED) resources make the efficient allocation of ED resources increasingly important. Forecasting is a vital activity that guides decision-making in many areas of economic, industrial, and scientific planning, but has gained little traction in the health care industry. There are few studies that explore the use of forecasting methods to predict patient volumes in the ED. OBJECTIVES The goals of this study are to explore and evaluate the use of several statistical forecasting methods to predict daily ED patient volumes at three diverse hospital EDs and to compare the accuracy of these methods to the accuracy of a previously proposed forecasting method. METHODS Daily patient arrivals at three hospital EDs were collected for the period January 1, 2005, through March 31, 2007. The authors evaluated the use of seasonal autoregressive integrated moving average, time series regression, exponential smoothing, and artificial neural network models to forecast daily patient volumes at each facility. Forecasts were made for horizons ranging from 1 to 30 days in advance. The forecast accuracy achieved by the various forecasting methods was compared to the forecast accuracy achieved when using a benchmark forecasting method already available in the emergency medicine literature. RESULTS All time series methods considered in this analysis provided improved in-sample model goodness of fit. However, post-sample analysis revealed that time series regression models that augment linear regression models by accounting for serial autocorrelation offered only small improvements in terms of post-sample forecast accuracy, relative to multiple linear regression models, while seasonal autoregressive integrated moving average, exponential smoothing, and artificial neural network forecasting models did not provide consistently accurate forecasts of daily ED volumes. CONCLUSIONS This study confirms the widely held belief that daily demand for ED services is characterized by seasonal and weekly patterns. The authors compared several time series forecasting methods to a benchmark multiple linear regression model. The results suggest that the existing methodology proposed in the literature, multiple linear regression based on calendar variables, is a reasonable approach to forecasting daily patient volumes in the ED. However, the authors conclude that regression-based models that incorporate calendar variables, account for site-specific special-day effects, and allow for residual autocorrelation provide a more appropriate, informative, and consistently accurate approach to forecasting daily ED patient volumes.

Assessing the Quality of Clinical Data in a Computer-based Record for Calculating the Pneumonia Severity Index

This paper describes the basic philosophy and implementation of MPLUS (M+), a robust medical text analysis tool that uses a semantic model based on Bayesian Networks (BNs). BNs provide a concise and useful formalism for representing semantic patterns in medical text, and for recognizing and reasoning over those patterns. BNs are noise-tolerant, and facilitate the training of M+.