Curtis P. Langlotz

Toward Best Practices in Radiology Reporting

The goals and current efforts of the Radiological Society of North America Radiology Reporting Committee are described. The committees charter provides an opportunity to improve the organization, content, readability, and usefulness of the radiology report and to advance the efficiency and effectiveness of the reporting process.

Structured Reporting: Patient Care Enhancement or Productivity Nightmare?

David L. Weiss, MD Curtis P. Langlotz, MD, PhD The man we now know as St Lawrence lived his life as Lawrence of Rome and served as a deacon in the 3rd century Roman Empire. In 258 AD he was executed during the persecution of Christians by the Emperor Valerian. Historians detail his martyrdom as being grilled alive. Many scholars consider this account well documented while others have cast some doubt on its accuracy. They point out that as a Roman citizen under a sentence of death Lawrence would likely have been afforded the “privilege” of being beheaded. Some believe the discrepancy can be traced to the error of an early scribe. The Latin “passus est” (he died) may have been incorrectly transcribed as “assus est” (he was roasted) (1). That a seemingly trivial transcription error could have such profound historical and ecclesiastic consequences should not be surprising to a radiologist. Our specialty is rife with anecdotes of the trivial typographical error that is perhaps not quite as gruesome, but potentially life-threatening if left uncorrected. Although the first radiology report was generated over a century ago, the process of radiology reporting has changed very little until the past decade. The use of speech-to-text software has been available for some time (2), but speech recognition use did not become widespread until hardware and software matured in the late 1990s. The use of speech recognition in radiology reporting has resulted in improved turnaround time and reported cost savings (3,4), metrics that delight hospital and department administrators. Radiologists, however, have found that the software tends to decrease productivity and has the potential to cause distraction during image interpretation (5,6). In the past several decades, structured reporting systems have become available, first as research systems (7,8), then as production systems, particularly in mammography and ultrasonography (9). As with speech recognition software, current structured reporting systems are more sophisticated and acceptable as a result of advances in hardware, software, and workflow design (10). Will these new systems result in improved workflow or are these reports merely administrative agitprop designed to convince radiologists to adopt one more reporting system that may ultimately have negative consequences for the user? In the sections that follow, we will discuss these issues in a classic dialogue format, alternatively presenting the pros (C.P.L.) and cons (D.L.W) of structured reporting from two radiologists’ points of view.

The Radiology Report of the Future: A Summary of the 2007 Intersociety Conference

A radiology report is the official record documenting the contribution of a radiologist to a patients care. The use of structured reports and a common lexicon will help referring physicians better understand the contents of reports. These same features in electronic health records will enable radiologists to mine reports for utilization management information as well as form the basis for clinical investigations.

Accuracy of MR imaging for staging prostate cancer: A meta-analysis to examine the effect of technologic change

RATIONALE AND OBJECTIVES The purpose of this study was to summarize the accuracy of magnetic resonance (MR) imaging for staging prostate cancer and to determine the effect of high magnetic field strength, use of the endorectal coil, use of fast spin-echo (SE) imaging, and study size on staging accuracy. MATERIALS AND METHODS A literature search and review yielded 27 studies comparing MR imaging to a pathologic standard in patients with clinically limited prostate cancer. Subgroup analyses examined magnetic field strength, use of an endorectal coil, use of fast SE imaging, publication date, and study size. RESULTS A summary receiver operating characteristic curve for all studies had a maximum joint sensitivity and specificity of 74%. At a specificity of 80% on this curve, sensitivity was 69%. Subgroup analyses showed that fast SE imaging was statistically significantly more accurate than conventional SE techniques (P < .001). Unexpectedly, studies employing higher magnetic field strength and those employing an endorectal coil were less accurate. CONCLUSION Seemingly small technologic advances may influence test accuracy. Early and small studies, however, may overstate accuracy because of publication bias, bias in small samples, or earlier studies being performed by the experts who developed the technology itself.

Accuracy of MR imaging in the work-up of suspicious breast lesions: A diagnostic meta-analysis

RATIONALE AND OBJECTIVES The authors performed a systematic, critical review of the literature on magnetic resonance (MR) imaging for primary breast cancer detection in patients with suspicious breast lesions, analyzed MR test performance in the articles meeting study criteria, and used this information to examine the cost-effectiveness of preoperative MR imaging. MATERIALS AND METHODS A structured, predefined MEDLINE search was conducted to identify potentially relevant, peer-reviewed, English-language references from January 1996 through August 1997 on the diagnostic accuracy of breast MR imaging. This information was supplemented by manually searching bibliographies of the retrieved articles for additional potentially relevant references. All studies were independently abstracted by two reviewers using a prospectively designed worksheet. Abstraction results were analyzed with the summary receiver operating characteristic (ROC) method. RESULTS Of 41 identified studies, 16 met the inclusion criteria. These studies reported sensitivities ranging from 63% to 100% and specificities ranging from 21% to 100%. Maximum joint sensitivity and specificity of the summary ROC curve was 89% (95% confidence interval [CI]: 82%, 93%). At a sensitivity of 95%, specificity was 67%. When test performance values were applied to a previous cost-effectiveness analysis, the cost-effectiveness of preoperative MR imaging relative to that of excisional biopsy was confirmed, but its cost-effectiveness relative to that of needle core biopsy varied widely. CONCLUSION For MR imaging to be a cost-effective alternative to excisional biopsy for diagnosis of suspicious breast lesions, its diagnostic test performance must be equal to or better than the best results in recently published studies.

Structured Radiology Reporting: Are We There Yet?

In this issue of the journal, Johnson and colleagues provide us with the first clear evidence of where roadblocks can be found on the road to structured reporting.

Accuracy of CT angiography versus pulmonary angiography in the diagnosis of acute pulmonary embolism: Evaluation of the literature with summary ROC curve analysis

RATIONALE AND OBJECTIVES The authors performed this study to estimate, by using published data, the sensitivity and specificity of computed tomographic (CT) angiography in the evaluation of suspected acute pulmonary embolism (PE). MATERIALS AND METHODS Summary receiver operating characteristic (ROC) curve analysis was used to determine the sensitivity and specificity of CT angiography in the diagnosis of acute PE. Pulmonary angiography was used as the diagnostic standard of reference. The authors reviewed the results of 11 independent studies published in the English-language literature between January 1992 and June 1999. RESULTS The sensitivity of CT angiography in the diagnosis or exclusion of PE in the central pulmonary arteries (to the level of the segmental pulmonary arteries) ranged from 0.74 to 0.81 on the basis of specificities of 0.89-0.91. The sensitivity of CT angiography in the diagnosis or exclusion of PE in all pulmonary arteries (to the level of the subsegmental pulmonary arteries) was 0.68 on the basis of a specificity of 0.91. CONCLUSION On the basis of the studies in the current literature, most of which used 5.0-mm collimation and single-detector CT, CT angiography may be less accurate in the diagnosis of PE than previously reported. With improvements in data acquisition, particularly the use of thinner section collimation and multidetector CT, and in the increased use of workstations for data analysis, the accuracy and utility of CT angiography will require continued investigation.

A Methodology for Generating Computer-based Explanations of Decision-theoretic Advice

Decision analysis is an appealing methodology with which to provide decision support to the practicing physician. However, its use in the clinical setting is impeded because computer- based explanations of decision-theoretic advice are difficult to generate without resorting to mathematical arguments. Nevertheless, human decision analysts generate useful and in tuitive explanations based on decision trees. To facilitate the use of decision theory in a computer-based decision support system, the authors developed a computer program that uses symbolic reasoning techniques to generate nonquantitative explanations of the results of decision analyses. A combined approach has been implemented to explain the differences in expected utility among branches of a decision tree. First, the mathematical relationships inherent in the structure of the tree are used to find any asymmetries in tree structure or inequalities among analogous decision variables that are responsible for a difference in expected utility. Next, an explanation technique is selected and applied to the most significant variables, creating a symbolic expression that justifies the decision. Finally, the symbolic expression is converted to English-language text, thereby generating an explanation that justifies the desirability of the choice with the greater expected utility. The explanation does not refer to mathematical formulas, nor does it include probability or utility values. The results suggest that explanations produced by a combination of decision analysis and symbolic processing techniques may be more persuasive and acceptable to clinicians than those produced by either technique alone. Key words: automated explanation; artificial intelligence; decision theory; decision support systems; medical informatics; stochastic simulation. (Med Decis Making 8:290-303, 1988)

The effect of PACS on the time required for technologists to produce radiographic images in the emergency department radiology suite.

The purpose of this study was to evaluate the effect of a switch to a filmless image management system on the time required for technologists to produce radiographic images in the emergency department (ED) after controlling for exam difficulty and a variable workload. Time and motion data were collected on patients who had radiographic images taken while being treated in the emergency department over the 3½-year period from April 1997 to November 2000. Event times and demographic data were obtained from the radiology information system, from the hospital information system, from emergency department records, or by observation by research coordinators. Multiple least squares regression analysis identified several independent predictors of the time required for technologists to produce radiographic images. These variables included the level of technologist experience, the number of trauma-alert patient arrivals, and whether a filmless image management system was used (all P <.05). Our regression model explained 22% of the variability in technologist time (R2 Adjusted, 0.22; F = 24.01; P <.0001). The regression model predicted a time saving of 2 to 3 minutes per patient in the elapsed time from notification of a needed examination until image availability because of the implementation of PACS, a delay of 4 to 6 minutes per patient who were imaged by technologists who spent less than 10% of their work assignments within the ED, and a delay of 18 to 27 minutes in radiology workflow because of the arrival of a trauma alert patient. A filmless system decreased the amount of time required to produce radiographs. The arrival of a trauma alert patient delayed radiology workflow in the ED. Inexperienced technologists require 4 to 6 minutes of additional time per patient to complete the same amount of work accomplished by an experienced technologist.

Information extraction from multi-institutional radiology reports

OBJECTIVES The radiology report is the most important source of clinical imaging information. It documents critical information about the patients health and the radiologists interpretation of medical findings. It also communicates information to the referring physicians and records that information for future clinical and research use. Although efforts to structure some radiology report information through predefined templates are beginning to bear fruit, a large portion of radiology report information is entered in free text. The free text format is a major obstacle for rapid extraction and subsequent use of information by clinicians, researchers, and healthcare information systems. This difficulty is due to the ambiguity and subtlety of natural language, complexity of described images, and variations among different radiologists and healthcare organizations. As a result, radiology reports are used only once by the clinician who ordered the study and rarely are used again for research and data mining. In this work, machine learning techniques and a large multi-institutional radiology report repository are used to extract the semantics of the radiology report and overcome the barriers to the re-use of radiology report information in clinical research and other healthcare applications. MATERIAL AND METHODS We describe a machine learning system to annotate radiology reports and extract report contents according to an information model. This information model covers the majority of clinically significant contents in radiology reports and is applicable to a wide variety of radiology study types. Our automated approach uses discriminative sequence classifiers for named-entity recognition to extract and organize clinically significant terms and phrases consistent with the information model. We evaluated our information extraction system on 150 radiology reports from three major healthcare organizations and compared its results to a commonly used non-machine learning information extraction method. We also evaluated the generalizability of our approach across different organizations by training and testing our system on data from different organizations. RESULTS Our results show the efficacy of our machine learning approach in extracting the information models elements (10-fold cross-validation average performance: precision: 87%, recall: 84%, F1 score: 85%) and its superiority and generalizability compared to the common non-machine learning approach (p-value<0.05). CONCLUSIONS Our machine learning information extraction approach provides an effective automatic method to annotate and extract clinically significant information from a large collection of free text radiology reports. This information extraction system can help clinicians better understand the radiology reports and prioritize their review process. In addition, the extracted information can be used by researchers to link radiology reports to information from other data sources such as electronic health records and the patients genome. Extracted information also can facilitate disease surveillance, real-time clinical decision support for the radiologist, and content-based image retrieval.