Mary H. Scanlon

Increased Error Rates in Preliminary Reports Issued by Radiology Residents Working More Than 10 Consecutive Hours Overnight

RATIONALE AND OBJECTIVES To determine if the rate of major discrepancies between resident preliminary reports and faculty final reports increases during the final hours of consecutive 12-hour overnight call shifts. MATERIALS AND METHODS Institutional review board exemption status was obtained for this study. All overnight radiology reports interpreted by residents on-call between January 2010 and June 2010 were reviewed by board-certified faculty and categorized as major discrepancies if they contained a change in interpretation with the potential to impact patient management or outcome. Initial determination of a major discrepancy was at the discretion of individual faculty radiologists based on this general definition. Studies categorized as major discrepancies were secondarily reviewed by the residency program director (M.H.S.) to ensure consistent application of the major discrepancy designation. Multiple variables associated with each report were collected and analyzed, including the time of preliminary interpretation, time into shift study was interpreted, volume of studies interpreted during each shift, day of the week, patient location (inpatient or emergency department), block of shift (2-hour blocks for 12-hour shifts), imaging modality, patient age and gender, resident identification, and faculty identification. Univariate risk factor analysis was performed to determine the optimal data format of each variable (ie, continuous versus categorical). A multivariate logistic regression model was then constructed to account for confounding between variables and identify independent risk factors for major discrepancies. RESULTS We analyzed 8062 preliminary resident reports with 79 major discrepancies (1.0%). There was a statistically significant increase in major discrepancy rate during the final 2 hours of consecutive 12-hour call shifts. Multivariate analysis confirmed that interpretation during the last 2 hours of 12-hour call shifts (odds ratio (OR) 1.94, 95% confidence interval (CI) 1.18-3.21), cross-sectional imaging modality (OR 5.38, 95% CI 3.22-8.98), and inpatient location (OR 1.81, 95% CI 1.02-3.20) were independent risk factors for major discrepancy. CONCLUSIONS In a single academic medical center, major discrepancies in resident preliminary reports increased significantly during the final 2 hours of consecutive 12-hour overnight call shifts. This finding could be related to either fatigue or circadian desynchronization. Discrimination of these two potential etiologies requires additional investigation as major discrepancies in resident reports have the potential to negatively impact patient care/outcome. Cross-sectional imaging modalities including computed tomography and ultrasound (versus conventional radiography), as well as inpatient location (versus Emergency Department location), were also associated with significantly higher major discrepancy rates.

Changes in Technologist Productivity with Implementation of an Enterprisewide PACS

The purpose of this report is to determine what effect filmless operation has on technologist productivity when compared with traditional film-based operation. Retrospective data on technologist productivity was collected from the study institution before and after implementation of PACS using workload reports and payroll records. Departmentwide technologist productivity was defined as the number of examinations per full-time equivalent (exams/FTE) and correlated with local and nationwide standards operating in traditional film-based operations. During filmbased operation, technologist productivity was comparable between the study institution and nationwide standards, allowing for the unique examination volumes and modality mix. After implementation of a large-scale PACS, technologist productivity was found to increase 34% above that of national standards and 48% that of the local control site. Implementation of an enterprisewide PACS offers the potential to significantly improve departmentwide technologist productivity when compared with traditional film-based operation.

Using a Web-Based Application to Enhance Resident Training and Improve Performance On-call

RATIONALE AND OBJECTIVES It is common practice in academic hospitals for radiology residents to provide preliminary interpretations for radiologic examinations performed in the emergency department (ED) during off-hours. In this study, we used a software program called Minerva to identify and track discrepancies between resident and faculty interpretation of ED studies. The objective was to determine if missed case conferences could reduce the number of resident discrepancies related to the types of cases reviewed. MATERIALS AND METHODS We used Minerva to identify and grade faculty-modified resident preliminary reports as minor or major discrepancies depending on whether the discrepancy had the potential to affect patient management or outcome. Minor and major discrepancy rates were calculated for all residents to evaluate call performance, establish benchmarks, and develop interventions to reduce the number of discrepant cases. RESULTS The total discrepancy rate for all residents (n = 22) was 2.6% with a standard deviation (SD) of 0.7%. The average major discrepancy rate for all residents was 1.1% with a SD of 0.4%. Trend analysis of missed cases was used to generate topic-specific resident missed case conferences on acromioclavicular joint separation injuries, elbow joint effusions, and osteochondral fractures, which resulted in an overall 64% decrease in the number of missed cases related to these injuries. CONCLUSIONS The systematic evaluation of resident discrepancies using a simple software application provides a competency-based metric to assess call performance, establish benchmarks, and develop missed case conferences. This process is expected to result in further reduction in resident discrepancy rates and missed cases.

Using Focused Missed-Case Conferences to Reduce Discrepancies in Musculoskeletal Studies Interpreted by Residents On Call

OBJECTIVE The purpose of this study is to determine whether focused missed-case conferences can significantly reduce the number of major discrepancies in musculoskeletal imaging studies interpreted by residents on call. MATERIALS AND METHODS A review of major discrepancies in musculoskeletal conventional radiography imaging studies interpreted by radiology residents and fellows on call from July 2008 to July 2009 revealed 31 common and important musculoskeletal injuries missed or misinterpreted at our institution. These missed cases were presented during focused missed-case conferences from July through October 2009. Only residents attended missed-case conferences. RESULTS Over the 12 months before the missed-case conferences, there were 55 resident major discrepancies and 25 fellow major discrepancies, representing 31 common and important missed musculoskeletal injuries. Over the 12 months after the missed-case conferences, there were 18 resident major discrepancies and 21 fellow major discrepancies involving these injuries. This corresponds to a 67% reduction in the number of resident major discrepancies involving the 31 musculoskeletal injuries covered during the missed-case conferences (chi-square p < 0.001). The overall major discrepancy rate for all musculoskeletal conventional radiography studies was 1.19% for residents and 1.55% for fellows (not significant) before the missed-case conferences and 0.87% for residents and 1.46% for fellows (p < 0.05) after the missed-case conferences. During this time, fellows missed more musculoskeletal injuries related to the topics discussed during missed-case conferences (16) compared with residents (8) although fellows read significantly fewer studies overall. This accounted for 0.49% of the 0.59% difference between residents and fellows. CONCLUSION Focused missed-case conferences are an effective educational intervention to significantly reduce the number of major discrepancies in radiology resident interpretation of musculoskeletal imaging studies on call.

Orion: A Web-Based Application Designed to Monitor Resident and Fellow Performance On-Call

Radiology residency and fellowship training provides a unique opportunity to evaluate trainee performance and determine the impact of various educational interventions. We have developed a simple software application (Orion) using open-source tools to facilitate the identification and monitoring of resident and fellow discrepancies in on-call preliminary reports. Over a 6-month period, 19,200 on-call studies were interpreted by 20 radiology residents, and 13,953 on-call studies were interpreted by 25 board-certified radiology fellows representing eight subspecialties. Using standard review macros during faculty interpretation, each of these reports was classified as “agreement”, “minor discrepancy”, and “major discrepancy” based on the potential to impact patient management or outcome. Major discrepancy rates were used to establish benchmarks for resident and fellow performance by year of training, modality, and subspecialty, and to identify residents and fellows demonstrating a significantly higher major discrepancy rate compared with their classmates. Trends in discrepancies were used to identify subspecialty-specific areas of increased major discrepancy rates in an effort to tailor the didactic and case-based curriculum. A series of missed-case conferences were developed based on trends in discrepancies, and the impact of these conferences is currently being evaluated. Orion is a powerful information technology tool that can be used by residency program directors, fellowship programs directors, residents, and fellows to improve radiology education and training.

The High-Performing Radiology Residency: A Case Study

The primary goal of any radiology residency program is to train clinically competent radiologists. However, some radiology programs go above and beyond basic ACGME requirements and produce highly productive radiologists who contribute to the field in research, education and technology. At the core of such residencies are the people who comprise it, including self-motivated, personable residents and faculty who can facilitate excellence from their trainees. Indeed, radiology residencies which create unique programs in response to evolving resident interests can help foster resident-driven research, advocacy, technological innovation and early leadership. These unique programs, when interwoven with a strong educational foundation driven by evidence-based learning techniques and informatics-driven evaluation programs, function to produce highly competent clinicians who become leaders in the field.

Minerva: using a software program to improve resident performance during independent call (Withdrawal Notice)

This paper was presented at the SPIE conference indicated above and has been withdrawn from publication at the request of the authors.