Aaron S. Kusano

Metrics of success: Measuring impact of a departmental near-miss incident learning system

PURPOSE There is a growing interest in the application of incident learning systems (ILS) to radiation oncology. The purpose of the present study is to define statistical metrics that may serve as benchmarks for successful operation of an incident learning system. METHODS AND MATERIALS A departmental safety and quality ILS was developed to monitor errors, near-miss events, and process improvement suggestions. Event reports were reviewed by a multiprofessional quality improvement committee. Events were scored by a near-miss risk index (NMRI) and categorized by event point of origination and discovery. Reporting trends were analyzed over a 2-year period, including total number and rates of events reported, users reporting, NMRI, and event origination and discovery. RESULTS A total of 1897 reports were evaluated (1.0 reports/patient, 0.9 reports/unique treatment course). Participation in the ILS increased as demonstrated by total events (2.1 additional reports/month) and unique users (0.5 new users/month). Sixteen percent of reports had an NMRI of 0 (none), 42% had an NMRI of 1 (mild), 25% had an NMRI of 2 (moderate), 12% had an NMRI of 3 (severe), and 5% had an NMRI of 4 (critical). Event NMRI showed a significant decrease in the first 6 months (1.68-1.42, P < .001). Trends in origination and discovery of reports were broadly distributed between radiation therapy process steps and staff groups. The highest risk events originated in imaging for treatment planning (NMRI = 2.0 ± 1.1; P < .0001) and were detected in on-treatment quality management (NMRI = 1.7 ± 1.1; P = .003). CONCLUSIONS Over the initial 2-year period of ILS operation, rates of reporting increased, staff participation increased, and NMRI of reported events declined. These data mirror previously reported findings of improvement in safety culture endpoints. These metrics may be useful for other institutions seeking to create or evaluate their own ILS.

Measurable improvement in patient safety culture: A departmental experience with incident learning

PURPOSE Rigorous use of departmental incident learning is integral to improving patient safety and quality of care. The goal of this study was to quantify the impact of a high-volume, departmental incident learning system on patient safety culture. METHODS AND MATERIALS A prospective, voluntary, electronic incident learning system was implemented in February 2012 with the intent of tracking near-miss/no-harm incidents. All incident reports were reviewed weekly by a multiprofessional team with regular department-wide feedback. Patient safety culture was measured at baseline with validated patient safety culture survey questions. A repeat survey was conducted after 1 and 2 years of departmental incident learning. Proportional changes were compared by χ(2) or Fisher exact test, where appropriate. RESULTS Between 2012 and 2014, a total of 1897 error/near-miss incidents were reported, representing an average of 1 near-miss report per patient treated. Reports were filed by a cross section of staff, with the majority of incidents reported by therapists, dosimetrists, and physicists. Survey response rates at baseline and 1 and 2 years were 78%, 80%, and 80%, respectively. Statistically significant and sustained improvements were noted in several safety metrics, including belief that the department was openly discussing ways to improve safety, the sense that reports were being used for safety improvement, and the sense that changes were being evaluated for effectiveness. None of the surveyed dimensions of patient safety culture worsened. Fewer punitive concerns were noted, with statistically significant decreases in the worry of embarrassment in front of colleagues and fear of getting colleagues in trouble. CONCLUSIONS A comprehensive incident learning system can identify many areas for improvement and is associated with significant and sustained improvements in patient safety culture. These data provide valuable guidance as incident learning systems become more widely used in radiation oncology.

Survey of Bereavement Practices of Cancer Care and Palliative Care Physicians in the Pacific Northwest United States

PURPOSE Physicians caring for patients with cancer frequently encounter individuals who will die as a result of their disease. The primary aim of this study was to examine the frequency and nature of bereavement practices among cancer care and palliative care physicians in the Pacific Northwest United States. Secondary aims included identification of factors and barriers associated with bereavement follow-up. METHODS An institutional review board (IRB) -approved, anonymous online survey of cancer specialists and palliative care physicians in Alaska, Washington, Oregon, Idaho, Montana, and Wyoming was performed in fall 2010. Potential participants were identified through membership in national professional organizations. Summary statistics and logistic regression methods were used to examine frequency and predictors of bereavement practices. RESULTS A total of 194 (22.7%) of 856 physicians participated in the online survey, with 164 (19.1%) meeting study inclusion criteria. Overall, 70% of respondents reported always or usually making a telephone call to families, sending a condolence letter, or attending a funeral service after a patients death. The most common perceived barriers to bereavement follow-up were lack of time and uncertainty of which family member to contact. Sixty-nine percent of respondents did not feel that they had received adequate training on bereavement follow-up during postgraduate training. CONCLUSION Although a significant portion of respondents engaged in some form of bereavement follow-up, the majority felt inadequately trained in these activities. Efforts to identify available resources and address bereavement activities in postgraduate training may contribute to improved multidisciplinary treatment of patients with cancer and their families.

Burnout in United States academic chairs of radiation oncology programs

PURPOSE The aims of this study were to determine the self-reported prevalence of burnout in chairs of academic radiation oncology departments, to identify factors contributing to burnout, and to compare the prevalence of burnout with that seen in other academic chair groups. METHODS AND MATERIALS An anonymous online survey was administered to the membership of the Society of Chairs of Academic Radiation Oncology Programs (SCAROP). Burnout was measured with the Maslach Burnout Inventory-Human Services Survey (MBI-HSS). RESULTS Questionnaires were returned from 66 of 87 chairs (76% response rate). Seventy-nine percent of respondents reported satisfaction with their current positions. Common major stressors were budget deficits and human resource issues. One-quarter of chairs reported that it was at least moderately likely that they would step down in the next 1 to 2 years; these individuals demonstrated significantly higher emotional exhaustion. Twenty-five percent of respondents met the MBI-HSS criteria for low burnout, 75% for moderate burnout, and none for high burnout. Group MBI-HSS subscale scores demonstrated a pattern of moderate emotional exhaustion, low depersonalization, and moderate personal accomplishment, comparing favorably with other specialties. CONCLUSIONS This is the first study of burnout in radiation oncology chairs with a high response rate and using a validated psychometric tool. Radiation oncology chairs share similar major stressors to other chair groups, but they demonstrate relatively high job satisfaction and lower burnout. Emotional exhaustion may contribute to the anticipated turnover in coming years. Further efforts addressing individual and institutional factors associated with burnout may improve the relationship with work of chairs and other department members.

Trimodality Treatment of Malignant Pleural Mesothelioma: An Institutional Review.

Objective: Malignant pleural mesothelioma (MPM) is a deadly disease with varying treatment options. This study retrospectively describes treatment practices at the University of Washington Medical System from 1980 to 2011, and evaluates the impact of trimodality therapy and radiation (photon and neutron) on survival. Methods: A retrospective study was conducted on patients treated for MPM. Univariate and multivariate methods were utilized to evaluate potential factors associated with survival. Treatments received and baseline characteristics were included. Survival analysis of trimodality therapy was performed using a propensity score method to control for baseline characteristics. Results: Among 78 eligible patients, the median age at diagnosis was 59 years and the median survival was 13.7 months. On multivariate analysis, the significant predictors of improved survival were age, smoking history, location, and receipt of radiation therapy or chemotherapy. In the 48 patients receiving radiation therapy, the difference in survival between neutron therapy and non-neutron therapy patients was not statistically significant: hazard ratio, 1.20 (95% confidence interval, 0.68-2.13), P=0.52. Patients receiving trimodality therapy were more likely to have early-stage disease (60% vs. 30%) and epithelioid histology (86% vs. 58%). In a propensity score-weighted Cox proportional hazards model, trimodality therapy patients had improved overall survival, hazard ratio 0.45, P=0.004, median 14.6 versus 8.6 months. Conclusions: Trimodality therapy was significantly associated with prolonged survival in patients with MPM, even when adjusting for baseline patient factors. Radiation therapy was associated with improved survival, but the modality of radiation therapy used was not associated with outcome.

Fast neutron radiotherapy in the treatment of malignant pleural mesothelioma.

Introduction:Malignant pleural mesothelioma (MPM) is a fatal disease lacking standardized treatment. We describe the use of fast neutron radiation therapy in MPM patients referred to the Department of Radiation Oncology at the University of Washington Medical Center. Materials and Methods:Retrospective chart review of MPM patients receiving neutron radiotherapy treatment from 1980 to 2012. Results:A total of 30 MPM patients received fast neutron radiotherapy as part of their treatment regimen. Median age at diagnosis was 59.6 years (range, 46.6 to 72.3 y). Eighteen patients received fast neutron radiotherapy as a component of trimodality treatment. Median overall survival was 20.3 months (range, 5.5 to 73.3 mo) with 1 patient censored at 34.8 months and all other patients with confirmed dates of death. One patient receiving radiotherapy alone as a palliative measure died during radiation treatment. One patient was unable to tolerate radiotherapy and stopped before completing prescribed treatment. On univariate analysis, Brigham Stage at presentation was a significant predictor of survival (P<0.01). No significant differences in survival were observed when comparing patients who received trimodality treatment compared to those who did not. Conclusions:Fast neutron radiotherapy may be utilized in the management of MPM patients. However, treatment with fast neutron radiotherapy did not significantly improvement outcome, even when used in a trimodality regimen.

MO‐G‐BRE‐06: Metrics of Success: Measuring Participation and Attitudes Related to Near‐Miss Incident Learning Systems

PURPOSE Interest in incident learning systems (ILS) for improving safety and quality in radiation oncology is growing, as evidenced by the upcoming release of the national ILS. However, an institution implementing such a system would benefit from quantitative metrics to evaluate performance and impact. We developed metrics to measure volume of reporting, severity of reported incidents, and changes in staff attitudes over time from implementation of our institutional ILS. METHODS We analyzed 2023 incidents from our departmental ILS from 2/2012-2/2014. Incidents were prospectively assigned a near-miss severity index (NMSI) at multidisciplinary review to evaluate the potential for error ranging from 0 to 4 (no harm to critical). Total incidents reported, unique users reporting, and average NMSI were evaluated over time. Additionally, departmental safety attitudes were assessed through a 26 point survey adapted from the AHRQ Hospital Survey on Patient Safety Culture before, 12 months, and 24 months after implementation of the incident learning system. RESULTS Participation in the ILS increased as demonstrated by total reports (approximately 2.12 additional reports/month) and unique users reporting (0.51 additional users reporting/month). Also, the average NMSI of reports trended lower over time, significantly decreasing after 12 months of reporting (p<0.001) but with no significant change at months 18 or 24. In survey data significant improvements were noted in many dimensions, including perceived barriers to reporting incidents such as concern of embarrassment (37% to 18%; p=0.02) as well as knowledge of what incidents to report, how to report them, and confidence that these reports were used to improve safety processes. CONCLUSION Over a two-year period, our departmental ILS was used more frequently, incidents became less severe, and staff confidence in the system improved. The metrics used here may be useful for other institutions seeking to create or evaluate their own incident learning systems.

SU‐E‐T‐230: Patient Safety Improvement Related to Changes in Ongoing Radiation Treatment Plan Identified with Near‐Miss Incidents Reporting

Purpose: A significant portion of patients undergoing radiation experience a change in the treatment plan during the treatment course for a variety of reasons, including adaptive planning due to tumor response and change in fractionation due to patient clinical status. This study tests whether mid‐course changes to treatment plans increase risk of errors through the use of a large institutional near‐miss incident reporting system. Methods: We analyzed incidents from a departmental near‐miss incident reporting system launched in 2/2012. All incidents were prospectively reviewed weekly by a multi‐disciplinary team including physicians, therapists, dosimetrists, physicists, nurses, and administrative staff. Incidents were assigned a near‐miss severity score ranging from 0 to 4 (no impact/mild/moderate/severe/critical), reflecting the potential harm of the incident if it had reached the patient. Monthly root‐case‐analysis is performed on incidents with the highest severity. Incidents related to change‐in‐plan were flagged, and severity score for these were compared to other incidents via t‐test. Results: From 2/2012 through 12/2012, 662 incidents were submitted through the departmental near‐miss incident reporting system. On multi‐disciplinary review, 59(9%) incidents were directly attributable to a change‐in‐plan. Average severity score for the 59 change‐in‐plan incidents was 1.9, significantly higher than score of 1.5 for other incidents not related to change‐in‐plan (p=0.02). Three(5%) near‐miss incidents were assigned the highest severity score of 4, with issues including: wrong isocenter, wrong fields assigned to new plan, and wrong information in new treatment plan due to existing plan. Conclusion: Changes in treatment plan are sometimes required to provide highest quality of care for patients receiving radiation. Although near‐miss incidents related to change‐in‐plan are relatively uncommon, when they do occur they are more severe than other types of incidents that are observed in the course of clinical operations. Development is underway for new safety process specifically tailored to this identified high‐risk patient group.