Prithima Mosaly

Quantitative assessment of workload and stressors in clinical radiation oncology

PURPOSE Workload level and sources of stressors have been implicated as sources of error in multiple settings. We assessed workload levels and sources of stressors among radiation oncology professionals. Furthermore, we explored the potential association between workload and the frequency of reported radiotherapy incidents by the World Health Organization (WHO). METHODS AND MATERIALS Data collection was aimed at various tasks performed by 21 study participants from different radiation oncology professional subgroups (simulation therapists, radiation therapists, physicists, dosimetrists, and physicians). Workload was assessed using National Aeronautics and Space Administration Task-Load Index (NASA TLX). Sources of stressors were quantified using observational methods and segregated using a standard taxonomy. Comparisons between professional subgroups and tasks were made using analysis of variance ANOVA, multivariate ANOVA, and Duncan test. An association between workload levels (NASA TLX) and the frequency of radiotherapy incidents (WHO incidents) was explored (Pearson correlation test). RESULTS A total of 173 workload assessments were obtained. Overall, simulation therapists had relatively low workloads (NASA TLX range, 30-36), and physicists had relatively high workloads (NASA TLX range, 51-63). NASA TLX scores for physicians, radiation therapists, and dosimetrists ranged from 40-52. There was marked intertask/professional subgroup variation (P<.0001). Mental demand (P<.001), physical demand (P=.001), and effort (P=.006) significantly differed among professional subgroups. Typically, there were 3-5 stressors per cycle of analyzed tasks with the following distribution: interruptions (41.4%), time factors (17%), technical factors (13.6%), teamwork issues (11.6%), patient factors (9.0%), and environmental factors (7.4%). A positive association between workload and frequency of reported radiotherapy incidents by the WHO was found (r = 0.87, P value=.045). CONCLUSIONS Workload level and sources of stressors vary among professional subgroups. Understanding the factors that influence these findings can guide adjustments to the workflow procedures, physical layout, and/or communication protocols to enhance safety. Additional evaluations are needed in order to better understand if these findings are systemic.

Relating physician's workload with errors during radiation therapy planning.

PURPOSE To relate subjective workload (WL) levels to errors for routine clinical tasks. METHODS AND MATERIALS Nine physicians (4 faculty and 5 residents) each performed 3 radiation therapy planning cases. The WL levels were subjectively assessed using National Aeronautics and Space Administration Task Load Index (NASA-TLX). Individual performance was assessed objectively based on the severity grade of errors. The relationship between the WL and performance was assessed via ordinal logistic regression. RESULTS There was an increased rate of severity grade of errors with increasing WL (P value = .02). As the majority of the higher NASA-TLX scores, and the majority of the performance errors were in the residents, our findings are likely most pertinent to radiation oncology centers with training programs. CONCLUSIONS WL levels may be an important factor contributing to errors during radiation therapy planning tasks.

Identifying factors and root causes associated with near-miss or safety incidents in patients treated with radiotherapy: A case-control analysis

PURPOSE To identify factors associated with a near-miss or safety incident (NMSI) in patients undergoing radiotherapy and identify common root causes of NMSIs and their relationship with incident severity. METHODS We retrospectively studied NMSIs filed between October 2014 and April 2016. We extracted patient-, treatment-, and disease-specific data from patients with an NMSI (n = 200; incident group) and a similar group of control patients (n = 200) matched in time, without an NMSI. A root cause and incident severity were determined for each NMSI. Univariable and multivariable analyses were performed to determine which specific factors were contributing to NMSIs. Multivariable logistic regression was used to determine root causes of NMSIs and their relationship with incident severity. RESULTS NMSIs were associated with the following factors: head and neck sites (odds ratio [OR], 5.2; P = .01), image-guided intensity-modulated radiotherapy (OR, 3; P = .009), daily imaging (OR, 7; P < .001), and tumors staged as T2 (OR, 3.3; P = .004). Documentation and scheduling errors were the most common root causes (29%). Communication errors were more likely to affect patients ( P < .001), and technical treatment delivery errors were most associated with a higher severity score ( P = .005). CONCLUSION Several treatment- and disease-specific factors were found to be associated with an NMSI. Overall, our results suggest that complexity (eg, head and neck, image-guided intensity-modulated radiotherapy, and daily imaging) might be a contributing factor for an NMSI. This promotes an idea of developing a more dedicated and robust quality assurance system for complex cases and highlights the importance of a strong reporting system to support a safety culture.

Effects of scaffolding equipment interventions on muscle activation and task performance

Examination of a local power utility’s injury database revealed maintenance personnel to experience high injury rates. Maintenance jobs were analyzed using an ergonomic risk factor screening tool and scaffolding tasks, including walk-board tie-down and frame tube coupling, were found to pose high risks. Factors included high torques at joints and awkward posture positions. The purpose of this study was to conceptualize interventions to reduce risks and conduct experiments to assess the impact of interventions on worker muscle activation and performance. Nine male operators were recruited from the utility and participated in two tests of novel walk-board tie-down and frame coupling equipment. Muscle activation was measured using electromyography in scaffold assemble/disassemble tasks along with time-to-task completion. Results revealed plastic zip-ties and quick-clamping couplers to reduce mean normalized muscle responses and support performance comparable to conventional metal-wire ties and ratcheting clamps. These ergonomic interventions maybe implemented in other areas where scaffolding is used.

Relating task demand, mental effort and task difficulty with physicians’ performance during interactions With Electronic Health Records (EHRs)

ABSTRACT Objective was to assess the relationship between task demand, mental effort, task difficulty, and performance during physicians’ interaction with electronic health records (EHRs). Seventeen physicians performed three EHR-based scenarios with varying task demands. Mental effort was measured using eye tracking measures via task evoked pupillary responses (TEPR), blink frequency, and gaze speed; task difficulty (or user behavior) was measured using frequent mouse click patterns and task flow; user performance was quantified using two types of omission errors: (i) omission errors with no evidence of trying to complete the task and (ii) omission error with evidence of trying but unable to complete the task. The results indicated that task demand significantly increased mental effort, but not task difficulty. Task demand, mental effort, and task difficulty all predicted performance. Specifically, there was a significant relationship between (i) task demand, TEPR and omission errors with no evidence of trying to complete the task, and (ii) blink frequency, repeated search clicks and omission error with evidence of trying but unable to complete the task. In concert, results suggest that physicians’ performance during EHR interaction was negatively affected by task demands and increase in mental effort. This highlight the need for implementation of appropriate quality assurance (QA) measures, in addition to EHR usability improvement, to minimize omission errors and improve physician’s performance. Additionally, the lack of relationship between task demand and task difficulty highlights a need for further methodological and empirical studies to advance our understanding from theory to application during physician–EHR interaction.

Empirical Evaluation of Workload of the Radiation Oncology Physicist During Radiation Treatment Planning and Delivery

In recent years, the practice of radiation oncology has changed due to several technological advances. As such, there is growing interest in the evolving nature of safety and operational challenges faced by radiation oncology professionals. This research focuses on physicists who play an important role in the radiation therapy treatment planning and delivery process. Specifically, the purpose of our research is to assess their workload levels using the NASA TLX method in order to identify tasks that might compromise patient safety. Based on empirical observations, this study provides practical suggestions for lowering workload levels that ultimately can reduce the probability of errors.

Physiological Evaluation and Quantification of Physician’s Cognitive Workload During Interaction with Computer Based Clinical System

Tracking cognitive workload (CWL) of physicians interacting with health information technology (HIT) might be useful in order to identify high-risk tasks, and to flag situations when performance might be expected to decline. Eight physician radiation oncologists (3-faculty, 5-residents) pupillary responses were monitored during treatment-planning tasks. The average change in task evoked pupillary response (TEPR) from pre-set baseline was calculated and the percent of time that the TEPR dilated by ≥0.45 mm (from historical studies) was taken as a measure of CWL where performance degradation could be expected. Physician performance was assessed subjectively (willingness-to-approve the treatment-plan) and objectively (number and severity of errors). There was an association between CWL and subjective performance (p 0.05) as assessed using logistic regression analysis. Future research is needed to further advance available methods to quantify the relationship between CWL and performance during physicians-HIT interactions.

Classification of EEG Features for Prediction of Working Memory Load

The objective of this research was to compare classification methods aimed at predicting working memory (WM) load. Electroencephalogram (EEG) data was collected from physicians while performing basic WM tasks and simulated medical scenarios. Data processing was performed to remove noise from the signal used for analysis (e.g., muscle activity, eye-blinks). The data from basic WM tasks was used to develop and test the four classification models (LASSO regression, support vector machines (SVM), nearest shrunken centroids (NSC), and iterated supervised principal components (ISPC) to predict a WM state indicative of physicians’ optimal performance. The naive misclassification rate was 19.74 %; LASSO and SVM outperformed this threshold: 18.10 and 12.21 % respectively). Both classification models had relatively high-specificity (LASSO: 97.2 %; SVM: 99.8 %); but relatively low-sensitivity LASSO: 20.7 %; SVM: 39.6 %). Results from simulated medical scenarios suggest that physicians were approximately 83 % of the time in the WM state that is likely indicative of optimal performance.

Quantification of baseline pupillary response and task-evoked pupillary response during constant and incremental task load

Abstract The methods employed to quantify the baseline pupil size and task-evoked pupillary response (TEPR) may affect the overall study results. To test this hypothesis, the objective of this study was to assess variability in baseline pupil size and TEPR during two basic working memory tasks: constant load of 3-letters memorisation-recall (10 trials), and incremental load memorisation-recall (two trials of each load level), using two commonly used methods (1) change from trail/load specific baseline, (2) change from constant baseline. Results indicated that there was a significant shift in baseline between the trails for constant load, and between the load levels for incremental load. The TEPR was independent of shifts in baseline using method 1 only for constant load, and method 2 only for higher levels of incremental load condition. These important findings suggest that the assessment of both the baseline and methods to quantify TEPR are critical in ergonomics application, especially in studies with small number of trials per subject per condition. Practitioner Summary: Quantification of TEPR can be affected by shifts in baseline pupil size that are most likely affected by non-cognitive factors when other external factors are kept constant. Therefore, quantification methods employed to compute both baseline and TEPR are critical in understanding the information processing of humans in practical ergonomics settings.

Usability Evaluation of Electronic Health Record System (EHRs) using Subjective and Objective Measures

The objective of this pilot study was to explore the applicability of various evaluation methods (subjective and objective) for assessing usability of electronic health record system (EHRs) during physician interaction on simple vs. complex tasks. Five physicians performed two simulated clinical scenarios consisting of 9 tasks using the EHR. Tasks were categorized into simple vs. complex tasks based on the user, task and context characteristics by the subject matter expert. Usability was assessed using four methods, (1) subjectively using subjects informal feedback and usability experts heuristics, (2) workload measures using eye tracking, (3) behavior measures using clicks and navigation windows, and (4) performance measures using actual time on task and predictive time based on CogTool ©. Overall, the results suggest that heuristic methods (1) are highly effective in identifying usability issues, with other methods (2-4) providing complementary analysis to identify differences is task complexly and user experience with EHRs.