Melanie C. Wright

Objective measures of situation awareness in a simulated medical environment

One major limitation in the use of human patient simulators is a lack of objective, validated measures of human performance. Objective measures are necessary if simulators are to be used to evaluate the skills and training of medical practitioners and teams or to evaluate the impact of new processes or equipment design on overall system performance. Situation awareness (SA) refers to a person’s perception and understanding of their dynamic environment. This awareness and comprehension is critical in making correct decisions that ultimately lead to correct actions in medical care settings. An objective measure of SA may be more sensitive and diagnostic than traditional performance measures. This paper reviews a theory of SA and discusses the methods required for developing an objective measure of SA within the context of a simulated medical environment. Analysis and interpretation of SA data for both individual and team performance in health care are also presented.

Can We Make Postoperative Patient Handovers Safer? A Systematic Review of the Literature

Postoperative patient handovers are fraught with technical and communication errors and may negatively impact patient safety. We systematically reviewed the literature on handover of care from the operating room to postanesthesia or intensive care units and summarized process and communication recommendations based on these findings. From >500 papers, we identified 31 dealing with postoperative handovers. Twenty-four included recommendations for structuring the handover process or information transfer. Several recommendations were broadly supported, including (1) standardize processes (e.g., through the use of checklists and protocols); (2) complete urgent clinical tasks before the information transfer; (3) allow only patient-specific discussions during verbal handovers; (4) require that all relevant team members be present; and (5) provide training in team skills and communication. Only 4 of the studies developed an intervention and formally assessed its impact on different process measures. All 4 interventions improved metrics of effectiveness, efficiency, and perceived teamwork. Most of the papers were cross-sectional studies that identified barriers to safe, effective postoperative handovers including the incomplete transfer of information and other communication issues, inconsistent or incomplete teams, absent or inefficient execution of clinical tasks, and poor standardization. An association between poor-quality handovers and adverse events was also demonstrated. More innovative research is needed to define optimal patient handovers and to determine the effect of handover quality on patient outcomes.

Adaptive automation of human-machine system information-processing functions.

The goal of this research was to describe the ability of human operators to interact with adaptive automation (AA) applied to various stages of complex systems information processing, defined in a model of human-automation interaction. Forty participants operated a simulation of an air traffic control task. Automated assistance was adaptively applied to information acquisition, information analysis, decision making, and action implementation aspects of the task based on operator workload states, which were measured using a secondary task. The differential effects of the forms of automation were determined and compared with a manual control condition. Results of two 20-min trials of AA or manual control revealed a significant effect of the type of automation on performance, particularly during manual control periods as part of the adaptive conditions. Humans appear to better adapt to AA applied to sensory and psychomotor information-processing functions (action implementation) than to AA applied to cognitive functions (information analysis and decision making), and AA is superior to completely manual control. Potential applications of this research include the design of automation to support air traffic controller information processing.

Teamwork training with nursing and medical students: does the method matter? Results of an interinstitutional, interdisciplinary collaboration

Objectives The authors conducted a randomised controlled trial of four pedagogical methods commonly used to deliver teamwork training and measured the effects of each method on the acquisition of student teamwork knowledge, skills, and attitudes. Methods The authors recruited 203 senior nursing students and 235 fourth-year medical students (total N=438) from two major universities for a 1-day interdisciplinary teamwork training course. All participants received a didactic lecture and then were randomly assigned to one of four educational methods: didactic (control), audience response didactic, role play and human patient simulation. Student performance was assessed for teamwork attitudes, knowledge and skills using: (a) a 36-item teamwork attitudes instrument (CHIRP), (b) a 12-item teamwork knowledge test, (c) a 10-item standardised patient (SP) evaluation of student teamwork skills performance and (d) a 20-item modification of items from the Mayo High Performance Teamwork Scale (MHPTS). Results All four cohorts demonstrated an improvement in attitudes (F1,370=48.7, p=0.001) and knowledge (F1,353=87.3, p=0.001) pre- to post-test. No educational modality appeared superior for attitude (F3,370=0.325, p=0.808) or knowledge (F3,353=0.382, p=0.766) acquisition. No modality demonstrated a significant change in teamwork skills (F3,18=2.12, p=0.134). Conclusions Each of the four modalities demonstrated significantly improved teamwork knowledge and attitudes, but no modality was demonstrated to be superior. Institutions should feel free to utilise educational modalities, which are best supported by their resources to deliver interdisciplinary teamwork training.

Assessing teamwork in medical education and practice: relating behavioural teamwork ratings and clinical performance.

Background: Problems with communication and team coordination are frequently linked to adverse events in medicine. However, there is little experimental evidence to support a relationship between observer ratings of teamwork skills and objective measures of clinical performance. Aim: Our main objective was to test the hypothesis that observer ratings of team skill will correlate with objective measures of clinical performance. Methods: Nine teams of medical students were videotaped performing two types of teamwork tasks: (1) low fidelity classroom-based patient assessment and (2) high fidelity simulated emergent care. Observers used a behaviourally anchored rating scale to rate each individual on skills representative of assertiveness, decision-making, situation assessment, leadership, and communication. A checklist-based measure was used to assess clinical team performance. Results: Moderate to high inter-observer correlations and moderate correlations between cases established the validity of a behaviourally anchored team skill rating tool for simulated emergent care. There was moderate to high correlation between observer ratings of team skill and checklist-based measures of team performance for the simulated emergent care cases (r = 0.65, p = 0.06 and r = 0.97, p < 0.0001). Conclusions: These results provide prospective evidence of a positive relationship between observer ratings of team skills and clinical team performance in a simulated dynamic health care task.

Time of day effects on the incidence of anesthetic adverse events

Background: We hypothesized that time of day of surgery would influence the incidence of anesthetic adverse events (AEs). Methods: Clinical observations reported in a quality improvement database were categorized into different AEs that reflected (1) error, (2) harm, and (3) other AEs (error or harm could not be determined) and were analyzed for effects related to start hour of care. Results: As expected, there were differences in the rate of AEs depending on start hour of care. Compared with a reference start hour of 7 am, other AEs were more frequent for cases starting during the 3 pm and 4 pm hours (p<0.0001). Post hoc inspection of data revealed that the predicted probability increased from a low of 1.0% at 9 am to a high of 4.2% at 4 pm. The two most common event types (pain management and postoperative nausea and vomiting) may be primary determinants of these effects. Conclusions: Our results indicate that clinical outcomes may be different for patients anesthetized at the end of the work day compared with the beginning of the day. Although this may result from patient related factors, medical care delivery factors such as case load, fatigue, and care transitions may also be influencing the rate of anesthetic AEs for cases that start in the late afternoon.

Adaptation of a Postoperative Handoff Communication Process for Children With Heart Disease A Quantitative Study

Handoff communication is a point of vulnerability when valuable patient information can be inaccurate or omitted. An institutional protocol was implemented in 2005 to improve the handoff from the operating room to the intensive care unit after pediatric cardiac surgery. A cross-sectional study of the present process was performed to understand how users adapt a communication intervention over time. Twenty-nine handoff events were observed. Individuals required for the handoff were present at 97% of the events. Content items averaged a 53% reporting rate. Some clinical information not specified in the protocol demonstrated a higher reporting rate, such as echocardiogram results (68%) and vascular access (79%). A mean of 2.3 environmental distractions per minute of communication were noted. Participant-directed adjustments in content reporting suggest that a facilitator in process improvement is user-centered innovation. Future handoff communication interventions should reduce nonessential distractions and incorporate a discussion of the anticipated patient course.

Effects of automation of information-processing functions on teamwork.

We investigated the effects of automation as applied to different stages of information processing on team performance in a complex decision-making task. Forty teams of 2 individuals performed a simulated Theater Defense Task. Four automation conditions were simulated with computer assistance applied to realistic combinations of information acquisition, information analysis, and decision selection functions across two levels of task difficulty. Multiple measures of team effectiveness and team coordination were used. Results indicated different forms of automation have different effects on teamwork. Compared with a baseline condition, an increase in automation of information acquisition led to an increase in the ratio of information transferred to information requested; an increase in automation of information analysis resulted in higher team coordination ratings; and automation of decision selection led to better team effectiveness under low levels of task difficulty but at the cost of higher workload. The results support the use of early and intermediate forms of automation related to acquisition and analysis of information in the design of team tasks. Decision-making automation may provide benefits in more limited contexts. Applications of this research include the design and evaluation of automation in team environments.

Investigation of multi-modal interface features for adaptive automation of a human-robot system

The objective of this research was to assess the effectiveness of using a multi-modal interface for adaptive automation (AA) of human control of a simulated telerobotic (remote-control, semi-autonomous robotic) system. We investigated the use of one or more sensory channels to cue dynamic control allocations to a human operator or computer, as part of AA, and to support operator system/situation awareness (SA) and performance. It was expected that complex auditory and visual cueing through system interfaces might address previously observed SA decrements due to unannounced or unexpected automation-state changes as part of adaptive system control. AA of the telerobot was based on a predetermined schedule of manual- and supervisory-control allocations occurring when operator workload changes were expected due to the stages of a teleoperation task. The task involved simulated underwater mine disposal and 32 participants were exposed to four types of cueing of task-phase and automation-state changes including icons, earcons, bi-modal (combined) cues and no cues at all. Fully automated control of the telerobot combined with human monitoring produced superior performance compared to completely manual system control and AA. Cueing, in general, led to better performance than none, but did not appear to completely eliminate temporary SA deficits due to changes in control and associated operator reorienting. Bi-modal cueing of dynamic automation-state changes was more supportive of SA than modal (single sensory channel) cueing. The use of icons and earcons appeared to produce no additional perceived workload in comparison no cueing. The results of this research may serve as an applicable guide for the design of human-computer interfaces for real telerobotic systems, including those used for military tactical operations, which support operator achievement and maintenance of SA and promote performance in using AA.

Comparison of Performance Effects of Adaptive Automation Applied to Various Stages of Human-Machine System Information Processing

Limitations in automation (expert system) capabilities and negative human performance consequences of automation in complex systems have led to the contention that use of computer assistance in high-level human-machine system information processing may be inappropriate. Adaptive automation (AA) has been explored as a solution to these problems; however, research has focused on the performance effects of dynamic control allocations of early sensory and information acquisition functions between human operators and computer controllers of complex systems. It has examined to a limited extent the human performance and workload effects of AA of cognitive tasks, such as decision-making, or of psychomotor functions such as response execution. This research compared the affects of AA applied to psychomotor tasks and cognitive tasks, including information monitoring, information analysis, decision-making, and action implementation, on overall human-machine system performance. Results demonstrated that operators are better able to adapt to AA when applied to lower level functions, such as information acquisition and action implementation, as compared to AA of information analysis and decision making tasks. The results also provided support for the use of AA, as compared to completely manual control.