Jonas Rybing

Studying distributed cognition of simulation-based team training with DiCoT

Abstract Health care organizations employ simulation-based team training (SBTT) to improve skill, communication and coordination in a broad range of critical care contexts. Quantitative approaches, such as team performance measurements, are predominantly used to measure SBTTs effectiveness. However, a practical evaluation method that examines how this approach supports cognition and teamwork is missing. We have applied Distributed Cognition for Teamwork (DiCoT), a method for analysing cognition and collaboration aspects of work settings, with the purpose of assessing the methodology’s usefulness for evaluating SBTTs. In a case study, we observed and analysed four Emergo Train System® simulation exercises where medical professionals trained emergency response routines. The study suggests that DiCoT is an applicable and learnable tool for determining key distributed cognition attributes of SBTTs that are of importance for the simulation validity of training environments. Moreover, we discuss and exemplify how DiCoT supports design of SBTTs with a focus on transfer and validity characteristics. Practitioner Summary: In this study, we have evaluated a method to assess simulation-based team training environments from a cognitive ergonomics perspective. Using a case study, we analysed Distributed Cognition for Teamwork (DiCoT) by applying it to the Emergo Train System®. We conclude that DiCoT is useful for SBTT evaluation and simulator (re)design.

Designing a Digital Medical Management Training Simulator Using Distributed Cognition Theory

Background. Training of medical professionals is important to improve care during mass-causality events. Therefore, it is essential to extend knowledge on how to design valid and usable simulation-based training environments. Purpose. This article investigates how distributed cognition and simulation theory concepts can guide design of simulation-based training environments. We present the design and user evaluation of DigEmergo, a simulator for training and assessing emergency medicine management. Design approach. A prior Distributed Cognition in Teamwork (DiCoT) analysis of the Emergo Train System (ETS) guided the design process. The design objective of DigEmergo was to be useful, usable, retain distributed cognition features of ETS, and strengthen validity and output reliability. Evaluation. Eight expert ETS instructors participated in a formative system evaluation. The Technology Assessment Model (TAM) questionnaire was used to measure usefulness and ease of use. Observations and post-test interviews were conducted to contextualize the measures. Results. The results showed that DigEmergo was perceived as somewhat to quite useful and somewhat easy to use. Overall, expert users considered DigEmergo promising and successful in retaining core ETS features. Conclusions. The study indicates that a design methodology based on distributed cognition and simulation theory can be successfully combined to guide simulator (re)design and strengthen simulator validity.

Exploring prehospital C2-work during a mass gathering event

Purpose– The purpose of this paper is to explore the workload and shared workload awareness in a staff performing command and control (C2) work during a planned major incident (MI) empirical case in Sweden. Design/methodology/approach– Data on workload and shared awareness were collected during live C2-work using qualitative observations and in-situ interviews mixed with quantitative questionnaires. Findings– A content analysis of the qualitative data revealed categories of workload sources. Quantified workload estimates showed changes in workload levels over time and staff roles, which were also contextualized using the results of the qualitative data. Data on shared awareness indicated that team workload awareness shifted over time according to common patterns. This study demonstrates a promising methodology to study C2-related factors during live EMS work. Research limitations/implications– The observed variations in workload imply that research that relies only on post-task measurements of workload may be inaccurate. Future research could use this method to investigate the connection between workload and performance during different types of MIs. Originality/value– The results can be used to inform future Goteborgsvarvet C2-teams in terms of when, why, and for whom task load changes, which would support predictive allocation of resources.

Workload Differences Across Command Levels and Emergency Response Organizations During a Major Joint Training Exercise

OBJECTIVE This study reports on an initial test using a validated workload measurement method, the NASA Task Load Index (TLX), as an indicator of joint emergency exercise effectiveness. Prior research on emergency exercises indicates that exercises must be challenging, ie, result in high workload, to be effective. However, this is often problematic with some participants being underloaded and some overloaded. The NASA TLX was used to test for differences in workload between commanders and subordinates and among three different emergency response organizations during a joint emergency exercise. DESIGN Questionnaire-based evaluation with professional emergency responders. SETTING The study was performed in conjunction with a large-scale interorganizational joint emergency exercise in Sweden. PARTICIPANTS A total of 20 participants from the rescue services, 12 from the emergency medical services, and 12 from the police participated in the study (N=44). Ten participants had a command-level role during the exercise and the remaining 34 were subordinates. MAIN OUTCOME MEASURE(S) The main outcome measures were the workload subscales of the NASA TLX: mental demands, physical demands, temporal demands, performance, effort, and frustration. RESULTS The results showed that the organizations experienced different levels of workload, that the commanders experienced a higher workload than the subordinates, and that two out of three organizations fell below the twenty-fifth percentile of average workload scores compiled from 237 prior studies. CONCLUSIONS The results support the notion that the NASA TLX could be a useful complementary tool to evaluate exercise designs and outcomes. This should be further explored and verified in additional studies.

Evaluation of a Novel Method to Study Interorganizational Coordination in Medical Command and Control Centers

Study/Objective: Inter-organizational coordination is key to successful medical command and control (C2) during major incidents. However, evaluating this factor is often problematic, in particular during or after real emergencies as compared to controlled training scenarios. The purpose of this case study was to pilot test a non-intrusive data collection method for evaluating operative inter-organizational coordination during medical C2 situations.Background: This study was conducted during a planned major incident in Sweden. The major incident studied was the Goteborgsvarvet half-marathon, the largest half-marathon event in the world with more than 200 000 attending spectators and over 60 000 runners. The studied C2 center included representatives from local hospitals, Goteborgsvarvet organization, police, fire department, ambulance service, the local traffic and infrastructure management office, and emergency dispatch.Methods: A combination of qualitative and quantitative methods was employed in this study. The qualitative methods included an ethnographic field study with on-site observations and contextual inquiry interviews. The quantitative methods included validated and experimental questionnaires distributed to the command center personnel at pre-determined intervals. These questionnaires aimed at gathering data on workload, stress, and shared and individual situational awareness.Results: The data indicate that the qualitative methods were less intrusive than the quantitative methods. The observations and contextual inquiries could be performed without interruptions while periods of high workload resulted in lower or delayed response rates on the questionnaires. Simple questionnaires produced an 80% response rate, complex questionnaires only 40%.Conclusion: The employed method appears useful to evaluate inter-organizational coordination and showed potential to gather meaningful data without being intrusive or disturbing the operative C2 activities. Due to the time-sensitive nature of emergency C2-work, unobtrusive qualitative methods and short, easy to fill out questionnaires are recommended for future studies. The results from this pilot will inform future operative C2 studies during similar planned major incidents.

Evaluating, Learning and Simulation Exercise for Efficacy, A Course on Advanced Prehospital Trauma Care

Study/Objective: In this study, we aimed to design a questionnaire battery for course and simulation exercise evaluation, and pilot-test the battery by evaluating a course on Advanced Prehospital T ...

An exploratory study of a low-level shared awareness measure using mission-critical locations during an emergency exercise

A shared awareness of other teams’ roles and tasks has been linked to successful performance in joint ventures. However, emergency management organizations responding to incidents do not always share critical information necessary for maintaining shared awareness. An instrument called Shared Priorities has previously been applied to measure aspects of shared situation awareness at level 2 and 3 in Endsley’s (1995) model. This paper reports on a shared awareness instrument focused on level 1 situation awareness and its associated level of team shared awareness. Participants in a large emergency response exercise were asked to locate and rank geographical locations based on importance for overall mission success. The results show that organizations tended to rank locations relevant for their own work higher than positions relevant to other organization’s tasks. The different organizations displayed different levels of inter-rater agreement within themselves concerning the ranking of these positions.