Johannes Martinus Cornelis Schraagen

Systematic review of serious games for medical education and surgical skills training

The application of digital games for training medical professionals is on the rise. So‐called ‘serious’ games form training tools that provide a challenging simulated environment, ideal for future surgical training. Ultimately, serious games are directed at reducing medical error and subsequent healthcare costs. The aim was to review current serious games for training medical professionals and to evaluate the validity testing of such games.

A prospective study of paediatric cardiac surgical microsystems: assessing the relationships between non-routine events, teamwork and patient outcomes

Objective Paediatric cardiac surgery has a low error tolerance and demands high levels of cognitive and technical performance. Growing evidence suggests that further improvements in patient outcomes depend on system factors, in particular, effective team skills. The hypotheses that small intraoperative non-routine events (NREs) can escalate to more serious situations and that effective teamwork can prevent the development of serious situations were examined to develop a method to assess these skills and to provide evidence for improvements in training and performance. Methods This mixed-method design, using both quantitative and qualitative measures, relied on trained human factor observers who observed and coded NREs and teamwork elements from the time of patient arrival into the operating room to patient handover to the intensive care unit. Real-time teamwork observations were coupled with microsystem preparedness measures, operative duration, assessed difficulty of the operation and patient outcome measures. Behaviour was rated based on whether it hindered or enhanced teamwork. Results 40 paediatric cardiac surgery cases were observed. Surgeons displayed better teamwork during complicated procedures, particularly during the surgical bypass/repair epoch. More procedural NREs were associated with a more complicated postoperative course (Muncomplicated=9.08; Mminor complications=11.11; Mmajor morbidity=14.60, F(2,26)=3.46, p<0.05). Procedural NREs decreased substantially over time (M1=13.5; M2=7.1, F(1,37)=33.07, p<0.001). Conclusions Structured observation of effective teamwork in the operating room can identify substantive deficiencies in the system and conduct of procedures, even in otherwise successful operations. High performing teams are more resilient displaying effective teamwork when operations become more difficult.

Factual accuracy and trust in information: The role of expertise

In the past few decades, the task of judging the credibility of information has shifted from trained professionals (e.g., editors) to end users of information (e.g., casual Internet users). Lacking training in this task, it is highly relevant to research the behavior of these end users. In this article, we propose a new model of trust in information, in which trust judgments are dependent on three user characteristics: source experience, domain expertise, and information skills. Applying any of these three characteristics leads to different features of the information being used in trust judgments; namely source, semantic, and surface features (hence, the name 3S-model). An online experiment was performed to validate the 3S-model. In this experiment, Wikipedia articles of varying accuracy (semantic feature) were presented to Internet users. Trust judgments of domain experts on these articles were largely influenced by accuracy whereas trust judgments of novices remained mostly unchanged. Moreover, despite the influence of accuracy, the percentage of trusting participants, both experts and novices, was high in all conditions. Along with the rationales provided for such trust judgments, the outcome of the experiment largely supports the 3S-model, which can serve as a framework for future research on trust in information.

How conversations change over time in face-to-face and video-mediated communication.

An experiment was conducted to examine how communication patterns and task performance differ as a function of the groups communication environment and how these processes change over time. In a longitudinal design, three-person groups had to select and argue the correct answer out of a set of three alternatives for ten questions. Compared with face-to-face groups, video-teleconferencing groups took fewer turns, required more time for turns, and interrupted each other less. Listeners appeared to be more polite, waiting for a speaker to finish before making their conversational contribution. Although groups were able to maintain comparable performance scores across communication conditions, initial differences between conditions in communication patterns disappeared over time, indicating that the video-teleconferencing groups adapted to the newness and limitations of their communication environment. Moreover, because of increased experience with the task and the group, groups in both conditions needed less conversation to complete the task at later rounds. Implications are discussed for practice, training, and possibilities for future research.

Assessing and improving teamwork in cardiac surgery

Objective Cardiac surgery (PCS) has a low error tolerance, is dependent upon sophisticated organisational structures and demands high levels of cognitive and technical performance. The aim of the study was to assess the role of intraoperative non-routine events (NREs) and team performance on paediatric cardiac surgery outcomes. The current paper focuses on improving methods for studying teamwork; a companion paper will report on the empirical results. Methods The authors trained human factors observers to observe and code the NREs and teamwork from time of arrival of the patient into the operating room (OR) to the patient handover in the intensive care unit. The observers underwent immersive training in which each observer attended 10 operations, learnt in detail about the technical procedures and clinical tasks and received practice in coding teamwork. Two observers were used interchangeably to observe OR teamwork. The authors instigated a rigorous training and assessment protocol, with independent assessment of their performance by both senior medical and human factors experts using video-based assessment. Real-time teamwork observations were supplemented with process mapping, questionnaires on safety culture, level of preparedness by the team, difficulty of the operation and outcome measures. Results 19 PCS cases were observed. The observers observed a total of 255 hr of operations, including the first 10 training cases. We found that 68% of events were observed by one observer but only 32% of all events were observed by both observers. If an event was coded by both observers, 76% was coded in the same way, resulting in high levels of inter-rater agreement. The inter rater reliability of the four main teamwork categories was 91% with Cohen kappa of 0.77. Recommendations were developed for observing teamwork in the operating room, for instance ‘train observers on video recordings of real operations (not scripted performance), preferably of at least 1–2 h in duration’ and ‘Rate teamwork in real time and not afterwards.’ Conclusions PCS is an ideal model to explore team performance. A challenge for the future is to make observations of teamwork in healthcare settings more efficient and robust.

Trust in wikipedia: how users trust information from an unknown source

The use of Wikipedia as an information source is becoming increasingly popular. Several studies have shown that its information quality is high. Normally, when considering information trust, the source of information is an important factor. However, because of the open-source nature of Wikipedia articles, their sources remain mostly unknown. This means that other features need to be used to assess the trustworthiness of the articles. We describe article features - such as images and references - which lay Wikipedia readers use to estimate trustworthiness. The quality and the topics of the articles are manipulated in an experiment to reproduce the varying quality on Wikipedia and the familiarity of the readers with the topics. We show that the three most important features are textual features, references and images.

Topic familiarity and information skills in online credibility evaluation

With the rise of user-generated content, evaluating the credibility of information has become increasingly important. It is already known that various user characteristics influence the way credibility evaluation is performed. Domain experts on the topic at hand primarily focus on semantic features of information e.g., factual accuracy, whereas novices focus more on surface features e.g., length of a text. In this study, we further explore two key influences on credibility evaluation: topic familiarity and information skills. Participants with varying expected levels of information skills i.e., high school students, undergraduates, and postgraduates evaluated Wikipedia articles of varying quality on familiar and unfamiliar topics while thinking aloud. When familiar with the topic, participants indeed focused primarily on semantic features of the information, whereas participants unfamiliar with the topic paid more attention to surface features. The utilization of surface features increased with information skills. Moreover, participants with better information skills calibrated their trust against the quality of the information, whereas trust of participants with poorer information skills did not. This study confirms the enabling character of domain expertise and information skills in credibility evaluation as predicted by the updated 3S-model of credibility evaluation.

Information Sharing During Crisis Management in Hierarchical vs. Network Teams

This study examines the differences between hierarchical and network teams in emergency management. A controlled experimental environment was created in which we could study teams that differed in decision rights, availability of information, information sharing, and task division. Thirty-two teams of either two (network) or three (hierarchy) participants (N=80 in total) received messages about an incident in a tunnel with high-ranking politicians possibly being present. Based on experimentally induced knowledge, teams had to decide as quickly and as accurately as possible what the likely cause of the incident was: an attack by Al Qaeda, by anti-globalists, or an accident. The results showed that network teams were overall faster and more accurate in difficult scenarios than hierarchical teams. Network teams also shared more knowledge in the difficult scenarios, compared with the easier scenarios. The advantage of being able to share information that is inherent in network teams is thus contingent upon the type of situation encountered.

Propensity to trust and the influence of source and medium cues in credibility evaluation

Credibility evaluation has become a daily task in the current world of online information that varies in quality. The way this task is performed has been a topic of research for some time now. In this study, we aim to extend this research by proposing an integrated layer model of trust. According to this model, trust in information is influenced by trust in its source. Moreover, source trust is influenced by trust in the medium, which in turn is influenced by a more general propensity to trust. We provide an initial validation of the proposed model by means of an online quasi-experiment (n = 152) in which participants rated the credibility of Wikipedia articles. Additionally, the results suggest that the participants were more likely to have too little trust in Wikipedia than too much trust.

Improving decision making in crisis response through critical thinking support

In this study, we describe how to use innovative techniques to improve the decision-making process in crisis response organizations. The focus was on building situation awareness of a crisis and overcoming pitfalls such as tunnel vision and information bias through using critical thinking. We started by observing typical difficulties in crisis management in a field study. The essential elements of concern were a deficit in sharing and communicating understanding and a patchy overview of the topics communicated, within as well as between teams. Communication frequently did not entail the reasoning behind a decision that was made. We therefore developed a critical thinking tool that made the reasoning process more explicit and at the same time more robust by tying it to specific hypotheses. We studied a candidate support tool in a controlled setting and found that people made better judgments, particularly in situations where they would be prone to decision biases. We subsequently extended the critical thinking tool to a team setting. We list a number of requirements that are essential for support systems that intend to limit tunnel vision and alleviate communication and coordination problems in crisis response organizations.