Andrew Robb

Exploring agent physicality and social presence for medical team training

Mixed reality and 3D user interface technologies have increased the immersion, presence, and physicality of user interactions. These technologies can also increase the physicality of embodied conversational agents (ECAs) by making the ECAs occupy and interact with the physical space. We propose that increasing the physicality of an ECA can increase the ECAs social presence, that is, the feeling that the ECA is a real person. In this paper, we examine existing research and formalize the idea of ECA physicality. We also explored the relationship between physicality and social presence by conducting two user studies (n 18 and n 29). Both user studies took place in a medical team training context and involved virtual human ECAs as fellow team members. The first studys results suggested that increasing physicality increased social presence and elicited more realistic behavior. The second studys results suggested that individual dimensions of physicality affect social presence to different extents.

Leveraging Virtual Humans to Effectively Prepare Learners for Stressful Interpersonal Experiences

Stressful interpersonal experiences can be difficult to prepare for. Virtual humans may be leveraged to allow learners to safely gain exposure to stressful interpersonal experiences. In this paper we present a between-subjects study exploring how the presence of a virtual human affected learners while practicing a stressful interpersonal experience. Twenty-six fourth-year medical students practiced performing a prostate exam on a prostate exam simulator. Participants in the experimental condition examined a simulator augmented with a virtual human. Other participants examined a standard unaugmented simulator. Participants reactions were assessed using self-reported, behavioral, and physiological metrics. Participants who examined the virtual human experienced significantly more stress, measured via skin conductance. Participants stress was correlated with previous experience performing real prostate exams; participants who had performed more real prostate exams were more likely to experience stress while examining the virtual human. Participants who examined the virtual human showed signs of greater engagement; non-stressed participants performed better prostate exams while stressed participants treated the virtual human more realistically. Results indicated that stress evoked by virtual humans is linked to similar previous real-world stressful experiences, implying that learners real-world experience must be taken into account when using virtual humans to prepare them for stressful interpersonal experiences.

Virtual Role-Models: Using Virtual Humans to Train Best Communication Practices for Healthcare Teams

Due to logistical scheduling challenges, social training of conflict resolution skills with healthcare professionals is a difficult task. To overcome these challenges, we used virtual humans to fill in as surgical teammates and train conflict resolution skills in a surgical scenario. Surgical technologists were recruited at a United States teaching hospital to interact with a virtual nurse, virtual surgeon, and virtual anesthesiologist in a team training exercise. Leveraging social learning theory, the virtual nurse on the team modeled one of two conflict resolution strategies, either best practices or bad practices, during an important decision moment in the exercise. In a second important decision moment, we assessed if surgical technologists demonstrated the conflict resolution model they observed. We found human participants were successfully able to demonstrate the ideal conflict resolution strategy after observing the virtual nurse model best practices. While we found participants were positively influenced by the best practices model, we also found that conversely, the bad practices model negatively influenced participants’ conflict resolution behavior. If humans can be positively influenced by virtual humans, this form of social training could transform medical team training, empowering more healthcare professionals to speak up, and potentially decreasing the chances of patient morbidity or death in the OR.

Increasing agent physicality to raise social presence and elicit realistic behavior

The concepts of immersion and presence focus on the environment in a virtual environment. We instead focus on embodied conversational agents (ECAs). ECAs occupy the virtual environment as interactive partners. We propose that the ECA analogues of immersion and presence are physicality and social presence. We performed a study to determine the effect of an ECAs physicality on social presence and eliciting realistic behavior from the user. The results showed that increasing physicality can elicit realistic behavior and increase social presence but there was also an interaction effect with plausibility.

Using a Critical Incident Scenario With Virtual Humans to Assess Educational Needs of Nurses in a Postanesthesia Care Unit

Introduction: During critical incidents, teamwork failures can compromise patient safety. This study provides evidence that virtual humans can be used in simulated critical incidents to assess the learning needs of health professionals, and provide important information that can inform the development of continuing education programs in patient safety. We explored the effectiveness of information transfer during a devolving medical situation between postanesthesia care unit (PACU) nurses and a virtual attending physician. Methods: We designed a three‐stage scenario: tutorial, patient transfer, and critical incident. We developed 2 checklists to assess information transfer: Critical Patient Information and Interprofessional Communication Skills. All participants were videotaped; 2 raters reviewed all videos and assessed performance using the checklists. Results: Participants (n = 43) who completed all 3 stages scored 62.3% correct on critical patient information transfer and 61.6% correct on interprofessional communication skills. Almost 87% missed a fatal drug error. The checklists measured each item on a 1/0 (done/not) calculation. Additionally, no relationship was found between years of nursing experience and performance on either checklist. Discussion: The PACU nurses in this study did not consistently share critical information with an attending (virtual) physician during a critical incident, and most missed a fatal dosage error. These findings strongly suggest a crucial need for additional structured team training among practicing health care teams, and they demonstrate the utility of using virtual humans to simulate team members.

A Qualitative Evaluation of Behavior during Conflict with an Authoritative Virtual Human

This research explores the extent to which humans behave realistically during conflict with a virtual human occupying a position of authority. To this end, we created a virtual team to train nurses how to manage conflict in the operating room; the team’s virtual surgeon engages in reckless behavior that could endanger the safety of the team’s patient, requiring nurses to intervene and correct the virtual surgeon’s behavior. Results from post-hoc behavioral analysis and semi-structured interviews indicate that participants behaved realistically during conflict, as compared against existing behavioral frameworks. However, some participants reported perceiving their virtual teammates as strangers, which they felt may have caused them to behave differently than they would with their normal teammates.

Training Together: how another human Trainee's Presence affects Behavior during Virtual human- Based Team Training

Despite research showing that team training can lead to strong improvements in team performance, logistical difficulties can prevent team training programs from being adopted on a large scale. A proposed solution to these difficulties is the use of virtual humans to replace missing teammates. Existing research evaluating the use of virtual humans for team training has been conducted in settings involving a single human trainee. However, in the real world multiple human trainees would most likely train together. In this paper, we explore how the presence of a second human trainee can alter behavior during a medical team training program. Ninety-two nurses and surgical technicians participated in a medical training exercise, where they worked with a virtual surgeon and virtual anesthesiologist to prepare a simulated patient for surgery. The agency of the nurse and the surgical technician were varied between three conditions: human nurses and surgical technicians working together; human nurses working with a virtual surgical technician; and human surgical technicians working with a virtual nurse. Variations in agency did not produce statistically significant differences in the training outcomes, but several notable differences were observed in other aspects of the teams behavior. Specifically, when working with a virtual nurse, human surgical technicians were more likely to assist with speaking up about patient safety issues that were outside of their normal responsibilities; human trainees spent less time searching for a missing item when working with a virtual partner, likely because the virtual partner was physically unable to move throughout the room and assist with the searching process; and more breaks in presence were observed when two human teammates were present. These results show that some behaviors may be influenced by the presence of multiple human trainees, though these behaviors may not impinge on core training goals. When developing virtual human-based training programs, designers should consider that the presence of other humans may reduce involvement during training moments perceived to be the responsibility of other trainees, and should consider that a virtual teammates limitations may cause human teammates to limit their own behaviors in corresponding ways (e.g. searching less).

Guidelines on Successfully Porting Non-Immersive Games to Virtual Reality: A Case Study in Minecraft

Virtual reality games have grown rapidly in popularity since the first consumer VR head-mounted displays were released in 2016, however comparatively little research has explored how this new medium impacts the experience of players. In this paper, we present a study exploring how user experience changes when playing Minecraft on the desktop and in immersive virtual reality. Fourteen players completed six 45 minute sessions, three played on the desktop and three in VR. The Gaming Experience Questionnaire, the i-Group presence questionnaire, and the Simulator Sickness Questionnaire were administered after each session, and players were interviewed at the end of the experiment. Participants strongly preferred playing Minecraft in VR, despite frustrations with using teleporation as a travel technique and feelings of simulator sickness. Players enjoyed using motion controls, but still continued to use indirect input under certain circumstances. This did not appear to negatively impact feelings of presence. We conclude with four lessons for game developers interested in porting their games to virtual reality.