Benjamin Lok

The validity of a virtual human experience for interpersonal skills education

Any new tool introduced for education needs to be validated. We developed a virtual human experience called the Virtual Objective Structured Clinical Examination (VOSCE). In the VOSCE, a medical student examines a life-size virtual human who is presenting symptoms of an illness. The student is then graded on interview skills. As part of a medical school class requirement, thirty three second year medical students participated in a user study designed to determine the validity of the VOSCE for testing interview skills. In the study, participant performance in the VOSCE is compared to participant performance in the OSCE, an interview with a trained actor. There was a significant correlation (r(33)=.49, p<.005) between overall score in the VOSCE and overall score in the OSCE. This means that the interaction skills used with a virtual human translate to the interaction skills used with a real human. Comparing the experience of virtual human interaction to real human interaction is the critical validation step towards using virtual humans for interpersonal skills education.

Comparing Interpersonal Interactions with a Virtual Human to Those with a Real Human

This paper provides key insights into the construction and evaluation of interpersonal simulators¿systems that enable interpersonal interaction with virtual humans. Using an interpersonal simulator, two studies were conducted that compare interactions with a virtual human to interactions with a similar real human. The specific interpersonal scenario employed was that of a medical interview. Medical students interacted with either a virtual human simulating appendicitis or a real human pretending to have the same symptoms. In Study I (n = 24), medical students elicited the same information from the virtual and real human, indicating that the content of the virtual and real interactions were similar. However, participants appeared less engaged and insincere with the virtual human. These behavioral differences likely stemmed from the virtual humans limited expressive behavior. Study II (n = 58) explored participant behavior using new measures. Nonverbal behavior appeared to communicate lower interest and a poorer attitude toward the virtual human. Some subjective measures of participant behavior yielded contradictory results, highlighting the need for objective, physically-based measures in future studies.

Collaboration in tele-immersive environments

This paper describes a study of remote collaboration between people in a shared virtual environment. Seventeen subjects were recruited at University College London, who worked with a confederate at University of North Carolina Chapel Hill. Each pair was required to negotiate the task of handling an object together, and moving a few metres into a building. The DIVE system was used throughout, and the network support was Internet-2. This was an observational study to examine the extent to which such collaboration was possible, to explore the limitations of DIVE within this context, and to examine the relationship between several variables such as co-presence and task performance. The results suggest that although the task is possible under this framework, it could only be achieved by various software tricks within the DIVE framework. A new Virtual Environment system is required that has better knowledge of network performance, and that supports shared object manipulation across a network. The participant-study suggests that co-presence, the sense of being together with another person, was significantly and positively correlated with task performance.

Online model reconstruction for interactive virtual environments

We present a system for generating real-time 3D reconstructions of the user and other real objects in an immersive virtual environment (IVE) for visualization and interaction. For example, when parts of the users body are in his field of view, our system allows him to see a visually faithful graphical representation of himself, an avatar. In addition, the user can grab real objects, and then see and interact with those objects in the IVE. Our system bypasses an explicit 3D modeling stage, and does not use additional tracking sensors or prior object knowledge, nor do we generate dense 3D representations of objects using computer vision techniques. We use a set of outside-looking-in cameras and a novel visual hull technique that leverages the tremendous recent advances in graphics hardware performance and capabilities. We accelerate the visual hull computation by using projected textures to rapidly determine which volume samples lie within the visual hull. The samples are combined to form the object reconstruction from any given viewpoint. Our system produces results at interactive rates, and because it harnesses ever-improving graphics hardware, the rates and quality should continue to improve. We further examine realtime generated models as active participants in simulations (with lighting) in IVEs, and give results using synthetic and real data. Additional

The use of virtual patients in medical school curricula

The demonstration of patient-based cases using automated technology [virtual patients (VPs)] has been available to health science educators for a number of decades. Despite the promise of VPs as an easily accessible and moldable platform, their widespread acceptance and integration into medical curricula have been slow. Here, the authors review the technological underpinnings of VPs, summarize the literature regarding the use and limitations of VPs in the healthcare curriculum, describe novel possible applications of the technology, and propose possible directions for future work.

Virtual Human + Tangible Interface = Mixed Reality Human An Initial Exploration with a Virtual Breast Exam Patient

Virtual human (VH) experiences are receiving increased attention for training real-world interpersonal scenarios. Communication in interpersonal scenarios consists of not only speech and gestures, but also relies heavily on haptic interaction - interpersonal touch. By adding haptic interaction to VH experiences, the bandwidth of human-VH communication can be increased to approach that of human-human communication. To afford haptic interaction, a new species of embodied agent is proposed - mixed reality humans (MRHs). A MRH is a virtual human embodied by a tangible interface that shares the same registered space. The tangible interface affords the haptic interaction that is critical to effective simulation of interpersonal scenarios. We applied MRHs to simulate a virtual patient requiring a breast cancer screening (medical interview and physical exam). The design of the MRH patient is presented. This paper also presents the results of a pilot study in which eight (n = 8) physician-assistant students performed a clinical breast exam on the MRH patient. Results show that when afforded haptic interaction with a MRH patient, users demonstrated interpersonal touch and social engagement similarly to interacting with a human patient.

A crowdsourcing method to develop virtual human conversational agents

Educators in medicine, psychology, and the military want to provide their students with interpersonal skills practice. Virtual humans offer structured learning of interview skills, can facilitate learning about unusual conditions, and are always available. However, the creation of virtual humans with the ability to understand and respond to natural language requires costly engineering by conversation knowledge engineers (generally computer scientists), and incurs logistical cost for acquiring domain knowledge from domain experts (educators). We address these problems using a novel crowdsourcing method entitled Human-centered Distributed Conversational Modeling. This method facilitates collaborative development of virtual humans by two groups of end-users: domain experts (educators) and domain novices (students). We implemented this method in a web-based authoring tool called Virtual People Factory. Using Virtual People Factory, medical and pharmacy educators are now creating natural language virtual patient interactions on their own. This article presents the theoretical background for Human-centered Distributed Conversational Modeling, the implementation of the Virtual People Factory authoring tool, and five case studies showing that Human-centered Distributed Conversational Modeling has addressed the logistical cost for acquiring knowledge.

A Mixed Reality Approach for Merging Abstract and Concrete Knowledge

Mixed realitys (MR) ability to merge real and virtual spaces is applied to merging different knowledge types, such as abstract and concrete knowledge. To evaluate whether the merging of knowledge types can benefit learning, MR was applied to an interesting problem in anesthesia machine education. The virtual anesthesia machine (VAM) is an interactive, abstract 2D transparent reality simulation of the internal components and invisible gas flows of an anesthesia machine. It is widely used in anesthesia education. However when presented with an anesthesia machine, some students have difficulty transferring abstract VAM knowledge to the concrete real device. This paper presents the augmented anesthesia machine (AAM). The AAM applies a magic-lens approach to combine the VAM simulation and a real anesthesia machine. The AAM allows students to interact with the real anesthesia machine while visualizing how these interactions affect the internal components and invisible gas flows in the real world context. To evaluate the AAMs learning benefits, a user study was conducted. Twenty participants were divided into either the VAM (abstract only) or AAM (concrete+abstract) conditions. The results of the study show that MR can help users bridge their abstract and concrete knowledge, thereby improving their knowledge transfer into real world domains.

High score!: motivation strategies for user participation in virtual human development

Conversational modeling requires an extended time commitment, and the difficulty associated with capturing the wide range of conversational stimuli necessitates extended user participation. We propose the use of leaderboards, narratives and deadlines as motivation strategies to persuade user participation in the conversational modeling for virtual humans. We evaluate the applicability of leaderboards, narratives and deadlines through a user study conducted with medical students (n=20) for modeling the conversational corpus of a virtual patient character. Leaderboards, narratives and deadlines were observed to be effective in improving user participation. Incorporating these strategies had the additional effect of making user responses less reflective of real world conversations.

Evolving an immersive medical communication skills trainer

This paper presents our experiences in evolving the Virtual Objective Structured Clinical Exam (VOSCE) system. This system allows medical students to experience the interaction between a patient and a medical doctor using natural methods of interaction with a high level of immersion. These features enable the system to provide training on medical communication skills. We discuss the experiences of a group of medical and physician assistant students that pilot tested the system. Further, we examine the impact of evolving the system based on their feedback. The VOSCE systems performance in subsequent studies has indicated that end-user feedback improvements have significantly impacted overall performance and efficacy.