Brent Rossen

A crowdsourcing method to develop virtual human conversational agents

Educators in medicine, psychology, and the military want to provide their students with interpersonal skills practice. Virtual humans offer structured learning of interview skills, can facilitate learning about unusual conditions, and are always available. However, the creation of virtual humans with the ability to understand and respond to natural language requires costly engineering by conversation knowledge engineers (generally computer scientists), and incurs logistical cost for acquiring domain knowledge from domain experts (educators). We address these problems using a novel crowdsourcing method entitled Human-centered Distributed Conversational Modeling. This method facilitates collaborative development of virtual humans by two groups of end-users: domain experts (educators) and domain novices (students). We implemented this method in a web-based authoring tool called Virtual People Factory. Using Virtual People Factory, medical and pharmacy educators are now creating natural language virtual patient interactions on their own. This article presents the theoretical background for Human-centered Distributed Conversational Modeling, the implementation of the Virtual People Factory authoring tool, and five case studies showing that Human-centered Distributed Conversational Modeling has addressed the logistical cost for acquiring knowledge.

High score!: motivation strategies for user participation in virtual human development

Conversational modeling requires an extended time commitment, and the difficulty associated with capturing the wide range of conversational stimuli necessitates extended user participation. We propose the use of leaderboards, narratives and deadlines as motivation strategies to persuade user participation in the conversational modeling for virtual humans. We evaluate the applicability of leaderboards, narratives and deadlines through a user study conducted with medical students (n=20) for modeling the conversational corpus of a virtual patient character. Leaderboards, narratives and deadlines were observed to be effective in improving user participation. Incorporating these strategies had the additional effect of making user responses less reflective of real world conversations.

Human-Centered Distributed Conversational Modeling: Efficient Modeling of Robust Virtual Human Conversations

Currently, applications that focus on providing conversations with virtual humans require extensive work to create robust conversational models. We present a new approach called Human-centered Distributed Conversational Modeling. Using this approach, users create conversational models in a distributed manner. To do this, end-users interact with virtual humans to provide new stimuli (questions and statements), and domain-specific experts (e.g. medical/psychology educators) provide new virtual human responses. Using this process, users become the primary developers of conversational models. We tested our approach by creating an example application, Virtual People Factory. Using Virtual People Factory, a pharmacy instructor and 186 pharmacy students were able to create a robust conversational model in 15 hours. This is approximately 10% of the time typical in current approaches and results in more comprehensive coverage of the conversational space. In addition, surveys demonstrate the acceptability of this approach by both educators and students.

Interactive Virtual-Patient Scenarios: An Evolving Tool in Psychiatric Education

Standardized patients (SPs) are routinely used to teach medical students communication and physical diagnosis skills (1–5). Standardization and decreased risk of possible harm being committed by training students (6) are only a few of the advantages provided by SPs. SPs can provide a detailed history, can answer directly to the students, and give feedback for improvement (1, 6, 7). However, SP interactions are limited by the availability of the actors, time constraints, and costs (8). An alternative to SPs is use of computer simulation. Virtual-patient (VP) systems are computer programs that simulate real-life clinical scenarios in which the learner can complete a patient interview and physical exam, while making diagnostic and therapeutic decisions (9). VP systems allow standardized instruction, immediate and objective performance feedback, and unlimited opportunity for repetitive practice (10). VPs and computer simulations are already used in various branches of medicine to teach communication, counseling, crisis management, procedures, leadership, teamwork, and medical decision-making skills (11). Virtual reality is already used as exposure therapy to treat panic disorders, posttraumatic stress disorder, (12) social phobias, and specific phobias (13, 14). In 1966, ELIZA (15) became the first virtual psychiatric tool by engaging the user in a text-based psychotherapy session. It was never used as an actual treatment protocol, but it did set the stage for using computer simulation as a tool in diagnosing and treating mental illnesses (13). However, even with the foundation laid, there are only a few published applications of a psychiatric VP scenario used as a training modality in medical student education (8, 16). We present the preliminary evaluation of a web-based VP scenario that assesses students’ ability to elicit symptoms of major depression. The study was performed using the online application called Virtual People Factory (VPF; http://www. virtualpeoplefactory.com), (17) developed by the University of Florida’s Virtual Experiences Research Group. Currently, many VP systems entail the learners’ going through an evolving scenario where they sequentially answer multiple-choice questions about history, diagnosis, and management of patients. VPF is a web-based, instantmessaging application that elicits a VP’s medical history, based on the questions a user asks, therefore allowing students to make their own assessments of the most appropriate next step in the interview, rather than passively choosing from a list of possible answers. The process of creating a VPF interaction entails the following steps: Initially, an editor creates a script of the medical scenario, including what can be potentially said to the VP (questions/statements) and what the VP will respond back (speeches). Next, the scenario is sent out to users. Users log in to the system, are presented with a brief introduction of the patient similar to the introduction given to students before they enter the room to interview an SP, and then ushered into the chat interface. Once inside the interface, the user can engage the patient in conversation to obtain a full medical history. Every user’s input will yield a speech response. If there is not a speech response affiliated with a question or if the speech response is incorrect for a given question, the event is logged for the editor, to either create a new speech response or connect it correctly with the right question. Numerous user interactions will therefore allow for a robust database of the various questions and speech responses to be built for any medical scenario. Received June 17, 2010; revised November 22, 2010; accepted January 29, 2011. From Georgia Health Sciences University (HS, DL, SDL, AF), University of Florida (BR, BL), and Drexel University College of Medicine (SDL). Send correspondence to Dr. Foster; afoster@ georgiahealth.edu (e-mail). Copyright

Using virtual humans to bootstrap the creation of other virtual humans

Virtual human (VH) experiences are increasingly used for training interpersonal skills such as military leadership, classroom education, and doctor-patient interviews. These diverse applications of conversational VHs have a common and unexplored thread - a significant additional population would be afforded interpersonal skills training if VHs were available to simulate either interaction partner. We propose a computer-assisted approach to generate a virtual medical student from hundreds of interactions between a virtual patient and real medical students. This virtual medical student is then used to train standardized patients - human actors who roleplay the part of patients in practice doctor-patient encounters. Practice with a virtual medical student is expected to lead to greater standardization of roleplay encounters, and more accurate evaluation of medical student competency. We discuss the method for generating VHs from an existing corpus of human-VH interactions and present observations from a pilot experiment to determine the utility of the virtual medical student for training.

Constructionism of virtual humans to improve perceptions of conversational partners

We propose a methodology to help people improve the accuracy of their mental model of a conversational partner by creating a virtual human representation of the partner. By creating a virtual human, the users will be able to transfer their mental model of the partner to a virtual human representation. Other people can then interact with the virtual human and provide feedback. The feedback will help the creator reduce the gap between their mental model of a partner and the actual qualities of the partner. Reducing this gap in perception is important in learning interpersonal skills. We implemented this methodology in a health professions course using Virtual People Factory, an online application for creating and interacting with virtual humans. The applicability of the methodology to reduce gaps in perception models was investigated through a user study with health professions students (n=32). The results indicate that students can reduce gaps in perceptions of conversational partners by creating virtual humans.

Automated Generation of Emotive Virtual Humans

Emotive virtual humans (VHs) are important for affective interactions with embodied conversation agents [1]. However, emotive VHs require significant resources and time. As an example, the VHs in movies and video games require teams of animators and months of work. VHs can also be imbued with emotion using appraisal theory methods that use psychology based models to generate emotions by using the VHs goals and beliefs to evaluate external events. These external events require manual tagging or natural language understanding [2]. As an alternative approach, we propose tagging VH responses with emotions using textual affect sensing methods. The method developed by Neviarouskaya et al. [3] uses syntactic parses and a database of words and associated emotion intensities.We use this database, and because these emotions are associated with specific words, we can combine the emotions with audio timing information to generate lip-synched facial expressions. Our approach, AutoEmotion, allows us to automatically add basic emotions to VHs without the need for manual animation or tagging or natural language understanding.