Daniel Schulman

MAINTAINING ENGAGEMENT IN LONG-TERM INTERVENTIONS WITH RELATIONAL AGENTS

We discuss issues in designing virtual humans for applications that require long-term voluntary use and the problem of maintaining engagement with users over time. Concepts and theories related to engagement from a variety of disciplines are reviewed. We describe a platform for conducting studies into long-term interactions between humans and virtual agents and present the results of two longitudinal, randomized, controlled experiments in which the effect of manipulations of agent behavior on user engagement was assessed.

A reusable framework for health counseling dialogue systems based on a behavioral medicine ontology

Automated approaches to promoting health behavior change, such as exercise, diet, and medication adherence promotion, have the potential for significant positive impact on society. We describe a theory-driven computational model of dialogue that simulates a human health counselor who is helping his or her clients to change via a series of conversations over time. Applications built using this model can be used to change the health behavior of patients and consumers at low cost over a wide range of media including the web and the phone. The model is implemented using an OWL ontology of health behavior change concepts and a public standard task modeling language (ANSI/CEA-2018). We demonstrate the power of modeling dialogue using an ontology and task model by showing how an exercise promotion system developed in the framework was re-purposed for diet promotion with 98% reuse of the abstract models. Evaluations of these two systems are presented, demonstrating high levels of fidelity to best practices in health behavior change counseling.

Empathic Touch by Relational Agents

We describe a series of experiments with an agent designed to model human conversational touch-capable of physically touching users in synchrony with speech and other nonverbal communicative behavior-and its use in expressing empathy to users in distress. The agent is composed of an animated human face that is displayed on a monitor affixed to the top of a human mannequin, with touch conveyed by an air bladder that squeezes a users hand. We demonstrate that when touch is used alone, hand squeeze pressure and number of squeezes are associated with user perceptions of affect arousal conveyed by an agent, while number of squeezes and squeeze duration are associated with affect valence. We also show that, when affect-relevant cues are present in facial display, speech prosody, and touch used simultaneously by the agent, facial display dominates user perceptions of affect valence, and facial display and prosody are associated with affect arousal, while touch had little effect. Finally, we show that when touch is used in the context of an empathic, comforting interaction (but without the manipulation of affect cues in other modalities), it can lead to better perceptions of relationship with the agent, but only for users who are comfortable being touched by other people.

Automated interventions for multiple health behaviors using conversational agents

OBJECTIVE An automated health counselor agent was designed to promote both physical activity and fruit and vegetable consumption through a series of simulated conversations with users on their home computers. METHODS The agent was evaluated in a 4-arm randomized trial of a two-month daily contact intervention comparing: (a) physical activity; (b) fruit and vegetable consumption; (c) both interventions; and (d) a non-intervention control. Physical activity was assessed using daily pedometer steps. Daily servings of fruit and vegetables were assessed using the NIH/NCI self-report Fruit and Vegetable Scan. RESULTS Participants in the physical activity intervention increased their walking on average compared to the control group, while those in the fruit and vegetable intervention and combined intervention decreased walking. Participants in the fruit and vegetable intervention group consumed significantly more servings per day compared to those in the control group, and those in the combined intervention reported consuming more compared to those in the control group. CONCLUSION Automated health intervention software designed for efficient re-use is effective at changing health behavior. PRACTICE IMPLICATIONS Automated health behavior change interventions can be designed to facilitate translation and adaptation across multiple behaviors.

Practical approaches to comforting users with relational agents

Interactions in which computer agents comfort users through expressed empathy have been shown to be important in alleviating user frustration and increasing user liking of the agent, and may have important healthcare applications. Given the current state of technology, designers of these systems are forced to choose between (a) allowing users to freely express their feelings, but having the agents provide imperfect empathic responses, or (b) greatly restricting how users can express themselves, but having the agents provide very accurate empathic feedback. This study investigates which of these options leads to better outcomes, in terms of comforting users and increasing user-agent social bonds. Results, on almost all measures, indicate that empathic accuracy is more important than user expressivity.

DTask and LiteBody: Open Source, Standards-Based Tools for Building Web-Deployed Embodied Conversational Agents

Two tools for developing embodied conversational agents and deploying them over the world-wide web to standard web browsers are presented. DTask is a hierarchical task decomposition-based dialogue planner, based on the CEA-2018 task description language standard. LiteBody is an extensible, web-based BML renderer that runs in most contemporary web browsers with no additional software and provides a conversational virtual agent with a range of conversational nonverbal behavior adequate for many user-agent interaction applications. Together, these tools provide a complete platform for deploying web-based conversational agents, and are actively being used on two health counseling applications.

Tinker: a relational agent museum guide

A virtual museum guide agent that uses human relationship-building behaviors to engage museum visitors is described. The computer animated agent, named “Tinker”, uses nonverbal conversational behavior, empathy, social dialogue, reciprocal self-disclosure and other relational behavior to establish social bonds with users, and encourage continued interaction and repeated visits. Tinker describes exhibits in the museum, gives directions, and discusses technical aspects of her own implementation. Tinker also recognizes returning visitors through biometric analysis of their hand shapes and dialogue cues. Results from two experiments using Tinker are described. In the first, 29 returning visitors are randomized to interact with the agent with the biometric identification turned on or off. In the second experiment, 1,607 visitors are randomized to interact with versions of Tinker that have relationship-building behavior turned on or off. Results indicate that the use of relational behavior leads to significantly greater engagement by museum visitors, measured by session length, number of sessions, and self-reported attitude, as well as learning gains, as measured by a knowledge test, compared to the same agent that does not use relational behavior. Implications for museum exhibits and intelligent tutoring systems are discussed.

The comforting presence of relational agents

In this paper we describe an on-going experiment on the calming effects of a relational agent on users following a social bonding interaction. Applications to a range of health care problems are discussed.

Public displays of affect: deploying relational agents in public spaces

Design principles for deploying agents designed for social and relational interactions with users in public spaces are discussed. These principles are applied to the development of a virtual science museum guide agent that uses human relationship-building behaviors to engage visitors. The agent appears in the form of a human-sized anthropomorphic robot, and uses nonverbal conversational behavior, empathy, social dialogue, reciprocal self-disclosure and other relational behavior to establish social bonds with users. The agent also uses a biometric identification system so that it can re-identify visitors it has already talked to. Results from a preliminary study indicate that most users enjoy the conversational and relational interaction with the agent.

Reducing Preconception Risks Among African American Women with Conversational Agent Technology

Background: Systems and tools are needed to identify and mitigate preconception health (PCH) risks, particularly for African American (AA) women, given persistent health disparities. We developed and tested “Gabby,” an online preconception conversational agent system. Methods: One hundred nongravid AA women 18–34 years of age were screened for over 100 PCH risks and randomized to the Gabby or control group. The Gabby group interacted with the system for up to six months; the control group received a letter indicating their health risks with a recommendation to talk with their clinician. The numbers, proportions, and types of risks were compared between groups. Results: There were 23.7 (SD 5.9) risks identified per participant. Eighty-five percent (77 of 91) provided 6 month follow up data. The Gabby group had greater reductions in the number (8.3 vs. 5.5 risks, P < .05) and the proportion (27.8% vs 20.5%, P < 0.01) of risks compared to controls. The Gabby group averaged 63.7 minutes of interaction time. Seventy-eight percent reported that it “was easy to talk to Gabby” and 64% used information from Gabby to improve their health. Conclusion: Gabby was significantly associated with preconception risk reduction. More research is needed to determine if Gabby can benefit higher risk populations and if risk reduction is clinically significant.