Gale M. Lucas

Automatic Audiovisual Behavior Descriptors for Psychological Disorder Analysis

Abstract We investigate the capabilities of automatic audiovisual nonverbal behavior descriptors to identify indicators of psychological disorders such as depression, anxiety, and post-traumatic stress disorder. Due to strong correlations between these disordersas measured with standard self-assessment questionnaires in this study, we focus our investigations in particular on a generic distress measure as identified using factor analysis. Within this work, we seek to confirm and enrich present state of the art, predominantly based on qualitative manual annotations, with automatic quantitative behavior descriptors. We propose a number of nonverbal behavior descriptors that can be automatically estimated from audiovisual signals. Such automatic behavior descriptors could be used to support healthcare providers with quantified and objective observations that could ultimately improve clinical assessment. We evaluate our work on the dataset called the Distress Assessment Interview Corpus (DAIC) which comprises dyadic interactions between a confederate interviewer and a paid participant. Our evaluation on this dataset shows correlation of our automatic behavior descriptors with the derived general distress measure. Our analysis also includes a deeper study of self-adaptor and fidgeting behaviors based on detailed annotations of where these behaviors occur.

Perceived Support for Promotion-Focused and Prevention-Focused Goals

Perceived emotional support from close relationship partners in times of stress is a major predictor of well-being. However, recent research has suggested that, beyond emotional support, perceived support for achieving personal goals is also important for well-being. The present study extends such research by demonstrating that associations of perceived goal support with well-being differ depending on how people represent their goals and the general motivational context in which they pursue these goals. Among unmarried romantic partners, for whom the context of the relationship presumably is largely attainment oriented, perceived support for attainment-relevant (or promotion-focused) goals independently predicted relationship and personal well-being, whereas perceived support for maintenance-relevant (or prevention-focused) goals did not. In contrast, among married partners, for whom the context of the relationship presumably is both attainment and maintenance oriented, perceived support for both promotion-focused and prevention-focused goals independently predicted well-being. We discuss the implications for forecasting and improving well-being among married couples.

Self-Reported Symptoms of Depression and PTSD Are Associated with Reduced Vowel Space in Screening Interviews

Reduced frequency range in vowel production is a well documented speech characteristic of individuals with psychological and neurological disorders. Affective disorders such as depression and post-traumatic stress disorder (PTSD) are known to influence motor control and in particular speech production. The assessment and documentation of reduced vowel space and reduced expressivity often either rely on subjective assessments or on analysis of speech under constrained laboratory conditions (e.g. sustained vowel production, reading tasks). These constraints render the analysis of such measures expensive and impractical. Within this work, we investigate an automatic unsupervised machine learning based approach to assess a speakers vowel space. Our experiments are based on recordings of 253 individuals. Symptoms of depression and PTSD are assessed using standard self-assessment questionnaires and their cut-off scores. The experiments show a significantly reduced vowel space in subjects that scored positively on the questionnaires. We show the measures statistical robustness against varying demographics of individuals and articulation rate. The reduced vowel space for subjects with symptoms of depression can be explained by the common condition of psychomotor retardation influencing articulation and motor control. These findings could potentially support treatment of affective disorders, like depression and PTSD in the future.

Negotiation as a Challenge Problem for Virtual Humans

We argue for the importance of negotiation as a challenge problem for virtual human research, and introduce a virtual conversational agent that allows people to practice a wide range of negotiation skills. We describe the multi-issue bargaining task, which has become a de facto standard for teaching and research on negotiation in both the social and computer sciences. This task is popular as it allows scientists or instructors to create a variety of distinct situations that arise in real-life negotiations, simply by manipulating a small number of mathematical parameters. We describe the development of a virtual human that will allow students to practice the interpersonal skills they need to recognize and navigate these situations. An evaluation of an early wizard-controlled version of the system demonstrates the promise of this technology for teaching negotiation and supporting scientific research on social intelligence.

Opponent Modeling for Virtual Human Negotiators

Negotiation is a challenging domain for virtual human research. One aspect of this problem, known as opponent modeling, is discovering what the other party wants from the negotiation. Research in automated negotiation has yielded a number opponent modeling techniques but we show that these methods do not easily transfer to human-agent settings. We propose a more effective heuristic for inferring preferences both from a negotiator’s pattern of offers and verbal statements about their preferences. This method has the added advantage that it can detect negotiators that lie about their preferences. We discuss several ways the method can enhance the capabilities of a virtual human negotiator.

The Benefits of Virtual Humans for Teaching Negotiation

This article examines the potential for teaching negotiation with virtual humans. Many people find negotiations to be aversive. We conjecture that students may be more comfortable practicing negotiation skills with an agent than with another person. We test this using the Conflict Resolution Agent, a semi-automated virtual human that negotiates with people via natural language. In a between-participants design, we independently manipulated two pedagogically-relevant factors while participants engaged in repeated negotiations with the agent: perceived agency (participants either believed they were negotiating with a computer program or another person) and pedagogical feedback (participants received instructional advice or no advice between negotiations). Findings indicate that novice negotiators were more comfortable negotiating with a computer program (they self-reported more comfort and punished their opponent less often) and expended more effort on the exercise following instructional feedback (both in time spent and in self-reported effort). These findings lend support to the notion of using virtual humans to teach interpersonal skills.

Changing Me to Keep You: State Jealousy Promotes Perceiving Similarity Between the Self and a Romantic Rival

Individuals sometimes alter their self-views to be more similar to others—traditionally romantic partners—because they are motivated to do so. A common motivating force is the desire to affiliate with a partner. The current research examined whether a different motivation—romantic jealousy—might promote individuals to alter their self-views to be more similar to a romantic rival, rather than a partner. Romantic jealousy occurs when individuals perceive a rival as a threat to their relationship and motivates individuals to defend their relationship. We proposed that one novel way that individuals might defend their relationship is by seeing themselves as more similar to a perceived romantic rival. We predicted individuals would alter their self-views to be more similar to a rival that they believed their partner found attractive. Importantly, we predicted that state romantic jealousy would motivate these self-alterations. Three studies confirmed these hypotheses.

The effect of operating a virtual doppleganger in a 3D simulation

Recent advances in scanning technology have enabled the widespread capture of 3D character models based on human subjects. Intuition suggests that, with these new capabilities to create avatars that look like their users, every player should have his or her own avatar to play video games or simulations. We explicitly test the impact of having ones own avatar (vs. a yoked control avatar) in a simulation (i.e., maze running task with mines). We test the impact of avatar identity on both subjective (e.g., feeling connected and engaged, liking avatars appearance, feeling upset when avatars injured, enjoying the game) and behavioral variables (e.g., time to complete task, speed, number of mines triggered, riskiness of maze path chosen). Results indicate that having an avatar that looks like the user improves their subjective experience, but there is no significant effect on how users perform in the simulation.

Choking Under Social Pressure Social Monitoring Among the Lonely

Lonely individuals may decode social cues well but have difficulty putting such skills to use precisely when they need them—in social situations. In four studies, we examined whether lonely people choke under social pressure by asking participants to complete social sensitivity tasks framed as diagnostic of social skills or nonsocial skills. Across studies, lonely participants performed worse than nonlonely participants on social sensitivity tasks framed as tests of social aptitude, but they performed just as well or better than the nonlonely when the same tasks were framed as tests of academic aptitude. Mediational analyses in Study 3 and misattribution effects in Study 4 indicate that anxiety plays an important role in this choking effect. This research suggests that lonely individuals may not need to acquire social skills to escape loneliness; instead, they must learn to cope with performance anxiety in interpersonal interactions.

Trust me: multimodal signals of trustworthiness

This paper builds on prior psychological studies that identify signals of trustworthiness between two human negotiators. Unlike prior work, the current work tracks such signals automatically and fuses them into computational models that predict trustworthiness. To achieve this goal, we apply automatic trackers to recordings of human dyads negotiating in a multi-issue bargaining task. We identify behavioral indicators in different modalities (facial expressions, gestures, gaze, and conversational features) that are predictive of trustworthiness. We predict both objective trustworthiness (i.e., are they honest) and perceived trustworthiness (i.e., do they seem honest to their interaction partner). Our experiments show that people are poor judges of objective trustworthiness (i.e., objective and perceived trustworthiness are predicted by different indicators), and that multimodal approaches better predict objective trustworthiness, whereas people overly rely on facial expressions when judging the honesty of their partner. Moreover, domain knowledge (from the literature and prior analysis of behaviors) facilitates the model development process.