Celso M. de Melo

Expression of Emotions Using Wrinkles, Blushing, Sweating and Tears

Wrinkles, blushing, sweating and tears are physiological manifestations of emotions in humans. Therefore, the simulation of these phenomena is important for the goal of building believable virtual humans which interact naturally and effectively with humans. This paper describes a real-time model for the simulation of wrinkles, blushing, sweating and tears. A study is also conducted to assess the influence of the model on the perception of surprise, sadness, anger, shame, pride and fear. The study follows a repeated-measures design where subjects compare how well is each emotion expressed by virtual humans with or without these phenomena. The results reveal a significant positive effect on the perception of surprise, sadness, anger, shame and fear. The relevance of these results is discussed for the fields of virtual humans and expression of emotions.

Expression of Moral Emotions in Cooperating Agents

Moral emotions have been argued to play a central role in the emergence of cooperation in human-human interactions. This work describes an experiment which tests whether this insight carries to virtual human-human interactions. In particular, the paper describes a repeated-measures experiment where subjects play the iterated prisoners dilemma with two versions of the virtual human: (a) neutral, which is the control condition; (b) moral, which is identical to the control condition except that the virtual human expresses gratitude, distress, remorse, reproach and anger through the face according to the action history of the game. Our results indicate that subjects cooperate more with the virtual human in the moral condition and that they perceive it to be more human-like. We discuss the relevance these results have for building agents which are successful in cooperating with humans.

The influence of emotions in embodied agents on human decision-making

Acknowledging the social functions that emotions serve, there has been growing interest in the interpersonal effect of emotion in human decision making. Following the paradigm of experimental games from social psychology and experimental economics, we explore the interpersonal effect of emotions expressed by embodied agents on human decision making. The paper describes an experiment where participants play the iterated prisoners dilemma against two different agents that play the same strategy (tit-for-tat), but communicate different goal orientations (cooperative vs. individualistic) through their patterns of facial displays. The results show that participants are sensitive to differences in the facial displays and cooperate significantly more with the cooperative agent. The data indicate that emotions in agents can influence human decision making and that the nature of the emotion, as opposed to mere presence, is crucial for these effects. We discuss the implications of the results for designing human-computer interfaces and understanding human-human interaction.

The impact of emotion displays in embodied agents on emergence of cooperation with people

Acknowledging the social functions of emotion in people, there has been growing interest in the interpersonal effect of emotion on cooperation in social dilemmas. This paper explores whether and how facial displays of emotion in embodied agents impact cooperation with human users. The paper describes an experiment where participants play the iterated prisoners dilemma against two different agents that play the same strategy (tit-for-tat), but communicate different goal orientations (cooperative vs. individualistic) through their patterns of facial displays. The results show that participants are sensitive to differences in the displays of emotion and cooperate significantly more with the cooperative agent. The results also reveal that cooperation rates are only significantly different when people play first with the individualistic agent. This is in line with the well-known black-hat/white-hat effect from the negotiation literature. However, this study emphasizes that people can discern a cooperator (white-hat) from a noncooperator (black-hat) based only on emotion displays. We propose that people are able to identify the cooperator by inferring, from the emotion displays, the agents goals. We refer to this as reverse appraisal, as it reverses the usual process in which appraising relevant events with respect to ones goals leads to specific emotion displays. We discuss implications for designing human–computer interfaces and understanding human–human interaction.

Humans versus Computers: Impact of Emotion Expressions on People's Decision Making

Recent research in perception and theory of mind reveals that people show different behavior and lower activation of brain regions associated with mentalizing (i.e., the inference of others mental states) when engaged in decision making with computers, when compared to humans. These findings are important for affective computing because they suggest peoples decisions might be influenced differently according to whether they believe emotional expressions shown in computers are being generated by algorithms or humans. To test this, we had people engage in a social dilemma (Experiment 1) or negotiation (Experiment 2) with virtual humans that were either perceived to be agents (i.e., controlled by computers) or avatars (i.e., controlled by humans). The results showed that such perceptions have a deep impact on peoples decisions: in Experiment 1, people cooperated more with virtual humans that showed cooperative facial displays (e.g., joy after mutual cooperation) than competitive displays (e.g., joy when the participant was exploited) but, the effect was stronger with avatars (d = .601) than with agents (d = .360); in Experiment 2, people conceded more to angry than neutral virtual humans but, again, the effect was much stronger with avatars (d = 1.162) than with agents (d = .066). Participants also showed less anger towards avatars and formed more positive impressions of avatars when compared to agents.

INFLUENCE OF AUTONOMIC SIGNALS ON PERCEPTION OF EMOTIONS IN EMBODIED AGENTS

Specific patterns of autonomic activity have been reported when people experience emotions. Typical autonomic signals that change with emotion are wrinkles, blushing, sweating, tearing, and respiration. This article explores whether these signals can also influence the perception of emotion in embodied agents. The article first reviews the literature on specific autonomic signal patterns associated with certain affective states. Next, it proceeds to describe a real-time model for wrinkles, blushing, sweating, tearing, and respiration that is capable of implementing those patterns. Two studies are then described. In the first, subjects compare surprise, sadness, anger, shame, pride, and fear expressed in an agent with or without blushing, wrinkles, sweating, or tears. In the second, subjects compare excitement, relaxation, focus, pain, relief, boredom, anger, fear, panic, disgust, surprise, startle, sadness, and joy expressed in an agent with or without typical respiration patterns. The first study shows a statistically significant positive effect on perception of surprise, sadness, anger, shame, and fear. The second study shows a statistically significant positive effect on perception of excitement, pain, relief, boredom, anger, fear, panic, disgust, and startle. The relevance of these results to artificial intelligence and intelligent virtual agents is discussed.

The Effect of Agency on the Impact of Emotion Expressions on People's Decision Making

Recent research in neuroeconomics reveals that people show different behavior and lower activation of brain regions associated with mentalizing (i.e., the inference of others mental states) when engaged in decision making tasks with a computer, when compared to a human. These findings are important for affective computing because they suggest peoples decision making might be influenced differently according to whether they believe the emotional expressions shown by a computer are being generated by a computer algorithm or a human. To test this, we had people engage in a social dilemma (Experiment 1) or a negotiation (Experiment 2) with virtual humans that were either agents (i.e., controlled by computers) or avatars (i.e., controlled by humans). The results show a clear agency effect: in Experiment 1, people cooperated more with virtual humans that showed facial cooperative displays (e.g., joy after mutual cooperation) rather than competitive displays (e.g., joy when the participant was exploited) but, the effect was only significant with avatars, in Experiment 2, people conceded more to an angry than a neutral virtual human but, once again, the effect was only significant with avatars.

Real-time expression of affect through respiration

Realistic character animation requires elaborate rigging built on top of high quality 3D models. Sophisticated anatomically based rigs are often the choice of visual effect studios where life-like animation of CG characters is the primary objective. However, rigging a character with a muscular-skeletal system is very involving and time-consuming process, even for professionals. Although, there have been recent research efforts to automate either all or some parts of the rigging process, the complexity of anatomically based rigging nonetheless opens up new research challenges. We propose a new method to automate anatomically based rigging that transfers an existing rig of one character to another. The method is based on a data interpolation in the surface and volume domain, where various rigging elements can be transferred between different models. As it only requires a small number of corresponding input feature points, users can produce highly detailed rigs for a variety of desired character with ease. Copyright

People Do Not Feel Guilty About Exploiting Machines

Guilt and envy play an important role in social interaction. Guilt occurs when individuals cause harm to others or break social norms. Envy occurs when individuals compare themselves unfavorably to others and desire to benefit from the others’ advantage. In both cases, these emotions motivate people to act and change the status quo: following guilt, people try to make amends for the perceived transgression, and following envy, people try to harm envied others. In this article, we present two experiments that study participants’ experience of guilt and envy when engaging in social decision making with machines and humans. The results showed that, though experiencing the same level of envy, people felt considerably less guilt with machines than with humans. These effects occurred both with subjective and behavioral measures of guilt and envy, and in three different economic games: public goods, ultimatum, and dictator game. This poses an important challenge for human-computer interaction because, as shown here, it leads people to systematically exploit machines, when compared to humans. We discuss theoretical and practical implications for the design of human-machine interaction systems that hope to achieve the kind of efficiency -- cooperation, fairness, reciprocity, etc. -- we see in human-human interaction.

Affective engagement to emotional facial expressions of embodied social agents in a decision-making game

Previous research illustrates that people can be influenced by the emotional displays of computer‐generated agents. What is less clear is if these influences arise from cognitive or affective process (i.e., do people use agent displays as information or do they provoke user emotions). To unpack these processes, we examine the decisions and physiological reactions of participants (heart rate and electrodermal activity) when engaged in a decision task (prisoners dilemma game) with emotionally expressive agents. Our results replicate findings that peoples decisions are influenced by such emotional displays, but these influences differ depending on the extent to which these displays provoke an affective response. Specifically, we show that an individual difference known as electrodermal lability predicts the extent to whether people will engage affectively or strategically with such agents, thereby better predicting their decisions. We discuss implications for designing agent facial expressions to enhance social interaction between humans and agents. Copyright