Improving Humanness of Virtual Agents and Users' Cooperation through Emotions
IImproving Humanness of Virtual Agents and Users’ Cooperationthrough Emotions
Moojan Ghafurian
School of Computer ScienceUniversity of WaterlooWaterloo, Ontario, N2L [email protected]
Neil Budnarain
School of Computer ScienceUniversity of WaterlooWaterloo, Ontario, N2L [email protected]
Jesse Hoey
School of Computer ScienceUniversity of WaterlooWaterloo, Ontario, N2L [email protected]
ABSTRACT
In this paper, we analyze the performance of an agent developed ac-cording to a well-accepted appraisal theory of human emotion withrespect to how it modulates play in the context of a social dilemma.We ask if the agent will be capable of generating interactions thatare considered to be more human than machine-like. We conductan experiment with 117 participants and show how participantsrate our agent on dimensions of human-uniqueness (which sepa-rates humans from animals) and human-nature (which separateshumans from machines). We show that our appraisal theoreticagent is perceived to be more human-like than baseline models, bysignificantly improving both human-nature and human-uniquenessaspects of the intelligent agent. We also show that perception ofhumanness positively affects enjoyment and cooperation in thesocial dilemma.
KEYWORDS
Affective computing; Human likeness, OCC, prisoner’s dilemma
In this paper, we investigate a theory of emotion as the methodfor generating artificially intelligent agents that seem more human-like. We argue that much Artificial Intelligence (AI) research hasfocused on building intelligence based on individual attributes thatnon-human animals do not possess, but that machines inherentlydo possess.However, human intelligence also requires emotional attributesand social support, attributes that machines do not possess, whereasnon-human animals do.The concept of “human-ness” has seen much debate in socialpsychology, particularly in relation to work on stereotypes and de-humanization or infra-humanisation [9]. Since Mori wrote aboutthe “uncanny valley” [31], artificial intelligence researchers (particu-larly in the fields of affective computing and human-computer/robotinteraction) have shown an interest in this issue. In a comprehen-sive series of experiments, Haslam et al. [17] examined how peoplejudge others as human or non-human (dehumanized). In theirmodel, humanness is broken down into two factors. First,
HumanUniqueness (HU) distinguishes humans from animals (but not nec-essarily from machines). Second,
Human Nature (HN) distinguisheshumans from machines (but not necessarily from animals). Hu-man uniqueness traits are civility, refinement, moral sensibility,rationality and maturity, as opposed to lack of culture, coarseness,amorality, irrationality and childlikeness. Human nature traits areemotionality, warmth, openness, agency (individuality), and depth, as opposed to inertness, coldness, rigidity, passivity and superfi-ciality. Thus, while one can imagine both humans and machineshaving human uniqueness traits, animals would not tend to havethese (they are coarse, amoral, etc). Similarly, while one can imag-ine both humans and animals having human nature traits, machineswould not tend to have these (they are inert, cold, rigid, etc). Whilemuch research in AI is trying to build machines with HU traits (thusseparating AI from animals), there is much less work on trying tobuild machines with HN traits (thus separating AI from machines).While both problems present challenges, the former problem isalready “solved” to a certain degree by simply having a machinein the first place, as humans and machines are on the same sideof the human uniqueness divide anyway. The latter problem ismore challenging as the human nature dimension is exactly thedimension on which machines differ most from humans.Several studies have verified that humanness and emotions ofvirtual agents can affect people’s behaviour and strategies [3, 6]. Forexample, Chowanda et al. [3] captured players’ emotions throughtheir facial expressions and showed that Non-Player Charactersthat have personalities and are capable of perceiving emotions canenhance players’ experience in the game. Further, Nonverbal be-haviour such as body gesture and gaze direction affects perceptionof cooperativeness of an agent [43].In this paper, we use an appraisal-based emotional model in thesame spirit as EMA [16], where emotional displays are made usingthe Ortony, Clore and Collins (OCC) model [33], and a set of copingrules are implemented to map the game history augmented withemotional appraisals to actions for the virtual agent. We refer tothe agent based on this model as the
OCC agent.The
OCC agent uses emotions to generate expectations aboutfuture actions [13, 49]. That is, it sees emotions as being relatedto an agent’s assessments of what is going to happen next, bothwithin and without the agent’s control. The
OCC agent computesexpectations with respect to the denotative meaning (or causalinterpretation [16]) of the situation, and these expectations aremapped to emotion labels. The generated emotions are then usedwith a set of coping rules to adjust future actions.We present results from a study involving N =
117 participantswho played a simple social dilemma game with a virtual agentnamed “Aria”. The game was a variation of Prisoner’s Dilemma(PD), in which each player could either give two coins or take onecoin from a common pile. Players could maximize their returns bydefecting while their partner cooperated, and although the Nashequilibrium is mutual defection, the players can jointly maximizetheir scores through mutual cooperation. The participants wereawarded a bonus according only to their total score in the game, a r X i v : . [ c s . A I] M a r nd so had incentive to cooperate as much as possible. Partici-pants played a series of 25 games in a row, and then answered aquestionnaire on how they felt about Aria on dimensions of human-uniqueness and human-nature taken from [17]. The participantswere evenly split into three conditions that differed only in theemotional displays. The virtual human, Aria, displayed facial ex-pressions and uttered canned sentences that were consistent withthe game context and the emotional state given the condition. Onecondition was based on the OCC model, while two were baselines,one with randomly selected emotions, and one with no emotions.Our hypothesis was that the
OCC agents would show more human-nature traits than the baselines.The primary contribution of this paper is to evaluate how ap-praised emotions relate to two important dimensions of humanness,and to investigate the impact of appropriate emotion modeling onperceived humanness of a virtual agent and users’ cooperation. Sec-ondary contributions are a complete description of the prisoner’sdilemma in terms of OCC emotions and a demonstration in a simpleenvironment.
Affective computing (AC) has formed as a sub-discipline of artifi-cial intelligence seeking to understand how human emotions canbe computationally modeled and implemented in virtual agents.While much current work is focused on social signal processing,the emphasis is on the detection, modeling and generation of sig-nals relating to social interactions without considering the controlmechanisms underlying the function of emotion [45]. In a broadsurvey [38], Reisenzein et al. define emotional functions as being informational , attentional , and motivational , but point to a lack ofexplicit mechanisms for computational implementation.Much of the work in AC on the function of emotions has fo-cused on appraisal theories of emotion, as these give clear rulesmapping denotative states to emotions and show a clear path forimplementation. The appraisal model of Ortony, Clore and Collins(OCC) [33] describes appraisals of the consequences of events andthe actions of agents in relation to self or other, and is perhapsthe most well used in AC. Scherer’s component process model [40]breaks emotion processes down into five components of appraisal,activation, expression, motivation, and feeling.Emotion is proposed as a facilitator of learning and as a mech-anism to signal, predict, and select forthcoming action, althoughno precise definitions of such a mechanism is proposed. Ortony,Norman and Revelle [34] describe a general cognitive architec-ture that incorporates affect at three levels of processing (reactive,routine and reflective) and four domains of processing (affect, moti-vation, cognition, and behaviour). The reactive level is relegated toa lower-level (e.g. hardware on a robot) process that encodes motorprograms and sensing mechanisms. They claim that appraisal andpersonality arise primarily at the routine level, but this is somewhatcontradictory since appraisals involve reasoning about long-termgoals (for example), something that is defined as being only at thereflective level in [34].Efforts at integrating appraisal models in artificial agents startedwith Elliott’s use of an OCC model augmented with “Love”, “Hate”and “Jealousy” to make predictions about human’s emotional rat-ings of semantically ambiguous storylines and to drive a virtual character [10]. This was followed by the work of Gratch [15, 16],and OCC models were integrated with probabilistic models for in-telligent tutoring applications in [4, 39]. A general-purpose gameengine for adding emotions is described in [37].The role of affect in decision making has long been problematic,and focus on bounded rationality following Simon’s articulationof emotions as interrupts to cognitive processing [41] has beenpursued by many authors [1, 23, 28]. Usually, these approachestake the stance that the agent is still acting on rational and deci-sion theoretic principles, but has a “modified” utility function [2],with some tuning parameter that trades off social normative ef-fects modeled as intrinsic rewards with the usual extrinsic rewards.The affect-as-cognitive-feedback approach [19] slightly rearrangesthings, and proposes that affect serves as a reward signal for thedefault cognitive processing mechanisms associated with the situa-tion. The BayesACT model uses the concepts of identity to frameaction based on affect alone, considering it to be a key componentof the social glue that enables collaboration [18].Expected and immediate emotions have been related to expectedutility and modify action choices accordingly [14, 25]. Other ap-proaches have attempted to reverse-engineer emotion through Re-inforcement Learning (RL) models that interpret the antecedentsof emotion as aspects of the learning and decision-making process,but relegate the function of emotion to characteristics of the RLproblem [30]. Many of these approaches borrow from behavioraleconomics and cognitive science to characterize the consequencesof emotion in decision making and integrate this knowledge as “cop-ing rules” [16], “affect heuristics” [42], or short-circuit “impulsivebehaviours” [11] to characterize or influence agents’ behaviour.Investigations into the role of emotions in modifying behavioursin a PD have looked at how disappointment and anger can be usedto promote forgiveness and retaliation, respectively [48]. Guilt hasalso been examined and human estimates of behaviour conditionedon facial expressions or written descriptions of this emotion havebeen shown to align with appraisal theoretic predictions [8]. Hu-man and bot play has been analyzed using BayesACT [21], whoshowed that BayesACT could replicate some aspects of human playin PD. Poncela et al. [36] demonstrated how humans split into dif-ferent player types when playing PD, which are loosely defined ontwo axes of optimism-pessimism and envy-trust.Finally, general personalities based on emotional factors of tol-erance and responsiveness, and the emotion of admiration wereconsidered in PD-like situations [24].Work on negotiations is based in similar theoretical ideas [22],where expressions of anger (or happiness) signal that an agent’sgoals are higher (or lower) than what the agent currently perceivesas the expected outcome.These findings about human-human negotiations are replicatedin a human-agent study, in which verbal and non-verbal displaysof anger and happiness are also compared [7].Finally, the same findings were partially replicated in real (un-scripted) human-human negotiations [44], where it was reportedthat negative displays increased individual gain, but led to worselonger-term outcomes. igure 1: An example of the game setting. The two coin pileson the left show the number of coins that the participant andAria have earned so far in the game. Currently, the playerhas chosen to give 2 and look ”happy” for the next round.Aria’s current emotion is ”sorry”. Our prisoner’s dilemma game setting was implemented using Phaser(open source HTML5 game framework) and is shown in Figure 1.The female Virtual Human was originally developed for speech andlanguage therapy [46] and has been also used in assistive technol-ogy applications [26]. Here, we call her Aria. In the game, Aria, thevirtual opponent, is sitting in front of the participants. On the right,there is a large pile of coins that the players use. On the left thereare two piles of coins, one pile representing the coins that Aria hasreceived so far and the other showing participants’ coins. In eachround, the players’ decisions are hidden from each other (hiddenbehind the red barriers). They can both choose between givingthe other player two coins, or taking one for themselves. Afterthe participants choose their actions, they will see the emoji list(and an emotion word describing each emoji) at the bottom of thepage. After deciding on the action, participants are asked to chooseone emoji based on their emotion. Upon selecting both action andemotion, the results will be revealed (coins will appear on the tableand will move to the players’ piles) and both players will see theother player’s emotion. Aria’s emotions are reflected through herfacial expressions and utterances, and players’ emotions are shownvia the emoji that they have selected.
We use a set of 20 emotions compiled from the OCC model (seeSection 4) and these are mapped to a three dimensional emotionspace with dimensions of Evaluation (E), Potency (P) and Activity(A) which is sometimes known as the VAD model where the termsare Valence (E), Activity (A) and Dominance (P) [35]. We use an emotion dictionary consisting of a set of 300 emotionwords rated on E,P and A by 1027 undergraduates at the Universityof Indiana in 2002-2004 [12]. We manually find the same or asynonymous word in the dictionary for each of the 20 OCC emotionwords. We also select a set of 20 emojis, one per OCC emotion word,to be used in the game play as described above.An important emotion in PD is regret. If a player defects, butregrets, it is quite different than when a player defects but showsno regret. In the following, we define regret as any of the fourOCC emotions ”remorse”, ”distress”, ”shame” or ”fears-confirmed”.Only this last term does not have a direct equivalent in the Indianadictionary, and for it we find the term ”heavy hearted” (E -1.03; P:-0.55; A: -1.15).
Aria’s facial expressions are generated with three controls thatmap to specific sets of facial muscles. We refer to this three di-mensional space of control as the “HSF”space: (1) Happy/Sad, (2)Surprise/Anger, and (3) Fear/Disgust. A setting of these three con-trols yields a specific facial expression by virtually moving the ac-tion units in the face corresponding to that emotion by an amountproportional to the control. For example, ”happy” is expressedwith AU6 + AU12: cheek raiser and lip corner puller. Althoughthe virtual human’s face can be controlled by moving individualmuscles, like the inner eyebrow raise, groups of these are highlycorrelated and move in recognizable patterns. Therefore, thesethree dimensions of musculature movement are deemed sufficient.To map from an emotion label from our set of 20, first the emotionword is mapped to EPA space using the Indiana dictionary, and thenthe distance to each end-point of the HSF space is computed usingthe EPA ratings shown below, also from the Indiana dictionary, andthese distances are used to set the HSF controls directly.
Emotion Symbol E,P,Ahappy / sad h + / h − (3.45,2.91,0.24) / (-2.38,-1.34,-1.88)surprise / anger h + / h − (1.48,1.32,2.31) / ( -2.03,1.07,1.80)fear / disgust h + / h − (-2.41,-0.76,-0.68) / (-2.57,0.27,0.43) Aria has a normal ”quiescent” state in which she blinks andslightly moves her head from side to side in a somewhat ran-dom way. The emotions are applied for 10.5 seconds and the facesmoothly transitions to a weaker representation of the same emo-tion and the quiescent state.
Aria also utters sentences from 8 predefined sets, one for eachcombination of agent action, human action, and binary indicatorof valence (E) being positive. Speaking and facial expressions arepossible at the same time. Lip movement during speech is based ona proprietary algorithm.An embedding (vector) for each emotion label is computed usingthe pretrained Word2Vec model which was trained on part of theGoogle News dataset where the model contains 3 million wordsand 300-dimensional vectors [29]. Phrases are embedded as fol-lows. Stop words are removed using the stop word list providedby the NLTK library for the English language. The embedding of aphrase is then simply computed by taking the mean of the Googleembeddings of all the words in a phrase. Given an emotion label,the closest phrase is queried by computing the cosine similarity dot product) of the vector representing the emotion label with allof the phrase vectors. According to the OCC model [34], emotions arise as a valencedreaction to the consequences of events, to the actions of agents,and to the aspects of objects. In our game situation, the aspects ofobjects (leading to the emotions of love and hate) do not change andso may influence overall mood but will not change substantiallyover the course of the interaction. We therefore focus on actionsand events only. Emotions in these categories are caused by theimmediate payoffs, or by payoffs looking into the future and past.Within each category, there are a number of further distinctions,such as whether focus is on the self or the other, and whether theevent is positive or negative. There are 20 emotions in the modelafter removing ”love” and ”hate”.
Table 1 gives the OCC interpretation of emotions in the Prisoner’sDilemma game. Each row gives the most recent move of bothagent (Aria) and player (human), and the momentary emotionsappraised on the consequences of events and on the actions of agents ,both of which are appraised for both self and other. The prospect-based consequences of events are evaluated on each subsequent turn,leading to emotions shown in the last two columns based only onthe player’s previous move. Emotional intensity is not modeled,but could be added to increase realism.Let us consider the immediate payoffs first. If Aria gets a payoffof 2 or more, she is pleased, leading to joy when considering conse-quences for self where prospects are irrelevant (the gains directlylead to joy). Alternatively, for payoffs less than 2, Aria is distressedby the loss. When considering her own actions, Aria is approving(and so feels pride) if she gives, because this seems an appropriateaction in hindsight. However, if Aria takes, she disapproves ofher own action because she has done something wrong, leading toshame. Unless if the player shows a negative emotion and gives 2,then Aria will approve of her action, because she predicts that it isnot going well anyway.When considering the consequences for the player, if he gets2 or more, then Aria estimates that it is desirable for him. If hegets 2, then Aria is pleased, and is happy-for him, otherwise she isdispleased (if he gets 3) and feels resentment. If the player gets 0and shows a negative emotion, Aria is pleased and gloats, but if apositive emotion is shown, Aria is displeased and feels pity. Simi-larly, if the player gives then Aria is approving and feels admiration,unless the player shows a negative emotion, in which case Aria isambivalent. Aria is also ambivalent if the player takes but showsregret, otherwise she disapproves and feels reproachful.Prospect-based emotions arise because Aria looks into the futureand predicts how things will evolve. If she gets 2 or more, she feelspleased and hope is elicited because she estimates things will gowell. However, if the player shows a negative emotion and Ariahas taken, then she feels fear because she predicts a reprisal. Ifshe gets less than 2 she feels fear about the future, however, if theplayer shows regret then she feels hopeful. Looking into the past,if the player’s action changed from the last time, Aria’s hopes andfears are disconfirmed, leading to the prospect-based emotion of disappointment. If the player’s action does not change, then Aria’shopes and fears are confirmed, leading to prospect-based emotionsof satisfaction (multiple give actions) or fears-confirmed (multipletake actions). At the start of the game (not shown in Table 1), Ariafeels hopeful because she is pleased about the prospect of positivepayoffs in the game.
Once an emotion is appraised, coping is used to figure out whataction to take.Five coping strategies are taken from [16]: acceptance, seekingsupport, restraint, growth, and denial, and these are applied asshown in Table 2. At the game’s start, Aria’s hope leads to the sup-port seeking coping mechanism, and thus to an initial cooperativeaction.
To assess how humanness of the OCC agent is perceived, we ran anexperiment on Mechanical Turk, where the participants played thePrisoner’s Dilemma against different agents with different strategiesand emotional displays. We then asked participants to evaluateeach agent on Human Nature and Human Uniqueness traits.
The experiment consisted of two parts: the Prisoner’s Dilemmagame and a questionnaire. Participants played 25 rounds of thegame against an agent, which was randomly assigned based on theexperimental condition. Afterwards, they filled out a questionnaireassessing how they perceived different aspects of humanness ofthe agent that they were assigned to. We ensured that the partici-pants would pay attention and try to maximize their outcome byproviding a bonus according to the points they earned in the game.The amount of the bonus was significantly larger than the initialpayment. Further, participants were told that they will play up to 30rounds because knowing the number of rounds can affect people’sstrategy in the final rounds.
The experiment had three between-participant conditions. In all conditions, participants played thesame number of game rounds against an opponent (Aria) and an-swered the same survey. However, the behavior and emotionaldisplays of the agent changed depending on the experimental con-dition. The conditions of the experiment were as follows:(1)
OCC:
Agent acts according to Table 2. Emotional displaysare selected randomly from the set defined in Table 1, andfacial expressions and utterances are applied as describedabove.(2)
Emotionless:
Agent plays tit-for-2-tats (cooperates im-mediately upon cooperation, but defects only after two de-fections), shows no emotional expressions in the face andsays nothing. This agent still shows quiescent behaviours.(3)
Random:
Agent plays tit-for-2-tats. Emotional displaysare randomly drawn from the set of 20 emotions, and facialexpressions and utterances are selected on the basis of that.The Emotionless agent is added to ensure that the participantsare paying attention to the emotions when rating the humannessof the agents. The Random condition enables us to study whether able 1: OCC-based emotional appraisals in the PD game. The “consequences” and “actions of agents” correspond to the OCCdecision tree. means “pleased”, U means approving. ♥ means desirable, and D means confirmed. Aria is ambivalent for alllines not shown. For example, in the case where Aria gives while the player takes but shows regret, Aria does not disapproveof the player’s action anymore (because he is showing regret), but does not actually approve of it either, so sits on the fenceand does not feel admiration or reproach. GAME PLAY VALENCED APPRAISALS APPRAISED EMOTIONS
Consequencesother self ActionsPrevious Most Recent prospects relevant? of agentsMove Emotion ♥ yes no Momentary Prospect-Basedfor ? self otherPlayer Aria Player Player ? other? future present D ? U ? U ? Single Compound Singlegive 2 give 2 give 2 any yes yes happy-foryes hopeyes yes satisfactionyes yes joy,pride gratificationyes yes joy,admiration gratitudetake 1 give 2 give 2 any yes yes happy-foryes hopeyes no reliefyes yes joy,pride gratificationyes yes joy,admiration gratitudegive 2 take 1 give 2 positive no no pityyes hopeyes yes satisfactionyes yes joy,admiration gratitudeno shametake 1 take 1 give 2 positive no no pityyes hopeyes yes reliefyes yes joy,admiration gratitudeno shamegive 2 take 1 give 2 negative yes no gloatingno fearyes yes satisfactionyes yes pride, joy gratificationtake 1 take 1 give 2 negative yes no gloatingno fearyes no reliefyes yes pride,joy gratificationgive 2 give 2 take 1 no regret no yes resentmentno fearno no disappointmentno no distress,reproach angeryes pridetake 1 give 2 take 1 no regret no yes resentmentyes fearno yes fears-confirmedno no distress,reproach angeryes pridegive 2 give 2 take 1 regret no yes resentmentyes hopeno no disappointmentno distressyes pridetake 1 give 2 take 1 regret no yes resentmentyes hopeno yes fears-confirmedno distressyes pridetake 1 take 1 take 1 any no no pityno fearno yes fears-confirmedno no distress,shame remorseno no distress,reproach angergive 2 take 1 take 1 any no no pityno fearno no disappointmentno no distress,shame remorseno no distress,reproach anger the participants pay attention to the differences in the emotionaldisplays, and the relationship between their actions/emotions andthe agent’s emotions, when rating the humanness of the agents. We use four sets of questions before andafter the game. These questions are as follows:(1)
Demographic Questionnaire:
Before the game, partici-pants were asked to provide their demographic information (i.e, age and gender). We use this information to controlfor possible effects of gender and age on perception of thehumanness of the agents. Participants could decide not todisclose this information.(2)
Humanness Questionnaire:
After the game, we used thehumanness assessing questionnaire to assess participants’ able 2: Coping strategies for the PD bot, including last player emotion, and the last two player moves. Player moves Player coping nextt-2 t-1 emotion (t-1) strategy example movetake 1 take 1 - acceptance: live with bad outcome “oh well, we’re doomed” take 1take 1 give 2 positive growth: positive reinterpretation “this might be turning around” give 2take 1 give 2 negative growth+denial: positive reinterpretation “maybe he didn’t mean that emotion” give 2give 2 take 1 regret restraint: hold back negative, keep trying “he’s a good person really” give 2give 2 take 1 not regret denial: deny reality, continue to believe “maybe its not so bad” give 2give 2 give 2 - seek support: understanding and sympathy “Let’s cooperate together on this” give 2 perception of agents’ emotions and behaviors. The hu-manness questionnaire consisted of 18 questions. The firsttwo questions asked participants to rate to what extendthey thought that the agent behaved human-like/animal-like. and to what extend they thought it behaved human-like/machine-like. The following 16 questions used thetraits proposed by Haslam et al. [17] and assessed differ-ent
Human Nature and
Human Uniqueness traits in moredetails.(3)
Enjoyment Question:
After the Humanness Question-naire, participants were asked to rate how much they en-joyed playing the game. We used this question to studywhether different emotional displays can affect partici-pants’ satisfaction.(4)
IDAQ Questionnaire:
After answering all other ques-tions, participants answered the IDAQ questionnaire pro-posed by Waytz et al. [47]. The results from this question-naire was used to account for individual differences in thegeneral tendency to anthropomorphize.A continuous slider was used in all questions, except IDAQ,which uses an 11-scale, the standard scale for this questionnaire [47].In addition to these questions, a total of six sanity-check questionswith clear answers (e.g., “How many ’a’s are in the word “Aria”’?”)were randomly placed among the questions in the Humanness andIDAQ questionnaires to ensure that the participants paid attention.
Participants first signed the consent form andprovided their demographic information. Then they played 25rounds of the game against one of the agents, which was randomlyassigned to them. After completing all 25 rounds of the game, par-ticipants answered the aforementioned set of questions regardinghumanness natures, enjoyment, and IDAQ. Repeated participationwas not allowed. We ensured that the participants saw and heardAria, and used a browser that was compatible with our platform.
Participants were recruited on Amazon Me-chanical Turk. 124 participants completed the game and the ques-tionnaire (74 male, 48 female, 1 other, and 1 did not wish to share,age: [21,74]). The data from 6 participants (4 male and 2 female)were removed as they failed to pass the attention checks. Data from1 participant (male) was removed as he was not able to hear theagent properly. Participants received an initial payment of $0.7and a bonus according to their performance in the game ($0.05 foreach point they earned). Participation was limited to residents ofNorth America, who had completed at least 50 HITs and had a priorMTurk approval rate of 96%. The experiment was approved by theUniversity of Waterloo’s Research Ethics Board. l Emotionless OCCRandom
Human Nature H u m an U n i quene ss Figure 2: Rating of humanness for all conditions. Xaxis shows the rating of Human Nature traits (emotional-ity, warmth, openness, individuality, and depth) and Y axisshows the rating of Human Uniqueness traits (civility, re-finement, maturity, rationality, and moral sensibility). 95%confidence intervals are demonstrated.
In this section, we will first demonstrate how playing against dif-ferent agents affected perception of humanness. We then show theeffects on the cooperation rate and on participants’ enjoyment.
We assessed all the agents based on the rating of HN and HU traits.Figure 2 shows the results. As hypothesized, the OCC model wasperceived to be more human-like on both HN and HU aspects. Wefit two linear mixed effect models predicting HN and HU ratingsbased on experimental condition. IDAQ, the general tendency toanthropomorphize, was controlled for. We also controled for possi-ble effects of age, gender, and bonus (as the final bonus may affectpeople’s perception of the agent). A random effect based on the dayon which the experiment was run is fitted. The modeling results forHN and HU ratings are shown in Tables 3 and 4, respectively. TheOCC agent’s HN traits were perceived to be significantly higherthan the Emotionless agent, and its HU traits were perceived to besignificantly higher than the Random agent. That is to say, overall, able 3: Linear mixed-effects model predicting the ratingsfor Human Nature traits based on condition. Age, gender,and anthropomorphism tendency (acquired using IDAQ) arecontrolled for. A random effect is fit based on the day. Covariate Estimate SE t Pr ( > | t | )Intercept 0.268 1.931 0.139 0.890Random -0.434 0.335 -1.296 0.198Emotionless -0.786 0.343 -2.296 < . < . Table 4: Linear mixed-effects model predicting the ratingsfor Human Uniqueness traits based on condition. Age, gen-der, and anthropomorphism tendency are controlled for. Arandom effect is fit based on the day.
Covariate Estimate SE t Pr ( > | t | )Intercept 10.449 3.708 2.818 0.006Random -2.285 0.644 -3.551 < . < . the OCC was perceived to be significantly more human-like ascompared to the other two conditions.Further, the bonus has significantly affected perception of theHU traits, as these traits mostly describe perception of the agent’sactions (e.g., rationality, sensibility). However, we did not see anyeffect of bonus on perception of HN traits. Next, we asked whether different emotional displays affected par-ticipants’ strategies. All agents played the same strategy (i.e., tit-for-two-tats); therefore, the difference in cooperation rates amongconditions can reflect the effect of the different emotional displayson participants’ tendency to cooperate (in other words, trustingthe agent). Figure 3 shows the results. OCC has the highest coop-eration rate and seems to encourage cooperation. This differenceis significant between the OCC and Random agent ( se = . , t = − . , p < . We know that perception of humanness of virtual agents can affectpeople’s enjoyment in games [3]. Here we ask what attributes ofhumans contribute to this effect. Therefore, we look into HN and
Condition P l a y e r C oope r a t i on N u m be r Figure 3: Number of rounds in which the participants choseto cooperate with Aria. The maximum number of roundswere 25 for all conditions. 95% confidence intervals are visu-alized.
Condition E n j o y m en t Human Nature Ratings E n j o y m en t (a) (b) Figure 4: (a) Enjoyment rating for all conditions. (b) En-joyment rating based on perception of the HN traits. 95%confidence intervals are visualized.Table 5: Linear mixed-effects model predicting the enjoy-ment ratings based on perception of HU and HN traits. An-thropomorphism tendency, and bonus are controlled for.Two random effect based on condition and day are fitted.
Covariate Estimate SE t Pr ( > | t | )Intercept -1.747 1.308 -1.336 0.184HN 0.475 0.095 5.005 < . < . < . HU traits independently, hypothesizing that HN traits are the keyfactors for enjoyment, as they distinguish humans from machines.Figure 4(a) shows the enjoyment ratings for each condition. Play-ing against the OCC agent has improved users’ experience of play-ing the game. We fit a model to look directly at how perceptionof HN and HU traits affects users’ enjoyment in the game. Table 5shows the results. Perception of HU traits does not seem to affect sers’ enjoyment, however, perception of HN traits significantlyaffected enjoyment in the game. That is, playing against an agentthat is perceived to be more human-like in HN traits (with the exactsame strategy and actions) significantly increased enjoyment inthe game. These results are visualized in Figure 4(b). As expected,the anthropomorphism tendency and the final bonus both affectedhappiness; therefore they are controlled for in the model. Virtual agents and assistants are used in many domains to enhancepeople’s quality of life. We are especially interested in applicationof virtual agents in health care, for assisting people with cognitivedisabilities such as Alzheimer’s Disease in performing daily activi-ties. Such assistants can decrease the dependence on the caregiverand reduce their burden. One challenge, however, is that the agentshould be designed in a way that it can be successfully adoptedby older adults with less exposure to technology. We know thataffective experience increases engagement [32] of users, improvesloyalty [20], and influences perception of humanness of the agents,which can affect people’s behaviour and enjoyment [3]. Therefore,in this paper, we studied how emotions affect perception of differentdimensions of humanness of the agents, and users’ trust.We utilized Haslam et al. ’s definition of humanness [17] to studyhow emotions affect perception of
Human Nature (HN) , distinguish-ing humans from machines, and
Human Uniqueness (HU) , distin-guishing humans from animals. We asked how emotions affectpeoples’ perception of HU and HN traits of an agent, and as a result,their opinion and behaviour towards the agents. We hypothesizedthat although there is an emphasis on improving the HU dimen-sion of computers (as a result of making computers more refined,rational, and moral), improving the HN dimension has seen rela-tively limited attention. With emotionality being an aspect of HN,we hypothesize that agents capable of showing emotions will beperceived more human-like, especially on the HN traits.We used a social dilemma, the prisoner’s dilemma, to test thishypothesis. In prisoner’s dilemma, the players cooperate if theytrust the opponent, so this game enables us study how emotionsand perception of humanness of agents can affect trust. Althoughall the agents (opponents) played the same tit-for-2-tats strategy,they differed in the emotional displays, which significantly affectedperception of human-like traits. The OCC agent, capable of showingmeaningful emotions, was perceived significantly more human-likeon both HN and HU traits. This significantly improved partici-pants’ cooperation rate and enjoyment.
Any expression of emotion,even by the random agent, improved perception of Human Naturetraits (warmth, openness, emotionality, individuality, and depth).However, displaying random emotion negatively affected HumanUniqueness traits (civility, refinement, moral sensibility, rationalityand maturity), as it can make the agent look irrational and immature.That is to say, while showing proper emotions enables computeragents with Human Nature traits and fills the gap between humansand machines, showing random emotions that are not necessarilymeaningful is even worse than showing no emotions for the HumanUniqueness traits, making the agent more animal-like.An interesting observation was that the general anthropomor-phism tendency (measured through the IDAQ questionnaire) sig-nificantly and negatively affected cooperation. This may suggest an uncanny valley effect: those who anthropomorphized moreperceived Aria to be more similar to humans, which resulted indisliking Aria and not trusting her [27].Finally, age significantly affected ratings of Human Nature traits.All the agents were perceived more human-like on the HN traitswhen age increased. This may be because the younger adults aremore used to seeing avatars and characters in computer games,which look similar to humans, thus have a higher standard in mindregarding virtual agents. Another interesting observation was thatthe amount of bonus significantly affected perception of HU traits.Possibility because the participants believed that the agent (i.e. theiropponent in the game) was in fact responsible for what they earned(the results), and associated a higher bonus to a better performanceof the opponent (despite the fact that the strategy of all the agentswas the same).Our work has a number of limitations. First, the
OCC copingmechanism is theoretically difficult to justify and is usually specifiedin a rather ad-hoc manner [5]. In the simple game considered here,it provides a reasonable approximation and yields a strategy oftenused by humans (tit-for-two-tat). Similar coping strategies couldbe defined using an “intrinsic reward” generated by the appraisalvariables. As reviewed by Broekens et al. [30], this intrinsic rewardrequires some weighting factor (e.g. ϕ in [30]) which is difficultto specify. In this simple game, we could, for example, considerthat motivational relevance, which is inversely proportional to thedistance from the goal, may be larger in cases where the agentpredicts cooperation (e.g., the other player cooperates, or defectsbut shows regret), and smaller when the agent predicts defection(e.g., the other player defects and shows no regret, or defects mul-tiple times). Motivational relevance would add intrinsic rewardto the cooperation option, making it game theoretically optimalcompared to defection. In the give2-take1 game, this would requireadding a reward of 1 to the cooperation option when motivationalrelevance was high (when there was “hope”) and not doing so whenmotivational relevance was low (when there was “fear”).A second limitation is that emotions are displayed in the facebased only on a dimensional emotion model (EPA space), and ne-glects semantic context. For example, while repentant and reverent have different meanings which should result in different facial ex-pressions, their EPA ratings are almost identical. Therefore thereare some instances where mapping from EPA → facial expressionsdoes seem accurate, but emotion label → facial expression not somuch. A third limitation is the limited number of emojis used toallow human players to express emotions, and the interpretationgiven to those emojis by the participants. While the emotion wordcan be seen by hovering, a better method would involve facialexpression recognition to extract emotional signals directly. This paper described our work towards understanding the effectof emotions on different dimensions of humanness of computeragents, as well as on users’ cooperation tendency and enjoyment.We studied traits distinguishing humans from machines (
HumanNature ), and those distinguishing humans from animals (
HumanUniqueness ), and showed that proper expressions of emotions in-creases perception of human nature of agents. While researcherscan successfully improve perception of
Human Uniqueness traits y making agents smarter, emotions are critical for perception of Human Nature traits. This improvement also positively affectedusers’ cooperation with the agent and their enjoyment. Further,we showed that if emotions are not reflected properly (e.g., gen-erated randomly), they can have negative effects on perception ofhumanness (HU traits) and can reduce the quality of social agents,even compared to when the agent does not reflect any emotions.Therefore, it is important to find models that accurately understandand express emotions, and utilize them properly in developingvirtual agents, should those agents need to be perceived as morehuman-like.
ACKNOWLEDGEMENTS
This work was supported by Natural Sciences and EngineeringResearch Council of Canada (NSERC) and Social Sciences and Hu-manities Research Council of Canada (SSHRC) through the Trans-Atlantic Platform’s Digging into Data Program. We thank NattawutNgampatipatpong and Sarel van Vuuren for help with the virtualhuman.
REFERENCES [1] Dimitrios Antos and Avi Pfeffer. 2011. Using Emotions to Enhance Decision-Making. In
Proc. International Joint Conferences on Artificial Intelligence .Barcelona, Spain.[2] Roland B´enabou and Jean Tirole. 2006. Incentives and Prosocial Behaviour.
American Economic Review
96 (2006), 1652–1678.[3] Andry Chowanda, Martin Flintham, Peter Blanchfield, and Michel Valstar. 2016.Playing with social and emotional game companions. In
International Conferenceon Intelligent Virtual Agents . Springer, 85–95.[4] Cristina Conati and Heather Maclaren. 2009. Empirically Building and Evaluatinga Probabilistic Model of User Affect.
User Modeling and User-Adapted Interaction
19 (2009), 267–303.[5] Carlos Crivelli and Alan J. Fridlund. 2018. Facial Displays are Tools for SocialInfluence.
Trends in Cognitive Science (2018).[6] Celso M De Melo, Peter Carnevale, and Jonathan Gratch. 2010. The influenceof emotions in embodied agents on human decision-making. In
InternationalConference on Intelligent Virtual Agents . Springer, 357–370.[7] Celso M. de Melo, Peter Carnevale, and Jonathan Gratch. 2011. The Effect ofExpression of Anger and Happiness in Computer Agents on Negotiations withHumans. In
Proc. International Conference on Autonomous Agents and MultiagentSystems (AAMAS) . Taipei, Taiwan.[8] Celso M. de Melo, Peter Carnevale, Stephen Read, Dimitrios Antos, and JonathanGratch. 2012. Bayesian Model of the Social Effects of Emotion in Decision-Making in Multiagent Systems. In
Proc. International Conference on AutonomousAgents and Multiagent Systems (AAMAS) . Valencia, Spain.[9] St´ephanie Demoulin, Jacques-Philippe Leyens, Maria-Paola Paladino, Ram´onRodriguez-Torres, Armando Rodriguez-Perez, and John Dovidio. 2004. Dimen-sions of ”uniquely” and ”non-uniquely” human emotions.
Cognition and emotion
18, 1 (2004), 71–96.[10] Clark Elliott. 1998. Hunting for the Holy Grail with ’Emotionally Intelligent’Virtual Actors.
ACM SIGART Bulletin
9, 1 (1998), 20–28.[11] P. Gomez Esteban and D. Rios Insua. 2017. An Affective Model for a non-Expensive Utility-based Decision Agent.
IEEE Transactions on Affective Comput-ing (2017), 1–1. https://doi.org/10.1109/TAFFC.2017.2737979[12] Clare Francis and David R. Heise. 2006. Mean Affective Ratings of 1,500 Conceptsby Indiana University Undergraduates in 2002-3. [Computer file] Distributed atAffect Control Theory Website. (2006).[13] Nico H. FRidja. 2010. The Psychologists’ Point of View. In
Handbook of Emotions (third edition ed.). The Guildford Press, 68–77.[14] Moojan Ghafurian and David Reitter. 2018. Impatience in Timing Decisions:Effects and Moderation.
Timing & Time Perception
6, 2 (2018), 183–219.[15] Jonathan Gratch. 2000. Emile: Marshalling Passions in Training and Education. In
Proceedings of the 4th International Conference on Autonomous Agents . Barcelona,Spain.[16] Jonathan Gratch and Stacy Marsella. 2004. A domain-independent frameworkfor modeling emotion.
Cognitive Systems Research
5, 4 (2004), 269 – 306. https://doi.org/10.1016/j.cogsys.2004.02.002[17] N. Haslam, S. Loughnan, Y. Kashima, and P. Bain. 2008. Attributing and denyinghumanness to others.
European Review of Social Psychology
19, 1 (2008), 55–85. [18] Jesse Hoey, Tobias Schr¨oder, and Areej Alhothali. 2016. Affect Control Processes:Intelligent Affective Interaction using a Partially Observable Markov DecisionProcess.
Artificial Intelligence
230 (January 2016), 134–172.[19] Jeffrey R. Huntsinger, Linda M. Isbell, and Gerald L. Clore. 2014. The AffectiveControl of Thought: Malleable, Not Fixed.
Psychological Review
Proceedings of the 2000 ACM SIGCPR conference onComputer personnel research . ACM, 77–85.[21] Joshua D.A. Jung and Jesse Hoey. 2016. Grounding Social Interaction withAffective Intelligence. In
Proceedings of the Canadian Conference on AI . Victoria,BC.[22] Gerben A. Van Kleef, C De Dreu, and A. Manstead. 2004. The interpersonaleffects of anger and happiness in negotiations.
Journal of Personality and SocialPsychology
86 (2004), 57–76.[23] Christine Laetitia Lisetti and Piotr Gmytrasiewicz. 2002. Can a rationalagent afford to be affectless? a formal approach.
Applied Artificial Intel-ligence
MABS 2012: 13thInternational Workshop on Multi-Agent Based Simulation . Valencia, Spain.[25] G. Loewenstein and J.S. Lerner. 2003. The Role of Affect in Decision Making. In
Handbook of Affective Sciences , R.J. Davidson, K.R. Sherer, and H.H. Goldsmith(Eds.). Oxford Univ. Press, 619–642.[26] Aarti Malhotra, Jesse Hoey, Alexandra K¨onig, and Sarel van Vuuren. 2016. Astudy of elderly people’s emotional understanding of prompts given by VirtualHumans. In
Proc. International Conference on Pervasive Computing Technologiesfor Healthcare . Cancun, Mexico, 13–16.[27] Maya B Mathur and David B Reichling. 2016. Navigating a social world withrobot partners: A quantitative cartography of the Uncanny Valley.
Cognition
Journal of Experimental Psychology: General
ArXiv e-prints (Oct. 2013). arXiv:cs.CL/1310.4546[30] Thomas M. Moerland, Joost Broekens, and Catholijn M. Jonker. 2017. Emotionin Reinforcement Learning Agents and Robots: A Survey.
Machine Learning
Energy
7, 4 (1970), 33fi?!35.[32] Heather L O’Brien and Elaine G Toms. 2008. What is user engagement? Aconceptual framework for defining user engagement with technology.
Journal ofthe American society for Information Science and Technology
59, 6 (2008), 938–955.[33] A. Ortony, G.L. Clore, and A. Collins. 1988.
The Cognitive Structure of Emotions .Cambridge University Press.[34] A. Ortony, D. Norman, and W. Revelle. 2005. Affect and proto-affect in effectivefunctioning. In
Who needs emotions: The brain meets the machine , J. Fellous andM. Arbib (Eds.). Oxford University Press, 173–202.[35] Charles E. Osgood, William H. May, and Murray S. Miron. 1975.
Cross-CulturalUniversals of Affective Meaning . University of Illinois Press.[36] Julia Poncela-Casasnovas, Mario Guti´errez-Roig, Carlos Gracia-L´azaro, JulianVicens, Jes´us G´omez-Gardena, Josep Perell´o, Yamir Moreno, Jordi Duch, andAngel S´anchez. 2016. Humans display a reduced set of consistent behavioralphenotypes in dyadic games.
Science Advances
2, 8 (2016).[37] Alexandru Popescu, Joost Broekens, and Maarten van Someren. 2014. GAMYG-DALA: an Emotion Engine for Games.
IEEE Transactions on Affective Computing
5, 1 (2014), 32–44.[38] Rainer Reisenzein, Eva Hudlicka, Mehdi Dastani, Jonathan Gratch, Koen Hin-driks, Emiliano Lorini, and John-Jules Ch Meyer. 2013. Computational modelingof emotion: Toward improving the inter-and intradisciplinary exchange.
IEEETransactions on Affective Computing
4, 3 (2013), 246–266.[39] Jennifer Sabourin, Bradford Mott, and James C. Lester. 2011. Modeling learneraffect with theoretically grounded dynamic Bayesian networks. In
Proc. AffectiveComputing and Intelligent Interaction . Springer-Verlag, 286–295.[40] Klaus R. Scherer, A. Schorr, and T. Johnstone. 2001.
Appraisal Processes in Emotion .Oxford University Press.[41] Herbert A. Simon. 1967. Motivational and Emotional Controls of Cognition.
Psychological Review
74 (1967), 29–39.[42] Paul Slovic, Melissa Finucane, Ellen Peters, and Donald G. MacGregor. 2007.The Affect Heuristic.
European Journal of Operations Research
International Conference on Intelligent Virtual Agents . Springer, 15–28.[44] Giota Stratou, Rens Hoegen, Gale Lucas, and Jonathan Gratch. 2015. EmotionalSignaling in a Social Dilemma: an Automatic Analysis. In
Humaine Association onference on Affective Computing and Intelligent Interaction (ACII) . Xi’an, China.[45] Alessandro Vinciarelli, Maja Pantic, Dirk Heylen, Catherine Pelachaud, IsabellaPoggi, Francesca D’Errico, and Marc Schroeder. 2012. Bridging the gap betweensocial animal and unsocial machine: A survey of social signal processing. IEEETransactions on Affective Computing
3, 1 (2012), 69–87.[46] Sarel Van Vuuren and L.R. Cherney. 2014. A Virtual Therapist for Speech andLanguage Therapy. In
Proc. Intelligent Virtual Agents, LNAI 8637 . Springer, 438–448.[47] Adam Waytz, John Cacioppo, and Nicholas Epley. 2010. Who sees human?The stability and importance of individual differences in anthropomorphism.
Perspectives on Psychological Science
5, 3 (2010), 219–232.[48] Maarten J.J. Wubben, David De Cremer, and Eric van Dijk. 2009. How emotioncommunication guides reciprocity: Establishing cooperation through disappoint-ment and anger.
Journal of Experimental Social Psychology
45, 4 (2009), 987 – 990.https://doi.org/10.1016/j.jesp.2009.04.010[49] R.B. Zajonc. 2000. Feeling and Thinking: Closing the Debate over the Indepen-dence of Affect. In
Feeling and Thinking: The Role of Affect in Social Cognition.Studies in Emotion and Social Interaction , J. P. Forgas (Ed.). Vol. 2. CambridgeUniversity Press, New York, 31–58., J. P. Forgas (Ed.). Vol. 2. CambridgeUniversity Press, New York, 31–58.