How Much Does Users' Psychology Matter in Engineering Mean-Field-Type Games
aa r X i v : . [ c s . G T ] F e b How Much Does Users’ Psychology Matter inEngineering Mean-Field-Type Games
Giulia Rossi, Alain Tcheukam and Hamidou Tembine ∗† February 28, 2017
Abstract
Until now mean-field-type game theory was not focused on cognitively-plausible models of choices in humans, animals, machines, robots, software-defined and mobile devices strategic interactions. This work presents someeffects of users’ psychology in mean-field-type games. In addition to thetraditional “material” payoff modelling, psychological patterns are intro-duced in order to better capture and understand behaviors that are ob-served in engineering practice or in experimental settings. The psycho-logical payoff value depends upon choices, mean-field states, mean-fieldactions, empathy and beliefs. It is shown that the affective empathyenforces mean-field equilibrium payoff equity and improves fairness be-tween the players. It establishes equilibrium systems for such interac-tive decision-making problems. Basic empathy concepts are illustratedin several important problems in engineering including resource sharing,packet collision minimization, energy markets, and forwarding in Device-to-Device communications. The work conducts also an experiment with47 people who have to decide whether to cooperate or not. The basic In-terpersonal Reactivity Index of empathy metrics were used to measure theempathy distribution of each participant. Android app called Empathizeris developed to analyze systematically the data obtained from the par-ticipants. The experimental results reveal that the dominated strategiesof the classical game theory are not dominated any more when users’psychology is involved, and a significant level of cooperation is observedamong the users who are positively partially empathetic.
Keywords:
Psychology, empathy, game theory, mean-field, belief, consis-tency ∗ Part of this work appeared in [2]. † The authors are with Learning & Game Theory Laboratory, New York University AbuDhabi, [email protected] ontents Introduction
Until now, mean-field-type game theory was not focused on cognitively-plausiblemodels of choices in humans, animals, machines, robots, software-defined andmobile devices strategic interactions. This paper studies behavioral and psycho-logical games of mean-field type. Psychological games seems to explain behav-iors that are better captured in experiments or in practice than classical game-theoretic equilibrium analysis. It takes in consideration psychological patternsof the decision-makers in addition to the traditional “material” payoff mod-elling. The payoff value depends upon choice consequences, mean-field states,mean-field actions and on beliefs about what will happen. The psychologicalgame theory framework can link cognition, emotion, and express emotions, guilt,empathy, altruism, spitefulness (maliciousness) of the decision-makers. It alsoinclude belief-dependent and other-regarding preferences in the motivations.One motivating example of psychological game theory is trying to understandhow her users and consumers will perceive a product or are thinking about aproduct in web online shop and will engage in empathy in the interaction. Thereare several definitions of empathy in the literature (see [4]). Cognitive empathyof a player, sometimes also called perspective taking, is the ability to identifythe felling and emotions of other players. Perspective taking empathy is con-sidered as the experience of understanding another player’s state and actionsfrom their perspective or mutual perspective via several channels. A decision-maker can place herself in the shoes of the others and feel what they are feeling.This is a particularly useful concept in the context of psychological game the-ory. Indeed, it helps to anticipate, compute and to react to the behavior ofthe others thanks to empathy. Note that, empathy is different than sympathywhich is the ability to select appropriate emotional responses for the apparentemotional states of others. In other words, sympathy is not about feeling thesame thing that somebody else is feeling, but an appropriate emotion to comple-ment theirs. Another notion is compassion which heuristically is to treat othersas you would like to be treated. It consists in selecting the appropriate actionin response to the apparent emotional states of another. This active versionof empathy may result in partial altruism in the preferences formation of theplayers. In game theory, the strategy and the resulting actions play key roles inthe outcomes. A player may use empathy in different ways. Examples includeempathy-selfishness, empathy-altruism and empathy-spitefulness. In this work,we examine basic empathy subscales: perspective taking (PT), empathy con-cern (EC), fantasy scale (FS) and personal distress (PD) that will be evaluatedthrough the Interpersonal Reactivity Index (IRI, [33]).
We overview some prior works on empathy in game theory. The motivation ofdecision makers who care for various emotions, intentions-based reciprocity, orthe opinions of others may depend directly on beliefs (about choices, states, be-haviors, or information). The study in [4] tries to explain how we can understand3hat someone else feels when he or she experiences simple emotions. Some psy-chological factors are considered in a game theoretic context [5, 6, 9, 11, 12, 18].The work in [25] investigates the neural basis of complex decision making usinga game theory. The authors of [26] study neuroeconomics approach to decision-making by combining game theory with psychological and neuroscientific meth-ods. The work in [19] underlined how empathy leads to fairness and [20] studiedthe correlation between empathy, anticipated guilt and prosocial behaviour; inhis study he found out that empathy affects prosocial behaviour in a more com-plex way than the one represented by the classical model of social choices. Theauthors in [6] propose and synthesize a large body of experimental and theo-retical analysis on multi-agent interactions, in psychology as well as economics.The work in [27] presents a game theoretic approach to empathy, reportingon current knowledge of the evolutionary, social, developmental, cognitive, andneurobiological aspects of empathy and linking this capacity to human com-munication, including in clinical practice and medical education. Under mildconditions, [30] shows that such empathetic preferences requiring us to see thingsfrom another’s point of view can be summarized by empathic payoff function.The idea of empathic preferences was followed in [28] where the importance ofempathic payoff function is illustrated. The work in [32] presents an overview onempathy and mind reading in some detail and have pointed out other-regardingpreferences in game theory. In the classical approach, they are taken to be, inthe ’worst’ case, a purely selfish or, in the best case, self- regarding individuals.In the psychological payoffs, players may care about the betterment of othersas well as themselves but their position, state or consumption is simply anotherargument of their own preferences. Other-regarding preferences are sometimespresented not simply as being concerned with the payoff of specific others, butmay incorporate more general concerns, such as equity, risk and fairness. Theproblem of comparing the payoff of different players is vital for the questionof fairness and equity [29]. The work in [21] proposes an evolutionary game-theoretic approach to study the evolutionary effect of empathy on cooperativegames. The origin, the source and learnability of empathy remains open. Theevolution of empathy and its connection to reciprocal altruism are discussed in[31]. Building on dynamic interactive epistemology, [10] proposes a more gen-eral framework that includes higher-order beliefs, beliefs of others, and plansof action may influence motivation, dynamic psychological patterns (such assequential reciprocity, psychological forward induction, and regret). These pre-liminary studies enrich classical game theory by empirical knowledge and makesit significantly closer to what is needed for real-world applications. Thousandsof theoretical papers have been published about the prisoner’s dilemma gameand more than 30 experiments about have been conducted in the literature,only few of them are dedicated to the emergence of cooperative behaviors inone-shot games [37]. The works in [38, 39, 40] consider decision-making prob-lems. However, the effect of users’ psychology on its decision is not examinedin these previous works. None of these previous works considered a mean-field-type game setup. Finally, the range of applications covered by these papers islimited compared to the current work. 4he goal of this paper is to examine the effect of empathy on players’ be-havior and outcomes in mean-field-type game theory. The psychology of theplayers is analyzed in several engineering applications.
Our contribution can be summarized as follows. This paper illustrates how someinsights from the psychology literature on empathy can be incorporated into amean-field-type payoff function, and demonstrate the potential interaction ofbeliefs, strategies, mean-field through the channel of empathy. It establishesmean-field equilibrium systems with psychological payoffs. • Empathy as perspective taking may induce a partial altruism. To apartially altruistic player we would be considering a payoff in the form r λj = r j + P j ′ = j λ jj ′ r j ′ , where λ jj ′ ≥ . It is shown that empathy-altruism promotes fairness in terms of mean-fieldequilibrium payoffs in wide range of mean-field-type games. We illustrateempathy concepts in several important engineering problems. – We provide an experimental evidence that the degree of empathy canshift the decisional balance in a one-shot forwarding dilemma game.To this end, we have conducted an experimental test with 47 peoplecarrying mobile devices. We have developed a android app called em-pathizer to measure dimensional empathy of the participant (Fig.1).It turns out that empathy induces some cooperative behaviors whichconsist to forward the data of the other wireless nodes in a network.Applied to D2D communications and WiFi Direct technology, thisexperiment helps us to estimate the proportion of users who are po-tentially interesting in enabling their platform to the others. Thesame method can be useful in other contexts such as mobile crowdsensing which pertains to the monitoring of large-scale phenomenathat cannot be easily measured by a single individual user. For exam-ple, intelligent transportation systems may require traffic congestionmonitoring and air pollution level monitoring. These phenomena canbe measured accurately only when many individuals provide speedand air quality information from their daily commutes, which arethen aggregated spatio-temporally to determine congestion and pol-lution levels in smart cities. It is thus important to estimate thenumber of potential participants who decide their level of participa-tion to the crowdsensing when these users are carrying power-hungrydevices to serve the cloud data. – Empathy-altruism provides a better explanation of resource sharingoutcomes. – The empathy-altruism of the users may help in reducing packets col-lision in wireless medium access channel and hence reducing conges-tion. 5igure 1: Empathizer app: Sample welcome screen on android platform formeasuring empathy at NYUAD L&G Lab. – Empathy-altruism reduces energy consumption during peak hours. – Empathy-altruism of prosumers improves their equilibrium revenues. • Can empathy at times be harmful?We do not restrict ourselves to the positive part of empathy. Empathymay have a ’dark’ or at least costly side specially when the environmentis strategic and interactive as it is the case in games.Can empathy be bad for the self? Empathy can be used, for example, bya other player attacker to identify the weak nodes in the network.Can empathy be bad for others? Empathetic users may use their ability todestroy the opponents. In strategic interaction between people, empathymay be used to derive antipathetic response (distress at seeing others’pleasure, or pleasure at seeing others’ distress).We illustrate this in a context of auction in prosumer (consumer-producer)markets. A prosumer who is bidder might be losing the auction due highlycompetitive prices. Yet she participates in the auction because she wantsto minimize the negative payoff on losing by making her competitor, whowould win the auction, gets low reward by selling its energy at almost zero6rice or negative price and hence the other get a high price for the win.This negative dependence of payoff on others’ surplus is referred to asspiteful behavior. We associate a certain spitefulness coefficient − α jj ′ ≤ j. A spiteful player j maximizes the weighted difference ofher own profit r j and his competitors’ profits r j ′ for all j ′ = j. The payoffof a spiteful (antipathetic) player is r αj := α j r j − X j ′ = j α jj ′ r j ′ (1)Obviously, setting α jj ′ = 0 , j = j and α j = 1 yields a selfishness (whosepayoff equals his exact profit) whereas α j = 0 , α jj ′ = 1 defines a mali-cious player (jammer) whose only goal is to minimize the global profitof other players. For α j = 0 , we can scale the payoff by α j , to get α j h r j − P j ′ = j α jj ′ α j r j ′ i , which is equivalent to focusing on r λj = r j − X j ′ = j λ jj ′ r j ′ where λ jj ′ := α jj ′ α j . This class of games captures a very extreme scenarioin which everyone dislikes all the others. It is shown that the empathy-spitefulness of prosumers decreases the optimal bidding price of the win-ners. This means that the spitefulness of the prosumers may benefit tothe consumers.
The rest of the paper is organized as follows. In Section 2 we provide motivatingexamples illustrating how empathy-altruism improves fairness. A generic mean-field-type game is presented and analyzed in Section 3. Section 3.6 illustratesempathy in variance reduction problems. Section 4 presents an experimentalsetup for participation in data forwarding in D2D communications. Section 5concludes the paper.Notations used in the text are available in Table 1.3.
We consider the local empathetic preference, the outcome of agent i for action a i combines her intrinsic preference for a i with the intrinsic preference of i ’sneighbors, N i , where the weight given to the preference of any neighbor j ∈ N i depends on the strength of the relationship between i and j. A basic setup andfor illustration purpose this can be captured with a number λ ij but it could bea general map with beliefs. 7eaning NotationHorizon { , , . . . , T − } Number of players n Time index t Noise η t State s t State mean-field m st Action profile a t Action mean-field m at Instant material payoff of i r it ( s t , m st , m at , a t )Terminal material payoff of i g iT ( s T , m sT )Instant empathic payoff of i r λit ( s t , m st , m at , a t )Terminal empathic payoff of i g λiT ( s T , m sT )Degree of empathy of i λ i ∈ R n − Table 1: Notations
By a self-regarding player we refer to a player in a game who maximizes his ownpayoff r i . A self-regarding player i thus cares about the behavior and payoffsthat impact her own payoff r i . An other-regarding player i considers not only her own payoff r i but also someof her network members’ payoffs ( r j ) j ∈N i . Then, the player will include these inher preferences and create an empathic payoff. She is still acting to maximizeher new empathic payoff.
There are many interactive decision-making situations where both positive andnegative reciprocal behaviors are observed. A user carrying a wireless devicemay favorably accept to forward the data of another temporary device (a newjoiner at a public place, conference or airport), and that device reciprocates thefavor although it is unlikely that they will ever meet again. In order to capturesuch a phenomenon in the preferences, the kindness between players and higherorder reciprocity terms [43, 44] will be introduced below.
The question of whether there is a fixed distribution of degrees of other-regardingbehaviour in the network is important. We investigate the effect of dynamic em-pathy on the payoffs in mean-field-type games. The basic experiment reported8elow reveals that there is a distribution of empathy across the population (seeFigure 13 and Table 7) and it is context-specific.
In this section we discuss the interplay of self-regarding and other-regardingbehavior through motivating examples.
As inspired by the Ultimatum Game, we consider two sisters who are asked tosplit/share a cake. The first sister, the proposer, makes an offer of how to splitthe cake. The second sister, the responder, either accepts the offer, in whichcase the cake is split as agreed, or rejects it, in which case neither sister receivesanything and a restart the process. If we model only with material payoff, andthe horizon is 1 then a good strategy for the proposer is to offer the smallestpossible positive share of cake and for the responder sister to accept it. Howeverthe material payoff is not what is widely observed in engineering practice. Why?One possible explanation may come from psychology using the empathy of thesisters for each other. The sisters do not behave this way, however, and insteadtend to offer nearly 50% of the cake and to reject offers below 20%. Empathymeans that individuals make offers which they themselves would be preparedto accept. If only below 20% of the cake were proposed to yourself you wouldnot accept so you will not propose an offer below 20%. Following that idea wewill see that empathy can lead to the evolution of fairness in the interaction.
Consider n wireless devices sharing a common medium channel using Aloha-likeprotocol. If two or more users transmit simultaneously, there is a collision andthe packets are lost. If only one user transmits at a time slot with transmissionpower ¯ p > β i . The success conditionis 1l
SNR i ≥ β i which is a random variable which is equal to 1 if SN R i ≥ β i and 0otherwise. This is an interactive decision-making framework for channel accessPlayer ITransmit WaitPlayer II Transmit (0,0) (1l SNR ≥ β , , SNR ≥ β ) (0,0)Figure 2: Random payoff matrix of wireless collision channel game with self-regarding players. “T” is for Transmit and “W” for Wait.point in wireless networks where the outcome is influenced not only by thedecisions of the users but also by a random variable representing the channelstate. This belongs to the class of random matrix games (RMGs, [1]) because9he SNR appears in the entries of the payoff-matrix as the SNR depends on thechannel state which is random process.Considering the expected material payoff, it is not difficult to observe that( δ T , yδ T + (1 − y ) δ W ) is an equilibrium for any y ∈ [0 , . In particular the pureaction profile (
T, T ) is an equilibrium. Thus the payoff gap between the payoffsin equilibrium is IN (0) = max { P ( SN R i ≥ β i ) , i ∈ { , }} . We denote by λ i the degree of empathy-altruism of user i. Then, the empathic-altruism payoff matrix is given byPlayer IT WPlayer II T (0,0) ( a , b )W ( a , b ) (0,0)Figure 3: Random payoff matrix of wireless collision channel game with partialempathy-altruism.where ( a , b ) = (1l SNR ≥ β , λ SNR ≥ β ) , ( a , b ) = ( λ SNR ≥ β , SNR ≥ β ) . With this expected empathic payoff, the profile (
T, W ) is an equilibriumand the pure action profile (
T, T ) is no longer an equilibrium if max { P ( SN R i ≥ β i ) , i ∈ { , }} > . Thus, the equilibrium payoff gap between the users is IN ( λ ) = max { (1 − λ j ) P ( SN R i ≥ β i ) , i, j ∈ { , }} which is smaller than IN (0) for any λ ∈ (0 , . This says that the concept ofempathy-altruism helps to reduce collisions in wireless medium access control(see Figure 4).
Consider n consumers interacting in an energy market. The electricity priceis a function of the aggregated supply S (of the producers) , the aggregateddemand D (of the consumers) and on mismatch between supply and demand.The payoff r i of consumer i depends on the amount of energy consumed d i , onhis/her own degree of satisfaction w i ( d i ) (a typical satisfaction function would be w i ( d i ) := 1 − e − d i ) and on the electricity price p. Let r i = w i ( d i ) − p ( D, S ) d i bethe material payoff. The interior equilibrium (if any) when users are empathic-selfish satisfies w ′ i − p ′ d i d i = p and the solution is denoted by d ∗ i (0) . Let λ ∈ (0 ,
1) be a parameter modeling the degree of empathy-altruism of aconsumer in the power network. We denote by ˜ r i = r i + λr j the empathic payoff10 empathy−altruism ( λ ) pa y o ff gap Collision channels reduction
Figure 4: Impact of empathy-altruism on collision channel reduction. As thealtruism level of the users increases the collision ratio decreases.of prosumer i given his/her empathy-altruism toward consumers j, j = i. Theinterior equilibrium when users are empathic-altruistic satisfies w ′ i ( d i ) − p ′ d i d i = p − λd j , that solution is denoted by d ∗ i ( λ ) . Since p − λd j ≤ p and the function d i w ′ i ( d i ) − p ′ d i d i is non-decreasing, and a one-to-one mapping within its range, itturns out d ∗ i ( λ ) ≤ d ∗ i (0) , ∀ λ ∈ (0 , . Summing over the consumers, we obtain D ∗ ( λ ) := X i d ∗ i ( λ ) ≤ D ∗ (0) = X i d ∗ i (0) , which means that empathy-altruism reduces the energy consumption. In partic-ular it reduces the global peak demand during peak hours. Figure 5 representsthe total demand curve D ( t ) for 1 day. One can observe two important peakswhich are significantly reduced when players are empathic. The question ofhow to incentivize users’ to be more empathic is an interesting direction thatwe leave for future research. Each prosumer has a unit production cost and quantity q. The production c j isa random variable with support in [0 , ¯ c ] and with cumulative function equals to F ( . ) . Each prosumer knows its own production cost, its spitefulness coefficient,its bid but not the production cost of the other bidders. Each bidder knows thecumulative distribution of the others. 11 t D ( t ) population testempathy−altruism population Figure 5: Impact of empathy-altruism on peak energy demand reductionThe game is played as follows. Each prosumer bids a (unit) price p j withthe production quantity q j > . The expected material payoff is r j = q j ( p j − c j )1l ( p j < min j ′6 = j p j ′ ( c j ′ )) − e j , where e j is the entry cost to the energy market. The empathic-spiteful payoffof prosumer j is r λj = r j − λr i with λ > . We are interested in structural results of equilibrium strategy under spitefulcoefficient. Let X be a random variable drawn from the interval [0 , ¯ c ] withcumulative distribution function I. Then, the conditional expectation of X giventhat X is greater than c is given by E ( X | X > c ) = Z ¯ cc x I ′ ( x )1 − I ( c ) dx. The optimal bidding (price) strategy of the prosumer is p ∗ ( c, F, λ ) = E ( X λ | X λ > c ) , where X λ is a random variable with cumulative function I λ ( c ) = P ( X λ < c ) = 1 − (1 − F ( c )) λ . It can easily be checked that I λ ( c ) is indeed a valid cumulative distributionfunction I λ (0) = 0 , I λ (1) = 1 , and I λ is non-decreasing and differentiable. Notethat the optimal bidding price of a winner is likely to be above c so that theprosumer gets some benefits in selling electricity to the market. The optimalbidding price p ∗ ( c, F, λ ) decreases as the spitefulness parameter λ increases.12 .3.5 Empathy-Altruism of Prosumers Improves the revenue of theProsumers We now examine the effect of Empathy-Altruism in the revenue of the pro-sumers. From the above analysis, the altruism strategy is obtained by changingthe sign of λ. The optimal bidding price p ∗ ( c, F, − λ ) for partially altruistic pro-sumers increases as λ increases. This will help the prosumers to save more: thebenefit p ∗ − c increases with λ ∈ (0 , . In the next section we present a class of mean-field-type games [7, 8, 34, 35]and explain how the above preliminary results in Subsections (1)-(5) on thepsychology of the players can be extended to this context.
Definition 1 (Mean-Field-Type Game)
A mean-field-type game is a gamein which the instantaneous payoffs and/or the state dynamics coefficient func-tions involve not only the state and the action profile but also the joint distribu-tions of state-action pairs (or its marginal distributions, i.e., the distributionsof states or the distribution of actions). A typical example of payoff function ofplayer j has the following structure: r j : S × A × P ( S × A ) → R , with r j ( s, a, D ( s,a ) ) where ( s, a ) is the state-action profile of the players and D ( s,a ) is the distribution of the state-action pair ( s, a ) , S is the state space and A is the action profile space of all players. From Definition 1, a mean-field-type game can be static or dynamic in time.In mean-field-type games, the number of players is arbitrary: it can be finiteor infinite [22, 23, 24]. The indistinguishability property (invariance in law bypermutation of index of the players) is not assumed. A single player may havea non-negligible impact of the mean-field. This last property makes a strongdifference between “mean-field games” and “mean-field-type games”.One may think that “mean-field-type games” is a small and particular classof games. However, this class includes the classical games in strategic formbecause any payoff function r j ( s, a ) can be written as r j ( s, a, D ) where D ( s,a ) is the distribution of the state-action pair ( s, a ) . Thus, the form r j ( s, a, D ) ismore general and includes non-von Neumann payoff functions. Example 1 (Mean-variance payoff )
The payoff function of agent i is E [ r i ( s, a )] − λ p var [ r i ( x, a )] , λ ∈ R which can be written as a function of r i ( s, a, D ( s,a ) ) . Forany number of interacting players, the term D s i ,a i ) plays a non-negligible rolein the standard deviation p var [ r i ( s, a )] . Therefore, the impact of agent i in theindividual mean-field term D ( s i ,a i ) cannot be neglected. Example 2 (Aggregative games)
The payoff function of each player dependson its own action and an aggregative term of the other actions. Example of pay-off functions include r i ( a i , P j = i a αj ) , α > and r i ( s i a i , P j = i s j a j ) .
13n the non-atomic setting, the influence of an individual state s j and indi-vidual action a j of any user j will have a negligible impact on mean-field term D ( s,a ) . In that case, one gets to the so-called mean-field game.
Example 3 (Population games)
Consider a large population of agents. Eachagent has a certain state/type s ∈ S and can choose a control action a ∈ A ( s ) . Let m the proportion of type-action of the population. The payoff of the agentwith type/state s, control action a when the population profile m is r ( s, a, m ) . Global games with continuum of players is based on the Bayesian games anduses the proportion of actions (mean-field of actions).
In the case where both non-atomic and atomic terms are involved in thepayoff, one can write the payoff function as r j ( s, a, D, ˆ D ) where ˆ D is the pop-ulation state-action measure. User j may influence D j (distribution of its ownstate-action pairs) but its influence on ˆ D may be limited. Empathic payoff
The instant empathic payoff of i is r λi ( s, m s , m a , a ) := r i + X j ∈N i \{ i } λ ij r j . Selfish λ ij = 0 PartiallySpiteful λ ij < λ ij > i towards j for different sign values of λ ij . • Selfishness: If λ ij = 0 we say that i is empathic-selfish towards j. Player i is self-regarding if λ ij = 0 for all j = i. If all the λ ij are zeros for every i, j then every player focuses on her own-payoff functions. • Partially Altruistic: If λ ij ∈ (0 ,
1) we say that i is partially empathic-altruistic towards j. If all the λ ij are positive for every i, j every player isconsidering the other players in its decision in a partially altruistic way. • Partially Spiteful/Malicious: If λ ij < i is partially empathic-spiteful towards j. If all the λ ij are negative for every i, j every player isconsidering the other players in her decision in a partially spiteful way.14 Mixed altruism-spitefulness-neutrality: The same player i may have dif-ferent empathetic behaviors towards her neighbors. If λ ij > , λ ik < λ il = 0 for j, k, l ∈ N i \{ i } then player i is partially altruistic towards j, and partially spiteful towards k and neutral towards l. Reciprocity payoff
Define i ’s kindness to player j as κ ij ( a i , ( b ij ) j = i ) = r j ( a i , ( b ij ) j = i ) − " sup a ′ i r j ( a ′ i , ( b ij ) j = i ) + inf a ′ i r j ( a ′ i , ( b ij ) j = i ) where r j ( a i , ( b ij ) j = i ) the material payoff that player i believes that player j willreceive. We say that i is kind to j if κ ij > . i is unkind to j if κ ij < . b ij is i ’s belief on player j ’s strategy. In order to define reciprocity, we introduce asecond order reasoning. Let ˜ b ijk is k ’s belief about others.The reciprocal perceived kindness of j towards i is˜ κ iji = r i ( b ij , (˜ b ijk ) k = j ) − " sup b ′ ij r i ( b ′ ij , (˜ b ijk ) k = j ) + inf b ′ ij r i ( b ′ ij , (˜ b ijk ) k = j ) which is what i believes that j believes that i will receive. If ˜ κ iji > i perceives that j is kind to him.The empathetic reciprocity payoff is r λi ( a, b, ˜ b ) = r i + P j ∈N i \{ i } λ ij κ ij ˜ κ iji , where λ ij is i ’s reciprocity sensitivity towards j. If λ ij > κ ij . ˜ κ iji > m a k = b jk = ˜ b ijk . Consider a dynamic mean-field-type game setup with the following data:
Time step: t ≤ T Set of Players: { , . . . , n } Initial state : s ∼ m State dynamics: s t +1 ∼ q t +1 ( . | s t , m st , m at , a t )Instant material payoff of i : r it ( s t , m st , m at , a t )Terminal material payoff of i : g iT ( s T , m sT )Instant psychological payoff of i : r λit ( s t , m st , m at , a t )Terminal psychological payoff of i : g λiT ( s T , m sT )Degree of empathy/reciprocity of i : λ i = ( λ ij ) j where T is the duration of the interaction, a t = ( a t , . . . , a nt ) =: ( a it , a − i,t )represents a control-action profile of all players at time t. a i,t ∈ A i , the space15f actions of i at time t, m st is the distribution of state at time t , m at is thedistribution of actions at time t. Definition 2 (Behavioral pure strategy)
A behavioral pure strategy of player i at time t is a mapping from the available information to the set of actions.The set of pure strategies of i is denoted by A i . Player i ’s cumulative empathic payoff is R λi ( m s , a ) = E T − X t =0 r λit ( s t , m st , m at , a t ) + g λiT ( s T , m sT ) . Next we define the response of a player to the others and the mean-field.
Definition 3 (Best response)
A strategy a i of player i is a best-response to ( a − i , m a − i ) if R λi ( a ) = sup a ′ i R λi ( a ′ i , a − i ) . The set of best response strategies of player i defines a best response correspon-dence BR λi . The existence of a pure best-response strategy can be obtained in numberof classes of games. When a pure best response strategy fails to exist, one canuse behavioral mixed strategies. Using weak compactness of the set of probabil-ities on A i , the existence of mixed behavioral best response can be establishedfollowing standard assumptions. Next, we define a mean-field equilibrium. Definition 4 (Mean-field equilibrium)
A strategy profile a generates a mean-field equilibrium if for every player i, the strategy a i of i is best-response to theothers’ strategies a i ∈ BR λi ( a − i ) , and it generates a consistent distribution. The existence of mean-field equilibria is not a trivial task. Sufficiency con-ditions for existence of equilibria can be obtained using fixed-point theory. Todo so, we provide an optimality system for empathic mean-field-type games.
Let the expected empathic payoff in terms of the measure m t .Er λit ( s t , m st , m at , a t ) = Z r λit (¯ s, m st , m at , a t ) m st ( d ¯ s )= ˆ r λit ( m t , a t )where ˆ r λit depends only on the measure m t and the strategy profile a t . Similarlyone can rewrite the expected value of the terminal payoff as Eg λiT ( s T , m sT ) = Z g λiT (¯ s, m sT ) m sT ( d ¯ s ) = ˆ g λiT ( m sT ) . roposition 1 On the space of measures, one has a deterministic dynamicgame problem over multiple stages. Therefore a dynamic programming principle(DPP) holds: ˆ v λit ( m st ) = sup a ′ i (cid:8) ˆ r λit ( m t , a ′ it , a − i,t )+ˆ v λi,t +1 ( m st +1 ) (cid:9) m st +1 ( ds ′ ) = R s q t +1 ( ds ′ | s, m st , m at , a t ) m st ( ds )This optimality system extends the works in [13, 14, 16, 15, 17] to the mean-field-type game case. Note, however that one cannot directly use DPP with thestate ( s, Em s , Em a ) because of non-Markovian structure. It turns out that onecan map the state dynamics to the measure dynamics, and the measure shouldbe the state of the DPP. Proposition 2
Suppose a sequence of real-valued function ˆ v λit , t ≤ T definedon the set of probability measures over S is satisfying the DPP relation above.Then ˆ v λit is the value function on P ( S ) starting from m t = m. Moreover if thesupremum is attained for some a ∗ i ( ., m ) , then the best response strategy is in(state-and-mean-field) feedback form. The equilibrium payoff is R λi ( a ∗ ) = ˆ v λi ( m ) . Proposition 2 provides a sufficiency condition for best-response strategiesin terms of ( s, m st ) . The proof is immediate and follows from the verificationtheorem of DPP in deterministic dynamic games.
Suppose that the state space S and the action spaces are nonempty and finite.Let the state transition be P ( s t +1 = s ′ | s t , m st , m at , a t ) = q t +1 ( s ′ | s t , m st , m at , a t ) , DPP becomes ˆ v λit ( m st ) = sup a ′ i (cid:8) ˆ r λit ( m st , a ′ it , a − i,t )+ˆ v λi,t +1 ( m st +1 ) (cid:9) m st +1 ( s ′ ) = P s ∈ S q t +1 ( s ′ | s, m st , m at , a t ) m st ( s ) Proposition 3
A pure mean-field equilibrium may not exist in general. By ex-tending the action space to the set of probability measures on A and the functions ˆ r λit , ˆ g λiT , q t +1 one gets the existence of mean-field equilibria in behavioral (mixed)strategies. .4.2 Continuous state space Consider the state dynamics s t +1 = s t + b t +1 ( s t , m st , m at , a t , η t +1 )where η is a random process. The transition kernel of s t +1 given s t , m st , m at , a t is q t +1 ( ds ′ | s t , m st , m at , a t ) = Z η P ( ds ′ ∋ s t + b t +1 ( s t , m st , m at , a t , η )) L η t +1 ( dη )where L η t +1 ( dη ) denotes the probability distribution of η t +1 . If r i ( s, a, m s , m a ) = r i ( s, a ) and g i ( s, m s ) = g i ( s ) for every player i thenˆ r λi ( m t , a t ) = Z s r λi ( s, a t ) m t ( ds ) . There exists a function v λi such thatˆ v λi ( m t ) = h v λi , m t i = Z s v λi ( s ) m t ( ds ) ,v λi ( s ) is a mean-field free function. In this case, the mean-field-type dynamicprogramming reduces to v λit ( s ) = sup a ′ it H λi ( s, a ′ it , a − i,t ) , where the Hamiltonian is H λi = r λit ( s, a ′ it , a − i,t ) + Z s ′ v λi,t +1 ( s ′ ) q t +1 ( ds ′ | s t , a ′ it , a − i,t )We retrieve the classical Bellman operator in the mean-field-free case. The following result holds:
Proposition 4
Empathy-altruism reduces equilibrium payoff inequality gap andimproves fairness. As λ increases towards the equilibrium payoff gap betweenplayers i and j, ˆ v λi − ˆ v λj decreases. .6 Variance Reduction Problem over a Network Consider a common state dynamics between players represented by a stochasticdifference equation. The material cost functional of player i is L i ( a ) = q iT s T + ¯ q iT ( E [ s T ]) + P T − t =0 q it s t + ¯ q it ( E [ s t ]) + c it a it . (2)which is composed of a terminal material cost q iT s T + ¯ q iT ( E [ s T ]) and a runningmaterial cost of q it s t + ¯ q it ( E [ s t ]) + c it a it . The coefficients q i , ¯ q i , c it are assumedto be positive real numbers.The players interact through the common state s which influences the ma-terial cost function. This is a dynamic mean-field-type game where the playersare not necessarily indistinguishable because the coefficients q i , ¯ q i , c i , b i , may bedifferent from one player to another. Moreover, each player i influences themean-field term E [ s ] through its control a i . In this model, the contribution of asingle player (say i ) in the mean-field term E [ s ] cannot be neglected. Let N i bethe set of players that are neighbors of player i. Player i is empathic-altruistictowards her neighbors. The empathic cost functional of i is L λi ( a ) = L i ( a ) + X j ∈N i \{ i } λ ij L j ( a ) . We have omitted the term that is not controlled by i : X j ∈N i \{ i } λ ij c jt a jt inf a i ∈A i E [ L λi ( a , . . . , a n )] subject to s t +1 = n αs t + ¯ αEs t + P nj =1 b j a jt o + σW t ,s ∼ L ( S ) , E [ S ] = m (3)given the strategy ( a j ) j = i of the others’ players.Let t ∈ { , . . . , T − } be the time step, q λjt = P i ∈N j [ q jt + λ ji q it ] ≥ , ( q λjt +¯ q λjt ) ≥ , c jt > , and given linear state-and-mean-field feedback of the otherplayers, the problem (3) has a unique best-response of player i and it is givenby 19 a λit = η it ( s t − Es t ) + ¯ η it Es t ,η it = − [ αb i β i,t +1 + b i β i,t +1 P j = i b j η jt ] c it + b i β i,t +1 , ¯ η it = − b i γ i,t +1 ( α +¯ α + P j = i b j ¯ η j,t ) c it + b i γ i,t +1 ,β it = q λit + β i,t +1 { α + 2 α P j = i b j η jt + [ P j = i b j η jt ] }− [ αb i β i,t +1 + b i β i,t +1 P j = i b j η jt ] c it + b i β i,t +1 β iT = q λiT ≥ γ it = ( q λit + ¯ q λit ) + γ i,t +1 ( α + ¯ α + P j = i b j ¯ η jt ) − ( b i γ i,t +1 ( α +¯ α + P j = i b j ¯ η jt )) c it + b i γ i,t +1 γ iT = q λiT + ¯ q λiT ≥ i is E [ L λi ( a )] = Eβ i ( s − Es ) + γ i ( Es ) + T − X t =0 β i,t +1 σ . We examine the effect of λ on the mean state Es t . Let the real numbers α, ¯ α, b i and ( Es ) be nonnegative. Es λt +1 = ( Es ) t Y k =0 [ α + ¯ α + n X i =1 b i ¯ η ik ] . It follows that γ λ ≥ λ and the coefficient ¯ η ik decreases with λ. We conclude that the empathy-altruism parameter helps to lower the meanstate while helping the others in their variance reduction problem.
This section presents an experimental evidence of psychological factors in users’behaviors for data forwarding in Device-to-Device (D2D) communications.
The explosion of wireless applications creates an ever-increasing demand formore radio spectrum. The presence of Device-to-Device -enabled mobile usersdefines an extended network coverage and its co-existence with device-to-infrastructurenetworks is not without challenges. In this context, each device can move inde-pendently, and will therefore change its links to other devices frequently due toconnectivity issues. Relay-enabled wireless device may be requested to forwardtraffic unrelated to its own use, and therefore be a temporary router or a relay.If the receiving device can also play the role of relay then it will forward the datato the next hop after sensing the channel again. For given routing path, the data20eed to be forwarded at each intermediary hop until the end destination. Theintermediary nodes are relays or regular nodes that are willing to forward. Ifmany nodes are participating in the forwarding process, every node can benefitfrom that service, and hence it is a public good, which we refer to as mobilecrowdforwarding . By analogy with crowdfunding, crowdsourcing, crowdsensing,the concept of mobile crowdforwarding consists to call for contributors (mobiledevices) who are willing to forward data in mobile ad hoc networks by meansof incentive schemes. However, most of the current smart devices are battery-operated mobile devices that suffer from a limited battery lifetime. Hence, auser who is forwarding a data needs also to balance with the remaining energyby limiting the energy consumptions. When decision-makers are optimizingtheir payoffs, a dilemma arises because individual and social benefits may notcoincide. Since nobody can be excluded from the use of a public good, a usermay not have an incentive to forward the data of others. One way of solving thedilemma is to give more incentive to the users. It can be done by slightly chang-ing the game, for example, by adding a second stage in which a reward (fair) canbe given to the contributors (non-free-riders). Consider n transmitter-receiverpairs in a Device-to-Device (D2D) communication, with n ≥ . Player i ’s actionspace is { nF, F } , where F means the player is participating in the forwardingprocess of the other players’ data, and nF refers to not forwarding. There is aneed for a critical number m ∗ ≥ r i = Q d i k =1 { SINR hk − hk ≥ β hk } if m > m ∗ , a i = nF Q d i k =1 { SINR hk − hk ≥ β hk } − m ∗ m α if m > m ∗ , a i = F m < m ∗ , a i = nF − m ∗ m γ if m < m ∗ , a i = F where α > , γ > , m := P nj =1 { a j =1 } and t i = h , h h , . . . , h l − h d i a21ultihop path from the transmitter of i to the end-to-end destination d i . Wedenote by p i := Q d i k =1 { SINR hk − hk ≥ β hk } . Nash equilibria of the game with material payoff
If players are only interested in their material payoff, it leads to the situationwhere no one would participate in the forwarding of everyone else data. This isan equilibrium because it is not beneficial to forward the data when no else isforwarding. Moreover the deviant to pay the cost − m ∗ α as a single deviator. Nash equilibria of the game with empathy
One single deviant does not induce a big degradation If λ ij > i and j then the empathetic payoff of i with a i = F when thenumber of cooperators exceeds m ∗ + 1 is p i − m ∗ m α + X j ∈N i \{ i } λ ij ( p j − m ∗ m α ) + X j ∈N i λ ij p j and the empathetic payoff for a i = nF becomes p i + X j ∈N i λ ij ( p j − m ∗ m α ) + X j ∈N i \{ i } λ ij p j In this case a single deviant does not induce big degradation in the payoff.
One single deviant limits the performance of the network If λ ij > i and j then the empathetic payoff of i with a i = F when thenumber of cooperators is m ∗ is p i − m ∗ m α + X j ∈N i \{ i } λ ij ( p j − m ∗ m α ) + X j ∈N i λ ij p j and the empathetic payoff for a i = nF (i.e. m = m ∗ − < m ∗ ) becomes0 + X j ∈N i λ ij ( p j − m ∗ m α ) + X j ∈N i \{ i } λ ij p j If p i − m ∗ m α > m ∗ and the user can take the advantage ofthe public good. Nash equilibria of the game with reciprocity
If there are enough cooperators, the kindness function yields − m ∗ m α when a i = nF and + m ∗ m α when a i = F. Similarly, if the number of cooperators is below m ∗ then the kindness function yields − m ∗ m γ when a i = nF and + m ∗ m γ when a i = F. m ∗ cooperators does induce a positive payoff to the usersthanks to the kindness and reciprocity of the some of the others and maintainthe public good. Thus, altruism and specially reciprocity of the nodes mattersin the forwarding process. This is practically observed in the experiment below. In order to understand the effect of empathy on the behavior of people choicebehind the machine, we have conducted an experimental test at NYUAD Learn-ing & Game Theory Laboratory. We consider a sample population of 47 peoplecarrying wireless devices with 19 men and 28 women, with different cultures,and nationalities and from 18 to 40 years old. Participants include engineers,psychologists, students, non-students and professional staff members. In orderto quantify the degree of empathy, a multidimensional index measure (Interper-sonal Reactivity Index [33, 36]) is used for each member of the population. Aparticipant can move around and may be within a D2D-enabled area if there isa device within a certain range as illustrated in Figure 8. In order to setup aD2D communication network a crucial step is the approval from the users: theirdecision to cooperate or not in forwarding the data of other devices.Figure 8: D2D and WiFi-Direct enabled technology areaEach person carrying a mobile device was invited to fill a form on its choicein the forwarding dilemma when facing different configurations. Due to the ran-domness in wireless channel communications, the forwarding problem becomesa game under uncertainty. In the forwarding game, the realized payoffs are influ-enced by the actions of the wireless devices and a random variable representingthe channel state. Such games are called Random Matrix Forwarding Games.Given random payoff matrices, the question arise as what is meant by playing23he random matrix game (RMG, [1]) in an optimal way. Because now the actualpayoff of the game depends not only on the action profile picked by the wirelessdevices but also on the sample point of realized state of the channel. Thereforethe devices cannot guarantee themselves a certain payoff level. The wirelessdevices will have to gamble depending on the channel state. The question ofhow one gambles in an optimal way needs to be defined. Different approacheshave been proposed: expectation approach, variance reduction, mean-varianceapproach, multi-objective approach. The signal-to-interference-plus-noise ratio(SINR) for transmission from node S1 to S2 is given by
SIN R S S = p | h | N + I S , where p > S , N > h S S is channel state between S1 and S2, and I S ≥ { SINR S S ≥ β } indicates the indicator functionon the event { SIN R S S ≥ β } , i.e., it is equal to 1 if SIN R S S ≥ β and 0otherwise. Let m = 1l { SINR S S ≥ β } . { SINR S D ≥ β } ,n = 1l { SINR S S ≥ β } . { SINR S D ≥ β } ,n = 1l { SINR S S ≥ β } . { SINR S D ≥ β } ,m = 1l { SINR S S ≥ β } . { SINR S D ≥ β } . Since h = ( h S S , h S S , h S D , h S D ) is a random vector, the coefficients m , n , n , m are random. This leads to a random matrix forwarding gamebetween wireless devices S1 and S2 as described in Table 2.S1 \ S2 F nFF ( m − c , n − c ) ( − c , n ) nF ( m , − c ) (0 , h. We denote by a ij := E [ m ij ]and b ij := E [ n ij ] . Then expected payoff matrix is given by Table 3.S1 \ S2 F nFF ( a − c , b − c ) ( − c , b )nF ( a , − c ) (0 , No empathy implies no network in most interesting cases
We analyze the normal form game of Table 3. If a − c < a then the rowplayer will not forward, and hence the column player as well. This leads to24ash equilibrium strategy ( nF, nF ) . If a − c > a then the row player willforward, and the column player will forward if b − c > b leading to ( nF, nF ),else if b − c < b then the equilibrium is ( F, nF ) . Similar reasoning can beconducted by inverting the roles. Thus, when taking into consideration thepower-limited of the mobile devices, the classical material payoff analysis leadsto the outcome ( nF, nF ) i.e., non-cooperation between the mobile users, and noforwarding implies no network.
Effect of empathy on the forwarding decision
Now we involve possible empathetic situations. Two contexts were availablein the game situation. The first context is a situation where the two personsinvolved in the game are friends. The second context is when they do not knoweach other (they meet for example during in their way in the public transporta-tion, and do not have an a priori relationship. The empathy measure used forthe experiment is the so-called the Interpersonal Reactivity Index (IRI) whichcomprised of four scales: empathic concern (EC), perspective taking (PT), fan-tasy scale (FS) and personal distress (PD). • The EC scale aims to assess the affective outcomes, the tendency to expe-rience other-oriented feelings and the response to distress in others withthe reactive response of sympathy and compassion. • The PT scale aims to measure the process of role taking, the tendency toadopt the psychological points of view of others. • The PD scale demonstrates an affective outcome, and is designed to tapones’ own feelings of personal unease and discomfort in reaction to theemotions of others. • the FS aims to measure the tendency to transpose oneself into feelingsand actions of fictitious characters. Participants were run individually, although they were led to believe that an-other person was also taking part. The experimenters explained to each partic-ipant that the study involved two participants, and that they were being placedtemporarily at different places. The experimenters then escorted the partici-pant to the NYUAD Learning & Game Theory Laboratory left her alone toread a written instruction that allows us the measure its empathy subscales,followed by another instruction on the packet forwarding and participation intoD2D technology. The test also distinguishes the gender of the participant, inorder to make a refined study with several types and subpopulations. Afterparticipants read the questionnaire (see Table 10), the experimenter answeredany questions, and informed them that they and the other participant in thesession had been randomly assigned to and the experimenter returned. If care-fully filled, the instructions reveal a significant empathy scale, the latency per25uestion and the decision of the participant in the forwarding game in two dif-ferent situations: close relationship with other participant that was fictitiousin the test or no prior relationship with the participant. All participants havewireless devices that have the capability in enabling WiFi direct and D2D tech-nology when the users decide to do so. They have the possibility in acceptingor rejecting (to enable or to disable) to forwarding the data of the others.
In the men population only two questions have been left in IRI, with a 99.63%of responsiveness to the four different scales. In women population we had threequestions that have been left in IRI, with a 99.62% of responsiveness to the samefour different scales. awful bad average good excellent448501930 Empathy Scale Quality: Perspective Taking (PT) T o t a l S c o r e o f A n s w e r s WomenMen F FnF nFF nFnF1916410 Outcomes of the Data Forwarding Game N u m b e r o f ” y e s ” WomenMen • Women population: Player IF nFPlayer II F 19 16nF 4 16A more refined version of the cooperators among the women populationwith FnF/nFF outcomes (15/28) is obtained:Player IF nFPlayer II F 10 1nF 5 13 • Men population: Player IF nFPlayer II F 11 8nF 3 13A more refined version of the cooperators among men population with FnF/nFFoutcomes shows (6/19) proportion of cooperators. • Although they read identical notes, we expected that participants who hadclose relationship would experience more empathy for the other participantthan would participants who do not each other and never met before(either virtually or physically). We checked this expectation with the self-reports of IRI response that participants made after reading an alternativequestion on what would be their decision if they do not know the otherparticipant. It turns out that only 1 person (out of 47 people) will change27heir opinion if the other user is unknown to them. Thus, both empathyand closiness affect the decision-making of the users. • Our second observation is that the experiment exhibits a strong correlationbetween the scale of IRI and the choice of people. Figure 9 illustrates arelationship with PT scale and percentage of cooperators.Figure 9: Impact of positive empathy (PT) in the decision-making of the peopleUsing total probability theorem we obtain P ( F ) = P ( F | P (1) + P ( F | P (0) , where P ( F | i ) is the conditional probability of forwarding the data of theothers (cooperation) assuming i. We use the sample statistics to computethe probability to cooperate through the number of occurrences of F. • Deviation to the material payoff outcomes: What if participants in a one-shot prisoner’s dilemma game know before making their decision that theother person has already decided not to forward (defected)? From the per-spective of classic game theory with material payoff, a dilemma no longerexists because of dominating strategy. It is clearly in their best interestto defect too. The empathy-based test predicts, however, that if some ofthem feel empathy for the other, then a forwarding dilemma remains: self-interest counsels not to forward (defection); empathy-induced behaviormay counsel not. Based on the experiment we have look at the outcomes(
F, nF ) and ( nF, nF ) from the choices of 47 participants. Among thosenot induced to feel empathy, very few (3/47) did not defect in return.Among those induced to feel empathy for the other, (26/47) did not de-fect. These experimental results highlight the power of empathy-inducedbehavior to affect decisions in one-shot forwarding dilemma game. • This experimental test reveals that empathy seems far more effective thanmost other techniques that have been proposed to increase cooperation inone-shot games.Based on these experimental results, we believe that the idea of using (pos-itive) empathy to increase cooperation in a one-shot forwarding dilemma andmore generally in a public good games should be explored in more details.28an we use psychological payoff functions to explain the behaviors observedin the experiment?To answer this question we introduce a psychological payoff that is not onlyself-interested but also other-regarding through the two random variables λ and λ . S1 \ S2 F nFF ( m λ , n λ ) ( m λ , n λ ) nF ( m λ , n λ ) (0 , m λ = m − c + λ ( n − c ) , n λ = λ ( m − c ) + n − c m λ = − c + λ n , n λ = − λ c + n m λ = m − λ c , n λ = λ m − c • Case 0: In absence of empathy: λ = 0 , λ = 0 corresponds to the self-regarding payoffs case. The game leads to the outcome (nF,nF) when m − c < m and n − c < n . • Case 1: λ > , λ > – FF: If m λ ≥ m λ or n λ ≥ n λ then the strategy nF is not dominat-ing anymore and in this case, forwarding is a good candidate for Nashequilibrium of the psychological one-shot forwarding game. In addi-tion, if m ii ≥ λ ≥ c + m − m n and λ > c + n − n m then fullcooperation ( F, F ) becomes a Nash equilibrium. If c + m − m n and c + n − n m belongs to (0 ,
1) and then a mixed strategy equilibriumemerges in addition to the pure ones, which explains the observedvariation of percentages of cooperators depending on the empathyindex measured from the experiment. – FnF is an equilibrium if m λ ≥ n λ ≥ n λ . This means that λ n ≥ c , λ m + n − n − c ≤ , i.e., λ positively high enoughand λ is low. – Similarly when λ is low and λ positively high enough then nF F becomes an equilibrium. – nFnF is an equilibrium when { m λ ≤ , n λ ≤ } which means λ n ≤ c , λ m ≤ c . – If λ ∈ ( c + m − m n , c n ) and λ ∈ ( c + n − n m , c m ) then there arethree equilibria: FF, nFnF and a mixed equilibrium. – If λ > c n and λ > c m then FF is the unique equilibrium because F is a dominating strategy for both users.29 If both users have low empathy λ < c + m − m n and λ < c + n − n m then nF is a dominating strategy for both users, hence nF nF is anequilibrium. • Case 2: λ < , λ < • Case 3: λ > , λ < nF nF otherwise. At the threshold value of λ such that m λ = 0 , everypartially mixed strategy profile ( yδ F + (1 − y ) δ nF , nF ) with y ∈ [0 ,
1] isan equilibrium. • Case 4: λ < , λ > nF nF otherwise. -1/114 0 1/114 2/3 1-1/11401/1142/31 highmedium λ λ mediumnegativelow lownegative nFnF F nF FFF high nFF F FF FF nF nF nF nF nF nF F pF + (1-p)nF nFnFnF nF F nFnFnFnF Figure 10: Outcome based on empathy distributionWhen the parameters lead to an equality of payoff, there may be infinitenumber of (mixed) equilibria. We have omitted these degenerate cases since λ will be a continuous random variable (Figure 10 and table 5). Table 5 summa-rizes the outcomes when the entries m, n are non-zero depending on the affectiveempathy level of the users: negative (spiteful), low (positive), medium, and highwhen c + m − m n < c n and c + n − n m < c m . This experimental test reveals a distribution of empathy across the pop-ulation of men and women (see Figure 13). Thus, a natural question is the30ser 1 \ User 2 λ Negative Low Medium High λ High FnF FnF F F FF λ Medium nFnF nFnF FF,nFnF, p F+(1-p)nF FF λ Low nFnF nFnF nFnF nFF λ Negative nFnF nFnF nFnF nFFTable 5: Summary of the outcomes. For user 1, Low empathy means λ ∈ (0 , c + m − m n ) , Medium empathy means λ ∈ ( c + m − m n , c n ) and High em-pathy means λ > c n . For user 2, low empathy means λ ∈ (0 , c + n − n m ) . Medium empathy means λ ∈ ( c + n − n m , c m ) and High empathy meansmeans λ > c m . probability to endup with FF as an outcome when people are drawn from theempathy population sampling distribution over [ − , . Below we examine the extreme cases with two types: High PT and Se dis-tributed according to (1 − µ, µ ) for some µ ∈ (0 , . The resulting interactiondepends on the type of the users carrying the wireless nodes. We denote by
P T a high level of positive empathy and by Se a user with a very low level ofempathy. PT \ PT FF ( m − c , m − c ) ∗ PT \ Se F nFF ( m − c , m − c ) ( − c , m )Se \ PT FF ( m − c , m − c )¯ F ( m , − c )Se \ Se F nFF ( m − c , m − c ) ( − c , m ) nF ( m , − c ) (0 , • In a PT-PT interaction, the equilibrium structure is to forward wheneverthe channel is good enough. • In a Se-PT or PT-Se interaction, the selfish wireless node has to compare.In a Se-PT interaction, if m − c > m then the selfish node 1 willchoose F otherwise will not forward ( nF ). The equilibrium structure ofSe-PT interaction is – ( F, F ) if m − c > m , ( nF , F ) if m − c < m , – (any mixed strategy , F ) if m − c = m Similarly, the equilibrium structure of PT-Se interaction is – ( F, F ) if m − c > m , – ( F, nF ) if m − c < m , – ( F, any mixed strategy) if m − c = m , • In a Se-Se interaction, the equilibrium structure is described in a similarlyway as in S1-S2 in Table 3.Note that when we put the pure strategies together in a population contextwhere (1 − µ ) fraction of the people are empathic PT and µ ∈ (0 ,
1) fractionare selfish nodes, the resulting outcome is well-mixed of (
F, F ) , ( F, nF ) , ( nF, F )and ( nF, nF ) , which strengthen the observations of the experiment. It alsoprovides the possibility to observe the Bayesian Hannan set [3] or Bayesiancoarse correlated equilibria in experimental one-shot games. Screen shots of the Empatizer app are given in Figures 11 and 12. As expected,the classification is incomplete and the subscales are not statistically indepen-dent. They are correlated and possibly overlapping. This is represented inFigure 13, Table 7.
Scale Type Women Men Total
PT 5 3 8EC 2 - 2FS 3 3 6PD 2 4 6PT - PD 1 1 2FS - PD 2 - 2EC - PD - 3 3EC - FS - 1 1PT - EC - PD - 1 1EC - FS - PD 1 - 1PT - EC - FS - PD - 1 1Other scale 12 2 141 1 2Participants 28 19 47Table 7: IRI scale distribution across the populationIn Table 10 we have completed the subscales PD, FS, EC. We observe thatboth positive and negative correlation between cooperation and empathy sub-scale can be obtained from the experiment, in particular for PD. This also reveals32igure 11: Empathizer measures the multidimensional empathy of each partic-ipant.a kind of spitefulness behavior. However, it is a mixture of several things. forexample, if player i is spiteful towards j and j is PT, the resulting outcome isunclear. 33igure 12: Empathizer sample result. awful bad average good excellent694922481830 Empathy Scale Quality: Empathy Concern (EC) T o t a l S c o r e o f A n s w e r s WomenMen Figure 13: Empathy scale distribution across the population of participants awful bad average good excellent16322629430 Empathy Scale Quality: Personal Distress (PD) T o t a l S c o r e o f A n s w e r s WomenMen wful bad average good excellent324437430 Empathy Scale Quality: Fantasy (FS) T o t a l S c o r e o f A n s w e r s WomenMen
In Table 8 we compute the correlation between the subscale of empathy fromthe data collected from the participants.Pearson correlation PT EC FS PDPT - -0,3462FS - - - -PD - - - -Table 8: subscale correlationCooperation levelPT + EC 50%PT + FS 62,5%PT + PD 66,66 %EC + FS 75%EC + PD 50%Table 9: Level of cooperation mixed scalesThe more refined result on the level of cooperation corresponding to themixed IRI scale is given in Table 9. We can observe that a high level of coop-eration associated to a high correlation coefficient correspond to the Empathy-Altruism behavior (namely PT + FS and EC + FS ). A high level of coopera-tion associated to a negative or low correlation coefficient correspond to a sort36f empathy-spitefulness behavior (namely PT + PD, EC + PD). In particular,the usage of empathy could be different across of the population and it is alwaysin a positive sense.The experiment reveals also that “other” scale of empathy may be useful:(i) involvement of the users in technology is different across of the population[42], and (ii) empathy anger may be correlated with the fact that some peopleare helping and some others have punishing desires [41]. We leave these refinedempathy concepts for future investigation.
Mean-field-type game theory is an emerging interdisciplinary toolbox with whichone can describe situations where multiple persons make decisions and influenceeach other with state, type, actions and distributions of these. Psychologicalmean-field-type game theory is as an extension of those methods, with which onecan design, analyze, identify how various psychological aspects, which classicalmodels typically do not take into account, affect behaviour of the decision-makers. One can, for example, study the importance of empathy and emotions,for example reciprocity, disappointment, regret, anger, shame, and guilt, thatis the propensity to return favors, take revenge or being malicious or spiteful.Prior works on mean-field-type games have assumed that people’s behavior ismotivated solely by their own material payoff. Other aspects of motivation, forexample the empathy and emotions, have been disregarded. But this is a majordrawback as empathy and emotions often influence behaviour and outcomes.In this paper, we have proposed and examined the role of empathy in mean-field-type games. We established optimality system for such games when em-pathic player are involved. It is shown that empathic-altruism helps in reducingcollision channel, securing the mean state and reducing electricity peak hours.Empathy-spitefulness of prosumers lowers electricity price and hence it helpsconsumers. Empathy-altruism reduces inequality between the payoffs in mean-field-type games. The experiment with 47 people carrying mobile devices hasdemonstrated that using WiFi direct, D2D or other relaying technology on cellphones, tablets and laptops while moving or being in downtown or at airportdegrades performance if the number of cooperators is not sufficient enough, in-creasing the response time of the servers, particularly among the users who arefar to the access points, and can lead to interruption and call blocking. Thereis a need of coalition among a certain of number of nodes to maintain a mini-mum connectivity level. The users’ who are aware of such a situation may beempathetic. However, empathy can be used in different directions and differentstrategic ways: self-regarding, other-regarding, mutual-regarding, spitefulness,and indirect network effect etc. The experiment reveals that more cooperationcan be observed even in one-shot games when users’ are empathetic, and thisholds in both women population and men population.Number of questions remain unanswered: (i) it would be interesting to ex-amine the formation and the evolution of empathy as time goes, for example by37eans of learning process. (ii) time delayed empathy and forgiveness. We havepresented some of the extremes situations to illustrate clearly the influence ofpsychological factors. However, the interaction in engineering games is not lim-ited to empathy-altruism and empathy-spitefulness. There are multiple factorsand multiple possibles cross-factors: one simple behavior to examine is the effectin the network when a user i is helping user j but not user k and j is helping k but not l etc. It is unclear who is helping whom in the multi-hop networkthrough to the indirect path. We leave these issues for future investigation. References [1] M. A. Khan, H. Tembine, Random matrix games in wireless networks, IEEEGlobal High Tech Congress on Electronics (GHTCE 2012), November 18-20, 2012, Shenzhen, China.[2] G. Rossi, A. Tcheukam and H. Tembine, How Much Does Users’ Psychol-ogy Matter in Engineering Mean-Field-Type Games, Workshop on GameTheory and Experimental Methods June 6-7, 2016, Second University ofNaples, Department of Economics Convento delle Dame Monache, Capua(Italy)[3] J. Hannan. Approximation to Bayes risk in repeated play. In M. Dresher,A. W. Tucker, and P. Wolfe (Eds.), Contributions to the Theory of Games,Vol. III, Ann. Math. Stud. 39, pp. 97-139. Princeton Univ. Press, 1957.[4] Preston, S.D., & de Waal, F.B.M. (2002). Empathy: its ultimate and prox-imate bases. Behavioral and Brain Sciences, 25(1), 1-71.[5] Jan Grohn, Steffen Huck, Justin Mattias Valasek, A note on empathy ingames, Journal of Economic Behavior & Organization, Volume 108, De-cember 2014, Pages 383-388,[6] Camerer CF (2003). Behavioral Game Theory. Princeton: Princeton Uni-versity Press.[7] H. Tembine: Psychological mean-field-type games, Preprint, 2017.[8] H. Tembine: Mean-field-type games, Preprint, 2017.[9] Page K, Nowak M (2002), Empathy leads to fairness. Bull Math Biol 64:1101-1116.[10] Pierpaolo Battigalli, Martin Dufwenberg, Dynamic psychological games,Journal of Economic Theory, vol. 144, Issue 1, January 2009, pp. 1-35.[11] G. Attanasi, R. Nagel, A survey of psychological games: Theoretical find-ings and experimental evidence, in: A. Innocenti, P. Sbriglia (Eds.), Games,Rationality and Behaviour. Essays on Behavioural Game Theory and Ex-periments, Palgrave McMillan, Houndmills, 2007, pp. 204-232.3812] J. Geanakoplos, D. Pearce, E. Stacchetti: Psychological games and sequen-tial rationality, Games Econ. Behav. 1 (1989) 60-79.[13] Jovanovic, B. (1982): Selection and the Evolution of Industry, Economet-rica 50, 649-670.[14] B. Jovanovic and R. W. Rosenthal (1988). Anonymous sequential games,Journal of Mathematical Economics, vol. 17, pp. 77-87.[15] D. Bauso, B. M. Dia, B. Djehiche, H. Tembine, R. Tempone (2014), Mean-Field Games for Marriage, PLoS One, 9(5): e94933.[16] Andersson, D. and Djehiche, B. (2010), A maximum principle for SDE’s ofmean-field type.
Appl. Math. Optim.
Proof of Proposition 1:
The evolution of the distribution of states m s under state-and-mean-field feed-back strategies is given by m st +1 ( ds ′ ) = Z s q t +1 ( ds ′ | s, m st , m at , a t ) m st ( ds )This is a deterministic dynamics over { , , . . . , T } . Since the expected payoffcan be rewritten as a function of m s and the action, one can use a classical DPPwith m as a state.Applying the classical dynamic programming principle (DPP) yields ˆ v λit ( m st ) = sup a ′ i (cid:8) ˆ r λit ( m st , a ′ it , a − i,t )+ˆ v λi,t +1 ( m st +1 ) (cid:9) m st +1 ( ds ′ ) = R s q t +1 ( ds ′ | s, m st , m at , a t ) m st ( ds ) . This completes the proof.
Proof of Proposition 3:
We know from classical optimal control theory that a pure optimal strategymay fail to exist in general. However, one can extend the action space to theset of probability measures on A and the underlying functions ˆ r λit , ˆ g λiT , q t +1 canbe extended. Then, the convexity of the action space is obtained. In addition,if the continuity of the Hamiltonian holds then one gets the existence of bestresponse in behavioral (mixed) strategies. Using the multi-linearity property ofthe mixed extension procedure, one can use the Kakutani fixed-point theoremto obtain the existence of equilibria in behavioral (mixed) strategies. Proof of Proposition 4:
We now show that if all the players are empathy-altruistic then the payoff gapis reduced across the entire network. Let λ ij = λ ji = λ ∈ (0 , . Observing that R λi − R λj = R i − R j + X k = i λ ik R k − X k = j λ jk R k (5)= R i − R j − λ ( R i − R j ) (6)= (1 − λ )( R i − R j ) , (7)41hus, from (7) one obtains the following result: If R i − R j = 0 , then theinequality ratio is | R λi − R λj || R i − R j | = 1 − λ < . which completes the proof. 42able 10: IRI subscales. Extension of the empathy measure of Davis 1980,Yarnold et al.1996 and Vitaglione et al. 2003. The star sign (*) denotes anopposite (reversed) counting/scoring.Abridged item Women (60%) Men(40%)PT EC FS PD PT EC FS PD(1) Daydream and fantasize (FS)(2) Concerned with unfortunates (EC) 0.6(3) Can’t see others’ views ∗ (PT)(4) Not sorry for others ∗ (EC)(5) Get involved in novels (FS) 0.8(6) Not-at-ease in emergencies (PD) 0.7(7) Not caught-up in movies ∗ (FS)(8) Look at all sides in a fight (PT) 0.9124 0.2444(9) Feel protective of others (EC) 0.3(10) Feel helpless when emotional (PD)(11) Imagine friend’s perspective (PT) 0.8393 0.824(12) Don’t get involved in books ∗ (FS)(13) Remain calm if other’s hurt ∗ (PD)(14) Others’ problems none mine ∗ (EC)(15) If I’m right I won’t argue ∗ (PT)(16) Feel like movie character (FS)(17) Tense emotions scare me (PD)(18) Don’t feel pity for others ∗ (EC)(19) Effective in emergencies ∗ (PD)(20) Touched by things I see (EC) -0.3452(21) Two sides to every question (PT)(22) Soft-hearted person (EC)(23) Feel like leading character (FS)(44) Lose control in emergencies (PD)(25) Put myself in others’ shoes (PT)(26) Image novels were about me (FS)(27) Other’s problems destroy me (PD)(28) Put myself in other’s place (PT) 0.42Decision outcome Women (60% of the whole population) Men(40%)PT EC FS PD PT EC FS PD(FF) 10/28 7/19(FnF )(nFF)(nFnF) 3/28 6/1943 iography Giulia Rossi received her Master degree with summa cum laude in ClinicalPsychology in 2009 from the University of Padova. She worked as independentresearcher in the analysis and prevention of psychopathological diseases and inthe intercultural expression of mental diseases. Her research interests includebehavioral game theory, social norms and the epistemic foundations of mean-field-type game theory. She is currently a research associate in the Learning &Game Theory Laboratory at New York University Abu Dhabi.
Alain Tcheukam received his PhD in 2013 in Computer Science and Engi-neering at the IMT Institute for Advanced Studies Lucca. His research interestsinclude crowd flows, smart cities and mean-field-type optimization. He receivedthe Federbim Valsecchi award 2015 for his contribution in design, modelling andanalysis of smarter cities, and a best paper award 2016 from the InternationalConference on Electrical Energy and Networks. He is currently a postdoctoralresearcher with Learning & Game Theory Laboratory at New York UniversityAbu Dhabi.