Emergent route towards cooperation in interacting games: the dynamical reciprocity
Qinqin Wang, Rizhou Liang, Jiqiang Zhang, Guozhong Zheng, Lin Ma, Li Chen
EEmergent route towards cooperation in interacting games: the dynamical reciprocity
Qinqin Wang, ∗ Rizhou Liang, ∗ Jiqiang Zhang,
2, 3
Guozhong Zheng, Lin Ma, and Li Chen
1, 4, † School of Physics and Information Technology, Shaanxi Normal University, Xi’an 710062, China School of Physics and Electronic-Electrical Engineering, Ningxia University, Yinchuan 750021, P. R. China Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University, Beijing 100191, P. R. China Robert Koch-Institute, Nordufer 20, 13353 Berlin, Germany
The success of modern civilization is built upon widespread cooperation in human society, deciphering themechanisms behind has being a major goal for centuries. However, a crucial fact is missing in most priorstudies that games in the real world are typically played simultaneously and interactively rather than separatelyas assumed. Here we introduce the idea of interacting games that different games coevolve and influence eachother’s decision-making. We show that as the game-game interaction becomes important, the cooperation phasetransition dramatically improves, a fairly high level of cooperation is reached for all involved games wheninteraction goes to be extremely strong. A mean-field theory indicates that a new mechanism — the dynamicalreciprocity, as a counterpart to the well-known network reciprocity, is at work to foster cooperation, which isconfirmed by the detailed analysis. This revealed reciprocity is robust against variations in the game type, thepopulation structure, and the updating rules etc, and more games generally yield a higher level of cooperation.Our findings point out the great potential towards high cooperation for many issues occurred in the real world,and may also provide clues to resolve those imminent crises like the climate change and trade wars.
PACS numbers: 87.23.Ge, 02.50.Le, 89.75.Fb, 87.23.Kg
Introduction.—
Recent withdrawals of the United Statesfrom a couple of “groups” like WHO, Paris Agreement, UN-ESCO signifies a degraded cooperation at the global scale.Any solution to this sort of problems requires an understand-ing of what processes drive and maintain human cooperationand what measures or institutions could be implemented forits promotion. The key question to be addressed is: why en-tities help each other who could potentially be in competitionand incur a cost to themselves? As the paradigm of homo eco-nomicus shows, people always try to maximize their earningsand avoid irrational investments, which inevitably leads to thetragedy of the commons [1].Important progresses have been made with the help of evo-lutionary game theory [2] by analysing the stylized socialdilemmas such as prisoner’s dilemma and the public goodsgame. Several mechanisms are proposed [3] in the past sev-eral decades, such as reward and punishment [4], social diver-sity [5], direct [6] or indirect reciprocity [7], kin [8] or groupselection [9, 10], spatial or network reciprocity [11]. In par-ticular, theoretically accounting for the fact that human pop-ulations are highly organized and individuals interact repeat-edly with their immediate neighbors can support cooperation[11]. The rationale behind is that a structured neighborhoodfacilitates the formation of cooperator clusters, which effec-tively resist the invasion of defectors, as opposed to the well-mixed scenario. The ensuing years have witnessed a wealth oftheoretical studies that further confirm this so-called networkreciprocity for various population structures [12]. However,recent human behavioral experiments show that, structuredpopulations do not promote cooperation in general [13, 14],at least some conditions combining game parameters and thepopulation structure must be met for cooperation to thrive[15]. One explanation is that the complexities of human psy-chology make humans switch strategies frequently that the as-sortment fails in the static networks [16]. But dynamic net- works indeed offer an escape because the players are allowedto adjust social ties and they are more like to cooperate underthis peer pressure [17, 18]. This unsatisfactory situation im-plies that some essential elements could be missing in currentgame-theoretic models and the experiment-driven modeling isrequired. Note that, in most of these studies a single game isconsidered, and they focus on the factors of interest like thepopulation structure; the conclusions drawn are supposed toapplicable to a wide range of circumstances.Games, however, may not be unfolded in isolation but of-ten in parallel. For instance, we humans are engaged in differ-ent activities, works, sports, and recreations; colleagues couldwork on a couple of concurrent projects; and countries haveto deal with a whole range of conflicts such as trade war, se-curity issues, diplomatic crisis etc. Only when the evolutionof these games is independent from each other, the modelingefforts based on a single game are then reasonable as most ofexisting work assumed. Observations in aforementioned con-texts, however, suggest that the decision-making of entities inone game is often conditioned by what happened in another.Similar observations are also made in biological games, likechimpanzees are more likely to groom their fellows if theyare skillful in hunting, and vice versa, and actually these twobehaviors together with sharing food, joint patrol the borders,support one another in conflicts etc are all correlated [19].A closely related research line is the multigame dynam-ics, the existing work shows that dynamical inconsistenciesare already possible when two or more non-repeated gamesare coupled [20–23], meaning that the eventual fate of gamescannot be inferred from the single game dynamics. A morerecent work starts to study the repeated scenario and an evo-lutionary framework of the so-called multichannel games isproposed [24], where they find that the fixed game linkage isable to enhance cooperation in all games engaged in general.Still, fundamental questions remain: what typical evolution- a r X i v : . [ phy s i c s . s o c - ph ] J a n ary dynamics are expected when more games are engaged, towhat extent would such game-game interaction alter the clas-sic cooperation mode of single game, and any new coopera-tion scenario arises therein? In this Letter, we mainly study two symmetrically interact-ing games, where they have a stake in each other, and focuson clarifying the impact of game-game interaction. We reveala new type of reciprocity rooted in the game-game interactionthat is able to maintain high levels of cooperation. In partic-ular, fairly high cooperation is expected when the interactiongoes to the extreme that the decision-making of a given gameis completely conditioned by the other and vice versa. Themechanism behind lies in the new types of interactions thatlead to a persistent advantage of cooperators. Furthermore,the uncovered reciprocity is found to be quite robust and moregames generally lead to be more cooperative.
Modeling two interacting games.—
Suppose that twogames G = { G , G } are played simultaneously in a popu-lation composed of N players, where they are located on an L × L square lattice with a periodic boundary condition. Theycan adopt one of the two strategies for each game: cooperation( C ) or defection ( D ), i.e. S = { C, D } . Therefore, there arefour possible states S = { XY | CC, CD, DC, DD } in thetwo interacting games, where X, Y represent the state regard-ing game G , respectively. For simplicity, we resort to thepairwise game defined as follows: mutual cooperation bringsboth a reward R , mutual defection leads to a punishment P for each, and mixed encounter yields the cooperator a sucker’spayoff S yet a temptation T for the defector. Their ranking de-termines the game type. Here, we follow the common practicefor a weak prisoner’s dilemma (PD) with R = 1 , P = S = 0 , T = b > for both games if not stated otherwise.Following the standard Monte Carlo (MC) simulation pro-cedure, firstly a game g ∈ G is chosen at random to play in anelementary step, a player i is then randomly chosen and accu-mulates its payoff Π i . Next, one of i ’s neighbors j is pickedrandomly, and acquires its payoff Π j as well. Lastly, player i adopts j ’s strategy regarding game g with a probability ac-cording to the Fermi rule [25] W gj → i = 11 + exp[( (cid:98) Π gi − (cid:98) Π gj ) /K ] , (1) (cid:98) Π G , i,j = (1 − θ )Π G , i,j + θ Π G , i,j , (2)where (cid:98) Π gi,j is the effective payoffs , which captures the realitythat to imitate, players compare the overall profiles of payoffsrather than simply the one under play. Therefore the decisionis made based upon a combination of both payoffs. We inter-pret the weight θ ∈ [0 , as the game interaction strength, alarger value of θ means a stronger impact of the other game;two extreme cases θ = 0 , correspond to the two indepen-dent games and the cross-playing scenario, respectively. K isa temperature-like parameter, measuring the uncertainties inthe imitation process, its inverse can be interpreted as the se- (a) (b) FIG. 1. (Color online) The cooperation evolution of two symmetri-cally interacting PD on the 2d square lattice. (a) Phase transitions ofcooperation prevalence regarding game G ( f G C = f CC + f CD )versus the temptation b is shown for interaction strengths θ =0 , . , . , . and 1. Due to the symmetry, f G C ≈ f G C (datanot shown for visual clarity). (b) Typical time series are shown withfixed b = 1 . . Parameters: L = 1024 and K = 0 . , the random ini-tial condition for both games, data over 50 ensemble averages aftertransient in (a). lection intensity in biology or the bounded rationality in eco-nomical contexts. A full MC step consists of × L × L elemen-tary steps, where every player is updated once for each gameon average. Simulations are carried out for L = 1024 , and thedata for the cooperator fractions are averaged over MCsteps after a transient period of steps. For general modeldescriptions, see the companion long paper (CLP) [26]. Results. —
Varying the game interaction strength θ , we ob-serve a continuing rise in the cooperation prevalence f c as afunction of the temptation b [see Fig. 1(a)]. For the indepen-dent game case where θ = 0 , a second-order phase transi-tion (PT) for cooperation is seen but the cooperation region israther small with the critical temptation b c ≈ . , beyondwhich the cooperator becomes extinct. As θ is increased, b c is shifted to the right, the prevalence also becomes higher, thecooperation is promoted. Finally, as θ → , this promotion ismaximal, where the PTs become absent and nearly full coop-eration is seen across the whole parameter region ≤ b ≤ for both games. This is quite unexpected since in the cross-playing scenario ( θ = 1 ), the decision-making of a game is DCCCDDCD
FIG. 2. (Color online) The evolution of cooperation patterns for θ = 0 (top row), 0.5 (middle row), and 1(bottom row). The system isprepared with full cooperators CC within upper half domain versusfull defectors DD within another half (the leftmost panel). The char-acteristic snapshots are taken at t = 0 , , , , for θ = 0 ,whereas t = 0 , , , , for the other two. Parame-ters: L = 128 , K = 0 . , and b = 1 . . entirely blind to its own payoff. This observation of promo-tion is strengthened by the corresponding time series by fixing b = 1 . shown in Fig. 1(b), where the initial decrease in f c isalso inhibited when θ becomes large.To gain some intuition of how the game-game interactionaffects cooperation, we first look at how the spatiotemporalevolution is influenced. Fig. 2 shows the case of θ = 0 , . , ,but starting from a bulk initial condition because it is moreintuitive. Without interaction θ = 0 , defectors dominate inboth games, DD invades the CC ’s domain, and cooperatorsquickly go extinct. At the intermediate strength θ = 0 . , thisadvantage disappears where all four fractions coexist. In theother extreme, a reversed invasion is seen where CC domi-nates and takes over the whole domain in the end. This sug-gests that a reversed advantage is expected between coopera-tors and defectors as the game-game is engaged. A mean-field theory.—
To understand the rationale ofpromotion, we develop a mean-field theory based on thereplicator equation [27, 28], where the evolution of the fourfractions with respect to each game depends on their relativefitness measured by the payoffs that can be formally describedas ˙ f s = f s ( (cid:98) Π s − ¯Π) , where g ∈ G , s ∈ S , and ¯Π = (cid:80) s f s (cid:98) Π s is the average fitness. With some algebra (see [26]), we obtainthe ordinary differential equations of cooperator fraction forgame G (i.e. f G C = f CC + f CD , the exchange of 1 and 2applies for game G ) as ˙ f G C = f G C f G D (Π G C − Π G D )+( f CC f DD − f CD f DC )(Π G C − Π G D ) , (3) where Π G , C,D are the fitness in game G or G respectively forthe cooperators and defectors. The first term in the rhs. iswell-known [29] that comes from the game under play, mean- (b)(a) FIG. 3. (Color online) Time evolution of all six interface proportionsfor two interacting PD games starting with random initial conditionsfor θ = 1 (a) and . (b), respectively. Inset show the relative frac-tion of the two non-neutral types P r = P CD − DC /P CC − DD . Param-eters: b = 1 . and L = 1024 for the 2d square lattice. ing that the fitness advantage in cooperators Π G C > Π G D con-verts the defectors into cooperators when they meet up. Thesecond term is new that captures the game-game interaction.Specifically, the impact of the other game is through two inter-acting pairs: i) when CC players come across DD , the advan-tage of cooperators in game G ( Π G C > Π G D ) also facilitatesthe proliferation of cooperators in game G due to the game-game correlation; ii) unexpectedly, in the opposite case when Π G D > Π G C , the advantage of defectors in G also helps thegrowth of cooperators in G when CD encounters DC play-ers. Therefore, the above analysis shows that potentially thereare now new dynamical routes at work towards cooperation inaddition to the one in the single game case. Mechanism analysis.—
To be more specific, it’s helpful tolook into all six interactions in details in our lattice system,as listed in Table I. Here, we distinguish two scenarios — in-dividual and bulk scenarios. In the former, we only focus onthe evolution of the interaction pairs when without knowledgeof their surroundings such as the random state configuration.The bulk scenarios apply for the circumstance when playersof the same type are well-bulked, both intra- and inter-bulkplay are incorporated. The two scenarios are typically presentin the early phase of evolution and afterwards, respectively.For simplicity, we consider the cross-playing case, where thesix pairs of interactions can be classified into three categories:invasion, neutral, and catalyzed type for both scenarios (Ta-ble I). While the neutral type of interactions brings no net ef-fect on cooperation, the other two categories determine thecooperation prevalence, though they always have the oppositeeffects either in individual or bulk scenario.Typical evolution of all interactions is shown in Fig. 3(a)starting from random initial conditions, where the evolutioncan be roughly divided into two stages. (i) At the earlystage t < t c ( t c ≈ MC steps) when no clear clusters areformed and thus the individual scenario applies, the propor-tion P CD − DC > P CC − DD is detected, meaning that catalyzedinteractions dominate over the invasion ones; and accordingto the evolutionary dynamics in Table I, a net production ofcooperation is expected. (ii) When t > t c , clusters are grad-ually formed, where both the size and compactness increase Individual scenario Bulk scenarioInvasion CC + DD G /G −−−−→ DC/CD + DD CC + DD G /G −−−−→ CC + CD/DCNeutral CC + DC G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G /G −−−−→ CD/DC + CC CD + DC G /G −−−−→ DC/CD + DDTABLE I. Classification of interactions in two interacting PD games for the cross-playing scenario ( θ = 1 ), where all six pairwise interactionscan be classified into three categories, in either individual or bulk scenario. In invasion interactions, the payoff advantage in a given gameis explicitly transformed into its reproduction. For neutral type, the state change is purely random, no net conversion towards cooperation ordefection is expected. In the last category, the advantage of defectors/cooperators regarding a given game cross-catalyzes the production ofcooperators/defectors in the other game. The last two categories are only possible in interacting games. (see [26]), therefore the bulk scenario sets in. Interestingly,a crossover is seen that P CD − DC < P CC − DD , the reverseddominance again yields a net increase of cooperators since CC − DD pairs in bulk scenario favor the cooperators (Ta-ble I). Therefore, cooperation is preferred in the whole evolu-tionary processes. Back to the mean-field equation Eq.(3), ouranalysis indicates that the second term always brings a pos-itive contribution to the cooperation evolution. Thereinafter,we term this mechanism caused by game-game interactions asthe dynamical reciprocity . It also works for cases with θ < ,as shown in Fig. 3(b). However, the dynamical reciprocityonly works in structured population, no promotion is seen inthe well-mixed population (see [26]). Robustness.—
With this prerequisite, the revealed reci-procity is quite robust. In [26], we show that when the in-teracting game is extended to be general pairwise games (in-cluding snowdrift game and stage hunt etc), a continuing co-operation promotion is still observed irrespective of the gametype, also fairly high cooperation is expected for the whole pa-rameter domain when games are cross played. We also showit is also applicable to a multiplayer game (the public goodsgame); similar observations are made in model variants suchas asymmetrically interacting games, games with different up-dating rules (like replicator rule, Moran rule, follow-the-bestrule etc), with different time-scales, and even with two differ-ent games, i.e. a PD is coupled with a snowdrift game.Additional structural complexities [26] from underlyingpopulations like small-world networks and Erd˝os-R´enyi ran-dom topologies also do not change the working of the reci-procity. The structural heterogeneity neither alters the promo-tion trend, as shown in the case of scale-free networks.In particular, when the number of engaged games increases,a higher level of cooperation is expected in general, see Fig. 4,where equal contribution is assumed when the game is morethan one. Since potentially there are many issues interweavedwith each other in reality, much higher cooperation is ex-pected than the case when only a single game is unfolded.
Conclusions.—
In summary, the discussed game-game in-teraction is a natural ingredient that may underpin a wealth of (a) (b)
FIG. 4. (Color online) (a) Phase transitions of cooperation preva-lence for one-, two-, and three-game cases, where the each game is ofequal contribution in the effective payoffs for the later two cases (i.e. θ = 1 / and / , respectively). (b) Time series for fixed b = 1 . .Parameter: L = 1024 for the 2d square lattice and K = 0 . . issues, from complex behaviors in animals, to inter-personalactivities in daily life, and even to international relationshipsat the global scale. The potential for being highly coopera-tive, as revealed here points out a promising route towards acooperative world. It is worthwhile to emphasize that con-trary to the network reciprocity, where the underlying struc-ture of population plays the key role [11, 12], the mechanismbehind the promotion here stems instead from the dynamicalinteraction among different games. Our results suggest thatthe dynamical reciprocity could constitute a new category ofmechanisms behind the emergence of cooperation.On the theoretic side, our finding of “more is different” [30]calls for more systematic investigations in specific contexts,since the revealed mechanism may offer valuable policy in-spiration to tackle those imminent crisis. On the experimentalhand, behavioral experiments are needed to justice the dynam-ical reciprocity and unveil other complexities that may arise ininteracting games. Acknowledgements.—
This work is supported by the Na-tional Natural Science Foundation of China under Grants61703257 and 12075144, and by the Fundamental ResearchFunds for the Central Universities GK201903012. L. C. ac-knowledges the enlightening discussions with Dirk Brock-mann (HU and RKI) in the early phase of the project andYing-Cheng Lai (ASU) for helpful comments. ∗ Equal contribution † Email address: [email protected][1] G. Hardin, Science , 1243 (1968).[2] M. A. Nowak and S. Karl, Science , 793 (2004).[3] M. A. Nowak, Science , 1560 (2006).[4] K. Sigmund, C. Hauert, and M. A. Nowak, Proc. Natl. Acad.Sci. U.S.A. , 10757 (2001).[5] F. C. Santos, M. D. Santos, and J. M. Pacheco, Nature , 213(2008).[6] R. L. Trivers, The Quarterly Review of Biology , 35 (1971).[7] M. A. Nowak and K. Sigmund, Nature , 573 (1998).[8] W. D. Hamilton, Journal of Theoretical Biology , 17 (1964).[9] L. Keller, Levels of selection in evolution (Princeton UniversityPress, 1999).[10] D. C. Queller, Nature , 1145 (1964).[11] M. A. Nowak and R. M. May, Nature , 826 (1992).[12] G. Szab´o and G. Fath, Physics Reports , 97 (2007).[13] C. Gracia-L´azaro, J. A. Cuesta, A. S´anchez, and Y. Moreno,Scientific Reports , 325 (2012).[14] G.-L. Carlos, F. Alfredo, R. Gonzalo, T. Alfonso, J. A. Cuesta,S. Angel, and M. Yamir, Proc. Natl. Acad. Sci. U.S.A. ,12922 (2012).[15] D. G. Rand, M. A. Nowak, J. H. Fowler, and N. A. Christakis, Proc. Natl. Acad. Sci. U.S.A. , 17093 (2014).[16] A. Traulsen, D. Semmann, R. D. Sommerfeld, H.-J. Kram-beck, and M. Milinski, Proc. Natl. Acad. Sci. U.S.A. , 2962(2010).[17] D. G. Rand, A. Samuel, and N. A. Christakis, Proc. Natl. Acad.Sci. U.S.A. , 19193 (2011).[18] F. Katrin, V. D. Post, Daniel J, and S. Dirk, Ecology Letters ,546 (2011).[19] P. Hammerstein et al. , Genetic and cultural evolution of coop-eration (MIT press, 2003).[20] R. Cressman, A. Gaunersdorfer, and J.-F. Wen, InternationalGame Theory Review , 67 (2000).[21] M. Chamberland and R. Cressman, Games and Economic Be-havior , 319 (2000).[22] K. Hashimoto, Journal of Theoretical Biology , 669 (2006).[23] V. R. Venkateswaran and C. S. Gokhale, Proceedings of theRoyal Society B , 20190900 (2019).[24] K. Donahue, O. P. Hauser, M. A. Nowak, and C. Hilbe, NatureCommunications , 1 (2020).[25] G. Szab´o and C. T˝oke, Phys. Rev. E , 69 (1998).[26] “Dynamical reciprocity in interacting games: numerical resultsand mechanism analysis (the companion long paper),”.[27] P. D. Taylor and L. B. Jonker, Mathematical Biosciences ,145 (1978).[28] C. P. Roca, J. A. Cuesta, and A. S´anchez, Physics of Life Re-views , 208 (2009).[29] J. M. Smith, Evolution and the Theory of games (CambridgeUniversity Press, 1982).[30] P. W. Anderson, Science177