Dynamical reciprocity in interacting games: numerical results and mechanism analysis
Rizhou Liang, Qinqin Wang, Jiqiang Zhang, Guozhong Zheng, Lin Ma, Li Chen
DDynamical reciprocity in interacting games: numerical results and mechanism analysis
Rizhou Liang, Qinqin Wang, Jiqiang Zhang,
2, 3
Guozhong Zheng, Lin Ma, and Li Chen
1, 4, ∗ School of Physics and Information Technology, Shaanxi Normal University, Xi’an 710062, People’s Republic of China School of Physics and Electronic-Electrical Engineering,Ningxia University, Yinchuan 750021, People’s Republic of China Beijing Advanced Innovation Center for Big Data and Brain Computing,Beihang University, Beijing 100191, People’s Republic of China Robert Koch-Institute, Nordufer 20, 13353 Berlin, Germany (Dated: February 2, 2021)We study the evolution two mutually interacting games with both pairwise games as well as the public goodsgame on different topologies. On 2d square lattices, we reveal that the game-game interaction can promote thecooperation prevalence in all cases, and the cooperation-defection phase transitions even become absent andfairly high cooperation is expected when the interaction goes to be extremely strong. A mean-field theory isdeveloped that points out new dynamical routes arising therein. Detailed analysis shows indeed that there arerich categories of interactions in either individual or bulk scenario: invasion, neutral, and catalyzed types; theircombination put cooperators at a persistent advantage position, which boosts the cooperation. The robustness ofthe revealed reciprocity is strengthened by the studies of model variants, including asymmetrical or time-varyinginteractions, games of different types, games with time-scale separation, different updating rules etc. The struc-tural complexities of the underlying population, such as Newman–Watts small world networks, Erd˝os–R´enyirandom networks, and Barab´asi–Albert networks, also do not alter the working of the dynamical reciprocity. Inparticular, as the number of games engaged increases, the cooperation level continuously improves in general.Our work thus uncovers a new class of cooperation mechanism and indicates the great potential for humancooperation where concurrent issues are so often seen in the real world.
PACS numbers: 87.23.Ge, 02.50.Le, 89.75.Fb, 87.23.Kg
I. INTRODUCTION
The rise of human civilization is built upon the widespreadcooperation at almost every corner of socioeconomical andother activities [1]. Its decay, by contrast, generally leads toregress of human welfare or even wars. The recent decadeshave witnessed some imminent crises such as global warming,trade wars, and more recently COVID-19 [2] etc. Solutions toany of them require multilateral cooperation, a throughout un-derstanding of what motivates cooperation and how it evolves,and why it fails, is then needed. According to the Darwinism,however, natural selection favors the fittest, those who are al-truistic incur a cost to themselves, leading to a less chanceto survive. Cooperation is not a reasonable option by logic.Revealing the hidden mechanisms behind is therefore of fun-damental importance and has been listed as one of the grandscientific challenges within this century [3].Within the framework of evolutionary game theory, manyimportant progresses have been made [4, 5]. By analyz-ing those canonical models such as prisoner’s dilemma (PD),the snowdrift game (SG), the public good game, and thecollective-risk dilemma etc, valuable insights are obtained andseveral mechanisms are revealed. These include direct [6] andindirect reciprocity [7], kin [8] and group selection [9, 10],spatial or network reciprocity [11], reward and punishment[12], social diversity and hierarchy [13–15], and so on. In par-ticular, it is found that the presence of population structures,either static or dynamic, is able to promote cooperation com- ∗ Email address: [email protected] pared to the well-mixed scenario, because cooperators formclusters that can prevail against the invasion of defectors. Thisso-called network reciprocity has been extensively studied inthe past few decades [16, 17].Alongside these theoretic insights, recent behavioral exper-iments also expand our understanding of cooperation [18].As a new paradigm, the recruiting volunteers are configuredwith some given topologies and rule settings, they are well-motivated to play the games. But due to the complexities likehuman psychology, cultures or personalities etc, inconsisten-cies with theoretic predictions are often unveiled [19]. Forexample, experiments with static structured population do notfound their advantage in promoting cooperation in general asthe network reciprocity predicts [20–22]. Some extra condi-tions regarding the game experiment settings have to be sat-isfied for cooperation to thrive [23]. These facts imply thatsome essential factors could be missing in the most of cur-rent game-theoretic models, and the experiment-driven modelimprovement effort is required in the future.As a common practice, most of the existing works mainlyfocus on a single game scenario, with a belief that when thedynamics of single game is well-understood, the wisdom ob-tained is supposed to be applicable to many other situations,even with more games being evolved. The philosophy behindis the reductionism essentially. In the real world, however,entities are generally evolved in multiple games simultane-ously, where they could influence each other’s evolution. Forexample, colleagues potentially work in a couple of concur-rent projects, nations are involved in several issues such astrade, security, culture etc. In these contexts, the progress orconflicts in one issue is likely to affect the evolution of oth-ers, they have a stake in each other explicitly or implicitly. a r X i v : . [ phy s i c s . s o c - ph ] J a n In non-human species, field research have made similar ob-servations in the chimpanzee societies, their activities suchas grooming, hunting, sharing meat, supporting one anotherin conflicts, border patrols etc are found to be closely corre-lated, e.g. a male with good hunting skill is more likely to begroomed by others and vice versa [24].Till now, only a few models have considered the multiplegames, but with different emphases. One line is along themulti-game dynamics. An early conceptually related work isthe study by Cressman et al. [25] where two 2-strategy gamesare played and the eventual states can be described by the dy-namics of the separate game. Later research show that the fateof a single game generally cannot be determined without in-corporating the messages of other games [26–28], e.g. persis-tent cycles could arise within coupled one-equilibrium games.These works are done in a mean-field sense, and only con-sider one-shot game scenario where the potential reciprocityin evolution is beyond their scope.Another line is within the framework of interdependent net-works, where different games are played on different layers ofnetworks and they are coupled by means of the payoff/utilityfunction [29–37]. Zhen et al. [29] consider two public goodsgames being played on two symmetrically connected lattices,and the utility function includes not only the contribution ofthe payoff of the focal site plus its neighborhood’s payoffs,but also the contributions of the neighborhood in the otherlattice. They found that as the neighborhoods’ contributionin both lattices increases, the cooperation level is promoted.G´omez-Gardenes et al. [30] extend the study to arbitrary num-ber of layers where they adopt Erd˝os-R´eny random graphsand PD for each layer, and the net payoff is through the equalcontribution among all layers used for the strategy updating.Their work shows a resilience of cooperation for extremelylarge values of temptation to defect and this resilience is in-trinsically related to a non-trivial organization of cooperationacross different layers. Asymmetrical game settings are alsostudied [33, 34], where different games are unfolded on differ-ent layers. For example, Santos et al. [33] considers PD andSG being posed on two layers of regular random networks re-spectively, individuals imitate neighbors from the same layerwith a probability, and neighbors from the second layer witha complementary probability. Therefore the strategy transferis allowed between layers and they find that while such cou-pling is able to promote the cooperation in the PD layer, but itis detrimental for the cooperation in the SG layer. Within theframework of interdependent networks, its specific construc-tion matters [31, 32, 35, 36]. In particular, the link overlapacross different layers is shown to be crucial [35], there is nobenefit for cooperation if without any structural correlation.The impact of other topological effects such as degree mixingis also studied in [36], and some other dynamical processeslike spontaneous symmetry breaking between different layersare uncovered in [37]. All these promotions are attribute tothe interdependent network reciprocity [29], a subcategory ofnetwork reciprocity. Intuitively, one can view it as an efficientconstruction of population relationship that tends to maximizethe previously uncovered network reciprocity.The most related work is by Donahue et. al [38], where they propose a framework termed multichannel games. Eachchannel represents a repeated game and players interact overmultiple channels and these channels are dependent with eachother. Based on two donation games, they uncovered the evo-lutionary advantage of cooperation due to the game linkage.This finding acts as a good starting point towards a new funda-mental category of reciprocity — dynamical reciprocity , as acounterpart of the well-studied network reciprocity. It empha-sizes that the reciprocity stems from the game-game dynamicsrather than the underlying topology of population. A bunch offundamental questions, however, remain unanswered: what’sthe typical evolutionary dynamics when more games are en-gaged? how robust is the dynamical reciprocity? what’smechanism behind this new type of reciprocity?
These arewhat we are trying to answer in this work.The aim of the present paper is to introduce and formal-ize the interacting games and systematically investigate theimpact of game interplay on the cooperation prevalence. Dif-ferent games interact through a perceived payoff, a functionof payoffs in all games. To our surprise, we are not only ableto see the cooperation promotion, but also the cooperation-defection phase transitions could disappear, where an absorb-ing state of nearly full cooperation is approached. More nu-merical studies show this sort of promotion is quite robust,confirmed by variants with both different game dynamics (thetypes of game, the updating rule and synchronicity, the gamecoupling etc.) and different underlying topologies. A mean-field treatment indicates that new dynamical routes towardscooperation come into play, which is confirmed by detailedmechanism analysis. There, apart from the invasion categoryof interaction that is also present in the single game case, twoother new categories of interactions — neutral and catalyzed— are identified. Working together, these three categories ofinteractions lead to a persistent advantage of cooperators overthe defectors. The promotion is further enhanced when moregames are engaged, where the revealed mechanism still holds.The paper is organized as follows: In Sec. II, we formu-late the interacting games of arbitrary number. In Sec. III,the preliminary results of two symmetrically interacting PDgames on 2d regular lattice are shown. In Sec. IV, we present amean-field treatment within the framework of replicator equa-tions to see what dynamics would arise in the presence ofgame-game interactions. In Sec. V, the dynamical mecha-nism of the revealed reciprocity is discussed in details by clas-sifying all interacting pairs. More numerical results regard-ing the robustness of the revealed reciprocity are provided re-garding dynamical variations in Sec. VI, and structural vari-ations in Sec. VII. More games are considered in Sec. VIIIto extrapolate what would be expected when the number ofgames increases. Finally, some concluding remarks are givenin Sec. IX, together with the implications and some open ques-tions being listed.A short version of the work is presented in a companionshort letter (CSL) [39]. (a) (b) G G Player y G G ⚔ Player xx y effectivepayoffs
FIG. 1. Modeling two interacting games. (a) Consider a groupof networked players, playing two games G , simultaneously, twopayoffs are obtained accordingly, which can be interpreted as the fit-ness in their evolution. (b) When two neighboring individuals, sayplayer x and y , are to update their strategies with respect to a givengame (e.g. game G here), the update not only depends on the pay-offs obtained in G (with weight − θ ), but also the one in the othergame G (with θ ), termed as the effective payoffs, see Eq.(3). II. GENERAL FORMULATION
We formulate the interacting games within the evolution-ary framework for where m games are denoted as G = { G , G , ..., G m } , they are played simultaneously in the pop-ulation of size N sharing the same underlying topology. Thecorresponding strategy set is denoted as S m . Assume that eachplayer could be in one of two states in each game: coopera-tion (C) or defection (D), i.e. S = { C, D } . By combinationthere are m elements in S m , e.g. the strategy set for m = 2 is S = { XY | CC, CD, DC, DD } , where X , Y correspondto the state in game G , G , respectively.In our study, we adopt the general pairwise games (GPG)and public goods game (PGG). The GPG is defined as follows:when both players cooperate, each gets a reward R ; whenboth defect, then each gets a punishment P ; and the mixedencounter yields a temptation T for the defector while the co-operator becomes a sucker with a payoff S . Different rankingsof the four payoffs lead to different game types. Specifically,four types of games are defined in S − T parameter space byfixing R = 1 and P = 0 : i) < T < and < S < forthe snowdrift game (SG); ii) < T < and < S < forthe harmony game (HG); iii) < T < and − < S < for stag hunt game (SH); iv) < T < and − < S < for prisoner’s dilemma (PD). The PGG can be considered asan extension of PD where an arbitrary number of players canplay together in a group. It is defined as follows: in eachround, every player in the group chooses either to contribute1 to the common pool as a cooperator, or nothing as a defec-tor; the sum of the contribution is then multiplied by a gainfactor r > , reflecting the synergetic effect; finally, the re-sulting amount of benefit is equally shared among all mem-bers in the group, including those defectors. Therefore thenet payoff of cooperators have to subtract 1 from the sharedamount, whereas the defectors don’t need to do so. If withoutany mechanism, defection is preferred.The system is initialized with random conditions if notstated otherwise where each player randomly chooses to co-operate or defect in each game. The evolution follows thestandard Monte Carlo (MC) procedure. At an elementary step, a random game g ∈ G is chosen, and then we choosea random player x , accumulate its payoff Π x according to thegame setting. Next, one of its neighbors y ∈ Ω x is pickedrandomly, also its payoff Π y is computed. Lastly, player x adopts y ’s strategy according to some function W ( s gy → s gx ) = W (Π x , Π y ) that translates their payoff difference intothe learning propensity. In our study, the Fermi rule [40] isadopted as W ( s gy → s gx ) = 11 + exp[( (cid:98) Π gx − (cid:98) Π gy ) /K ] , (1)where K is a temperature-like parameter, measuring the un-certainties in the strategy adoption, its inverse can be inter-preted as the selection pressure. K is fixed at 0.1 throughoutthe work if not stated otherwise. (cid:98) Π gx is the effective payoff perceived by player x in game g defined below that is used toupdate its strategy. A full MC step is comprised of m × N such elementary steps, meaning that every player is going toupdate its strategy once for each game on average.The effective payoff Π gx that is formally defined as (cid:98) Π gx = E (Π G x , Π G x , ... Π G m x ; P g ( θ , θ , ..., θ m )) , (2)by which the game-game interactions come into play. It cap-tures the facts that the decision-making of a given game g would base upon a perceived payoff through integrating pay-offs in all games instead of simply the one under play. Here θ i ∈ [0 , is the contribution weight of game G i . The dis-tribution P g ( θ , θ , ..., θ m ) then determines to how m gamesinfluence the perceived payoffs in game g .Note that, the implemented MC simulation procedure isto mimic the continuous-time evolution as in the real world,where the strategy updating is asynchronous. To our inter-est, we also consider synchronous updating (SU) scheme asfollows. All m games are repeatedly played in circular or-der G , G , ..., G m , G , ... ; for a given game, all players si-multaneously compute and compare their payoffs to one ran-domly chosen neighbor, and make strategy update accordingto Eq.(1). m such discrete steps in the SU scheme guaranteethat every game is precisely played once for each player.Four more different updating rules [41]: Moran-like rule,replicator rule, multiple replicator rule, and the unconditionalrule are also investigated in Sec. VI for robustness studies.We adopt 2d square lattices with the size of N = L × L as the underlying structure for most studies, where each indi-vidual plays the games with its four nearest neighbors, andthe periodic boundary condition is assumed. We will alsostudy Newman-Watts networks, Erd˝os-R´enyi networks, andBarab´asi–Albert networks in Sec. VII. As noted above that allgames are assumed to share the same set of links, this is notvery realistic of course, since many games are unfolded ontheir own sets of connections, as in the multiplex networks.However, these structural intricacies in multiplex networkswould bring additional complexities that will confound ourunderstanding of the reciprocity purely from the dynamicalpart, therefore we would like to avoid them. T -101 S T -101 0 1 2 T -101 00.51 SDHGSH PD (a) (b) (c)
FIG. 2. (Color online) Color-coded fraction of cooperators regarding the first game ( f G C = f CC + f CD ) for the general pairwise game withinthe S − T parameter space with θ = 0 , 0.5, and 1 on the 2d square lattice, respectively shown in (a-c). Due to the symmetry, f G C ≈ f G C .Four quadrants correspond to four different games (defined in Sec. II). Parameters: R = 1 , P = 0 , L = 128 . III. PRELIMINARY RESULTS FOR 2 INTERACTING PDGAMES ON 2D SQUARE LATTICE
In this section, we shall only discuss the case of two inter-acting games G = { G , G } with perfect symmetry, they areunfolded on a 2d square lattice (see Fig. 1). A simple linearcombination is used for the effective payoff as (cid:98) Π G , x = (1 − θ )Π G , x + θ Π G , x , (3)where P G ( θ , θ ) = { − θ, θ } and P G ( θ , θ ) = { θ, − θ } are also symmetrical. Here we interpret the contributionweight θ as the game-game interaction strength . The largerthe value of θ , the stronger impact of the other game is posed.The case of θ = 0 reduces the model into two indepen-dent games, while the other extreme θ = 1 corresponds tothe cross-playing scenario where the decision-making of agiven game is entirely determined by the payoffs in the othergame. More often cases in reality are supposed to occur inbetween. These constitute a parsimonious model of two inter-acting games.In CSL [39], a monotonic promotion of cooperation is ob-served as the interaction strength θ increases in two symmet-rically interacting PD games (PD is a typical case of GPG).Here, we further show that this promotion is universal for allgame types within the GPG formulation, see Fig. 2. We seethat as θ increases, the defection region shrinks. In particular,when θ → , the cooperation maintains at a fairly high level( > . ) for the whole parameter S − T space, the differenceamong the four games nearly disappears.In fact, a similar observation is also made in two symmet-rically interacting PGG. Fig. 3 shows the prevalence of coop-eration within a two-parameter space. While a larger value ofnormalized gain factor ˆ r tends to raise the cooperation propen-sity, a stronger interaction strength θ generally facilitates co-operation as well. Together with Fig. 2, we conclude thatas the game-game interaction becomes stronger, cooperationcontinues to improve, and fairly high cooperation is seen forthe whole parameter spaces as θ → , irrespective of the gametype. IV. A MEAN-FIELD ANALYSIS
In theory, the evolutionary games can be described by amean-field treatment based on the replicator equation (RE)[41], which was introduced in 1978 by Taylor and Jonker[42]. RE characterizes the evolution of frequencies or frac-tions of different species in the population by taking into ac-count their mutual influence on each other’s fitness. Math-ematically, it successfully captures the selection process andprovides a bridge between the Nash equilibrium in static pay-off matrix and the evolutionary stable strategies in evolution.However, there are some assumptions required explicitly orimplicitly in the derivation of RE as follows. (1) The popu-lation is infinitely large; (2) The population is well-mixed sothat each individual interacts with an equal probability witheveryone else, or equivalently a full connected graph is as-sumed; (3) No mutation is allowed for the strategies, theirfrequency changes are only due to the reproduction; (4) Theevolution of frequencies is linearly proportional to their fitnessdifference. Derivations from the finite-size effect or from thestructured property in the real population are expected accord-ing to assumptions (1) and (2).With these assumptions, let’s consider a population with m games, their evolution can formally be described as ˙ f s = f s (Π s − ¯Π) , (4)where f s is the frequency or fraction of population within state s ∈ S m , Π s is its fitness Π s = (cid:80) g ∈ G Π gs , and ¯Π = (cid:80) s f s Π s is the average fitness of the whole population. The specieswith a high fitness tends to increase its fraction, the one witha low value tends to reduce instead. A. A single pairwise game
Let’s first recall the well-known single pairwise game case,where m = 1 and S = { C, D } . f C,D are the fraction ofcooperators and defectors respectively, with f C + f D = 1 . By FIG. 3. (Color online) Color-coded fraction of cooperation preva-lence f G C in θ − ˆ r parameter space for two symmetrically interactingpublic goods games. ˆ r = r/ ( k + 1) is the normalized gain factor,and k + 1 is the number of games the individual is evolved. Also f G C ≈ f G C due to the symmetry. Here L = 128 and K = 0 . . applying the payoff matrix given in Sec. II, the RE is then ˙ f C = f C f D [ f C ( R − T ) + f D ( S − P )]= f C (1 − f C )(Π C − Π D ) , (5)where Π C = Rf C + S (1 − f C ) and Π D = T f C + P (1 − f C ) .There are three fixed points: f ∗ C = 0 , , P − SR + P − T − S , (6)which correspond to full defection, full cooperation, and amixed strategy, respectively. Their stabilities are determinedby doing linearization of Eq. (5) and computing the corre-sponding eigenvalues around these fixed points. Only negativeeigenvalues guarantee a stable solution. Well-known facts ofthe stable solution are as follows [43]: full defection for PD,mixed strategy for SD, coexistence of full cooperation and fulldefection for SH depending on the initial condition, and fullcooperation for HG.
B. Two interacting pairwise games
Now let’s extend the RE treatment to the case of twosymmetrically interacting pairwise games, where S = { CC, CD, DC, DD } . The four fractions satisfy (cid:80) s f s =1 , s ∈ S . The overall fitness of a given state s is Π s = (cid:80) g Π gs = Π G s + Π G s . Along the setup in numerical simula-tions, the two games are symmetrical both in the game param-eterization and interactions. Without game interactions, thefitness of the four species in the two games are Π G CC Π G CD Π G DC Π G DD = R R S SR R S ST T P PT T P P f CC f CD f DC f DD , (7) and Π G CC Π G CD Π G DC Π G DD = R S R ST P T PR S R ST P T P f CC f CD f DC f DD . (8)With these, we define the effective fitness analogously as (cid:18)(cid:98) Π G s (cid:98) Π G s (cid:19) = (cid:18) − θ θθ − θ (cid:19) (cid:18) Π G s Π G s (cid:19) , (9)corresponding to the effective payoffs as in Eq. (3) in thenumerical simulations. The RE Eq. (4) is then rewritten as ˙ f s = f s ( (cid:98) Π s − ¯Π) , (10)where (cid:98) Π s = (cid:80) g (cid:98) Π gs = (cid:98) Π G s + (cid:98) Π G s . And the mean fitness is ¯Π = (cid:80) s f s (cid:98) Π s .With some algebra (see Appendix A for details), we can de-rive the evolution of cooperator fraction with regard to eithergame, say game G ( f G C = f CC + f CD ), as follows ˙ f G C = f G C f G D (Π G C − Π G D )+( f CC f DD − f CD f DC )(Π G C − Π G D ) , (11) where Π G C = Π G CC = Π G CD , Π G D = Π G DD = Π G DC , Π G C =Π G CC = Π G DC , and Π G D = Π G DD = Π G CD . The first termof the rhs. is exactly the same as in the single game dynam-ics shown in Eq. (5). It means that the cooperator fractionin game G tends to increase when Π G C > Π G D . The newdynamics lies in the second term, which captures the game-game interaction via the interaction pairs of CC – DD and CD– DC. Specifically, when CC individuals meet up with DDones, if their fitness satisfies Π G C > Π G D the advantage of co-operators in game G transfer DD to be CD due to the gamecorrelation. By contrast, when CD individuals meet up withDC ones, this advantage is instead to reduce the cooperationprevalence, transferring CD to DD. But for the way around(i.e. Π G D > Π G C ), however, the advantage of defectors ingame G leads to the growth of cooperators in G , transfer-ring DC to CC, which is unexpected when games evolve inde-pendently. Therefore, the above equation explicitly capturesthe cooperation dynamics from both intra- and inter-game in-teractions. Note that, by the exchange of game label 1 and2 in Eq. (11), the equation describes the cooperator fractionevolution of game G , i.e. f G C = f CC + f DC .By analytically solving the mean field Eqs. (10) (see Ap-pendix B), we found that the stable solutions are exactly thesame as the single game case, meaning the game-game inter-action brings no impact on the cooperation evolution at themean-field level. The two games are decoupled essentiallyin the well-mixed scenario. This means that the dynamicalreciprocity from the game-game interaction should go hand inhand with network reciprocity — it only works in the struc-tured population. Numerical simulations of two interactingPD games in a fully connected population confirm this con-clusion as shown in Fig. 4, where by starting from randominitial conditions, mixed strategies, full cooperation, coexis-tence of full cooperation and full defection, full defection are Individual interaction Bulk interactionInvasion type CC + DD G /G −−−−→ DC/CD + DD CC + DD G /G −−−−→ CC + CD/DCNeutral type CC + DC G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G /G −−−−→ CD/DC + CC CD + DC G /G −−−−→ DC/CD + DDTABLE I. Categories of interactions in 2 interacting PD games within the cross-playing scheme ( θ = 1 ). Six pairwise interactions are classifiedinto three categories in either individual or bulk scenarios. respectively seen in the quadrant I to IV, in accordance withthe analytical results. In the following section, however, weshow that the second term in Eq. (11) does play a role inthe structured population, which yields fundamentally differ-ent dynamical route towards cooperation. V. DYNAMICAL MECHANISMA. The category of interactions for two games
To understand the physics behind Eq. (11), and ultimatelytowards a thorough comprehension of the dynamical reci-procity, let’s list all possible interactions in the two interactinggames on 2d square lattices and classify them into differentcategories according to the net effect in their offspring repro-duction (see Table I). For simplicity, we focus on the cross-playing case ( θ = 1 ) in pairwise games, where the classifica-tion is most clear, but qualitatively the following analysis canalso be applied to the cases with θ < . FIG. 4. (Color online) The cooperator fraction f G C for the two in-teracting pairwise games in the well-mixed population within S − T parameter space. The resulting cooperation prevalence exactly corre-sponds to the solution of single game; the less significant bistabilityin SH game is simply due to the random initial conditions, where f C ≈ f D ≈ / . Due to the symmetry, f G C ≈ f G C in all cases,except very few case along the line S = T − in SH, depending ontheir initial conditions. Other parameters: R = 1 , P = 0 , N = 2 . Before proceed, we need to distinguish two interacting sce-narios — individual and bulk interactions. Consider twoneighboring players x and y , and assume s x (cid:54) = s y . In theindividual scenario, we only compute the payoffs of the twoplayers through the interacting pair x − y . This scenario ap-plies to the context that their surroundings are unknown, likethe random initial condition case, where the information oftheir neighbors is stochastic. Instead, when players of thesame state are well-bulked, their payoffs can be explicitly esti-mated by incorporating both the intra- and inter-bulk gaming.Specifically for player x , its payoff is assumed to include thepayoffs from its three neighbors with s = s x (intra-bulk) andthe one from x − y gaming (inter-bulk). These two scenar-ios are two extreme cases in structured population, the actualevolution should occur somewhere in between.Since there are four different types of individuals, there aresix interacting pairs by combination, which can be classifiedinto the following three categories for either individual or thebulk interactions, see Table I. (i) Invasion type — for the individual interaction, DD dom-inates over CC for either game thus the CC individual tends tobecome CD or DC; while when bulk CCs meet up bulk DDs,CCs at the interface have higher payoff than DDs (typically R > T for the lined up interface), CC bulks are supposed toinvade into DD regions. In either case, the two cannot coexistand invasion happens, where the one at the disadvantageousposition will be invaded and become CD or DC. (ii) Neutral type — for both scenarios, there is a neu-tral outcome, where statistically no net change in coopera-tion is expected. Consider CC–CD interface as an exam-ple, the two individuals are of identical state regarding game G , Π G CC = Π G CD = R and R for individual and bulk in-teractions, respectively. And since the evolution in game G is determined by the difference of (cid:98) Π G CC,CD = Π G CC,CD , (cid:98) Π G CC − (cid:98) Π G CD = 0 means that the evolution of CC–CD inter-face is neutral W ( CD → DC ) = W ( DC → CD ) = 1 / according to Eq. (1), a random-walk-like movement of theirboundaries is expected. This argument is applicable to theother three interfaces alike. (iii) Catalyzed type — when a CD meets up a DC and theyplay game G ; accordingly (cid:98) Π G CD = Π G CD = T and (cid:98) Π G DC =Π G DC = S with T > S , the DC individual is then likely to be-come CC by learning (i.e. CD + DC G −→ CD + CC ). Anal-ogously, when they play game G , the CD individual tendsto become CC (i.e. CD + DC G −→ CC + DC ). As a con-sequence, CC individuals are continuously produced. In thebulk scenario, however, the situation is reversed. When bulkCDs and bulk DCs meet up, at the lined-up interface (cid:98) Π G CD =Π G CD = T and (cid:98) Π G DC = Π G DC = 3 R with T < R , CD indi-viduals then tend to become DD while DC remains unchangedwhen playing game G (i.e. CD + DC G −→ DD + DC ). AndDC individuals likewise are likely to become DD when play-ing game G (i.e. CD + DC G −→ CD + DD ). Therefore,DDs are continuously produced instead for bulk interactions.In all these cases, the advantage or disadvantage regarding agiven game, is not directly translated to any strategy update inthe game per se , but the state changes in the other game. Thisis reminiscent of catalyzed reactions in Chemistry. The pres-ence of one game is to “catalyze” the evolution of the othergame. They mutually catalyze each other’ evolution.With this classification, it’s straightforward to understandthe impact of game-game interactions, or more specifically thesecond term in mean-field equation Eq. (11). First, since thefour neutral interactions give rise to null net effect, no relatedterm is included in Eq. (11). Second, the CC–DD and CD–DCinteracting pairs do have net effect in the cooperation evolu-tion therefore they appear in the equation, but because theyhave the opposite impact on cooperation in either individualor bulk scenario, they appear to have opposite signs. Third,the mean-field treatment assumes a well-mixed setup, whichactually corresponds to the individual scenario, where the CD-DC pairs contribute to the increase of cooperation while theCC–DD pairs do the opposite given the cooperators are at adisadvantage position, i.e. Π G , C − Π G , D < . B. Numerical evidences for the two scenarios
To validate the arguments of the three categories, we pro-vide some numerical experiments. To prepare the two scenar-ios, we respectively start with random and half-half patchedinitial conditions to mimic the individual and bulk circum-stances respectively, and focus on the early stage of evolu-tion, since longer evolution is going to ruin the randomness orcompactness. Each time we only place two species on the 2dsquare lattice for clarity.Figure 5 shows the time series for all six binary combina-tions in the individual scenario. Category (i) corresponds toFig. 5(a), a mixture of CC and DD individuals. As can beseen, the fraction of CC decreases rapidly at the early fewsteps, DDs’ density doesn’t increase either, instead the partialcooperators CD and DC show considerable increases. This isline with our argument that CC has no advantage against DDindividually, and CD and DC are produced. The evolution atthe later stage, the well-mixture is ruined and the CC playersgain their strength to increase. Figure 5(b)-5(e) reports theevolution of Category (ii). We observe that only the two pre-pared species are present, the others are of zero density. Thisis because a partial absorbing state is reached, e.g. CC–CD
CCDCCDDD (a) (b) (c) (d) (f)(e)
FIG. 5. (Color online) Time series of all four fractions for two cross-played ( θ = 1 ) PD games starting from random initial conditions.All six binary compositions are included: (a) CC–DD, (b) CC–CD,(c) CC–DC, (d) CD–DD, (e) DC–DD, (f) DC–DC. (a), (b-e), (f)correspond to category (i – iii), respectively. Parameters: S = 0 , T = 1 . , R = 1 , P = 0 , L = 1024 for the 2d square lattice. in Fig. 5(b), game G is in the absorbing state with full coop-eration, the fraction of DC or DD continues to be zero. Theremaining two fractions fluctuate around 1/2, the neutrality oftheir evolution is then confirmed. Category (iii) is illustratedin Fig. 5(f), where CD and DC players are randomly blended.As expected, in the first few steps, the fraction of CC rapidlyincreases together with some DD individuals, and the frac-tions of CD and DC themselves decease. Later on, the fractionof CC individuals continue to increase as all other three frac-tions decrease, but the population by then is not well-mixedanymore and some clusters are formed.Figure 6 shows the evolution in the bulk circumstance. Cat-egory (i) is shown in Fig. 6(a), where the evolution of the twofractions is now qualitatively different from Fig. 5(a) — bulkCC individuals are now in an apparent advantage position overDDs, its fraction continues to increase and the faction of DDdecreases until distinction. The fractions of CD and DC keepat relatively low densities. Typical spatiotemporal snapshotsfor this case are shown in Fig. 3 in CSL [39]. Category (ii)is illustrated in Fig. 6(b-e). As can be seen, the dynamicalevolution is qualitatively the same as shown in Fig. 5(b-e), allfluctuating around 1/2. Category (iii) is shown in Fig. 6(f),which is fundamentally different from the process in Fig. 5(f), CCDCCDDD (a) (b) (c) (d) (e) (f)
FIG. 6. (Color online) Time series of all four fractions for two cross-played ( θ = 1 ) PD games starting from half-half patched initial con-ditions. (a) CC–DD, (b) CC–CD, (c) CC–DC, (d) CD–DD, (e) DC–DD, (f) CD–DC. (a), (b-e), (f) correspond to category (i – iii), re-spectively. Parameters: S = 0 , T = 1 . , R = 1 , P = 0 , L = 1024 for the 2d square lattice. but in line with our arguments above. It shows that once CDand DC meet up in bulks, the payoffs from the intra-clustergaming reverse the advantages, favoring the DD individuals.As a result, the fractions of CD and DC continue to decreaseand the density of DD monotonically increases without anyCC individuals being seen.Put together, the numerical experiments in Fig. 5 and Fig. 6perfectly justify the classification of the three categories. C. Dynamical realities on 2d square lattices
However, these above two initial conditions are peculiarthat correspond to two extreme circumstances and only lastfor a short-term time window. What really happened to thesediverse interactions along the whole time evolution? Figuringout these facts is the key to understanding the working of dy-namical reciprocity, since the invasion and catalyzed types ofinteraction always produce the opposite results, the classifica-tion itself cannot explain why cooperation is preferred.Here, by adopting random initial conditions, we monitorthe long-term evolution for both θ = 0 . and 1, the dynamicalprocesses typically experience two stages: (a) (b) (c) (d) FIG. 7. (Color online) Time evolution of interacting pair propor-tions for two interacting PD games. Both random (a, c) and half-halfpatched (b, d) initial conditions are considered. (a, b) and (c, d) cor-respond θ = 0 . and 1, respectively. P r = P CD − DC /P CC − DD is to compare the relative proportion for CD–DC and CD–DD pairs.Parameters: S = 0 , T = 1 . , R = 1 , P = 0 , L = 1024 for the 2dsquare lattice. i) At early stage t < t c ( t c ∼ MC steps for the size of × ), no sizeable clusters are supposed to be presentin the system. Hence the individual interaction scenario ap-plies. Statistically, as Fig. 7(a,c) show that, there is a de-tectable quantity difference in CC–DD and CD–DC pairs thatthe proportion P CD − DC > P CC − DD , meaning the catalyzedinteractions occur more frequently. Net production of cooper-ators is thus expected. Though this stage is relatively short.ii) When t > t c , clusters are gradually formed. Oncethe cluster property is strengthened, the bulk interaction sce-nario comes into play. Interestingly, a proportion crossover isfound now that P CD − DC < P CC − DD , whereby cooperationis again enhanced since net cooperators are also favored in thisscenario according to Table I.Figure 7(a,c) further show that the four neutral interactingpairs have the major proportions in most of the time for bothcases, though they bring no net production of cooperators ordefectors. The difference between Fig. 7(a) and 7(c) is thatan equilibrium state of coexistence is reached for the case of θ = 0 . , while an absorbing state regarding G is approachedfor θ = 1 where four interfaces except for CC–CD and CC–DC are vanishing.Put together, for the whole processes, the system self-organizes into states with different relative proportions of in-vasion and catalyzed interaction type that make cooperatorscontinuously be produced, no matter clusters are formed ornot. Look back to the mean-field equation Eq. (11), weare now sure that it captures the cooperation mechanism alsofor structured population. When the strategies are not clus-tered, defectors dominate i.e. Π G , C − Π G , D < , but dueto the number difference of the interacting pairs, f CC f DD − f CD f DC < , the second term in Eq. (11) is thus posi-tive. When the strategies are clustered, the opposite is true Π G , C − Π G , D > , f CC f DD − f CD f DC > , again thesecond term is positive and cooperation is enhanced. Thismeans that the game-game interactions prefer cooperation inthe whole evolutionary process and thus explains the dynami-cal reciprocity.A simpler case is starting from half-half patched initialconditions, where bulk scenario sets in from the very begin-ning, see Fig. 7. We can see that the interface proportion ofCC–DD decreases and all others increase, but the inequal-ity P CD − DC < P CC − DD holds along the whole process, nocrossover as in Fig. 7(a,c) is seen. The ensuing dynamics ex-hibits insensitivity to the initial conditions, the evolution isqualitatively the same as in Fig. 7(a,c) in the long term. Thismeans that in the absence of stage i), the dynamics in bulkscenario is still sufficient to yield high level of cooperation. D. Cluster size and compactness analysis
To validate the appropriateness of the two-scenario divisionin the random initial condition case, we further conduct clus-ter size and compactness analysis. While the former is easyto understand and often adopted, the latter is to measure howcompact of clusters, which is defined as the fraction of neigh-bors with identical state regarding the central player (here weadopt Moore neighborhood with eight neighbors), its averagecharacterizes the overall compactness of clusters. The situ-ation with both large average cluster size and compactnessprovides an ideal circumstance for bulk interaction scenarioto play, the individual interaction scenario works for just theopposite case. There may also be cases that the average sizeis large but with small compactness or the way around, whichare in between the two interaction scenarios discussed above.Figure 8 provides the statistical properties of the clustersize for both θ = 0 . and 1. As can be seen, the initialsize of clusters is pretty small, but they become larger as timeevolves, and get saturated in the case of θ = 0 . at around t (cid:38) t c [Fig. 8(a)]. For the cross-play case ( θ = 1 ), however,the cluster sizes continue to increase except the DD players[Fig. 8(b)]. The PDF shown in Fig. 8(c,d) shows that the clus-ter size could typically reach an order of at the late stagebut not for CC or DD clusters in the case of θ = 1 , where theyare either too huge or too small.Figure 9 shows the corresponding compactness analysis.Fig. 9(a,b) show similar profiles of time series when com-pare with Fig. 8(a,b). Note that the peaks of DD species inboth Fig. 8(a,b) and Fig. 9(a,b) are simply due to the invasionof defectors at the initial stage for t < t c . The PDF showsquite a few individuals are of high compactness, especiallyfor the case of θ = 1 , where there are some considerably largeclusters (expect DD clusters) within the population. These ob-servations justified our the two-scenario division, whereby theabove mechanism analysis of the dynamical reciprocity seemsreasonable. CCDCCDDD (b)(a) P D F cluster size P D F cluster size CC (c) DCCD DD P D F cluster size P D F cluster size CDCC (d)
DCDD
FIG. 8. (Color online) Cluster size distribution for two interactingPD games starting from random initial conditions. (a, b) are timeevolution of the average cluster size of the four species for θ = 0 . and 1, respectively. (c, d) are probability function distributions ofcluster size at t = 1000 for θ = 0 . and 1, respectively. In (d), thereis a giant CC cluster with the size comparable to the population size( ∼ N ), which is not shown. Parameters: S = 0 , T = 1 . , R = 1 , P = 0 , L = 1024 for the 2d square lattice. VI. ROBUSTNESS STUDIES
In this section, we turn to provide more evidences to ex-amine the robustness of the dynamical reciprocity by study-ing different variants of the above model, such as asymmet-rically interacting games, games of the time-scale separation,and various updating rules etc.0
CCDCCDDD (b)(a)
CCCD DDDC (c)
CC DCCD DD (d)
FIG. 9. (Color online) Cluster compactness analysis. (a, b) Timeevolution of compactness of the four species for θ = 0 . and 1,respectively. (c, d) Probability function distributions of cluster com-pactness at t = 1000 for θ = 0 . and 1, respectively. Same settingsas in Fig. 8. A. Asymmetrically interacting games
The first relaxation of the above model is to remove thesymmetry assumption by using asymmetric settings, whichcould be more realistic. One variant could be that the twogames are identical, while their impact on each other is as-sumed to be asymmetric. The effective payoffs defined in (a) (b)
FIG. 10. (Color online) Color coded cooperation prevalences fortwo asymmetrically interacting PD games on 2d square lattice within θ − θ parameter space, (a) and (b) are for game G , , respectively.Other parameters: S = 0 , T = 1 . , R = 1 , P = 0 , L = 128 . Eq. (3) are then naturally written as (cid:18)(cid:98) Π G x (cid:98) Π G x (cid:19) = (cid:18) − θ θ θ − θ (cid:19) (cid:18) Π G x Π G x (cid:19) , (12)where the interaction strengths θ , ∈ [0 , represent the con-tribution fraction of game G , in the other game’s effectivepayoff. If a game is more important for the other game thanthe way around, then generally G (cid:54) = G . Note that θ , are not necessarily negatively or positively correlated, bothcould view the other game important or unimportant in theirdecision-makings, up to the specific context.Figure 10(a,b) show respectively the cooperation levels forthe two games within the θ − θ parameter space, where thesymmetrical case studied above is along θ = θ . The overalltrend is qualitatively the same as the symmetrical case that ahigh level of cooperation is expected when θ , become large.In particular, a high cooperation level of a given game, saygame G , is more likely to happen when the other game’scontribution θ is large given its own’s contribution is not toosmall ( θ (cid:38) . ); and nearly full cooperation is reached when θ approaches one. But due to the asymmetry, however, thecooperation prevalences of the two games could be very dif-ferent. B. Interacting games composed of two different types
Another variant is to remove the symmetry in the twogames per se. Interacting games in the real world more ofteninvolve different issues, which should be modeled by differ-ent types of game. Thus, here we study the case of interactinggames composed of two different types of pairwise game, i.e.one is PD game, the other is SD, but with the same interactionstrength θ .To reduce the parameters, a convenient way to do is as fol-lows [33, 34]: the two games share three identical parameters R , P , T , but with opposite signs in S , since < S < for SDand − < S < for PD are required by definition. Figure 11shows three typical interaction strengths in S − T parame-ter space. Without game interaction, they are reduced intotwo independent games, almost no cooperation is expected1 (a) (b) (c) SDPD
FIG. 11. (Color online) Color-coded cooperation prevalence in S − T parameter space for two interacting games — one is PD and the otheris SD, on the 2d square lattice, for θ = 0 , . , (a-c). They haveopposite sign in S , e.g. the SD at top left corner ( T = 1 , S = 1) interacts with PD with ( T = 1 , S = − at the bottom left. Otherparameters: R = 1 , P = 0 , and L = 128 for the lattice. for PD. As the interaction becomes stronger, both cooperationlevels are promoted. Significant promotion is possible whenthe scenario becomes cross-playing, again high cooperationprevalences arise for the whole parameter space. Due to thedifference of the two games, now the cooperation levels arenot homogeneous, but with certain nontrivial distribution inthe parameter space, remaining for further investigation.Note that here the game-game interaction promotes cooper-ation in both games, which is superior to the previous resultswithin the framework of interdependent network. In [33, 34],the PD and SD are placed on two interdependent networks,and they are coupled through network interdependency. Theirstudy shows that with the increment of coupling, the coopera-tion promotion in PD is at the expense of cooperation declinein SD. C. Interacting games with time-scale separation
An underlying assumption in the above studies is that thetime-scales of involved games are comparable, their updatingrates are assumed to be identical. A further relaxed conditionis to allow for different time scales, or even time-scale sepa-ration — some of them are fast games while others are slowones. This could be the case in reality since different issues bynature has its own paces and hence is reasonably of differenttime scales.To see the impact of the time-scale separation, we studytwo interacting PD games, one is fast, the other is slow. Thetime-scale separation is characterized by the time scale ratio T r ≥ : when the slow game is updated by one generationfor each player, the fast game is updated T r generations onaverage. Apparently, the case of T r = 1 is reduced to identicaltime scale scenario as studied above; and as T r increases, thetime-scale separation becomes stronger.Figure 12 shows the cooperation prevalence for a widerange of time-scale separation for θ = 0 . and 1. In both (b)(a) FIG. 12. (Color online) The impact of time-scale separation of thetwo interacting PD games. The cooperation prevalence f c versus thetime-scale ratio T r for θ = 0 . (a) and θ = 1 (b). No cooperation isseen for θ = 0 for the given parameters. Other parameters: S = 0 , T = 1 . , R = 1 , P = 0 , L = 1024 for the 2d square lattice. cases, the cooperation prevalence in the slow game monotoni-cally declines, while the cooperation level for fast game keepslargely unchanged or even increases within T r < in thecase of θ = 0 . . As T r further increases, the declines of bothgames are observed in Fig. 12(a), while the cooperation levelfor slow game keeps round 0.5 and full cooperation for the fastgame in Fig. 12(b). These results mean that a fairly high levelof cooperation is still able to be maintained when the timescales are not very much separated. A systematic account oftime-scale separation’s impact will be presented elsewhere. D. Different updating rules and sychronicities
The update rule determines how exactly the strategy of in-dividuals evolves in time. There is a variety of update rulesthat have been adopted in the literature, each being conceivedin different backgrounds, and sometimes they yield fairly dif-ferent cooperation outcomes. For completeness, here we alsoexamine four other often used updating rules [41]:(i)
Replicator rule , also known as the proportional imita-tion rule , is inspired by the replicator dynamics. The proce-dure is similar as the Fermi rule case that we randomly pickone node x and one of its neighbor y and the imitation proba-bility linearly depends on the payoff difference as: W gxy ≡ W ( s gy → s gx ) = (cid:40) ( (cid:98) Π gy − (cid:98) Π gx ) / (cid:98) Π g , (cid:98) Π gy > (cid:98) Π gx , , (cid:98) Π gy ≤ (cid:98) Π gx , (13)where (cid:98) Π g = k (max(1 , T ) − min(0 , S )) to ensure the proba-bility W gxy ∈ [0 , .(ii) Multiple replicator rule is a variation of the replicatorrule, where we now check simultaneously the whole neigh-borhood of x , and therefore it’s more probable to change thestrategy of x . With this rule, the probability of player x main-tains its strategy is W ( s gx → s gx ) = (cid:89) y ∈ Ω x (1 − W gxy ) , (14)where W gij is given by (13). The more neighbors an individualhas, the less likely to maintain its strategy.2 -101 00.51 a b c -101 00.51 d e f -101 00.51 g h i j k l FIG. 13. (Color online) Cooperation phase diagram for f G C with other four update rules with AU, for three interaction strengths θ =0 , . , . The update rules are: replicator rule (first row, a-c), multiple replicator rule (second row, d-f), Moran rule (third row, g-i), andunconditional imitation (bottom row, j-l). Random initial conditions are adopted. The four numbers are the average cooperation prevalence forthe corresponding quadrants. Also f G C ≈ f G C due to the symmetrical settings. Parameters: R = 1 , P = 0 , and L = 128 for the 2d squarelattice. (iii) Moran-like rule , also know as death-birth rule , is in-spired by the Moran process in biology. With this rule, anindividual randomly picks one site in its neighborhood includ-ing itself, with the imitation probability defined as W ( s gy → s gx ) = (cid:98) Π gy − (cid:98) Π g (cid:80) i ∈ Ω ∗ x ( (cid:98) Π gi − (cid:98) Π g ) , (15)where Ω ∗ x = Ω x ∪ { x } , the constant (cid:98) Π g is to guarantee the nu- merator positive, (cid:98) Π g = max j ∈ Ω ∗ x ( k j ) min(0 , S ) for pairwisegames.(iv) Unconditional imitation rule , also known as follow-the-best rule , is a deterministic rule. At each time step, everyplayer adopts the strategy of the individual who has the high-est payoff in its neighborhood, given this payoff is greater thanits own.Another complication of the model study is the synchronic-ity of the strategy update. As described in Sec. II, the one used3 -101 00.51 cba -101 00.51 d e f -101 00.51 g h i j k l
FIG. 14. (Color online) Cooperation phase diagram with other four update rules but with SU, for three interaction strengths θ = 0 , . , .Other settings are exactly the same as in Fig. 13. above is the random sequential updating, or simply termed as asynchronous updating (AU). We also examine synchronousupdating (SU), where each individual is updated simultane-ously, thus each player is updated exactly once per generation.Previous studies show that the synchronicity issue is often rel-evant for the cooperation outcome [41]. In the following, weinvestigate the cooperation dynamics of two interacting PDgames using the above four updating rules, with both AU andSU.Figure 13 presents the cooperation prevalence of two inter-acting PD games for the above four updating rules, for AU. Without game interaction, the phase diagram are very simi-lar, except the unconditional imitation rule case [Fig. 13(j)],where its phase diagram differs significantly from the others.As game-game interaction is engaged and becomes stronger,the overall cooperation is promoted in general, and this pro-motion reaches maximal as θ → , the cross-playing sce-nario, in line with the Fermi rule studied above. In this sce-nario, however, there are some new dynamical features emerg-ing. The most significant distinction is that there is no mono-tonic dependence of cooperation on the temptation T , an in-termediate value of T yields a cooperation valley for repli-4 (a) (b) FIG. 15. (Color online) Two interacting PD games on Newman-Watts small-world networks. (a) The cooperation prevalence f c ver-sus the temptation T = b for a couple of interaction strengths θ . (b)The cooperation prevalence f c as a function of φ for three temptationvalues b with θ = 0 . . f G C ≈ f G C due to the symmetrical settings.Parameters: P = S = 0 , R = 1 for the two PD games, the networksize N = 2 with φ = 0 . . cator rule, multiple replicator rule and unconditional imita-tion [Fig. 13(c,f,l)]. Besides, many “defection islands” arisewithin the parameter space for the the Moran rule [Fig. 13(i)].Furthermore, the cooperation prevalence in some quadrantsinstead decreases, like HG and SH.SU doesn’t alter the overall phase diagram of cooperation,as shown in Fig. 14. The cooperative behaviors are quali-tatively the same as the cases with AU, and values are evenslightly larger compared to Fig. 13, where the monotonic de-pendence of T and “defection islands” in the cross-playingscenario remain. Detailed analysis requires inspecting theevolutionary game dynamics, which will be presented else-where.Taken together, the observations in Fig. 13 and Fig. 14mean that the dynamical reciprocity is robust against differ-ent rules and the synchronicity of the strategy updating. VII. INTERACTING GAMES ON COMPLEX NETWORKS
While the above robustness studies focus on the variationsin dynamical aspects, we now turn to the impact of structuralcomplexities, since the underlying connectivities of real pop-ulations are far more complex than regular lattices we studied.The stylized models for complex networks include Erd˝os–R´enyi (ER) random networks, small-world (SW) networks,scale-free (SF) networks, the impact of which on coopera-tion in the single game case has been studied extensively [16],both theoretically and experimentally. Here we are not aimingto examine exhaustedly different topologies, rather we onlystudy three of them and focus on the question: whether thestructural complexities alter qualitatively the working of dy-namical reciprocity?
A. Small-world networks
We adopt Newman–Watts network for the SW networks[44], which are derived from d -dimensional square lattice(here d = 2 ) by adding some additional random links. This (b)(a) FIG. 16. (Color online) Two interacting PD games on Erd˝os-R´enyirandom networks. (a) The cooperation prevalence f c versus thetemptation T = b for a couple of interaction strengths θ . (b) Thecooperation prevalence f c as a function of the average degree forthree temptation values b with θ = 0 . . f G C ≈ f G C due to the sym-metrical settings. Parameters: P = S = 0 , R = 1 for the two PDgames, the network size N = 2 with (cid:104) k (cid:105) = 4 . SW model is thought to be better behaved than the originalnetwork model [45], such as the exclusion of detached possi-bility. Instead of rewiring, shortcuts are added with a proba-bility φ corresponding to each bond of original lattice, so thatthere are dN φ shortcuts on average. The average degree isthen (cid:104) k (cid:105) = 2 d (1+ φ ) . By tuning the parameter φ , the topologycan vary continuously from the regular lattice to small-worldnetworks, and to random networks in principle.Fig. 15(a) provides phase transitions for a couple of inter-action strength θ for φ = 0 . , showing that as θ increases,the cooperation prevalence is continuously promoted; when θ → , the phase transition disappears, and a fairly high levelof cooperation is also observed, irrespective of the control pa-rameter b . Fig. 15(b) shows the cooperation dependence onthe network parameter φ for three b . While a higher tempta-tion b reduces the cooperation prevalence as expected, a larger φ generally enhances the cooperation for large T . However,when the temptation is small (e.g. b = 1 . ), there is an op-timal φ that promotes the cooperation to the largest degree.This observation is in line with previous findings in the singlegame case that a moderate amount of randomness in small-world networks is found to best enhance cooperation [46]. B. Random networks
Erd˝os–R´enyi random networks [47] represent a class oftopologies found in nature. Its construction starts with an en-semble of N isolated individuals, any two nodes are then con-nected with a given probability. In such a way, the degreesof the resulting networks follow a Poisson distribution aroundthe mean value (cid:104) k (cid:105) .The evolutionary outcome of two interacting PD games onER networks is shown in Fig. 16(a). Due to the structural dis-order, the prevalence of cooperation is higher than the caseof 2d square lattice in the absence of game interaction. Byincreasing θ , a continuing promotion of cooperation is ob-served as well, with the cross-playing scenario being the op-timal case likewise. Fig. 16(b) shows the dependence of co-5 FIG. 17. (Color online) Two interacting PD games on BA scale-free networks. The cooperation prevalence f c versus the temptation b for a couple of interaction strengths θ . f G C ≈ f G C due to thesymmetrical settings. Parameters: P = S = 0 , R = 1 for the twoPD games, the network size N = 2 with (cid:104) k (cid:105) = 4 , Now K =0 . due to the average payoff scheme, the flipping noise p f =0 . . operation prevalence on the average degree (cid:104) k (cid:105) , where thereis an optimal (cid:104) k (cid:105) for each case, further increasing the valueof degree results in a cooperation decline. This is because thecase of (cid:104) k (cid:105) → N − is equivalent to the well-mixed pop-ulations, where no cooperation is expected according to theabove mean-field analysis. C. Scale-free networks
We adopt Barab´asi–Albert (BA) networks [48] for the sim-ulations on the scale-free network. The construction is via thegrowth and preferential attachment. The network is startedwith a small fully connected graph as the initial core, andevery newly added node is going to connect (cid:104) k (cid:105) / existingnodes, with a probability proportional to their degrees. Thegenerated networks follow a power-law degree distributionwith an exponent − .Different from the SW or ER random networks, which aretaken as homogeneous networks, scale-free networks are typ-ical heterogeneous networks. This heterogeneity is able toboost cooperation considerably, yielding fairly high level ofcooperation even for a single game. The reason lies in thefact that the hubs are easily to form cooperator backbonesthat drives the whole network to be cooperative. This boost ishowever based upon the accumulated total payoffs, wherebythe hubs are very likely to have higher payoffs and thus themodel players. Once the accumulated payoff is replaced bythe average payoff (i.e. ¯Π x = Π x /k i ), which is argued morereasonable in reality in some previous studies [15, 49, 50], theheterogeneity-induced-enhancement is less significant. Here,we adopt the average payoff scheme in our simulations of twointeracting games.As can be seen in Fig. 17, without game interaction ( θ = 0 ),the enhancement effect of heterogeneity is very much inhib-ited — a low level of cooperation is seen. As the game interac-tion θ increases, the cooperation curve is monotonically lifted, also the cross-playing scenario work best. Note that, in scale-free network case, a flipping noise in posed to inhibited thestrong fluctuations caused by the strong degree heterogeneityin the following way: in each elementary step, with a smallprobability p f to flip the state of the focal player, and with − p f to conduct the standard MC procedure as in Sec. II.With flipping noise, the absorbing state is thus avoided, and itcan be interpreted as the deviation from the logic of imitationdue to the irrationality.In brief, additional complexities in the underlying networksof population only bring some quantitative difference compareto the lattice case, the dynamical reciprocity still works in thecomplex networked populations. VIII. MORE GAMES
Given the results of two interacting games, one naturallywants to see what’s the trend if more games are engaged, sincein the real world there are many more issues unfolded simul-taneously. The question of interest is: what could be expectedif the number of games involved increases, and whether theabove mechanism still holds?
In CSL [39], we have shown that by assuming equal con-tribution for each game, the phase transitions and typical timeseries show clearly that a higher level of cooperation is ex-pected when more games are involved ( m = 1 , , ). Basedupon these observations, one can reasonably extrapolate thata continuing promotion in cooperation is expected when moreand more games are engaged. Here, we plan to study the threegame case in a bit more depth.The effective payoffs following the linear combination forthe three interacting games are now written as (cid:98) Π G x (cid:98) Π G x (cid:98) Π G x = − θ − θ θ θ θ − θ − θ θ θ θ − θ − θ Π G x Π G x Π G x , (16)where the interaction strength θ , , are respectively the con-tribution weights of G , , in other games’ effective payoffs.To reduce the number of parameters, here we adopt a circularparameterization as follows, (cid:98) Π G x (cid:98) Π G x (cid:98) Π G x = − θ (cid:48) − θ (cid:48) θ (cid:48) θ (cid:48) θ (cid:48) − θ (cid:48) − θ (cid:48) θ (cid:48) θ (cid:48) θ (cid:48) − θ (cid:48) − θ (cid:48) Π G x Π G x Π G x , (17)where θ (cid:48) , ∈ [0 , and θ (cid:48) + θ (cid:48) ≤ . By varying these twointeraction strengths, we can systematically study the case ofthree interacting games.The simplest case where θ (cid:48) = θ (cid:48) = θ (cid:48) is shown inFig. 18(a). When θ (cid:48) (cid:46) . , cooperators cannot survivewithin the population for the given parameters, further in-creasing the interaction strength, a continuous phase transi-tion is seen and the cooperation prevalence goes be fairly highwhen θ (cid:48) → / , which corresponds to a cross-playing sce-nario in the three interacting game case.6 (a) (b) FIG. 18. (Color online) Three interacting games. (a) Phase tran-sitions of the three cooperation prevalences versus the interactionstrength θ (cid:48) by assuming θ (cid:48) = θ (cid:48) = θ (cid:48) . (b) Color-coded fractionof cooperators regarding game G ( f G C = (cid:80) X,Y ∈ S f CXY ) withinthe interaction space θ (cid:48) − θ (cid:48) . Due to the restriction of θ (cid:48) + θ (cid:48) ≤ ,the upper right half is unphysical. Note that the cooperation preva-lence for the three games f G , , C are approximately identical due tothe symmetrical setting, as shown in (a). Other parameters: S = 0 , R = 1 , P = 0 , T = 1 . , L = 1024 for the 2d square lattice. Fig. 18(b) provides the more general case where θ (cid:48) , canbe different. We see that a stronger game interaction leads tobetter cooperation holds in general. In particular, high coop-eration does not require the symmetry between θ (cid:48) , ; in fact,as long as θ (cid:48) + θ (cid:48) → , the cross-playing scenario, fairly highcooperation is guaranteed.For three games playing together, The mechanism behindthe promotion is the same as the case of the two interactinggames and the above classification still holds. Specifically,there are now 8 different states and 28 pairwise interactionsby combination. Even Though, these interactions can still beclassified into the above three categories also within a cross-playing scenario θ (cid:48) = θ (cid:48) = 1 / for simplicity, as listed inTable II.Overall, also two scenarios are considered — individualand bulk interactions. And all interacting pairs can similarlybe classified into invasion, neutral, and catalyzed categories.While no net production in cooperation is expected in the neu-tral category, cooperators are either yielded or ruined in theother two categories, and the net effect is opposite in two sce-narios. Additional complexities here, however, are that for agiven pairwise interaction, it could be classified into differ-ent categories for different games; e.g. DCC-CDD belongs tocatalyzed type when playing game G , but is of neutral typewhen playing game G or G . All classifications are also con-firmed by numerical experiments (not shown). IX. SUMMARY AND DISCUSSIONS
In summary, motivated by the facts that different gamesare often coupled in the real world, we formulate their evo-lution in the framework of interacting games. We show thatthis game-game interaction generally boosts the propensitiesof cooperation in all evolved games. To our surprise, the opti-mal promotion occurs when the decision-making of the gamesis completely bind to their own’s payoffs. A rough mean-field treatment reveals that two additional new routes to coopera-tion arise, which are confirmed by further analysis. All thesefindings suggest a new mechanism for cooperation that haslong been neglected — dynamical reciprocity. Exhausted nu-merical evidences for variants both in dynamical and struc-tural aspects have verified the robustness of the dynamicalreciprocity.While the network reciprocity is to facilitate the growth ofcooperator clusters via the structured connectivities [16, 51],the mechanism to lift cooperation in the dynamical reciprocityis through the game-game correlation instead, more specifi-cally through the invasion and catalyzed types of interactions.In this sense, the dynamical reciprocity manifests itself as anentirely new different mechanism. Nonetheless, as shown inthe well-mixed case, the interacting games doesn’t show anyimprovement compare to the single game case. This meansthat the dynamical reciprocity only works in structured popu-lation. Since most populations in the real world are structured,this precondition is easy to meet. Therefore, the two reciproc-ities are expected to go hand in hand to maintain high levelsof cooperation in reality.In the present work, we have only treated the linear gameinteraction case as shown Eq.(3). There, the effective pay-offs perceived is a linear combination of payoffs for differentgames, and the interaction strength is encoded within the com-bination weights. This is of course a simplified formulation inthe real cases, more realistic modeling could be more complexfunctions of the payoffs and the weights. In addition, the ef-fective payoff should also probably incorporate a memory ofpast history for all evolved games, which implies that a non-Markovian model [52, 53] seems more proper.The five strategy updating rules used in this work essen-tially belong to the outward learning, where they imitatethose who are better off. This is the mainstream paradigmof modeling updating. However, a new paradigm proposedrecently [54, 55] is through the inward learning, where thedecision-making of individuals is through introspective ac-tions based on their history. With the help of machine learn-ing [56], they also model the evolution of cooperation, but islimited to the single game case. It would be interesting to seewhat if more games are played in this shifted paradigm, doesthe dynamical reciprocity still work?As the next step, maybe the most important open question,is to verify the dynamical reciprocity in behavioral experi-ments. But due to the great complexities in human beings, theexperiment needs to be carefully designed to be convincing.Ideally, the game-game correlation is tunable; also the com-parison of interacting games within structured and unstruc-tured populations is also necessary according to our theoreti-cal results.Back to the real world, the implications of our works isat least two facets. On one hand, since different issues areoften interweaved in the real world, and highly cooperativebehaviors are abundant, the dynamical reciprocity seeminglyprovides a natural causality explanation for these two obser-vations. On the other hand, to handle those cooperation fail-ures in some vital issues, such as global warming and tradewar, our work suggests that the players should get involved7
Individual interaction Bulk interactionInvasion type CCC + DDD G /G /G −−−−−−−→ DCC/CDC/CCD + DDDCCC + DDC G /G −−−−→ DCC/CDC + DDCCCC + CDD G /G −−−−→ CDC/CCD + CDDCCC + DCD G /G −−−−→ DCC/CCD + DCDDDD + CCD G /G −−−−→ DDD + DCD/CDDDDD + DCC G /G −−−−→ DDD + DDC/DCDDDD + CDC G /G −−−−→ DDD + DDC/CDD CCC + DDD G /G −−−−→ CCC + CDD/DCD/DDCCCC + DDC G /G −−−−→ CCC + CDC/DCCCCC + CDD G /G −−−−→ CCC + CCD/CDCCCC + DCD G /G −−−−→ CCC + CCD/DCCDDD + CCD G /G −−−−→ CDD/DCD + CCDDDD + DCC G /G −−−−→ DCD/DDC + DCCDDD + CDC G /G −−−−→ CDD/DDC + CDCNeutral type CCC + DCC G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ CCD + CDC or DCD + DDCCCD + DDC G −−→ CCD + DCC or CDD + DDCDCC + CDD G −−→ DCC + CCD or DDC + CDDDCC + CDD G −−→ DCC + CDC or DCD + CDDCDC + DCD G −−→ CDC + CCD or DDC + DCDCDC + DCD G −−→ CDC + DCC or CDD + DCD CCC + DCC G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ G −−→ CCD + CDC or DCD + DDCCCD + DDC G −−→ CCD + DCC or CDD + DDCDCC + CDD G −−→ DCC + CCD or DDC + CDDDCC + CDD G −−→ DCC + CDC or DCD + CDDCDC + DCD G −−→ CDC + CCD or DDC + DCDCDC + DCD G −−→ CDC + DCC or CDD + DCDCatalyzed type CDC + DCC G /G −−−−→ CDC/DCC + CCCCCD + CDC G /G −−−−→ CCD/CDC + CCCCCD + DCC G /G −−−−→ CCD/DCC + CCCCDD + DCD G /G −−−−→ CDD/DCD + CCDDCD + DDC G /G −−−−→ DCD/DDC + DCCCDD + DDC G /G −−−−→ CDD/DDC + CDCDCC + CDD G −−→ CCC + CDDCDC + DCD G −−→ CCC + DCDCCD + DDC G −−→ CCC + DDC CDC + DCC G /G −−−−→ DDC + DCC or CDC + DDCCCD + CDC G /G −−−−→ CDD + CDC or CCD + CDDCCD + DCC G /G −−−−→ DCD + DCC or CCD + DCDCDD + DCD G /G −−−−→ DCD/CDD + DDDDCD + DDC G /G −−−−→ DDC/DCD + DDDCDD + DDC G /G −−−−→ DDC/CDD + DDDDCC + CDD G −−→ DCC + DDDCDC + DCD G −−→ CDC + DDDCCD + DDC G −−→ CCD + DDDTABLE II. Categories of interactions in 3 interacting PD games G , , within a cross-playing scheme ( θ (cid:48) = θ (cid:48) = 1 / in Eq.(17)). C = 28 pairwise interactions by combination can still be classified into three categories in either individual or bulk scenarios. in as many games as possible, whereby a decent cooperationshould be expected as a result. This advice could be applicableto different contexts from international affairs to interpersonalrelationships.Finally, our work may also provide an inspiration in thecomplexity science. In complexity science, many systems arestudied within the framework of structure plus dynamics (i.e.the function), where the structure and its impact on dynamicshave been extensively studied with the rise of network science. Our findings suggest that the dynamics-dynamics interactionsmay also harbor a great amount of complexities, which havelargely been underestimated before. Another good example of“more is different” [57] in dynamics is modeling the spreadof multiple infectious diseases [58–61], where the physics re-vealed is fundamentally different from the single one case. Wehope the related communities could put more attention to thepotential complexities arising from the dynamics-dynamicsinteractions in the future.8 Acknowledgements —
This work is supported by the Na-tional Natural Science Foundation of China under Grant Nos61703257 and 12075144, and by the Fundamental ResearchFunds for the Central Universities GK201903012. L. C. ac- knowledges the enlightening discussions with Dirk Brock-mann (HU and RKI) in the early phase of the project, andYing-Cheng Lai (ASU) for helpful comments.
Appendix A: Detailed mean-field treatment
Following Eq.(10) in Sec. IV, we explicitly write down the effective fitness defined as (cid:98) Π G , s = (1 − θ )Π G , s + θ Π G , s , s ∈ S that players perceived and used in their strategy updating, as follows (cid:98) Π G CC = (1 − θ )( f CC R + f CD R + f DC S + f DD S ) + θ ( f CC R + f CD S + f DC R + f DD S ) , (cid:98) Π G CC = (1 − θ )( f CC R + f CD S + f DC R + f DD S ) + θ ( f CC R + f CD R + f DC S + f DD S ) , (cid:98) Π G CD = (1 − θ )( f CC R + f CD R + f DC S + f DD S ) + θ ( f CC T + f CD P + f DC T + f DD P ) , (cid:98) Π G CD = (1 − θ )( f CC T + f CD P + f DC T + f DD P ) + θ ( f CC R + f CD R + f DC S + f DD S ) , (cid:98) Π G DC = (1 − θ )( f CC T + f CD T + f DC P + f DD P ) + θ ( f CC R + f CD S + f DC R + f DD S ) , (cid:98) Π G DC = (1 − θ )( f CC R + f CD S + f DC R + f DD S ) + θ ( f CC T + f CD T + f DC P + f DD P ) , (cid:98) Π G DD = (1 − θ )( f CC T + f CD T + f DC P + f DD P ) + θ ( f CC T + f CD P + f DC T + f DD P ) , (cid:98) Π G DD = (1 − θ )( f CC T + f CD P + f DC T + f DD P ) + θ ( f CC T + f CD T + f DC P + f DD P ) . (A1)The overall effective payoffs are then (cid:98) Π XY = (cid:98) Π G XY + (cid:98) Π G XY = (1 − θ )Π G XY + θ Π G XY + (1 − θ )Π G XY + θ Π G XY = Π G XY + Π G XY = Π XY . (A2)The key observation here is that the interaction strength θ is cancelled out due to the linear combination, which is also reasonablesince θ is the contribution weight that only adjusts the payoff values perceived in a specific game but not the overall payoffs.Specifically, we have (cid:98) Π CC = (2 f CC + f CD + f DC ) R + (2 f DD + f CD + f DC ) S, (cid:98) Π CD = ( f CC + f CD ) R + ( f DC + f DD ) S + ( f CC + f DC ) T + ( f CD + f DD ) P, (cid:98) Π DC = ( f CC + f CD ) T + ( f DC + f DD ) P + ( f CC + f DC ) R + ( f CD + f DD ) S, (cid:98) Π DD = (2 f CC + f CD + f DC ) T + (2 f DD + f CD + f DC ) P. (A3)And the mean fitness is ¯Π = (cid:88) s ∈ S f s (cid:98) Π s = f CC (cid:98) Π CC + f CD (cid:98) Π CD + f DC (cid:98) Π DC + f DD (cid:98) Π DD . (A4)Inset these terms into Eq. (10), the replicator equations are then ˙ f CC = f CC [ f CD ( (cid:98) Π CC − (cid:98) Π CD ) + f DC ( (cid:98) Π CC − (cid:98) Π DC ) + f DD ( (cid:98) Π CC − (cid:98) Π DD )] , ˙ f CD = f CD [ f CC ( (cid:98) Π CD − (cid:98) Π CC ) + f DC ( (cid:98) Π CD − (cid:98) Π DC ) + f DD ( (cid:98) Π CD − (cid:98) Π DD )] , ˙ f DC = f DC [ f CC ( (cid:98) Π DC − (cid:98) Π CC ) + f CD ( (cid:98) Π DC − (cid:98) Π CD ) + f DD ( (cid:98) Π DC − (cid:98) Π DD )] , ˙ f DD = f DD [ f CC ( (cid:98) Π DD − (cid:98) Π CC ) + f CD ( (cid:98) Π DD − (cid:98) Π CD ) + f DC ( (cid:98) Π DD − (cid:98) Π DC )] , (A5)where we used the normalization condition f CC + f CD + f DC + f DD = 1 . These equations can actually be summarized as ˙ f s = (cid:88) s (cid:48) ∈ S [ f s f s (cid:48) ( (cid:98) Π s − (cid:98) Π s (cid:48) )] . (A6)The structure of Eq. (A6) is similar to Eq. (5) and its meaning is straightforward that the change in f s comes from the interactionof individuals within state s and s (cid:48) and their effective payoff difference. The overall effective fitness differences are as follows (cid:98) Π CC − (cid:98) Π CD = ( f CC + f DC )( R − T ) + ( f CD + f DD )( S − P ) , (cid:98) Π CC − (cid:98) Π DC = ( f CC + f CD )( R − T ) + ( f DD + f DC )( S − P ) , (cid:98) Π CC − (cid:98) Π DD = (2 f CC + f CD + f DC )( R − T ) + (2 f DD + f CD + f DC )( S − P ) , (cid:98) Π CD − (cid:98) Π DC = ( f CD − f DC )( R − T ) + ( f DC − f CD )( S − P ) , (cid:98) Π CD − (cid:98) Π DD = ( f CC + f CD )( R − T ) + ( f DC + f DD )( S − P ) , (cid:98) Π DC − (cid:98) Π DD = ( f CC + f DC )( R − T ) + ( f CD + f DD )( S − P ) . (A7)9Now let us consider the evolution of overall cooperator fraction with regard to G by adding the first two equations in Eqs. (A5)and insert the above related terms ˙ f G C = ˙ f CC + ˙ f CD = f CC { f DC ( (cid:98) Π CC − (cid:98) Π DC ) + f DD ( (cid:98) Π CC − (cid:98) Π DD ) } + f CD { f DC ( (cid:98) Π CD − (cid:98) Π DC ) + f DD ( (cid:98) Π CD − (cid:98) Π DD ) } = f CC { f DC [( f CC + f CD )( R − T ) + ( f DC + f DD )( S − P )] + f DD [(2 f CC + f CD + f DC )( R − T ) + (2 f DD + f CD + f DC )( S − P )] } + f CD { f DC [( f CD − f DC )( R − T ) + ( f DC − f CD )( S − P )] + f DD [( f CC + f CD )( R − T ) + ( f DC + f DD )( S − P )] } = ( f CC f DC + f CD f DD )[( f CC + f CD )( R − T ) + ( f DC + f DD )( S − P )] + f CC f DD [(2 f CC + f CD + f DC )( R − T )+(2 f DD + f CD + f DC )( S − P )] + f CD f DC [( f CD − f DC )( R − T ) + ( f DC − f CD )( S − P )] . By replacing f CC f DC + f CD f DD = ( f CC + f CD )( f DC + f DD ) − ( f CC f DD + f CD f DC ) , ˙ f G C = ( f CC + f CD )( f DC + f DD )[( f CC + f CD )( R − T ) + ( f DC + f DD )( S − P )]+ f CC f DD { [(2 f CC + f CD + f DC )( R − T ) + (2 f DD + f CD + f DC )( S − P )] − [( f CC + f CD )( R − T ) + ( f DC + f DD )( S − P )] } + f CD f DC { [( f CD − f DC )( R − T ) + ( f DC − f CD )( S − P )] − [( f CC + f CD )( R − T ) + ( f DC + f DD )( S − P )] } = f G C f G D [ f G C ( R − T ) + f G D ( S − P )] + ( f CC f DD − f CD f DC )[ f G C ( R − T ) + f G D ( S − P )]= f G C f G D (Π G C − Π G D ) + ( f CC f DD − f CD f DC )(Π G C − Π G D ) . Similarly, one can also obtain the equation for G by adding the first and the third equations in Eqs. (A5). ˙ f G C = ˙ f CC + ˙ f DC = f G C f G D (Π G C − Π G D ) + ( f CC f DD − f CD f DC )(Π G C − Π G D ) . (A8)Equation (11) is then obtained. Appendix B: Analytical solutions
To solve the mean-field equation Eq. (A5), we make some arrangements that lead to ˙ f CC = 2 f CC f G D [ f G C ( R − T ) + f G D ( S − P )] , ˙ f CD = f CD ( − f G C + f G D )[ f G C ( R − T ) + f G D ( S − P )] , ˙ f DC = f DC ( − f G C + f G D )[ f G C ( R − T ) + f G D ( S − P )] , ˙ f DD = − f DD f G C [ f G C ( R − T ) + f G D ( S − P )] , (B1)where f G C = f CD + f CC and f G D = f DC + f DD by definition. Based on the observations in the numerical simulation as wellas the symmetrical game setting, it’s reasonable to assume f gCD = f gDC , g ∈ G . In addition, f CC + f CD + f DC + f DD = 1 isalways required. Therefore, only two independent variables (let’s select f CC and f DD ) are present within the four fractions, theothers can be expressed by f CD = f DC = 1 − f CC − f DD , (B2) f G C = 1 + f CC − f DD , (B3) f G D = 1 − f CC + f DD . (B4)Their equations are ˙ f CC = f CC (1 − f CC + f DD )[ (1 + f CC − f DD )2 ( R − T ) + (1 − f CC + f DD )2 ( S − P )] , ˙ f DD = − f DD (1 + f CC − f DD )[ (1 + f CC − f DD )2 ( R − T ) + (1 − f CC + f DD )2 ( S − P )] . (B5)By setting ˙ f CC = ˙ f DD = 0 , we obtain the four solutions: (1) f CC = f DD = 0; (B6) (2) f CC = 1 , f DD = 0; (B7) (3) f CC = 0 , f DD = 1; (B8) (4) (1 + f CC − f DD )2 ( R − T ) + (1 − f CC + f DD )2 ( S − P ) = 0 . (B9)0The stability of these solutions is determined by the eigenvalues of the corresponding Jacobian matrix as J = ∂ ˙ f CC ∂f CC ∂ ˙ f CC ∂f DD ∂ ˙ f DD ∂f CC ∂ ˙ f DD ∂f DD . (B10)(1) For f CC = f DD = 0 , J = ( R + S − T − P )2 00 − ( R + S − T − P )2 . (B11) We have λ = ± ( R + S − T − P )2 , the eigenvalues couldn’t be both negative, thus this solution is unstable in any case.(2) For f CC = 1 , f DD = 0 , J = − ( R − T ) ( R − T )0 − R − T ) . (B12) We have λ = − ( R − T ) , λ = − R − T ) . Therefore, this solution is stable only when T < R .(3) For f CC = 0 , f DD = 1 , J = S − P ) 0 − ( S − P ) ( S − P ) . (B13) We have λ = ( S − P ) , λ = 2( S − P ) . This solution is stable only when S < P .(4) For (1 + f CC − f DD )2 ( R − T ) + (1 − f CC + f DD )2 ( S − P ) = 0 , or equivalently f G C = P − SR + P − S − T , J = f CC (1 − f CC + f DD ) ( R − S − T + P )2 − f CC (1 − f CC + f DD ) ( R − S − T + P )2 − f DD (1 + f CC − f DD ) ( R − S − T + P )2 f DD (1 + f CC − f DD ) ( R − S − T + P )2 . (B14) We have λ = 0 , λ = ( f CC + f DD − f CC − f DD ) ( R − S − T + P )2 . This solution is stable only when ( R − S − T + P ) < ,since f CC + f DD − f CC − f DD > .Put together, the stable solutions of the mean-field equations are exactly recovered to the single pairwise game: the mixedstate (solution (B9)) is stable for SD game region, full cooperation (solution (B7)) is stable for HG game region, full defection(solution (B8)) is stable for PD game region, and SH region is bistable (full cooperation or full defection) because it is theoverlapped region for solution (B7) and (B8). [1] R. Axelrod, The evolution of cooperation (Revised edition) , 93 (2005).[4] M. A. Nowak, Science , 1560 (2006).[5] M. Perc, J. J. Jordan, D. G. Rand, Z. Wang, S. Boccaletti, andA. Szolnoki, Phys. Rep. , 1 (2017).[6] R. L. Trivers, The Quarterly Review of Biology , 35 (1971).[7] M. A. Nowak and K. Sigmund, Nature , 573 (1998).[8] W. D. Hamilton, Journal of Theoretical Biology , 17 (1964).[9] L. Keller, Levels of selection in evolution (Princeton UniversityPress, 1999). [10] D. C. Queller, Nature , 1145 (1964).[11] M. A. Nowak and R. M. May, Nature , 826 (1992).[12] K. Sigmund, C. Hauert, and M. A. Nowak, Proc. Natl. Acad.Sci. U.S.A. , 10757 (2001).[13] F. C. Santos and J. M. Pacheco, Phys. Rev. Lett. , 098104(2005).[14] F. C. Santos, M. D. Santos, and J. M. Pacheco, Nature , 213(2008).[15] R.-Z. Liang, J.-Q. Zhang, G.-Z. Zheng, and L. Chen, PhysicaA , 125726 (2021).[16] G. Szab´o and G. Fath, Physics Reports , 97 (2007).[17] Z. Wang, L. Wang, A. Szolnoki, and M. Perc, Eur. Phys. J. B , 124 (2015). [18] D. G. Rand and M. A. Nowak, Trends in Cognitive Sciences ,413 (2013).[19] S. Dirk, Proc. Natl. Acad. Sci. U.S.A. , 12846 (2012).[20] A. Traulsen, D. Semmann, R. D. Sommerfeld, H.-J. Kram-beck, and M. Milinski, Proc. Natl. Acad. Sci. U.S.A. , 2962(2010).[21] G.-L. Carlos, F. Alfredo, R. Gonzalo, T. Alfonso, J. A. Cuesta,S. Angel, and M. Yamir, Proc. Natl. Acad. Sci. U.S.A. ,12922 (2012).[22] C. Gracia-L´azaro, J. A. Cuesta, A. S´anchez, and Y. Moreno,Scientific Reports , 325 (2012).[23] D. G. Rand, M. A. Nowak, J. H. Fowler, and N. A. Christakis,Proc. Natl. Acad. Sci. U.S.A. , 17093 (2014).[24] P. Hammerstein et al. , Genetic and cultural evolution of coop-eration (MIT press, 2003).[25] R. Cressman, A. Gaunersdorfer, and J.-F. Wen, InternationalGame Theory Review , 67 (2000).[26] M. Chamberland and R. Cressman, Games and Economic Be-havior , 319 (2000).[27] K. Hashimoto, Journal of Theoretical Biology , 669 (2006).[28] V. R. Venkateswaran and C. S. Gokhale, Proceedings of theRoyal Society B , 20190900 (2019).[29] Z. Wang, A. Szolnoki, and M. Perc, Scientific Reports , 1183(2013).[30] J. G´omez-Gardenes, I. Reinares, A. Arenas, and L. M. Flor´ıa,Scientific Reports , 620 (2012).[31] B. Wang, X. Chen, and L. Wang, Journal of Statistical Mechan-ics: Theory and Experiment , P11017 (2012).[32] Z. Wang, A. Szolnoki, and M. Perc, Scientific Reports , 2470(2013).[33] M. D. Santos, S. N. Dorogovtsev, and J. F. Mendes, ScientificReports , 4436 (2014).[34] B. Wang, Z. Pei, and L. Wang, Europhysics Letters , 58006(2014).[35] F. Battiston, M. Perc, and V. Latora, New Journal of Physics , 073017 (2017).[36] Z. Wang, L. Wang, and M. Perc, Phys. Rev. E , 052813(2014).[37] Q. Jin, L. Wang, C.-Y. Xia, and Z. Wang, Scientific Reports ,4095 (2014).[38] K. Donahue, O. P. Hauser, M. A. Nowak, and C. Hilbe, Nature Communications , 1 (2020).[39] “Emergent route towards cooperation in interacting games: thedynamical reciprocity (the companion short letter),”.[40] G. Szab´o and C. T˝oke, Phys. Rev. E , 69 (1998).[41] C. P. Roca, J. A. Cuesta, and A. S´anchez, Physics of Life Re-views , 208 (2009).[42] P. D. Taylor and L. B. Jonker, Mathematical Biosciences ,145 (1978).[43] J. M. Smith, Evolution and the Theory of games (CambridgeUniversity Press, 1982).[44] M. E. J. Newman and D. J. Watts, Phys. Rev. E , 7332 (1999).[45] D. J. Watts and S. H. Strogatz, Nature , 440 (1998).[46] J. Ren, W.-X. Wang, and F. Qi, Phys. Rev. E , 045101 (2007).[47] B. Bollob´as, Random Graphs (Cambridge University Press,2001).[48] A.-L. Barab´asi and R. Albert, Science , 509 (1999).[49] Z.-X. Wu, J.-Y. Guan, X.-J. Xu, and Y.-H. Wang, Physica A , 672 (2007).[50] A. Szolnoki, M. Perc, and Z. Danku, Physica A , 2075(2008).[51] M. A. Nowak and S. Karl, Science , 793 (2004).[52] W.-X. Wang, J. Ren, G. Chen, and B.-H. Wang, Phys. Rev. E , 056113 (2006).[53] N. Kampen, Stochastic Processes in Physics and Chemistry(3rd edition) (North-Holland Personal Library, 2007).[54] S.-P. Zhang, J.-Q. Zhang, L. Chen, and X.-D. Liu, NonlinearDyn. , 3301 (2020).[55] J.-Q. Zhang, S.-P. Zhang, L. Chen, and X.-D. Liu, Phys. Rev.E , 042402 (2020).[56] G. Carleo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld,N. Tishby, L. Vogt-Maranto, and L. Zdeborov´a, Rev. Mod.Phys. , 045002 (2019).[57] P. W. Anderson, Science , 393 (1972).[58] W. Cai, L. Chen, F. Ghanbarnejad, and P. Grassberger, Nat.Phys. , 936 (2015).[59] P. Grassberger, L. Chen, F. Ghanbarnejad, and W. Cai, Phys.Rev. E , 042316 (2016).[60] L. Chen, F. Ghanbarnejad, and D. Brockmann, New J. Phys. , 103041 (2017).[61] L. Chen, Phys. Rev. E99