Exit rights open complex pathways to cooperation
Chen Shen, Marko Jusup, Lei Shi, Zhen Wang, Matjaz Perc, Petter Holme
EExit rights open complex pathways to cooperation
Chen Shen , , ∗ Marko Jusup , ∗ Lei Shi , † Zhen Wang , ‡ Matjaz Perc , , , and Petter Holme
1. School of Statistics and Mathematics, Yunnan University of Finance and Economics, Kunming 650221, China2. Tokyo Tech World Hub Research Initiative, Institute of Innovative Research,Tokyo Institute of Technology, Tokyo 152-8550, Japan3. Center for OPTical IMagery Analysis and Learning (OPTIMAL) and School of Mechanical Engineering,Northwestern Polytechnical University, Xi’an 710072, China4. Faculty of Natural Sciences and Mathematics, University of Maribor, 2000 Maribor, Slovenia5. Department of Medical Research, China Medical University Hospital, China Medical University, Taichung 404, Taiwan6. Complexity Science Hub Vienna, 1080 Vienna, Austria (Dated: October 1, 2020)We study the evolutionary dynamics of the prisoner’s dilemma game in which cooperators anddefectors interact with another actor type called exiters. Rather than being exploited by defectors,exiters exit the game in favour of a small payoff. We find that this simple extension of the gameallows cooperation to flourish in well-mixed populations that adhere to either direct or indirectreciprocity. In combination with network reciprocity, however, the exit option is less conducive tocooperation. Instead, it enables the coexistence of cooperators, defectors, and exiters through cyclicdominance. Other outcomes are also possible as the exit payoff increases or the network structurechanges, including network-wide oscillations in actor abundances that may cause the extinctionof exiters and the domination of defectors, although game parameters should favour exiting. Thecomplex dynamics that emerges in the wake of a simple option to exit the game implies that nuancesmatter even if our analyses are restricted to incentives for rational behaviour.
Keywords: Evolutionary game theory; Cooperation; Coexistence; Cyclic dominance; Oscillations
In economic game theory, the conditions and conse-quences of quitting a game [1], and voluntary participa-tion in general, are fundamental topics [2]. In the the-ory of the evolution of cooperation, however, they arerarer guests [3, 4]. Because evolutionary game theorytraditionally concerns the competition between species,it is not surprising that the primary focus is on involun-tary interactions [5]. Nevertheless, there is an increasinginterest in modelling the interface between cooperationand social behaviour in human populations. We willalso take this route and extend the canonical model ofcooperation between selfish individuals – the prisoner’sdilemma [2, 6, 7] – with an option of exiting the game.To more realistically incorporate sociality, our players, oractors, will interact over model social networks [7–9].There are historical examples where the option to exita game could have had a dramatic impact on the out-come. In the final years of the 1950s, China carried outfar-reaching collectivisation of its society. Everyone inthe countryside had to belong to a ‘people’s commune’where people shared everything – farming tools, seedingcrops, draft animals, kitchens, and health care. Even pri-vate cooking was banned and replaced by communal can-teens. Between 1958 and 1962, one of the worst faminesin the history of humanity struck the country [10]. Eversince then, scholars have debated the connections be-tween these social changes and the famine [11]. ∗ Equal contribution † shi [email protected] ‡ [email protected] One intriguing theory was proposed in 1990 by theeconomist Justin Yifu Lin of Peking University [12]. Hepointed out that with the establishment of the people’scommunes, leaving a collective was no longer an option.He reasoned that this revocation of the right to exit tookaway a disincentive to free ride, as now farmers could nolonger avoid negative feedback loops of perfidy. Just howimportant this mechanism was in the onset of famine hasbeen debated. For example, Refs. [13, 14] contend thatLin was wrong using various economic arguments, whileRefs. [15, 16] lend support to the general idea of exitoptions promoting cooperation.We will not dwell further on the question of how wellLin’s hypothesis explains the connection between the col-lectivisation and famine. Instead, intrigued by this his-torical example, we will investigate in a more genericsetting how much a simple right to exit can impact theevolution of cooperation. Our starting point is the pris-oner’s dilemma – a basic mathematical formulation of thesituation in which cooperation would be most beneficialin the long run, but only considering the next interac-tion, defection would be advantageous [2, 5, 7]. Thereare many mechanisms promoting cooperation in the pris-oner’s dilemma. Ref. [17] divides these mechanisms intofive categories – kin and group selection, as well as direct,indirect, and network reciprocity. Others try to identifycommon principles behind all these mechanisms [18, 19].People interact in social networks [7]. The structure ofthe networks can influence the game dynamics. There-fore, many authors have investigated games in which ac-tors interact over model networks [8, 9]. We will inves-tigate the prisoner’s dilemma with an exit option on the a r X i v : . [ q - b i o . P E ] S e p TABLE I. Payoff matrix for the weak prisoner’s dilemmawith an exit option.
C D EC
D b
E (cid:15) (cid:15) (cid:15)
The first row indicates that when a cooperator, C , meetsanother cooperator, defector D or exiter E , she earns a payoffequal to one, zero or zero, respectively. Analogously, whena defector meets a cooperator, defector or exiter, she earn apayoff equal to b ∈ (1 , (cid:15) , 0 ≤ (cid:15) <
1, irrespective ofwhom they meet. regular lattice, as well as three additional types of net-work models: (i) small-world networks that have manytriangles and short path-lengths characteristic of socialnetworks [20], (ii) random regular graphs known to bevery robust to perturbations, and (iii) scale-free networksthat have fat-tailed degree distributions characteristic ofsocioeconomic systems [21].In the extension of the prisoner’s dilemma game thatwe consider, we assume that exiting is a third strategyalongside cooperating and defecting. An exiter receivesa small reward for never playing the game again. We be-gin our analysis with a well-mixed population where weanalyse both one-shot and iterated versions of the pris-oner’s dilemma games with an exit option. After that,we progressively add more network structural complexityby considering populations in a lattice formation, as wellas homogeneous and heterogeneous networks.
RESULTS
Well-mixed populations.
We start our analysis fromone of the simplest possible situations. Specifically, weconsider a one-shot prisoner’s dilemma with an exit op-tion in a well-mixed population. We simplify the ex-position without much loss of generality by assuming thepayoff structure of the weak prisoner’s dilemma (Table I)[22]. Under these conditions, the existence of the exit op-tion is in no way helpful in establishing cooperation; seeSupplementary Information (SI) Remark 1. Actors sim-ply choose to exit the game even if the payoff obtainedby doing so is arbitrarily small; it is better to have somereturn with certainty than to risk getting exploited bydefectors.The situation changes when we replace the single-shotgame by an iterated game. Iterations, provided the gameproceeds sufficiently many rounds, may favour coopera-tion; see SI Remark 2 and Ref. [6]. The exit option helpsto eliminate defection irrespective of how small the exitpayoff is. Without the fear of defection, actors ultimatelychoose to cooperate because cooperation is more benefi-cial than exiting the game. If we extend the game byadding a variable representing actor reputation, the ef-
FIG. 1.
Cooperation is sustained, but rarely dominant,in networked populations with exit.
We plot the full (cid:15) - b phase diagram as obtained by Monte Carlo simulations ofthe weak prisoner’s dilemma comprising an exit option in alattice. When the exit option is highly rewarding, (cid:15) (cid:39) . (cid:15) (cid:47) .
52, leadsto four different outcomes. If temptation is small, b (cid:47) . eo ipso ensures that cooperators remainin the population indefinitely alongside defectors (the C + D phase). Larger temptation values, b (cid:39) .
04, lead to defectordomination for (cid:15) ≤ D phase), but otherwise sustainthe coexistence of all three actor types (the C + D + E phase)or lead to cooperator domination (the C phase). Note thatpurely cooperative outcomes emerge only over a small domainof the (cid:15) - b phase plane. fect is the same; see SI Remark 3. Our theory thus showsthat for well-mixed populations, the availability of theexit option supports cooperation, but only accompaniedby another mechanism, e.g., direct or indirect reciprocity,that makes cooperation a viable option in the first place.These results open the question of what happens whenthe exit option is available in conjunction with networkreciprocity. Regular lattice.
To answer the question of how co-operation fares in networked populations with an exitoption, we resorted to numerical Monte Carlo simula-tions; see Methods for details. We first performed simula-tions in lattices characterised by the von Neumann neigh-bourhood and the periodic boundary conditions. Thegame parameters were the payoffs b , 1 < b ≤
2, and (cid:15) , − . < (cid:15) < (cid:15) (cid:39) .
52, exiters outcompete otheractor types (the E phase in Fig. 1). Conversely, when FIG. 2.
Time dependence of ac-tor abundances reveals cyclicdominance or sole game win-ners. A,
In the D phase, de-fectors win by eliminating first ex-iters, then cooperators. B, In the C + D + E phase, oscillating actorabundances are a signature of cyclicdominance. C, In the C phase, therise of exiters drives defectors to ex-tinction, while rare cooperators sur-vive and later prosper. There is,however, a narrow margin for thisto happen. D, In the E phase, therise of exiters wipes out cooperatorseven before defectors. We show theresults for over 10 timesteps, whichwas sufficient for actor abundancesto stabilise. (cid:15) (cid:47) .
52, there are four possible outcomes. Small temp-tation b (cid:47) .
04 allows network reciprocity alone to securethe coexistence of cooperators and defectors, while ex-iters get eliminated from the population (the C + D phasein Fig. 1). Larger temptation b (cid:39) .
04 gives rise either to(i) defector domination for (cid:15) ≤
0, (ii) the coexistence ofall three actor types, or (iii) cooperator domination (re-spectively, the D phase, the C + D + E phase, and the C phase in Fig. 1). A chief distinction between well-mixedand networked populations emerging from these resultsis that the latter permit dimorphic and trimorphic equi-libria in which the different types of actors coexist. Theexit option thus seems unable to entirely displace defec-tion in networked populations, which is in contrast to ourfindings in well-mixed populations with either direct orindirect reciprocity, as described above.We can gain a better understanding of how the threetypes of actors affect one another by looking at thechange of their abundances through time (Fig. 2). Inthe D phase (Fig. 2A), exiters are the first to give way todefectors, followed shortly thereafter by cooperators. Inthe C + D + E phase (Fig. 2B), it is cooperators who startgiving way to defectors, but then – with less cooperatorsaround – exiters temporarily outnumber defectors. Fewerdefectors, in turn, allow cooperators to partly recover atthe expense of exiters. This proceeds until recovering co-operators once more start giving way to defectors. Wehave thus described a phenomenon called cyclic domi-nance by which three actor types dominate one anotherin an intransitive manner. In our case, cooperators dom-inate exiters who dominate defectors who dominate co-operators. Cyclic dominance has proven influential in ecological [23] and evolutionary game-theoretic [24] con-texts, especially in voluntary dilemmas and extensionsthereof [3, 25, 26].The phenomenon of cyclic dominance disappears in the C phase (Fig. 2C) because here, a substantial rise of ex-iters drives defectors to extinction. At the same time, atiny fraction of cooperators survive and, in the absenceof defectors, eventually takes over the lattice. The riseof exiters in the E phase (Fig. 2D), however, is so force-ful that they wipe out cooperators even before defectors,thus remaining the sole actor type in the lattice.The relationships between actor types described herecould be seen as power relations. In Fig. 3, we fur-ther analyse such relations by examining the equilibriumabundances of cooperators, defectors, and exiters alongseveral transects of the phase space. These horizontaltransects reveal power relations between the three ac-tor types depending on the temptation payoff, b . In theusual weak prisoner’s dilemma without exit, this payoffis equivalent to dilemma strength [27] and thus a cru-cial determinant of the game outcome. Here, we findthat when exiting is neutral or costly ( (cid:15) ≤ b (cid:47) . < (cid:15) (cid:47) . b (cid:47) . FIG. 3.
Power relations be-tween cooperators, defectors,and exiters exhibit intricatepatterns. A,
Along the horizon-tal transect of the (cid:15) - b phase planeat (cid:15) = 0, network reciprocity aloneis enough to secure the coexistenceof cooperators and defectors for b (cid:47) .
04. Thereafter, defectors prevail. B, Along the horizontal transectat (cid:15) = 0 .
46, cooperators, defec-tors, and exiters coexist over thetemptation range 1 . (cid:47) b (cid:47) . b (cid:39) .
90, cooperators dominate. C, Along the vertical transect of the (cid:15) - b phase plane at b = 1 .
02, net-work reciprocity secures the coexis-tence of cooperators and defectorsup to a relatively large exit payoffof (cid:15) ≈ .
45. Between 0 . (cid:47) (cid:15) (cid:47) .
50, all three actor types coexist,whereas between 0 . (cid:47) (cid:15) (cid:47) . D, Along the vertical tran-sect at b = 1 .
4, defectors dominatefor (cid:15) ≤
0, the three actor types co-exist for 0 < (cid:15) (cid:47) .
49, coopera-tors dominate over a narrow stripbetween 0 . (cid:47) (cid:15) (cid:47) .
52, and fi-nally, exiters dominate thereafter.Symbols (squares, circles, and tri-angles) indicate the average steady-state abundances of the three actortypes. pense of defectors and later of cooperators. Temptationthus fails to entice defection but instead pushes actors toexit the game. This ultimately hurts defectors who caneven go extinct by temptation being too large ( b (cid:39) . . (cid:47) (cid:15) (cid:47) . (cid:15) - b phase plane also reveal power relations be-tween the three actor types, but this time depending onthe exit reward (cid:15) . For small temptation values, b (cid:47) . (cid:15) ≈ .
45, ex-iters are able to reduce the abundance of defectors, andafter crossing (cid:15) ≈ .
50, defectors are eliminated, thusallowing cooperators to flourish (Fig. 3C). Cooperatordomination, however, is short-lived because already be- yond (cid:15) ≈ .
52, cooperators die out ahead of defectors,so exiters ultimately prevail (Fig. 3C). For larger temp-tation (1 . (cid:47) b (cid:47) . (cid:15) ≈ .
49 there is again a narrowstrip of cooperator domination, followed by a region ofexiter domination (Fig. 3D). The situation changes below (cid:15) ≈ .
49 because network reciprocity is replaced by cyclicdominance, which ensures the coexistence of all three ac-tor types between 0 < (cid:15) (cid:47) .
49. Defectors prevail if (cid:15) ≤ (cid:15) ≈ .
21, then goes downin favour of cooperators from (cid:15) ≈ .
30, only to go upone more time at the expense of defectors from (cid:15) ≈ . FIG. 4.
Snapshots of evolutionary dynamics expose in detail the interactions between actor types.
When theexit payoff is negative (top row), both cooperators and defectors oust exiters, who get eliminated first. Afterwards, defectorsprevail. When the exit payoff is small-but-positive (second row), cyclic dominance ensues as recognisable by the eventualpatchy distribution of actor types. A larger positive exit payoff (third row) enables exiters to eliminate defectors, but in thestruggle between remaining cooperators and exiters, the former prevails. When the positive payoff is even larger (bottom row),defectors and exiters ousts cooperators. Defectors ultimately lose to exiters who dominate alone. All simulations were runwith temptation b = 1 . eration will soon increase. The margin for cooperatordomination is generally narrow and widens only for thelargest temptation that we consider 1 . (cid:47) b ≤ C -phase in Fig. 1).In contrast to the time-series in Fig. 2, which showthe aggregate development of actor abundances along thetemporal dimension, snapshots of evolutionary dynam-ics provide insights into the development of local actorabundances along both spatial and temporal dimensions(Fig. 4). Snapshots thus open up the opportunity toreexamine the described phenomena from a microscopicperspective. Fixing temptation to b = 1 .
9, we learn thatnon-positive exit payoffs make exiters weaker than coop-erators or defectors (top row in Fig. 4). Consequently,cooperators and defectors jointly eliminate exiters, af- ter which cooperators succumb to defectors (top row inFig. 4). This sequence of events no longer transpireswhen the exit payoff turns positive. Then, instead, allthree actor types get perpetually stuck in a loop of cyclicdominance (second row in Fig. 4). Making the exit pay-off even more positive allows small pockets of coopera-tors to survive until the elimination of defectors by ex-iters. Afterwards, cooperators dominate exiters (thirdrow in Fig. 4). Finally, if the exit payoff becomes toolarge, then even cooperators cannot stand up to exiters.Pressured from both defectors and exiters, cooperatorsget eliminated first, while defectors experience the samefate shortly thereafter, leaving exiters to dominate alone(bottom row in Fig. 4).The above analysis of evolutionary snapshots demon-
FIG. 5.
Network structure is an important determinant of evolutionary dynamics in the studied game. A,
Regular small-world networks have a smaller diameter, but otherwise remain similar to the regular lattice from which they wereconstructed. Consequently, the results here resemble those in Fig. 3C, D for both small temptation ( b = 1 .
02) in the upperpanel and larger temptation ( b = 1 .
4) in the lower panel. B, Random regular networks differ more extensively from the regularlattice than regular small-world networks. This affects evolutionary dynamics. For example, there is no cyclic dominance inthe upper panel when temptation is small ( b = 1 . b = 1 . . (cid:47) (cid:15) (cid:47) . C, Scale-free networks, constructed using the Barab´asi-Albert algorithm [28], give rise to fundamentally different evolutionarydynamics compared to other network structures. These networks support a large cooperator abundance even at temptation b = 2, as seen in the upper panel. Some level of cooperation is possible even at temptation b = 4, as seen in the lower panel.The coexistence of all three actor types, which arises at relatively large values of the exit payoff, is supported by a mechanismdifferent from cyclic dominance. Finally, the sole domination of exiters is seen only at the near-maximum values of the exitpayoff. Symbols (squares, circles, and triangles) indicate the average steady-state abundances of the three actor types. strates that network reciprocity combines with the exitoption differently than direct and indirect reciprocity. Incombination with the latter two reciprocities, an arbitrar-ily small-but-positive exit payoff can undermine defection(SI Fig. S1). In contrast, in combination with networkreciprocity, cooperation can only happen via the coex-istence of all three actor types due to cyclic dominance.How general are these observations? To answer this ques-tion, we proceed to examine whether and how the under-lying network structure affects evolutionary dynamics. Other networks.
To understand the effects of networkstructure on evolutionary dynamics, we ran simulations along the vertical transects of the (cid:15) - b phase plane in threeadditional network types: regular small-world, randomregular, and scale-free (Fig. 5). The results of these sim-ulations are thus analogous – and best understood bycomparing – to the results in Fig. 3C, D. In constructingregular small-world networks, we started with the regularlattice and used random rewiring with the probability of3 % to disconnect two neighbouring nodes and connecttwo nodes that had been distant before. This construc-tion reduced the network diameter but left other prop-erties, e.g., the density of squares, almost unchanged,which is why the simulation results for this network type FIG. 6.
Nature of coopera-tor, defector, and exiter coex-istence changes with networkstructure. A,
In the regularlattice, initial large oscillations inaverage actor abundances quicklydampen and give way to muchsmaller oscillations that are a sig-nature of cyclic dominance. B, Regular small-world networks havea smaller diameter than the regu-lar lattice, but a similar density ofsquares, which somewhat increasesthe amplitude of oscillations. C, Random regular networks have notonly a smaller diameter than theregular lattice, but also a muchsmaller density of squares. This issufficient to trigger global-scale os-cillations that may be large enoughto eliminate exiters in some in-stances. D, In scale-free networks,hub nodes are predominantly co-operative, while small-degree nodesswitch between defection and exit-ing. Temptation is b = 1 . (cid:15) = 0 .
3, except in thescale-free network where b = 4 and (cid:15) = 0 . and the regular lattice are similar (Fig. 5A). The onlynoteworthy difference is that for small temptation values( b = 1 . b = 1 . b = 1 . b = 1 . b = 2, net-work reciprocity supports a large cooperator abundanceup to the exit payoff of (cid:15) ≈ .
48 (upper panel in Fig. 5C). After that, the abundance of exiters increases linearlywith the exit payoff up to (cid:15) ≈ .
91, when this actor typefinally prevails. A similar picture holds even for temp-tation b = 4, except that the abundances of cooperatorsand defectors switch places (lower panel in Fig. 5C).It is illustrative at this point to look at the time-seriesof cooperator, defector, and exiter abundances when allthree actor types coexist (Fig. 6). In the regular lat-tice, we find that initial large oscillations subside ratherquickly, after which there are only small oscillationsaround the average abundances that are characteristicof cyclic dominance (Fig. 6A). The situation is similarin regular small-world networks, although the amplitudeof oscillations around the average abundances is largerthan before (Fig. 6B). The similarity between the time-series in these two cases seems to arise from almost thesame density of squares in regular small-world networksas in the regular lattice. It would appear that squareskeep oscillations local, and thereby small in amplitude(SI Fig. S2A). This is perhaps expected for the regularlattice that lacks long-distance links, but less so for reg-ular small-world networks that are much more compact.Consistent with the above ideas, we further observethat as the density of squares approaches zero in ran-dom regular networks, oscillations become network-wideand develop very large amplitudes (Fig. 6C; see alsoSI Fig. S2). There are even instances in which ampli-tudes are large enough to exterminate exiters, which isthe reason why defector domination appears in the lowerpanel of Fig. 5B when the exit payoff should stronglysupport exiters. We also find that the nature of ac-tor coexistence in scale-free networks is entirely differ-ent compared to other network structures. Hub nodestend to cooperate, while small-degree nodes tend toswitch between defection and exiting, which ultimatelycreates noisy rather than oscillating time-series of actorabundances (Fig. 6D). We visualise the described coexis-tence patterns by animated movies that are available at doi.org/10.17605/OSF.IO/GRHSB . DISCUSSION
We have shown that adding an exceedingly simple exitoption to a weak variant of the prisoner’s dilemma isenough to generate complicated dynamics. In particular,we have seen that in well-mixed populations, an arbitrar-ily small-but-positive exit payoff, (cid:15) >
0, is sufficient todestabilise defection; see SI Remark 1. If there is alsoa viable reciprocity mechanism, cooperators can invadethe population as long as their initial fraction is above (cid:15) (SI Fig. S1B).Combining the exit option with network reciprocityproduces outcomes that differ greatly from those in well-mixed populations [5]. Here, an arbitrarily small-but-positive exit payoff typically leads to the coexistence ofcooperators, defectors, and exiters through cyclic domi-nance. Yet other outcomes are possible as the exit payoffincreases or as the underlying network structure changes.It is particularly interesting that square-dense networkstructures, like the regular lattice, keep cyclic dominancelocal. In contrast, networks without squares, such asrandom regular networks, turn cyclic dominance into aglobal phenomenon. Dramatic oscillations may ensue (SIFig. S2), giving rise to sudden extinction of exiters andsubsequent unexpected domination of defectors.Parallels between our exiters and well-known loners,which rose to prominence as a mechanism behind cyclicdominance [3, 25, 26], undoubtedly invite comparisonsbetween the two. In this sense, the fact that exitersare just as responsible for cyclic dominance in networkedpopulations as are loners shows that non-zero payoffsreceived by cooperators and defectors when interactingwith loners are practically irrelevant for the observed dy-namic phenomena. The exit option can thus be deemed,if not more basic, than at least more economical than theloner option. Exiters, furthermore, leave both coopera-tors and defectors completely hanging when they walkaway from the game, which seems to correspond to vari-ous real-world situations. To exemplify, if completion ofa scientific project rests on collaboration, two genuinelycooperative researchers should be able to complete theproject as planned. If one of the researchers has free-riding tendencies, the project may still get completed,but the invested effort will be asymmetric. If, however,one researcher outright abandons the project for anotherproject with a smaller-but-immediate payoff, the remain- ing researcher is left with little hope for success.The view that exit option is beneficial for cooperationis being challenged by a new psychological study [30]. Ifthe exit payoff is too large, our model supports this ob-servation. The authors of Ref. [30] proceed to concludethat “both research and practice can gain greatly in rich-ness by giving more consideration to exit options in thestudy of cooperation”, which is – given the richness ofour results – a sentiment that we wholeheartedly agreewith.Returning to the example of the Chinese famine, ourresults agree with Ref. [12] in that having an exit op-tion could save cooperation in the system. It is hard tointerpret more of our results in that context; in conform-ing the model to networked evolutionary games, we lostthe connection to that motivational example. Instead, wediscovered that a seemingly minute adjustment to includeexiters, leads to a plethora of dynamic phenomena. Thisshows that nuances matter even if we restrict ourselvesto the goal of economic and evolutionary game theory,that is, to elucidate incentives for rational behaviours.If we wanted to raise the bar and proceed to modellinggeneral human behaviour [19, 31], details of the modelwould be even more important.
METHODS
The key elements of our modelling approach comprised(i) actions and payoffs, (ii) population structure, (iii) ac-tion selection, and (iv) simulation settings. We proceedto briefly describe each of these elements.
Actions and payoffs.
For the sake of simplicity, wechose to base our model on the weak prisoner’s dilemma.In this game, cooperators encountering cooperators re-ceive the payoff equal to unity. Cooperators encoun-tering defectors receive nothing. Conversely, defectorsencountering cooperators receive the temptation payoff b >
1. Defectors encountering defectors receive nothing.We added a third action to this setup, dubbed exit, suchthat exiters typically receive a small-but-positive payoff (cid:15) > b ≤ (cid:15) < Population structure.
We assumed two general typesof populations, well-mixed and networked. In the for-mer case, an actor can encounter any other actor. In thelatter case, an actor encounters only their neighbours asprescribed by the network. The basic network structureused in simulations was the regular lattice in two dimen-sions with the von Neumann neighbourhood. We alsogenerated regular small-world networks and random reg-ular networks by rewiring the underlying lattice, wherethe probability of rewiring any particular link rangedfrom 1 % (small-world) to 99 % (random). To generatescale-free networks for simulation purposes, we used theBarab´asi-Albert algorithm [28].
Action selection.
In well-mixed populations, actionselection followed the usual replicator dynamics. Actorsin networked populations selected their actions throughimitation. Specifically, denoting the payoff earned by afocal actor i with Π i and the payoff of a randomly se-lected neighbour j with Π j , the probability of the actor i imitating the neighbour j was given by the Fermi rule: W i ← j = 11 + exp (cid:16) Π j − Π i K (cid:17) , (1)where K measures the irrationality of selection. Notethat as K →
0, the Fermi rule turns into the Heavi-side step-function such that W i ← j = 0 if Π i < Π j , and W i ← j = 1 if Π i > Π j , while W i ← j = 0 . i = Π j holdsby definition. We set K = 0 . Simulation settings.
We arranged simulations in a se-ries of Monte Carlo timesteps. In each timestep, we ran-domly selected a focal actor who then played the gamewith all their neighbours. We thereafter randomly se-lected one of the focal actor’s neighbours, and allowedthis neighbour to play the game with all their neighboursas well. We finally compared the payoffs of the focal ac-tor and the selected neighbour to determine whether thefocal actor imitates the neighbour or not.We paid special attention in simulations to ensure that(i) transient dynamics had subsided and that (ii) finite-size effects had been eliminated. We thus ran simula- tion for O (cid:0) (cid:1) timesteps, typically 50,000, while aver-aging actor abundances over the last O (cid:0) (cid:1) timesteps,typically 5,000. Networks used in the study contained O (cid:0) (cid:1) nodes, typically 5,000. ARTICLE INFORMATION
Acknowledgements.
This research was supported bythe National Natural Science Foundation of China (grantno. 11931015) to L. S. We also acknowledge supportfrom (i) the China Scholarship Council (scholarshipno. 201908530225) to C. S., (ii) the Japan Society forthe Promotion of Science (grant no. 20H04288) toM. J. as a co-investigator, (iii) the National NaturalScience Foundation of China (grant no. 11671348) toL. S., (iv) the National Natural Science Foundation ofChina (grant no. U1803263), the Thousand Talents Plan(no. W099102), the Fundamental Research Funds for theCentral Universities (grant no. 3102017jc03007), and theChina Computer Federation–Tencent Open Fund (grantno. IAGR20170119) to Z. W, and (v) the Japan Soci-ety for the Promotion of Science (grant no. 18H01655)and the Sumitomo Foundation (grant for basic scienceresearch projects) to P. H.
Author contributions.
C. S., M. J., L. S., and M. P.conceived research. C. S. and M. J. performed simula-tions. All co-authors discussed the results and wrote themanuscript.
Conflict of interest.
Authors declare no conflict of in-terest.
Code availability.
The code used in the study is freelyavailable at doi.org/10.17605/OSF.IO/GRHSB . [1] E. Solan, N. Vieille, Quitting games: An example, Int. J.Game Theory 31 (3) (2003) 365–381.[2] R. B. Myerson, Game Theory: Analysis of Conflict, Har-vard University Press, Cambridge MA, 1991.[3] G. Szab´o, C. Hauert, Evolutionary prisoners dilemmagames with voluntary participation, Phys. Rev. E 66 (6)(2002) 062903.[4] A. Ginsberg, F. Fu, Evolution of cooperation in publicgoods games with stochastic opting-out, Games 10 (1)(2019) 1.[5] J. Hofbauer, K. Sigmund, Evolutionary games and popu-lation dynamics, Cambridge University Press, Cambridge,1998.[6] R. Axelrod, W. D. Hamilton, The evolution of coopera-tion, Science 211 (4489) (1981) 1390–1396.[7] M. O. Jackson, Social and economic networks, PrincetonUniversity Press, 2010.[8] G. Szab´o, G. Fath, Evolutionary games on graphs, PhysicsReports 446 (4-6) (2007) 97–216.[9] Z. Wang, L. Wang, A. Szolnoki, M. Perc, Evolutionarygames on multilayer networks: a colloquium, The Euro-pean Physical Journal B 88 (5) (2015) 124. [10] B. Ashton, K. Hill, A. Piazza, R. Zeitz, Famine in China,1958–61, in: D. L. Poston Jr, D. Yaukey (Eds.), The pop-ulation of modern China, Springer, 1992, pp. 225–271.[11] X. Meng, N. Qian, P. Yared, The institutional causes ofChina’s great famine, 1959–1961, Rev. Econ. Stud. 82 (4)(2015) 1568–1611.[12] J. Y. Lin, Collectivization and China’s agricultural crisisin 1959-1961, J. Polit. Econ. 98 (6) (1990) 1228–1252.[13] X.-Y. Dong, G. K. Dow, Does free exit reduce shirking inproduction teams?, J. Comp. Econ. 17 (2) (1993) 472–484.[14] W. B. MacLeod, The role of exit costs in the theory ofcooperative teams: A theoretical perspective, J. Comp.Econ. 17 (2) (1993) 521–529.[15] J. M. Orbell, P. Schwartz-Shea, R. T. Simmons, Do coop-erators exit more readily than defectors?, Am. Polit. Sci.Rev. 78 (1) (1984) 147–162.[16] J. M. Orbell, R. M. Dawes, Social welfare, cooperators’advantage, and the option of not playing the game, Am.Sociol. Rev. (1993) 787–800.[17] M. A. Nowak, Five rules for the evolution of cooperation,Science 314 (5805) (2006) 1560–1563.[18] I. Eshel, L. L. Cavalli-Sforza, Assortment of encountersand evolution of cooperativeness, Proc. Natl. Acad. Sci. USA 79 (4) (1982) 1331–1335.[19] J. Newton, Evolutionary game theory: A renaissance,Games 9 (2) (2018) 31.[20] D. J. Watts, S. H. Strogatz, Collective dynamics of‘small-world’ networks, nature 393 (6684) (1998) 440–442.[21] F. C. Santos, J. M. Pacheco, Scale-free networks providea unifying framework for the emergence of cooperation,Phys. Rev. Lett. 95 (9) (2005) 098104.[22] M. A. Nowak, R. M. May, The spatial dilemmas of evo-lution, Int. J. Bifurc. Chaos Appl. Sci. Eng. 3 (1) (1993)35–78.[23] T. Reichenbach, M. Mobilia, E. Frey, Mobility promotesand jeopardizes biodiversity in rock–paper–scissors games,Nature 448 (7157) (2007) 1046–1049.[24] A. Szolnoki, M. Mobilia, L.-L. Jiang, B. Szczesny, A. M.Rucklidge, M. Perc, Cyclic dominance in evolutionarygames: a review, J. R. Soc. Interface 11 (100) (2014)20140735.[25] G. Szab´o, C. Hauert, Phase transitions and volunteeringin spatial public goods games, Phys. Rev. Lett. 89 (11)(2002) 118101.[26] H. Guo, Z. Song, S. Geˇcek, X. Li, M. Jusup, M. Perc,Y. Moreno, S. Boccaletti, Z. Wang, A novel route to cyclicdominance in voluntary social dilemmas, J. R. Soc. Inter-face 17 (164) (2020) 20190789.[27] Z. Wang, S. Kokubo, M. Jusup, J. Tanimoto, Universalscaling for the dilemma strength in evolutionary games,Phys. Life Rev. 14 (2015) 1–30.[28] R. Albert, A.-L. Barab´asi, Statistical mechanics of com-plex networks, Rev. Mod. Phys. 74 (1) (2002) 47.[29] P. Holme, C. R. Edling, F. Liljeros, Structure and timeevolution of an Internet dating community, Soc. Netw.26 (2) (2004) 155–174.[30] T. Haesevoets, D. H. Bostyn, C. Reinders Folmer,A. Roets, A. Van Hiel, Decision making in the prisoner’sdilemma game: The effect of exit on cooperation and so-cial welfare, J. Behav. Decis. Mak. 32 (1) (2019) 61–78.[31] H. Gintis, Behavioral game theory and contemporaryeconomic theory, Anal. und Krit. 27 (1) (2005) 48–72. SUPPLEMENTARY INFORMATION FOR“EXIT RIGHTS OPEN COMPLEX PATHWAYS TO COOPERATION”
Remark 1.
We analyse, by means of the replicator equations, the evolutionary dynamics of a one-shot prisoner’sdilemma with exit in a well-mixed population. Let x , y , and z respectively denote the densities of cooperators ( C ),defectors ( D ), and exiters ( E ) in the population, where 0 ≤ x, y, z ≤ x + y + z = 1. The replicator equationsare: ˙ x = x (cid:0) Π C − Π (cid:1) , ˙ y = y (cid:0) Π D − Π (cid:1) , ˙ z = z (cid:0) Π E − Π (cid:1) . (S1)The symbols Π C , Π D , and Π E denote, in that order, the expected payoff from cooperating, defecting, and exiting,whereas Π = x Π C + y Π D + z Π E is the expected per-capita payoff of the whole population. Based on the payoffsdefined in Table I of the main text: Π C = x, Π D = bx, Π E = (cid:15). (S2)Using the constraint z = 1 − x − y , we obtain:˙ x = f ( x, y ) = x [(1 − x ) (Π C − Π E ) − y (Π D − Π E )] = x [(1 − x ) ( x − (cid:15) ) − y ( bx − (cid:15) )] , ˙ y = g ( x, y ) = y [(1 − y ) (Π D − Π E ) − x (Π C − Π E )] = y [(1 − y ) ( bx − (cid:15) ) − x ( x − (cid:15) )] , (S3)This system of equations has four equilibrium points: (1 , , , , , , (cid:15), , − (cid:15) ). To examine thestability of these equilibria, we calculate the Jacobian matrix: J = (cid:34) ∂f ( x,y ) ∂x ∂f ( x,y ) ∂y∂g ( x,y ) ∂x ∂g ( x,y ) ∂y (cid:35) , (S4)where ∂f ( x,y ) ∂x = x (2 − x − by ) + ( − x + y ) (cid:15), ∂f ( x,y ) ∂y = x ( − bx + (cid:15) ) , ∂g ( x,y ) ∂x = y ( b − x − by + (cid:15) ) , ∂g ( x,y ) ∂y = x ( b − x − by ) + ( − x + 2 y ) (cid:15), (S5)and then look at the determinant and the trace of matrix J . The results of the stability analysis are: • For point (1 , , J = ( − b ) ( − (cid:15) ) <
0, indicating that the equilibrium is unstable. • For point (0 , , J = 0 and tr J = (cid:15) ≥
0, again indicating that the equilibrium is unstable forany 0 < (cid:15) < • For point (0 , , J = (cid:15) ≥ J = − (cid:15) ≤
0, indicating that the equilibrium is stable forany 0 < (cid:15) < • Finally, for point ( (cid:15), , − (cid:15) ), we obtain det J = (cid:15) ( − b ) (1 − (cid:15) ) ≥ J = ( b − (cid:15) ) (cid:15) ≥
0, indicating onemore time that the equilibrium is unstable.In summary, all actors exit the game provided that for doing so they receive an arbitrarily small payoff.
Remark 2.
We extend our previous analysis of the one-shot prisoner’s dilemma with exit in a well-mixed populationto a situation when the game is iterated. Iterations mean that two actors play the game for an unspecified numberof rounds determined by a termination probability q . Precisely, the game may be terminated after each round withprobability q , or it may continue with probability 1 − q . We furthermore assume that cooperative actors in theiterated game cooperate conditionally, that is, resort to the tit-for-tat strategy, because it is a well-known result ofevolutionary game theory that unconditional cooperation is easily undermined by defection [1]. Cooperative actorsthus start the game with cooperation, and proceed in the same fashion unless they are paired with a defector, in whichcase defection in the previous round is met with defection by the cooperative actor in the current round. Under suchcircumstances, the payoff matrix transforms into: q b (cid:15)q (cid:15)q (cid:15)q . (S6)2Based on this matrix, the expected payoffs from cooperating, defecting, and exiting become:Π C = xq , Π D = bx, Π E = (cid:15)q . (S7)Consequently, for the functions f ( x, y ) and g ( x, y ) as defined in Eq. (S3), we get: f ( x, y ) = x (cid:104) (1 − x ) x − (cid:15)q − y (cid:16) bx − (cid:15)q (cid:17)(cid:105) ,g ( x, y ) = y (cid:104) (1 − y ) (cid:16) bx − (cid:15)q (cid:17) − x x − (cid:15)q (cid:105) . (S8)The iterated game has the same four equilibrium points as the single-shot game: (1 , , , , , , (cid:15), , − (cid:15) ). To examine the stability of these equilibria, we calculate the elements of the Jacobian matrix (Eq. S4): ∂f ( x,y ) ∂x = xq (2 − x − bqy ) + ( − x + y ) (cid:15)q , ∂f ( x,y ) ∂y = x (cid:16) − bx + (cid:15)q (cid:17) , ∂g ( x,y ) ∂x = yq ( bq − x − bqy + (cid:15) ) , ∂g ( x,y ) ∂y = xq ( bq − x − bqy ) + ( − x + 2 y ) (cid:15)q , (S9)and then look at the determinant and the trace of matrix J . The results of the stability analysis are: • For point (1 , , J = q ( − bq ) ( − (cid:15) ) and tr J = q ( − bq + (cid:15) ). The determinant ispositive, while the trace is negative if q < b . A sufficiently low termination probability, therefore, makesthe fully cooperative equilibrium stable. • For point (0 , , J = 0 and tr J = (cid:15)q ≥
0. Accordingly, irrespective of the termination probability,the fully defecting equilibrium is unstable when there is an arbitrarily small exit payoff. • For point (0 , , J = (cid:15) q ≥ J = − (cid:15)q ≤
0. This indicates that the fully exiting equilibriumis stable provided there is an arbitrarily small payoff associated with the exit option. • For point ( (cid:15), , − (cid:15) ), we obtain det J = (cid:15) q ( − bq ) (1 − (cid:15) ) and tr J = (cid:16) b − (cid:15)q (cid:17) (cid:15) . The determinant here canbe positive if q > b , but in that case the trace is also positive, indicating that the mixed cooperative-exitingequilibrium is unstable.In summary, iterations may overcome the dilemma in favour of cooperation if the termination probability is sufficientlylow. The exit option meanwhile eliminates defection regardless of how small the exit payoff is (Fig. S1). In the mixof cooperators and exiters, the smaller the values of q and (cid:15) , the more likely it is for cooperation to prevail. Remark 3.
The success of iterations in promoting cooperation is due to a mechanism called direct reciprocity thatmanifests in the tit-for-tat strategy of conditional cooperators [2]. The gist of direct reciprocity is that cooperativeactors have the time to assess whether they are paired with cooperators and then act accordingly. This naturallyleads to a question if there could be other, more indirect, ways to assess whether someone is a cooperator. It turns outthat reputation precisely serves this purpose, leading to the evolution of cooperation through indirect reciprocity [3].Assuming that an actor’s reputation is known with a probability p , the payoff matrix of a single-shot game transformsinto: − p ) b (cid:15) (cid:15) (cid:15) . (S10)The expected payoffs from cooperating, defecting, and exiting then become:Π C = x, Π D = (1 − p ) bx, Π E = (cid:15), (S11)while the functions f ( x, y ) and g ( x, y ) as defined in Eq. (S3) turn into: f ( x, y ) = x [(1 − x ) ( x − (cid:15) ) − y ((1 − p ) bx − (cid:15) )] ,g ( x, y ) = y [(1 − y ) ((1 − p ) bx − (cid:15) ) − x ( x − (cid:15) )] . (S12)The single-shot game with indirect reciprocity has the same four equilibrium points as before: (1 , , , , , , (cid:15), , − (cid:15) ). To examine the stability of these equilibria, we calculate the elements of the Jacobian3matrix (Eq. S4): ∂f ( x,y ) ∂x = − x − (1 − y ) (cid:15) + 2 x [1 − b (1 − p ) y + (cid:15) ] , ∂f ( x,y ) ∂y = x [ − b (1 − p ) x + (cid:15) ] , ∂g ( x,y ) ∂x = y [ − x + b (1 − p ) (1 − y ) + (cid:15) ] , ∂g ( x,y ) ∂y = − x + b (1 − p ) x (1 − y ) + ( − x + 2 y ) (cid:15), (S13)and then look at the determinant and the trace of matrix J . The results of the stability analysis are: • For point (1 , , J = [1 − b (1 − p )] (1 − (cid:15) ) and tr J = − b (1 − p ) + (cid:15) . The determinant ispositive, while the trace is negative if p > b − b . A sufficiently high probability of knowing the other actor’sreputation thus makes the fully cooperative equilibrium stable. • For point (0 , , J = 0 and tr J = (cid:15) ≥
0. Accordingly, irrespective of the probability of knowingthe other actor’s reputation, the fully defecting equilibrium is unstable when there is an arbitrarily small exitpayoff. • For point (0 , , J = (cid:15) ≥ J = − (cid:15) ≤
0. This indicates that the fully exiting equilibriumis stable provided there is an arbitrarily small payoff associated with the exit option. • For point ( (cid:15), , − (cid:15) ), we obtain det J = (cid:15) [ − b (1 − p )] (1 − (cid:15) ) and tr J = [ b (1 − p ) − (cid:15) ] (cid:15) . The determinanthere can be positive if p > b − b , but in that case the trace is also positive, indicating that the mixed cooperative-exiting equilibrium is unstable.In summary, indirect reciprocity may overcome the dilemma in favour of cooperation if the probability of knowing theother actor’s reputation is sufficiently high. The exit option meanwhile eliminates defection regardless of how smallthe exit payoff is (Fig. S1). In the mix of cooperators and exiters, the larger the value of p and the smaller the value (cid:15) , the more likely it is for cooperation to prevail.4 SUPPLEMENTARY FIGURES
FIG. S1.
Exiting destabilises defection, while reciprocity ensures cooperation.
In well-mixed populations with director indirect reciprocity, there are three monomorphic evolutionary equilibria (cooperation, defection, and exiting respectivelydenoted C , D , and E ) and one dimorphic equilibrium (cooperation-exiting). Any positive exit payoff, (cid:15) >
0, destabilisesthe D equilibrium. In contrast, whether the C equilibrium is stable or not depends on the strength of reciprocity, that is,the game termination probability under direct reciprocity or the probability of knowing the other actor’s reputation underindirect reciprocity. A, The panel shows a ternary plot of evolutionary dynamics under indirect reciprocity when the exitpayoff is (cid:15) = 0 .
2. The C and E monomorphic equilibria are stable (black disks), whereas the two other equilibria are unstable(white disks). Because of the relatively large exit payoff, evolutionary dynamics converge to the E equilibrium when the initialabundance of cooperators is small. Otherwise, cooperation evolves. B, In the limit of an infinitesimally small exit payoff (here, (cid:15) = 0 . E equilibrium and the dimorphic equilibrium coincide. Cooperation evolves irrespective of howsmall the initial cooperator abundance is. In both ternary plots, the probability of knowing the other actor’s reputation is p = 0 .
75, whereas temptation is b = 1 .
2. Analogous ternary plots can be obtained under direct reciprocity too. FIG. S2.
How local cyclic dominance turns into global oscillations of actor abundances. A,
The panel displays thevariance of actor abundances (estimated over the last 5,000 simulation timesteps) as the functions of the density of squares,where this latter quantity was calculated following the definition in Ref. [4]. Larger variances reflect larger oscillation amplitudesaround equilibrium actor abundances. The density of squares decreases from the regular lattice to random regular networks byincreasing the link rewiring probability from zero to unity. Here, temptation is b = 1 . (cid:15) = 0 . B–G,
Inrandom regular networks (constructed from the regular lattice with the rewiring probability of 0.99), the oscillation amplitudeof actor abundances is large for certain sets of parameter values. Here, temptation is b = 1 .
4, while the exit payoff progressivelyincreases from (cid:15) = 0 .
25 to (cid:15) = 0 .
30 in steps of 0.01. The exiter abundance oscillates with the largest amplitude which is whythis actor type goes extinct when the exit payoff is between 0 . (cid:47) (cid:15) (cid:47) .6