Deciphering chaos in evolutionary games
DDeciphering chaos in evolutionary games
Archan Mukhopadhyay ∗ and Sagar Chakraborty † Department of Physics, Indian Institute of Technology Kanpur, Uttar Pradesh 208016, India (Dated: February 22, 2021)Discrete-time replicator map is a prototype of evolutionary selection game dynamical models thathave been very successful across disciplines in rendering insights into the attainment of the equilib-rium outcomes, like the Nash equilibrium and the evolutionarily stable strategy. By construction,only the fixed point solutions of the dynamics can possibly be interpreted as the aforementionedgame-theoretic solution concepts. Although more complex outcomes like chaos are omnipresent inthe nature, it is not known to which game-theoretic solutions they correspond. Here we constructa game-theoretic solution that is realized as the chaotic outcomes in the selection monotone gamedynamic. To this end, we invoke the idea that in a population game having two-player–two-strategyone-shot interactions, it is the product of the fitness and the heterogeneity (the probability of findingtwo individuals playing different strategies in the infinitely large population) that is optimized overthe generations of the evolutionary process.
I. MOTIVATION
An authoritative book [1] on evolutionary game dy-namics states: “Complex dynamic behaviour can resultfrom elementary discrete-time evolutionary processes.This is one of the main reasons dynamic evolutionarygame theory deals primarily with continuous-time dynam-ics” . This statement implicitly highlights the fact thatthe casting aside of the complex dynamical behaviour—which essentially means dynamics that don’t settle downonto a stable fixed point solution—is not a rare practicein the literature of evolutionary game dynamics. Thisis very surprising given that the set of evolutionary pro-cesses with exclusively simple and predictable outcomesspans only a minor fraction of enormous possibilities [2–5]. Probably the most omnipresent complex determinis-tic unpredictable behaviour is due to chaos. Of course, weare not implying that the chaos has not been explored inthe context of the evolutionary game dynamics. Withinthe paradigms of the game theory and the theory of evo-lution, issues related to chaos have been presented in thecontext of learning [6–9], emergence of cooperation [10–14], mutation [15, 16], fictitious play [17], imitation gameof bird song [18], Darwinian evolution [19, 20], conscious-ness [21], law and economics [22], and language acquisi-tion [23].What, to the best of our knowledge, is lacking is anattempt to connect the chaotic behaviour with a game-theoretic concept. The reason behind this is also not hardto understand: The field of classical game theory, as ineconomics and other social sciences, has heavily revolvedaround the static equilibrium concept of the Nash equi-librium (NE) [24, 25]; researchers try to refine the notionin order to overcome the stringent requirement of ratio-nality expected from the players of the game and to selectthe best equilibrium among the many possible simulta- ∗ [email protected] † [email protected] neous NEs. In the context of evolutionary biology, theequilibrium outcome of the evolutionary dynamics is ex-pected to be an evolutionarily stable strategy (ESS) [26]that surprisingly turns out to be a refinement of the NE,although the concept of rationality is not invoked whiledefining the ESS. The concept of the NE becomes evenmore welcome to the biologists and the ethologists in thelight of the fact the simple fixed point solutions of theparadigmatic replicator equation [27–29] can be tied tothe ESSs (or the NEs) of the underlying game throughthe folk theorem of the evolutionary games [30]. But,unfortunately, the non-fixed point complex solutions arenot amenable to such simple convenient interpretation. II. INTRODUCTION
In classical two-player–two-strategy one-shot game theNE corresponds to the strategy pair such that the strate-gies in the pair are the best response to each other,thereby, denying any player any gain following its unilat-eral strategy-deviation. The concept is easily extendablefor games—which need not even be one-shot—involvingmany players and many strategies. Consider a normalform game with N pure strategies and real payoff ma-trix U which is N × N dimensional. A mixed strat-egy, p , thus, belongs to an N − dimensional simplex, Σ N whose vertices are the pure strategies. Using thisas the underlying game, we can construct a populationgame [1, 31, 32] between n (pheno-)types that constitutefractions x , x , · · · , x n of an infinitely large population.We represent the state of the population as a columnvector, x = ( x , x , · · · , x n ) T , that specifies a point onan n − dimensional simplex Σ n . Here, the superscript‘ T ’ stands for the transpose operation. Every type canbe mapped on to some strategy in Σ N . Specifically, i thtype—equivalently, the i the vertex of Σ n —in the popu-lation game can be seen as a (possibly mixed) strategy p i ∈ Σ N . The fitness of the i th type can be representedas ( Π x ) i where the ( i, j ) th element of the n × n payoff ma-trix Π of the population game is given by p Ti U p j . In the a r X i v : . [ q - b i o . P E ] F e b simple case of all individual playing the same role in thepopulation, we have a symmetric population game wherea state ˆ x is NE if ˆ x T Π ˆ x ≥ x T Π ˆ x , for all x ∈ Σ n . If theNE is not a pure state, then the ‘ ≥ ’ sign can be strictlyreplaced by an ‘ = ’ sign. The state ˆ x is furthermore anESS of the population game if there exists a neighbour-hood B ˆ x of ˆ x such that for all x ∈ B ˆ x \ ˆ x , the followinginequality holds: ˆ x T Π x > x T Π x . The idea of ESS playsthe central role in the evolutionary game theory sincewhen a population is in this state, the population cannotbe successfully invaded by an infinitesimal fraction of mu-tants with alternative strategy. Strict NE implies ESS,and ESS implies NE. This is the right place to highlightexplicitly that, by definition, the NEs (and the ESSs) canbe connected with only the fixed points of the replicatorequation (or any such deterministic selection dynamics, ifat all) that essentially is a differential equation dictatinghow the state of the population, x , evolves in time.It is not an exaggeration in commenting that the folktheorem has brought about a paradigm shift in the waypopulation biologists interpret the interactions in the sys-tems of their interest; now they can treat the individualsof the system as rational players even though in realityit is the natural selection that fashions their behaviourleading to a mathematically stable solution of the modelequation (e.g., replicator dynamic). This is justified be-cause, owing to the folk theorem, the evolutionary out-come can be predicted conveniently by finding the NE ofthe game modelling the inter-player competition. Natu-rally, when a complex chaotic dynamics is under obser-vation, the folk theorem and the NEs are of no use. Onedoesn’t even know how to interpret chaos in the contextof the plethora of game equilibria available in the richliterature. Primarily the attempt to avoid exactly thissituation resonates in the quote given in the beginningof this paper because chaos is readily witnessed in thediscrete-time evolutionary processes, e.g., the one mod-elled by the discrete-time replicator equation in the mostbasic setting of two-player–two-strategy games.Usually, the discrete-time replicator dynamic is as-sociated with the populations having nonoverlappinggenerations, whereas its continuous-time version mod-els the case of overlapping generations. However, thediscrete-time replicator equation has been proposed forthe overlapping generations case [31, 33, 34] as well.While a continuous-time differential equation can betime-discretized to arrive its discrete version, a discrete-time dynamic stands on its own; e.g., the place of thediscrete-time version [35] of the continuous-time logisticequation [36] in the theory of chaos is paramount. III. REPLICATOR MAP ANDGAME-THEORETIC EQUILIBRIA
A two-player–two-strategy (one dimensional) discrete-time replicator map [37–39] may be written as follows: x ( k +1) = x ( k ) + H x ( k ) (cid:104) ( Π x ( k ) ) − ( Π x ( k ) ) (cid:105) , (1)such that ≤ x ≤ . Here ‘ ( k ) ’ denotes the time step orgeneration and H x ( k ) ≡ x ( k ) (1 − x ( k ) ) is the heterogene-ity for a population state x ( k ) = ( x ( k ) , − x ( k ) ) T . It maybe noted that H x ( k ) is the probability that two arbitrarilychosen individuals belong to two different phenotypes atthe k th generation. In the context of one-locus–two-alleletheory in the population genetics an analogous, heterozy-gosity, quantifies the proportion of heterozygous individ-uals [40].Our strong motivation to work with Eq. (1) arises fromthe facts that (a) it is in line with the Darwinian tenetof the natural selection, i.e, only the types with fitnessesmore than the average fitness of the population have posi-tive growth rate; (b) its fixed points are related to the NEand the ESS through the folk and related theorems; and(c) it exhibits with chaotic solutions even in the simplestcase of two strategies [37, 38], specifically for the anti-coordination games like [41, 42] the leader game and thebattle of sexes. We want to find the hitherto unknowngame-theoretic interpretation of such chaotic solutions.The seed of this endeavour has been sown in a recentpaper [39] that shows how the concepts of the NE and theESS may be extended to show that periodic orbits canbe evolutionarily stable. All that is required is to realizethat instead of fitness, heterogeneity weighted fitness, canbe used to define both the (mixed) NE and the ESS—respectively redefined through H ˆ x (cid:2) ˆ x T Π ˆ x (cid:3) = H ˆ x (cid:2) x T Π ˆ x (cid:3) and H x (cid:2) ˆ x T Π x (cid:3) > H x (cid:2) x T Π x (cid:3) — using a positive definite H ˆ x ∈ (0 , . . One should note that it makes sense toexclusively work with mixed states since our goal is tocomprehend the game-theoretic meaning of the non-fixedpoint outcomes while any pure state is a fixed point ofthe replicator map.Specifically, a periodic orbit is a heterogeneity orbit(HO( m ), which is not to be confused with the abbrevia-tions of the homoclinic orbit or the heteroclinic orbit) andif it is asymptotically stable, it is heterogeneity stable or-bit (HSO( m )); the HO( m ) and the HSO( m ) respectivelyboil down to the NE and the ESS—HO( ) and HSO( )respectively—when one considers fixed point as a trivial1-period orbit. Moreover, HSO( m ) implies HO( m ) justas (mixed) ESS implies (mixed) NE. In this context it isuseful to explicitly define the HO( m ) and the HSO( m ):A sequence of states { ˆ x ( k ) : ˆ x ( k ) ∈ (0 , , k = 1 , , · · · , m } where ˆ x ( k ) (cid:54) = ˆ x ( k ) for all k (cid:54) = k , of a map— x ( k +1) = f ( x ( k ) ) —is an HO( m ) if for all l ∈ { , , · · · , m } , m (cid:88) k =1 H ˆ x ( k ) ˆ x ( l ) T Π ˆ x ( k ) = m (cid:88) k =1 H ˆ x ( k ) x T Π ˆ x ( k ) , (2)for any mixed state x . The sequence is furthermore anHSO( m ) if m (cid:88) k =1 H x ( k ) ˆ x (1) T Π x ( k ) > m (cid:88) k =1 H x ( k ) x (1) T Π x ( k ) , (3)for any trajectory { x ( k ) : x ( k ) ∈ (0 , k = 1 , , · · · , m } of the map starting in an infinitesimal neighbourhood B ˆ x (1) \{ ˆ x (1) } of ˆ x (1) . IV. HETEROGENEITY ADVANTAGEOUSORBIT
We observe that since an HSO( m ) must obey bothEq. (2) and Eq. (3) that on rearrangement yield a com-bined inequality defining the HSO( m ): m (cid:88) k =1 (cid:104) H ˆ x ( k ) (cid:16) ˆ x (1) T Π ˆ x ( k ) (cid:17) − H ˆ x ( k ) (cid:16) x (1) T Π ˆ x ( k ) (cid:17)(cid:105) < m (cid:88) k =1 (cid:104) H x ( k ) (cid:16) ˆ x (1) T Π x ( k ) (cid:17) − H x ( k ) (cid:16) x (1) T Π x ( k ) (cid:17)(cid:105) , (4)where the left hand side is identically zero. Consider thatthe mixed state ˆ x (1) when matched against the HSO( m )equilibrium, accumulates a heterogeneity weighted fit-ness; and so does the mixed state x (1) —a state in theinfinitesimal neighbourhood of the initial state of theHSO( m ). Eq. (4) implies that the amount by which theformer is more than the latter is less than the similar dif-ference between the accumulated heterogeneity weightedfitnesses obtained against the trajectory starting in theinfinitesimal neighbourhood of the HSO( m ) equilibrium.In this sense, to be in the HSO( m ) equilibrium appears tobe disadvantageous for the individuals of the population.This motivates the question that what sequence ofstates is advantageous in the sense described above?Could such an orbit exist in the evolutionary dynam-ics? To this end first we define, what we aptly call het-erogeneity advantageous orbit (HAO( m )), as follows: Asequence of states { ˆ x ( k ) : ˆ x ( k ) ∈ (0 , , k = 1 , , · · · , m } ,where ˆ x ( k ) (cid:54) = ˆ x ( k ) for all k (cid:54) = k , of a map— x ( k +1) = f ( x ( k ) ) —is an HAO( m ) if, m (cid:88) k =1 H ˆ x ( k ) (cid:104) ˆ x (1) T Π ˆ x ( k ) − x (1) T Π ˆ x ( k ) (cid:105) > m (cid:88) k =1 H x ( k ) (cid:104) ˆ x (1) T Π x ( k ) − x (1) T Π x ( k ) (cid:105) , (5)for any trajectory { x ( k ) : x ( k ) ∈ (0 , k = 1 , , · · · , m } of the map starting in an infinitesimal neighbourhood B ˆ x (1) \{ ˆ x (1) } of ˆ x (1) . Note that in contrast with Eq. (4),the HAO( m ) represents a set of states over a few consec-utive generations such that on average any individual is more efficient (in fetching heterogeneity weighted fitness) with respect to an individual in an alternate mutant-invaded state ( { x (1) , x (2) , · · · , x ( m ) } ) playing against thepopulation in the HAO( m ) ( { ˆ x (1) , ˆ x (2) , · · · , ˆ x ( m ) } ) whencompared with the similar play against the populationin the alternate state. In other words, the individualsof the population in the HAO( m ) enjoy an advantagecompared to being in the alternate mutant-invaded pop-ulation state. Obviously, the set of all possible HAOs andthe set of all possible HSOs must be mutually exclusiveand consequently, a stable periodic orbit—always beingan HSO( m ) [39]—can never be an HAO( m ). However,it does not imply that all unstable periodic orbits areHAO( m ) because unstable periodic orbit can be HSO( m )as well [39].Chaos, observed in nature as seemingly erratic unpre-dictable outcomes of deterministic systems, is character-ized in many different ways [43, 44]. For our purpose, wedefine [45] a chaotic orbit as the bounded orbit that isnot asymptotically periodic and has positive maximumLyapunov exponent. Note that unlike the HO( m ), theHSO( m ), and the HAO( m ) (which are game-theoreticconcepts such that they can, in principle, be defined us-ing only the game payoff matrix when the sequences ofstates of interest are given), an orbit of the correspond-ing map and its the dynamical stability are not deter-mined by any game-theoretic considerations but ratherby the theory of dynamical systems. Thus, whether andhow a chaotic orbit corresponds to the aforementionedgame-theoretic concepts is an interesting question. It ishowever obvious that a chaotic orbit, being aperiodic andnon-terminating, cannot correspond to an HO( m ). Nat-urally, it is essential to relax the requirement of the van-ishing of the left hand side of Eq. (5) only while dealingwith the chaotic orbits in our scheme of things.Further considerations bring us to the central result ofthis paper. V. CHAOS AND THE HAO
Theorem:
A heterogeneity advantageous orbit of thereplicator map, Eq. (1), corresponding to the two-player–two-strategy game is either an unstable periodic orbit ora chaotic orbit.
Proof:
Let a sequence { ˆ x (1) , ˆ x (2) , · · · , ˆ x ( m ) } , where ˆ x ( k ) (cid:54) = ˆ x ( k ) for all k (cid:54) = k , be an HAO( m ) of the repli-cator map corresponding to the two-player-two-strategygames. Therefore there exists an infinitesimal neighbour-hood B ˆ x (1) of ˆ x (1) such that for all x (1) ∈ B ˆ x (1) \{ ˆ x (1) } (i.e., | x (1) − ˆ x (1) | → + ), Eq. (5) is satisfied. The in-equality can easily be arranged into the following form: (cid:0) x (1) − ˆ x (1) (cid:1) A m > , where A m ≡ m (cid:88) k =1 H x ( k ) (cid:104) ( Π x ( k ) ) − ( Π x ( k ) ) (cid:105) − m (cid:88) k =1 H ˆ x ( k ) (cid:104) ( Π ˆ x ( k ) ) − ( Π ˆ x ( k ) ) (cid:105) . (6)Consequently, A m / [2 (cid:0) x (1) − ˆ x (1) (cid:1) ] > , and hence lim | x (1) − ˆ x (1) |→ + (cid:12)(cid:12)(cid:12)(cid:12) A m x (1) − ˆ x (1) ) (cid:12)(cid:12)(cid:12)(cid:12) > . (7)On further noticing that the m th iterate, f m ( x ) , of thereplicator map is given by, f m = x (1) + m (cid:88) k =1 H x ( k ) (cid:104) ( Π x ( k ) ) − ( Π x ( k ) ) (cid:105) , (8)Eq. (7) gets recast into lim | x (1) − ˆ x (1) |→ + (cid:12)(cid:12)(cid:12)(cid:12) f m ( x (1) ) − f m (ˆ x (1) ) x (1) − ˆ x (1) (cid:12)(cid:12)(cid:12)(cid:12) > . (9)The inequality (9) implies that lim | x (1) − ˆ x (1) |→ + (cid:20) m ln (cid:12)(cid:12)(cid:12)(cid:12) f m ( x (1) ) − f m (ˆ x (1) ) x (1) − ˆ x (1) (cid:12)(cid:12)(cid:12)(cid:12)(cid:21) > . (10)We straightaway conclude that if the HAO( m ) consistsof a finite number of elements, it is an unstable peri-odic orbit of the map. But if the HAO( m ) consists ofa nonterminating trajectory, i.e., if m → ∞ , then theleft-hand side of Eq. (10) is the (maximum) Lyapunovexponent if the limit exists [44, 46, 47]. Consequently,we conclude that in such a case the HAO( ∞ ) is a chaoticorbit. Q.E.D.
The converse of the theorem isn’t necessarily true: Re-call Eq. (7) to note that the condition of same sign of A m and ( x (1) − ˆ x (1) ) is only a sufficient condition forEq. (10) to be satisfied. In fact, if they are of differentsigns such that ( x (1) − ˆ x (1) ) A m < − x (1) − ˆ x (1) ) , eventhen Eq. (10) is satisfied (also in the limit m → ∞ , giventhe limit exists). The condition can explicitly be writtenas, m (cid:88) k =1 (cid:104) H ˆ x ( k ) (cid:16) ˆ x (1) T Π ˆ x ( k ) (cid:17) − H ˆ x ( k ) (cid:16) x (1) T Π ˆ x ( k ) (cid:17)(cid:105) − m (cid:88) k =1 (cid:104) H x ( k ) (cid:16) ˆ x (1) T Π x ( k ) (cid:17) − H x ( k ) (cid:16) x (1) T Π x ( k ) (cid:17)(cid:105) < − x (1) − ˆ x (1) ) , (11)that clearly is not the condition of the HAO(m) but sat-isfies the one defining the HSO( m ) (see Eq. (4)). Weobserve that Eq. (11) is the sufficient condition for anHSO( m ) to be an unstable periodic orbit.Note that, by construction, an HAO(1) is a mixed statethat is not an ESS. (Similarly, HAO( m ), for any finite m , is not the extension of ESS, viz., HSO( m ).) We, how-ever, know [31] that ESS may be absent for a game payoffmatrix or may even be non-unique, and so it is naturalto wonder how the evolutionary trajectories would be insuch scenarios. Moreover, even if ESS is present, it maybe unattainable, e.g., in the case of continuously many in-finite strategies [48]. In other words, mathematical exis-tence of ESS and its practical realisation are two different m . . . . . . x ( m ) (a) m . . . . . . x ( m ) (b) FIG. 1.
Unattainment of ESS in chaotic two-strategy discrete-time replicator map, Eq. (1).
In subplot (a) black-triangle andblue-circle solid lines depict the chaotic time series (Lyapunovexponent= . ± . ) for the leader game [38, 41, 42] payoffmatrix, Π = (cid:0) . .
50 0 (cid:1) , starting at x = 0 . and x = 0 . respectively, both of which are in the infinitesimal neighbour-hood of the ESS, ˆ x ≈ . (red dashed line). Similarly, insubplot (b) black-triangle and blue-circle solid lines presentthe chaotic time series (Lyapunov exponent= . ± . )for the game of battle of sexes [38, 42] with payoff matrix, Π = (cid:0) . .
80 0 (cid:1) , starting at x = 0 . and x = 0 . respec-tively, both of which are in the infinitesimal neighbourhoodof the ESS, ˆ x ≈ . (red dashed line). aspects that are heavily dependent on the evolutionarydynamics under consideration. For example, FIG. 1 illus-trates that in the case of simple two-player–two-strategyscenario—when the payoff matrix corresponds to someanti-coordination games [49]—although a mixed ESS ispresent, dynamically only chaotic orbits are witnessedunder the discrete-time replicator map; the continuoustime replicator equation [27], however, leads to the ESSin this case. Such chaotic orbits naturally deserve a gametheoretic explanation that is what HAO( ∞ ) may some-times offer.The presence of chaos in replicator equation is inter-esting from ecological perspective as well: Any ecologicalpopulation dynamics model can be thought of as an evo-lutionary game with strategy dependent system parame-ters [50]. Specifically, dynamics of any n -strategy replica-tor equation can be mapped [32, 51] to the n − dimen-sional Lotka–Volterra equation [52, 53] which is widelyused as a basic population dynamics model in theoret-ical ecology [54] and mathematical biology [55, 56]. Itis easy to show that the discrete replicator map as usedin this paper can similarly be mapped to the discrete-time versions of the Lotka–Volterra dynamics [50, 57–62]which are known to lead to chaotic outcomes [61, 62] inthe ecological context. VI. DISCUSSION AND CONCLUSION
The connection between the stable fixed points andthe ESS (a refinement of the NE) is very well known [30,38, 63]. This connection has been further extended [39]to show the stable periodic orbits are nothing but theHSO( m )—a generalization of the concept of the ESS. Ithas been argued that in very simple reinforcement learn-ing games (modelled by replicator dynamics and the rock-paper-scissors game) [6], the players do not play NE butrather their strategies evolve chaotically over time hint-ing that rationality may be an unrealistic condition evenin the simplest setting. Furthermore, economists havealso pointed out that there is a lack of any compellingreason that real agents should play the NE [64]; HomoEconomicus remains elusive in the real world [65]. Thus,dynamically—at least in the mean-field level if one con-siders unavoidable stochastic effects too—it is undeniablethat unpredictable chaotic evolution of strategies shouldbe present in the real world strategic interactions. Thispaper has presented possible evolutionary game-theoreticinterpretations of such chaotic orbits (along with the un-stable period orbits) arising in the replicator map.What is crucial for such interpretations of non-convergent outcomes is to appreciate that the evolution-ary game dynamics, as fashioned by the replicator map,is mathematically not about optimizing the fitness of thephenotypes; rather it is the heterogeneity weighted fit-ness that has to be taken into account. This is the mostimportant implicit message of this paper. It is inter-esting to note that the heterogeneity can be taken as ameasure of diversity in the population; it takes the maxi-mum value when both the types of individuals are equallypresent and the minimum value when only one type ofthe individuals is present. While basic mathematical con-ditions of the NE and the ESS remain effectively intacteven on using the heterogeneity weighted fitness, it pavesway for associating evolutionary meaning to the non-fixedpoint outcomes in the game dynamics. In the process,we find that a chaotic attractor—that has a countablyinfinite number of unstable periodic orbits embedded inan uncountably infinite number of chaotic orbits—in adiscrete-time replicator dynamics essentially correspondsto a collection of game-theoretic equilibria (the HSO( m )and the HAO( m )).Eq. (1) and its related forms are useful in modellingreinforcement learning [66], intergenerational culturaltransmission [67, 68], and imitational behaviour [69]. Incertain social scenarios, this map may be derived us-ing the players’ rational behaviour. [1]. Our replicatorequation exhibits selection monotone dynamics—a classof widely used evolutionary dynamical models [1, 32]which includes replicator dynamics [27], sampling best re- sponse dynamics [70], and stochastically perturbed bestresponse dynamics [71]. We draw special attention to-wards the time-discrete selection monotone i-logit mapthat approximates both replicator map and best responsemap [72] depending on the intensity of myopic ratio-nality. The i-logit map leads to complex chaotic out-come [72], surprisingly, for more rational players. Itshould be mathematically straightforward to extend theresults presented thus far to the entire class of the selec-tion monotone maps including more than two-strategycases (see Appendices A and B).Ever since the focus has shifted from the existence ofthe static equilibrium concepts in the classical game the-ory of von Neumann and Morgenstern [73] towards howthese equilibria are attained, the evolutionary (and simi-lar) game dynamics have come to the fore. Since the con-vergent fixed-point outcomes are not an exhaustive repre-sentation of the real world, we strongly believe that—ashas been the goal of this paper—one must develop newgame-theoretic solution concepts that are realized as thecomplex dynamical outcomes, so common in nature. ACKNOWLEDGMENTS
The authors are thankful to Jayanta Kumar Bhat-tacharjee for helpful discussions.
AIP PUBLISHING DATA SHARING POLICY
Data sharing is not applicable to this article as no newdata were created or analyzed in this study.
Appendix A: Two-player– n -strategy game We consider a population game among n phenotypessuch that i th type individuals constitute fraction x i ofthe infinite population. Thus, the state of the popu-lation can be written as x = ( x , x , · · · , x n ) T that isspecified by a point on an n − dimensional simplex Σ n . A very important point to note is that, althoughthere are now more than two types of individuals, anyinteraction between the individuals is still supposed tobe only pairwise; i.e., as is done customarily in the repli-cator equation, we still have a unique payoff matrix (now n × n ) that specifies outcomes of any one-shot interac-tion since multiplayer interactions are not supposed tobe occurring. This provides a hint that the heterogeneitydefined in the main text for the two-player–two-strategycase should still remain pairwise in the description ofthe system: H ij x ( k ) ≡ x ( k ) i x ( k ) j is the heterogeneity whichfor a population state x ( k ) has been defined in a pair-wise fashion. Every type contributes in ( n − differentpairwise heterogeneities. Clearly, H ij x ( k ) is the probabil-ity that two arbitrarily chosen individuals belong to twodifferent phenotypes— i th and j th types—at the k th gen-eration. It is hence not surprising that H ij x ( k ) appear ex-plicitly in a two-player– n -strategy ( n − dimensional)discrete-time replicator map [1, 37–39, 66–69] that canbe written as, x ( k +1) i = x ( k ) i + n (cid:88) j =1 j (cid:54) = i H ij x ( k ) (cid:104) ( Π x ( k ) ) i − ( Π x ( k ) ) j (cid:105) , (A1)such that ≤ x i ≤ for all i . Here ‘ ( k ) ’ denotes the timestep or generation.It is easy to extend the concept of heterogeneity orbit(HO( m )) and heterogeneity stable orbit (HSO( m )) foran n -strategy game [39]: A sequence of states { ˆ x ( k ) :ˆ x ( k ) ∈ (0 , , k = 1 , , · · · , m } where ˆ x ( k ) (cid:54) = ˆ x ( k ) ∀ k (cid:54) = k , of a map— x ( k +1) i = f ( x ( k ) i ) —is an HO( m ) if ∀ i ∈{ , , · · · , n } and l ∈ { , , · · · , m } , the following holds: m (cid:88) k =1 n (cid:88) j =1 j (cid:54) = i H ij ˆ x ( k ) ˆ x ( l ) Tij Π ˆ x ( k ) = m (cid:88) k =1 n (cid:88) j =1 j (cid:54) = i H ij ˆ x ( k ) x Tij Π ˆ x ( k ) . (A2)Here ˆ x ij (or x ij ) is a mixed state having same fractionof i th type as that of ˆ x (or x ) but consists exclusively of i th and j th types; e.g., ˆ x = (ˆ x , , , − ˆ x , , · · · , and x = (1 − x , , x , , , · · · , . The sequence isfurthermore an HSO( m ) if m (cid:88) k =1 n (cid:88) j =1 j (cid:54) = i H ij x ( k ) ˆ x (1) Tij Π x ( k ) > m (cid:88) k =1 n (cid:88) j =1 j (cid:54) = i H ij x ( k ) x (1) Tij Π x ( k ) , (A3)for any trajectory { x ( k ) : x ( k ) ∈ (0 , k = 1 , , · · · , m } of the map starting in an infinitesimal neighbourhood B ˆ x (1) \{ ˆ x (1) } of ˆ x (1) . It can be again shown [39] that aperiodic orbit must be an HO( m ) and if additionally itis asymptotically stable, it is an HSO( m ) as well. More-over, as is desirable, the HO( m ) and the HSO( m ) boildown to the NE and the ESS respectively when a fixedpoint is considered as a trivial 1-period orbit. Addition-ally, HSO( m ) implies HO( m ) just as (mixed) ESS implies(mixed) NE.Subsequently, in line with the main text, we defineheterogeneous advantageous orbit (HAO( m )) for two-player– n -strategy games as follows: A sequence of states { ˆ x ( k ) : ˆ x ( k ) ∈ (0 , , k = 1 , , · · · , m } , where ˆ x ( k ) (cid:54) = ˆ x ( k ) for all k (cid:54) = k , of a map— x ( k +1) i = f ( x ( k ) i ) —is anHAO( m ) if ∀ i ∈ { , , · · · , n } , m (cid:88) k =1 n (cid:88) j =1 j (cid:54) = i H ij ˆ x ( k ) (cid:104) ˆ x (1) Tij Π ˆ x ( k ) − x (1) Tij Π ˆ x ( k ) (cid:105) > m (cid:88) k =1 n (cid:88) j =1 j (cid:54) = i H ij x ( k ) (cid:104) ˆ x (1) Tij Π x ( k ) − x (1) Tij Π x ( k ) (cid:105) , (A4)for any trajectory { x ( k ) : x ( k ) ∈ (0 , k = 1 , , · · · , m } of the map starting in an infinitesimal neighbourhood B ˆ x (1) \{ ˆ x (1) } of ˆ x (1) . It is straightforward to concludethat a stable periodic orbit can never be an HAO( m ) be-cause all possible HAOs and the set of all possible HSOsmust be mutually exclusive by definition.We do not explicitly show the completely analogoussteps of the proof given in the main text but it canbe easily concluded following little inspection that aheterogeneity advantageous orbit of the replicator map,Eq. (A1), corresponding to the two-player– n -strategygame is either an unstable periodic orbit or a chaoticorbit. Appendix B: Selection Monotone Map
For a two-player– n -strategy games, the map, x i ( k +1) = x i ( k ) + φ i ( x ( k ) ); i = 1 , , · · · , n ; (B1)is a selection dynamics in simplex Σ n , if the followingconditions are satisfied [1]:1. The simplex Σ n is forward invariant.2. For all x ( k ) ∈ Σ n , (cid:80) ni =1 φ i ( x ( k ) ) = 0 .3. φ i ( x ( k ) ) , for all i , is a Lipschitz continuous functionon some open neighbourhood in the simplex Σ n .4. φ i ( x ( k ) ) /x ( k ) i , for all i , is continuous real-valuedfunctions on the simplex Σ n .Now, the dynamics is a monotone selection dynamicsif we impose the following condition of monotonicity: φ i ( x ( k ) ) /x ( k ) i > φ j ( x ( k ) ) /x ( k )) j (whenever, i (cid:54) = j ) if andonly if ( Π x ( k ) ) i > ( Π x ( k ) ) j .The population games, having payoff or fitness beinglinear in the frequencies of the types of the individuals,are known as the matrix games [30]. For such populationthe form of the selection monotone map can easily beargued to be such that for all j ∈ { , , · · · , n } and j (cid:54) = i , φ i ( x ( k ) ) x ( k ) i − φ j ( x ( k ) ) x ( k ) j = β ( x ( k ) ) (cid:104) ( Π x ( k ) ) i − ( Π x ( k ) ) j (cid:105) , (B2)where β ( x ( k ) ) is a positive-definite real function andshould ensure that the simplex remains forward invari-ant. On rearranging, we get x ( k ) j φ i ( x ( k ) ) − x ( k ) i φ j ( x ( k ) ) = β ( x ( k ) )2 H ij x ( k ) (cid:104) ( Π x ( k ) ) i − ( Π x ( k ) ) j (cid:105) , (B3)where the term H ij x ( k ) = 2 x ( k ) i x ( k ) j is the pairwise het-erogeneity. Now if we sum Eq. (B3) for all the possible j values, use condition 2 of selection monotonicity, andEq. (B1), we get the general form of monotone selectiondynamics for two-player– n -strategy matrix games as fol-lows: x ( k +1) i = x ( k ) i + n (cid:88) j =1 j (cid:54) = i β ( x ( k ) )2 H ij x ( k ) (cid:20)(cid:16) Π x ( k ) (cid:17) i − (cid:16) Π x ( k ) (cid:17) j (cid:21) . (B4)On comparing with Eq. (A1), it can be easily seen thatour results of the main text can readily be extended for aselection monotone map if we redefine heterogeneity byscaling with β ( x ( k ) ) as β ( x ( k ) ) H ij x ( k ) .For non-matrix games (non-linear payoff function) thedefinition of the evolutionarily stable state (ESS) itself is somewhat problematic [30]. Our main result in themain text is based on the modification of ESS beyondone-shot games. Thus, in order to extend our ideasfor the selection monotone map corresponding to non-matrix game one needs to be more cautious and its care-ful treatment has been left for the near future. We how-ever comment that it appears that the quantity whicha chaotic orbit in it would optimize should be of theform: β ˆ x ( k ) H ij ˆ x ( k ) ˆ x (1) Tij F (cid:0) Π x ( k ) (cid:1) where F is a real-valued n -dimensional function that is monotonically increasingwith respect to its argument Π x ( k ) . [1] R. Cressman, Evolutionary Dynamics and ExtensiveForm Games , 1st ed., MIT Press Books, Vol. 1 (The MITPress, Cambridge, MA, 2003).[2] B. Skyrms, J. Logic Lang. Infor. , 111 (1992).[3] B. Skyrms, PSA: Proceedings of the Biennial Meeting ofthe Philosophy of Science Association , 374 (1992).[4] R. Ferriere and G. A. Fox, Trends Ecol. Evol. , 480(1995).[5] M. Doebeli and I. Ispolatov, Evolution , 1365 (2014).[6] Y. Sato, E. Akiyama, and J. D. Farmer, Proc. Natl.Acad. Sci. U.S.A. , 4748 (2002).[7] Y. Sato and J. P. Crutchfield, Phys. Rev. E , 015206(2003).[8] Y. Sato, E. Akiyama, and J. P. Crutchfield, Physica D:Nonlinear Phenomena , 21 (2005).[9] J. Sanders, J. Farmer, and T. Galla, Sci. Rep. , 4902(2017).[10] M. Nowak and K. Sigmund, Proc. Natl. Acad. Sci. U.S.A. , 5091 (1993).[11] M. Perc, EPL , 841 (2006).[12] T. You, M. Kwon, H.-H. Jo, W.-S. Jung, and S. K. Baek,Phys. Rev. E , 062310 (2017).[13] M. Krieger, S. Sinai, and M. Nowak, Nat. Commun. ,2192 (2020).[14] R. Chattopadhyay, S. Sadhukhan, and S. Chakraborty,Chaos: An Interdisciplinary Journal of Nonlinear Science , 113111 (2020).[15] W. Schnabl, P. F. Stadler, C. Forst, and P. Schuster,Physica D: Nonlinear Phenomena , 65 (1991).[16] J. M. Bahi, C. Guyeux, and A. Perasso, Int. J. Biomath. , 1650076 (2016).[17] C. Sparrow and S. van Strien, “Dynamics, games and sci-ence i. springer proceedings in mathematics,” (Springer,Berlin, Heidelberg, 2011) Chap. Dynamics Associated toGames (Fictitious Play) with Chaotic Behavior.[18] K. Kaneko, Artif. Life , 163 (1993).[19] D. S. Robertson, J. Theor. Biol. , 469 (1991).[20] D. S. Robertson and M. C. Grant, Complexity , 10(1996).[21] L. R. Vandervert, New Ideas Psychol. , 107 (1995).[22] M. J. Roe, Harv. L. Rev. , 641 (1996).[23] W. G. Mitchener and M. A. Nowak, Proc. R. Soc. Lond.Series B: Biological Sciences , 701 (2004).[24] J. F. Nash, Proc. Natl. Acad. Sci. , 48 (1950).[25] E. van Damme, Stability and Perfection of Nash Equilib- ria (Berlin: Springer Berlin Heidelberg, 1991).[26] J. M. Smith,
Evolution and the Theory of Games (Cam-bridge University Press, Cambridge, 1982).[27] P. D. Taylor and L. B. Jonker, Math. Biosci. , 145(1978).[28] P. Schuster and K. Sigmund, J. Theor. Biol. , 533(1983).[29] K. M. Page and M. A. Nowak, J. Theor. Biol. , 93(2002).[30] R. Cressman and Y. Tao, Proc. Natl. Acad. Sci. ,10810 (2014).[31] J. W. Weibull, Evolutionary Game Theory , 1st ed., Vol. 1(MIT Press Books, The MIT Press,USA, 1997).[32] J. Hofbauer and K. Sigmund,
Evolutionary Gamesand Population Dynamics (Cambridge University Press,Cambridge, 1998).[33] A. Cabrales and J. Sobel, J. Econ. Theory , 407 (1992).[34] K. Binmore, Fun and Games , 1st ed. (D. C. Heath, Lex-ington, MA, 1992).[35] R. May, Nature , 459 (1976).[36] H.-O. Peitgen, H. Jürgens, and D. Saupe,
Chaos andfractals - new frontiers of science (2. ed.). (Springer,2004).[37] D. Vilone, A. Robledo, and A. Sánchez, Phys. Rev. Lett. , (2011).[38] V. Pandit, A. Mukhopadhyay, and S. Chakraborty,Chaos , 033104 (2018).[39] A. Mukhopadhyay and S. Chakraborty, J. Theor. Biol. , 110288 (2020).[40] S. H. Rice, Evolutionary Theory: Mathematical and Con-ceptual Foundations , First ed. (Massachusetts: SinauerAssociates, Inc., 1961).[41] A. Rapoport, Behav. Sci. , 81 (1967).[42] S. Hummert, K. Bohl, D. Basanta, A. Deutsch,S. Werner, G. Theißen, A. Schroeter, and S. Schuster,Mol. BioSyst. , 3044 (2014).[43] M. Cencini, F. Cecconi, and A. Vulpiani, in Series on Ad-vances in Statistical Mechanics , Vol. 17 (World ScientificPublishing Co. Pte. Ltd., 5 Toh Tuck Link, Singapore,2010).[44] J. P. Eckmann and D. Ruelle, Rev. Mod. Phys. , 617(1985).[45] K. T. Alligood, T. Sauer, and J. Yorke, in Textbooks inMathematical Sciences (Springer-Verlag New York, NewYork, USA, 1996). [46] V. I. Oseledets, Tr. Mosk. Mat. Obs. , 179 (1968).[47] M. S. Raghunathan, Israel J. Math. , 356 (1979).[48] M. Nowak and K. Sigmund, Acta Appl Math , 247(1990).[49] F. Kojima and S. Takahashi, Int. Game Theory Rev. ,667 (2007).[50] T. L. Vincent and J. S. Brown, Annu. Rev. Ecol. Evol.Syst. , 423 (1988).[51] I. M. Bomze, Biol. Cybern. , 201 (1983).[52] A. J. Lotka, J. Am. Chem. Soc. , 1595 (1920).[53] V. Volterra, Mem. R. Accad. Naz. dei Lincei , 31 (1926).[54] R. M. May, Stability and complexity in model ecosystems (Princeton University Press,New Jersey,USA, 1973).[55] J. D. Murray, in
Interdisciplinary Applied Mathematics ,Vol. 17 (Springer-Verlag New York,USA, 2002).[56] J. D. Murray, in
Interdisciplinary Applied Mathematics ,Vol. 18 (Springer-Verlag New York,USA, 2003).[57] M. Hassell and H. Comins, Theor. Popul. Biol. , 202(1976).[58] W. M. Schaffer, Ecology , 93 (1985).[59] A. Hastings, C. L. Hom, S. Ellner, P. Turchin, andH. C. J. Godfray, Annu. Rev. Ecol. Evol. Syst. , 1(1993).[60] R. Q. Grafton and J. Silva-Echenique, Mar. Resour.Econ. , 127 (1997). [61] D. Blackmore, J. Chen, J. Perez, and M. Savescu, Chaos,Solitons and Fractals , 2553 (2001).[62] G. Bischi and F. Tramontana, Comm. Nonlinear Sci. Nu-mer. , 3000 (2010).[63] J. H. Nachbar, International Journal of Game Theory ,59 (1990).[64] D. M. Kreps, Game Theory and Economic Modelling (Oxford: Oxford University Press, 1990).[65] Y. Babichenko and A. Rubinstein, in