Periodic Orbit can be Evolutionarily Stable: Case Study of Discrete Replicator Dynamics
PPeriodic Orbit can be Evolutionarily Stable: Case Study of Discrete ReplicatorDynamics
Archan Mukhopadhyay ∗ and Sagar Chakraborty † Department of Physics, Indian Institute of Technology Kanpur, Uttar Pradesh 208016, India
In evolutionary game theory, it is customary to be partial to the dynamical models possessing fixedpoints so that they may be understood as the attainment of evolutionary stability, and hence, Nashequilibrium. Any show of periodic or chaotic solution is many a time perceived as a shortcomingof the corresponding game dynamic because (Nash) equilibrium play is supposed to be robust andpersistent behaviour, and any other behaviour in nature is deemed transient. Consequently, thereis a lack of attempt to connect the non-fixed point solutions with the game theoretic concepts.Here we provide a way to render game theoretic meaning to periodic solutions. To this end, weconsider a replicator map that models Darwinian selection mechanism in unstructured infinite-sizedpopulation whose individuals reproduce asexually forming non-overlapping generations. This is oneof the simplest evolutionary game dynamic that exhibits periodic solutions giving way to chaoticsolutions (as parameters related to reproductive fitness change) and also obeys the folk theoremsconnecting fixed point solutions with Nash equilibrium. Interestingly, we find that a modifiedDarwinian fitness—termed heterogeneity payoff—in the corresponding population game must beput forward as (conventional) fitness times the probability that two arbitrarily chosen individualsof the population adopt two different strategies. The evolutionary dynamics proceeds as if theindividuals optimize the heterogeneity payoff to reach an evolutionarily stable orbit, should it exist.We rigorously prove that a locally asymptotically stable period orbit must be heterogeneity stableorbit—a generalization of evolutionarily stable state.
Keywords: Game theory, Evolutionary dynamics, Replicator map, Periodic orbits, Evolutionary stability
I. INTRODUCTION
The influence of Malthus and contemporaryeconomists on the development of the theory ofnatural selection is well-documented [1]; and so isvery well known that it was Darwin [2] who first gave ascientific argument for why the sex ratio in most sexuallyreproducing species is approximately 1:1 between malesand females, or in modern game theoretic parlance,why the sex ratio 1:1 is an evolutionarily stable strat-egy (ESS) [3, 4]. The symbiotic relationship betweeneconomists and biologists (also, sociologists, ethologists,and related researchers) through the evolutionary gametheory, thus, started with Darwin’s work and is stillgoing very strong: While Dawkins [5] remarks that theevolutionary game theoretical concept of evolutionarystability is one of the most important advances inevolutionary theory, the economists do advocate for evo-lutionary theorizing in economics so that the agents arenot seen as merely uncompromising maximisers of profitbut as driven by some sort of selection process [6–9].In a monomorphic population invaded by a tiny frac-tion of mutants, ESS either renders a higher expectedpayoff through its performance against itself or, in casethe strategy ties with that of the mutant’s, it fetches moreexpected payoff against the mutant compared to what themutant would. It is remarkable [10] that a strict Nashequilibrium (NE) is ESS and ESS must be NE. It just so ∗ [email protected] † [email protected] happens that the condition for NE—needed for existenceof ESS—is enough to explain quite a few biological con-flict scenarios. Hence, it not surprising that the conceptof the evolutionary stability has publicised the idea ofNE for non-economists.It may not be very wrong to remark that many re-finements of NE, that solely bank on the ideas relatedto rationality, are not very satisfactory; at least, therationality-based mechanism leading to them in a gamedefinitely is not so. As a result, over the last thirty yearsor so, evolutionary models are slowly but surely beingpreferred to rationality-based models. This is more sobecause now-a-days, rather than interpreting a game asan idealized rational interaction, it is more sensible tointerpret it as a model of an actual interaction whereinan equilibrium is seen as the result of a dynamic adjust-ment process. In this context, it is worth noting that theevolutionary game theory manifests itself through math-ematical models that describe adaptions of the players’behaviours over the course of repeated plays of a gameas a dynamic process. Evolutionary stability concept isnot only able to rule out some of the NEs in case thereare more than one of them, it also unravels the non-rationality-based mechanism for the realizable NEs. Forexample, ESS is known [11] to be both proper (Nash)equilibrium [12] and trembling hand perfect (Nash) equi-librium [13]. Thus, if one sees the attainment of theequilibrium as a consequence of an evolutionary-basedmechanism, then one is no longer faced with the diffi-cultly of explaining why rational players should tremblein a rationality-based model.The most used mathematical model in the evolution-ary game theory is without doubt the replicator equa- a r X i v : . [ q - b i o . P E ] F e b tion [14]—a highly simplified model of selection and repli-cation. There are folk theorems connecting stable fixedpoints of the replicator dynamics with static solution con-cepts of noncooperative game theory played by rationalplayers [15]. It enables a biologist to predict the dy-namical outcome by finding the Nash equilibrium (NE)of the corresponding one shot game. This is the casefor other monotone dynamics in evolutionary game the-ory as well. In literature there are many different evo-lutionary dynamical models: some of them can be ob-tained by varying revision protocol [16, 17], e.g., replica-tor dynamics [14], best response dynamics [18], Brown–von Neumann–Nash dynamics [19], Smith dynamics [20]and logit dynamics [21]; whereas some can be seen asvariants of incentive dynamics [22], e.g., replicator dy-namics [14], best response dynamics [18], logit dynam-ics [21] and projection dynamics [23]. All the aforemen-tioned dynamics have the common property of convergingtowards NE [24–27]. It has also been shown in the lit-erature that evolutionarily stable state can be related tolocally asymptotically stable fixed point for both replica-tor map [27] and replicator flow [15].Although it is not justified to argue that equilibriumplay should explain the outcomes of every games in ev-ery possible scenario, this idea of equilibrium being con-nected with convergence to the fixed point is so deep-rooted that it is commonly held that any non-equilibriumbehaviour is necessarily transient, and only equilibriumbehaviour is persistent and robust to be ultimately re-alized. As a consequence, it appears very intriguingwhen the possibilities of non-convergence to NE solu-tions show up as robust asymptotic solutions—and not astransients—in evolutionary models. There is no knownway (such as folk theorems) to predict such non-NE out-comes of dynamics from the knowledge of one-shot non-cooperative game theory.Few examples of such non-fixed point robust asymp-totic dynamical solutions in evolutionary game theory areas follows: Replicator dynamics, Brown–von Neumann–Nash dynamics, and Smith dynamics can show limitcycle as possible outcome [17]. Chaotic behaviour hasbeen found in replicator dynamics [28, 29] and Brown–von Neumann–Nash dynamics [30]. Logit dynamics, thenoisy version of best response dynamics [31], can alsolead to periodic solutions. Replicator map can show bothchaotic and periodic outcomes along with convergence tofixed point [27, 32–34]. While one could say that the ap-pearance of non-fixed point solutions is shortcoming ofthe corresponding model and there should exist an evo-lutionary model that unfailingly ensures convergence toNE, there is no strong logic to presume that actual be-haviour would be in line with such a model.In view of the above, in this paper, we take first step ofproposing the hitherto ill-understood question: Is thereany game theoretic argument possible that we can con-nect with the periodic outcomes of the dynamical modelsin population games and what could be the physical im-plication of the periodic outcome? Given the plethora of models of evolutionary dynamics, we decide to workwith the replicator map adapted for the two-player-two-strategy games [27] because of the following concrete rea-sons: (i) this evolutionary game dynamic models Dar-winian selection, (ii) relationship of its fixed points withNE and ESS is well established through standard folktheorems and related theorems, and (iii) it is able to showperiodic solutions (that bifurcate into chaotic solution).Since our main aim is to find game theoretic connectionfor periodic orbits, we restrict our analyses only to themixed strategy domain. This is so because any pure stateis a fixed point of the replicator map.We end up convincingly showing that it is very muchpossible to interpret locally asymptotically stable peri-odic orbits as representing evolutionarily stable scenarioin the setting of a repeated game. All one needs to do isto appropriately generalize the concepts of ESS and NEsuch that the individuals in the population game appearto be optimizing their way to ‘survival of the fittest’. Wefind that the effective ‘fitness’ in the game must be (re-)interpreted as fitness multiplied by the probability thattwo arbitrarily chosen members of the population belongto two different phenotypes.However before embarking on the technical discussionof the repeated games and the corresponding game the-oretical concepts of equilibrium, without further ado werevisit the dynamics of the replicator map that we haveused as the paradigmatic model in this paper. II. PERIODIC ORBITS IN REPLICATOR MAP
Consider the population game where there is an un-derlying normal form game with, say, N pure strategiesand an N × N payoff matrix U . Any mixed strategy is anelement of the corresponding simplex Σ N . Subsequently,one could define a population game on Σ n between n (pheno-)types with fractions x , x , · · · , x n such that ev-ery type can be mapped to a particular strategy in Σ N ;say, i th type in the population game is realized as strat-egy p i ∈ Σ N . An element, π ij , of n × n payoff matrix Π (say) of the population game is p i · U p j and the fit-ness of i th type is given by ( Π x ) i = (cid:80) π ij x j . Connectionbetween the two aforementioned simplices is that the dy-namics of state x ∈ Σ n induces a dynamics for averagepopulation strategy ¯ p = (cid:80) ni =1 p i x i on Σ N .In classical game theory, rational players optimizetheir payoff following the concept of NE strategy profilewherein the strategies of the players are best responsesto each other. Hence, the underlying game with payoffmatrix U has a mixed NE ( ˆ p ) which is mathematicallydefined as ˆ p T U ˆ p = p T U ˆ p , ∀ p ∈ Σ N . (1)In evolutionary game theory the concept of ESS playsthe central role as it ensures that a population adopt-ing this strategy can’t be invaded by any infinitesimalfraction of mutants adopting an alternative strategy.Mathematically, the strategy ˆ p is ESS of the underly-ing game if there exists a neighbourhood B ˆ p of ˆ p suchthat ∀ p ∈ B ˆ p \ ˆ p the following inequality holds: ˆ p T U p > p T U p . (2)It is straightforward to show that ESS implies NE.Along the line of the above discussion the idea of NEand ESS can be extended to the population game withpayoff matrix Π . However now the (pheno-)types are thepossible strategies. Hence it is defined in terms of state ofthe population consisting those phenotypes. Specifically,for symmetric population game with payoff matrix Π , if ˆx is the mixed NE (state) then ˆ x T Π ˆ x = x T Π ˆ x , ∀ x ∈ Σ n . (3)The state ˆ x is ESS (evolutionarily stable state) of thepopulation game if there exists a neighbourhood B ˆ x of ˆ x such that ∀ x ∈ B ˆ x \ ˆ x the following inequality holds, ˆ x T Π x > x T Π x . (4)Again, ESS implies mixed NE.The replicator map maps frequency of a phenotypein k th generation to its frequency in ( k + 1) th non-overlapping generation. Specifically, the map is expressedin the following form [27, 32–34]: x ( k +1) i = f i ( x ) = x ( k ) i + x ( k ) i (cid:104) ( Π ˆ x ( k ) ) i − ˆ x ( k ) T Π ˆ x ( k ) (cid:105) . (5)Here x ( k ) i is the frequency of i th phenotype in the popu-lation of k th generation and x ( k ) = ( x ( k )1 , x ( k )2 , · · · , x ( k ) n ) . Π is the payoff matrix of the corresponding one shot gameassociated with the dynamics. In general, the map cangive rise to unphysical solutions, i.e., x i / ∈ [0 , for some i ; therefore, not all n × n payoff matrices are physicallyallowed.For two-player-two-strategy game i ∈ { , } and Π isa × matrix. On rewriting x ( k )1 = x ( k ) and, hence, x ( k )2 = 1 − x ( k ) , Eq. (5) becomes, x ( k +1) = f (cid:0) x ( k ) (cid:1) = x ( k ) + x ( k ) (1 − x ( k ) ) (cid:104) ( Π ˆ x ( k ) ) − ( Π ˆ x ( k ) ) (cid:105) . (6)The fixed point of this replicator map given by Eq. (6)is connected with game theoretic outcomes through folktheorems. The NE state are the fixed point of this mapwhereas any locally asymptotically stable interior fixedpoint is known to be ESS [27]. By the concept of strongstability [10] it can also be shown that if the average strat-egy of the population in the underlying game asymptot-ically converges to the strategy adopted by a phenotypein the undelying game then that strategy must be ESSof the underlying game. Hence, in a way one can con-nect the dynamics of this map with the game theoreticoutcome of the underlying game. For game theoretic studies, it suffices to work with thefollowing form of × payoff matrix: Π = (cid:20) ST (cid:21) ; S, T ∈ R , (7)as it is capable of representing all the twelve ordinalclasses of symmetric games found in the standard lit-erature. The dynamics of replicator map for this form ofpayoff matrix has been studied in literature and specificconditions for which this map gives strict physical solu-tions are known [27]. The reason behind choosing thisgame to be symmetric is that we assume, as is normin the evolutionary game theory, that (i) the players’strategy sets are identical, (ii) the payoff received by aplayer playing against an opponent doesn’t change withthe identities of the players, and (iii) players don’t maketheir choices of strategy based on features of the oppo-nent.The simple map given, by Eq. (6), has rich dynam-ical properties showing wide range of asymptotic be-haviours, e.g., fixed points, periodic orbits, and chaotictrajectories [27]: The map, in general, has two bound-ary fixed points, x = 0 and , and one interior fixedpoint, x = S/ ( S + T − . The interior fixed point ofthis map is stable when S ( T − / ( S + T − < andundergoes flip bifurcation at S ( T − / ( S + T −
1) = 2 ,giving rise to a two-period orbit. Subsequently, as onedrives S ( T − / ( S + T − further away from , a pe-riod doubling cascade—giving rise to higher period orbitsand ultimately chaos—is observed.Now, let’s assume that a sequence of states, { ˆ x ( k ) :ˆ x ( k ) ∈ (0 , , k = 1 , , · · · , m } , with ˆ x ( i ) (cid:54) = ˆ x ( j ) ∀ i (cid:54) = j ,represents a periodic orbit with prime-period m . Thenfor any k ∈ { , , · · · , m } , by construction, we have ˆ x ( k +1) = ˆ x ( k ) +ˆ x ( k ) (1 − ˆ x ( k ) ) (cid:104) ( Π ˆ x ( k ) ) − ( Π ˆ x ( k ) ) (cid:105) , (8)where naturally, ˆ x ( m +1) = ˆ x (1) . Summing all the m ex-pressions implied by Eq. (8), we get m (cid:88) k =1 ˆ x ( k ) (1 − ˆ x ( k ) ) (cid:104) ( Π ˆ x ( k ) ) − ( Π ˆ x ( k ) ) (cid:105) = 0 . (9)It is interesting to note that x ( k ) (1 − ˆ x ( k ) ) is the prob-ability that two arbitrarily chosen members of the popu-lation belong to two different phenotypes. In populationgenetics of the simple case of one-locus-two-allele, un-der Hardy–Weinberg assumptions, the analogous expres-sion is called heterozygosity that measures the propor-tion of heterozygous individuals in the population [35].For future convenience, we denote x ( k ) (1 − ˆ x ( k ) ) by H ˆ x ( k ) and call it heterogeneity as it is a measure of howheterogeneous-strategied the population is. III. EXTENSION OF NE AND ESS
Our ultimate aim concerns with relating the dynam-ical outcomes with the corresponding population gametheoretic outcomes, we must first discuss how we can ex-tend the existing framework of equilibrium states for aset of states (containing m elements) that may be peri-odic orbit. Analogously, we want to study the scenarioof m -period games (with payoff matrix U ) that is an ex-tensive form of game where a base game is played m times. Unless otherwise specified, for the sake of simplic-ity, we discuss the case of two-player-two-strategy game(i.e., N = 2 ) throughout the paper (see, however, Ap-pendix A). The caveat we must keep in mind is that whilea standard m -period game is usually all about maximis-ing (given the belief about the opponent) the total payoffaccumulated over all the stages, the m -period games weare interested are in the context of the replicator mapwhere the dynamics at each stage (generation) is drivenby the payoffs of immediately preceding stage. It alreadygives us hint that players playing m -period game in linewith the replicator map need not be optimizing the ac-cumulated payoff. So what do they optimize? A. Heterogeneity Equilibrium
In classical game theory, rational players optimizetheir payoff following the concept of NE strategy profilewherein the strategies of the players are best responses toeach other. However, rationality is a redundant conceptin evolutionary dynamics that is governed by, say, Dar-winian selection. While a posteriori justification is fur-nished later in the form of a successful self-consistency,we propose to study a scenario where rather than the ex-pected payoff, heterogeneity weighted expected payoff (orsimply heterogeneity payoff, for the sake of brevity) is be-ing optimized. By heterogeneity payoff we merely meanthat payoff is weighted (multiplied) by the heterogene-ity defined using opponent’s mixed strategy, p ( k ) (say),i.e., H p ( k ) ≡ p ( k ) (1 − p ( k ) ) . Here p ( k ) = ( p ( k ) , − p ( k ) ) is the completely mixed strategy of the opponent. Notethat now we are using strategy, and not state, to de-fine heterogeneity. We reiterate that we work only inthe completely mixed strategy domain as our main in-tention is to justify the emergence of the periodic orbits(of prime period more than one) which must be totallymixed states as all the pure states are fixed points of thereplicator map.The condition of mixed NE as expressed by Eq. (1) canbe trivially rewritten as, H ˆ p (cid:2) ˆ p T U ˆ p (cid:3) = H ˆ p (cid:2) p T U ˆ p (cid:3) ; 0 < H ˆ p ≤ . . (10)It is clear that Eq. (1) considers expected payoff as incen-tive while Eq. (10) considers heterogeneity payoff as in-centive. Obviously, they both have mixed NE as a uniquesolution should it exist, implying that a mixed NE pro- vides indifference in heterogeneity payoff for unilateraldeviation in case of 1-period game.We contextually propose that there exists an equilib-rium among the mixed strategies for m -period games,that we name as heterogeneity weighted Nash equilib-rium or heterogeneity equilibrium (HE) for brevity. TheHE( m ) strategy profile (where m denotes that m -periodgame is under consideration) consists of pairs of strate-gies that are best responses to each other in the followingsense: Assuming that a player plays the strategy of anystage of the m -period in all the m stages of play withits opponent, the player cannot get more accumulatedheterogeneity payoff by deviating unilaterally. Mathe-matically, we have the following: Definition:
The sequence of strategies { ˆ p ( k ) : ˆ p ( k ) ∈ (0 , , k = 1 , , · · · , m } over m -period game is an HE( m )if ∀ j ∈ { , , · · · , m } , m (cid:88) k =1 H ˆ p ( k ) (cid:104) ˆ p ( j ) T U ˆ p ( k ) (cid:105) = m (cid:88) k =1 H ˆ p ( k ) (cid:104) p T U ˆ p ( k ) (cid:105) ; ∀ p ∈ (0 , . (11)We have already observed that mixed NE profile is theunique HE profile for 1-period game. A closer look re-veals that any single mixed NE profile played over allthe stages of the m -period game induces an HE( m ): ofcourse, if Eq. (11) holds then ddp (cid:80) mk =1 H ˆ p ( k ) (cid:104) p T U ˆ p ( k ) (cid:105) =0 , implying m (cid:88) k =1 H ˆ p ( k ) (cid:104) ( U ˆ p ( k ) ) − ( U ˆ p ( k ) ) (cid:105) = 0 . (12)It is now easily comprehensible that a set of strategiesthat is essentially a single mixed NE repeated m times isan HE( m ). To see it more transparently, we recall thatthe term in the third bracket in Eq. (12) vanishes indi-vidually if ˆ p ( k ) is mixed NE strategy. More interesting,however, is the non-trivial scenario when Eq. (12) is ful-filled by a set of strategies that is not a single mixed NErepeated m times. However, whether such solutions existdepends on the exact structure of the payoff matrix. Wenote that, if n is a multiplicative factor of m , then theset of strategies forming HE( n ) repeated m/n times forman HE( m ). B. Heterogeneity Orbit
The concept of HE( m ) in repeated games (with pay-off matrix U ) can be adapted to define an equilibriumusing a set of states for a population consisting of twotypes (with payoff matrix Π ). Hence, in what follows, wepropose heterogeneity weighted Nash equilibrium orbitor heterogeneity orbit (HO) for brevity. Definition:
The sequence of states { ˆ x ( k ) : ˆ x ( k ) ∈ (0 , , k = 1 , , · · · , m } where ˆ x ( i ) (cid:54) = ˆ x ( j ) for i (cid:54) = j , is an HO( m ) if ∀ j ∈ { , , · · · , m } , m (cid:88) k =1 H ˆ x ( k ) (cid:104) ˆ x ( j ) T Π ˆ x ( k ) (cid:105) = m (cid:88) k =1 H ˆ x ( k ) (cid:104) x T Π ˆ x ( k ) (cid:105) ; ∀ x ∈ (0 , . (13)We have already observed that mixed NE profileis the unique HO( ). If Eq. (13) holds then ddx (cid:80) mk =1 H ˆ x ( k ) (cid:104) x T Π ˆ x ( k ) (cid:105) = 0 , implying m (cid:88) k =1 H ˆ x ( k ) (cid:104) ( Π ˆ x ( k ) ) − ( Π ˆ x ( k ) ) (cid:105) = 0 . (14)We note that a set of states formed by a single mixed NErepeated m times becomes a trivial HO( m ) if we removethe restriction imposed by ˆ x ( i ) (cid:54) = ˆ x ( j ) for i (cid:54) = j . However,whether non-trivial solutions to Eq. (14) exist depends onthe exact structure of the payoff matrix Π . Furthermore,on comparing Eq. (14) with Eq. (9), one notes that m -period orbit of replicator map must be HO( m ) and viceversa . C. Heterogeneity Stable Orbit
Having generalized mixed NE to HO, in this subsectionwe ask what the generalization of evolutionary stabilityis and how that generalization of ESS will relate to HO.In evolutionary game theory the concept of ESS plays thecentral role as it ensures that a population in this statecan’t be invaded by an infinitesimal fraction of mutanthaving some alternative state. As the stable fixed pointsof the replicator map are ESS, so it can be claimed thatnatural selection alone is sufficient to stop invasion byany alternative state once the population is fixed at ESS.Now, given the idea of heterogeneity payoff, is there anystate profile for m -period orbit, that is resilient againstan infinitesimal mutant fraction? ESS ˆ x is the state ofthe population that is resilient against an infinitesimalmutant fraction— (cid:15) fraction with any state x ( m ) (cid:54) = ˆ x ; informal mathematical notations, ˆ x T Π (cid:104) (1 − (cid:15) )ˆ x + (cid:15) x ( m ) (cid:105) > x T ( m ) Π (cid:104) (1 − (cid:15) )ˆ x + (cid:15) x ( m ) (cid:105) (15)for (cid:15) (cid:28) . Now, let’s define x ≡ (1 − (cid:15) )ˆ x + (cid:15) x ( m ) . One canconstruct neighbourhood B ˆ x of ˆ x such that x ∈ B ˆ x \{ ˆ x } .Multiplying both sides of Inequality (15) by (cid:15) and adding (cid:0) − (cid:15) (cid:1) (cid:2) ˆ x T Π x (cid:3) to both the sides, we arrive the conditionof ESS given by Inequality (4).We can rewrite Inequality (4) as, H x (cid:2) ˆ x T Π x (cid:3) > H x (cid:2) x T Π x (cid:3) , (16)whenever x in the neighbourhood B ˆ x , i.e., x ∈ B ˆ x \{ ˆ x } .Inequality (16) is another equivalent definition of ESS.In line with the concept of HO, we propose heterogeneityweighted evolutionarily stable orbit—for brevity, hetero-geneity stable orbit, HSO( m )—as an extension of ESS. Definition:
HSO( m ) of a map— x ( k +1) i = g ( x ( k ) i ) —is a sequence of states, { ˆ x ( k ) : ˆ x ( k ) ∈ (0 , k =1 , , · · · , m ; ˆ x ( i ) (cid:54) = ˆ x ( j ) ∀ i (cid:54) = j } such that m (cid:88) k =1 H x ( k ) ˆ x (1) T Π x ( k ) > m (cid:88) k =1 H x ( k ) x (1) T Π x ( k ) , (17) for any orbit { x ( k ) : x ( k ) ∈ (0 , k = 1 , , · · · , m } of the map starting in some infinitesimal neighbourhood B ˆ x (1) \{ ˆ x (1) } of ˆ x (1) . It is easy to observe that mixed ESS is the uniqueHSO( ). In passing, we remark that the concept of in-centive stable state equilibrium [22] to describe incentivedynamics is simply HSO( ). We also observe that onecould in principle replace states by strategies and useappropriate payoff matrix ( U , say) in Inequality (17) toanalogously define heterogeneity stable strategy (HSS).Since we know that ESS serves as a refinement of NE,it would be rather satisfying if HSO( m ) serves as a re-finement of HO( m ). For m = 1 , the sought refinementis mere tautology because the concepts of HSO and HOboil down to the concepts of ESS and NE respectively.For the case of any general m , owing to Inequality (17)there exists a neighbourhood N ˆ x (1) of ˆ x (1) in (0 , suchthat ∀ x (1) ∈ N ˆ x (1) \{ ˆ x (1) } (where B ˆ x (1) = N ˆ x (1) × (0 , ),the following holds: (cid:0) x (1) − ˆ x (1) (cid:1) m (cid:88) k =1 H x ( k ) (cid:104)(cid:16) Π x ( k ) (cid:17) − (cid:16) Π x ( k ) (cid:17) (cid:105) < , (18) = ⇒ lim x (1) − ˆ x (1) → m (cid:88) k =1 H x ( k ) (cid:104)(cid:16) Π x ( k ) (cid:17) − (cid:16) Π x ( k ) (cid:17) (cid:105) = 0 , (19) = ⇒ m (cid:88) k =1 H ˆ x ( k ) (cid:104) ( Π ˆ x ( k ) ) − ( Π ˆ x ( k ) ) (cid:105) = 0 . (20)Comparing Eq (20) with Eq. (14) we conclude thatHSO( m ) implies HO( m ).Henceforth, unless otherwise specified, all further dis-cussions involve only HSO( m ) of replicator map (cf. Ap-pendix B). IV. HSO AND DYNAMICAL STABILITY
It is clear that HSO( ) is nothing but evolutionarilystable state and it has been shown in literature that lo-cally asymptotically stable fixed point of the replicatormap is HSO( ) [27]. We emphasize that HSO( ) is locallyasymptotically stable fixed point even for the replicatorequation [15]. It, thus, is very natural to suspect thatthere must be a connection between stable periodic orbitof period m and HSO( m ). In fact, the following theoremtells us that so is the case: Theorem:
If the sequence of states { ˆ x ( k ) : ˆ x ( k ) ∈ intΣ ; k = 1 , , · · · , m } , where ˆ x ( i ) (cid:54) = ˆ x ( j ) for i (cid:54) = j ,is a locally asymptotically stable m -period orbit of the replicator map for two-player-two-strategy game, then itmust be HSO( m ). Proof:
Let sequence of states { ˆ x (1) , ˆ x (2) , · · · , ˆ x ( m ) } be a locally asymptotically stable periodic orbit of thereplicator map given in Eq. (6). Then, by the definition ofperiod orbit, each of the state from the set must be a fixedpoint of the map f m ( x ) . We assume that the states arearranged in temporal order. Since we have assumed localasymptotic stability, by construction, ∃ a neighbourhood N ˆ x (1) of ˆ x (1) in (0 , such that ∀ x (1) ∈ N ˆ x (1) \{ ˆ x (1) } wehave, || f m ( x (1) ) − ˆ x (1) |||| x (1) − ˆ x (1) || < . (21)Recalling f ( x ( j ) ) = x ( j +1) and using the explicit form ofthe replicator map, the above inequality can be rewrittenas, || x (1) − ˆ x (1) + (cid:80) mk =1 H x ( k ) (cid:2) ( Π x ( k ) ) − ( Π x ( k ) ) (cid:3) |||| x (1) − ˆ x (1) || < . (22)Here, || · · · || stands for an appropriate norm which we canconveniently take as the Euclidean norm. Inequality (22)implies that x (1) − ˆ x (1) must have a sign that is oppositeto that of (cid:80) mk =1 H x ( k ) (cid:2) ( Π x ( k ) ) − ( Π x ( k ) ) (cid:3) . Therefore, ∀ x (1) ∈ B ˆ x (1) \{ ˆ x (1) } where B ˆ x (1) = N ˆ x (1) × (0 , , (cid:0) x (1) − ˆ x (1) (cid:1) m (cid:88) k =1 H x ( k ) (cid:104) ( Π x ( k ) ) − ( Π x ( k ) ) (cid:105) < ⇒ m (cid:88) k =1 H x ( k ) (cid:2) ˆ x (1) T Π x ( k ) (cid:3) > m (cid:88) k =1 H x ( k ) (cid:2) x (1) T Π x ( k ) (cid:3) . (23)Comparing this expression with Eq. (17) it is clear thatit is nothing but the condition for HSO( m ). Hence, lo-cally asymptotically stable periodic orbits of period m are HSO( m ). Q.E.D.
The converse of this theorem is not true, i.e., anHSO( m ) need not always be a locally asymptoti-cally stable periodic orbit of period m . Inequal-ity (23) is only a necessary condition for the ful-filment of Inequality (22); one additionally requires (1 / (cid:12)(cid:12)(cid:80) mk =1 H x ( k ) (cid:2) ( Π x ( k ) ) − ( Π x ( k ) ) (cid:3)(cid:12)(cid:12) < | x (1) − ˆ x (1) | for the converse to hold true. Therefore, a HSO( m ), thatalso happens to be an orbit, is a locally asymptoticallystable m -periodic orbit of the replicator map if and onlyif < m (cid:88) k =1 H x ( k ) (cid:104) ˆ x (1) T Π x ( k ) − x (1) T Π x ( k ) (cid:105) < | x (1) − ˆ x (1) | . (24)The importance of this theorem is akin to that of thefolk theorems: One can deduce on the asymptotic pe-riodic outcome of the replication-selection dynamics bystudying the payoff matrix of the game keeping in mindthe concept of HSO. Thus, we believe that this represen-tative theorem has far reaching implications on the study of evolutionary dynamics. This theorem enables one tounderstand what the game-theoretic interpretation of arobust stable periodic orbit is. Recall that periodic orbitsare a common occurrence in many dynamical systems ofevolutionary game theory. V. STRONGLY STABLE STRATEGY SET
Though our study has associated periodic orbit withevolutionarily stability, we lack the corresponding insightin the underlying normal form game where a particu-lar strategy corresponds to a particular (pheno-)type inthe population game. We already know that unlike thecontinuous time dynamics, ESS or HSO( ) need not bethe stable fixed point of replicator map. As an exam-ple, Leader game can lead to periodic or chaotic outcomeeven though it possesses ESS [27]. It hints that the aver-age population strategy in normal form game also don’tconverge to any particular strategy that happen to beevolutionary stable strategy of the corresponding normalform game. It, thus, is important to understand the dy-namics from the point of view of underlying normal formgame.Our studied population game corresponds to twotypes, i.e., n = 2 . Let’s consider that the type with fre-quency x is using strategy p and the other type with fre-quency − x is using strategy p where both p , p ∈ Σ N .The average population strategy at k th generation isgiven as, ¯ p ( k ) = x ( k ) p + (1 − x ( k ) ) p ∈ Σ N . Hence wecan rewrite the condition such that the sequence of states { ˆ x ( k ) : ˆ x ( k ) ∈ (0 , k = 1 , , · · · , m } is a m -periodic or-bit of replicator map (refer Eq. (9)) in terms of undelyingnormal form game in the following form, m (cid:88) k =1 H ˆ x ( k ) (cid:104) p . U (cid:98) ¯ p ( k ) − p . U (cid:98) ¯ p ( k ) (cid:105) = 0 , (25)where (cid:98) ¯ p ( k ) = ˆ x ( k ) p + (1 − ˆ x ( k ) ) p . Hence, the aver-age population strategy traverses through the sequenceof strategies { (cid:98) ¯ p ( k ) : (cid:98) ¯ p ( k ) ∈ (0 , k = 1 , , · · · , m } peri-odically. Definition:
A sequence of strategies { (cid:98) ¯ p ( k ) : (cid:98) ¯ p ( k ) = (cid:80) i =1 ˆ x ( k ) i p i ; ˆ x ( k ) ∈ (0 , ∀ k = 1 , , · · · , m ; p i ∈ Σ N } where (cid:98) ¯ p ( i ) (cid:54) = (cid:98) ¯ p ( j ) ∀ i (cid:54) = j is strongly stable strategy set (SSSS( m )) if anyinitial average population strategy ¯ p ( k ) , that is sufficientlyclose to SSSS( m ), converges to SSSS( m ). Theorem: If { (cid:98) ¯ p ( k ) : k = 1 , , · · · , m } is SSSS( m )then { ˆ x ( k ) : k = 1 , , · · · , m } is HSO( m ). Proof:
By definition, (cid:98) ¯ p ( k ) = ˆ x ( k ) p + (cid:0) − ˆ x ( k ) (cid:1) p .Any infinitesimal perturbation around an element ofSSSS( m ) can be represented as (cid:98) ¯ p ( k ) + (cid:15) ( p − p ) = (cid:0) ˆ x ( k ) + (cid:15) (cid:1) p + (cid:0) − ˆ x ( k ) − (cid:15) (cid:1) p where (cid:15) → . Thus,we note that if any initial average population strategy is − . − . . . . (cid:15) × − × − (a) F , F − . − . . . . (cid:15) × − − . − . − . − . − . − . . F × − (b) FIG. 1. Locally asymptotically stable -period orbit, (ˆ x (1) , ˆ x (2) ) ≈ (0 . , . , of Battle of Sex game is HSO( ). Insubplot (a) blue solid curve and black dashed curve respec-tively represent F and F for (0 . , . plotted against (cid:15) .Subplot (b) depicts F vs. (cid:15) for (ˆ x (1) , ˆ x (2) ) = (ˆ x, ˆ x ) where ˆ x is mixed NE ( ˆ x ≈ . ). − . − . . . . (cid:15) × − . . . . . . . . × − (a) F , F − . − . . . . (cid:15) × − − . − . − . − . − . − . − . − . . F × − (b) FIG. 2. Locally asymptotically stable -period orbit, (ˆ x (1) , ˆ x (2) ) ≈ (0 . , . , of Leader game ( S = 5 . , T = 6 . )is HSO( ). In subplot (a) blue solid curve and black dashedcurve respectively represent F and F for (0 . , . plottedagainst (cid:15) . Subplot (b) depicts F vs. (cid:15) for (ˆ x (1) , ˆ x (2) ) = (ˆ x, ˆ x ) where ˆ x is mixed NE ( ˆ x ≈ . ). sufficiently close to an element of SSSS( m ), then in thepopulation dynamics the initial state is sufficiently closeto the corresponding element of the sequence of states { ˆ x ( k ) : k = 1 , , · · · , m } . Since by the definition the ini-tial average population strategy converges to SSSS( m ),if we start sufficiently close to any state of the sequence { ˆ x ( k ) : k = 1 , , · · · , m } , the population state must con-verge to this set. Hence, the set of states is locally asymp-totically stable m -periodic orbit that must be HSO( m ) inline with the theorem proved in Section IV. The converseof the theorem does not always hold good as an HSO( m )need not be locally asymptotically stable m -periodic or-bit. VI. ILLUSTRATIVE EXAMPLES
In order to make the concepts introduced in this pa-per more accessible, we now take the examples of threegames, viz., Prisoner’s Dilemma, Battle of Sex, and − . − . . . . (cid:15) × − . . . . . . × − (a) F , F − . − . . . . (cid:15) × − . . . . . . × − (b) F , F − . − . . . . (cid:15) × − − − − − − − × − (c) F , F − . − . . . . (cid:15) × − − − − − − − F × − (d) FIG. 3. Unstable -period orbits of Leader game may or maynot be HSO(2). Blue solid curve and black dashed curve re-spectively represent F and F , and red solid curve standfor | (cid:15) | (see Inequality (24)). Subplots (a), (b), and (c)are respectively for -period orbits (ˆ x (1) , ˆ x (2) ) ≈ (0 . , . , (0 . , . and (0 . , . of Leader game ( S = 7 . , T =8 . ). Subplot (d) depicts F vs. (cid:15) for (ˆ x (1) , ˆ x (2) ) = (ˆ x, ˆ x ) where ˆ x is mixed NE ( ˆ x ≈ . ). Leader game, where we confine ourselves to the caseof m = 2 . Out of the possible physical solutions—fixed points and prime -periodic orbits—of equation f ( x ) = x , we consider only the prime -period solutionsas they may be connected to HO( ) and HSO( ).In the subsections to follow, we are specifically goingto elaborate the following points in the context of theaforementioned games:• In case the dynamical outcome of a game is a -period orbit, (ˆ x (1) , ˆ x (2) ) , then that periodic orbitmust be HO( ) defined by Eq. (14). By definitionof HO( ) given by Eq. (13) any unilateral deviationby a player does not fetch more accumulated het-erogeneity payoff when played against the periodicorbit.• We know that a locally asymptotically stable -period orbit must be HSO( ) as defined by Eq. (17).In order to check this, it is convenient to define F j ≡ (cid:88) k =1 H x ( k ) (cid:104) ˆ x ( j ) T Π x ( k ) − x ( j ) T Π x ( k ) (cid:105) , j ∈ { , } ; (26)and observe that { ˆ x ( k ) : k = 1 , } is HSO( m ) if F j > for any x ( j ) = ˆ x ( j ) + (cid:15) where | (cid:15) | < ¯ (cid:15) forsome positive ¯ (cid:15) ≤ .• Furthermore, if an unstable -periodic orbits isHSO( ), it must violate Inequality (24), i.e., F j < | (cid:15) | does not hold true.Without any loss of generality, the above points can easilybe adapted to find HE and HSS of any two-player-two-strategy game (with payoff matrix U ). A. Prisoner’s Dilemma
We consider the form of payoff matrix given by Eq. (7)where S = − . and T = 2 . stands for Prisoner’sDilemma game. The discrete replicator dynamics of thisgame doesn’t have any physical -periodic orbit [27]. Asperiodic orbit must be HO( ), we remark that this gamedoesn’t have any HO( ). By definition, HSO( ) mustbe HO( ). Hence, this game doesn’t have any HSO( )either. B. Battle of Sex S = 5 . and T = 4 . (refer Eq. (7)) makes forthe payoff matrix of Battle of Sex game. Dynamics ofthis game has only one physical -period orbit given by (ˆ x (1) , ˆ x (2) ) ≈ (0 . , . . Hence, (0 . , . is HO( ).As implied by the definition of HO( ), even mixed NE ˆ x ( ˆ x ≈ . ) must fetch same accumulated heterogeneitypayoff as any other arbitrary state, when played againstthe periodic orbit. It is indeed the case: We compare theaccumulated heterogeneity payoffs for each of the threestates ˆ x , ˆ x (1) , and ˆ x (2) when played against the periodicorbit. They come out to be approximately . .This periodic orbit further happens to be locallyasymptotically stable and hence it must be HSO( ). Thisis indeed the case as depicted in Fig. 1a which shows that F > and F > . This finding is non-trivial in thesense that even the mixed NE repeated twice doesn’t sat-isfy the condition of HSO( ). This is showcased in Fig. 1bwhere F < . This fact is in accordance with the factthat the NE is an unstable fixed point. C. Leader Game
1. Case I S = 5 . and T = 6 . gets us the payoff matrix of theLeader game. This game has only one physical -periodorbit given by (ˆ x (1) , ˆ x (2) ) ≈ (0 . , . which is locallyasymptotically stable. Hence, (0 . , . is an HO( ).As before, the accumulated heterogeneity payoff of eachof the three states ˆ x (ˆ x ≈ . , ˆ x (1) , and ˆ x (2) whenplayed against the periodic orbit is same ( ≈ . ).Being locally asymptotically stable, the -period or-bit must be HSO( ). We confirm this through Fig. 2awhere we observe that F , F > . Additionally, Fig. 2bstresses the fact that the mixed NE repeated over twogenerations doesn’t satisfy the condition of HSO( ).
2. Case II
Another payoff matrix of Leader game may be re-alized by setting S = 7 . and T = 8 . . This gamehas interestingly three physical -period orbits, givenby (ˆ x (1) , ˆ x (2) ) ≈ (0 . , . . , . . , . .Hence, (0 . , . . , . and (0 . , . areHO( ). For these three HO( ), the accumulated het-erogeneity payoffs are approximately . , . , and . respectively, irrespective of what state plays against thecorresponding periodic orbits; as before we have checkedthis fact using the three states ˆ x ( ˆ x ≈ . ; mixed NE), ˆ x (1) , and ˆ x (2) .However, all of the three -period orbits are unsta-ble. Some or all of them may be HSO( ) that requires F , F > . In Fig. 3a,b we note that (0 . , . and (0 . , . are HSO( ), and they also violate Inequal-ity (24) as expected. The remaining periodic orbit is notan HSO( ) as seen in Fig. 3c where F , F < . Yetagain, we observe in Fig. 3d that the mixed NE repeatedtwice doesn’t satisfy the condition of HSO( ). VII. DISCUSSION AND CONCLUSION
We remind ourselves that evolutionary game dynamicshave been successfully used to model real life problemsin diverse fields, like, biology, economics, sociology, be-havioural science, etc. Replicator dynamics is used as amodel in problems involving social dilemma [36], molec-ular and cell biology [37], economy [38]. The field ofgrammar learning has been studied using replicator mu-tator as a dynamical model [39, 40]. Logit dynamics onthe other hand, have mainly been applied in economicalmodels [41, 42] along with social and behavioural sci-ence [31, 43]. Brown–von Neumann–Nash dynamics hasapplications in economic scenarios [30] and evolution ofheterogenous forecasting [44]. Projection dynamics wasproposed as a model in transport system [23] and laterapplied to complimentary formalism [45]. Best responsedynamics have found applications in complex social net-works [46], internet and network economics [47]. Clearlythe vast applications of the aforementioned evolutionarydynamics in different domains are quite appealing andmotivate one to study their dynamics in depth while ap-preciating their implications.It so happens that all the aforementioned dynamicsshow periodic and chaotic behaviours that are not merelythe transient phases of the dynamics. We do not believethat it is justified to attribute these robust non-triviallimit sets of phase trajectories to the inapplicability ofthe models just because the behaviours are not in directconformity with the game theoretic concept of NE. Theemergence of chaos and periodic orbits in evolutionarydynamics for two-player-two-strategy games simply in-dicates that the assumption of rationality may be un-realistic even in the simplest setting. There are lackof compelling reasons behind how agents might havelearned how to play NE [48]. In a learning process, ina population of players meeting randomly and repeat-edly, the players are endowed with some behavioural rulesof selecting strategies based on their experiences. Sato et al. [29] have illustrated that learning through a repli-cator model even in an elementary setting of rock-paper-scissors games is practically impossible because the re-sulting dynamics becomes chaotic.In view of the above, in this paper, we argue that oneneeds to generalize the game theoretic concepts appropri-ately in order to appreciate any non-fixed point behaviourof evolutionary game dynamics. To this end, we take ananalytically tractable version of replicator dynamic—thereplicator map for two-player-two-strategy game—thatis known to not only possess periodic and chaotic or-bits, but also satisfy the folk theorems connecting fixedpoint solutions to NE. We, then, introduce the conceptsof new equilibria—termed HE and HSS—in the contextof m -period games, and define them in terms of the statesof the population to introduce the concepts of HO andHSO respectively. We can summarize the main math-ematical results in the following points: (i) HSO mustbe HO (and similarly, HSS must be HE), (ii) a periodicorbit of replicator map must be HO, and (iii) a locallyasymptotically stable periodic orbit is HSO. Thus, one isenabled to predict dynamical outcome just by studyingthe payoff matrix of corresponding one shot game evenwhen the dynamic outcome is a period solution—this isa clear development over the standard folk theorems forthe replicator dynamics.What, however, is even more intriguing is that thereplicator dynamics, or in more fashionable terms, Dar-winian selection is such that it may not be the expectedpayoff or fitness that individuals optimize. Rather the fit-ness weighted by heterogeneity, termed heterogeneity pay-off, is what appears as being optimized when replication-selection process is in action. We further remark that as a chaotic attractor has adense set of countably infinite number of unstable pe-riodic orbits, our study on periodic orbits may poten-tially excite interest among researchers to understand themeaning of chaos from the perspective of game theory. Inparticular, it would be an interesting problem to find outhow to generalize the concept of ESS so as to connectit with the asymptotically stable nature of the chaoticattractor.We conclude by pointing out scope for extending theresults reported in this paper. We remind ourselves thatwe have exclusively worked with time-discrete dynam-ics in this paper. Thus, the extension of NE and ESSfor periodic orbits in continuous replicator dynamics re-mains an open problem. Furthermore, what happens ifone relaxes the condition of infinite population is alsoquite an interesting question. Specifically, it is a naturalquestion to ask how HSO or HSS can be defined for fi-nite population in line with the concept of ESS in finitepopulation [49].
ACKNOWLEDGEMENTS
The authors are grateful to Vimal Kumar and VarunPandit for many insightful discussions on game theory.
APPENDICESAppendix A: Two-Player-n-Strategy Game
For two-player-n-strategy game the dimension of payoffmatrix Π is n × n . The condition (cid:80) ni =1 x ( k ) i = 1 impliesthat the effective dynamics is modelled by an ( n − -dimensional dynamical system. The replicator map (referEq. (5)) has the following form for n -strategy populationgame: x ( k +1) i = x ( k +1) i + n (cid:88) h =1 h (cid:54) = i x ( k ) i x ( k ) h (cid:104) ( Π x ( k ) ) i − ( Π x ( k ) ) h (cid:105) , (A1) ∀ i ∈ { , , · · · , n } . Let { ˆ x ( k ) : ˆ x ( k ) i ∈ (0 , , k =1 , , · · · , m } where ˆ x ( i ) (cid:54) = ˆ x ( j ) ∀ i (cid:54) = j represent an m -periodic orbit of replicator map, then in line with thederivation of Eq. (9) we arrive at m (cid:88) k =1 n (cid:88) h =1 h (cid:54) = i ˆ x ( k ) i ˆ x ( k ) h (cid:104) ( Π ˆ x ( k ) ) i − ( Π ˆ x ( k ) ) h (cid:105) = 0 , (A2) ∀ i ∈ { , , · · · , n } .Now, in order to extend the definitions of HO and HSOto the general n -strategy games, we first need to appro-priately define heterogeneity. Noting that even thoughtthe population now has n types, the interactions are stillconfined to two-player interactions. One can thus in-tuit that the heterogeneity must still be defined pair-wise. Consequently, for any arbitrary mixed state x ( k ) ,we define pairwise heterogeneity for any two pure typestagged by the indices, say, h and i where h (cid:54) = i and i, h ∈ { , , · · · , n } ) as H ih x ( k ) ≡ x ( k ) i x ( k ) h . Thus, everytype has contribution in ( n − different pairwise het-erogeneities. Definition of HO( m ): The sequence of states { ˆ x ( k ) :ˆ x ( k ) ∈ (0 , , k = 1 , , · · · , m } where ˆ x ( i ) (cid:54) = ˆ x ( j ) ∀ i (cid:54) = j , isan HO( m ) if ∀ i ∈ { , , · · · , n } and ∀ j ∈ { , , · · · , m } , m (cid:88) k =1 n (cid:88) h =1 h (cid:54) = i H ih ˆ x ( k ) ˆ x ( j ) Tih Π ˆ x ( k ) = m (cid:88) k =1 n (cid:88) h =1 h (cid:54) = i H ih ˆ x ( k ) x Tih Π ˆ x ( k ) , (A3)where ˆ x ( j ) ih (or x ih ) is a mixed state that has same fractionof i th type as that of ˆ x ( j ) (or x ) but comprises only of i th and h th types; e.g., ˆ x ( j )13 = (ˆ x ( j )1 , , − ˆ x ( j )1 , , · · · , and x = (1 − x , , , x , , · · · , . Definition of HSO( m ): HSO( m ) of a map— x ( k +1) i = g ( x ( k ) i ) —is a sequence of states, { ˆ x ( k ) : ˆ x ( k ) ∈ (0 , k = 1 , , · · · , m ; ˆ x ( i ) (cid:54) = ˆ x ( j ) ∀ i (cid:54) = j } such that m (cid:88) k =1 N (cid:88) h =1 h (cid:54) = i H ih x ( k ) ˆ x (1) Tih Π x ( k ) > m (cid:88) k =1 N (cid:88) h =1 h (cid:54) = i H ih x ( k ) x (1) Tih Π x ( k ) , (A4) for any orbit { x ( k ) : x ( k ) ∈ (0 , k = 1 , , · · · , m } of the map starting in some infinitesimal neighbourhood B ˆ x (1) \{ ˆ x (1) } of ˆ x (1) . Without giving the tedious but straightforward details,we comment that in line with the theorem proven forthe case of 2-strategy games, following propositions holdtrue even for any general n -strategy games: m -periodicorbit of replicator map is HO( m ) and vice versa; andlocally asymptotically stable m -period orbit of the repli-cator map is HSO( m ). Appendix B: Case of Monotone Selection Dynamics
Here we intend to show how the concepts of HO( m )and HSO( m ) can be generalized for a rather broad classof dynamics, called monotone selection dynamics [50]which has replicator map as a special case. Specializingfor the case of two-player-two-strategy games, a map, x i ( k +1) = x i ( k ) + φ i ( x ( k ) ) , i = 1 , , (B1)models the selection dynamics, if the conditions givenbelow are satisfied: 1. The simplex, σ , is a forward invariant of this map.2. φ (cid:0) x ( k ) (cid:1) + φ (cid:0) x ( k ) (cid:1) = 0 for all non-negative integer k .3. Both φ (cid:0) x ( k ) (cid:1) and φ (cid:0) x ( k ) (cid:1) are Lipschitz continu-ous on some open neighbourhood in the simplex.4. φ (cid:0) x ( k ) (cid:1) /x ( k ) and φ (cid:0) x ( k ) (cid:1) / (1 − x ( k ) ) are contin-uous real valued functions on the simplex.Now to ensure that the dynamics is a monotone selec-tion dynamics we impose the condition of monotonic-ity: φ (cid:0) x ( k ) (cid:1) /x ( k ) > φ (cid:0) x ( k ) (cid:1) / (cid:0) − x ( k ) (cid:1) if and only if ( Π x ( k ) ) > ( Π x ( k ) ) . Hence, we demand, φ (cid:0) x ( k ) (cid:1) x ( k ) − φ (cid:0) x ( k ) (cid:1)(cid:0) − x ( k ) (cid:1) = β (cid:104) ( Π x ( k ) ) − ( Π x ( k ) ) (cid:105) , (B2)where β must be positive at all times. Now using condi-tion 2 given above and Eq. (B2), we can write the generalform of monotone selection dynamics for two-player-two-strategy game as follows: x ( k +1) = x ( k ) + β H x ( k ) (cid:104)(cid:16) Π x ( k ) (cid:17) − (cid:16) Π x ( k ) (cid:17) (cid:105) . (B3)On comparing Eq. B3 with Eq. (6), it can easily be seenthat one can still connect HO(m) and HSO(m) with them-period orbit of replicator map and its evolutionarilystability if we simply work with rescaled heterogeneityas H x ( k ) → βH x ( k ) . [1] C. Darwin, The life and letters of Charles Darwin, in-cluding an autobiographical chapter , Second ed., editedby S. Darwin, Francis, Vol. 1 (London: John Murray,1887).[2] C. Darwin,
The Descent of Man, and Selection in Rela-tion to Sex , First ed. (London: John Murray, 1871).[3] J. Maynard Smith,
Game theory and the evolution offighting (Edinburgh: Edinburgh University Press, 1972)pp. 8–28.[4] J. Maynard Smith and G. R. Price, Nature , 15(1973).[5] R. Dawkins,
The Selfish Gene (Oxford: Oxford Univer-sity Press, 1989).[6] A. A. Alchian, J. Political Econ. , 211 (1950).[7] M. Friedman, Essays in Positive Economics (Chicago:University of Chicago Press, 1953).[8] R. R. Nelson and S. G. Winter, J. Econ. Perspect. , 23(2002).[9] L. Samuelson, J. Econ. Perspect. , 47 (2002).[10] J. Hofbauer and K. Sigmund, Bull. Am. Math. Soc. ,479 (2003).[11] E. van Damme, Stability and Perfection of Nash Equilib-ria (Berlin: Springer Berlin Heidelberg, 1991).[12] R. B. Myerson, Internat. J. Game Theory , 73 (1978).[13] R. Selten, Internat. J. Game Theory , 25 (1975).[14] P. D. Taylor and L. B. Jonker, Math. Biosci. , 145 (1978).[15] R. Cressman and Y. Tao, Proc. Natl. Acad. Sci. ,10810 (2014).[16] R. Lahkar and W. H. Sandholm, Games Econ. Behav. , 565 (2008), special Issue in Honor of Michael B.Maschler.[17] J. Hofbauer and W. H. Sandholm, Theor. Econ , 341(2011).[18] I. Gilboa and A. Matsui, Econometrica , 859 (1991).[19] G. W. Brown and J. von Neumann, “Solutions of gamesby differential equations,” in Contributions to the Theoryof Games I , Vol. 24, edited by H. W. Kuhn and A. W.Tucker (Princeton: Princeton University Press, 1950) pp.73–79.[20] M. J. Smith, Transp. Sci. , 245 (1984).[21] L. E. Blume, Games Econ. Behav. , 387 (1993).[22] M. Harper and D. Fryer, Dyn. Games Appl. , 318 (2015).[23] A. Nagurney and D. Zhang, Transp. Sci. , 147 (1997).[24] W. H. Sandholm, E. Dokumacı, and R. Lahkar, GamesEcon. Behav. , 666 (2008), special Issue in Honor ofMichael B. Maschler.[25] J. Hofbauer, J. Oechssler, and F. Riedel, Games Econ.Behav. , 406 (2009).[26] M. Feldman, Y. Snappir, and T. Tamir, in Algorith-mic Game Theory , edited by V. Bilò and M. Flam-mini (Cham: Springer International Publishing, 2017) pp. 186–198.[27] V. Pandit, A. Mukhopadhyay, and S. Chakraborty,Chaos , 033104 (2018).[28] B. Skyrms, PSA: Proceedings of the Biennial Meeting ofthe Philosophy of Science Association , 374 (1992).[29] Y. Sato, E. Akiyama, and J. D. Farmer, Proc. Natl.Acad. Sci. , 4748 (2002).[30] G. A. Waters, J. Econ. Dyn. Control , 1201 (2009).[31] D. Ferraioli, SIGecom Exch. , 34 (2013).[32] T. Börgers and R. Sarin, J. Econ. Theory , 1 (1997).[33] J. Hofbauer and K. H. Schlag, J. Evol. Econ. , 523(2000).[34] D. Vilone, A. Robledo, and A. Sánchez, Phys. Rev. Lett. , 038101 (2011).[35] S. H. Rice, Evolutionary Theory: Mathematical and Con-ceptual Foundations , First ed. (Massachusetts: SinauerAssociates, Inc., 1961).[36] S. Iyer, J. Reyes, and T. Killingback, PLoS One , 1(2014).[37] S. Hummert, K. Bohl, D. Basanta, A. Deutsch,S. Werner, G. Theißen, A. Schroeter, and S. Schuster,Mol. BioSyst. , 3044 (2014).[38] D. Friedman, J Evol Econ , 15 (1998).[39] N. L. Komarova, P. Niyogi, and M. A. Nowak, J. Theor. Biol. , 43 (2001).[40] M. A. Nowak, N. L. Kamarova, and P. Niyogi, Science , 114 (2001).[41] D. Fudenberg and T. Strzalecki, Econometrica , 651(2015).[42] J. Lu, Econometrica , 1983 (2016).[43] V. Auletta, D. Ferraioli, F. Pasquale, P. Penna, andG. Persiano, Algorithmica , 511 (2015).[44] G. Waters, Unpublished manuscript (2009).[45] W. Heemels, J. Schumacher, and S. Weiland, Oper. Res.Lett. , 83 (2000).[46] M. Fazli, A. Maazallahi, J. Habibi, and M. Habibi, IEEETNSE (2018), 10.1109/TNSE.2018.2814590.[47] N. Nisan, M. Schapira, and A. Zohar, in Internet andNetwork Economics , edited by C. Papadimitriou andS. Zhang (Berlin, Heidelberg: Springer-Verlag, 2008) pp.531–538.[48] D. M. Kreps,
Game Theory and Economic Modelling (Oxford: Oxford University Press, 1990).[49] M. A. Nowak,
Evolutionary Dynamics: Exploring theEquations of Life (Harvard University Press, 2006).[50] R. Cressman,