Natural Selection as an Inhibitor of Genetic Diversity: Multiplicative Weights Updates Algorithm and a Conjecture of Haploid Genetics
NNatural Selection as an Inhibitor of Genetic Diversity
Multiplicative Weights Updates Algorithm and a Conjecture of Haploid GeneticsRuta MehtaGeorgia Institute of [email protected] Ioannis PanageasGeorgia Institute of [email protected] PiliourasCalifornia Institute of [email protected]
Abstract
In a recent series of papers [7, 8, 6] a surprisingly strong connection was discoveredbetween standard models of evolution in mathematical biology and Multiplicative WeightsUpdates Algorithm, a ubiquitous model of online learning and optimization. These pa-pers establish that mathematical models of biological evolution are tantamount to applyingdiscrete replicator dynamics [19, 13], a close variant of MWUA, on coordination games. Thisconnection allows for introducing insights from the study of game theoretic dynamics intothe field of mathematical biology.Using these results as a stepping stone, we show that mathematical models of haploidevolution imply the extinction of genetic diversity in the long term limit, a widely believedconjecture in genetics [4]. In game theoretic terms we show that in the case of coordinationgames, under minimal genericity assumptions, discrete replicator dynamics converge to pureNash equilibria for all but a zero measure of initial conditions. This result holds despite thefact that mixed Nash equilibria can be exponentially (or even uncountably) many, completelydominating in number the set of pure Nash equilibria. Thus, in haploid organisms thelong term preservation of genetic diversity needs to be safeguarded by other evolutionarymechanisms such as mutations and speciation. a r X i v : . [ m a t h . D S ] O c t Introduction
Decoding the mechanisms of biological evolution has been one of the most inspiring contestsfor the human mind. The modern theory of population genetics has been derived by combiningthe Darwinian concept of natural selection and Mendelian genetics. Detailed experimentalstudies of a species of fruit fly, Drosophila, allowed for a unified understanding of evolutionthat encompasses both the Darwinian view of continuous evolutionary improvements and thediscrete nature of Mendelian genetics. The key insight is that evolution relies on the progressiveselection of organisms with advantageous mutations. This understanding has lead to precisemathematical formulations of such evolutionary mechanisms, dating back to the work of Fisher,Haldane, and Wright [5] in the beginning of the twentieth century.The existence of dynamical models of genotypic evolution, however, does not offer by it-self clear, concise insights about the future states of the phenotypic landscape. Which allelecombinations, and as a result, which attributes will take over?
Prediction of the evolution ofthe phenotypic landscape is a key, alas not well understood, question in the study of biologicalsystems [36].
Despite the advent of detailed mathematical models, still at the forefront of our understand-ing lie experimental studies and simulations. Of course, this is to some extent inevitable sincethe involved dynamical systems are nonlinear and hence a complete theoretical understandingof all related questions seems intractable [29, 11]. Nevertheless, some rather useful qualitativestatements have been established.Nagylaki [20] showed that, when mutations do not affect reproduction success by a lot ,the system state converges quickly to the so-called Wright manifold, where the distribution ofgenotypes is a product distribution of the allele frequencies in the population. In this case, inorder to keep track of the distribution of genotypes in the population it suffices to record thedistribution of the different alleles for each gene. The overall distribution of genotypes can berecovered by simply taking products of the allele frequencies. Nagylaki et al. [21] have alsoshown that under hyperbolicity assumptions ( e.g. , isolated equilibria) such systems converge.Recently, Chastain et al. has built on Nagylaki’s work by establishing an informative connec-tion between these mathematical models of population genetics and the multiplicative updatealgorithm (MWUA). MWUA is a ubiquitous online learning dynamic [2], which is known toenjoy numerous connections to biologically relevant mathematical models. Specifically, its con-tinuous time limit is equivalent to the replicator dynamics (in its standard continuous form) [15]and its equivalent up to a smooth change of variables to the Lotka-Volterra equations [13]. In[7, 8] another strong such connection was established. Specifically, under the assumption of weakselection standard models of population genetics are shown to be closely related to applyingdiscrete replicator dynamics on a coordination game.The resulting coordination game is as follows: Each gene is an agent and its available strate-gies are its alleles. Any combination of strategies/alleles (one for each gene/agent) gives rise to aspecific genotype/individual. The common utility of each gene/agent at that genotype/outcomeis equal to the fitness of that phenotype . If we interpret the frequency of the allele in the popu-lation as mixed (randomized) strategies in this game then the population genetics model reducesto each agent/gene updating their distribution according to discrete replicator dynamics.This connection allows for the translation of results between the areas of genetics and gametheory. We begin with a brief overview of some key insights that have emerged thus far. See Section 3.3 for (non-technical) definition of biological terms. This is referred to as the weak selection regime and it corresponds to a well supported principle known asKimura’s neutral theory. This MWUA variant, which Chastain et al. refer to as discrete MWUA, is already a well established dynamicin the literature of mathematical biology and evolutionary game theory [19, 13] under the name discrete (time,version of) replicator dynamics and to avoid confusion we will refer to it by its standard name. In the weak selection regime this is a number in [1-s,1+s] for some small s >
1n discrete replicator dynamics the rate of increase of the probability of a given strategy isdirectly proportional to its current expected utility. In population genetic terms, this expectedutility reflects the average fitness of a specific allele when matched with the current mixture ofalleles of the other genes. Livnat et al. [17] coined the term mixability to refer to this beneficialattribute. In other words, an allele with high mixability achieves high fitness when pairedagainst the current allele distribution. Naturally, this trait is not a standalone characteristic ofan allele but depends on the current state of the system, i.e. , the other allele frequencies. Anallele that enjoys a high mixability in one distribution of alleles, might exhibit a low mixabilityin another. So, although mixability offers a palpable interpretation of how evolutionary modelsbehaves in a single time step, it does not offer insights about the long term behavior.Game theory, however, can provide us with clues about the long term behavior as well.Specifically, discrete replicator dynamics converges to sets of fixed points in variants of coordi-nation games [19, 25]. This allows for a concise characterization of the possible limit points ofthe population genetics model, since they coincide with the set of equilibria (fixed-points). . In[7, 8] it was observed that random two agent coordination games (in the weak selection payoffregime) exhibit (in expectation) exponentially many such mixed strategies. The abundance ofsuch mixed Nash equilibria seems like a strong indicator that (i) the long term system behaviorwill result in a state of high genetic variance (highly mixed population), (ii) we cannot even ef-ficiently enumerate the set of all biologically relevant limiting behaviors, let alone predict them.We show that this intuition does not reflect accurately the dynamical system behavior. Our game theoretic results.
We show that given a generic two agent coordination games,starting from all but a zero measure of initial conditions, discrete MWUA converges to pure,strict Nash equilibria. The genericity assumption is minimal and merely requires that anyrow/column of the payoff matrix have distinct entries . Our results carry over even if the gamehas uncountably many Nash equilibria.Conceptually, our paper is based on one key idea. Equilibria (fixed-points) of a dynamicalsystem may be unstable.
An example of an unstable equilibrium is an ideal coin that lies onits edge. This is a fixed point/equilibrium of the dynamical system (on paper), but it has nopredictive value in terms of actual system behavior. It corresponds to a probability zero eventin the sense that for the coin to land on its edge the set of allowable starting conditions isnegligible. Even if we try to place an idea coin on its edge, the inherent instability of the statewill amplify even the most minute of disturbances fast and cause it to topple and land on oneof its two stable equilibria, either heads or tails. If we think of this knife edge equilibrium asencoding a mixed state (symmetric 50% heads 50% tails), in the resulting stable states thissymmetry has collapsed and we end up with a pure state. This high level intuition captures theessence behind our theorem. This mixability/symmetry breaking argument is universal in thesense that it holds for all mixed states.Technically, our paper is based mostly on two prior works. In [15] the generic instabilityof mixed Nash was established for other variants of MWUA, including the replicator equation.Our instability analysis follows along similar lines. Any linearly stable equilibrium is shownto be a weakly stable Nash equilibrium . This is a strong refinement of the Nash equilibrium A mixed strategy profile is an equilibrium of our system if and only if, for each agent, all strategies whichare played with positive probability have equal expected utility. This set encompasses the set of Nash equilibriaof the underlying game, since they furthermore require that any strategy that is not played by any agent mustresult in expected utility that is no greater than his current expected payoff. This requirement is not present indiscrete replicator dynamics, since it does not explore new strategies. This genericity assumption, for example, is trivially satisfied with probability one, if the entries of the matrixare iid from a distribution that is continuous and symmetric around zero, say uniform in [ − ,
1] as in [7, 8]. Thisclass of games contains instances with uncountably many Nash equilibria, i.e., A = . This Nash equilibrium refinement was introduced in [15]. A weakly stable Nash is a Nash equilibrium that i.e. , the distance of the state from the set of equilibria goes to zero, instead of the strongerpoint-wise convergence, i.e. , every trajectory has a unique (equilibrium) limit point. Set-wiseconvergence allows for complicated non-local trajectories that weave infinitely often in and outof the neighborhood of an equilibrium making topological arguments hard. Once point-wiseconvergence has been established, the continuum of equilibria can be chopped down into count-able many pieces via Lindel˝of’s lemma and once again standard union bound arguments suffice.Nagylaki et al. pointwise convergence result [21] does not apply here, because their hyperbol-icity assumption is not satisfied. Further, assuming s →
0, they analyze a continuous timedynamical system governed by a differential equation. Unlike Nagylaki the system we analyzeis discrete MWUA, and establish point-wise convergence to pure Nash equilibria almost alwaysfollowing the work of Losert and Akin [19], even if hyperbolicity is not satisfied (uncountablymany equilibria).We close our paper with some technical observations about the speed of divergence from theset of unstable equilibria as well as discussing an average case analysis approach for predictingthe probabilities that we converge to any of the pure equilibria given a random initial condition.We believe that these observations could stimulate future work in the area.
Biological Interpretation.
Our work sheds new light on the role of natural selection in hap-loid genetics. We show that natural selection acts as an antagonistic process to the preservationof genetic diversity. The long term preservation of genetic diversity needs to be safeguarded byevolutionary mechanisms which are orthogonal to natural selection such as mutations and speci-ation. This view, although may appear linguistically puzzling at first, is completely compatiblewith the mixability interpretation of [17, 6]. Mixability implies that good “mixer” alleles ( i.e. ,alleles that enjoy high fitness in the current genotypic landscape) gain an evolutionary advantageover their competition. On the other hand, the preservation of mixed populations relies on thisevolutionary race between alleles having no clear long term winner with the average-over-timemixability of two, or more, alleles being roughly equal . As with actual races, ties are rare andhence mixability leads to non-mixed populations in the long run.According to recent PNAS commentary [4] some of the points in [6] raised questions whencompared against commonly held beliefs in mathematical biology.“Chastain et al. (1) suggest that the representation of selection as (partially) maxi-mizing entropy may help us understand how selection maintains diversity. However, satisfies the extra property that if you force any single randomizing agent to play any strategy in his currentsupport with probability one, all other agents remain indifferent between the strategies in their support. In game theoretic terms, in order for two strategies to be played with positive probability by the same agentin the long run, it must be the case that the time-average expected utilities of these two strategies are roughlyequal. The time average here is over the history of play so far.
3t is widely believed that selection on haploids (the relevant case here) cannot main-tain a stable polymorphic equilibrium. There seems to be no formal proof of this inthe population genetic literature. . . ”Our argument above helps bridge this gap between belief and theory.
MWUA, Genetics, Ecology and Biology.
The earliest connection, to our knowledge, be-tween MWUA and genetics lies in [15], where such a connection is established between MWUA(in its usual exponential form) and replicator dynamics [31, 28], one of the most basic tools inmathematical ecology, genetics, and mathematical theory of selection and evolution. Specifi-cally, MWUA is up to first order approximation equivalent to replicator dynamics. Since theMWUA variant examined in [6] is an approximation of its standard exponential form, theseresults follow a unified theme. MWUA in its classic form is up to first order approximationequivalent to models of evolution. The MWUA variant examined in [6] was introduced by Losertand Akin in [19] in a paper that also brings biology and game theory together. Specifically, theyuse game theoretic analysis to prove the first point-wise convergence to equilibria for a class ofevolutionary dynamics resolving an open question at the time. We build on the techniques ofthis paper, while also exploiting the (in)stability analysis of mixed equilibria along the lines of[15]. The connection between MWUA and replicator dynamics by [15] also immediately impliesconnections between MWUA and mathematical ecology. This is because replicator dynamicsis known to be equivalent (up to a diffeomorphism) to the classic prey/predator populationmodels of Lotka-Voltera [13].
MWUA (and variants) in game theory.
As a result of the discrete nature of MWUA,its game theoretic analysis tends to be trickier than that of its continuous time variant, thereplicator. Analyzed settings of this family of dynamics include zero-sum games [1, 27], potential(congestion) games [15], games with non-converging behavior [14, 9, 12, 3, 16] and as well asfamilies of network games [25, 26]. New techniques can predict analytically the limit point ofreplicator systems starting from randomly chosen initial condition. This approach is referredto as average case analysis of game dynamics [23].
Genetics and Computer Science.
In the last couple of years we have witnessed an accu-mulation of papers and problem proposals in the intersection of computer science and genetics[18, 33, 32, 34, 10]. In the closing sections of our paper, we add to this exciting discussion bypointing out some new challenges along these lines.
In this section we formally describe the two player coordination games, the dynamics underconsideration, and its equivalence with MWUA in evolution. First we start with some notations.
Notations:
All vectors are in bold-face letters, and are considered as column vectors. Todenote a row vector we use x T . The i th coordinate of x is denoted by x i , and for l < k , x ( l : k )denote subvector ( x l , x l +1 , . . . , x k ). For two vectors x , y let ( x ; y ) denote the concatenation oftwo vectors. For k ∈ R , k n × n represents n × n matrix with all entries set to k . We denote set { , . . . , n } by [1 : n ]. int A is the interior of set A .4 .1 Two-player Games and Nash equilibrium In this paper we consider two-player games, where each player has finitely many pure strate-gies (moves). Let S i , i = 1 , i , and let m def = | S | and n def = | S | .Then such a game can be represented by two payoff matrices A and B of dimension m × n ,where payoff to the players are A ij and B ij respectively if the first-player plays i and the secondplays j .Players may randomize among their strategies. The set of mixed strategies for the firstplayer is ∆ = { x = ( x , . . . , x m ) | x ≥ , (cid:80) mi =1 x i = 1 } , and for the second player is ∆ = { y =( y , . . . , y n ) | y ≥ , (cid:80) nj =1 y j = 1 } . The expected payoffs of the first-player and second-playerfrom a mixed-strategy ( x , y ) ∈ ∆ × ∆ are, respectively (cid:88) i,j A ij x i y j = x T A y and (cid:88) i,j B ij x i y j = x T B y Lemma 1. (Nash Equilibrium [35]) A strategy profile is said to be a Nash equilibrium strategyprofile (NESP) if no player achieves a better payoff by a unilateral deviation [22]. Formally, ( x , y ) ∈ ∆ m × ∆ n is a NESP iff ∀ x (cid:48) ∈ ∆ m , x T A y ≥ x (cid:48) T A y and ∀ y (cid:48) ∈ ∆ n , x T B y ≥ x T B y (cid:48) . Given strategy y for the second-player, the first-player gets ( A y ) k from her k th strategy.Clearly, her best strategies are arg max k ( A y ) k , and a mixed strategy fetches the maximumpayoff only if she randomizes among her best strategies. Similarly, given x for the first-player,the second-player gets ( x T B ) k from k th strategy, and same conclusion applies. These can beequivalently stated as the following complementarity type conditions, ∀ i ∈ S , x i > ⇒ ( A y ) i = max k ∈ S ( A y ) k ∀ j ∈ S , y j > ⇒ ( x T B ) j = max k ∈ S ( x T B ) k (1) Symmetric Game.
Game (
A, B ) is said to be symmetric if B = A T . In a symmetric gamethe strategy sets of both the players are identical, i.e., m = n , and S = S . We will use n , S and ∆ n to denote number of strategies, the strategy set and the mixed strategy set respectivelyof the players in such a game. A Nash equilibrium profile ( x , y ) ∈ ∆ n × ∆ n is called symmetric if x = y . Note that at a symmetric strategy profile ( x , x ) both the players get payoff x T A x .Using (1) it follows that x ∈ X is a symmetric NE of game ( A, A T ), with payoff π to bothplayers, if and only if, ∀ i ∈ S, x i > ⇒ ( A x ) i = max k ( A x ) k (2) Coordination Game.
In a coordination game B = A , i.e., both the players get the samepayoff regardless of who is playing what. Note that such a game always has a pure equilibrium,namely arg max ( i,j ) A ij . A dynamical system is a smooth action of the reals or the integers on another object (usuallya manifold). When the integers are acting, the system is called discrete and is given by thefollowing update rule: x ( n + 1) = f ( x ( n ))with n ∈ N or Z where f is called the rule/map of the dynamic. A point x ∗ is called fixedpoint or equilibrium if f ( x ∗ ) = x ∗ . A trajectory of the dynamical system is a (infinite) sequenceof vectors x (0) , f ( x (0)) , f ( x (0)) , ... where f n is the composition of f for n times. Dynamicalsystem theory is the branch of mathematics that tries to understand the behavior of dynamicalsystems. To understand their behavior, there are plenty of questions one needs to answer. Doesthe system converge? What is the rate of convergence? Which are the stable fixed points?5 ames and discrete MWUA of [6]. Chastain et. al. [6] observed that the update rulederived by Nagylaki [20] for allele frequencies, during evolutionary process under weak-selection,is exactly multiplicative weight update algorithm (MWUA) applied on coordination game, wheregenes are players and alleles are their strategies. Formally, if fitness values of a genome definedby a combination of alleles (strategy profile) is from [1 − s, s ] for a small s > B = m × n + (cid:15)C , where each C ij ∈ Z and (cid:15) <<
1. This, defines a coordination game (
B, B ).Further, the change in allele frequencies in each new generation is as per the following rule: ∀ i ∈ S , x i ( t + 1) = x i ( t )(1 + (cid:15) ( C y ( t )) i )1 + (cid:15) x ( t ) T C y ( t ) ; ∀ j ∈ S , y j ( t + 1) = y j ( t )(1 + (cid:15) ( C T x ( t )) j )1 + (cid:15) x ( t ) T C y ( t ) (3)Using the fact that B = m × n + (cid:15)C , this can be reformulated as, x i ( t )(1 + (cid:15) ( C y ( t )) i )1 + (cid:15) x ( t ) T C y ( t ) = x i ( t ) ( B y ( t )) i x T ( t ) B y ( t ) ; y j ( t )(1 + (cid:15) ( C T x ( t )) j )1 + (cid:15) x ( t ) T C y ( t ) = y j ( t ) ( B T x ( t )) j x T ( t ) B y ( t )Thus, in this paper we study convergence of discrete MWUA through this reformulation.Thisreformulation is also known as discrete replicator dynamics . In general, given a game ( A, B )consider the update rule (map) f : ∆ m × ∆ n → ∆ m × ∆ n ,For ( x , y ) ∈ ∆ m × ∆ n if ( x (cid:48) , y (cid:48) ) = f ( x , y ), then ∀ i ∈ S , x (cid:48) i = x i ( A y ) i x T A y ∀ j ∈ S , y (cid:48) j = y j ( x T B ) j x T B y (4)Clearly, x (cid:48) ∈ ∆ m , y (cid:48) ∈ ∆ n , and therefore f is well-defined. Starting with ( x (0) , y (0)), thestrategy profile at time t ≥ x ( t ) , y ( t )) = f ( x ( t − , y ( t − f t ( x (0) , y (0)). Losert and Akin [19].
Losert and Akin showed a very interesting result on the convergenceof discrete replicator dynamics when applied on evolutionary games [5] with positive matrix.These games are symmetric games, where pure strategies are species and the player is playingagainst itself, i.e., symmetric strategy ( x = y ). Consider a symmetric game ( A, A T ) where A isan n × n positive matrix, and the following dynamics starting with z (0) ∈ ∆ n . z i ( t + 1) = z i ( t ) ( A z ( t )) i z ( t ) T A z ( t ) (5)Clearly, z ( t + 1) ∈ ∆ n , ∀ t ≥
1. Thus, there is a map f s : ∆ n → ∆ n corresponding to theabove dynamics, where if z (cid:48) = f s ( z ) then z (cid:48) i = z i ( A z ) i z T A z , implying z ( t + 1) = f s ( z ( t )) = f t +1 s ( z (0)) (6)If z ( t ) is a fixed-point of f s then z ( t (cid:48) ) = z ( t ) , ∀ t (cid:48) ≥ t . Losert and Akin [19] proved that theabove dynamical system converges pointwise to fixed-point, and that map f is a diffeomorphismin an open set that contains ∆ n . Formally: Theorem 2. [19] Let { z ( t ) } be an orbit for the dynamic of (5). As t approaches ∞ , z ( t ) converges to a unique fixed-point q . Additionally, the map f s corresponding to (5) is a diffeo-morphism, i.e. it is a one-to-one, onto, and smooth function whose inverse function is alsosmooth. .3 Terms used in biology We provide brief non-technical definitions of a few biological terms that we use in this paper.
Gene.
A unit that determines some characteristic of the organism, and passes traits to off-springs. All organisms have genes corresponding to various biological traits, some of which areinstantly visible, such as eye color or number of limbs, and some of which are not, such as bloodtype.
Allele.
Allele is one of a number of alternative forms of the same gene, found at the same placeon a chromosome, Different alleles can result in different observable traits, such as differentpigmentation.
Genotype.
The genetic constitution of an individual organism.
Phenotype.
The set of observable characteristics of an individual resulting from the interactionof its genotype with the environment.
Diploid.
Diploid means having two copies of each chromosome. Almost all of the cells in thehuman body are diploid.
Haploid.
A cell or nucleus having a single set of unpaired chromosomes. Our sex cells (spermand eggs) are haploid cells that are produced by meiosis. When sex cells unite during fertiliza-tion, the haploid cells become a diploid cell.
In this section we show that the discrete replicator dynamics of (4), when applied to atwo-player coordination game (
B, B ), converges pointwise to a fixed-point of f under weakselection. Further, map f is diffeomorphism. Essentially we will reduce the problem to applyingdiscrete replicator dynamics on symmetric game with positive matrix and then use the resultof Losert and Akin [19] (Theorem 2).Under weak selection regime we have B ij ∈ [1 − s, s ] , ∀ ( i, j ), for some s <
1. Let (cid:15) < − s , and consider the following matrix A = (cid:20) (cid:15) m × m B − (cid:15)B T − (cid:15) (cid:15) n × n (cid:21) (7)We will show that applying dynamics of (4) on game ( B, B T ) starting at ( x (0) , y (0)) is sameas applying (5) on game ( A, A T ) starting at z (0) = ( x (0)2 ; y (0)2 ). Lemma 3.
Given ( x (0) , y (0)) ∈ ∆ , ∆ , let z (0) = ( x (0)2 ; y (0)2 ) , then ∀ t ≥ , ( x ( t ); y ( t )) =2 ∗ z ( t ) , where x ( t ) and y ( t ) are as per (4) and z ( t ) is as per (5). Proof
We will show the result by induction. By hypothesis the base case of t = 0 holds.Suppose, it holds up to time t , then let x = x ( t + 1), y = y ( t + 1) and ( x (cid:48) ; y (cid:48) ) = z ( t + 1). Now, ∀ i ≤ m + n, z i ( t + 1) = z i ( t ) ( A z ( t )) i z ( t ) T A z ( t ) together with z ( t ) = ( x ( t ); y ( t )) gives us ∀ i ≤ m, x (cid:48) i = x i ( t )2 (cid:15) (cid:80) i x i ( t )2 + ( B y ( t )) i − (cid:15) (cid:80) j y j ( t )2 x ( t ) T B y ( t )4 + y ( t ) B T x ( t )4 = 2 x i ( t )4 ( B y ( t )) i x ( t ) T B y ( t ) = x i ∀ j ≤ n, y (cid:48) j = y j , and the lemma follows.7emmas 3 establishes equivalence between games ( B, B ) and (
A, A T ) in terms of dynamics,and thus the next theorem follows using Theorem 2. Theorem 4.
Let { x ( t ) , y ( t ) } be an orbit for the dynamic of (4). As t approaches ∞ , ( x ( t ) , y ( t )) converges to a unique fixed-point ( p , q ) . Additionally, the map F corresponding to (4) is adiffeomorphism, i.e. it is a one-to-one, onto, smooth function whose inverse function is alsosmooth. In Section 4 we saw that dynamics of (4) converges to a fixed point regardless of where westart in coordination games with weak-selection. However, which equilibrium it converges todepends on the starting point. In this section we show that it almost always converge to a pureNash equilibrium under mild genericity assumptions on the game matrix. In the light of theknown fact that a coordination game (
B, B ), where B ij s are chosen uniformly at random from[1 − s, s ], may have exponentially many mixed NE [7, 8], this result comes as a surprise.To show the result, we use the concept of weakly stable Nash equilibrium [15]. This isa refinement of the classic notion of equilibrium and we show that for coordination games itcoincides with pure NE under some mild assumptions. Further, we connect them to stable fixed-points of f (4) by showing that all stable fixed points of f are weakly stable Nash equilibria.Finally, using the Center Stable Manifold Theorem [30] we show that dynamics defined by f converges to stable fixed-points except for a zero-measure set of starting points. Definition 5. [15]A Nash equilibrium ( x , y ) is called weakly stable if fixing one of the playersto choosing a pure strategy in the support of her strategy with probability one, leaves the otherplayer indifferent between the strategies in his support, e.g., let T and T are supports of x and y respectively, then for any i ∈ T if the first player plays i with probability one then the secondplayer is indifferent between all the strategies of T , and vice-versa. Note that pure
NE are always weakly stable, and coordination games always have pure NE.Further, for a mixed-equilibrium to be weakly stable , for any i ∈ T all the Bij s correspondingto j ∈ T are the same. Thus, the next lemma follows. Lemma 6.
If coordinates of a row or a column of B are all distinct, then every weakly stable equilibrium is a pure Nash equilibrium. Proof
To the contrary suppose ( x , y ) is a mixed weakly stable NE, then for T = { i | x i > } and T = { j | y j > } we have ∀ i ∈ T , B ij = Bij (cid:48) , ∀ j (cid:54) = j (cid:48) ∈ T , a contradiction. Remark 7.
We note that the games analyzed in [7, 8], where entries of matrix B are chosenuniformly at random from the interval [1 − s, s ] , will have distinct entries in each of itsrows/columns with probability one, and thereby due to Lemma 6 all its weakly stable NE arepure NE.
Stability of a fixed-point is defined based on eigenvalues of Jacobian matrix evaluated at thefixed-point. So let us first describe the Jacobian matrix of function f . We denote this matrixby J which is m + n × m + n , and let f k denote the function that outputs k th coordinate of f .Then, ∀ i (cid:54) = i (cid:48) ≤ m and ∀ j (cid:54) = j (cid:48) ≤ nJ ii = df i dx i = ( By ) i x T By − x i (cid:16) ( By ) i x T By (cid:17) , J ( m + j )( m + j ) = df m + j dy j = ( B T x ) i x T By − y i (cid:16) ( B T x ) i x T By (cid:17) J ii (cid:48) = df i dx i (cid:48) = − x i ( By ) i · ( By ) i (cid:48) ( x T By ) , J ( m + j )( m + j (cid:48) ) = df m + j dy j (cid:48) = − y j ( B T x ) j · ( B T x ) j (cid:48) ( x T By ) J i ( m + j ) = df i dy j = x i B ij · ( x T By ) − ( By ) i ( B T x ) j ( x T By ) , J ( m + j ) i = df m + j dx i = y j B ij · ( x T By ) − ( B T x ) j ( By ) i ( x T By ) Center Stable Manifold Theorem (see Theorem 14), we need a map whosedomain is full-dimensional around the fixed-point. However, an n -dimensional simplex (∆ n ) in R n has dimension n −
1, and therefore the domain of f , namely ∆ m × ∆ n is of dimension m + n − R m + n . Therefore, we need to take a projection of the domain space and accordinglyredefine the map f . We note that the projection we take will be fixed-point dependent; this isto keep of the proof of Lemma 10 relatively less involved later.Let r = ( p , q ) be a fixed-point of map f in ∆ m × ∆ n . Define i ( r ) and j ( r ) to be coordinatesof p and q respectively that are non-zero, i.e. p i ( r ) > q j ( r ) >
0. Consider the mapping z r : R m + n → R m + n − so that we exclude from each player 1 , x i ( r ) , y j ( r ) respectively.We substitute the variables x i ( r ) with 1 − (cid:80) i (cid:54) = i ( r ) x i and y j ( r ) with 1 − (cid:80) j (cid:54) = j ( r ) y j . Considermap f under the projection z r , and let J r denote the projected Jacobian at r . Then, ∀ i, i (cid:48) ∈ [1 : m ] \ { i ( r ) } and ∀ j, j (cid:48) ∈ [1 : n ] \ { j ( r ) } , J r ii = ( By ) i x T By − x i (cid:16) ( By ) i x T By (cid:17) + x i ( By ) i · ( By ) i ( r ) ( x T By ) J r ( m + j )( m + j ) = ( B T x ) j x T By − y j (cid:16) ( B T x ) j x T By (cid:17) + y j ( B T x ) j · ( B T x ) j ( r ) ( x T By ) J r ii (cid:48) = − x i ( By ) i · ( By ) i (cid:48) ( x T By ) + x i ( By ) i · ( By ) i ( r ) ( x T By ) J r ( m + j )( m + j (cid:48) ) = − y j ( B T x ) j · ( B T x ) j (cid:48) ( x T By ) + y j ( B T x ) j · ( B T x ) j ( r ) ( x T By ) J r ij = x i B ij · ( x T By ) − ( By ) i ( B T x ) j ( x T By ) − x i B ij ( r ) · ( x T By ) − ( By ) i ( B T x ) j ( r ) ( x T By ) J r ( m + j ) i = y j B ij · ( x T By ) − ( B T x ) j ( By ) i ( x T By ) − y j B i ( r ) j · ( x T By ) − ( B T x ) j ( By ) i ( r ) ( x T By ) (8)The characteristic polynomial of J r at r is (cid:89) i : p i =0 (cid:18) λ − ( B q ) i p T B q (cid:19) (cid:89) i : q j =0 (cid:18) λ − ( B T p ) j p T B q (cid:19) × det ( λI − J r )where J r corresponds to J r at r by deleting rows i ,columns j such that p i = 0 and q j = 0. Definition 8.
A fixed point r is called linearly stable, if the eigenvalues of J r at r have absolutevalue less than or equal to 1. Otherwise it is called linear unstable. The definition above is a slight modification of the classic definition of a stable fixed point,and has been tailored so that use of Theorem 14 becomes easier. The intuition here is thatlinearly unstable fixed points are going to be discarded by the dynamics in a robust manner, soit suffices to characterize the set of linearly stable fixed points. Throughout the paper when werefer to (un)stable fixed points, we refer to this definition of stability.
Lemma 9.
Every linearly stable fixed point is a Nash Equilibrium.
Proof
Assume that a linearly stable fixed point r is not a Nash equilibrium. Without lossof generality suppose player t = 1 can deviate and gain. Since r is a fixed-point of map f , ∀ p i > ⇒ ( B q ) i = p T B q . Hence, there exists a strategy i ≤ m such that p i = 0 and( B q ) i > p T B q . Then the characteristic polynomial has ( B q ) i p T B q > Lemma 10.
Every linearly stable fixed point is a weakly stable Nash equilibrium. roof Let k × k be the size of matrix J r . If k = 0 then the equilibrium is pure and thereforeis stable. For the case when k >
0, let T p and T q be the support of p and q respectively, i.e., T p = { i | p i > } and similarly T q . If we show that ∀ i, i (cid:48) ∈ T p and ∀ j, j (cid:48) ∈ T q , M i,i (cid:48) ,j,j (cid:48) =( B ij − B i (cid:48) j ) − ( B ij (cid:48) − B i (cid:48) j (cid:48) ) = 0, then using argument similar to Theorem 3.8 in [15], the lemmafollows. We show this using the expression of tr (( J r ) ). Claim 11. tr (( J r ) ) = k + p T B q ) (cid:88) i
Since J r ii (cid:48) = 0, J r ( m + j )( m + j (cid:48) ) = 0 for i (cid:54) = i (cid:48) and j (cid:54) = j (cid:48) , and J r ii = 1, J r ( m + j )( m + j ) = 1 weget that tr (( J r ) ) = k + (cid:88) i,j J r i ( m + j ) J r ( m + j ) i We consider the following cases: • Let i < i (cid:48) with i, i (cid:48) (cid:54) = i ( r ) and j < j (cid:48) with j, j (cid:48) (cid:54) = j ( r ) and we examine the term p T B q ) p i q j p i (cid:48) q j (cid:48) in the sum and we get that it appears with[[ M i,i (cid:48) ,j,j ( r ) ] × [ M i,i ( r ) ,j,j (cid:48) ] + [ M i,i (cid:48) ,j,j ( r ) ] × [ M i ( r ) ,i (cid:48) ,j,j (cid:48) ]+[ M i,i (cid:48) ,j ( r ) ,j (cid:48) ] × [ M i,i ( r ) ,j,j (cid:48) ] + [ M i,i (cid:48) ,j ( rr ) ,j (cid:48) ] × [ M i ( r ) ,i (cid:48) ,j,j (cid:48) ]=( M i,i (cid:48) ,j,j (cid:48) ) • Let i (cid:54) = i ( r ) and j (cid:54) = j ( r ). The term p T B q ) p i q j p i ( r ) q j ( r ) in the sum appears in multipli-cation with ( M i,i ( r ) ,j,j ( r ) ) . • Let i < i (cid:48) with i, i (cid:48) (cid:54) = i ( r ) and j (cid:54) = j ( r ). The term p T B q ) p i q j p i (cid:48) q j ( r ) in the sum appearswith [ M i,i (cid:48) ,j,j ( r ) ] × [ M i,i ( r ) ,j,j ( r ) ] + [ M i,i (cid:48) ,j,j ( r ) ] × [ M i ( r ) ,i (cid:48) ,j,j ( r ) ]= ( M i,i (cid:48) ,j,j ( r ) ) • Similarly to the previous case, for j < j (cid:48) with j, j (cid:48) (cid:54) = j ( r ) and i (cid:54) = i ( r ). The term p T B q ) p i q j p i ( r ) q j (cid:48) in the sum appears with ( M i,i ( r ) ,j,j (cid:48) ) .Trace of ( J r ) can not be larger than k , otherwise there exists an eigenvalue with absolutevalue greater than one contradicting r being a stable fixed-point. From the above claim, it isclear that tr (( J q ) ) ≥ k and it is exactly k if and only if M i,i (cid:48) ,j,j (cid:48) = 0, ∀ i, i (cid:48) ∈ T and j, j (cid:48) ∈ T ,and the lemma follows.In Appendix A we show that except for zero measure starting points ( x (0) , y (0)) the dy-namics of (4) converges to stable fixed-points using the Center Stable Manifold Theorem , whichproves the next theorem.
Theorem 12.
The set of initial conditions in ∆ m × ∆ n so that the dynamical system convergesto unstable fixed points has measure zero. Theorem 12 together with Lemmas 6 and 10 gives the following main result.
Theorem 13.
For all but measure zero initial conditions in ∆ m × ∆ n , the dynamical system(4) when applied to a coordination game ( B, B ) with B ij ∈ [1 − s, s ] , ∀ ( i, j ) for s < ,converges to weakly stable Nash equilibria. Furthermore, assuming that entries in each row andcolumn of B are distinct, it converges to pure Nash equilibria. Conclusion
We show that standard mathematical models of haploid evolution imply the extinction ofgenetic diversity in the long term limit. This reflects a widely believed conjecture in populationgenetics [4]. We prove this via recent established connections between game theory, learningtheory and genetics [7, 8, 6]. Specifically, in game theoretic terms we show that in the case ofcoordination games, under minimal genericity assumptions, discrete MWUA converges to pureNash equilibria for all but a zero measure of initial conditions. This result holds despite thefact that mixed Nash equilibria can be exponentially (or even uncountably) many, completelydominating in number the set of pure Nash equilibria. Thus, in haploid organisms the long termpreservation of genetic diversity needs to be safeguarded by other evolutionary mechanisms suchas mutations and speciation.The intersection between computer science, genetics and game theory has already providedsome unexpected results and interesting novel connections. As these connections become clearer,new questions emerge alongside the possibility of transferring knowledge between these areas.In appendix C we raise some novel questions that have to do with speed of dynamics as wellas the possibility of understanding the evolution of biological systems given random initialconditions. Such an approach can be thought of as a middle ground between Price of Anarchy(worst case scenario) and Price of Stability (best case scenario) in game theory. We believe thatthis approach can also be useful from the standard game theoretic lens [23].
We would like to thank Prasad Tetali for helpful discussions and suggestions.
References [1] E. Akin and V. Losert. Evolutionary dynamics of zero-sum games.
J. of Math. Biology ,20:231–258, 1984.[2] Sanjeev Arora, Elad Hazan, and Satyen Kale. The multiplicative weights update method:a meta algorithm and applications. Technical report, 2005.[3] Maria-Florina Balcan, Florin Constantin, and Ruta Mehta. The weighted majority al-gorithm does not converge in nearly zero-sum games. In
ICML Workshop on Markets,Mechanisms and Multi-Agent Models , 2012.[4] Nicholas H Barton, Sebastian Novak, and Tiago Paix˜ao. Diverse forms of selection inevolution and computer science.
Proceedings of the National Academy of Sciences (PNAS) ,111(29):10398–10399, 2014.[5] Peter J. Bowler.
Evolution: The History of an Idea . University of California Press, 1989.[6] Erick Chastain, Adi Livnat, Christos Papadimitriou, and Umesh Vazirani. Algorithms,games, and evolution.
Proceedings of the National Academy of Sciences , 2014.[7] Erick Chastain, Adi Livnat, Christos H. Papadimitriou, and Umesh V. Vazirani. Multi-plicative updates in coordination games and the theory of evolution.
CoRR , abs/1208.3160,2012.[8] Erick Chastain, Adi Livnat, Christos H. Papadimitriou, and Umesh V. Vazirani. Multi-plicative updates in coordination games and the theory of evolution. In
ITCS [7], pages57–58. 119] Constantinos Daskalakis, Rafael Frongillo, Christos H. Papadimitriou, George Pierrakos,and Gregory Valiant. On learning algorithms for nash equilibria. In Spyros Kontogiannis,Elias Koutsoupias, and PaulG. Spirakis, editors,
Algorithmic Game Theory , volume 6386of
Lecture Notes in Computer Science , pages 114–125. Springer Berlin Heidelberg, 2010.[10] Narendra M. Dixit, Piyush Srivastava, and Nisheeth K. Vishnoi. A finite populationmodel of molecular evolution: Theory and computation.
Journal of Computational Bi-ology , 19(10):1176–1202, 2012.[11] Michael Doebeli and Iaroslav Ispolatov. Chaos and unpredictability in evolution.
Evolution ,68(5):1365–1373, 2014.[12] A. Gaunersdorfer and J. Hofbauer. Fictitious play, shapley polygons and the replicatorequation.
Games and Economic Behavior , 11(2):279–303, 1995.[13] J. Hofbauer and K. Sigmund.
Evolutionary Games and Population Dynamics . CambridgeUniversity Press, Cambridge, 1998.[14] R. Kleinberg, K. Ligett, G. Piliouras, and ´E. Tardos. Beyond the Nash equilibrium barrier.In
Symposium on Innovations in Computer Science (ICS) , 2011.[15] Robert Kleinberg, Georgios Piliouras, and ´Eva Tardos. Multiplicative updates outper-form generic no-regret learning in congestion games. In
ACM Symposium on Theory ofComputing (STOC) , 2009.[16] K. Ligett and G. Piliouras. Beating the best Nash without regret.
SIGecom Exchanges 10 ,2011.[17] Adi Livnat, Christos Papadimitriou, Jonathan Dushoff, and Marcus W. Feldman. A mix-ability theory for the role of sex in evolution.
Proceedings of the National Academy ofSciences , 105(50):19803–19808, 2008.[18] Adi Livnat, Christos Papadimitriou, Aviad Rubinstein, Andrew Wan, and Gregory Valiant.Satisfiability and evolution. In
FOCS , 2014.[19] V. Losert and E. Akin. Dynamics of games and genes: Discrete versus continuous time.
Journal of Mathematical Biology , 1983.[20] T Nagylaki. The evolution of multilocus systems under weak selection.
Genetics ,134(2):627–47, 1993.[21] T. Nagylaki, J. Hofbauer, and P. Brunovsk´y. Convergence of multilocus systems underweak epistasis or weak selection.
J. Math Biol. , (2):103–33, 1999.[22] J. Nash. Equilibrium points in n-person games.
Proceedings of the National Academy ofSciences , pages 48–49, 1950.[23] I. Panageas and G. Piliouras. http://arxiv.org/pdf/1403.3885v4.pdf, 2014.[24] Lawrence Perko.
Differential Equations and Dynamical Systems . Springer, 3nd. edition,1991.[25] G. Piliouras and J. S. Shamma. Optimization despite chaos: Convex relaxations to complexlimit sets via Poincar´e recurrence. In
SODA , 2014.1226] Georgios Piliouras, Carlos Nieto-Granda, Henrik I. Christensen, and Jeff S. Shamma. Per-sistent patterns: Multi-agent learning beyond equilibrium and utility. In
Proceedings of the2014 International Conference on Autonomous Agents and Multi-agent Systems , AAMAS’14, pages 181–188, Richland, SC, 2014. International Foundation for Autonomous Agentsand Multiagent Systems.[27] Yuzuru Sato, Eizo Akiyama, and J. Doyne Farmer. Chaos in learning a simple two-persongame.
Proceedings of the National Academy of Sciences , 99(7):4748–4751, 2002.[28] Peter Schuster and Karl Sigmund. Replicator dynamics.
Journal of Theoretical Biology ,100(3):533 – 538, 1983.[29] Michael Scriven. Explanation and prediction in evolutionary theory: Satisfactory expla-nation of the past is possible even when prediction of the future is impossible.
Science ,130(3374):477–482, 1959.[30] Michael Shub.
Global Stability of Dynamical Systems . Springer-Verlag, 1987.[31] Peter D. Taylor and Leo B. Jonker. Evolutionary stable strategies and game dynamics.
Mathematical Biosciences , 40(12):145–156, 1978.[32] Nisheeth K. Vishnoi. Evolution without sex, drugs andboolean functions, 2012. http://research.microsoft.com/en-us/um/people/nvishno/site/Publications files/evolution-without-sex.pdf.[33] Nisheeth K. Vishnoi. Making evolution rigorous: The error threshold. In
Proceedings of the4th Conference on Innovations in Theoretical Computer Science , ITCS ’13, pages 59–60,New York, NY, USA, 2013. ACM.[34] Nisheeth K. Vishnoi. The speed of evolution, 2013. http://research.microsoft.com/en-us/um/people/nvishno/Site/Publications files/VishnoiMixing.pdf.[35] Bernhard von Stengel. Equilibrium computation for two-player games in strategic andextensive form.
Chapter 3, Algorithmic Game Theory, eds. N. Nisan, T. Roughgarden, E.Tardos, and V. Vazirani , pages 53–78, 2007.[36] Carl Zimmer. The new science of evolutionary forecasting.
Quanta
Proof of Theorem 12
To prove Theorem 12, we will make use of the following important theorem in dynamicalsystems.
Theorem 14. (Center and Stable Manifolds, p. 65 of [30]) Let p be a fixed point for the C r local diffeomorphism h : U → R n where U ⊂ R n is an open (full-dimensional) neighborhood of p in R n and r ≥ . Let E s ⊕ E c ⊕ E u be the invariant splitting of R n into generalized eigenspacesof Dh ( p ) corresponding to eigenvalues of absolute value less than one, equal to one, and greaterthan one. To the Dh ( p ) invariant subspace E s ⊕ E c there is an associated local h invariant C r embedded disc W scloc of dimension dim ( E s ⊕ E c ) , and ball B around p such that: h ( W scloc ) ∩ B ⊂ W scloc . If h n ( x ) ∈ B for all n ≥ , then x ∈ W scloc (9)To use the theorem above we need to project the vector field to a lower dimensional space.We consider the (diffeomorphism) function g that is a projection of the points ( x , y ) ∈ R m + n to R m + n − by excluding a specific (the ”first”) variable for each player (we know that the proba-bilities must sum up to one for each player). Let N = m + n , then we denote this projection of∆ N by g (∆ N ), i.e., ( x , y ) → g ( x (cid:48) , y (cid:48) ) where x (cid:48) = ( x , . . . , x n ) and y (cid:48) = ( y , . . . , y n ). Further,recall the fixed-point dependent projection function z r defined in Section 5, where we remove x i ( r ) and y j ( r ) .Map f is one corresponding to dynamical system (4). For an unstable fixed point r we considerthe function ψ r ( v ) = z r ◦ f ◦ z − r ( v ) which is C local diffeomorphism (due to theorem 4 we knowthat the rule of the dynamical system is a diffeomorphism), ( v ) ∈ R N − . Let B r be the (open)ball that is derived from Theorem 14 and we consider the union of these balls (transformed in R N − ) A = ∪ r A r where A r = g ( z − r ( B r )) ( z − r ”returns” the set B r back to R N ). Set A r is an open subset of R N − (by continuity of z r ). Taking advantage of separability of R N − we have the followingtheorem. Theorem 15. (Lindel˝of ’s lemma) For every open cover there is a countable subcover.
Therefore due to the above theorem, we can find a countable subcover for A , i.e., thereexists fixed-points r , r , . . . such that A = ∪ ∞ m =1 A r m .For a t ∈ N let ψ t, r ( v ) the point after t iteration of dynamics (4), starting with v , underprojection z r , i.e., ψ t, r ( v ) = z r ◦ f t ◦ z − r ( v ). If point ( v ) ∈ int g (∆ N ) (which corresponds to g − ( v ) in our original ∆ N ) has as unstable fixed point as a limit, there must exist a t and m so that ψ t, r m ◦ z r m ◦ g − ( v ) ∈ B r m for all t ≥ t (we have point-wise convergence from theorem4) and therefore again from Theorem 14 we get that ψ t , r m ◦ z r m ◦ g − ( v ) ∈ W scloc ( r m ), hence v ∈ g ◦ z − r m ◦ ψ − t , r m ( W scloc ( r m )).Hence the set of points in int g (∆ N ) whose ω -limit has an unstable equilibrium is a subsetof C = ∪ ∞ m =1 ∪ ∞ t =1 g ◦ z − r m ◦ ψ − t, r m ( W scloc ( r m )) (10)Since r m is unstable corresponding dim ( E u ) ≥
1, and therefore dimension of W scloc ( r m ) is atmost N −
3. Thus, the manifold W scloc ( r m ) has Lebesgue measure zero in R N − . Finally since g ◦ z − r m ◦ ψ − t, r m : R N − → R N − is continuously differentiable, ψ t, r m is C and locally Lipschitz Jacobian of h evaluated at p C is a countable union of measure zero sets, i.e., is measure zero as well, and Theorem 12follows. Lemma 16.
Let g : R m → R m be a locally Lipschitz function, then g is null-set preserving,i.e., for E ⊂ R m if E has measure zero then g ( E ) has also measure zero. Proof
Let B γ be an open ball such that || g ( y ) − g ( x ) || ≤ K γ || y − x || for all x , y ∈ B γ . Weconsider the union ∪ γ B γ which cover R m by the assumption that g is locally Lipschitz. ByLindel˝of’s lemma we have a countable subcover, i.e., ∪ ∞ i =1 B i . Let E i = E ∩ B i . We will provethat g ( E i ) has measure zero. Fix an (cid:15) >
0. Since E i ⊂ E , we have that E i has measure zero,hence we can find a countable cover of open balls C , C , ... for E i , namely E i ⊂ ∪ ∞ j =1 C j so that C j ⊂ B i for all j and also (cid:80) ∞ j =1 µ ( C j ) < (cid:15)K mi . Since E i ⊂ ∪ ∞ j =1 C j we get that g ( E i ) ⊂ ∪ ∞ j =1 g ( C j ),namely g ( C ) , g ( C ) , ... cover g ( E i ) and also g ( C j ) ⊂ g ( B i ) for all j . Assuming that ball C j ≡ B ( x , r ) (center x and radius r ) then it is clear that g ( C j ) ⊂ B ( g ( x ) , K i r ) ( g maps the center x to g ( x ) and the radius r to K i r because of Lipschitz assumption). But µ ( B ( g ( x ) , K i r )) = K mi µ ( B ( x , r )) = K mi µ ( C j ), therefore µ ( g ( C j )) ≤ K mi µ ( C j ) and so we conclude that µ ( g ( E i )) ≤ ∞ (cid:88) j =1 µ ( g ( C j )) ≤ K mi ∞ (cid:88) j =1 µ ( C j ) < (cid:15) Since (cid:15) was arbitrary, it follows that µ ( g ( E i )) = 0. To finish the proof, observe that g ( E ) = ∪ ∞ i =1 g ( E i ) therefore µ ( g ( E )) ≤ (cid:80) ∞ i =1 µ ( g ( E i )) = 0. B Figure of stable/unstable manifolds in simple example
The figure 1 corresponds to a two agent coordination game with payoff structure B = (cid:20) (cid:21) . Since this game has two agents with two strategies each, in order to capture thestate space of game it suffices to describe one number for each agent, namely the probabilitywith which he will play his first strategy. This game has three Nash equilibria, two pure ones(0 , , (1 ,
1) and a mixed one ( , ). We depict them using small circles in the figure. Themixed equilibrium has a stable manifold of zero measure that we depict with a black line. Incontrast, each pure Nash equilibrium has region of attraction of positive measure. The stablemanifold of the mixed NE separates the regions of attraction of the two pure equilibria. The(0 ,
0) equilibrium has larger region of attraction, represented by darker region in the figure. Itis the risk dominant equilibrium of the game. Recently, in [23] techniques have been developedto compute such objects (stable manifolds, volumes of region of attraction) analytically.
C Discussion
Building on the observation of [6] that the process of natural selection under weak-selectionregime can be modeled as discrete Multiplicative weight update dynamics on coordinationgames, we showed that it converges to pure NE almost always in the case of two-player games.As a consequence natural selection alone seem to lead to extinction of genetic diversity in thelong term limit, a widely believed conjecture of haploid genetics [4]. Thus, the long termpreservation of genetic diversity must be safeguarded by evolutionary mechanisms which areorthogonal to natural selection such as mutations and speciation. This calls for modeling andstudy of these latter phenomenon in game theoretic terms under discrete replicator dynamics.Additionally below we observe that in some special cases, ( i ) the rate of convergence ofdiscrete replicator dynamics is doubly exponentially fast in some special cases, and ( ii ) the15igure 1: Regions of attraction for B = [1 0; 0 3], where ◦ correspond to NE points.expected fitness of the resulting population, starting with a random distribution, under suchdynamics is constant factor away from the optimum fitness. It will be interesting to get similarresults for the general case of two-player coordination games. Rate of Convergence.
Let’s consider a special case where B is a square diagonal matrix.In that case, starting from any point ( x (0) , y (0)) observe that after one time step, we get that x (1) = y (1) (i.e f ( x (0)) = f ( y (0))). Therefore without loss of generality let us assume that x (0) = y (0). Then both the players get the same payoff from each of their pure strategies inthe first play as B = B T . And thus it follows that f n ( x (0)) = f n ( y (0)) for all n ≥
1. Let U i ( t )be the payoff that both gets from their i th strategy at time t (both will get the same payoff).Suppose for i (cid:54) = j we have U i (0) = cU j (0), then U i ( t ) U j ( t ) = (cid:18) B ii x i ( t − B jj x j ( t − (cid:19) = (cid:18) U i ( t − U j ( t − (cid:19) = (cid:18) U i (0) U j (0) (cid:19) t = c t Thus the ratio between payoffs from each pure strategy increases doubly exponentially, andthe next lemma follows.
Lemma 17. If z = min j U i ∗ (0) U j (0) where i ∗ ∈ arg max k U k (0) , we get that after O (log log z(cid:15) ) weare (cid:15) -close to a Nash equilibrium with support arg max k U k (0) (in terms of the total variationdistance). Average Price of Anarchy (APoA)
Following the work of [23] we can compute the average price of anarchy (APoA) for thefollowing case, where w > B = (cid:20) w (cid:21) Average Price of Anarchy (APoA) is defined w.r.t. a dynamics when the starting pointis picked uniformly at random from ∆ m × ∆ n . Dynamics under different starting points mayconverge to different NE. Let expected NE social welfare be the expected social welfare (SW) atthe Nash equilibrium to which dynamics may converge, then AP oA = Optimal SWExpected NE SW .Since both the players have only two strategies, probability of the first strategy is enough todescribe a profile. So let ( x, y ) denote the probabilities with which both plays first strategy, i.e.,( x , y ). Our game has three NE: (1 , , (0 ,
0) and ( w , w w ). Since the set of starting pointconverging to the mixed NE has measure zero (Theorem 13), we can ignore it. If ( x (0) , y (0)) =16 x, y ) is picked at random from [0 , then let A denote the area starting from where thedynamics (4) converges to (0 ,
0) where the SW is 2 w . Then, AP oA = w (2 w ∗ A )+2(1 − A ) = wwA +1 − A .Next we compute A .As discussed above after first step strategies of both the players are same, and there after if U (1) > U (1) then dynamics will converge to (0 , U (1) > U (1) ⇔ w (1 − x )(1 − y ) > xy ⇔ y < w (1 − x ) x (1 − w ) + w Thus A is the area under the curve y = w (1 − x ) w (1 − x )+ x , which is A = (cid:90) w (1 − x ) w (1 − x ) + x dx = [ w xw − w − w + w ) 11 − w ln( w + x (1 − w ))] = w − w − w ln w ( w − Replacing A in AP oA = w ( w ∗ A )+(1 − A ) , and setting its differentiation w.r.t. w to zero gives, w ln w (2 w + w + 1) = ( w − w + 3 w − w − w where APoA is maximum, and it turns out to bearound w = 2 .
02. APoA for w = 2 .
02 is 1 . Lemma 18.
For the class of coordination games ( B, B ) where B = (cid:20) w (cid:21) and w > , theAPoA is at most . under discrete replicator dynamics.under discrete replicator dynamics.