Evolutionary graph theory revisited: general dynamics and the Moran process
aa r X i v : . [ q - b i o . P E ] M a y Evolutionary graph theory revisited: general dynamicsand the Moran process
Karan Pattni ∗ , Mark Broom ∗ , Jan Rycht´aˇr † andLara J. Silvers ∗ October 9, 2018
Abstract
Evolution in finite populations is often modelled using the classical Moran pro-cess. Over the last ten years this methodology has been extended to structuredpopulations using evolutionary graph theory. An important question in any suchpopulation, is whether a rare mutant has a higher or lower chance of fixating (thefixation probability) than the Moran probability, i.e. that from the original Moranmodel, which represents an unstructured population. As evolutionary graph theoryhas developed, different ways of considering the interactions between individualsthrough a graph and an associated matrix of weights have been considered, as havea number of important dynamics. In this paper we revisit the original paper on evo-lutionary graph theory in light of these extensions to consider these developmentsin an integrated way. In particular we find general criteria for when an evolutionarygraph with general weights satisfies the Moran probability for the set of six commonevolutionary dynamics.
When modelling population evolution we are concerned with the spread of heritablecharacteristics in successive generations. The type of model that is used depends uponwhether the population size is assumed to be finite or infinite. The majority of classicalevolutionary models (see for example [1, 2]) use infinite populations, although finitepopulation models are also well estiablished, the most important models being thosein [3, 4]. These models are stochastic, and are solved using classical Markov chainmethodology [5, 6, 7]. See also [8, 9] for an extension to evolutionary games in finitepopulations.The populations in the models described above, however, were “well-mixed”, i.e.every individual was equally likely to encounter every other individual. Real populationsof course contain structural elements, such as geographical location or social relationship,which mean that some pairs individuals are more likely to interact than others. In suchcircumstances we need to be able to identify distinct individuals (or at least distinctclasses of individuals), and considering finite populations is perhaps more natural thaninfinite ones (although finite structures each containing an infinite number of individuals,so called “island models”, were considered in [10]). In [11] the modelling ideas of [3]were extended to consider such structured populations based upon graphs, known asevolutionary graph theory. This has proved very successful, spawning a large number ofpapers (for example [12, 13, 14, 15, 16, 17, 18, 19]). For informative reviews see [20, 21].In an evolving population, we need to consider the mechanism of how the populationchanges, called the dynamics. Informally, the dynamics specify the way in which herita-ble characteristics are passed on from one generation to the next. For infinite populations ∗ Department of Mathematics, City University London, Northampton Square, London, EC1V 0HB,UK † Department of Mathematics and Statistics, The University of North Carolina at Greensboro, Greens-boro, NC 27412, USA
We shall first describe the population model of [11], which generalises the model of [3] byincorporating a replacement structure. The notation used in this paper is summarisedin Table 1.
The population has a constant size N ∈ Z , N ≥ , consisting of individuals I , . . . , I N .Every individual is either of type A or B . This implies that there are 2 N different states of the population given by the combi-nation of type A and B individuals. We represent each state by a set S such that n ∈ S if an individual I n is of type A . We can easily revert to using the number of type A individuals, | S | , if the population is homogeneous. The states ∅ and N = { , , . . . , N } have only type B and A individuals respectively. Individuals have a constant fitness that may depend upon their type.
The fitness of individuals in state S is thus given by the vector F ( S ) = ( F n ( S )) n =1 , ,...,N where F n ( S ) = ( n / ∈ S,r ∈ (0 , ∞ ) n ∈ S, is the fitness of I n . Here the fitness r of a type A individual is given relative to the fitnessof a type B individual assumed to be 1. During a stochastic replacement event (that happens in an instant) an exact copy of anindividual I i replaces an individual I j . The replacement events may be restricted in the sense that not all individuals canreplace one another. To enforce such restrictions, [11] imposed a replacement structureusing a weighted directed graph given by the tuple (
D, w ) where D = ( V, E ) is a directedgraph, with sets V of vertices and E of directed edges, and w is a map that assigns aweight to each edge such that w : V × V → [0 , ∞ ) : ( i, j ) w ij . Each vertex n ∈ V represents I n therefore V = { , , . . . , N } so | V | = N . We assume that ( i, j ) ∈ E if andonly if w ij >
0, which indicates that I i can replace I j . Note that we allow w ii > I i can replace itself. All the information contained within the weighted digraph( D, w ) is conveniently summarised by the N × N weighted adjacency matrix W = ( w ij )and therefore we will refer to ( D, w ) using W , which we call the replacement matrix .The replacement events are stochastic which means that there is a probability r ij = r ij ( F ( S ) , W ) associated with (a copy of) I i replacing I j . There are several potential evolutionary dynamics on graphs that govern how the probability is determined. Therethree main types of dynamics that are summarised below, see also [21]. We use theconvention that I i is chosen for birth and I j is chosen for death.1. Birth-Death (BD): I i is chosen first then I j . We have that i ∈ V is chosen withprobability b i and then ( i, j ) ∈ E i is chosen with probability d ij , where E i are all2 ummary of Notation Symbol Definition Description N ∈ Z + \ { , } Population size.
A, B
The two types of individuals in population. I n Individual n . S = { n : I n of type A } State of the population. N = { , , . . . , N } State in which all I n of type A . r ∈ (0 , ∞ ) Fitness of a type A individual. F n ( S ) ∈ { , r } Fitness of I n in state S . D = ( V, E ) Replacement digraph with vertices V where | V | = N and directed edges E . w ij ∈ [0 , ∞ ) Edge weight such that w ij > i, j ) ∈ E . W = ( w ij ) Replacement matrix: N × N weighted adjacency ma-trix of tuple ( D, w ). T + n = P Nj =1 w nj Out temperature:
Sum of all outgoing edge weightsof vertex n ∈ V . T − n = P Ni =1 w in In temperature : Sum of all incoming edge weights ofvertex n ∈ V . b i ∈ [0 ,
1] Probability I i chosen for birth. d ij ∈ [0 ,
1] Probability a copy of I i replaces I j given I i was cho-sen for birth, i.e. replacement by death. d j ∈ [0 ,
1] Probability I j chosen for death. b ij ∈ [0 ,
1] Probability a copy of I i replaces I j given I j is chosenfor death, i.e. replacement by birth. r ij ∈ [0 ,
1] Probability a copy of I i replaces I j . P SS ′ ∈ [0 ,
1] State transition probability. S = ( P SS ′ ) State transition matrix. E ∗ , W ,r Stochastic process with state transition matrix S such that ∗ dynamics are used on graph W and type A individuals have fitness r . ρ AS ∈ [0 ,
1] Fixation probability of type A individual given initialstate S . W Set of all strongly connected replacement matrices. W C { W : T + n = T − n ∀ n } Replacement matrices that are circulations. W I { W : T + i = T − j ∀ i, j } Replacement matrices that are isothermal. W R { W : T + n = 1 ∀ n } Right stochastic replacement matrices. W L { W : T − n = 1 ∀ n } Left stochastic replacement matrices. C N Replacement matrices whose digraphs are cycles oflength N . f R ( w ij ) ( w ij / P n w in ) Map from W to W R . f L ( w ij ) ( w ij / P n w nj ) Map from W to W L . f ′ ( w ij ) ( w ij / P n,k w nk ) Map from W to W . M ∗ Replacement matrices for which E ∗ is ρ -equivalent toa Moran process when ∗ dynamics are used.Table 1: Notation used in this paper.edges starting in vertex i . d ij is used to signify that there is ‘replacement by death’.Finally, r ij = b i d ij .2. Death-Birth (DB): I j is chosen first then I i . We have that j ∈ V is chosen withprobability d j and then ( i, j ) ∈ E j is chosen with probability b ij , where E j are alledges ending in vertex j . b ij is used to signify that there is ‘replacement by birth’.Finally, r ij = d i b ij .3. Link (L): I i and I j are chosen simultaneously. In this case ( i, j ) ∈ E is simplychosen with probability r ij .For each type of these dynamics, the natural selection can, through the fitness parameter,3rocess P ( I i replaces I j ) Order chosen P (Chosen first) P (Chosen second)BDB [11] r ij = b i d ij I i then I j b i = F i ( S ) X n F n ( S ) d ij = w ij X n w in BDD [26] r ij = b i d ij I i then I j b i = 1 N d ij = w ij /F j ( S ) X n w in /F n ( S )DBD [13] r ij = d i b ij I j then I i d j = 1 /F j ( S ) X n /F n ( S ) b ij = w ij X n w nj DBB [16] r ij = d i b ij I j then I i d j = 1 N b ij = w ij F i ( S ) X n w nj F n ( S )LB [11] r ij = w ij F i ( S ) X n,k w nk F n ( S ) Simultaneous N/A N/ALD [23] r ij = w ij /F j ( S ) X n,k w nk /F k ( S ) Simultaneous N/A N/ATable 2: List of the dynamics used in this paper. Note that L will be used in place ofLB and LD where appropriate.influence either the choice at birth (resulting in adding “B”) or at death (adding “D”).It yields 6 kinds of evolutionary dynamics on graphs summarized in Table 2. Thesedynamics have been extensively studied, in particular, see [26] for a detailed comparisonof them. Of these, the BDB and LB dynamics were used in [11]. The fixation probability, ρ AS = ρ AS ( ∗ , W , r ), is the probability that the population withinitial state S is absorbed in N where ∗ is the dynamics being used.Given that the replacement events are random, the transitions between the states ofthe population are described by a stochastic process, which we denote E . The propertiesof E can be investigated once the state transition probabilities of moving from state S to S ′ , P SS ′ = P SS ′ ( ∗ , W , r ), are calculated using the replacement probabilities as follows: P SS ′ = X i/ ∈ S r ij ( F ( S ) , W ) if S ′ = S \ { j } for some j ∈ S, X i ∈ S r ij ( F ( S ) , W ) if S ′ = S ∪ { j } for some j / ∈ S, X i,j ∈ S ∨ i,j / ∈ S r ij ( F ( S ) , W ) if S ′ = S. The transition probabilities, P SS ′ , satisfy the Markov property because they only dependupon the state S , that is, the probability of transitioning from the present state to anotherstate is independent of any past and future state of the population. The stochastic process E ∗ , W ,r with state transition matrix S = S ( ∗ , W , r ) = ( P SS ′ ) S,S ′ ⊂{ , ,...,N } is thereforea Markov chain. The Markov chain E ∗ , W ,r is part of the class of evolutionary Markovchains described in [27].The absorbing states of E ∗ , W ,r are ∅ , N , which means that if the population is ineither one of these states then it remains there indefinitely. This property of E ∗ , W ,r canbe used to measure the success of a type A individual by calculating the probability thatit fixates, that is, everyone in the population is of type A . The fixation probability isthen given by solving ρ AS = X S ′ ⊂{ , ,...,N } P SS ′ ρ AS ′ (1)4ith boundary conditions ρ A ∅ = 0 and ρ A N = 1.As demonstrated in [26], LB and LD dynamics may differ in time scale but they yieldthe same fixation probabilities when fitness is constant (which is our case). Thus, for ourpurposes the dynamics are the same and we will thus consider them together and denotethem by L. The Moran process [3], a stochastic birth-death process on finite fixed homogenous pop-ulation, can be reconstructed as E BDB , W H ,r for a constant replacement matrix W H = (1 /N ) i,j . (2)For any r ∈ (0 , ∞ ) and any S ⊂ { , . . . , N } , the fixation probability for this process, or Moran probability , is given by ρ AS = − r −| S | − r − N if r = 1 , | S | /N if r = 1 . We are interested in characterizing graphs (and evolutionary dynamics) that yield thesame fixation probabilities as the homogeneous matrix W H given in (2). We note that forthis matrix all of the transition probabilities r ij take the same value independent of i, j or the dynamics, and consequently the fixation probability under each of the dynamicsis the same. The set of all admissible replacement matrices is defined as follows W = { W : for every i, j , there is n such that ( W n ) i,j > } . This definition means that W is strongly connected as for any pair of vertices i and j ,there is a path (of length n ) going from i to j . Unless specified otherwise, we will consideradmissible replacement matrices only.As in [11], for any W (admissible or not) we define the in temperature of I n , T − n , andthe out temperature of I n , T + n , by T − n = N X j =1 w jn and T + n = N X j =1 w nj . W is called a circulation if T + n = T − n , for all n ∈ V and it is called isothermal if T + i = T − j , for all i, j ∈ V . W is called right stochastic if T + n = 1, for all n ∈ V and itis called left stochastic if T − n = 1, for all n ∈ V . The sets of all circulations, isothermalmatrices, right stochastic matrices, and left stochastic matrices, respectively are denotedby W C , W I , W R , and W L respectively.The set C N denotes the sets of matrices representing cycles of length N , more specif-ically, for ( w ij ) ∈ C N we have w ii = 1 / i = 1 , , . . . N , w i i = · · · = w i n i n +1 = · · · = w i N − i N = w i N i = 1 / i , i , . . . , i N of the sequence 1 , , . . . , N ,and w ij = 0 otherwise.We also define the maps f R : W → W R , f L : W → W L , and f ′ : W → W respectively,by f R (( w ij )) = (cid:18) w ij P n w in (cid:19) , f L (( w ij )) = (cid:18) w ij P n w nj (cid:19) , and f ′ (( w ij )) = w ij P n,k w nk ! . Note that f R preserves right stochastic matrices and f L preserves left stochastic matrices.Moreover, f R ( W ) = f L ( W ) for all W ∈ W I . Also, since f ′ simply involves multiplying W by the constant 1 / P n,k w nk , it implies that W ∈ W C ⇔ f ′ ( W ) ∈ W C .5hen the dynamics ∗ , matrices W and W , and fitness r are given, we say that anevolutionary Markov chain E ∗ , W ,r is ρ -equivalent to E ∗ , W ,r if for every S ⊂ { , . . . , N } , ρ AS ( ∗ , W , r ) = ρ AS ( ∗ , W , r ), in which case we write W ∼ ∗ ,r W .We are specifically interested in finding matrices equivalent to the Moran process.For a dynamics ∗ , we define M ∗ = { W : W ∼ ∗ ,r W H for all r > } . The map f R preserves the equivalence classes of BDB and BDD dynamics, f L preservesthe equivalence classes of DBB and DBD dynamics and f ′ preserves the equivalenceclasses for link dynamics. Specifically, as one can see from the proofs in the Appendix,for any W and any r > W ∼ BDB ,r f R ( W ) , (3) W ∼ BDD ,r f R ( W ) , W ∼ DBB ,r f L ( W ) , W ∼ DBD ,r f L ( W ) , W ∼ L ,r f ′ ( W ) . We thus obtain the following results, which completely specify the graphs which areequivalent to the homogeneous matrix W H for each of our evolutionary dynamics. Proposition 1 (Link) . M L = W C . More precisely, the following statements are equiv-alent:(a) W is a circulation.(b) For all r > , W ∼ L,r W H .(c) There is r > such that W ∼ L,r W H . We note that W C = f ′− ( W C ) = { W ; f ′ ( W ) ∈ W C } and thus, similarly to Proposi-tion 2 below, Proposition 1 can be written as M L = f ′− ( W C ). Proposition 2 (BDB and DBD) . M BDB = f − R ( W C ) and M DBD = f − L ( W C ) . Moreprecisely, the following statements are equivalent:(a) f R ( W ) is a circulation.(b) For all r > , W ∼ BDB ,r W H .(c) There is r > such that W ∼ BDB ,r W H The equivalent conditions for DBD are similar to the above for BDB but f R is replacedby f L . Proposition 3 (BDD and DBB) . M BDB = f − R ( { W H }∪ C N ) and M DBB = f − L ( { W H }∪ C N ) . More precisely, the following statements are equivalent:(a) f R ( W ) = W H or f R ( W ) ∈ C N .(b) For all r > , W ∼ BDD ,r W H .The equivalent conditions for DBB are similar to the above for BDD but f R is replacedby f L . In particular, M BDD ⊂ M BDB and M DBB ⊂ M DBD . The sets M ∗ are illustrated in Figure1. Note that unlike in Propositions 1 and 2, Proposition 3 does not contain “any r implies all r ”. In fact, when r = 1, there is no selection and thus the dynamics BDB andBDD are the same (and also the dynamics DBB and DBD are the same). Consequently,by Proposition 2, W ∼ BDD , W H ⇔ f R ( W ) ∈ W C ⇔ W ∈ M BDB , W ∼ DBB , W H ⇔ f L ( W ) ∈ W C ⇔ W ∈ M DBD . W W W W W W W W W = W I ∩ f − R ( { W H } ∪ C N )= W I ∩ f − L ( { W H } ∪ C N ) W = W I \ f − R ( { W H } ∪ C N )= W I \ f − L ( { W H } ∪ C N ) W = W C \ W I W = (cid:0) f − R ( W C ) \ W C (cid:1) ∩ f − R ( { W H } ∪ C N ) W = (cid:0) f − R ( W C ) \ W C (cid:1) \ f − R ( { W H } ∪ C N ) W = (cid:0) f − L ( W C ) \ W C (cid:1) ∩ f − L ( { W H } ∪ C N ) W = (cid:0) f − L ( W C ) \ W C (cid:1) \ f − L ( { W H } ∪ C N ) W = W \ S i =1 W i M L ≡ W C W W W W W W W W W M BDB
W W W W W W W W W M BDD
W W W W W W W W W M DBD
W W W W W W W W W M DBB
W W W W W W W W W M BDD ∩ M DBB
W W W W W W W W W Figure 1: The diagram on top shows eight partitions W i of W (labelled using their index i = 1 , , . . . , M ∗ . Below this, aseparate diagram for each of the standard dynamics is given showing the partitions thatmake up M ∗ . The bottom right diagram shows the partition where E is ρ -equivalent toa Moran process regardless of the standard dynamics on the graph being used, that is, M L ∩ M BDB ∩ M BDD ∩ M DBD ∩ M DBB ≡ M BDD ∩ M DBB . For the LB dynamics, Proposition 1 was stated and proved in [11] as the Circulationtheorem. For the LD dynamics, Proposition 1 follows from the Circulation theorem andthe result of [26] that the fixation probabilities for LB and LD are the same.As shown in Appendix A.1, BDB is the same as the LB dynamics for right stochasticmatrices (in particular, for BDB dynamics, Proposition 2 can be seen as the Isothermaltheorem from [11]). Proposition 2 thus follows from Proposition 1 thanks to (3). Thenatural symmetries between f R and f L and BDB and DBD dynamics allow us to extendthe Isothermal theorem to DBD dynamics as well (see also [28]).Overall, Propositions 1 and 2 and the occurrence of W C within them are consis-tent with the claim made in [11] that the circulation criterion completely classifies allreplacement matrices where E ∗ , W ,r is ρ -equivalent to a Moran process.Our most important new result is Proposition 3. It shows that the BDD and DBBdynamics require very strict conditions to yield the Moran process. Either the populationstructure is homogeneous, or it is a directed cycle. This latter structure is an interestingtheoretical example, but is unlikely to apply to real populations, meaning that the ho-mogeneous population is practically the only way to get the Moran process for a realisticpopulation. 7 .2 The importance of self-loops in BDD and DBB dynamics Proposition 3 by definition requires that w ii > ∀ i = 1 , , . . . , N . Without such self-loops, E BDD , W ,r , E DBB , W ,r cannot ever be ρ -equivalent to the Moran process. The abilityof an individual to replace itself therefore plays an important role in the replacementstructure of the population and cannot be discounted. For BD dynamics, when increasingthe diagonal weights of W , the fixation probability decreases for BDB and increases forBDD. For DB dynamics, the increase in fixation probability DBB is greater than thatfor DBD. For LB dynamics, the fixation probability remains the same.With BDD and DBD evolutionary dynamics on graphs one may encounter the fol-lowing problems if there are no self-loops [29, page 245]. For DBB dynamics, a type Aindividual with almost infinite fitness still has a fixation probability bounded away from1 because even type A individuals can be randomly picked for death and replaced bytype B individuals. With self-loops, however, a type A individual will almost always bereplaced by itself (or another type A individual) and therefore has a fixation probabilityapproaching 1. Similarly, for BDD dynamics, a type A individual with almost zero fitnessdoes not have near probability 0 of fixating as type A individuals can be randomly pickedfor birth and replace type B individuals. With self-loops, such an individual will almostalways pick itself (or another type A) to replace and therefore its fixation probability isnear 0. Thus the inclusion of self-loops removes some problematic features of the BDDand DBB dynamics, and makes them more attractive dynamics to use in models. In this paper we have considered an evolutionary graph theory model of a populationinvolving general weights and a variety of evolutionary dynamics based upon the workof [11], which was a development of the classical population model of [3]. In such pop-ulations, the population size is fixed at all times and at successive discrete time pointsone replacement event occurs. Like the aforementioned papers we consider two typesof individuals, where fitness depends upon type but no other factors (i.e. there are nogame-theoretic interactions). In particular the single most important property of such aprocess is the fixation probability, the probability that a randomly placed mutant indi-vidual of the second type will eventually completely replace the population of the firsttype.This fixation probability depends upon the fitnesses of the two types of individuals,but can also be heavily influenced by the population structure as given by the weights,and by the evolutionary dynamics used. These effects are commonly observed, althoughin some circumstances evolution proceeds as if as on a well-mixed population as fromthe original work of [3], dependent only upon the fitnesses of the two types, and someimportant results in this regard were already given in [11]. The aim of this paper was toprovide a generalised set of conditions for when this would be the case.By defining what is meant by fixation-equivalence to the Moran process, we provided ageneral result which, independent of the specific dynamics used, helps identify graphs thatdo not affect the fixation probability. With respect to each of the standard dynamics,we then classified sets of evolutionary graphs that have the same fixation probabilityas the Moran process (or well mixed population). These sets include graphs that arecirculations and therefore generalises the work of [11].An important new result shows that the set of weights for which we obtain fixationequivalence to the Moran process for the BDD and DBB dynamics is very restricted, andso that for most populations with any structure this equivalence will not hold for thesedynamics. We note also that the inclusion of non-zero self weights w ii eliminates someproblematic features of these two dynamics (i.e. that individuals with 0 fitness couldfixate or those with infinite fitness could be eliminated) and so improves the applicabilityof these dynamics.Presenting evolutionary dynamics on graphs in the way that we have allows one toincorporate a variety of dynamics in their analysis, both of standard type and otherdefinitions. This will improve our understanding of dynamics on graphs in general. Wenote that the list of dynamics in Table 2 is not exhaustive. For example, [30] used8mitation dynamics, which is a class of DBB dynamics with an additional requirement w ii > ∀ i .In general the inclusion of non-zero self weights, in contrast to many earlier evolution-ary graph theory works, allows for a greater flexibility of modelling. We note that thisis consistent with the original work of [3], which allowed self-replacement as an integralpart of the process. For well-mixed populations it does not matter much whether thispossibility is included or not (at least for sufficiently large populations with intermediatefitness values), and it is likely that it has often been excluded for reasons of conveniencebecause of this without the ramifications being fully considered in many later works. Itis thus important to consider whether to include such self weights when modelling spatialstructure using evolutionary graph theory. References [1] Maynard Smith J. Evolution and the Theory of Games. Cambridge university press;1982.[2] Hofbauer J, Sigmund K. Evolutionary games and population dynamics. CambridgeUniversity Press; 1998.[3] Moran PAP. Random processes in genetics. In: Mathematical Proceedings of theCambridge Philosophical Society. vol. 54. Cambridge Univ Press; 1958. p. 60–71.[4] Moran PAP. The statistical processes of evolutionary theory. Clarendon Press,Oxford; 1962.[5] Karlin S, Taylor HM. A first course in stochastic processes. London, AcademicPress; 1975.[6] Landauer R, Buttiker M. Diffusive traversal time: Effective area in magneticallyinduced interference. Physical Review B. 1987;36(12):6255.[7] Antal T, Scheuring I. Fixation of strategies for an evolutionary game in finitepopulations. Bulletin of Mathematical Biology. 2006;68(8):1923–1944.[8] Taylor C, Fudenberg D, Sasaki A, Nowak MA. Evolutionary game dynamics in finitepopulations. Bulletin of Mathematical Biology. 2004;66(6):1621–1644.[9] Nowak MA, Sasaki A, Taylor C, Fudenberg D. Emergence of cooperation and evo-lutionary stability in finite populations. Nature. 2004;428(6983):646–650.[10] Maruyama T. Effective number of alleles in a subdivided population. TheoreticalPopulation Biology. 1970;1(3):273–306.[11] Lieberman E, Hauert C, Nowak MA. Evolutionary dynamics on graphs. Nature.2005;433(7023):312–316.[12] Allen B, Traulsen A, Tarnita CE, Nowak MA. How mutation affects evolutionarygames on graphs. Journal of Theoretical Biology. 2012;299:97–105.[13] Antal T, Redner S, Sood V. Evolutionary dynamics on degree-heterogeneous graphs.Physical Review Letters. 2006;96(18):188104.[14] Broom M, Rycht´aˇr J. An analysis of the fixation probability of a mutant on specialclasses of non-directed graphs. Proceedings of the Royal Society A: Mathematical,Physical and Engineering Science. 2008;464(2098):2609–2627.[15] Broom M, Rycht´aˇr J, Stadler BT. Evolutionary dynamics on graphs-the effectof graph structure and initial placement on mutant spread. Journal of StatisticalTheory and Practice. 2011;5(3):369–381.[16] Ohtsuki H, Hauert C, Lieberman E, Nowak MA. A simple rule for the evolution ofcooperation on graphs and social networks. Nature. 2006;441(7092):502–505.917] Ohtsuki H, Nowak MA, Pacheco JM. Breaking the symmetry between interactionand replacement in evolutionary dynamics on graphs. Physical Review Letters.2007;98(10):108106.[18] Shakarian P, Roos P. Fast and deterministic computation of fixation probability inevolutionary graphs. DTIC Document; 2012.[19] Voorhees B, Murray A. Fixation probabilities for simple digraphs. Proceedings of theRoyal Society A: Mathematical, Physical and Engineering Science. 2013;469(2154).[20] Allen B, Nowak MA. Games on graphs. EMS Surveys in Mathematical Sciences.2014;1(1):113–151.[21] Shakarian P, Roos P, Johnson A. A review of evolutionary graph theory with ap-plications to game theory. Biosystems. 2012;107(2):66–80.[22] Taylor PD, Jonker LB. Evolutionary stable strategies and game dynamics. Mathe-matical Biosciences. 1978;40(1):145–156.[23] Masuda N, Ohtsuki H. Evolutionary dynamics and fixation probabilities in directednetworks. New Journal of Physics. 2009;11(3).[24] Hadjichrysanthou C, Broom M, Rycht´aˇr J. Evolutionary games on star graphs undervarious updating rules. Dynamic Games and Applications. 2011;1(3):386–407.[25] Slatkin M. Fixation probabilities and fixation times in a subdivided population.Evolution. 1981;p. 477–488.[26] Masuda N. Directionality of contact networks suppresses selection pressure in evo-lutionary dynamics. Journal of Theoretical Biology. 2009;258(2):323–334.[27] Allen B, Tarnita CE. Measures of success in a class of evolutionary models withfixed population size and structure. Journal of Mathematical Biology. 2014;68(1-2):109–143.[28] Kaveh K, Komarova NL, Kohandel M. The duality of spatial death–birth andbirth–death processes and limitations of the isothermal theorem. Royal SocietyOpen Science. 2015;2(4).[29] Broom M, Rycht´aˇr J. Game-theoretical models in biology. CRC Press, Boca Raton,FL; 2013.[30] Ohtsuki H, Nowak MA. The replicator equation on graphs. Journal of TheoreticalBiology. 2006;243(1):86–97.[31] Kemeny J, Snell JL. Finite Markov Chains. Van Nostrand; 1960.
AppendixA Proofs
A.1 BDB is the same as LB for right stochastic matrices
For BDB dynamics we have r ij = b i d ij . By definition P ij b i d ij = 1, we can thereforewrite this as r ij = b i d ij .P n,k b n d n,k . Substituting b i = F i .P Nm =1 F m gives r ij = d ij F i .P Nm =1 F m P n,k (cid:16) d nk F n .P Nm =1 F m (cid:17) = d ij F i P n,k d nk F n . If W is right stochastic, i.e. P Nn =1 w in = 1 for all i = 1 , , . . . N , for BDB dynamicswe have that d ij = w ij .P Nn =1 w in = w ij giving r ij = w ij F i .P n,k w nk F n which is theLB dynamics as required. We also have that DBD is the same as LD for left stochasticmatrices. The explanation follows the same procedure as above.10 .2 Lemma 1 (Forward Bias) The key Lemma 1 stated below is used in the proofs of all propositions and it reliesheavily on the notion of forward bias of state S which is then given by the ratio of theprobabilities of a forward transition to a backward transition from S . A forward andbackward transition from S occurs when the number of type A individuals increase anddecrease by one respectively, which happen with probability P + S = X n/ ∈ S P S,S ∪{ n } and P − S = X n ∈ S P S,S \{ n } . Lemma 1 (Constant Forward Bias) . Let E be an evolutionary process on states S ⊂{ , , . . . , N } with transition probabilities P S,S ′ that satisfy • P S,S ′ > only if S and S ′ differ in at most one element • for every S = ∅ , { , . . . , N } , there are S + and S − such that | S + | = | S | + 1 and | S − | = | S | − and P S,S + > , P S,S − > .Then, the following are equivalenta) There is a constant c > such that for all S ⊂ { , , . . . , N } ρ AS = − c −| S | − c − N if c = 1 , | S | /N if c = 1 b) E has constant forward bias, that is, there is a constant d such that for all S ⊂{ , , . . . , N } P + S (cid:14) P − S = d. Moreover, if either (a) or (b) hold, then c = d . Note that a similar result is given in [11, 20] where the forward bias is explicitlydefined as r X a ∈ S X b/ ∈ S w ab ,X a ∈ S X b/ ∈ S w ba , which is what one gets when using Link dynamics, or BDB dynamics if W ∈ W R . Notethat in Lemma 1 the forward bias is defined independent of the dynamics and thereforeapplies to all dynamics that satisfy the assumptions. Proof. “(a) ⇒ (b)”: Take any S ⊂ { , , . . . , N } . It is known that ρ AS = X S ′ P S,S ′ ρ AS ′ = P S,S ρ AS + X n/ ∈ S (cid:16) P S,S ∪{ n } ρ AS ∪{ n } (cid:17) + X n ∈ S (cid:16) P S,S \{ n } ρ AS \{ n } (cid:17) and using P S,S = 1 − P + S − P − S gives0 = X n/ ∈ S (cid:16) P S,S ∪{ n } (cid:16) ρ AS ∪{ n } − ρ AS (cid:17)(cid:17) + X n ∈ S (cid:16) P S,S \{ n } (cid:16) ρ AS \{ n } − ρ AS (cid:17)(cid:17) . (4)For c = 1, equation (4) simplifies to0 = 1 − c −| S |− − c −| S | − c − N P + S + 1 − c −| S | +1 − c −| S | − c − N P − S ⇒ P + S (cid:14) P − S = c −| S | − c −| S | +1 c −| S |− − c −| S | = 1 − cc − − c. For c = 1, equation (4) simplifies to0 = ( | S | + 1 − | S | ) P + S + ( | S | − − | S | ) P − S ⇒ P + S (cid:14) P − S = 1 . ⇐ (a)”: The state transition matrix S = ( P S,S ′ ) can be scaled to give S ′ = ( P ′ S,S ′ )such that P ′ S,S = 0 and P ′ S,S ′ = P S,S ′ / (1 − P S,S ) = P S,S ′ / ( P + S + P − S ) where S is a non-absorbing state. The fixation probability ρ AS will be the same whether S ′ or S is used.This is because equation (1) can be rearranged as follows ρ AS = X S ′ P SS ′ ρ AS ′ ⇒ ρ AS = P SS ρ AS + X S ′ : S ′ = S P SS ′ ρ AS ′ ⇒ ρ AS (1 − P SS ) = X S ′ : S ′ = S P SS ′ ρ AS ′ ⇒ ρ AS = X S ′ : S ′ = S P SS ′ P + S + P − S ρ AS ′ . Let {S , S , . . . , S N } be a partition of the states S such that S ∈ S i if | S | = i . Theprobability P i,j ( S ) of transitioning from state S ∈ S i to lumped state S j with respect to S ′ is P i,j ( S ) = j = i ± , / ( d + 1) j = i − ,d/ ( d + 1) j = i + 1 for i = 1 , , . . . , N − . (5)This can be easily verified, for example, take j = i − P i,i − ( S ) = X S ′ ∈S i − P ′ S,S ′ = X S ′ ∈S i − P S,S ′ P + S + P − S = P − S P + S + P − S = 11 + d since the forward bias is equal to d . Equation (5) satisfies the necessary and sufficientcondition for the Markov chain with state transition matrix S ′ to be lumpable withrespect to the partition {S , S , . . . , S N } (Theorem 6.3.2 page 124, [31]). Let ˆ S = ( P i,j )be the state transition matrix for this lumped Markov chain then the probability P i,j oftransitioning from lumped states S i to S j is given by P i,j = P i,j ( S ) . The state transition matrix ˆ S describes a random walk with absorbing barriers andtherefore the probability ρ Ai of type A individuals fixating when the population starts inlumped state S i is calculated using the methods in [5] to give ρ Ai = 1 + i − X j =1 j Y k =1 P k,k − P k,k +1 , N − X j =1 j Y k =1 P k,k − P k,k +1 . In this case, ρ Ai = − d − i − d − N d = 1 ,i/N d = 1since P k,k − /P k,k +1 = 1 /r for k = 1 , , . . . , N −
1. By definition, ρ AS = ρ Ai where i = | S | as required. A.3 Proposition 1 (Link)
The following statements are equivalent:(a) W is a circulation.(b) For all r > W ∼ L ,r W H .(c) There is r > W ∼ L ,r W H . (d) For all r > S ⊂ { , , . . . , N } , the forward bias of E L , W ,r is r , i.e. P + S (cid:14) P − S = r. r > a ∈ { , , . . . , N } , the forward bias of the oneelement set S = { a } is r , i.e. X b = a P { a } , { a,b } P a, ∅ = r. Proof.
For LB dynamics the forward bias is given by P + S P − S = X a ∈ S X b/ ∈ S w ab F a X n,k w nk F n X a ∈ S X b/ ∈ S w ba F b X n,k w nk F n = r X a ∈ S X b/ ∈ S w ab X a ∈ S X b/ ∈ S w ba . For LD dynamics the forward bias is given by P + S P − S = X a ∈ S X b/ ∈ S w ab /F b X n,k w nk /F k X a ∈ S X b/ ∈ S w ba /F a X n,k w nk /F k = r X a ∈ S X b/ ∈ S w ab X a ∈ S X b/ ∈ S w ba . “(a) ⇒ (d)”: W is a circulation i.e. T + n = T − n for all n ∈ { , . . . , N } and thus X a ∈ S X b/ ∈ S w ab = X a ∈ S (cid:18) X n w an − X k ∈ S w ak (cid:19) = X a ∈ S (cid:18) T + a − X k ∈ S w ak (cid:19) ⇒ X a ∈ S X b/ ∈ S w ab = X a ∈ S (cid:18) T − a − X k ∈ S w ka (cid:19) = X a ∈ S (cid:18) X n w na − X k ∈ S w ka (cid:19) ⇒ X a ∈ S X b/ ∈ S w ab = X a ∈ S X b/ ∈ S w ba . Note that P a ∈ S P b/ ∈ S w ab = 0 because W is admissible and represents a strongly con-nected graph. Thus, the forward bias for both LB and LD is equal to r .“(d) ⇒ (e)” is trivial as (d) is much stronger than (e).“(e) ⇒ (a)” Let a and r is fixed. By above calculations of the forward bias, we have X b/ ∈ S = { a } w ab = X b/ ∈ S = { a } w ba ⇒ − w aa + N X i =1 w ai = − w aa + N X i =1 w ia ⇒ N X i =1 w ai = N X i =1 w ia therefore W is a circulation.“(d) ⇒ (b)” follows from Lemma 1.“(b) ⇒ (c)” is trivial.“(c) ⇒ (e)” follows from Lemma 1. A.4 Proposition 2 (BDB and DBD)
More precisely, the following statements are equivalent:(a) f R ( W ) is a circulation.(b) For all r > W ∼ BDB ,r W H .(c) There is r > W ∼ BDB ,r W H (d) For all r > S ⊂ { , , . . . , N } , the forward bias of E BDB , W ,r is r , i.e. P + S (cid:14) P − S = r. r > a ∈ { , , . . . , N } , the forward bias of E BDB , W ,r ofthe one element set S = { a } is r , i.e. X b = a P { a } , { a,b } P a, ∅ = r. Proof.
Let U = ( u ij ) = f R ( W ) = ( w ij / P n w in ) then for BDB dynamics the forwardbias of E BDB, W ,r is given by P + S P − S = X a ∈ S X b/ ∈ S F a X n F n w ab X n w an X a ∈ S X b/ ∈ S F b X n F n w ba X n w bn = r X a ∈ S X b/ ∈ S u ab X b/ ∈ S X a ∈ S u ba and therefore the forward bias of E BDB , W ,r is the same as forward bias of E BDB , U ,r .Similarly, with almost identical working as above, when V = f L ( W ), the forwardbias of E DBD , W ,r is the same as forward bias of E DBD , V ,r and is given by P + S P − S = X a ∈ S X b/ ∈ S /F b X n /F n w ab X n w nb X a ∈ S X b/ ∈ S /F a X n /F n w ba X n w na = X a ∈ S X b/ ∈ S v ab r X a ∈ S X b/ ∈ S v ba . and the proof of the Proposition for DBD closely follows the one for BDB given belowwith U and f R appropriately replaced by V and f L .“(a) ⇒ (d)”: If U = f R ( W ) ∈ W C , i.e. if U is doubly stochastic, then the forwardbias (for S = ∅ , N ) is equal to P + S P − S = r X a ∈ S (cid:18) X n ( u an ) − X k ∈ S ( u ak ) (cid:19)X a ∈ S (cid:18) X n ( u na ) − X k ∈ S ( u ka ) (cid:19) = r (cid:18) | S | − X a ∈ S X k ∈ S u ak (cid:19) | S | − X a ∈ S X k ∈ S u ka = r “(d) ⇒ (e)” is trivial as (d) is stronger than (e).“(e) ⇒ (a)” Let a and r is fixed. By above calculations of the forward bias, we have X a ∈ S X b/ ∈ S u ab = X a ∈ S X b/ ∈ S u ba . Consider the states S = { a } in which there is only one individual of type A then X b/ ∈ S u ab = X b/ ∈ S u ba ⇒ − u aa + N X i =1 u ai = − u aa + N X i =1 u ia ⇒ N X i =1 u ia is true for all a = 1 , , . . . , N and therefore U is doubly stochastic and thus f R ( W ) is acirculation.“(d) ⇒ (b)” follows from Lemma 1.“(b) ⇒ (c)” is trivial.“(c) ⇒ (e)” follows from Lemma 1. 14 .5 Proposition 3 (BDD and DBB) The following statements are equivalent:(a) f R ( W ) = W H or f R ( W ) ∈ C N .(b) For all r > W ∼ BDD ,r W H . Proof.
The replacement probabilities r ij ( F ( S ) , W ) for BDD dynamics can be rewritten as r ij ( F ( S ) , U ) where U = ( u ij ) = f R ( W ) = ( w ij / P n w in ) by multiplying the numeratorand denominator with P n w in as follows r ij ( F ( S ) , W ) = 1 N w ij /F j ( S ) P n w in /F n ( S ) = 1 N w ij / ( F j ( S ) P n w in ) P n w in / ( F n ( S ) P n w in ) ⇒ u ij /F j ( S ) P n u in /F n ( S ) = r ij ( F ( S ) , U )and therefore we have that W ∼ BDD ,r U , for all r >
0. The forward bias using U forstate S is given by P + S P − S = X a ∈ S X b/ ∈ S N u ab /F b X n u an /F n X a ∈ S X b/ ∈ S N u ba /F a X n u bn /F n = X a ∈ S X b/ ∈ S u ab X n u an /F n r X a ∈ S X b/ ∈ S u ba X n u bn /F n . (6)Similarly, let V = ( v ij ) = f L ( W ) = ( w ij / P n w nj ). Then for DBB dynamics we have b ij = w ij F i P n w nj F n = w ij F i / P n w nj P n w nj F n / P n w nj = v ij F i P n v nj F n and therefore the forward bias when using V is given by P + S P − S = X a ∈ S X b/ ∈ S N v ab F a X n v nb F n X a ∈ S X b/ ∈ S N v ba F b X n v na F n = r X a ∈ S X b/ ∈ S v ab X n v nb F n X a ∈ S X b/ ∈ S v ba X n v na F n . The proof of the Proposition for DBB closely follows the one for BDD given below with U and f R appropriately replaced by V and f L . A.5.1 If U ∈ C N , then U ∼ BDD ,r W H If U ∈ C N then there are only two nonzero elements in each row. In particular, in row i of U we have that u ii , u ik i = 1 / k i = i . In the numerator of equation (6) for a ∈ S , b / ∈ S and k a = a we have that for all Su ab X n u an /F n ( S ) = u ab u aa /F a ( S ) + u ak a /F k a ( S ) = ( b = k a , / / r +1 / if b = k a . Similarly, in the denominator of equation (6) for a ∈ S , b / ∈ S and k b = b we have thatfor all S u ba X n u bn /F n ( S ) = u ba u bb /F b ( S ) + u bk b /F k b ( S ) = ( a = k b , / / / r if a = k b . S can be written as x/ / r + 1 / (cid:30) r y/ / / r = rx/y where x ( y ) is the number of nonzero u ab ( u ba ) terms in the numerator (denominator).If we partition the vertices of the digraph of U into any two sets V , V then the numberof edges e ( i, j ) and e ( j, i ) for i ∈ V and j ∈ V are by definition the same because it isa cycle. This means that for a ∈ S and b / ∈ S the number of nonzero u ab , u ba terms inthe numerator and denominator respectively are the same hence x = y and rx/y = r asrequired. As per Lemma 1, E BDD , U ,r is ρ -equivalent to the Moran process. A.5.2 If U ∼ BDD ,r W H for all r > , then U = W H or U ∈ C N By Lemma 1, the forward bias (6) is equal to r for all S ⊂ { , . . . , N } giving X a ∈ S X b/ ∈ S u ab X n u an /F n = X a ∈ S X b/ ∈ S u ba X n u bn /F n ⇒ X a ∈ S X b/ ∈ S u ab X j / ∈ S u aj + 1 r X i ∈ S u ai = X b/ ∈ S X a ∈ S u ba X j / ∈ S u bj + 1 r X i ∈ S u bi . (7)Note that if r = 1, (7) holds for all U ∈ W C . From now, we will consider r = 1 only.For clarity, the remainder of this section of the proof is broken down into the followingsix steps. Step 1: Derivation of general state dependent row-sum equation
Let U ( a, S ) = P i ∈ S u ai , i.e. 1 − U ( a, S ) = P j / ∈ S u aj . Equation (7) thus becomes X a ∈ S − U ( a, S )1 − U ( a, S ) + U ( a, S ) /r = X b/ ∈ S U ( b, S )1 − U ( b, S ) + U ( b, S ) /r ⇒ X a ∈ S
11 + U ( a, S )(1 /r −
1) = N X n =1 U ( n, S )1 + U ( n, S )(1 /r − . (8)Equation (8) can be written as a Taylor series as follows X a ∈ S ∞ X k =0 ( − k (1 /r − k [ U ( a, S )] k = N X n =1 U ( n, S ) ∞ X k =0 ( − k (1 /r − k [ U ( n, S )] k ⇒ X a ∈ S ∞ X k =0 (1 − /r ) k [ U ( a, S )] k = N X n =1 ∞ X k =0 (1 − /r ) k [ U ( n, S )] k +1 (9)For equation (9) to hold for all r the coefficients of (1 − /r ) k should be same, that is,for all k X a ∈ S [ U ( a, S )] k = N X n =1 [ U ( n, S )] k +1 . (10) Step 2: The diagonal of U consists of non-zero elements
Consider the state S = { a } then equation (10) gives u kaa = N X n =1 u k +1 na . (11)If u aa = 0 or 1 then (11) implies that all off-diagonal terms in column n are zero whichis a contradiction with W (and thus also U = f R ( W )) being strongly connected, whichmeans that 0 < u aa <
1. 16 tep 3: The n th column of U contains m n nonzero elements, all equal to /m n Since 0 < u aa <
1, we can divide equation (11) by u kaa giving1 = N X n =1 u na (cid:18) u na u aa (cid:19) k . (12)We have that lim k →∞ (cid:18) u na u aa (cid:19) k = ∞ u na > u aa , u na = u aa , u na < u aa , and therefore (12) implies that 0 ≤ u na ≤ u aa . There must be n = a such that u na = u aa as otherwise, by (12), we would have u aa = 1. Let C a = { i : u ia = u aa } . (12) becomes1 = (cid:18) X i ∈ C a u aa (cid:19) + (cid:18) X j / ∈ C a u k +1 ja u kaa (cid:19) = | C a | u aa + (cid:18) X j / ∈ C a u k +1 ja u kaa (cid:19) . (13)As k → ∞ , (13) implies that u aa = 1 / |C a | . Thus, again by (13), u ja = 0 for all j / ∈ C a .This means that in column n of U there should be m n = | C n | with 2 ≤ m n ≤ N nonzeroelements, including u nn , that are all equal to 1 /m n . Step 4: m n is the same for all n Considering state S = { i, j } and using u aa = 1 /m a , (10) can be written as follows( u ii + u ij ) k + ( u ji + u jj ) k = α m k +1 i + β m k +1 j + γ (cid:18) m i + 1 m j (cid:19) k +1 (14)where α, β, γ are the number of rows where 1 /m i is adjacent to 0, 0 is adjacent to 1 /m j ,and 1 /m i is adjacent to 1 /m j in columns i and j respectively. More precisely, α is thecardinality of the set K iij = { n : u ni = 1 /m i , u nj = 0 } , β is the cardinality of the set K jij = { n : u ni = 0 , u nj = 1 /m j } and γ is the cardinality of the set K ijij = { n : u ni =1 /m i , u nj = 1 /m j } .Since C i = K iij ∪ K ijij and C j = K jij ∪ K ijij , we have that m i = α + γ and m j = β + γ .Since K iij , K jij , K ijij are disjoint, we have α + β + γ ≤ N . Now, consider the differentpossibilities we can have on the left-hand side of equation (14). Case 1: u ii = 1 /m i , u ij = 0 in row i and u ji = 1 /m i , u jj = 1 /m j in row j . Thus α, γ ≥ m ki + (cid:18) m i + m j m i m j (cid:19) k = αm k +1 i + βm k +1 j + γ (cid:18) m i + m j m i m j (cid:19) k +1 ⇒ α + γ ) k + (cid:18) α + β + 2 γ ( α + γ )( β + γ ) (cid:19) k = α ( α + γ ) k +1 + β ( β + γ ) k +1 + γ (cid:18) α + β + 2 γ ( α + γ )( β + γ ) (cid:19) k +1 ⇒ ( β + γ ) k + ( α + β + 2 γ ) k [( α + γ )( β + γ )] k = α ( β + γ ) k +1 + β ( α + γ ) k +1 + γ ( α + β + 2 γ ) k +1 [( α + γ )( β + γ )] k +1 ⇒ ( β + γ ) k + ( α + β + 2 γ ) k = α ( β + γ ) k +1 + β ( α + γ ) k +1 + γ ( α + β + 2 γ ) k +1 ( α + γ )( β + γ ) ⇒ ( β + γ ) k + ( α + β + 2 γ ) k = α ( β + γ ) k α + γ + β ( α + γ ) k β + γ + ( αγ + βγ + 2 γ )( α + β + 2 γ ) k αβ + αγ + βγ + γ ⇒ γ ( β + γ ) k α + γ = β ( α + γ ) k β + γ + ( γ − αβ )( α + β + 2 γ ) k αβ + αγ + βγ + γ . As k → ∞ , we get ( β + γ ) k = ( α + γ ) k ± ( α + β + 2 γ ) k since α + β + 2 γ > β + γ, α + γ hence we want γ = αβ to get rid off ( α + β + 2 γ ) k . This implies that β + γ = α + γ ⇒ α = β ⇒ α = β = γ giving m i = m j . 17 ase 2: u ii = 1 /m i , u ij = 1 /m j in row i and u ji = 0 , u jj = 1 /m j in row j . This case issymmetrical to Case 1 and therefore we get that α = β = γ giving m i = m j . Case 3: u ii = 1 /m i , u ij = 1 /m j in row i and u ji = 1 /m i , u jj = 1 /m j in row j . Thus γ ≥ (cid:18) m i + m j m i m j (cid:19) k = αm k +1 i + βm k +1 j + γ (cid:18) m i + m j m i m j (cid:19) k +1 ⇒ (cid:18) α + β + 2 γ ( α + γ )( β + γ ) (cid:19) k = α ( β + γ ) k +1 + β ( α + γ ) k +1 + γ ( α + β + 2 γ ) k +1 [( α + γ )( β + γ )] k +1 ⇒ α + β + 2 γ ) k = α ( β + γ ) k +1 + β ( α + γ ) k +1 + γ ( α + β + 2 γ ) k +1 ( α + γ )( β + γ ) ⇒ α + β + 2 γ ) k = α ( β + γ ) k α + γ + β ( α + γ ) k β + γ + ( αγ + βγ + 2 γ )( α + β + 2 γ ) k αβ + αγ + βγ + γ ⇒ (2 αβ + αγ + βγ )( α + β + 2 γ ) k αβ + αγ + βγ + γ = α ( β + γ ) k α + γ + β ( α + γ ) k β + γ . As k → ∞ , we get ( α + β + 2 γ ) k = ( β + γ ) k + ( α + γ ) k since α + β + 2 γ > β + γ, α + γ hence we want 2 αβ + αγ + βγ = 0 ⇒ α, β = 0 giving m i = m j . Case 4: u ii = 1 /m i , u ij = 0 in row i and u ji = 0 , u jj = 1 /m j in row j . Thus α, β ≥ /m ki + 1 /m kj = αm k +1 i + βm k +1 j + γ (cid:18) m i + m j m i m j (cid:19) k +1 ⇒ α + γ ) k + 1( β + γ ) k = α ( α + γ ) k +1 + β ( β + γ ) k +1 + γ (cid:18) γ + β + 2 γ ( α + γ )( β + γ ) (cid:19) k +1 ⇒ ( β + γ ) k + ( α + γ ) k [( α + γ )( β + γ )] k = α ( β + γ ) k +1 + β ( α + γ ) k +1 + γ ( α + β + 2 γ ) k +1 [( α + γ )( β + γ )] k +1 ⇒ ( β + γ ) k + ( α + γ ) k = α ( β + γ ) k +1 + β ( α + γ ) k +1 + γ ( α + β + 2 γ ) k +1 ( α + γ )( β + γ ) ⇒ ( β + γ ) k + ( α + γ ) k = α ( β + γ ) k α + γ + β ( α + γ ) k β + γ + γ ( α + β + 2 γ ) k +1 αβ + αγ + βγ + γ . As k → ∞ , we get 0 = ( α + β + 2 γ ) k since α, β ≥ γ = 0 to getan equality. Conclusion from all the cases above
We see that m i = m j is potentially possible only in Case 4. However, U is stronglyconnected. If one connects i and j by a path i = i , i , i , . . . i n = j , then one has m i k = m i k +1 as i k and i k +1 must fall into Case 1, Case 2 or Case 3 above. Thus m i = m j . This implies that every column of U has 2 ≤ m ≤ N nonzero elements,including u nn , that are all equal to 1 /m . This is also true for every row of U because itis right stochastic by definition. Step 5: There exists state S such that C a = C a ′ for all a, a ′ ∈ S We can define the state R x = { n : u xn = u xx } then, by definition, x ∈ R x and |R x | = m since there are m nonzero elements in row x of U . Consider the state S = R x \ { y } for y ∈ R x \ { x } . For this S (as well as any other state), we have thatif n ∈ S then 1 /m if n / ∈ S then 0 ) ≤ U ( n, S ) ≤ min( m, | S | ) m .
18e can therefore write equation (10) in the form min( m, | S | ) X i =1 λ S ( i ) (cid:18) im (cid:19) k = min( m, | S | ) X i =0 λ ′ S ( i ) (cid:18) im (cid:19) k +1 (15)where λ S ( i ) is the number of U ( n, S ) terms equal to i/m for n ∈ S and λ ′ S ( i ) is thenumber of U ( n, S ) terms equal to i/m for n ∈ N , which means that λ ′ S ( i ) ≥ λ S ( i ) for i = 0. The ratio of the left-hand side and right-hand side of equation (15) should alwaysbe equal to one. Therefore, as k → ∞ , we require that λ S ( i max ) = λ ′ S ( i max ) i max m where i max is the largest i such that λ S ( i ) > i max = m − | S | = m − U ( x, S ) =( m − /m . This means that for state S , as k → ∞ , we require that λ S ( m −
1) = λ ′ S ( m − m − m . Since λ S ( m −
1) is an integer, λ ′ S ( m −
1) has to be a multiple of m and the only possiblevalue that satisfies this criteria is λ ′ S ( m −
1) = m hence λ S ( m −
1) = m − λ ′ S ( m −
1) = m there exist m rows j , j , . . . , j m such that U ( j n , S ) = ( m − /m ,that is, u j n a = 1 /m ∀ a ∈ S . This means that C a = { j , j , . . . , j m } ∀ a ∈ S hence C a = C a ′ for all a, a ′ ∈ S . Step 6: m = 2 or m = N By contradiction, assume that 2 < m < N . We can consider another state S ′ = R x \ { z } such that z ∈ R x \ { x, y } . We then have that i max = m − | S ′ | = m − U ( x, S ′ ) = ( m − /m . As before, this means that C a = C a ′ for all a, a ′ ∈ S ′ . Since x ∈ S, S ′ and R x = S ∪ S ′ we have that C a = C a ′ for all a, a ′ ∈ R x . For2 < m < N this implies that vertices i ∈ R x are disconnected from j ∈ N \ R xx