Universality of weak selection
UUniversality of weak selection
Bin Wu,
1, 2, ∗ Philipp M. Altrock, Long Wang, and Arne Traulsen † Research Group Evolutionary Theory, Max-Planck-Institute for Evolutionary Biology,August-Thienemann-Str. 2, 24306 Pl¨on, Germany Center for Systems and Control, State Key Laboratory for Turbulence and Complex Systems,College of Engineering, Peking University, Beijing 100871, China (Dated: October 22, 2018)Weak selection, which means a phenotype is slightly advantageous over another, is an importantlimiting case in evolutionary biology. Recently it has been introduced into evolutionary game theory.In evolutionary game dynamics, the probability to be imitated or to reproduce depends on theperformance in a game. The influence of the game on the stochastic dynamics in finite populationsis governed by the intensity of selection. In many models of both unstructured and structuredpopulations, a key assumption allowing analytical calculations is weak selection, which means thatall individuals perform approximately equally well. In the weak selection limit many differentmicroscopic evolutionary models have the same or similar properties. How universal is weak selectionfor those microscopic evolutionary processes? We answer this question by investigating the fixationprobability and the average fixation time not only up to linear, but also up to higher orders inselection intensity. We find universal higher order expansions, which allow a rescaling of the selectionintensity. With this, we can identify specific models which violate (linear) weak selection results,such as the one–third rule of coordination games in finite but large populations.
PACS numbers: 87.23.-n, 05.40.-a, 02.50.-r
I. INTRODUCTION
In evolutionary game theory the outcome of strategicsituations determines the evolution of different traits ina population [1]. Typically, individuals are hardwired toa set of strategies. The performance in an evolutionarygame determines the rate at which strategies spread byimitation or natural selection. Due to differences in pay-off, different strategies spread with different rates undernatural selection. In infinitely large well–mixed popula-tions this is described by the deterministic replicator dy-namics [2–5]. In this set of non–linear differential equa-tions the intensity of selection, which determines howpayoff affects fitness, only changes the time scales, butnot the direction of selection or the stability properties.In finite populations fluctuations cannot be neglected [6–9]. The dynamics becomes stochastic: Selection drivesthe system into the same direction as the correspond-ing deterministic process, but sometimes the system canalso move into another direction. The strength of selec-tion determines the interplay between these two forces.The absence of selective differences is called neutral selec-tion: Moving into one direction is as probable as movinginto any other, independent of the payoffs. If selectionacts, the transition probabilities become payoff depen-dent and thus asymmetric. The asymmetry can be thesame in each state (constant selection) or state depen-dent (frequency dependent selection). In general, un-der frequency dependent selection the probability that ∗ Electronic address: [email protected] † Electronic address: [email protected] one strategy replaces another can be fairly complicated.However, under the assumption of weak selection, someimportant insights can be obtained analytically [9–16].It has to be pointed out that these results do in generalnot carry over to stronger selection.Weak selection describes situations in which the effectsof payoff differences are small, such that the evolution-ary dynamics are mainly driven by random fluctuations.This approach has a long standing history in populationgenetics [17, 18]. In evolutionary biology, a phenotypeis often found to be slightly advantageous over anotherphenotype [19, 20]. Further, a recent experiment suggeststhat some aspects of weak selection are reflected in hu-man strategy updating in behavioral games [21]. In thecontext of evolutionary game dynamics, however, weakselection has only recently been introduced by Nowak etal. [9]. The definition of weak selection is unambiguousin the case of constant selection, but there are differentways to introduce such a limit under frequency depen-dent selection [22].In the simplest case, frequency dependence can be in-troduced by an evolutionary game between two types A and B . In a one shot interaction (where strategies areplayed simultaneously) a type A interacting with anothertype A receives payoff a , two interacting B types get d each. Type A interacting with B gets b , whereas B ob-tains c . This symmetric 2 × (cid:32) A BA a bB c d (cid:33) . (1)Let i denote the number of A individuals in a populationof constant size N . Under the assumption of a well-mixed a r X i v : . [ q - b i o . P E ] O c t opulation, excluding self–interactions, the average pay-offs for individuals of either type are given by π A = a i − N − b N − iN − , (2) π B = c iN − d N − i − N − , (3)These expectation values are the basis for the evolution-ary game. In the continuous limit N → ∞ , the state ofthe system is characterized by the fraction of A individ-uals x = i/N . The dynamics are typically given by thereplicator dynamics ˙ x = x (1 − x )( π A − π B ), which has thetrivial equilibria ˆ x = 0 and ˆ x = 1. Additionally, therecan be a third equilibrium between zero and one, givenby x ∗ = ( d − b ) / ( a − b − c + d ). In finite populations,the probabilistic description does not allow the existenceof equilibrium points anymore. Moreover, the invarianceof the replicator dynamics under rescaling of the payoffmatrix [5] is lost in finite population models. Typically,the average payoffs are mapped to the transition proba-bilities to move from state i to other states, only i = 0and i = N are absorbing states. When only two typescompete and there is only one reproductive event at atime this defines a birth–death process. The transitionprobabilities from i to i + 1, and from i to i − T + i and T − i , respectively. They determinethe probability of the process to be absorbed at a cer-tain boundary, usually called fixation probability, as wellas the average time such an event takes, termed averagefixation time.An important result of evolutionary game dynamics infinite populations under weak frequency dependent selec-tion is the one–third rule. It relates the fixation proba-bility of a single type A individual, φ , to the positionof the internal equilibrium x ∗ in a coordination game,i.e. when a > c and d > b . If selection is neutral, we have φ = 1 /N . If the internal equilibrium is less than onethird, x ∗ < /
3, then φ > /N . Originally, this weak se-lection result has been found for large populations in thefrequency dependent Moran process [9]. Subsequently,the one–third rule has been derived from several relatedbirth–death processes [23–25], and also for the frequencydependent Wright-Fisher process [26, 27], which is still aMarkov process, but no longer a birth-death process. Ina seminal paper, Lessard and Ladret have shown that theone–third-rule is valid for any process in the domain ofKingman’s coalescence [28], which captures a huge num-ber of the stochastic processes typically considered inpopulation genetics. Essentially, this class of processesdescribes situations in which the reproductive success isnot too different between different types. Thus, the gen-erality of the one–third-rule under linear weak selectionis well established. Here we ask a slightly different ques-tion: To which order can two birth–death processes beconsidered as identical under weak selection? Some au-thors have considered higher weak selection orders forspecific processes [29–31]. We investigate two classes ofbirth–death processes, a general pairwise imitation pro- cess motivated by social learning and a general Moranprocess based on reproductive fitness. In this light, wealso discuss cases which violate the one–third rule.The manuscript is organized in the following way: InSec. II we compute the weak selection expansion of thefixation probability in a general case of our two classesof birth–death processes. In Sec. III, we perform thesame calculations for the significantly more complicatedfixation times. In Sec. IV we discuss our analytical resultsand conclude. Some detailed calculations can be foundin the Appendix. II. PROBABILITIES OF FIXATION
A birth–death process is characterized by the tran-sition probabilities from each state i to its neighboringstates, T + i and T − i . We assume that this Markov chainis irreducible on the interior states and we exclude muta-tions or spontaneous switching from one type to another.Thus, the process gets eventually absorbed at i = 0 or N . For any internal state, the probability to hit i = N starting from 0 < i < N , φ i , fulfills the recursion equa-tion φ i = (1 − T + i − T − i ) φ i + T − i φ i − + T + i φ i +1 [32–34].This recursion can be solved explicitly, respecting theboundary conditions φ = 0 and φ N = 1. For a single A individual in a populations of B , the probability to takeover the population is [32–34] φ = 11 + (cid:80) N − k =1 (cid:81) ki =1 T − i T + i . (4)In any model of neutral selection, the transition prob-abilities of the Markov chain fulfill T − i /T + i = 1, andhence the respective fixation probability of a single mu-tant amounts to 1 /N .In this section we focus on the weak selection approxi-mation of Eq. (4). We do this for two different approachesto evolutionary game theory: imitation dynamics and se-lection dynamics. In the former case, strategy spreadingis based on pairwise comparison and imitation, in thelatter it results from selection proportional to fitness andrandom removal. The most prominent examples are theFermi process and the Moran process, respectively. A. Pairwise comparison
In a pairwise comparison process, two individuals arechosen randomly to compare their payoffs from the evo-lutionary game, Eqs. (2) and (3). One switches to theothers strategy with a given probability, see Fig. 1. If se-lection is neutral, this probability is constant. If selectionacts, the larger the payoff difference, the higher the prob-ability that the worse imitates the better. But typicallythere is also a small chance that the better imitates theworse. Otherwise, only the strategy of the more success-ful individual is adopted. This would lead to a dynamics2 I m i t a t i on p r obab ili t y g ( ! " ) Payoff difference !" -1.0 -0.5 0.0 0.5 1.001234 F i t ne ss f ( ! ) Payoff ! f ( π ) = 1 + βπ f ( π ) = 1 + βπ + ( βπ ) + ( βπ ) f ( π ) = exp( βπ ) g (∆ π ) = ! π ≤ β ∆ π ∆ π > g (∆ π ) = 12 + β ∆ π g (∆ π ) = 11 + exp( − β ∆ π ) FIG. 1: (color online) Upper panel: Pairwise comparison pro-cesses are characterized by the probability g (∆ π ) to imitatethe strategy of someone else based on the payoff difference∆ π . With increasing payoff difference, the imitation proba-bility becomes higher, g ! (∆ π ) ≥
0. Weak selection implies aTaylor expansion at ∆ π = 0. Thus, it can only be invokedfor functions that are differentiable in 0. The figure showsthree examples of imitation probability functions, g (∆ π ) is alinear function (selection intensity β = 0 .
5) and g (∆ π ) is theFermi function ( β = 50). For the imitation function g (∆ π ),a meaningful weak selection limit does not exist since g (∆ π )is not differentiable in 0. Because g (∆ π ) = 0 for ∆ π < f ( π ). Fitness is a non–decreasing functionof the payoff, f ! ( π ) ≥
0. The figure shows three examples forpayoff to fitness mappings (selection intensity β = 1 for allthree functions). FIG. 1: (color online) Upper panel: Pairwise comparison pro-cesses are characterized by the probability g (∆ π ) to imitatethe strategy of someone else based on the payoff difference∆ π . With increasing payoff difference, the imitation proba-bility becomes higher, g (cid:48) (∆ π ) ≥
0. Weak selection implies aTaylor expansion at ∆ π = 0. Thus, it can only be invokedfor functions that are differentiable in 0. The figure showsthree examples of imitation probability functions, g (∆ π ) is alinear function (selection intensity β = 0 .
5) and g (∆ π ) is theFermi function ( β = 50). For the imitation function g (∆ π ),a meaningful weak selection limit does not exist since g (∆ π )is not differentiable in 0. Because g (∆ π ) = 0 for ∆ π < f ( π ). Fitness is a non–decreasing functionof the payoff, f (cid:48) ( π ) ≥
0. The figure shows three examples forpayoff to fitness mappings (selection intensity β = 1 for allthree functions). that is stochastic in the time spent in each interior state,but deterministic in direction [24]. Thus, given that allinterior states are transient, the fixation probabilities areeither 0 or 1 and there is no basis to discuss a weak se-lection limit.Selection is parameterized by the intensity of selec- tion β ≥
0. As a first example we consider the Fermiprocess [24, 35, 36]. Let the two randomly selected in-dividuals X and Y have payoffs π X and π Y . Then X adopts Y ’s strategy with probability g Fermi ( π Y − π X ) =1 / (cid:0) − β ( π Y − π X ) (cid:1) . Thus, the transition probabilitiesof an evolutionary game with payoffs Eqs. (2) and (3)are given by T ± i = iN N − iN
11 + exp ∓ β ( π A − π B ) . (5)The probability to stay in state i is 1 − T − i − T + i . TheFermi process is closely related to Glauber dynamics [37].If we define individuals’ energy as the exponential func-tion of payoff, then the Fermi process can be mappedonto the Ising model. The Fermi process has the com-fortable property that the ratio of transition probabili-ties simplifies to T − i /T + i = e − β ( π A − π B ) , such that theproducts in Eq. (4) can be replaced by sums in the ex-ponent. Defining u = ( a − b − c + d ) / ( N −
1) and v = ( N b − N d − a + d ) / ( N − π A − π B = u i + v ,leads to φ ( β ) = 11 + (cid:80) N − k =1 exp (cid:8) − β (cid:2) k u + k ( u + v ) (cid:3)(cid:9) . (6)For large N , the sum can be replaced by an integral,leading to a closed expression [24]. For weak selection, N β (cid:28)
1, Eq. (6) can be approximated by φ ≈ N + ( N − N + 1) u + 3 v )6 N β. (7)This can also be obtained directly from T − i /T + i ≈ − β ( π A − π B ). The fixation probability under weak selec-tion is greater than in the neutral case if the term linear in β is positive, N u + u +3 v >
0. In particular, for a coordi-nation game in a large population, this implies x ∗ < / φ concerning the probability of switching strate-gies, g (∆ π )? In a general framework, the probability that X switches to the strategy of Y , given the difference intheir payoffs, ∆ π = π X − π Y , is governed by the inten-sity of selection. We call g (∆ π ) the imitation probabilityfunction of a general pairwise comparison process. In awell mixed population, the transition probabilities read T ± i = iN N − iN g ( ± β ∆ π ) . (8)The larger the payoff difference, the more likely the worseindividual switches to the strategy of the better. There-fore the imitation function is nondecreasing, g (cid:48) (∆ π ) ≥ g (0) > φ ≈ N + C β + C β , (9)where C = ( N −
1) (( N + 1) u + 3 v )6 N g (cid:48) (0) g (0) , (10)and C = (cid:0) u ( N +1)( N +2) + 15 uv ( N +1) + 30 v (cid:1) (11) × ( N − N − (cid:18) g (cid:48) (0) g (0) (cid:19) .C is proportional to the increase of the imitation func-tion at ∆ π = 0, see Fig. 2. Note that for large N , C > N u + 3 v >
0, which for large N fur-ther simplifies to x ∗ < /
3. Thus, the one–third ruleholds for all pairwise comparison processes that fulfill g (cid:48) (0) (cid:54) = 0, and g (0) >
0. Moreover, C is proportional to2 g (cid:48) (0) /g (0), while C is proportional to the square of thisquantity. Thus, 2 g (cid:48) (0) /g (0) can be absorbed into the se-lection intensity by proper rescaling. Therefore, the morerapid the increases of the imitation function at ∆ π = 0,the stronger is the sensitivity of the fixation probabilityto changes in average payoff. For low switching proba-bilities in the neutral case, ∆ π = 0, we have a fixationprobability that changes rapidly when the payoff differ-ence becomes important, ∆ π (cid:54) = 0. While most previousmodels have either considered g (0) = 0 (which lies outof the scope of our approach, because it does not lead toa reasonable definition of weak selection) or g (0) = 0 . g (0). Forexample, Szab´o and Hauert have used the imitation func-tion g ( x ) = 1 / (1 + e − x + α ), where α is a constant [38]. Inthis case 2 g (cid:48) (0) /g (0) = 2 / (1+exp( − α )), thus, an increasein α is equivalent to an increase in the (small) selectionintensity.Now it is straightforward to come up with an imitationfunction that leads to a violation of the one–third rule,for example g (∆ π ) = 1 / (1 + exp {− ∆ π } ). Obviously, g ( β ∆ π ) satisfies the conditions g (cid:48) ( β ∆ π ) ≥
0, and g (0) (cid:54) =0. Further, both the first and the second order expansionsvanish. Therefore, the fixation probability under weakselection can only be approximated as φ ≈ N + C β , (12)where C can be derived in the same way as C and C . In special games, the sign of C can also change at x ∗ = 1 /
3, but in general this will not be the case due tothe complicated dependence of C on u and v . In moregeneral terms, the 1/3 rule is not sustained whenever thelinear approximation of g ( β ∆ π ) vanishes. !"!!! !"!! ! " $ % " &’ ( ) * &+$+ "," % - ( ! ! !" " !"!!! !"!! ! " $ % " &’ ( ) * &+$+ "," % - ( ! ! !" " !" ( FIG. 2: Approximation of the fixation probability of a singlemutant under weak selection. Upper panel: Pairwise compar-ison process with the Fermi function 1 / (1 + exp[ − β ∆ π ]) asimitation function. As shown in the main text, up to secondorder the approximation is valid for any imitation function g ( β ∆ π ) after appropriate rescaling of the selection intensity β . Lower panel: Moran process with fitness as a linear func-tion of the payoff, f = 1+ βπ . Any other function leads to thesame first order approximation after rescaling of β . However,the second order depends on choice of the function transform-ing payoff to fitness. Exact analytical results are numericalevaluations of Eq. (4). (Parameters N = 100, β = 1, a = 4, b = 1, c = 1, and d = 5 in both panels). FIG. 2: Approximation of the fixation probability of a singlemutant under weak selection. Upper panel: Pairwise compar-ison process with the Fermi function 1 / (1 + exp[ − β ∆ π ]) asimitation function. As shown in the main text, up to secondorder the approximation is valid for any imitation function g ( β ∆ π ) after appropriate rescaling of the selection intensity β . Lower panel: Moran process with fitness as a linear func-tion of the payoff, f = 1+ βπ . Any other function leads to thesame first order approximation after rescaling of β . However,the second order depends on choice of the function transform-ing payoff to fitness. Exact analytical results are numericalevaluations of Eq. (4). (Parameters N = 100, β = 1, a = 4, b = 1, c = 1, and d = 5 in both panels). B. Moran process
In the frequency dependent Moran process the pay-off π , given by Eqs. (2) and (3), is mapped to fitness f , as illustrated in Fig. 1. In each reproductive event,one individual is selected for reproduction (producing anidentical offspring) proportional to fitness. To keep thesize of the population to the constant value N , a ran-domly chosen individual is removed from the populationsubsequently. As in pairwise comparison processes, thestate i can at most change by one per time step.In the simplest case, fitness is a linear function of pay-off. With a background fitness of one, the fitnesses of4ype A and B read f A = 1 + β π A , and f B = 1 + β π B ,respectively. The quantity β ≥ β is bound such that fitness neverbecomes negative. The probability that the number of A individuals increases by one, i → i + 1, is given by T + i = if A if A + ( N − i ) f B N − iN . (13)The other possible transition, i → i −
1, occurs withprobability T − i = ( N − i ) f B if A + ( N − i ) f B iN . (14)When selection is neutral, β = 0, we have T ± i = i ( N − i ) /N . Up to linear order in β the Moran process has thesame fixation probability as the Fermi process, Eq. (7),such that in this approximation the one–third rule is ful-filled. This is because under first order weak selection, T − i /T + i is again a linear function of the payoff difference.In general, let fitness be any non-negative functionof the product of payoff and selection intensity, f ( βπ ), which fulfills f (cid:48) ( βπ ) ≥
0. For simplicity, we assume thatthe baseline fitness f (0) is one. The transition probabil-ities in a population with types A and B read T + i = if ( βπ A ) if ( βπ A ) + ( N − i ) f ( βπ B ) N − iN , (15) T − i = ( N − i ) f ( βπ B ) if ( βπ A ) + ( N − i ) f ( βπ B ) iN . (16)Note that T − i /T + i = f ( βπ B ) /f ( βπ A ). Up to second or-der in β , the fixation probability of a single A mutant ina population of B is (see Appendix A 2) φ ≈ N + D β + D β , (17)where D = ( N −
1) ( N + 1) u + 3 v N f (cid:48) (0) , (18)and D = (cid:34) u ( N + 1)( N + 2) + 15 uv ( N + 1) + 30 v (cid:35) ( N − N − f (cid:48) (0) − (cid:34) (2 a + 4 ab + 4 cd − d ) + (11 d + 2 cd − c − b − ab − a ) N + ( a + 2 ab + 3 b − c − cd − d ) N (cid:35) ( N − N (cid:0) f (cid:48) (0) − f (cid:48)(cid:48) (0) (cid:1) , (19)with u and v as above. Note that the first order termdepends on payoff differences only, but the second or-der term also depends on the payoff values directly. Anexample for such an approximation is shown in Fig. 2.The first order term D is proportional to the increasein fitness at π = 0, f (cid:48) (0). The first order term D isproportional to N u + 3 v for large N . Hence, the one–third rule holds for every Moran model for which f (cid:48) (0)does not vanish under weak selection. Additionally, f (cid:48) (0)can be absorbed into the selection intensity by rescaling:Changing this rate is equivalent to changing the inten-sity of selection. Note that this is not possible with D ,where not only the slope, but also the curvature of thefitness function at the origin plays a role. However, whenthe exponential fitness function f = exp( βπ ) is employed[39], the second term of Eq. (19) vanishes. This allowsto incorporate f (cid:48) (0) into the selection intensity even forthe second order term.Again, we conclude the section with an example wherethe one–third is violated. Consider the fitness function f ( βπ ) = 1+ β π , which obviously satisfies f (0) = 1, and f (cid:48) ( βπ ) ≥
0. Both, first and second order correction in β vanish, D = D = 0. Therefore, the first non-trivialapproximation of the fixation probability is φ ≈ N + D β . (20)If D changes sign at x ∗ = 1 /
3, we recover the one–thirdrule. This is only the case for very special games. Inanalogy to the previous section, the general one–thirdrule does not hold anymore.
III. TIMES OF FIXATION
In this section we address the conditional fixation time τ Ai . In a finite population of N − i individuals of type B and i individuals of type A , τ Ai measures the ex-pected number of imitation or birth–death events untilthe population consist of type A only, under the con-5ition that this event occurs. In general, the probabil-ity P Ai ( t ) that after exactly t events the process movedfrom any i to N , which is the all A state, obeys themaster equation P Ai ( t ) = (cid:0) − T + i − T − i (cid:1) P Ai ( t −
1) + T − i P Ai − ( t −
1) + T + i P Ai +1 ( t − τ Ai = (cid:80) ∞ t =0 t P Ai ( t ) /φ i is the stationary first mo-ment of this probability distribution, resulting from arecursive solution of φ i τ Ai = (cid:0) − T + i − T − i (cid:1) φ i τ Ai + T − i φ i − ( τ Ai − + 1) + T + i φ i +1 ( τ Ai +1 + 1). In a similar wayone can find τ Bi = (cid:80) ∞ t =0 t P Bi ( t ) / (1 − φ i ), such that thetotal average lifetime of the Markov process amounts to φ i τ Ai + (1 − φ i ) τ Bi [32, 40, 41]. Following the previoussection we restrict our analysis to the biologically mostrelevant case i = 1, which yields [32, 40] τ A = N − (cid:88) k =1 k (cid:88) l =1 φ l T + l k (cid:89) m = l +1 T − m T + m . (21)Maruyama and Kimura [42], Antal and Scheuring [41]as well as Taylor et al. [43] have shown that the condi-tional fixation time of a single mutant of either type isthe same, τ A = τ BN − . This remarkable identity holds forany evolutionary birth–death process, and is thus validfor any 2 × j > τ Aj (cid:54) = τ BN − j , unless β vanishes.Since τ A and τ BN − are identical up to any order in β , weobtain (cid:20) ∂ n ∂β n τ A (cid:21) β =0 = (cid:20) ∂ n ∂β n τ BN − (cid:21) β =0 (22)for any n . This symmetry can help to obtain severalproperties of the expansion of the conditional fixationtime, Eq. (21), without brute force calculations. A. Pairwise comparison
Let us first consider the fixation time in the special caseof the Fermi process, Eq. (5). When the selection inten-sity vanishes, β = 0, we have τ A (0) = 2 N ( N − N β (cid:28)
1, the conditional fixa-tion time is approximately τ A ≈ τ A (0)+ ∂ β τ A ( β ) | β =0 β + ∂ β τ A ( β ) | β =0 β /
2. For the Fermi process, the first orderterm is then given by [13] (cid:20) ∂∂β τ A (cid:21) β =0 = − u N ( N − N + N − , (23)where u stems from π A − π B = u i + v , compare App.II A. The first order expansion of τ A is only proportionalto the i dependent term u in this special case. This canalso be seen from a symmetry argument [41, 43]: Since τ A = τ BN − , the fixation time does not change under a ↔ d and b ↔ c . Since u , but not v , is invariant underthis exchange of strategy names, τ A can depend underlinear weak selection only on u , but not on v . The second order term of the conditional fixation time for the Fermiprocess yields (cid:20) d dβ τ A (cid:21) β =0 = E u + E uv + E v , (24)where E = − ( N − N − N − N + 177 N + 59 N ) ,E = − N (6 − N + N )18 , (25) E = 1 N E . Now, in contrast to the first order expansion Eq. (23),both u and v enter. An interesting relation is E = E /N . In the following, we show that this is found forany pairwise comparison process and not only in the spe-cial case of the Fermi process.For general pairwise comparison processes under neu-tral selection, the conditional fixation time is τ A (0) = N ( N − /g (0), where g (0) >
0. When selection acts,Eq. (8), the transition probabilities become dependenton the derivative of the imitation function, g (cid:48) (0) ≥ β . In general,the first order term in β reads ∂∂β τ A = (cid:88) | α | =1 N − (cid:88) k =1 k (cid:88) l =1 h α , (26) h α = (cid:18) ∂ α ∂β α T + i (cid:19)(cid:18) ∂ α ∂β α φ l (cid:19)(cid:32) ∂ α ∂β α k (cid:89) m = l +1 T − m T + m (cid:33) (27)with the multi–index α = ( α , α , α ), | α | = α + α + α ,see App. B 1 for details of the calculation. The generalstructure of this term is determined by h α , which is lin-ear in u and v , as | α | equals one. Thus, ∂ β τ A | β =0 = F u + F v is also of this form, where F and F only de-pend on the population size N . With the same symmetryargument as above, based on [41, 43], we can concludethat F = 0. This yields τ A = τ BN − ≈ N ( N − g (0) + F u β. (28)We can now calculate the payoff independent term F forany g (∆ π ) from the special case u = 1 and v = 0, whichreads F = − g (cid:48) (0) g (0) N ( N − N + N − . (29)Here, β can be rescaled by g (cid:48) (0) /g (0) . Changing g (cid:48) (0) or g (0) is equivalent to changing the selection intensity ap-propriately. In particular, when u >
0, which is true e.g.for coordination games such as the stag–hunt game [44],the conditional time it takes on average for a mutant6ype to take over decreases with the intensity of selec-tion. Moreover, for a > c and b > d in combination with u <
0, a mutant which is always advantageous over thewild type needs longer to reach fixation than a neutralmutant. This phenomenon, termed stochastic slowdownin [45], occurs in any imitation process, since Eq. (28)only depends on u .For the second order term in the expansion in β we canwrite ∂ ∂β τ A = (cid:88) | α | =2 N − (cid:88) k =1 k (cid:88) l =1 h α , (30) h α is of the form G u + G uv + G v . Thus ∂ β τ A | β =0 is also of this form, where the G i ’s only depend on N .Again, we consider the transformation a ↔ d and b ↔ c which corresponds to exchanging the names of the strate-gies. For the transformed game, we obtain ∂ β τ BN − | β =0 = G u + G u ˜ v + G ˜ v with ˜ v = ( N c − N a − d + a ) / ( N − G u ( v − ˜ v ) + G ( v − ˜ v ) = 0.With v + ˜ v = − N u , we then get G = G /N — thesymmetry discussed above for a special case holds for anyimitation function. Eventually, the second order term in β for general imitation probability is given by ∂ ∂β τ A = G u + G uv + G N v , (31)The special cases u = 1 , v = 0, as well as u = 0 , v = 1allow to compute G and G explicitly. Thus we have(see Appendix B 1) G = − ( N − N − N − N + 177 N + 59 N ) (cid:18) g (cid:48) (0)) g (0) (cid:19) − N ( N − N − (cid:18) g (cid:48)(cid:48) (0) g (0) (cid:19) , (32) G = − N (6 − N + N )18 (cid:18) g (cid:48) (0)) g (0) (cid:19) − N ( N − g (cid:48)(cid:48) (0) g (0) . (33)Obviously, Eq.(31) does not allow a rescaling of the inten-sity of selection. Instead, the properties of the imitationfunction enter in a more intricate way. An example ofthis approximation is shown in Fig. 3. B. Moran process
To close this section, we consider the Moran process,where selection at birth is proportional to fitness andselection at death is random. For neutral selection β = 0,it is well known that τ A (0) = N ( N −
1) [13, 33, 41]. Whenselection is weak β (cid:28)
1, the conditional mean fixationtime is approximately τ A ≈ τ A (0) + ∂ β τ A | β =0 β . For theMoran process with linear fitness function, f A = 1+ βπ A ,we have ∂ β τ A | β =0 = − u N ( N − N + 2) /
36, compare[13, 43]. The first order expansion of τ A again dependsonly on u , but not on v . This can be shown based on[41, 43] or explicitly [13].With general fitness mapping f ( βπ ) with transitionrates (15) and (16), we have (cid:20) ∂∂β τ A ( β ) (cid:21) β =0 = − f (cid:48) (0) N N − N + 236 u, (34)which allows a rescaling of the intensity of selection when τ A is approximated up to linear order.With general fitness function f ( x ), it becomes un-wieldy to calculate higher order terms in β . However, the general calculations are similar to that of the generalpairwise comparison rules. Eq. (19) reveals that alreadythe second order expansion of the fixation probability φ with general fitness mapping is tedious in form. Thusthe equivalent terms for the fixation time τ A are evenmore complicated and do not lead to further insight inthis case. Since it would be only an academic exerciseto calculate them, we do not give them explicitly here.It is clear that the weak selection approximation is notuniversal over a large class of processes in second orderin the fixation times. IV. DISCUSSION
In the past years, weak selection has become an impor-tant approximation in evolutionary game theory [9–15].Weak selection means that the game has only a smallinfluence on evolutionary dynamics. In evolutionary bi-ology and population genetics, the idea that most muta-tions confer small selective differences is widely accepted.In social learning models, it refers to a case where imita-tion is mostly random, but there is a tendency to imitateothers that are more successful. Since weak selection isthe basis of many recent results in evolutionary dynam-ics [10, 11, 46–48], it is of interest how universal theseresults are. It has been shown that they are remarkablyrobust and the choice of evolutionary dynamics has only7 "!!! !"!! ! " $ % &’ ( )* + , "-* ’ . ’ "-% & +/ ’ % . ’ "- +. ’ $ ) + ! " ! ! " ! !" !"!!! !"!! ! " $ % &’ ( )* + , "-* ’ . ’ "-% & +/ ’ % . ’ "- +. ’ $ ) + ! " ! " ! " ! $ % !" !" ( FIG. 3: Weak selection approximation of the conditional fix-ation time of a single mutant, the exact result is given byEq. (21). Upper panel: The approximations are shown forthe Fermi process, but they would be identical up to secondorder for any other pairwise comparison process after appro-priate rescaling of the selection intensity. Lower panel: Forany Moran process the first order approximation is indepen-dent of the precise function mapping payoff to fitness (hereit is linear). Any higher order approximation depends on thedetails of the function. Note that the first order approxima-tion in the two panels is not identical due to a difference inthe dependence on population size N (same parameters as inFig. 2) FIG. 3: Weak selection approximation of the conditional fix-ation time of a single mutant, the exact result is given byEq. (21). Upper panel: The approximations are shown forthe Fermi process, but they would be identical up to secondorder for any other pairwise comparison process after appro-priate rescaling of the selection intensity. Lower panel: Forany Moran process the first order approximation is indepen-dent of the precise function mapping payoff to fitness (hereit is linear). Any higher order approximation depends on thedetails of the function. Note that the first order approxima-tion in the two panels is not identical due to a difference inthe dependence on population size N (same parameters as inFig. 2) a small impact in unstructured populations [28, 49]. Instructured populations, however, the choice of evolution-ary dynamics can have a crucial impact on the outcome[11, 47, 50–54]. For example, for a prisoner’s dilemmaon a graph under weak selection, cooperation may be fa-vored by a death birth process while it is never favored bya birth death process. In a well mixed population, how-ever, the transition probabilities for those two processes are identical, thus they lead to the same result. However,in general spatial structure has a less pronounced effectunder weak selection than under strong selection [53, 54].We have addressed to what extent two evolutionaryprocesses can be considered as identical by investigat-ing the fixation probability and the fixation time. Forany given 2 × × × Acknowledgement
B.W. gratefully acknowledges the financial supportfrom China Scholarship Council (2009601286). L.W.acknowledges support by the National Natural ScienceFoundation of China (10972002 and 60736022). P.M.A.and A.T. acknowledge support by the Emmy-Noetherprogram of the Deutsche Forschungsgemeinschaft.8 ppendix A: Third order expansion of the fixation probabilities
Here, we expand the fixation probability φ for general birth–death processes up to the third order. Let γ i = T − i /T + i and (cid:20) ∂ s ∂β s γ i (cid:21) β =0 = p si . (A1)Note that the first index of p si refers to the order of the derivative and the second index gives the position in statespace. We expand Eq.(4) to the third order under weak selection γ i ≈ p i β + p i β / p i β /
6. Hence, we have k (cid:89) i =1 γ i ≈ k (cid:88) j =1 p j (cid:124) (cid:123)(cid:122) (cid:125) L k β + k (cid:88) j =1 ( p j − p j ) + k (cid:88) j =1 p j (cid:124) (cid:123)(cid:122) (cid:125) L k β k (cid:88) j =1 p j + 3 k (cid:88) j =1 p j (cid:32) k (cid:88) s =1 p s (cid:33) − k (cid:88) j =1 p j p j (cid:124) (cid:123)(cid:122) (cid:125) L k β . (A2)Then the fixation probability can be written as φ ≈ N + β N − (cid:88) k =1 L k (cid:124) (cid:123)(cid:122) (cid:125) Q + β N − (cid:88) k =1 L k (cid:124) (cid:123)(cid:122) (cid:125) Q + β N − (cid:88) k =1 L k (cid:124) (cid:123)(cid:122) (cid:125) Q − (A3) ≈ N − Q N β + (cid:20) Q N − Q N (cid:21) β − (cid:20) Q N − Q Q N + Q N (cid:21) β . (A4)This now serves as a starting point for our particular processes with certain choices of γ i = T − i /T + i and particular p si resulting from this.
1. General pairwise comparison process
For general switching probabilities in a pairwise comparison process, we have p i = − g (cid:48) (0) g (0) ∆ π i , (A5) p i = (cid:18) g (cid:48) (0) g (0) ∆ π i (cid:19) (A6) p i = − g (cid:48) (0)) − g (0) g (cid:48) (0) g (cid:48)(cid:48) (0) + g (0) g (cid:48)(cid:48)(cid:48) (0) g (0) (∆ π i ) . (A7)9nserting these quantities into Eqs.(A2) and (A3) leads to Q = − g (cid:48) (0) g (0) N − (cid:88) k =1 k (cid:88) i =1 ∆ π i , (A8) Q = (cid:18) g (cid:48) (0) g (0) (cid:19) N − (cid:88) k =1 (cid:32) k (cid:88) i =1 ∆ π i (cid:33) , (A9) Q = 2 6( g (cid:48) (0)) + 3 g (0) g (cid:48) (0) g (cid:48)(cid:48) (0) − g (0) g (cid:48)(cid:48)(cid:48) (0) g (0) N − (cid:88) k =1 k (cid:88) i =1 (∆ π i ) − g (cid:48) (0)) g (0) N − (cid:88) k =1 (cid:32) k (cid:88) i =1 ∆ π i (cid:33) (cid:32) k (cid:88) s =1 (∆ π s ) (cid:33) . (A10) Q and Q have been calculated in the main text. Note that they only depend on g (cid:48) (0) /g (0), whereas Q also dependson higher order derivatives of the imitation function. Thus, two pairwise comparison processes that are identical infirst order are also identical in second order. Only in third order, differences start to emerge.Let us briefly come back to our example of an imitation function that violates the 1 / g ( x ) = (1+exp( − x )) − .In this case, we have g (0) = 1 / g (cid:48) (0) = g (cid:48)(cid:48) (0) = 0 and g (cid:48)(cid:48)(cid:48) (0) = 3 /
2. Thus, both Q , and Q vanish and the thirdorder expansion of the fixation probability is φ ≈ N + N − N (cid:2) ( N + 1)(3 N − u + 15( N + 1) N u v + 30( N + 1) uv + 30 v (cid:3) β . (A11)
2. Moran processes
For Moran processes with general fitness functions, we have p i = − f (cid:48) (0)∆ π i and p i = 2( f (cid:48) (0)) π A ∆ π i − f (cid:48)(cid:48) (0)( π A + π B )∆ π i . Inserting these quantities into Eqs.(A2) and (A3) leads to Q = − f (cid:48) (0) N − (cid:88) k =1 k (cid:88) i =1 ∆ π i ,Q = (cid:0) ( f (cid:48) (0)) − f (cid:48)(cid:48) (0) (cid:1) N − (cid:88) k =1 k (cid:88) i =1 ( π A − π B ) + ( f (cid:48) (0)) N − (cid:88) k =1 (cid:32) k (cid:88) i =1 ∆ π i (cid:33) . (A12)Thus, the first and the second order expansion of the fixation probability of such processes are given by Eqs.(18) and(19), respectively. In particular for f ( π ) = 1 + π , both p i and p i vanish and p i = − π A − π B ). By Eq.(A3), thisyields φ = 1 N + 1 N N − (cid:88) k =1 k (cid:88) i =1 ( π A − π B ) (cid:124) (cid:123)(cid:122) (cid:125) D β + o ( β ) (A13)where D = N ( N − (cid:16) − c d ( N − N )(2 N − − cd ( N − N + 1)(3 N −
4) + 6 a b ( N − N − N + 2) + a ( a + 3 b )( N − N − N + 1) − c (1 + N )(3 N −
2) + 2 b (1 + N − N + 6 N ) − d ( N − − N + 12 N ) (cid:17) . Appendix B: Times of fixation
General expressions for the first and second order expansion of the fixation time for the birth–death process havebeen given in Eq(26) and Eq.(30). Based on these, we show the results for the general pairwise comparison rule firstand then discuss the Moran process. 10 . General pairwise comparison process
For the first order term of the fixation time, Eq.(26), each h α on the rhs. is proportional to g (cid:48) (0) /g (0). Thus, thefirst order term of the fixation time is of the form Rg (cid:48) (0) /g (0). In particular, when g (∆ π ) is the Fermi function, g (cid:48) (0) /g (0) is one. Hence the first order of the fixation time for the Fermi process is R , cf. Eq.(23). This leads to thefirst order expansion of the fixation time for general pairwise comparison rule, Eq.(29).For the second order, we write Eq. (30) explicitly as ∂ ∂β τ A = N − (cid:88) k =1 k (cid:88) l =1 h (2 , , (cid:124) (cid:123)(cid:122) (cid:125) K + N − (cid:88) k =1 k (cid:88) l =1 h (0 , , (cid:124) (cid:123)(cid:122) (cid:125) K + N − (cid:88) k =1 k (cid:88) l =1 h (0 , , (cid:124) (cid:123)(cid:122) (cid:125) K + 2 N − (cid:88) k =1 k (cid:88) l =1 h (1 , , (cid:124) (cid:123)(cid:122) (cid:125) K + 2 N − (cid:88) k =1 k (cid:88) l =1 h (1 , , (cid:124) (cid:123)(cid:122) (cid:125) K + 2 N − (cid:88) k =1 k (cid:88) l =1 h (0 , , (cid:124) (cid:123)(cid:122) (cid:125) K . (B1)As shown in the main text, the second order term is of the form of G u + G uv + G N v . Letting u = 1 and v = 0leads to K = N ( N − N − g (cid:48) (0)) − g (0) g (cid:48)(cid:48) (0) g (0) K = − N ( N − N − N + 16 N )2700 2( g (cid:48) (0)) g (0) K = N ( −
120 + 4 N + 350 N − N − N + 121 N )1800 2( g (cid:48) (0)) g (0) K = − N ( N − g (cid:48) (0)) g (0) K = N (2 − N + N )9 2( g (cid:48) (0)) g (0) K = − N (2 + 25 N − N − N + 13 N )180 2( g (cid:48) (0)) g (0) (B2)after some tedious calculations using the identity (cid:80) Mk =1 (cid:80) kl =1 = (cid:80) Ml =1 (cid:80) Mk = l [56]. Summing these K i ’s leads to G inEq.(32). On the other hand, letting u = 0 and v = 1 yields K = N ( N −
1) 2( g (cid:48) (0)) − g (0) g (cid:48)(cid:48) (0) g (0) K = N ( N − N − g (cid:48) (0)) g (0) K = N (4 N − N + 17 N − g (cid:48) (0)) g (0) K = − N ( N − g (cid:48) (0)) g (0) K = N ( N − N − g (cid:48) (0)) g (0) K = − N ( N − N − g (cid:48) (0)) g (0) (B3)Adding these K i ’s yields G /N as in Eq. (33). Thus, the quantities in Eq. (31) are finally derived.
2. Moran processes
For Moran processes, the approach is fully equivalent to pairwise comparison processes. However, the results donot only depend on payoff differences u and v , but on the full payoff matrix with entries a , b , c , and d . This makes11he calculations a matter of diligence and leads to quite long expressions, but not to additional insights. Thus, we donot give details of the derivation here. [1] J. Maynard Smith and G. R. Price, Nature , 15(1973).[2] P. D. Taylor and L. Jonker, Math. Biosci. , 145 (1978).[3] J. Hofbauer, P. Schuster, and K. Sigmund, J. Theor.Biol. , 609 (1979).[4] E. C. Zeeman, in Lecture Notes in Mathematics , 471(1980).[5] J. Hofbauer and K. Sigmund,
Evolutionary Gamesand Population Dynamics (Cambridge University Press,Cambridge, 1998).[6] G. Fogel, P. Andrews, and D. Fogel, Ecol. Model. ,283 (1998).[7] S. Ficici and J. Pollack, Effects of finite populations onevolutionary stable strategies., in
Proceedings GECCO ,edited by D. Whitley et al. , pp. 927–934, Morgan-Kaufmann, San Francisco, 2000.[8] S. Schreiber, Siam J. Appl. Math. , 2148 (2001).[9] M. A. Nowak, A. Sasaki, C. Taylor, and D. Fudenberg,Nature , 646 (2004).[10] A. Traulsen and M. A. Nowak, Proc. Natl. Acad. Sci.USA , 10952 (2006).[11] H. Ohtsuki, C. Hauert, E. Lieberman, and M. A. Nowak,Nature , 502 (2006).[12] H. Ohtsuki, M. A. Nowak, and J. M. Pacheco, Phys. Rev.Lett. , 108106 (2007).[13] P. M. Altrock and A. Traulsen, New J. Physics ,013012 (2009).[14] S. Kurokawa and Y. Ihara, Proc. R. Soc. B , 1379(2009).[15] C. S. Gokhale and A. Traulsen, Proc. Natl. Acad. Sci.U.S.A. , 5500 (2010).[16] H. Ohtsuki, J. Theor. Biol. , 136 (2010).[17] M. Kimura, Nature , 624 (1968).[18] T. Ohta, Proc. Natl. Acad. Sci. USA , 16134 (2002).[19] H. Akashi, Genetics , 1067 (1995).[20] J. Charlesworth and A. Eyre-Walker, Proc. Natl. Acad.Sci. U.S.A. , 16992 (2007).[21] A. Traulsen, D. Semmann, R. D. Sommerfeld, H.-J.Krambeck, and M. Milinski, Proc. Natl. Acad. Sci.U.S.A. , 2962 (2010).[22] A. Traulsen, Evolution , 316 (2010).[23] A. Traulsen, J. C. Claussen, and C. Hauert, Phys. Rev.Lett. , 238701 (2005).[24] A. Traulsen, M. A. Nowak, and J. M. Pacheco, Phys.Rev. E , 011909 (2006).[25] H. Ohtsuki, P. Bordalo, and M. A. Nowak, J. Theor.Biol. , 289 (2007).[26] L. A. Imhof and M. A. Nowak, J. Math. Biol. , 667(2006).[27] A. Traulsen, J. M. Pacheco, and L. A. Imhof, Phys. Rev.E , 021905 (2006).[28] S. Lessard and V. Ladret, J. Math. Biol. , 721 (2007).[29] A. Ross-Gillespie, A. Gardner, S. A. West, and A. S. Griffin, Am. Nat. , 331 (2007).[30] I. Bomze and C. Pawlowitsch, J. Theor. Biol. , 616(2008).[31] W. Huang and A. Traulsen, J. Theor. Biol. , 262(2010).[32] N. Goel and N. Richter-Dyn, Stochastic Models in Biol-ogy (Academic Press, New York, 1974).[33] W. J. Ewens,
Mathematical Population Genetics (Springer, NY, 2004).[34] M. A. Nowak,
Evolutionary Dynamics (Harvard Univer-sity Press, Cambridge, MA, 2006).[35] L. E. Blume, Games and Economic Behavior , 387(1993).[36] G. Szab´o and C. T˝oke, Phys. Rev. E , 69 (1998).[37] R. J. Glauber, J. Math. Phys. , 294 (1963).[38] G. Szab´o and C. Hauert, Phys. Rev. Lett. , 118101(2002).[39] A. Traulsen, N. Shoresh, and M. A. Nowak, Bull. Math.Biol. , 1410 (2008).[40] S. Karlin and H. M. A. Taylor, A First Course in Stochas-tic Processes , 2nd edition ed. (Academic, London, 1975).[41] T. Antal and I. Scheuring, Bull. Math. Biol. , 1923(2006).[42] T. Maruyama and M. Kimura, Evolution , 161 (1974).[43] C. Taylor, Y. Iwasa, and M. A. Nowak, J. Theor. Biol. , 245 (2006).[44] B. Skyrms, The Stag-Hunt Game and the Evolutionof Social Structure (Cambridge University Press, Cam-bridge, 2003).[45] P. M. Altrock, C. S. Gokhale, and A. Traulsen, Phys.Rev. E , 011925 (2010).[46] M. A. Nowak, Science , 1560 (2006).[47] C. E. Tarnita, T. Antal, H. Ohtsuki, and M. A. Nowak,Proc. Natl. Acad. Sci. USA , 8601 (2009).[48] T. Antal, H. Ohtsuki, J. Wakeley, P. D. Taylor, and M. A.Nowak, Proc. Natl. Acad. Sci. USA , 8597 (2009).[49] T. Antal, A. Traulsen, H. Ohtsuki, C. E. Tarnita, andM. A. Nowak, J. Theor. Biol. , 614 (2009).[50] H. Ohtsuki and M. A. Nowak, Proc. Roy. Soc. Lond. B , 2249 (2006).[51] C. E. Tarnita, H. Ohtsuki, T. Antal, F. Fu, and M. A.Nowak, J. Theor. Biol. , 570 (2009).[52] M. A. Nowak, C. E. Tarnita, and T. Antal, Phil. Trans.Roy. Soc. London B , 19 (2010).[53] C. P. Roca, J. A. Cuesta, and A. S´anchez, Phys. Rev. E , 046106 (2009).[54] C. P. Roca, J. A. Cuesta, and A. S´anchez, Physics of LifeReviews (2009).[55] A. J. Bladon, T. Galla, and A. J. McKane, Phys. Rev.E , 066122 (2010).[56] R. L. Graham, D. E. Knuth, and O. Patashnik, ConcreteMathematics , second ed. (Addison-Wesley, 1994)., second ed. (Addison-Wesley, 1994).