Replicators in Fine-grained Environment: Adaptation and Polymorphism
aa r X i v : . [ q - b i o . P E ] M a y Replicators in Fine-grained Environment: Adaptation and Polymorphism ∗ Armen E. Allahverdyan and Chin-Kun Hu , Yerevan Physics Institute, Alikhanian Brothers Street 2, Yerevan 375036, Armenia Institute of Physics, Academia Sinica, Nankang, Taipei 11529, Taiwan Center for Nonlinear and Complex Systems and Department of Physics,Chung Yuan Christian University, Chungli 32023, Taiwan (Dated: November 12, 2018)Selection in a time-periodic environment is modeled via the two-player replicator dynamics. Forsufficiently fast environmental changes, this is reduced to a multi-player replicator dynamics in aconstant environment. The two-player terms correspond to the time-averaged payoffs, while thethree and four-player terms arise from the adaptation of the morphs to their varying environment.Such multi-player (adaptive) terms can induce a stable polymorphism. The establishment of thepolymorphism in partnership games [genetic selection] is accompanied by decreasing mean fitnessof the population.
PACS numbers: 87.23.-n, 87.23.Cc, 87.23.Kg, 02.50.Le
Environmental impact on adaptation, selection andevolution is an important subject of biological research[1–9]. The main scenario of environmental adaptationon the population level is polymorphism [1–4]: two ormore clearly different types of phenotype (morph) existin one interbreeding population. The basic mechanismsof polymorphism are heterozygote advantage and inho-mogeneous (frequency, space and/or time-dependent) en-vironment [2, 4]. Polymorphism can be restricted to thephenotype level, or it may be controlled genetically bymultiple alleles at a single locus , e.g., human ABO bloodgroups [4]. Here is one example of polymorphism relatedto a time-dependent environment [4, 5]. Forrest popula-tions of the land snail
Cepaea Nemoralis consist of threemorphs having respectively brown, pink and yellow col-ored shells [4, 5]. The brown and pink morphs have anadvantage over the yellow morph at the spring time, sincethe background color makes them less visible for preda-tors [4, 5]; the yellow morph has an advantage at summerand autumn on the yellow-green substrate. In addition,the yellow morph is more resistant to high and low tem-peratures [4, 5]. Thus different morphs have differentadvantage under different environmental conditions [4].A varying environment has roughly three dimensions:it may be time or/and space dependent, predictable vs stochastic, and fine vs coarse-grained [1]. The lattermeans that each individual within population sees mainlyone fixed environment, which can change from one gen-eration to another, for example. A fine grained environ-ment changes many times during the life-time of eachindividual; see the above example of Cepaea and notethat this snail lives seven to eight years [4, 5].Much attention was devoted to modeling polymor-phism in various coarse-grained environments [1, 2, 6–9]. Fine-grained environments got less attention, since ∗ Published in Phys. Rev. Lett. , 058102 (2009) early theoretical results [1, 10] and the biological commonsense [2] implied that non-trivial polymorphism scenariosare absent. One expects that in this case the organismsees (thus adapts to) the average environment [1, 2, 10].However, recent experiments indicate that the evolvingpopulations can adapt to time-varying aspects of theirfine-grained environment [11–13]. In particular, they canrespond to the environmental patterns other than the en-vironmental mean [11]. Moreover, the total fitness dur-ing such an adaptation need not increase [13]. A propertheoretical model for such phenomena is still absent.Here we present a theory for polymorphism in fine-grained, time-periodic environment based on Evolution-ary Game Theory (EGT). Our main method is the time-scale separation in the replicator dynamics.EGT describes interacting agents separated into sev-eral groups [7, 14]. The reproduction of each group isgoverned by its fitness, which depends on interactionsbetween the groups. The most popular replicator dy-namics approach to EGT describes the time-dependentfrequency p k ( t ) of the group k , which is the number ofagents N k in the group k , over the total number of agentsin all n groups: p k = N k / P nk =1 N k . The fitness f k of thegroup k is a linear function of the frequencies [7, 14]: f k ( a, p ) = X nl =1 a kl p l , k = 1 , . . . , n, (1)where the payoffs a kl account for the interaction between(the agents from) groups k and l . The replicator dynam-ics [14] facilitates the (relative) growth of groups withfitness larger than the average fitness P nl =1 p l f l :˙ p k = p k [ f k ( a, p ) − X nl =1 p l f l ] ≡ G k [ a, p ] . (2)Within game theory the groups correspond to strategies,while the pay-offs a kl describe interaction between twoplayers: the probability p k of strategy k changes accord-ing to the average pay-off P l a kl p l received by one playerin response to applying the strategy k [7, 14].There are several applications of EGT and replicatordynamics in biology: i) Animal (agent) contests, wherethe groups correspond to the strategies of agent’s be-havior during the contest, while p k is the probability bywhich an agent applies the strategy k [14]. The actualmechanism by which p k changes depends on the concreteimplementation of the model (inheritance, learning, imi-tation, infection, etc). ii) Selection of genes, where p k isthe frequency of one-locus allele k in panmictic, asexual,diploid population, and where a kl = a lk refers to the se-lective value of the phenotype driven by the zygote ( kl )[7]. Then (1, 2) are the Fisher equations for the selec-tion with overlapping generations [7, 14]. iii) The basicLotka-Volterra equations of ecological dynamics can berecast in the form (2) and studied as replicators [14].Within the replicator approach polymorphism means astable state, where two or more p k are non-zero.We consider a varying, but predictable environment,which acts on the phenotypes making a kl periodic func-tions of time with period 2 π/ω [7, 15]: a kl ( τ ) = a kl ( τ +2 π ) , τ ≡ ωt . There are well-defined methods to decideto which extent a varying environment is predictable for agiven organism [9]. The oscillating payoffs can reflect thefact that different morphs (alleles, strategies) are dom-inating at different times. Let us assume that the en-vironment varies fast [fine-graining]: the average changeof the population structure over the environment period2 π/ω is small. We separate the time-dependent payoffs a kl ( τ ) into the constant part ¯ a kl and the oscillating part e a kl ( τ ) with zero time-average ¯ e a kl ≡ R π τ π e a kl ( τ ) = 0, a kl ( τ ) = ¯ a kl + e a kl ( τ ) , ∂ τ ˆ a kl ( τ ) = e a kl ( τ ) , ˆ a kl = 0 , (3)where ˆ a kl is defined to be the primitive of e a kl with itstime-average equal to zero.Following the Kapitza method [17], we represent p k asa slowly varying part ¯ p k plus ǫ k [¯ p ( t ) , τ ], which is smallerthan ¯ p k , fast oscillating on the environment time τ , andaveraging to zero: p k ( t ) = ¯ p k ( t ) + ǫ k [¯ p ( t ) , τ ] , (4)¯ ǫ k [¯ p ( t )] = Z π d τ π ǫ k [¯ p ( t ) , τ ] = 0 . (5)Here the average is taken over the fast time τ for a fixedslow time t . Note that the fast ǫ k depends on the slow ¯ p .Now put (4) into (2) and expand the RHS of (2) over ǫ :˙¯ p k + ˙¯ p α ∂ α ǫ k + ω∂ τ ǫ k = [1 + ǫ α ∂ α + O ( ǫ )] G k [ e a ( τ ) , ¯ p ] , (6)where the summation over the repeated Greek indices, isassumed and ∂ α X ≡ ∂∂ ¯ p α X . The fast factor ǫ i is searchedfor via expanding over ω : ǫ k = ω ǫ k, + ω ǫ k, + . . . . Sub-stitute this into (6), add and subtract suitable averages,and get for the fast terms: ∂ τ ǫ k, = G k [ e a ( τ ) , ¯ p ] + O ( ω ).After straightforward integration, we have ǫ k [¯ p ( t ) , τ ] = 1 ω G k [ˆ a ( τ ) , ¯ p ( t )] + O ( 1 ω ) , (7) ƒ ŸŸ Ÿ ƒƒ
10 p - Π Π FIG. 1: Schematic portrait of (15) for C = 0 [upper diagram]and for C satisfying conditions (17) [lower diagram]. Stable[unstable] fixed points are denoted by circles [squares]. Ar-rows indicate direction of flow in time. where ˆ a = { ˆ a kl } is defined in (3). Once (7) is separated,the remainder in (6) is the evolution of slow terms˙¯ p k = G k [¯ a, ¯ p ] + ǫ α [¯ p, τ ] ∂ α G k [ e a ( τ ) , ¯ p ] + O (1 /ω ) , (8)where the time-average is defined as in (5). Working out(8) and using ˆ a e b = − e a ˆ b we get again a replicator equation˙¯ p k = ¯ p k [ F k (¯ p ) − ¯ p α F α (¯ p )] , (9) F k (¯ p ) = ¯ a kα ¯ p α + b kαβ ¯ p α ¯ p β + c kαβγ ¯ p α ¯ p β ¯ p γ (10)where F k is the effective (already non-linear) fitness, and b klm ≡ ω e a kl ˆ a lm , c klmn ≡ ω ˆ a kl e a [s] mn . (11)Here we defined a [s] kl ≡ ( a kl + a lk ). Equations (9, 10)are our central result. Expectedly, the fast environmentcontributes the averaged payoffs ¯ a kl into F k . This is well-known for any fine-grained environment [1].However, besides this averaged two-party interaction,each group k gets engaged into three- and four-party in-teractions with payoffs b klm and c klmn , respectively. In-deed, recalling our discussion after (2), we can interpret P lm b klm ¯ p l ¯ p m in (10) as the average pay-off received byone of three players upon applying strategy k . The termswith b klm and c klmn in (10) exist due to adaptation ofthe morphs to their environment: while the frequenciesof the morphs fast oscillate on the environmental time,see (5), on longer times the population sees the fast en-vironment as an effective many-player model. Note thatthe terms with b klm and c klmn need not be small as com-pared to ¯ a kl -terms, since the derivation of (9–11) appliesfor ¯ p k ≫ ǫ k , which can hold even for ¯ a kl → b klm = c klmn = 0 [due to ˆ a kl e a kl = 0], if only one a kl varies in time, or if all e a kl oscillate at one phase: e a kl = a ( t ) ξ kl , where ξ kl are constant amplitudes. Indeed, theadaptive terms [with b klm and c klmn ] are non-zero due tointerference between the environmental oscillations of e a kl and those of ǫ k , which are delayed over the environmentaloscillations by phase π/
2; see (7). This is why there is nointerference if all e a kl oscillate in phase. Thus for havingthe adaptive terms we need at least two morphs reactingon the environment differently , e.g., due to different delaytimes of reaction. For the adaptive terms we also needthe frequency-dependent selection, e.g., they are absentfor haploid replicators ˙ p k = p k [ a k ( t ) − a α ( t ) p α ].Euation (9) implies that the relative growth of twomorphs at slow (long) times is determined by the ef-fective fitness difference: dd t (¯ p k / ¯ p l ) = (¯ p k / ¯ p l )( F k − F l ).Thus in the stable fixed points of (9) the (effective) fit-ness of surviving morphs are equal to each other, whilethe fitness of non-surviving (¯ p k →
0) morphs is smaller(Nash equilibrium) [14]. Another pertinent quantity, the(fast) time-averaged fitness ¯ f k [ p ( t )] is the cumulative ef-fect of the short-time replication intensities. Employing(4, 5, 7) we deduce that even when the adaptive termsare taken into account, these two quantities are equal F k = f k (¯ a + e a, ¯ p + ǫ ). The overall long-time fitness of thepopulation is characterized by the effective mean fitness¯ p α F α , which is also equal to its time-averaged analog:Φ ≡ (¯ p α + ǫ α )(¯ a αβ + e a αβ )(¯ p β + ǫ β ) = ¯ p α F α = ¯ a αβ ¯ p α ¯ p β + b αβγ ¯ p α ¯ p β ¯ p γ . (12)The contribution c αβγδ ¯ p α ¯ p β ¯ p γ ¯ p δ in ¯ p α F α nullifies due to(11) and ˆ a e b = − e a ˆ b .The mean fitness Φ is especially important for part-nership games [genetic selection]: a ik ( τ ) = a ki ( τ ), sincefor the constant payoff situation in the replicator equa-tion (2), Φ monotonically increases towards its nearestlocal maximum over the set of variables ¯ p k (fundamen-tal theorem of natural selection) [7, 14]. As follows from(11, 12), for a ik ( τ ) = a ki ( τ ), Φ reduces to the averagedtwo-player contribution: Φ = P αβ ¯ p α ¯ a αβ ¯ p β . However,the theorem is not valid in the presence of the adaptiveterms, and the mean fitness can decrease; see below.Let us now study concrete examples. For n = 2, Eq. (2)simplifies to a closed equation for the frequency p ˙ p = p (1 − p )[ A ( t ) − B ( t ) p ] , (13) A ( t ) ≡ a ( t ) − a ( t ) , B ( t ) ≡ a ( t ) − a ( t ) − a ( t ) , where without much loss of generality we adopted a ( τ ) = a ( τ ). Using notations (3) and defining C ≡ ω ˆ A e B = 1 ω [ ˆ a ( e a − e a ) + ˆ a e a ] , (14)for the adaptive factor, we deduce from (7, 8, 13)˙¯ p = ¯ p (1 − ¯ p )[ ¯ A − ¯ B ¯ p − C ¯ p (1 − ¯ p )] . (15) C vanishes for the symmetric homozygotes, a ( τ ) = a ( τ ), and for one recessive allele, e.g. a ( τ ) = a ( τ ).Both cases are easily solvable from (13) showing that thelong-time behavior of p is indeed governed by ¯ A .The vertices ¯ p = 1 and ¯ p = 0 are always fixed pointsof (15), while two interior fixed points are π , = 12 C [ ¯ B + C ∓ q ( ¯ B + C ) − AC ] , π < π . (16) - - Ÿ ƒ FIG. 2: The portrait of (18) for ¯ a = 0 .
1, ¯ a = 0 . a = − . κ = − .
65 and κ = 0 .
65; ¯ p and ¯ p are re-stricted by 0 ≤ ¯ p ≤
1, 0 ≤ ¯ p ≤
1, ¯ p + ¯ p ≤
1. Twofixed points are denoted by square (saddle) and cycle (cen-ter). The closed orbits contain the center in their interior;orbits from the second class converge to ¯ p = 1. These twoclasses of orbits are separated by a dashed curve (separatrix),which is made by joining together two unstable directions ofthe saddle. Arrows indicate direction of flow. If π and π are in (0 , π ( π ) is stable (unstable).The analysis of (15) reduces to the following scenarios. For ¯
A >
A > ¯ B [i.e., ¯ a > ¯ a > ¯ a ] themorph 1 globally dominates for C = 0, i.e., for all initialconditions ¯ p goes to 1 for large times; see Fig. 1. Theglobal dominance does not change for C <
0. One cancall this morph generalist [2], since its fitness does notoscillate in time ( ǫ k in (7) is zero for ¯ p = 1), and itsfitness is maximal; see also below. For C > ¯ B > B + C ) ≥ AC, (17)both π and π fall into the interval (0 , π and π are not in this interval. Thus if (17) hold, a stablefixed point π emerges, which attracts all the trajectoriesthat start from ¯ p (0) < π : the polymorphism is createdby the adaptation term ∝ C in (15). Initial conditionlarger than the unstable fixed point π , ¯ p (0) > π , stilltend to ¯ p = 1; see Fig. 1. Both stable fixed points π and 1 are Evolutionary Stable States (ESS), mean-ing that they cannot be invaded by a sufficiently smallmutant population [14]. The coexistence of two ESS oneof which is interior (i.e., polymorphic) is impossible fora two-player replicator equation with constant pay-offs[14], but it is possible for multi-player replicator equa-tion [16]. We thus saw above an example of this behaviorinduced by time-varying environment.As we discussed below (12), the mean fitness does notcontain the adaptive terms directly and is given as Φ =2 ¯ A ¯ p − ¯ B ¯ p (up to an irrelevant constant). For ¯ A >
A > ¯ B , Φ maximizes at ¯ p = 1, and this maximumis the only stable fixed point of the replicator dynamics(15) with C = 0. If however C satisfies conditions (17),in the stable fixed point ¯ p = π the mean fitness Φ issmaller than at the stable point ¯ p = 1. Moreover, forthe initial conditions π < ¯ p (0) < π , the mean fitnessΦ decreases in the course of the relaxation to π .The quantity which is increased by dynamics (15) isΨ = 2 ¯ A ¯ p − ¯ B ¯ p − C ¯ p (1 − p − C ¯ p (1 − p . Thus Ψ is the Lyapunov function: ˙Ψ ≥
0. ThoughΨ − Φ < C < minus a positive risk aversion factor; see [18] for reviews. For ¯
B > ¯ A > a > ¯ a , ¯ a ] and C = 0 thereis a stable polymorphism at the fixed point ¯ p = ¯ A/ ¯ B (heterozygote advantage). The presence of C = 0 in (15)does not change this polymorphism; only the value of thefixed point shifts to π . In contrast to the scenario , herethe response to slow environmental changes is reversible. For ¯
B < ¯ A < C = 0 there is an unstablepolymorphism: all the initial conditions with ¯ p (0) < ¯ A/ ¯ B end up at p = 0 (morph 2 dominates), while thosewith p (0) > ¯ A/ ¯ B finish at p = 1 (morph 1 dominates).Now C = 0 in (15) shifts the unstable fixed point to π .These are all possible scenarios for n = 2; other rela-tions between ¯ A and ¯ B lead to interchanging morphs.For three morphs, n = 3, we assume the zero-sum sit-uation in (2), a kl ( τ ) = − a lk ( τ ) [19]: the loss of the strat-egy l is equal to the gain of k . Equations (9–11) reduceto ˙¯ p i = ¯ p i ¯ a iα ¯ p α + κ i ¯ p ¯ p ¯ p , κ = 2 b , κ = 2 b , (18)where P i =1 κ i = 0. The four-party contribution disap-pears from (18). One can show that any interior fixedpoint of (18) can be either saddle (two real eigenvaluesof the Jacobian with different sign) or center (two imag-inary, complex conjugate eigenvalues). For ¯ a >
0, ¯ a > κ i = 0 the morph 1globally dominates: ¯ p = 1 is the only stable fixed point.For the existence of polymorphism it is necessary that κ <
0, i.e., the strategies 2 and 3 together win over 1,although separately they lose to 1. The dominance of 1 isstill kept when ¯ p or ¯ p are forced to decay, because thenthe adaptive term in (18) is irrelevant for sufficiently largetimes. This happens when ¯ a < κ < a > κ <
0. Apart from these cases the terms ∝ κ i in (18) can lead to polymorphism, provided that theirmagnitude is large enough; see Fig. 2. Besides the stablefixed point ¯ p = 1 of the κ i = 0 dynamics, two new fixedpoints emerge: stable (center) and unstable (saddle). Adomain around the saddle supports polymorphism withcyclic dominance of the morphs; see Fig. 2. We see ageneral feature of all the above examples: the adaptive(multi-party) terms do not influence the local stability ofthe vertices (where all but one ¯ p k ’s are zero). For ¯ a > , ¯ a < , ¯ a > κ i = 0: 1 wins over 2, which wins over 3, but 3 wins over 1 (rock-scissor-paper game). Now for κ i = 0 in (18) there is already one interior fixed point,and the trajectories are closed orbits around this fixedpoint. After including the adaptive ( ∝ κ k ) terms in (18)this fixed point is simply shifted, and no new fixed pointsappear for any size or magnitude of κ i . To summarize , we have shown that in addition to theaveraged payoffs, a fast [fine-grained] time-periodic en-vironment generates adaptive, multi-player terms in thereplicator dynamics, provided that at least two morphsreact on the environment differently. These terms cancreate a polymorphic stable state via adaptation of the“weak” morphs to environmental changes. This polymor-phism is related to decreasing mean fitness of the pop-ulation. This specific aspect of the polymorphism wasargued to be a prerequisite for the phenomenon of sym-patric speciation, where by contrast to the allopatric sce-nario the speciation is induced inside a single population[20]. Thus, our results hint at a sympatric speciation sce-nario due to a fine-grained, time-periodic environment.We thank M. Broom and K. Petrosyan for discussions,and K.-t. Leung for critical reading. The work wassupported by Volkswagenstiftung, grants NSC 96-2911-M 001-003-MY3 & AS-95-TP-A07, and National Centerfor Theoretical Sciences in Taiwan. [1] R. Levins,
Evolution in Changing Environments (Prince-ton University Press, 1968).[2] P.W. Hedrick et al. , Ann. Rev. Ecol. Syst. , 1 (1976);P.W. Hedrick, ibid , 535 (1986); ibid , 67 (2007).[3] L.A. Meyers and J.J. Bull, Tr. Ecol. Evol , 551 (2002).[4] V. Grant, Organismic Evolution (Freeman, SF, 1977).[5] L.M. Cook, Phil. Trans. R. Soc. B , 1577 (1998).[6] E. Dempster, Cold Spring Harbor Symp. Quant. Biol. ,25 (1955). J.B.S. Haldane and S.D. Jayakar, J. Genet. ,237 (1963). J.L. Cornette, J. Math. Biol. , 173 (1981).[7] Yu.M. Svirezhev and V.P. Passekov, Findamentals ofMathematical Genetics (Dordrecht, Kluwer, 1990).[8] J.H. Gillespie,
The Causes of Molecular Evolution (Ox-ford Univ. Press, Oxford, 1991).[9]
Lecture Notes on Biomathematics: Adaptation inStochastic Environments , ed. J. Yoshimura and C.W.Clark (Springer-Verlag, Berlin, 1991).[10] C. Strobeck, Am. Nat. , 419 (1975).[11] B.G. Miner and J.R. Vonesh, Ecol. Lett. , 794 (2004).[12] A. Winn, Evolution , 1111 (1996).[13] J.-N. Jasmin and R. Kassen, Proc. R. Soc. B , 2761(2007).[14] J. Hofbauer and K. Sigmund, Evolutionary Games andPopulation Dynamics (Cambridge Univ. Press, 1998).[15] M. Broom, Comp. Rend. Biol. , 403 (2005).[16] M. Broom, et al. , Bull. Math. Biol. , 931 (1997).L.A. Bach et al. , J. Theor. Biol. , 426 (2006).[17] L. D. Landau and E. M. Lifshitz, Mechanics (PergamonPress, Oxford, 1976).[18] S.C. Stearns, J. Biosci. , 221 (2000).[19] E. Akin and V. Losert, J. Math. Biology , 231 (1984). [20] M. Doebeli and U. Dieckmann, Am. Nat. , S77(2000); J. Evol. Biol.18