Stochasticity enhances the gaining of bet-hedging strategies in contact-process-like dynamics
SStochasticity enhances the gaining of bet-hedgingstrategies in contact-process-like dynamics
Jorge Hidalgo , Simone Pigolotti , and Miguel A. Mu˜noz ∗ Departamento de Electromagnetismo y F´ısica de la Materia,and Instituto Carlos I de F´ısica Te´orica y Computacional,Universidad de Granada, 18071 Granada, Spain. Departament de Fisica i Enginyeria Nuclear, Universitat Politecnica de Catalunya,Rambla Sant Nebridi 22, 08222 Terrassa, Barcelona, Spain. (Dated: November 10, 2018)In biology and ecology, individuals or communities of individuals living in unpredictable envi-ronments often alternate between different evolutionary strategies to spread and reduce risks. Suchbehavior is commonly referred to as “bet-hedging.” Long-term survival probabilities and populationsizes can be much enhanced by exploiting such hybrid strategies. Here, we study the simplest pos-sible birth-death stochastic model in which individuals can choose among a poor but safe strategy,a better but risky alternative, or a combination of both. We show analytically and computationallythat the benefits derived from bet-hedging strategies are much stronger for higher environmentalvariabilities (large external noise) and/or for small spatial dimensions (large intrinsic noise). Thesecircumstances are typically encountered by living systems, thus providing us with a possible justifi-cation for the ubiquitousness of bet-hedging in nature.
I. INTRODUCTION
In natural environments, individuals have to chooseamong a variety of evolutionary strategies, characterizedby different payoffs and risks, which, in their turn, maychange with time. Particularly relevant is the case inwhich a choice is to be made between a relatively safestrategy, with a low but stable payoff, and a potentiallyvery productive, but risky, variable strategy. Micro-organisms able to metabolize two resources [1–4], one ofthem consistently available at a fixed though low level,and the second, more abundant on average but fluctu-ating in time, constitute an example of this. In the ab-sence of specific knowledge about environmental condi-tions, individuals need to make a blind decision aboutwhether to specialize in consuming only one resource or,instead, develop a hybrid “bet-hedging” strategy by al-ternating both options. Similar forms of bet-hedging canalso emerge in cases where limited information from sen-sory systems is available [5, 6]. Finally, bet-hedging canbe exploited at a community level –rather than on anindividual basis– by developing, for example, phenotypicvariability among individuals in a population[4, 6].The concept of bet-hedging was first formalized in thecontext of information theory [7] and portfolio manage-ment [8]. Later, it was conjectured that living organismsmay employ bet-hedging strategies to decrease their riskin unpredictable environments [5, 9–12]. This idea hasbeen empirically confirmed in bacterial and viral com-munities [2–4, 13–16], in insects [17], in seed-dispersalstrategies developed by plants [18–20], and in a wealth ofother examples in population ecology, microbiology, andevolutionary biology [9]. ∗ Electronic address: [email protected]
Given their ubiquity, bet-hedging strategies have at-tracted a lot of interest from the perspective of evolution-ary game theory [21, 22]. An interesting and nontrivialresult in this context is the so-called Parrondo’s paradox[23, 24], in which an alternation of two “losing” strate-gies can lead to a “winning” one. However, most of theexisting theoretical studies of this effect, including ap-plications in population dynamics [9], rely on mean-fieldanalyses describing well-mixed communities.In this paper, we aim to extend previous approachesand to show that the competitive advantage of bet-hedging hybrid strategies is much stronger in the pres-ence of highly variable environments and/or in low-dimensional systems, where the effect of fluctuations, i.e.,demographic noise, is maximal and mean-field predic-tions do not hold.For this, we study a minimal mathematical model ofbet-hedging. It is based on the physics of the contactprocess (CP) [25–27], but with the twist that individ-uals can randomly choose between two strategies: onecharacterized by a fixed reproduction rate and the otherby fluctuating environmental-dependent rates. By com-bining analytical and computational results, we concludethat bet-hedging benefits are greatly enhanced in noisylow-dimensional environments such as the ones that liv-ing systems typically inhabit and end by discussing therelevance of our results for more realistic models of bio-logical populations.
II. THE MODEL: CONTACT PROCESS WITHHYBRID DYNAMICS
Our starting point is the simplest possible birth-deathstochastic model on a lattice, i.e., the CP [25–27] (seeFig. 1, left). Individuals occupy a (square) lattice ornetwork, with at most one individual per site. At every a r X i v : . [ q - b i o . P E ] M a r FIG. 1: Left: Sketch of the standard contact process dynam-ics: each occupied node (individual) in the lattice either (i)produces an offspring with probability p at a random neigh-boring site (provided it is empty) or (ii) is removed from thelattice with complementary probability 1 − p . Right: Contactprocess with hybrid dynamics: at each time step, every in-dividual chooses between the conservative (with probability1 − α ) and the risky (with probability α ) spreading strategy.The conservative dynamics is characterized by a constant, rel-atively low spreading probability p , while the risky one de-pends on a stochastic variable p ( t ), common to all particlesin the system. discrete time step, each individual can either produce(with probability p ) an offspring at a randomly chosenneighboring site –provided it is empty– or die and beremoved from the system (with probability 1 − p ). Thissimple dynamics can –depending on the value of p – eithergenerate an active phase characterized by a nonvanishingdensity of individuals or, alternatively, lead ineluctablyto the absorbing state in which the population becomesextinct. A critical point, p c , separates these two distinctphases [25–27].We consider a variant of the CP dynamics in which in-dividuals can choose between two strategies (see Fig. 1,right): a “conservative” one, corresponding to exploita-tion of a constantly available resource, and a “risky” one,exploiting a variable/unpredictable resource. The con-servative strategy corresponds to a CP in which p is keptconstant at a relatively low value, p . On the other hand,in the risky strategy, demographic probabilities dependupon variable external conditions, i.e., p = p ( t ), where p ( t ) is a random noise, common to all individuals in thecommunity. We focus on the simple case in which p ( t )is freshly drawn at every (Monte Carlo) time step, anddiscuss later the case in which the environment is tem-porally correlated.Individuals can hedge their bets by randomly pickingeither of the two competing strategies at each time step.This choice is controlled by the “risk parameter” α : ateach time step, each individual independently adopts therisky strategy with probability α or the conservative onewith probability 1 − α . In the language of game theory, α = 0 and α = 1 correspond to “pure strategies” and therange 0 < α < α ; variations in which α is individual-dependent are left for a future work. Key observablesare the stationary density of individuals, ρ , the averaged(exponential) growth rate, G , and the mean extinctiontime of small populations, τ (see below). III. THEORETICAL INSIGHTS
In game theory, it is known that a hybrid plan consist-ing in the alternation of two distinct pure strategies canoutperform both of them (see, e.g., [7, 9, 12, 23]), con-stituting an example of Parrondo’s paradox. This effectplays an important role for our aims in what follows. Inthis section, we discuss a simplified one-variable equationaimed at capturing the gist of our model.In particular, the simplifying assumptions we makehere are as follows: (i) We consider a mean-field limit inwhich spatial fluctuations are neglected. (ii) We neglectnonlinear saturation terms; this is a valid approximationonly for low densities. (iii) We consider a continuous timelimit, as usually done to analyze the physics of the CPand other particle systems [25, 27, 28] (a discrete-timecalculation is presented in Appendix A to prove the ro-bustness of our conclusions against this assumption). Inthe continuous-time limit, we consider the growth rate p ( t ) = ¯ p + σξ ( t ), where ¯ p and σ are constants –themean and amplitude of the stochastic risky strategy p ( t ),respectively–, and where ξ ( t ) is a Gaussian noise with (cid:104) ξ ( t ) (cid:105) = 0 and (cid:104) ξ ( t ) ξ ( t (cid:48) ) (cid:105) = δ ( t − t (cid:48) ) [observe that even ifwe maintain the same notation as above, p ( t ) and p needto be interpreted as transition rates in the continuous-time approach]. The choice of a Gaussian probabilitydistribution function for p ( t ) enables us to obtain analyt-ical calculations, but it has some “technical” drawbacks.In particular, being an unbounded distribution, p ( t ) cantake negative values; thus, in order to avoid interpreta-tion problems, we need to restrict ourselves to the case0 (cid:28) ¯ p − σ and ¯ p + σ (cid:28)
1, where these effects should benegligible. In any case, even if specific details may de-pend on this choice, the general results and conclusionspresented in what follows are robust against changes inthis probability. This is explicitly illustrated in AppendixA for the case of uniform bounded distributions.Under these assumptions, the density of individuals ρ ( t ) obeys the following rate equation:˙ ρ ( t ) = α [ p ( t ) ρ (1 − ρ ) − (1 − p ( t )) ρ ]+(1 − α ) [ p ρ (1 − ρ ) − (1 − p ) ρ ] . (1)Defining the average spreading rate, p av ( α ) = α ¯ p + (1 − α ) p , (2)Eq. (1) becomes˙ ρ ( t ) = (2 p av ( α ) − ρ − p av ( α ) ρ + 2 ασρ (cid:16) − ρ (cid:17) ξ ( t ) , (3)which, owing to the stochastic nature of p ( t ), is aLangevin equation, to be interpreted in the Itˆo sense.Up to leading linear order, we have the approximation˙ ρ ≈ (2 p av ( α ) − ρ + 2 σαρξ ( t ) , (4)valid for ρ (cid:28)
1. Now changing variables (using Itˆo cal-culus) to y = log( ρ ) and averaging over realizations (cid:104)·(cid:105) ,eq. (4) becomes ddt (cid:104) log ρ (cid:105) = G ( α ) , (5)where the sign of the exponential growth rate , G ( α ) = − σ α + 2 p av ( α ) − , (6)determines whether the population tends to shrink or[owing to the absence of the nonlinear saturation termsin this approximation, Eq. (4)] to grow unboundedly.These two regimes are separated by a critical point atwhich G ( α ) = 0.Keeping fixed all parameters but α , we define the op-timal strategy α ∗ ∈ [0 ,
1] as the one maximizing G ( α ).This can be either a pure or a hybrid strategy, depend-ing on parameter values. In particular, α ∗ = 0 for¯ p < p , α ∗ = 1 for ¯ p > p + 2 σ , and α ∗ = (¯ p − p ) / σ for intermediate values of ¯ p . Observe that the criti-cal point is obtained for p c ( α ) = + σ α , i.e., the α -dependent critical point interpolates quadratically be-tween the critical points of the pure strategies, p c (0) = and p c (1) = + σ . On the other hand, the averagespreading rate p av ( α ) [Eq. (2)] is a linear-in- α interpola-tion between the two limiting pure values.Figure 2 (top) shows the stationary density [obtainedvia numerical integration of Eq. (3)] for the conservative,the risky, and an intermediate hybrid strategy. The criti-cal points at which the nontrivial steady states emerge co-incide with the analytical predictions we have just made.As explained in the caption to Fig. 2, the differentfunctional dependences for p av ( α ) and p c ( α ) –linear andquadratic in α , respectively– enable the two curves to in-tersect each other, and thus, for intermediate values of α it is possible to have p av ( α ) > p c ( α ), i.e., a supercriticaldynamics, even in the case (illustrated in Fig. 2) in whichboth pure strategies, p and ¯ p , are subcritical. Similarly,when the two pure strategies are active/supercritical, thesame argument shows that a much higher stationary den-sity can be achieved by hybrid strategies.This graphical representation –which we believe is newin the literature– allows us to understand in a rela-tively simple and compact way the essence of Parrondo’sparadox. In what follows, we consider different typesof pure strategies, either absorbing/subcritical or ac-tive/supercritical, and quantify the gain induced by bet-hedging in different settings, including fully connected(FC) networks (where the above mean-field approachshould hold) and spatially explicit low-dimensional sys-tems (where mean-field conclusions might break down). r i s k p a r a m e t e r spreading rate s t a t i o n a r y d e n s i t y c o n s e r v a t i v e r i s ky h y b r i d FIG. 2: Top: Stationary density, computed via numerical in-tegration of Eq. (3), for three types of strategies –conservative( α = 0), risky ( α = 1), and a particular hybrid strategy(0 < α ∗ < p in the conservative case and ¯ p in the risky and hybrid cases.Critical points for the pure strategies can be obtained analyt-ically: p c ( α = 0) = 1 / p c ( α = 1) = 1 / σ . Criticalpoints for hybrid strategies lie between these two values. Thelestmost (blue) and rightmost (red) arrows indicate a specificchoice made for the two pure strategies, α = 0 and α = 1,respectively ( p = 0 .
47 and ¯ p = 0 .
72, with σ = 0 . α = α ∗ ). Bottom: Two lines are plotted as afunction of α : the effective value of the spreading rate p av ( α )and the location of the critical point, p c ( α ). The first oneinterpolates linearly between our two pure-strategy choices(arrows) for p and ¯ p in the pure cases (as indicated by theblue-red color gradient). On the other hand, the second line isa quadratic interpolation between the two pure-strategy crit-ical points (as indicated by the green-orange gradient). Asthe two lines intersect each other, there exists a range of val-ues of α for which p av ( α ) > p c ( α ) (supercritical) and othersfor which p av ( α ) < p c ( α ) (subcritical). In particular, for in-termediate values of α , such as the marked α ∗ = 0 .
5, thestochastic alternation of two subcritical strategies results in asupercritical one.
IV. COMPUTATIONAL RESULTS
The calculation in the previous section provides valu-able insight into why hybrid strategies can be important,but it has some important limitations. It is a mean-fieldcalculation, thus neglecting spatial structure. Moreover,Eq. (4) includes only linear terms, and thus it can onlydescribe exponential growth starting from low densityrather than the steady-state behavior of the nonlineardynamics. To go beyond these limitations, here we per-form direct Monte Carlo simulations of the discrete modeldefined in Sec. II in large FC networks, and later we com-pare them with similar simulations in one-dimensional(1D) two-dimensional (2D) and three-dimensional (3D)lattices.We implemented the CP dynamics using either syn-chronous/parallel or asynchronous/sequential types ofupdatings. Here, we mostly focus on the synchronouscase. In Appendix B, we show that asynchronous updat-ing leads to qualitatively similar results, even if quanti-tative differences emerge.Simulations are initialized with a fully occupied con-figuration; then the dynamics proceeds as follows: (i)At every step, a new value of p ( t ) is drawn from someprobability distribution in [0 , N (¯ p, σ ) (¯ p and σ are themean and variance, respectively, of the nontruncated dis-tribution) in which we fix possible values p ( t ) < p ( t ) = 0 and values p ( t ) > α and 1 − α ,each individual selects the risky or the conservative strat-egy respectively. (iii) Each individual either dies or re-produces with the corresponding probabilities; all dyingindividuals are removed from the system and afterwardoffspring are placed at random neighbors of their corre-sponding parents, keeping the constraint of a maximumoccupancy of one individual per site (i.e., offspring tryingto occupy an already full site are simply removed). Fi-nally, (iv) time is incremented in one unit and the processis iterated until a stationary state has been reached andsteady-state measurements (of, e.g., ρ ) are performed.We begin by verifying the possibility of obtaining anactive phase by combining two strategies, each of themleading to an absorbing/subcritical phase. To this aim,we fix the parameters ( p , ¯ p, σ ) to poise the respectivepure strategies ( α = 0 or α = 1) in the absorbing phaseand study the behavior of hybrid strategies at intermedi-ate values of α . To determine whether or not a strategyleads to an active phase, we measure the mean extinctiontime, τ , as a function of the system size N . Observe that,owing to fluctuations, any finite system is condemned toend up in the absorbing state. However, its mean lifetimeincreases exponentially with N , τ ∼ exp( N ) in the ac-tive phase [28], making the system stable in the large- N limit. Note that, in the presence of fluctuating param-eters, τ ( N ) can also scale as a power-law in the activephase [29]. On the other hand, a slow logarithmic in-crease, τ ∼ log( N ), is expected in the absorbing phase[28, 29]. As shown in Fig. 3 for different values of α and for different spatial dimensions, τ grows logarithmi-cally with N for the two pure strategies ( α = 0 , FIG. 3: Mean extinction times as a function of system size N for different strategies α and spatial dimensions. For ourparameter choice, the two pure strategies, α = 0 ,
1, havea logarithmic dependency (characteristic of subcritical be-havior [29]), while a range of hybrid strategies exhibits anexponential or power-law increase typical of active phases[29]. Parameter values ( p , ¯ p, σ ): 1D, (0 . , . , . . , . , . . , . , . . , . , . ity. We therefore conclude that in the CP the stochasticalternation of two absorbing dynamics can lead to anactive one, in agreement with the linear-approximationabove.Some remarks are in order. The advantageous conse-quences of bet-hedging are not limited to the mean-fieldcase, which can be simply interpreted in terms of Eq.(3), but are important also in low-dimensional systemswhere internal fluctuations play a key role. Observe alsothat, as the phase boundaries depend on dimensional-ity, different parameters are chosen for different panels inFig. 3. We discuss later a way to compare more clearlythe strength of the effect as the system dimensionality ischanged. Finally, we have made no attempt here to ac-curately determine the values of α delimiting the activephase for each dimension, but have just confirmed thestabilizing effect of hybrid strategies.The goal now is to quantitatively analyze how the ben-efits of bet-hedging depend on the level of stochastic-ity, both external (environmental) and intrinsic (demo-graphic). A. Environmental/external noise
First, we study the dependence on environmental vari-ability σ . To ease comparison, for each value of σ ,we fix the two pure strategies to have the same steady-state density, (cid:104) ρ ( α = 0) (cid:105) = (cid:104) ρ ( α = 1) (cid:105) = 0 .
3, by anappropriate choice of the only remaining free parame-ters, p and ¯ p , respectively. Observe that, at variancewith the previous section, here the pure strategies aretaken to be “equally” active (same steady-state density),but we could have also taken them to be equally absorb-ing (same extinction time). The reason for this choice isthat it allows for a much faster and easier computationalimplementation. We then analyze how the steady-stateaveraged density ρ depends on α for different values of σ . Figure 4(a) clearly illustrates that, in the case of FC(mean-field) lattices, more variable environments allowbet-hedging strategies to achieve much higher station-ary densities. The same trend holds for low-dimensionallattices (not shown): the larger the external noise, themore benefits a community can derive from convenientlyexploiting bet-hedging.This observation is consistent with the linear analy-sis embodied in Eq.(4). Using the definition of G ( α )and keeping the environmental variance σ as a controlparameter, p and ¯ p can be fixed by imposing identicalgrowth rates for the pure strategies, G (0) = G (1) ≡ G , .Under this constraint, the maximum possible growingrate is max( G ) = G ( α = 1 /
2) = G , + σ , (7)predicting a linear increase in the optimal G with σ .As a final remark, observe that, although the two purestrategies have been set to be equivalent (in the sensethat both lead to the same stationary density), the opti-mal strategy α ∗ in Fig. 4(a) (maximizing (cid:104) ρ (cid:105) ) tends tobe slightly larger than that provided by the approximateanalytical prediction α ∗ = 1 / G ). We havechecked that this “favoring” of the risky strategy stronglydepends on the details of the implementation, as we havenot observed it with asynchronous updating in the dy-namics (see Appendix B). So far, we have not attemptedto determine the optimal strategy α ∗ for each case, butjust to confirm the gain enhancement of bet-hedging inthe presence of larger fluctuations. B. Dimensionality and demographic/intrinsic noise
A main feature of low-dimensional models in statis-tical mechanics is that intrinsic fluctuations (or demo-graphic stochasticity, in the language of population dy-namics) play a more dramatic role than they do in highdimensions [30], where they can be safely neglected inmean-field-like approximations. We assume –and thenexplicitly verify– that smaller spatial dimensions effec-tively correspond to larger levels of demographic noise.We now explore the effect of dimensionality on bet-hedging; the spatial dimension of the systems is variedwhile keeping a fixed external noise variance σ . Asabove, to ease comparison, we choose pure strategies foreach dimension so that (cid:104) ρ (0) (cid:105) = (cid:104) ρ (1) (cid:105) = 0 . s t a t i o n a r y d e n s i t y s t a t i o n a r y d e n s i t y FIG. 4: (a) Effect of the external-noise variability ( σ ) on thestationary density for different bet-hedging strategies. Curvesare results of Monte Carlo simulations of the fully-connected(FC) CP with bet-hedging. As σ is increased, the optimalstrategy induces higher stationary densities, even if the purestrategies α = 0 , (cid:104) ρ ( α = 0 , (cid:105) =0 .
3. Parameter values are N = 10 , p = 0 . p, σ ) are(0 . , . . , . . , .
15) and (0 . , .
20) inthe different curves. (b) Effect of dimensionality at fixed σ .The net benefit of bet-hedging is much enhanced at lower-dimensions. Parameters: (cid:104) ρ ( α = 0 , (cid:105) = 0 . σ = 0 . N = 10 for 1D, 2D and FC, N = 10648 for 3D, and ( p , ¯ p )are 1D:(0 . , . . , . . , . . , . pure strategies are equally active) and measure compu-tationally (cid:104) ρ (cid:105) as a function of α for hybrid strategies ineach dimension.Figure 4(b) clearly illustrates that the benefits of bet-hedging are much enhanced as the system dimensional-ity is decreased, allowing for much higher densities. Inparticular, 1D systems can accommodate twice as muchdensity as FC (infinite-dimensional) lattices.A simple mathematical argument allows us to qualita-tively –even if not quantitatively– understand this find-ing. Demographic noise is the key ingredient, missing inthe mean-field limit. Therefore, we generalize Eq. (4) toinclude a demographic-noise term of tunable amplitude γ [28, 31] as well as the above-neglected leading nonlinearterm˙ ρ ( t ) ≈ (2 p av ( α ) − ρ − p av ( α ) ρ + 2 ασρξ ( t ) + γ √ ρη ( t ) , (8)where η ( t ) is a Gaussian white noise. As usual, thesquare-root factor multiplying the noise amplitude of de-mographic fluctuations is a direct consequence of the cen-tral limit theorem, which, in particular, imposes thatfluctuations disappear in the absence of activity ( ρ = 0)[28].Equivalently to Eq. (8), we can write down the Fokker-Planck equation for the probability distribution P ( ρ, t )[28]. To work in the quasi-stationary approximation [28,31, 32] (i.e., to avoid technical problems stemming fromthe existence of an absorbing state at ρ = 0), we includea small and constant drift ε , which is a constant addedon the right hand side of Eq. (8), giving: ∂ t P ( ρ, t ) = − ∂∂ρ (cid:2)(cid:0) ε + (2 p av ( α ) − ρ − p av ( α ) ρ (cid:1) P ( ρ, t ) (cid:3) + 12 ∂ ∂ρ (cid:2) ( β ρ + γ ) ρP ( ρ, t ) (cid:3) , (9)where, for convenience, we have introduced β = 2 ασ .The associated stationary probability distribution func-tion then reads: P st ( ρ ) = C ρ εγ − exp (cid:16) p av − ρ − p av ρ γ (cid:17) , β = 0 C ρ εγ (cid:0) γ + β ρ (cid:1) p av − β + p av γ β − εγ − × exp (cid:16) − p av ρβ (cid:17) , β > , (10)where C and C are normalization constants. From thisequation we can compute the averaged quasi-stationarydensity (cid:104) ρ (cid:105) = (cid:90) ∞ + dρ ρ P st ( ρ ) (11)as a function of parameter values. The effectivedimension-dependent value of γ can now be inferred fromthe condition that –fixing the remaining parameters (i.e., p , ¯ p , and σ ) as in each of the spatially explicit simula-tions [see caption of Fig. 4b]– the quasi-stationary den-sity in Eq. (11) satisfies (cid:104) ρ ( α = 0) (cid:105) = (cid:104) ρ ( α = 1) (cid:105) = 0 . γ are γ = 0 . , .
15, and 0 . γ –effectively representing the amplitude of demographicnoise– increases upon lowering the spatial dimension.Having determined, for each dimension, the level ofdemographic fluctuations γ , we now use Eqs. (10) and(11) to compute the maximum density as a function of α .Results are shown in Fig. 5, which reveals that the ben-efits of bet-hedging are enhanced for larger demographicnoises and, thus, for lower spatial dimensions. M F s t a t i o n a r y d e n s i t y a) D D D b) FIG. 5: (a) Stationary density of the optimal hybrid strat-egy as a function of the demographic noise amplitude γ : eachpoint was computed using the value of γ inferred from Eq.(10), for a different spatial dimension (1, 2, and 3), and γ = 0 for the mean-field case. Parameters p , ¯ p , and σ are the same as in Fig. 4(b) and γ is tuned to produce (cid:104) ρ ( α = 0) (cid:105) = (cid:104) ρ ( α = 1) (cid:105) = 0 .
3. The inferred γ changesslightly for α = 0 and α = 1 as reflected in the error bars(shaded region). These results confirm that the effective noiseamplitude γ increases as the system dimensionality decreasesand that the benefits of bet-hedging are enhanced by demo-graphic noise. However, the curve becomes nonmonotonousfor larger values of γ . (b) Density distributions in the optimalstrategy α ∗ for different demographic noise amplitudes γ : Werepresent the quasistationary solution of Eq. (10) ( ε = 10 − ).The calculus fails for higher values of γ , as the probability ofdecay into the absorbing state (emerging peak at ρ = 0) be-comes non-negligible. The functional dependence of (cid:104) ρ (cid:105) max ( γ )does not change qualitatively for other choices of this param-eter, such as ε = 10 − , − or 10 − . We remark that this phenomenological single-variabletheory only provides a qualitative explanation of the phe-nomenon and does not quantitatively reproduce the ac-tual stationary densities in Fig. 4(b). Observe also thatfor very high noise amplitudes the curve in Fig. 5 veersdown, while this effect is not seen when reducing the sys-tem dimensionality. A more rigorous analytical approachto this problem –including the explicitly spatial depen-dence in Eq. (8)– is a challenging task, beyond the scopeof the present work.
C. Time-correlated environments
In the model we have discussed, the timescale of en-vironmental changes is the same as the generation timescale. However, real biological populations have to copewith environmental conditions varying on time scalespossibly longer [33] than the individual generation time.To address this important generalization, we checked howour main results change upon varying the temporal cor-relation of the state of the environment. In particular, wesimply described p ( t ) as an Ornstein-Uhlenbeck process(see, e.g., [28]) of average ¯ p and variance σ and studythe effect of varying its correlation time.Detailed results of this study are summarized in Ap-pendix C. Our main conclusion, i.e., that the benefits ofbet-hedging strategies are enhanced in lower-dimensionalsystems, remains unaltered. In addition, considering anenvironment correlated over a few generations enhancesthe advantage of bet-hedging in all dimensions, althoughthis effect is significantly stronger in low dimensions. Fi-nally, the optimal strategy becomes more conservativefor environments characterized by a very long correla-tion time. These results can be intuitively understoodby thinking that, if the environment is persistently unfa-vorable for a long time, the extinction risk is very high,and bet-hedging strategies become more crucial for sur-vival. A much more detailed analytical characterizationof bet-hedging dynamics under correlated environmentsis left for a future study. V. CONCLUSIONS
Summarizing, our main finding is that the relativebenefit of developing bet-hedging strategies is stronglyenhanced in highly fluctuating low-dimensional systems,where both internal and external sources of variabilitygreatly foster dynamical fluctuations, leading to a strongdeparture from mean-field behavior. Given that theseconditions are often met by biological populations –asfor instance, in bacterial colonies competing at the frontof a range expansion in noisy environments [34–36]– ourresults support the importance of bet-hedging in na-ture. This being said, of course, more realistic models–including some realistic ingredients such as, for exam-ple, the possibility of “dormant” states and not just birthand death processes– would be required to approach vi-ral or bacterial communities and their bet-hedging moreclosely.The kind of trade-off considered in this paper, betweena stable and a fluctuating resource, is possibly the sim-plest example of bet-hedging, both biologically relevantand natural to understand using tools of nonequilibrium statistical physics. However, we conjecture that the in-creased strength of bet-hedging in low dimensionality isa general phenomenon, present in other recently studiedexamples of trade-offs, for example, between reproduc-tion and motility [37, 38] and in pairwise games [39].Our preliminary results presented in Appendix C showthat the effect described in this paper is still present incorrelated environments. However, for very long corre-lation times, bet-hedging strategies are disfavored com-pared to short-correlated environments, but they alwaysprovide an advantage with respect to pure strategies. Inview of these preliminary results, it will be of interest toinvestigate from a general perspective and in more depththe influence of environmental-noise temporal autocorre-lations on bet-hedging, as well as the difference betweenexploiting bet-hedging individually and exploiting it at acommunity level. We believe that this work will providea physical framework to answer these and similar chal-lenging questions which might be of interest in biologyand ecology.
Appendix A: Uniformly distributed environment
In this Appendix we study the case in which thespreading probability for the risky strategy, p ( t ), isbounded and uniformly distributed in the range [¯ p − δ, ¯ p + δ ], where the parameter δ encapsulates the level of en-vironmental variability. In particular, to avoid negativevalues, we take δ < ¯ p and δ < − ¯ p . This type of distribu-tion allows for analytical treatment using a discrete-timeapproach, which is common in the study of game theory[7]. To proceed, we take the linearized rate equation de-rived in the text, Eq. (4), and write it in a discrete-timeform (using one-unit time steps) and replace p ( t ) with itsexplicit form for the uniform distribution, ρ ( t ) = 2 p av ( α ) ρ ( t −
1) + 2 αδρ ( t − u ( t − , (A1)where u ( t ) is a uniformly distributed variable in the range[ − , t ≥ ρ (0) and discretizing therange of values of u , u = ( u , ..., u U ), the previous equa-tion becomes ρ ( t ) = U (cid:89) i =1 [2 p av ( α ) + 2 αδu i ] n i ρ (0) , (A2)where n i is the number of times that a value u i isobtained, and therefore, (cid:80) i n i = t . The exponen-tial growth rate is derived from its discrete form, G =lim t →∞ t log ρ ( t ) ρ (0) [7], G ( α ) = U (cid:88) i =1 n i t log [2 p av ( α ) + 2 αδu i ] , (A3) -0.0100.010.020.03 0 0.25 0.5 0.75 1risk parameter g r o w t h r a t e FIG. 6: Exponential growth rate G as a function of therisk parameter α as given by Eq. (A4) for uniformly dis-tributed environments: p ( t ) = ¯ p + δu ( t ), where ¯ p and δ areconstants and u ( t ) is a uniform noise distributed in [ − , δ ; in all cases the corresponding pure strategies havebeen tuned to give G ( α = 0) = G ( α = 1) = − .
005 inthe absorbing/subcritical phase. The optimal strategy is al-ways a hybrid one, very close to α ∗ = 0 .
5. In particular,for δ (cid:38) . G ( α ∗ ) >
0, allowing for active/supercriticaldynamics. Furthermore, the maximum growth rate for thisstrategy increases with the amplitude of the environmentalnoise. Parameter values: p = 0 .
498 fixed in all cases, and(¯ p, δ ) = (0 . , . , (0 . , . , (0 . , .
3) and (0 . , . δ < ¯ p and δ < − ¯ p in all cases. which in the continuum limit becomes G ( α ) = (cid:90) − du δ log [2 p av ( α ) + 2 αδu ] = log(2) − δα log ( p av ( α ) + αδ ) p av ( α )+ αδ ( p av ( α ) − αδ ) p av ( α ) − αδ , (A4)for any α ∈ (0 , G (0) = log (2 p ) for α = 0.Figure 6 shows the solution G as a function of α [Eq.(A4)] for different choices of the environmental variabil-ity δ . In this example, we have tuned the parameters p and ¯ p to be equally subcritical, i.e., G (0) = G (1) < G (0) = G (1) > G , always lies at intermediate values of α ;(ii) the growth rate for such optimal strategy increaseswith the amplitude of fluctuations δ ; and (iii) for suffi-ciently large values of δ , a combination of two subcriticalstrategies gives rise to a supercritical one, as G ( α ∗ ) > δ (cid:38) .
15. Moreover, we have tested these results inMonte Carlo simulations, as well as with different latticedimensions, obtaining plots similar to Fig. 4 and Fig. 7.Summarizing, in the case in which p ( t ) is uniformly dis-tributed, the same conclusion obtained for Gaussian dis-tributions holds: the benefits of bet-hedging are stronger in the presence of both extrinsic and intrinsic fluctua-tions. Appendix B: Model with asynchronous updating
In this Appendix, we verify the robustness of our re-sults when an asynchronous-updating version of the CP[25] is implemented. At each time step, one of the ex-isting N act ( t ) active particles is randomly selected; withprobability α , the particle chooses the risky strategy or,with complementary probability 1 − α , the conservativeone. As above, in the first case, it reproduces with prob-ability p ( t ) or dies with probability 1 − p ( t ), where p ( t )changes with the environment, while for the conservativestrategy it reproduces with probability p or dies withprobability 1 − p . Time is incremented in 1 /N act ( t ). Af-ter all particles in the network have been updated onceon average (i.e. after time increases in one unit) an-other value of p ( t ) is drawn from a Gaussian distribution N (¯ p, σ ).With this implementation, as in Fig. 4, we have againcomputed the curve (cid:104) ρ ( α ) (cid:105) provided that (cid:104) ρ (0 , (cid:105) = 0 . σ (inthe FC network) and for different network dimensions(fixing σ ). As illustrated in Fig. 7, the relative positionof all curves is the same as for the case of synchronousupdating: the benefits of bet-hedging are enhanced as thenoise amplitude is increased. However, quantitatively,the enhancement is smaller than in the synchronous case,discussed in the text.As a final remark, observe that one could have naivelyexpected that fluctuations derived from the sequentialupdating might contribute to an enhancement of the den-sity for such hybrid strategies. This difference stemsfrom the fact that in the sequential implementation ofthe model, not all individuals are necessarily updated atevery single Monte Carlo step; thus the stochasticity in-troduced by this type of updating may save populationsfrom extinctions in very unfavorable environments. Thisimplies that the community does not rely as strongly onbet-hedging to perdure. Appendix C: Effect of temporal correlations
A simple way to introduce temporal autocorrelationsin the environment is to take p ( t ) to follow a Ornstein-Uhlenbeck process [28], i.e., a Brownian particle movingin a parabolic potential. Mathematically, this processobeys [28] ˙ p = θ (¯ p − p ) + √ θσξ ( t ) , (C1)where ¯ p and σ represent, as before, the mean and varianceof p ( t ), respectively. With this choice, p ( t ) is Gaussiandistributed, N (¯ p, σ ). The new parameter θ controls thetemporal autocorrelations, as (cid:104) ( p ( t ) − ¯ p ) ( p ( t (cid:48) ) − ¯ p ) (cid:105) = s t a t i o n a r y d e n s i t y s t a t i o n a r y d e n s i t y FIG. 7: Effect (a) external-noise variability ( σ ) in the fullyconnected (FC) network and (b) dimensionality at fixed σ on the stationary density for different bet-hedging strategieswith asynchronous updating. Curves are results of MonteCarlo simulations of the CP with bet-hedging. Qualita-tively, the same conclusions are obtained as with the par-allel updating. However, the benefits of bet-hedging strate-gies become lower in this second implementation, especiallyfor the FC network. Parameter values are as follows. (a) N = 10 ; p = 0 . p, σ ) of (0 . , . . , . . , .
15) and (0 . , .
20) in the different curves. (b): σ = 0 . N = 10 for 1D, 2D, and FC, N = 10648for 3D, and ( p , ¯ p ) are 1D:(0 . , . . , . . , . . , . σ e − θ | t − t (cid:48) | ; consequently, θ → θ → ∞ representthe extreme cases of immutable (completely correlated)and delta-correlated environments, respectively.Equation (C1) can be integrated exactly, allowing fora recursive generation of values at successive time steps[40], p ( t + 1) = ¯ p (1 − e − θ ) + p ( t ) e − θ + σ (cid:112) − e − θ N (0 , , (C2) where N (0 ,
1) is a zero-mean unit-variance Gaussian ran-dom number.Fixing the environmental variance σ , we numericallystudy the effect of temporal correlations on bet-hedgingfor different values of θ in every dimension. Following thesame strategy as above, we tune the parameters p and ¯ p for each temporal autocorrelation θ to fix the stationarydensity at (cid:104) ρ ( α = 0 , (cid:105) , and measure (cid:104) ρ ( α ) (cid:105) . Resultsare summarized in Fig. 8. Some remarks are in order.(i) The optimal strategy is always a hybrid strategy be-tween α = 0 and α = 1. Additionally, curves coincidewith those in Fig. 4(b) when θ is high ( θ ∼ θ decreases moderately, the stationary densityat the optimal strategy becomes larger compared to thenoncorrelated case. In other words, bet-hedging strate-gies are more efficient for temporally autocorrelated en-vironments. This effect is stronger for lower dimensions,whereas it barely applies to higher dimensional lattices.(iii) When the autocorrelation is very large ( θ < . p risk parameter s t a t i o n a r y d e n s i t y FIG. 8: Stationary density as a function of the risk parame-ter for different lattice topologies in temporal-correlated en-vironments: the spreading probability of the risky strategy, p ( t ) now obeys an Ornstein-Uhlenbeck process with mean ¯ p ,variance σ , and exponential temporal correlations with acharacteristic time θ − . The benefits of the hybrid strategiesin the stationary density become enhanced for intermediatevalues of θ . This result is much intensified in lower dimen-sions, while it is imperceptible for the 3D and FC networks.Parameters: σ = 0 .
05; (¯ p, θ ) in 1D: (10 , . , . , . . , . , . , . , . . , . , . , . , . . , . , . , . , . . , . p and N , taken as in Fig. 4(b). θ at which this effect appearsvaries for different dimensions. (iv) Finally, the optimalstrategy becomes more conservative when temporal cor-relations are added to the environment, with a bias to α ∗ → θ decreases. It would be nice to have amore detailed analytical understanding of all this phe-nomenology, but we leave this challenging task for futurework. Acknowledgments
We are grateful to R. Rubio de Casas for illuminat-ing discussions and to P. Moretti for a critical read- ing of the manuscript. We acknowledge support fromJ. de Andaluc´ıa Grant No. P09-FQM-4682 and Span-ish MINECO Grant Nos. FIS2012-37655-C02-01 andFIS2013-43201-P. [1] J. Monod, Annual Reviews in Microbiology , 371 (1949).[2] J.-W. Veening, W. K. Smits, and O. P. Kuipers, AnnualReviews in Microbiology , 193 (2008).[3] I. G. de Jong, P. Haccou, and O. P. Kuipers, Bioessays , 215 (2011).[4] A. Solopova, J. van Gestel, F. J. Weissing, H. Bachmann,B. Teusink, J. Kok, and O. P. Kuipers, Proceedings ofthe National Academy of Sciences , 7427 (2014).[5] J. Seger, Oxford Surveys in Evolutionary Biology , 182(1987).[6] E. Kussell and S. Leibler, Science , 2075 (2005).[7] J. L. Kelly, The Bell System Technical Journal , 917(1956).[8] R. Fernholz and B. Shay, The Journal of Finance , 615(1982).[9] P. D. Williams and A. Hastings, Proceedings of the RoyalSociety B: Biological Sciences p. rspb20102074 (2011).[10] H. N. Comins, W. D. Hamilton, and R. M. May, Journalof Theoretical Biology , 205 (1980).[11] W. D. Hamilton and R. M. May, Nature , 578 (1977).[12] V. A. Jansen and J. Yoshimura, Proceedings of the Na-tional Academy of Sciences , 3696 (1998).[13] M. P. Stumpf, Z. Laidlaw, and V. A. Jansen, Proceedingsof the National Academy of Sciences , 15234 (2002).[14] D. M. Wolf, V. V. Vazirani, and A. P. Arkin, Journal ofTheoretical Biology , 227 (2005).[15] D. M. Wolf, V. V. Vazirani, and A. P. Arkin, Journal ofTheoretical Biology , 255 (2005).[16] S. Heilmann, K. Sneppen, and S. Krishna, Journal ofVirology , 3016 (2010).[17] K. R. Hopper, Annual Review of Entomology , 535(1999).[18] D. Z. Childs, C. Metcalf, and M. Rees, Proceedings ofthe Royal Society B: Biological Sciences p. rspb20100707(2010).[19] D. Venable and J. Brown, Vegetatio , 31 (1993).[20] D. Cohen and S. A. Levin, Theoretical Population Biol-ogy , 63 (1991).[21] J. M. Smith, Evolution and the Theory of Games (Cam-bridge university press, 1982).[22] M. A. Nowak,
Evolutionary dynamics (Harvard Univer- sity Press, 2006).[23] G. P. Harmer and D. Abbott, Nature , 864 (1999).[24] J. M. Parrondo, G. P. Harmer, and D. Abbott, PhysicalReview Letters , 5226 (2000).[25] J. Marro and R. Dickman, Nonequilibrium phase tran-sitions in lattice models (Cambridge University Press,2005).[26] G. Grinstein and M. Mu˜noz, in
Fourth Granada Lecturesin Computational Physics , pp. 223–270 (1997).[27] M. Henkel, H. Hinrichsen, S. L¨ubeck, and M. Pleim-ling,
Non-equilibrium phase transitions , vol. 1 (Springer,2008).[28] C. Gardiner,
Stochastic methods (Springer-Verlag,Berlin–Heidelberg–New York–Tokyo, 1985).[29] F. Vazquez, J. A. Bonachela, C. L´opez, and M. A. Mu˜noz,Physical Review Letters , 235702 (2011).[30] J. Binney, N. Dowrick, A. Fisher, and M. Newman,