Fluctuation Relations of Fitness and Information in Population Dynamics
aa r X i v : . [ q - b i o . P E ] M a y APS/123-QED
Fluctuation Relations of Fitness and Information in Population Dynamics
Tetsuya J. Kobayashi and Yuki Sughiyama ∗ :Institute of Industrial Science, the University of Tokyo,4-6-1 Komaba Meguro-ku, Tokyo 153-8505, Japan. (Dated: May 12, 2015)Phenotype-switching with and without sensing environment is a ubiquitous strategy of organismsto survive in fluctuating environment. Fitness of a population of organisms with phenotype-switchingmay be constrained and restricted by hidden relations as the entropy production in a thermal systemwith and without sensing and feedback is well-characterized via fluctuation relations (FRs) . In thiswork, we derive such FRs of fitness together with an underlying information-theoretic structure inselection. By using path-integral formulation of a multi-phenotype population dynamics, we clarifythat the optimal switching strategy is characterized as a consistency condition for time-forward andbackward path probabilities. Within the formulation, the selection is regarded as passive informationcompression, and the loss of fitness from the optimal strategy is shown to satisfy various FRs thatconstrain the average and fluctuation of the loss. These results are naturally extended to thesituation that organisms can use an environmental signal by actively sensing the environment. FRsof fitness gain by sensing are derived in which the multivariate mutual information among thephenotype, the environment and the signal plays the role to quantify the relevant information inthe signal for fitness gain. Submitted to PRL on 25/Jul/2014; resubmitted to PRL for revision on 10/Apr/2015.
PACS numbers: Valid PACS appear here
Phenotype-switching is a strategy of living systems tosurvive in stochastically changing environment[1]. Evenif no environmental information is available, diversifica-tion of phenotypes by stochastic switching (known asbethedging) can lead to gain of fitness when a subpopu-lation with a resistant phenotype can survive in a harshenvironment to feed the next generation[2–4]. If environ-mental signal that conveys information of the environ-ment is exploitable, further gain of fitness is possible byswitching into the phenotypes adapted to the future en-vironment (known as decision-making)[5–7]. Ubiquitousobservations of phenotype-switching and environmentalsensing in living systems from higher organisms down tobacteria implies its actual fitness advantage over the di-versification loss and the metabolic load of switching andsensing mechanisms[8–11].The fitness gain enjoyed by switching and sensing,however, must be constrained by the environmentalstatistics and the sensed information. On the one hand,previous investigations clarified such constrains for aver-age fitness gain at least in specific situations[2, 3, 7, 12–15]. On the other hand, the similarity between evolution-ary dynamics and statistical physics[16–18] suggests thatmore general relations may exist as the series of fluctua-tion relations (FRs) characterize not only the average butalso the fluctuation of entropy production in a thermalsystem with and without sensing and feedback[19, 20].Finding such relations is crucial to understand the con-straints and predicability of adaptive dynamics of organ-isms in ever changing environment (fitness seascapes)[21].In this work, by using a path-wise (path-integral) formu- ∗ [email protected]; http://research.crmind.net/ lation of the dynamics of growing population with multi-phenotypes[22–25], we reveal such relations of fitness to-gether with the underlying information-theoretic struc-ture of selection.Let x t ∈ S x and y t ∈ S y be a phenotype of a livingorganism and a state of environment at time t , respec-tively. For simplicity, possible phenotypic and environ-mental states are assume to be discrete. We also definepaths (histories) of phenotype and environment up totime t as X t := { x τ | τ ∈ [0 , t ] } and Y t := { y τ | τ ∈ [0 , t ] } ,respectively. The population size of organisms in a phe-notype x at time t under a realization of an environmen-tal path Y t is denoted as N Y t ( x ). When the populationsize is sufficiently large for all x , N Y t ( x ) can be approx-imated to be continuous as in [2, 22]. Phenotype of anorganism, in general, switches stochastically over timedepending on its state. The switching dynamics is mod-eled, for example, by a Markov transition probability T where P x ′ T ( x ′ | x ) = 1 as in [22]. In addition, an organ-ism with a phenotype x under an environmental state y isassumed to duplicate asexually to produce its e h ( x,y ) − h : S x × S y → R . Then, the time-discrete dynamics ofthe population size, N Y t ( x ), can be described (Fig. 1) as N Y t +1 ( x ′ ) = e h ( x ′ ,y t +1 ) X x T ( x ′ | x ) N Y t ( x ) . (1)Cumulative fitness of the population at t under an envi-ronmental path Y t is defined as Ψ[ Y t ] := ln P x ′ N Y t ( x ′ ) P x N ( x ) . Ifthe environmental path follows a path probability Q [ Y t ],we can define the environmental ensemble average ofthe cumulative fitness as h Ψ t i := h Ψ[ Y t ] i Q [ Y t ] . More-over, with additional assumptions on Y t and T , the time-average of the cumulative fitness, ψ ( t ) := h Ψ t i /t can Z t Y t Environment:
Time-forward sampled pathPhenotype-SwitchGrowth Time-backward sampled path
Signal: P Y B [ X t ] P F [ X t ] X t X t T ( x τ +1 | x τ ) Ti Q [ Y t , Z t ] τ τ +1 τ τ +1 e h ( x τ +1 ,y τ +1 ) FIG. 1. A schematic diagram of the population dynamicswith phenotype switching[22]. Note that the actual modelaccounts for sufficiently large population, and the lineage ofcells in the figure illustrates behavior of only a small subsetof the population. also reflect temporal averaging of long-term growth underone realization of the environment as lim t →∞ t Ψ[ Y t ] =lim t →∞ ψ ( t )[2]. In this work, finite t is considered.All formulations and definitions can be naturally ex-tended for the situation where an environmental signal z t ∈ S z is available. Let Z t := { z ( τ ) | τ ∈ [0 , t ] } bethe path of the signal, and Q [ Y t , Z t ] be the joint paht-probability of the environment and the signal. The pop-ulation dynamics with the signal can be obtained by sim-ply replacing T ( x ′ | x ) with T ( x ′ | x, z ′ ) as N Y , Z t +1 ( x ′ ) = e h ( x ′ ,y t +1 ) X x T ( x ′ | x, z t +1 ) N Y , Z t ( x ) . (2)We also define the cumulative fitness and its av-erage as Ψ[ Y t , Z t ] := ln P x ′ N Y , Z t ( x ′ ) P x N ( x ) , and h Ψ t i := h Ψ[ Y t , Z t ] i Q [ Y t , Z t ] .As indicated in [22, 23, 26], the total populationsize at time t , N Y t := P x ′ N Y t ( x ′ ), can be describedwith a path integral formulation. Let define a time-forward path probability of phenotype without sensingas P F [ X t ] := Q t − τ =0 T ( x τ +1 | x τ ) P F ( x ) where P F ( x ) := N Y ( x ) / N Y . In addition, a path-wise (historical) fit-ness of a phenotypic path X t under an environmentalpath Y t is defined as H [ X t , Y t ] := P t − τ =0 h ( x τ +1 , y τ +1 ).Then, the population size of organisms that experi-ence the phenotypic path X t under Y t at time t is N [ X t , Y t ] = e H [ X t , Y t ] P F [ X t ] N [26, 27]. Thus, we have N Y t = P X t N [ X t , Y t ] andΨ[ Y t ] = ln N Y t N Y = ln D e H [ X t , Y t ] E P F [ X t ] . (3)Because of this representation, Ψ[ Y t ] can also be repre-sented variationally asΨ[ Y t ] = max P [ X t ] [ h H [ X t , Y t ] i P − D [ P || P F ]] , (4) where D [ P || P F ] := P X t P [ X t ] ln P [ X t ] P F [ X t ] is the Kullback-Leibler divergence (KLD) between P and P F [26–28]. Thisvariational problem can be attained by a retrospective(time-backward) path probability [26, 29] defined as P Y B [ X t ] = e H [ X t , Y t ] − Ψ[ Y t ] P F [ X t ] . (5)With this backward path probability, we haveΨ[ Y t ] = h H [ X t , Y t ] i P Y B − D [ P Y B || P F ] . (6)This path-wise formulation generally holds for more gen-eral P F that may not be generated by Markov processes.In the following, therefore, we consider that P F [ X t ] canbe any path probabilities over the phenotypic history, X t .The forward and backward path probabilities, P F [ X t ]and P Y B [ X t ], have an obvious interpretation. P F [ X t ] is theprobability to observe an organism with a phenotypic his-tory X t when we randomly sample an organism from theinitial population at t = 0 and track it in a time-forwardmanner (Fig. 1). When the tracked organism duplicates,we choose one of the two daughters randomly. P B [ X t ], incontrast, is the probability to observe X t when we ran-domly sample an organism from the final population at t and track it back retrospectively (Fig. 1). Because thebackward path probability is defined for a fixed environ-mental history, Y t , we can also define a joint path proba-bility as P JB [ X t , Y t ] := P Y B [ X t ] Q [ Y t ]. We can also obtain amarginal path probabilities as P MB [ X t ] := P Y t P JB [ X t , Y t ].With these probabilities, we have the average of thecumulative fitness as h Ψ t i = h H [ X t , Y t ] i P JB − I X , Y B − D [ P MB || P F ] , (7)where I X , Y B is a backward mutual informa-tion between X t and Y t defined as I X , Y B := P X t , Y t P JB [ X t , Y t ] ln P JB [ X t , Y t ] P MB [ X t ] Q [ Y t ] [27]. This relation gen-erally holds for any strategy of phenotypic switching.Among them, we focus on a special strategy, ˆ P F [ X t ],that maximizes h Ψ t i . Because h Ψ t i is concave withrespect to P F , ˆ P F [ X t ] is unique in the convex spaceof path probabilities. However ˆ P F [ X t ] may not existswithin biologically realistic class of path probability,e.g., ones generated by Markov or causal processes. Ifexits, this strategy can be regarded as the strategy thathave adapted evolutionary to the environment definedby Q [ Y t ]. Even if not, ˆ P F [ X t ] and the corresponding D ˆΨ E := max P F h Ψ i still plays important roles as thebound of fitness in the FRs derived in the following. Suchoptimal strategy must satisfy the stationary condition, δ D ˆΨ t E = 0, for any perturbation of the strategy δ P F around ˆ P F . The condition can be explicitly representedas δ D ˆΨ t E = DP Y t ˆ P Y B [ X t ] Q [ Y t ]ˆ P F [ X t ] E δ P F = D ˆ P MB [ X t ]ˆ P F [ X t ] E δ P F = 0[27].From this equation, we obtain a consistency conditionbetween the forward and backward probabilities asˆ P MB [ X t ] = X Y t ˆ P Y B [ X t ] Q [ Y t ] = ˆ P F [ X t ] . (8)The consistency condition requires no time-directionalityin the phenotypic paths in the following sense. When wesample phenotypic paths in the time-forward manner, wehave the ensemble of paths, P F [ X t ], that contains no in-fluence from the environment. When we sample paths inthe time-backward manner without observing the envi-ronment, Y t , we have another ensemble of paths that fol-lows the marginal backward path probability ˆ P MB [ X t ](Fig.2 (A)). While the forward and the backward path prob-abilities are the same marginally under the consistencycondition, the selection induces correlation between thebackward phenotypic dynamics and the environmentalhistory. This fact is quantitatively described by the op-timal cumulative fitness obtained from eq. (7) as D ˆΨ t E = h H [ X t , Y t ] i ˆ P JB − ˆ I X , Y B , (9)where ˆ I X , Y B measures the correlation. This form of theoptimal cumulative fitness can be further represented asanother type of variational problem[27] as D ˆΨ t E = max P Y B [ X t ] h h H [ X t , Y t ] i P Y B [ X t ] Q [ Y t ] − I X , Y B i . (10)From the information-theoretic viewpoint, this is equiva-lent to the lossy soft compression or encoding of the envi-ronmental history, Y t , into the phenotypic history, X t , un-der a utility measure H (or equivalently distortion mea-sure − H )[30]. The environmental history is composed ofinformation relevant to and nothing to do with increaseof the path-wise fitness H . The variational form of D ˆΨ t E indicates that only information relevant for increasing H is imprinted or encoded into the phenotype history, andthe optimal ˆ P Y B [ X t ] is regarded as the optimal encoder.This relation clarifies that selection can be regarded as akind of passive information processing, and the backwardmutual information, ˆ I X , Y B , quantifies the information en-coded by the selection.From the consistency condition (eq. (8)), we have P Y t ˆ P Y B [ X t ] Q [ Y t ] / ˆ P F [ X t ] = P Y t e H [ X t , Y t ] − ˆΨ[ Y t ] Q [ Y t ] =1. This implies that ˆ P X B [ Y t ] := ˆ P JB [ X t , Y t ] / ˆ P MB [ X t ] = e H [ X t , Y t ] − ˆΨ[ Y t ] Q [ Y t ] holds for X t ∈ Supp[ˆ P F ] := { X t | ˆ P F [ X t ] = 0 } . By taking average with any P F [ X t ]sharing the same support with ˆ P F , we can easily seethat the fitness loss of a suboptimal strategy defined as∆Ψ[ Y t ] := ˆΨ[ Y t ] − Ψ[ Y t ] satisfies the following detailedFR: e − ∆Ψ[ Y t ] = ˆ P Y B [ X t ]ˆ P F [ X t ] P F [ X t ] P Y B [ X t ] = D ˆ P X B [ Y t ] E P F [ X t ] Q [ Y t ] . (11)An integral FR immediately follows as D e − ∆Ψ[ Y t ] E Q [ Y t ] = 1 , (12) Y t X t Z t I Y , Z I Y , Z | X B PhenotypeEnvironment Signal (A) (B) P Y B [ X t ] P F [ X t ] P F [ X t ] Q [ Y t ] .... .................... P Y ′ B [ X t ] P Y ′′ B [ X t ] P MB [ X t ] I X , Y , Z FIG. 2. (A) A schematic diagram of the forward, backwardand the marginal path-ensembles, P F [ X t ], P Y B [ X t ], and P MB [ X t ].(B) Venn’s diagram for information among phenotype, envi-ronment, and signal. The region circled in bold black is I Y , Z ,and those in green and red foreground are ˘ I Y , Z | X B and ˘ I X , Y , Z B ,respectively. Furthermore, we also have the Kawai-Parrondo-Broeck(KPB)-type FR as h ∆Ψ t i = D [ˆ P F || P F ] − D D [ˆ P Y B || P Y B ] E Q = D [ Q || D ˆ P X B E P F ] . (13)The second term shows that the average loss of a sub-optimal strategy, h ∆Ψ i , is determined by the strengthof contraction of the phenotypic path probabilities from D [ˆ P F || P F ] to D D [ˆ P Y B || P Y B ] E Q [ Y ] that is induced by selec-tion. The third term, in addition, shows that the lossis zero when D ˆ P X B [ Y t ] E P F equals to the statistics of en-vironment, Q [ Y t ]. In addition, this FRs can be used toquantify the loss by causal strategy even when ˆ P F is notcausal[27].All the result above, i.e., the consistency condition,the maximal cumulative fitness, and the FRs, can begeneralized for the situation where the environmen-tal signal, z t , is available. Let us define the for-ward path probability with the signal as P Z F [ X t ] := Q t − τ =0 T ( x τ +1 | x τ , z τ +1 ) P F ( x ) for T ( x ′ | x, z ). With thisforward path probability, we similarly have the cumula-tive fitness with the signal, Ψ[ Y t , Z t ] = ln (cid:10) e H [ X t , Y t ] (cid:11) P Z F [ X t ] ,and the backward path probability, P Y , Z B [ X t ] = e H [ X t , Y t ] − Ψ[ Y t Z t ] P Z F [ X t ]. As in the case without the signal,we consider general P Z F than Markov or causal ones. Thejoint and marginal backward probabilities are also de-fined as P JB [ X t , Y t , Z t ] := P Y , Z B [ X t ] Q [ Y t , Z t ], P MB [ X t , Z t ] := P Y t P JB [ X t , Y t , Z t ], P MB [ X t , Y t ] := P Z t P JB [ X t , Y t , Z t ], and P MB [ X t ] := P Y t , Z t P JB [ X t , Y t , Z t ]. Conditional probabili-ties are P Z B [ X t , Y t ] := P JB [ X t , Y t , Z t ] / Q [ Z t ], P Y B [ X t , Z t ] := P JB [ X t , Y t , Z t ] / Q [ Y t ], P M, Z B [ X t ] := P MB [ X t , Z t ] / Q [ Z t ], and P M, Y B [ X t ] := P MB [ X t , Y t ] / Q [ Y t ]. With these extensions, theaverage cumulative fitness, h Ψ t i := h Ψ[ Y t , Z t ] i Q [ Y t , Z t ] , is h Ψ t i = h H [ X t , Y t ] i P JB − I X , Y | Z B − D D [ P M, Z B [ X t ] || P Z F [ X t ]] E Q [ Z t ] , where I X , Y | Z B := D D [ P Z B [ X t , Y t ] || P M, Z B [ X t ] Q [ Y t | Z t ]] E Q [ Z t ] .We also have the optimal strategy with the signal as˘ P Z F := arg max P Z F h Ψ t i where we use ˘ to indicate the op-timal strategy with the signal to distinguish it from onewithout the signal. The optimal strategy satisfies thefollowing extended consistency condition as [27]˘ P M, Z B [ X t ] = X Y t ˘ P Y , Z B [ X t ] Q [ Y t | Z t ] = ˘ P Z F [ X t ] , (14)and the corresponding maximal cumulative fitness is D ˘Ψ t E = h H [ X t , Y t ] i ˘ P JB − ˘ I X , Y | Z B . (15)Similarly to the case without the signal, we can interpretthis relation as an information compression of Y t to X t with side information Z t [27].By using ˘ P X , Z B [ Y t ] := ˘ P JB / ˘ P MB [ X t , Z t ] = e H [ X t , Y t ] − ˘Ψ[ Y t , Z t ] Q [ Y t | Z t ] derived from the extendedconsistency condition, we similarly have the fitness loss,∆Ψ[ Y t , Z t ] := ˘Ψ[ Y t , Z t ] − Ψ[ Y t , Z t ], by a suboptimalstrategy with signal, P Z F as e − ∆Ψ[ Y t , Z t ] = D ˘ P X , Z B [ Y t ] E P Z F [ X t ] / Q [ Y t | Z t ] , (16)When ˆ P F [ X t ] shares the same support with ˘ P Z F [ X t ] , bychoosing the optimal strategy without signal as P Z F [ X t ] =ˆ P F [ X t ],the Sagawa-Ueda detailed FR [20] as e − ( ˆΨ[ Y t ]+ i [ Y t , Z t ] − ˘Ψ[ Y t , Z t ]) = ˆ P X B [ Y t ] / ˘ P X , Z B [ Y t ] , (17)where e i [ Y t , Z t ] := Q [ Y t , Z t ] / Q [ Y t ] Q [ Z t ]. The KPB-typeFR, D ˆΨ E + I Y , Z − D ˘Ψ E = D [˘ P JB || ˆ P Y B ˘ P MB ] ≥
0, shows that I Y , Z is an upper bound of the average gain of fitnessby sensing. Nonetheless, I Y , Z does not always properlyquantify the gain of fitness by sensing. For example, if allphenotypes have identical growth under two environmen-tal states, y and y ′ , i.e., h ( x, y ) = h ( x, y ′ ) for all x , theinformation in the signal to distinguish y and y ′ has nocontribution to fitness gain whereas I Y , Z increases. Theinformation relevant for fitness can be evaluated more tightly by the following FR as e − ( ˆΨ[ Y t ]+ i [ Y t , Z t ] − ˘ i X B [ Y t , Z t ] − ˘Ψ[ Y t , Z t ]) = ˆ P X B [ Y t ]˘ P M, X B [ Y t ] , (18)where e ˘ i X B [ Y t , Z t ] := ˘ P X , Z B [ Y t ] / ˘ P M, X B [ Y t ]. The KPB-typeFR shows that the multivariate mutual information,˘ I X , Y , Z B := I Y , Z − ˘ I Y , Z | X B , is the tighter bound for fitnessgain by sensing as D ˆΨ E + I X , Y , Z B − D ˘Ψ E = D D [˘ P M, X B || ˆ P X B ] E ˘ P MF ≥ , (19)[27]. In addition, the equality can be attained whenthe backward path probabilities of the optimal switchingwith and without sensing are identical as ˘ P M, X B = ˆ P X B . Be-cause ˘ I Y , Z | X B is the residual information of the signal onthe environment when we already know the phenotypepath (Fig. 2 (B)), ˘ I X , Y , Z B is the maximum informationof the signal that can be imprinted into the phenotypicdynamics by selection, i.e., the information of the signalconsumed and used in selection.In this work, we derived various FRs for fitness lossand gain with and without sensing the environment.These results generalize the previous results obtained byKelly[7], Hacco and Iwasa[2], and others for the averageof fitness gain and loss. In addition, by combining theFRs, we can also recover the result on the fitness gain bythe optimal causal strategy derived in [7](see [27]). Thekeystone for generalization was the introduction of path-wise formulation and the retrospective view of pheno-typic dynamics via the backward path probability. Thisalso enables us to clarify an information-theoretic aspectof selection as passive compression of environmental dy-namics onto the retrospective phenotypic one. Activeinformation processing by sensing interacts with this pas-sive processing, and thereby, the maximum gain of fitnessby sensing is quantified by the multivariate mutual infor-mation (eq. (19)). Because of the shared mathematicalstructures, this work will be the basis for the integrationof the information thermodynamics and evolutionary dy-namics to unveil the interdependencies among fitness, in-formation and entropy production[16–21].We thank Yoichi Wakamoto and Mikihiro Hashimotofor discussion. This research is supported partially byPlatform for Dynamic Approaches to Living Systemfrom MEXT, Japan, the Aihara Innovative Mathemat-ical Modelling Project, JSPS through the FIRST Pro-gram, CSTP, Japan, and the JST PRESTO program. [1] R. Levins, Evolution in Changing Environments, SomeTheoretical Explorations (Princeton University Press,1968).[2] P. Haccou and Y. Iwasa, Theoretical Population Biology , 212 (1995). [3] E. Kussell and S. Leibler, Science , 2075 (2005).[4] I. G. de Jong, P. Haccou, and O. P. Kuipers, Bioessays , 215 (2011).[5] T. J. Perkins and P. S. Swain, Mol Syst Biol , 326(2009). [6] T. J. Kobayashi and A. Kamimura, Adv. Exp. Med. Biol. , 275 (2012).[7] O. Rivoire and S. Leibler, J Stat Phys , 1124 (2011).[8] N. Q. Balaban, J. Merrin, R. Chait, L. Kowalik, andS. Leibler, Science , 1622 (2004).[9] R. Jayaraman, Journal of Biosciences , 795 (2008).[10] Y. Wakamoto, N. Dhar, R. Chait, K. Schneider,F. Signorino-Gelo, S. Leibler, and J. D. McKinney, Sci-ence , 91 (2013).[11] E. Ben-Jacob and D. Schultz, Proc. Natl. Acad. Sci.U.S.A. , 13197 (2010).[12] C. T. Bergstrom and M. Lachmann, pp. 50–54 (2004).[13] M. C. Donaldson-Matasci, C. T. Bergstrom, andM. Lachmann, Oikos , 219 (2010).[14] S. A. Frank, J. Evol. Biol. , 2377 (2012).[15] O. Rivoire and S. Leibler, Proceedings of the NationalAcademy of Sciences (2014).[16] Y. Iwasa, J. Theor. Biol. , 265 (1988).[17] H. P. de Vladar and N. H. Barton, arXiv pp. 424–432(2011), 1104.2854v1.[18] H. Qian, Quant Biol , 47 (2014).[19] U. Seifert, Rep. Prog. Phys. , 126001 (2012). [20] T. Sagawa, Thermodynamics of Information Processingin Small Systems (Springer, 2012).[21] V. Mustonen and M. L¨assig, Proceedings of the NationalAcademy of Sciences , 4248 (2010).[22] S. Leibler and E. Kussell, Proceedings of the NationalAcademy of Sciences , 13183 (2010).[23] Y. Wakamoto, A. Y. Grosberg, and E. Kussell, Evolution , 115 (2012).[24] G. Bianconi and C. Rahmede, Chaos, Solitons & Fractals , 555 (2012).[25] R. Oizumi and T. Takada, J. Theor. Biol. , 76 (2013).[26] Y. Sughiyama, T. J. Kobayashi, K. Tsumura, and K. Ai-hara, Phys Rev E Stat Nonlin Soft Matter Phys ,032120 (2015).[27] See Supplementary Material.[28] S. Kullback and R. A. Leibler, The Annals of Mathemat-ical Statistics pp. 79–86 (1951).[29] E. Baake and H.-O. Georgii, J. Math. Biol.54