Probabilistic model of N correlated binary random variables and non-extensive statistical mechanics
PProbabilistic model of N correlated binary random variables and non-extensivestatistical mechanics Julius Ruseckas ∗ Institute of Theoretical Physics and Astronomy, Vilnius University, A. Goˇstauto 12, LT-01108 Vilnius, Lithuania
The framework of non-extensive statistical mechanics, proposed by Tsallis, has been used todescribe a variety of systems. The non-extensive statistical mechanics is usually introduced in aformal way, thus simple models exhibiting some important properties described by the non-extensivestatistical mechanics are useful to provide deeper physical insights. In this article we present a simplemodel, consisting of a one-dimensional chain of particles characterized by binary random variables,that exhibits both the extensivity of the generalized entropy with q < q -Gaussian distributionin the limit of the large number of particles. I. INTRODUCTION
There exist a number of systems featuring long-range interactions, long-range memory, and anomalous diffusion, thatpossess anomalous properties in view of traditional Boltzmann-Gibbs statistical mechanics. Non-extensive statisticalmechanics is intended to describe such systems by generalizing the Boltzmann-Gibbs statistics [1–3]. In general, thenon-extensive statistical mechanics can be applied to describe the systems that, depending on the initial conditions,are not ergodic in the entire phase space and may prefer a particular subspace which has a scale invariant geometry,a hierarchical or multifractal structure. Concepts related to the non-extensive statistical mechanics have foundapplications in a variety of disciplines: physics, chemistry, biology, mathematics, economics, and informatics [4–6].The non-extensive statistical mechanics is based on a generalized entropy [1] S q = 1 − (cid:82) [ p ( x )] q dxq − , (1)where p ( x ) is a probability density function of finding the system in the state characterized by the parameter x , while q is a parameter describing the non-extensiveness of the system. Entropy (1) is an extension of the Boltzmann-Gibbsentropy S BG = − (cid:90) p ( x ) ln p ( x ) dx (2)which is recovered from Eq. (1) in the limit q → (cid:82) + ∞−∞ p ( x ) dx = 1 and (cid:82) + ∞−∞ x [ p ( x )] q dx (cid:82) + ∞−∞ [ p ( x )] q dx = σ q , (3)where σ q is the generalized second-order moment [22–24], one obtains the q -Gaussian distribution density p q ( x ) = C exp q ( − A q x ) . (4)Here exp q ( · ) is the q -exponential function, defined asexp q ( x ) ≡ [1 + (1 − q ) x ] − q + , (5) ∗ a r X i v : . [ c ond - m a t . s t a t - m ec h ] A ug with [ x ] + = x if x >
0, and [ x ] + = 0 otherwise. The q -Gaussian distribution (or distribution very close to it) appears inmany physical systems, such as cold atoms in dissipative optical lattices [25], dusty plasma [18], motion of hydra cells[26], and defect turbulence [27]. The q -Gaussian distribution is one of the most important distributions in the non-extensive statistical mechanics. It’s importance stems from the generalized central limit theorems [28–30]. Accordingto q -generalized central limit theorem, q -Gaussian can result from a sum of N q -independent random variables. The q -independence is defined in [28] through the q -product [31, 32], and the q -generalized Fourier transform [28]. When q (cid:54) = 1, q -independence corresponds to a global correlation of the N random variables. However, the rigorous definitionof q -independence is not transparent enough in physical terms.The non-extensive statistical mechanics is introduced in a formal way, starting from the maximization of thegeneralized entropy [1]. Therefore, simple models providing some degree of intuition about non-extensive statisticalmechanics can be useful for understanding it. There has been some effort to create such simple models. In Ref. [33]a system composed of N distinguishable particles, each particle characterized by a binary random variable, hasbeen constructed so that the number of states with non-zero probability grows with the number of particles N notexponentially, but as a power law. For such a system in the limit N → ∞ the ratio S q ( N ) /N is finite not for theBoltzmann-Gibbs entropy but for the generalized entropy with some specific value of q . The starting point in theconstruction is the Leibnitz triangle, then initial probabilities are redistributed into a small number of all the otherpossible states, in such a way that the norm is preserved. For example, in the restricted uniform model [33] for a fixedvalue of N all nonvanishing probabilities are equal. In the proposed models that yield q (cid:54) = 1 there are d + 1 no-zeroprobabilities and the value of q is given by q = 1 − /d .In Refs. [34, 35], the goal has been to construct simple models providing q -Gaussian distributions. As in [33], themodels considered in [34] consist of N independent and distinguishable binary variables, each of them having twoequally probable states. The models presented in [34] are strictly scale-invariant, however, they do not approach a q -Gaussian form when the number of particles N in the model increases [36]. The situation is different with themodels presented in [35]: the two proposed models do approach a q -Gaussian form, the second of them does so byconstruction. All models in [34, 35], except the last model of [35] are for q ≤
1. The drawback of the models fromRef. [35] is that the standard Boltzmann-Gibbs entropy remains extensive. In addition, the models are constructedartificially and it is hard to see how they can be related to real physical systems.The goal of this paper is to provide a simple model that achieves both the extensivity of the generalized entropywith q (cid:54) = 1 and q -Gaussian distribution in the limit of the large number of particles. In addition, we want to constructa model that is closer to situations in physical systems. We expect that such a model can provide deeper insights intonon-extensive statistical mechanics than the previously constructed simple models.The paper is organized as follows: To highlight differences from our proposed model, a simple model consisting ofuncorrelated binary random variables and leading to extensive Boltzmann-Gibbs entropy and a Gaussian distributionis presented in Section II. In Section III we construct a simple model exhibiting the extensivity of the generalizedentropy with q (cid:54) = 1 and q -Gaussian distribution in the limit of the large number of particles. Section IV summarizesour findings. II. MODEL OF UNCORRELATED BINARY RANDOM VARIABLES
At first let us consider a model consisting from N uncorrelated binary random variables. Physical implementationof such a model could be N particles of spin , the projection of each spin to the z axis can acquire the values ± .The microscopic configuration of the system can be described by a sequence of spin projections s s . . . s N , whereeach s i = ± . There are W = 2 N different microscopic configurations. As is usual in statistical mechanics for thedescription of a microcanonical ensemble, we assign to each microscopic configuration the same probability. Thus theprobability of each microscopic configuration is P = 1 W = 12 N . (6)Note, that this system has a property of composability: if we have two spin chains with W and W microscopicconfigurations, then we can join them to form a larger system. The description of a larger system is just concatenationof the descriptions of each subsystems and the number of microscopic configurations of the whole system is W = W W .The standard Boltzmann-Gibbs entropy S BG = k B ln W is extensive for this system: S BG = N k B ln 2 grows linearlywith N .Let us consider a macroscopic quantity, the total spin of the system M = N (cid:88) i =1 s i . (7) FIG. 1. One-dimensional spin chain having N spins. There are d spin flips, so that the spin chain consists of d + 1 domains ofspins pointing to the same direction. The lengths of the domains are n , n , . . . , n d +1 . The total spin can take values M = − N , − N + 1 , . . . , N − , N . The value of M can be obtained when there are n = M + N spins with the projection + , the remaining spins have projection − . The macroscopic configurationcorresponding to the given value of M can be realized by N ! n !( N − n )! microscopic configurations, thus using Eq. (6) theprobability of each macroscopic configuration is P M = 12 N N ! n !( N − n )! . (8)Note, that the probabilities of macroscopic configurations are normalized: N (cid:88) M = − N P M = 1 . (9)Using Eq. (8) we can calculate the average spin of the system (cid:104) M (cid:105) = 0 and the standard deviation (cid:112) (cid:104) M (cid:105) − (cid:104) M (cid:105) = √ N . (10)From Eq. (10) it follows that the relative width of the distribution of the total spin M decreases with number of spins N as √ N . When N is large then we can approximate the factorials using Stirling formula n ! ≈ √ πnn n e − n (11)and obtain a Gaussian distribution P M ≈ (cid:113) π N e − M N (12)The Gaussian distribution can be obtained by maximizing the Boltzmann-Gibbs entropy (2) with appropriate con-straints. III. MODEL OF CORRELATED SPINS
In this Section we investigate a system consisting from N correlated binary random variables. Similarly as in theprevious Section, we can think about a one-dimensional spin chain consisting from N spins. However, the spins arecorrelated: spins next to each other have almost always the same direction, except there are d cases when the nextspin has an opposite direction, as is shown in Fig. 1. Thus the spin chain consists from d + 1 domains with spinspointing in the same direction and has d boundaries between domains. This model is inspired by a connection betweennon-extensive statistics and critical phenomena [37–39]. As it has been shown in Refs. [37–39], the properties of asingle large cluster of the order parameter at a critical point in thermal systems can be described by non-extensivestatistics. The restriction of the number of allowed states in our model is very similar to the models presented in [33].Let us calculate the number of allowed microscopic configurations of the spin chain. If the length of i -th domain is n i then we need to calculate the number of possible partitions such that d +1 (cid:88) i =1 n i = N . (13)
20 40 60 80 100050100150200 N S q (cid:72) N (cid:76) q (cid:61) (cid:61) (cid:61) FIG. 2. Dependence of the generalized entropy S q on the number of particles N in the spin chain for various values of q . Thenumber of domain walls d = 4. Only for q = q stat = 1 − /d = 0 .
75 the limit lim N →∞ S q ( N ) /N has a finite value; the limitvanishes (diverges) for q > .
75 ( q < . This number is equivalent to the number of ways one can place d domain boundaries into N − W = 2( N − d !( N − d − d ! ( N − · · · ( N − d ) . (14)For large N (cid:29) d we have that the number of microscopic configurations grows as a power-law of the number of spins,not exponentially: W ∼ d ! N d . (15)Assigning to each allowed microscopic configuration the same probability P = 1 /W we get that in this situation thetraditional Boltzmann-Gibbs entropy is not linearly proportional to N and thus is not extensive. The generalizedentropy Eq. (1) for equal probabilities 1 /W takes the form S q = k B − W − q q − . (16)Using Eqs. (15) and (16) on gets that the generalized entropy is extensive (proportional to N ) only when q is q stat = 1 − d . (17)This is the same dependency of q on d as in [33].In Fig. 2 the dependence of the generalized entropy (16) on the size N of the spin chain for various values of q isshown. As one can see, only for q = q stat the limit lim N →∞ S q /N is finite, this limit vanishes or diverges for all othervalues of q . Thus our simple model can be characterized by the generalized entropy with q < M . If there are d + 1 domains and the total spin is M then we have the equality s d +1 (cid:88) i =1 ( − i − n i = M . (18)Eq. (18) can be written as N odd − N even = Ms , (19)where N odd = (cid:98) d (cid:99) (cid:88) k =0 n k (20)is the total length of odd-numbered domains, N even = (cid:98) d +12 (cid:99) (cid:88) k =1 n k (21)is the total length of even-numbered domains. Here (cid:98)·(cid:99) denotes an integer part of a number. In addition, the totallengths of odd- and even-numbered domains should obey the equation N odd + N even = N . (22)Eqs. (19) and (22) have only one solution N odd = 12 (cid:18) N + Ms (cid:19) , N even = 12 (cid:18) N − Ms (cid:19) . (23)The total length of odd-numbered domains N odd is a sum of (cid:4) d (cid:5) +1 terms, the total length of even-numbered domains N even is a sum of (cid:4) d +12 (cid:5) terms. Similarly as in Eq. (14), the number of ways to choose n , n , . . . , n d +1 is equal to thenumber of ways to place (cid:4) d (cid:5) domain boundaries into N odd − (cid:4) d +12 (cid:5) − N even − M , N and s are given is W ( N, M, s ) = 1 (cid:0)(cid:4) d (cid:5)(cid:1) ! (cid:18) N odd − (cid:19) · · · (cid:18) N odd − (cid:22) d (cid:23)(cid:19) (cid:0)(cid:4) d − (cid:5)(cid:1) ! (cid:18) N even − (cid:19) · · · (cid:18) N even − (cid:22) d − (cid:23)(cid:19) . (24)The number of microscopic configurations corresponding to a given value of M is W ( N, M ) = (cid:88) s = ± / W ( N, M, s ) . (25)Using Eqs. (23)–(25) we obtain that the number of microscopic configurations corresponding to a given value of M is W ( N, M ) = 12 d − (cid:2)(cid:0) d − (cid:1) ! (cid:3) (cid:0) ( N − − M (cid:1) · · · (cid:0) ( N − ( d − − M (cid:1) (26)when d is odd and W ( N, M ) = ( N − d )2 d − d (cid:2)(cid:0) d − (cid:1) ! (cid:3) (cid:0) ( N − − M (cid:1) · · · (cid:0) ( N − ( d − − M (cid:1) (27)when d is even. When N (cid:29) d then Eqs. (26) and (27) can be approximated as W ( N, M ) ≈ N d − d − (cid:0)(cid:4) d (cid:5)(cid:1) ! (cid:0)(cid:4) d − (cid:5)(cid:1) ! (cid:18) − M N (cid:19) (cid:98) d − (cid:99) . (28)Instead of the total spin M it is convenient to consider a scaled variable x = 2 MN . (29)From Eq. (28) we get that the distribution P x ( x ) of the variable x for large N is proportional to P x ( x ) ∝ (1 − x ) (cid:98) d − (cid:99) . (30) (cid:45) (cid:45)
200 0 200 4000.00000.00050.00100.00150.0020 MP (cid:72) M (cid:76) FIG. 3. Comparison of numerically obtained histogram of total spin M (gray area) with q -Gaussian distribution (solid redline). The histogram is calculated using an ensemble of randomly generated spin chains that have the length N = 1000 spinsand d = 5 spin flips. The q -Gaussian distribution is given by Eq. (32) with x q = N/ When q < q -Gaussian distribution (4) has compact support, the range of possible values of x is limitedby the condition | x | (cid:54) x q , where x q = 1 (cid:112) (1 − q ) A q . (31)Using the limiting value x q the expression (4) for the q -Gaussian distribution takes the form p q ( x ) = Γ (cid:16) − q − q ) (cid:17) √ πx q Γ (cid:16) − q − q (cid:17) (cid:20) − x x q (cid:21) − q + . (32)By rescaling the variable x the expression (32) for q -Gaussian distribution in the case of q < p q ( x ) ∝ (1 − x ) − q . (33)Comparing Eq. (30) with Eq. (33) we see that the distribution of the total spin M in our model in the limit of largenumber of spins N is a q -Gaussian with q dist = 1 − (cid:4) d − (cid:5) . (34)Comparison of the histogram of the total spin M , calculated using an ans amble of randomly generated spin chains,with a q -Gaussian distribution (32) is shown in Fig. 3. We see that the q -Gaussian describes the distribution of thetotal spin very well. By increasing the number of spin flips d , both q stat and q dist approaches the value of 1, as followsfrom Eqs. (17) and (34). In the limit of d → ∞ the distribution of total spin becomes Gaussian.From Eqs. (17) and (34) follows the relationship between the two q values:11 − q dist = (cid:22) (cid:18) − q stat − (cid:19)(cid:23) . (35)This equation is similar to relations presented in Eq. (18) of Ref. [40], in the footnote in page 15378 of Ref. [33], andEq. (5) in Ref. [41]. IV. DISCUSSION
Our simple model of a spin chain exhibits both non-extensive behavior of Boltzmann-Gibbs entropy and q -Gaussiandistribution of the total spin. This is in contrast to the other models involving N binary random variables: themodels presented in [33] demonstrate that the generalized entropy with q (cid:54) = 1 can be extensive, but they do notprovide q -Gaussian distribution, whereas the models from Ref. [35] yield q -Gaussians but for them the Boltzmann-Gibbs entropy is extensive. Thus the model, presented in this paper, is an improvement over earlier models and mayprovide deeper insights into non-extensive statistical mechanics.By comparing the model with the chain of uncorrelated spins, presented in Section II, we see that the reason forthe non-extensivity of Boltzmann-Gibbs entropy and the extensivity of the generalized entropy is the reduction ofthe number of allowed microscopic configurations, resulting from the restriction of the number of allowed spin flips.Similar reason is behind the differences in the distribution of the total spin M : the number of possible configurationshaving N odd spins pointing in one direction and N even spins in the opposite direction in the chain of uncorrelatedspins is the exponential function of N odd N even , resulting in the Gaussian function of M . When the number of spinflips is restricted, the number of configurations is a power-law function of N odd N even , resulting in the q -Gaussian.The model can be slightly modified by requiring that the number of spin flips is not constant, but can be a randomnumber not larger than d . Since the number of microscopic configurations having d spin flips grows as N d , thecontribution of configurations with smaller number of spin flips becomes negligible in the limit of large N . Thus theconclusions of Section III remain valid also for this modified model.Note, that the values of q obtained from the entropy (17) and from the distribution (34) are different. This differenceis not surprising and is similar to the q -triplet in the non-extensive statistical mechanics. Usually the systems describedby the non-extensive statistical mechanics have three different values of q , the q -triplet [1]. This triplet consist fromthe values of q , ( q sen , q rel , q stat ) obtained from the sensitivity to the initial conditions, the relaxation in phase-space,and the distribution of energies at a stationary state [1]. Since our model does not involve the energy and there isno evolution in time, we have obtained only two values q . It would be interesting task for the future to extend themodel and get the full q -triplet.The model presented here provides q -Gaussian distribution with q <
1. Thus another open question is whether itis possible to modify the model to obtain q -Gaussian distribution with q > [1] C. Tsallis, Introduction to Nonextensive Statistical Mechanics – Approaching a Complex World (Springer, New York, 2009).[2] C. Tsallis, Braz. J. Phys. , 337 (2009).[3] L. Telesca, Tectonophysics , 155 (2010).[4] C. M. Gell-Mann and C. Tsallis, Nonextensive Entropy—Interdisciplinary Applications (Oxford University Press, NY,2004).[5] S. Abe, Astrophys. Space Sci. , 241 (2006).[6] S. Picoli, R. S. Mendes, L. C. Malacarne, and R. P. B. Santos, Braz. J. Phys. , 468 (2009).[7] R. Hanel and S. Thurner, EPL , 20006 (2011).[8] R. Hanel, S. Thurner, and M. Gell-Mann, PNAS , 6390 (2011).[9] O. Afsar and U. Tirnakli, EPL , 20003 (2013).[10] U. Tirnakli, C. Tsallis, and C. Beck, Phys. Rev. E , 056209 (2009).[11] G. Ruiz, T. Bountis, and C. Tsallis, Int. J. Bifurcation Chaos , 1250208 (2012).[12] Z. Huang, G. Su, A. El Kaabouchi, Q. A. Wang, and J. Chen, J. Stat. Mech. , L05001 (2010).[13] J. Prehl, C. Essex, and K. H. Hoffman, Entropy , 701 (2012).[14] C. Beck and S. Miah, Phys. Rev. E , 031002 (2013).[15] A. A. Budini, Phys. Rev. E , 011109 (2012).[16] J.-L. Du, J. Stat. Mech. , P02006 (2012).[17] R. M. Pickup, R. Cywinski, C. Pappas, B. Farago, and P. Fouquet, Phys. Rev. Lett. , 097202 (2009).[18] B. Liu and J. Goree, Phys. Rev. Lett. , 055003 (2008).[19] L. Borland, Quantit. Finance , 1367 (2012).[20] S. Drozdz, J. Kwapien, P. Oswiecimka, and R. Rak, New J. Phys. , 105003 (2010).[21] V. Gontis, J. Ruseckas, and A. Kononovicius, Physica A , 100 (2010).[22] C. Tsallis, A. R. Plastino, and R. S. Mendes, Physica A , 534 (1998).[23] D. Prato and C. Tsallis, Phys. Rev. E , 2398 (1999).[24] C. Tsallis, Braz. J. Phys , 1 (1999).[25] P. Douglas, S. Bergamini, and F. Renzoni, Phys. Rev. Lett. , 110601 (2006).[26] A. Upaddhyaya, J. P. Rieu, J. A. Glazier, and Y. Sawada, Physica A , 459 (2001).[27] K. E. Daniels, C. Beck, and E. Bodenschatz, Physica D , 208 (2004).[28] S. Umarov, C. Tsallis, and S. Steinberg, Milan J. Math. , 307 (2008).[29] S. Umarov, C. Tsallis, M. Gell-Mann, and S. Steinberg, J. Math. Phys. , 033502 (2010).[30] S. Umarov and C. Tsallis, in Complexity, Metastability and Nonextensivity , American Institute of Physics ConferenceProceedings, Vol. 965, edited by S. Abe, H. Herrmann, P. Quarati, A. Rapisarda, and C. Tsallis (New York, 2007) pp.34–42. [31] L. Nivanen, A. Le Mehaute, and Q. A. Wang, Rep. Math. Phys. , 437 (2003).[32] E. P. Borges, Physica A , 95 (2004).[33] C. Tsallis, M. Gell-Mann, and Y. Sato, Proc. Natl. Acad. Sc. USA , 15377 (2005).[34] L. G. Moyano, C. Tsallis, and M. Gell-Mann, Europhys. Lett. , 813 (2006).[35] A. Rodriguez, V. Schwammle, and C. Tsallis, J. Stat. Mech. , P09006 (2008).[36] H. J. Hilhorst and G. Schehr, J. Stat. Mech. , P06003 (2007).[37] A. Robledo, Phys. Rev. Lett. , 2289 (1999).[38] A. Robledo, Mol. Phys. , 3025 (2005).[39] A. Robledo, Int. J. Mod. Phys. B , 3947 (2007).[40] G. Ruiz and C. Tsallis, Phys. Lett. A , 2451 (2012).[41] A. Celikoglu, U. Tirnakli, and S. Queir´os, Phys. Rev. E82