Analysis of the convergence of the degree distribution of contracting random networks towards a Poisson distribution using the relative entropy
aa r X i v : . [ phy s i c s . s o c - ph ] S e p Analysis of the convergence of the degree distribution ofcontracting random networks towards a Poisson distributionusing the relative entropy
Ido Tishby, Ofer Biham and Eytan Katzav
Racah Institute of Physics, The Hebrew University, Jerusalem 9190401, Israel
Abstract
We present analytical results for the structural evolution of random networks undergoing contrac-tion processes via generic node deletion scenarios, namely, random deletion, preferential deletionand propagating deletion. Focusing on configuration model networks, which exhibit a given degreedistribution P ( k ) and no correlations, we show using a rigorous argument that upon contractionthe degree distributions of these networks converge towards a Poisson distribution. To this end, weuse the relative entropy S t = S [ P t ( k ) || π ( k |h K i t )] of the degree distribution P t ( k ) of the contract-ing network at time t with respect to the corresponding Poisson distribution π ( k |h K i t ) with thesame mean degree h K i t as a distance measure between P t ( k ) and Poisson. The relative entropyis suitable as a distance measure since it satisfies S t ≥ P t ( k ), whileequality is obtained only for P t ( k ) = π ( k |h K i t ). We derive an equation for the time derivative dS t /dt during network contraction and show that the relative entropy decreases monotonically tozero during the contraction process. We thus conclude that the degree distributions of contract-ing configuration model networks converge towards a Poisson distribution. Since the contractingnetworks remain uncorrelated, this means that their structures converge towards an Erd˝os-R´enyi(ER) graph structure, substantiating earlier results obtained using direct integration of the masterequation and computer simulations [I. Tishby, O. Biham and E. Katzav, Phys. Rev. E , 032314(2019)]. We demonstrate the convergence for configuration model networks with degenerate de-gree distributions (random regular graphs), exponential degree distributions and power-law degreedistributions (scale-free networks). . INTRODUCTION Complex network architectures and dynamical processes taking place on them play acentral role in current research [1–3]. Since the 1960s, mathematical studies of networkswere focused on model systems such as the Erd˝os-R´enyi (ER) network [4–6], which exhibitsa Poisson degree distribution of the form π ( k | c ) = e − c c k /k !, where c is the mean degree[7]. In an ER network of N nodes, each pair of nodes is connected with probability p ,where p = c/ ( N − P ( k ) and no degree-degree correlations. During the contraction processthe degree distribution of the network evolves. We denote the degree distribution at time t by P t ( k ) and its mean degree by h K i t . We use the relative entropy S t = S [ P t ( k ) || π ( k |h K i t )]as a distance measure between the degree distribution P t ( k ) of the contracting network andthe corresponding Poisson distribution π ( k |h K i t ) with the same mean degree h K i t . Usingthis measure we obtain rigorous results for the convergence of the degree distribution ofcontracting networks towards a Poisson distribution. To this end, we derive an equation forthe time derivative dS t /dt of the relative entropy during network contraction. This equation3an be expressed in the form dS t /dt = ∆ A ( t ) + ∆ B ( t ). We show that ∆ A ( t ) < B ( t ) < B ( t ) termturns out to be small and has little effect on the convergence, which is dominated by ∆ A ( t ).This implies that the relative entropy decreases monotonically during the contraction pro-cess. Since the relative entropy satisfies S t ≥ P t ( k ), whileequality is obtained only for P t ( k ) = π ( k |h K i t ) we conclude that the degree distributions ofcontracting networks converge towards a Poisson distribution. This conclusion is corrobo-rated by the fact that the relative entropy provides an upper bound for the total variationdistance, which is a standard measure of the difference between probability distributions.We demonstrate the convergence for configuration model networks with a degenerate degreedistribution (random regular graphs), exponential degree distribution and power-law degreedistribution (scale-free networks).The paper is organized as follows. In Sec. II we present the three generic network con-traction scenarios studied in this paper. In Sec. III we present the master equation andshow that the Poisson distribution is a solution of the master equation for the three con-traction scenarios. In Sec. IV we present the relative entropy and express it in terms of theShannon entropy and the cross-entropy. In Sec. V we present rigorous results showing thatthe relative entropy decays to zero in any of the three contraction scenarios. In Sec. VIwe present analytical results and computer simulations for the contraction of configurationmodel networks with a degenerate degree distribution (random regular graphs), an expo-nential degree distribution and a power-law degree distribution (scale-free networks). Theresults are discussed in Sec. VII and summarized in Sec. VIII. II. NETWORK CONTRACTION PROCESSES
We consider network contraction processes in which at each time step a single node isdeleted together with its links. The initial network consists of N nodes, so at time t thenetwork size is reduced to N t = N − t nodes. The deletion of a node of degree k , whoseneighbors are of degrees k ′ i , i = 1 , , . . . , k , eliminates the deleted node from the degree4equence and reduces the degrees of its neighbors to k ′ i − i = 1 , , . . . , k . The nodedeleted at each time step is selected randomly. However, the probability of a node to beselected for deletion may depend on its degree, according to the specific network contractionscenario. Here we focus on three generic scenarios of network contraction: the scenario ofrandom node deletion that describes the random, inadvertent failure of nodes, the scenarioof preferential node deletion that describes intentional attacks that are more likely to focuson highly connected nodes and the scenario of propagating node deletion that describescascading failures and infectious processes that spread throughout the network.In the random deletion scenario, at each time step a random node is selected for deletion.In this scenario each one of the nodes in the network at time t has the same probabilityto be selected for deletion, regardless of its degree. Since at time t there are N t nodes inthe network, the probability of each one of them to be selected for deletion is 1 /N t . In thepreferential deletion scenario the probability of a node to be selected for deletion at time t is proportional to its degree at that specific time. This means that the probability of agiven node of degree k to be deleted at time t is k/ [ N t h K i t ]. This is equivalent to selectinga random edge in the network and randomly choosing for deletion one of the two nodes atits ends. In the propagating deletion scenario at each time step the node to be deleted israndomly selected among the neighbors of the node deleted in the previous time step. In casethat the node deleted in the previous time step does not have any yet-undeleted neighborwe pick a random node, randomly select one of its neighbors for deletion and continue theprocess from there.Here we focus on the contraction of undirected networks of initial size N , which are drawnfrom a configuration model network ensemble with a given initial degree distribution P ( k )and no degree-degree correlations. The degree distribution is bounded from above and belowsuch that k min ≤ k ≤ k max . For example, the commonly used choice of k min = 1 eliminatesthe possibility of isolated nodes in the network. Choosing k min = 2 also eliminates the leafnodes. Controlling the upper bound is important in the case of fat-tail degree distributionssuch as power-law degree distributions. The configuration model network ensemble is amaximum entropy ensemble under the condition that the degree distribution P ( k ) is imposed[32–37]. In such uncorrelated networks the deletion of a node at time t does not inducecorrelations between the remaining N t − N t , the resulting network remains a configuration model5etwork with a suitably adjusted degree distribution P t +1 ( k ). III. THE MASTER EQUATION AND ITS POISSON SOLUTION
Consider an ensemble of networks of size N and degree distribution P ( k ), with meandegree h K i . At each time step a single node is deleted from the network. In addition tothe primary effect of the loss of the deleted node, the damage to the network also includes asecondary effect as each neighbor of the deleted node loses one link. An intrinsic property ofthe secondary effect is that it is always of a preferential nature. This is due to the fact thatthe probability of a node of degree k ′ to be a neighbor of the deleted node is proportional to k ′ . The number of nodes in the network at time t is N t = N − t . The number of nodes ofdegree k at time t is denoted by N t ( k ), where P k N t ( k ) = N t . The time dependent degreedistribution is given by P t ( k ) = N t ( k ) N t . (1)The mean degree and the second moment of the degree distribution at time t are denotedby h K n i t where n = 1 and 2, respectively.The master equation [38, 39] for the temporal evolution of the degree distribution P t ( k )during network contraction processes was derived in Ref. [31]. To demonstrate the derivationof the master equation we consider below the relatively simple case of random node deletion.The time dependence of N t ( k ) depends on the primary effect, given by the probability thatthe node selected for deletion is of degree k , as well as on the secondary effect of node deletionon neighboring nodes of degrees k and k + 1. In random node deletion the probability thatthe node selected for deletion at time t is of degree k is given by N t ( k ) /N t . Thus, the rateat which N t ( k ) decreases due to the primary effect of the deletion of nodes of degree k isgiven by R t ( k → ∅ ) = N t ( k ) N t , (2)where ∅ represents the empty set. In case that the node deleted at time t is of degree k ′ ,it affects k ′ adjacent nodes, which lose one link each. The probability of each one of these k ′ nodes to be of degree k is given by kN t ( k ) / [ N t h K i t ]. We denote by W t ( k → k − k that lose a link at time t and6re reduced to degree k −
1. Summing up over all possible values of k ′ , we find that thesecondary effect of random node deletion on nodes of degree k amounts to W t ( k → k −
1) = kN t ( k ) N t . (3)Similarly, the secondary effect on nodes of degree k + 1 amounts to W t ( k + 1 → k ) = ( k + 1) N t ( k + 1) N t . (4)The time evolution of N t ( k ) can be expressed in terms of the forward difference∆ t N t ( k ) = N t +1 ( k ) − N t ( k ) . (5)Combining the primary and the secondary effects on the time dependence of N t ( k ) we obtain∆ t N t ( k ) = − R t ( k → ∅ ) + [ W t ( k + 1 → k ) − W t ( k → k − . (6)Since nodes are discrete entities the process of node deletion is intrinsically discrete. There-fore, the replacement of the forward difference ∆ t N t ( k ) by a time derivative of the form dN t ( k ) /dt involves an approximation. The error associated with this approximation wasevaluated in Ref. [31]. It was shown that except for the limit of extremely narrow degreedistributions the error is of order 1 /N t , which quickly vanishes in the large network limit.This means that the replacement of the forward difference by a time derivative has littleeffect on the results, and a clear technical advantage.Inserting the expressions for R t ( k → ∅ ), W t ( k → k −
1) and W t ( k + 1 → k ) from Eqs.(2), (3) and (4), respectively into Eq. (6) and replacing ∆ t N t ( k ) by dN t ( k ) /dt we obtain ddt N t ( k ) = ( k + 1)[ N t ( k + 1) − N t ( k )] N t . (7)The derivation of the master equation is completed by taking the time derivative of Eq. (1),which is given by ddt P t ( k ) = 1 N t ddt N t ( k ) − N t ( k ) N t ddt N t . (8)Inserting the time derivative of N t ( k ) from Eq. (7) into Eq. (8) and using the fact that dN t /dt = −
1, we obtain the master equation for the random deletion scenario, which isgiven by 7 dt P t ( k ) = 1 N t [( k + 1) P t ( k + 1) − kP t ( k )] . (9)The derivation of the master equations for the preferential deletion and the propagatingdeletion scenarios can be performed along similar lines [31]. Interestingly, the resultingmaster equations for these three network contraction scenarios can be written in a unifiedmanner, in the form ddt P t ( k ) = F A ( t ) + F B ( t ) , (10)where F A ( t ) = A t N t [( k + 1) P t ( k + 1) − kP t ( k )] (11)accounts for the secondary effect on the neighbors of the deleted node, which lose one linkeach, while F B ( t ) = − B t ( k ) N t P t ( k ) (12)accounts for the primary effect, namely, the loss of the deleted node [31]. The coefficients A t and B t ( k ) are given by A t = h K i t h K i t preferential deletion h K i t − h K i t h K i t propagating deletion , (13)and B t ( k ) = k −h K i t h K i t preferential deletion k −h K i t h K i t propagating deletion . (14)The master equation consists of a set of coupled ordinary differential equations for P t ( k ), k = 0 , , , . . . , k max , or in other words it is a partial difference-differential equation. Inorder to calculate the time evolution of the degree distribution P t ( k ) during the contractionprocess one solves the master equation using direct numerical integration [40], starting fromthe initial network that consists of N nodes whose degree distribution is P ( k ). For any8 (cid:1731)(cid:1837)(cid:1732)(cid:1842) ( (cid:1863) ) (cid:1863) (cid:1863)(cid:1863) + 1 (cid:1863) (cid:3398) (cid:1853) (cid:1854) FIG. 1: (Color online) Illustration of the time dependence of the degree distribution P t ( k ) duringnetwork contraction processes, described by the master equation (10). (a) In the trickle-down term F A ( t ), given by Eq. (11), the probability flows downwards step by step from degree k + 1 to k andfrom k to k −
1. This way high degree nodes become less probable and low degree nodes becomemore probable as the contraction process evolves. (b) In the redistribution term F B ( t ), given byEq. (12), for values of k above the mean degree h K i t the probability P t ( k ) decreases at a rateproportional to k − h K i t , while for values of k below h K i t the probability P t ( k ) increases at a rateproportional to h K i t − k . Here the flow of probability is non-local in the k axis, namely, probabilityis lost at high degrees and instantaneously emerges at low degrees. finite network the degree distribution is bounded from above by an upper bound denotedby k max , which satisfies the condition k max ≤ N −
1. Since the contraction process canonly delete edges from the remaining nodes and cannot increase the degree of any node, theupper cutoff k max is maintained throughout the contraction process.The F A ( t ) term of the master equation, given by Eq. (11), is referred to as the trickle-down term [41]. This term represents the step by step downwards flow of probability fromhigh to low degrees. This process is illustrated in Fig. 1(a). The coefficient A t of thetrickle-down term depends on the network contraction scenario according to Eq. (13). Inthe case of random node deletion A t = 1, because the probability of a node to be selectedfor deletion does not depend on its degree. In the case of preferential node deletion A t isproportional to h K i t because the probability of a node to be deleted is proportional to its9egree k while the magnitude of the secondary effect is also proportional to k .The F B ( t ) term of the master equation, given by Eq. (12), is referred to as the redistribu-tion term. As can be seen in Eq. (14), this term vanishes in the random deletion scenario.However, in the preferential and propagating deletion scenarios the redistribution term isnegative for k > h K i t and positive for k < h K i t . Thus the redistribution term decreasesthe probabilities P t ( k ) for values of k that are above the mean degree and increases themfor values of k that are below the mean degree, as illustrated in Fig. 1(b). The size of theredistribution term is proportional to the absolute value | k − h K i t | , which means that nodesof degrees that are much higher or much lower than h K i t are most strongly affected by thisterm.Consider an ER network of N t nodes with mean degree c t . Its degree distribution followsa Poisson distribution of the form π ( k | c t ) = e − c t c kt k ! . (15)The second moment of this degree distribution is equal to c t ( c t + 1). To examine thecontraction process of ER networks we start from an initial network of N nodes whosedegree distribution follows a Poisson distribution π ( k | c ), where c is the mean degree ofthe initial network. Inserting π ( k | c t ) into the master equation (10) we find that the timederivative on the left hand side is given by ddt π ( k | c t ) = − dc t dt (cid:18) − kc t (cid:19) π ( k | c t ) , (16)On the other hand, inserting π ( k | c t ) on the right hand side of Eq. (10), we obtain ddt π ( k | c t ) = A t N t ( c t − k ) π ( k | c t ) − B t ( k ) N t π ( k | c t ) , (17)In order that π ( k | c t ) will be a solution of Eq. (10), the right hand sides of Eqs. (16) and(17) must coincide. In the case of random deletion this implies that1 c t dc t dt = − N t . (18)Integrating both sides for t ′ = 0 to t , we obtain the solution c t = c N t /N . Repeatingthe analysis presented above for the cases of preferential deletion and propagating deletion10t is found that π ( k | c t ) solves the master equation (10) for the three network contractionscenarios, while the mean degree, c t decreases linearly in time according to c t = c − Rt, (19)where the rate R depends on the network contraction scenario, and is given by R = c N random deletion c +2 N preferential deletion c N propagating deletion . (20)This means that an ER network exposed to any one of the three contraction scenariosremains an ER network at all times, with a mean degree that decreases according to Eq.(19). IV. THE RELATIVE ENTROPY
In order to establish that networks exposed to these contraction scenarios actually con-verge towards the ER structure, it remains to show that the Poisson solution is attractive.To quantify the convergence of P t ( k ), whose mean degree is h K i t , towards a Poisson dis-tribution, we use the relative entropy (also referred to as the Kullback-Leibler divergence),defined by [42] S t = S [ P t ( k ) || π ( k |h K i t )] = ∞ X k =0 P t ( k ) ln (cid:20) P t ( k ) π ( k |h K i t ) (cid:21) , (21)where π ( k |h K i t ) is the Poisson distribution, given by Eq. (15), with the same mean degreeas P t ( k ), namely, h K i t . The relative entropy S t is a distance measure between the wholedegree distribution P t ( k ) and the reference distribution π ( k |h K i t ). It also quantifies theadded information associated with constraining the degree distribution P t ( k ) rather thanonly the mean degree h K i t , as nicely shown in Refs. [35–37]. The Poisson distribution is aproper reference distribution for the relative entropy because it satisfies π ( k |h K i t ) > k . Using the log-sum inequality [43], one can show that therelative entropy is always non-negative and satisfies S t = 0 if and only if P t ( k ) = π ( k |h K i t )[44, 45]. Therefore, S t can be used as a measure of the distance between a given networkand the corresponding ER network with the same mean degree.11he relative entropy S [ P ( k ) || π ( k | c )] of a degree distribution P ( k ) with mean degree h K i with respect to a Poisson distribution π ( k | c ) with mean degree c can be decomposed in theform S [ P ( k ) || π ( k | c )] = − S [ P ( k )] + C [ P ( k ) || π ( k | c )] (22)where S [ P ( k )] = − ∞ X k =0 P ( k ) ln[ P ( k )] (23)is the Shannon entropy [46] of P ( k ), while C [ P ( k ) || π ( k | c )] = − ∞ X k =0 P ( k ) ln[ π ( k | c )] , (24)is the cross-entropy [47] between P ( k ) and π ( k | c ). The Poisson distribution π ( k | c ) satisfiesln[ π ( k | c )] = − c + k ln( c ) − ln( k !) . (25)Inserting ln[ π ( k | c )] from Eq. (25) into Eq. (24), we obtain S [ P ( k ) || π ( k | c )] = ∞ X k =0 P ( k ) ln[ P ( k )] + c − h K i ln( c ) + ∞ X k =0 ln( k !) P ( k ) . (26)Eq. (26) provides the relative entropy of any degree distribution P ( k ) whose mean degreeis h K i , with respect to a Poisson distribution with mean degree c . In order to find the valueof c for which S [ P ( k ) || π ( k | c )] is minimal we differentiate S [ P ( k ) || π ( k | c )] with respect to c and solve the equation ddc S [ P ( k ) || π ( k | c )] = 1 − h K i c = 0 . (27)We find that S [ P ( k ) || π ( k | c )] is minimized when the condition c = h K i is satisfied. Thisimplies that for any degree distribution P ( k ) with mean degree h K i , the closest Poissondistribution π ( k | c ), in terms of the relative entropy, is the Poisson distribution with meandegree c = h K i .Using the result discussed above, one can express the relative entropy S [ P ( k ) || π ( k | c )] inthe form 12 [ P ( k ) || π ( k | c )] = S [ P ( k ) || π ( k |h K i )] + δS ( c, h K i ) (28)where S [ P ( k ) || π ( k |h K i )] is the relative entropy of P ( k ) with respect to a Poisson distributionwhose mean is h K i , and δS ( c, h K i ) = h K i (cid:20)(cid:18) c h K i − (cid:19) − ln (cid:18) c h K i (cid:19)(cid:21) (29)is the added entropy due to the difference between c and h K i . Note that δS ( c, h K i ) ≥ h K i > c >
0, while δS ( c, h K i ) = 0 only in the case that c = h K i .Going back to Eq. (22), the relative entropy S [ P ( k ) || π ( k |h K i )] can be expressed in theform S [ P ( k ) || π ( k |h K i )] = − S [ P ( k )] + C [ P ( k ) || π ( k |h K i )] , (30)where S [ P ( k )] is given by Eq. (23) and C [ P ( k ) || π ( k |h K i )] = h K i − h K i ln( h K i ) + ∞ X k =0 ln( k !) P ( k ) . (31)To evaluate the last term in Eq. (31) we recall that ln(0!) = ln(1!) = 0, while the k = 2term is ln(2) P (2). For k ≥ k !) = (cid:18) k + 12 (cid:19) ln( k ) − k + 12 ln(2 π ) . (32)Inserting ln( k !) for k ≥ C [ P ( k ) || π ( k | c )] = −h K i ln( h K i ) + ∞ X k =2 (cid:18) k + 12 (cid:19) ln( k ) P ( k )+ 12 ln(2 π ) −
12 ln(2 π ) P (0) + (cid:20) −
12 ln(2 π ) (cid:21) P (1)+ (cid:20) −
32 ln(2) −
12 ln(2 π ) (cid:21) P (2) , (33)where the terms involving P (0), P (1) and P (2) result from the adjustment of the summationdue to the fact that Eq. (32) is used only for k ≥
3. Note that in the case of distributionsin which k min ≥
1, one assigns P ( k ) = 0 for 0 ≤ k ≤ k min −
1. Using Eq. (33), the13elative entropy of the degree distribution P t ( k ) of a contracting network with respect to thecorresponding Poisson distribution π t ( k |h K i t ) with the same mean degree h K i t , is given by S t = ∞ X k =0 P t ( k ) ln[ P t ( k )] − h K i t ln( h K i t ) + ∞ X k =2 (cid:18) k + 12 (cid:19) ln( k ) P t ( k )+ 12 ln(2 π ) −
12 ln(2 π ) P t (0) + (cid:20) −
12 ln(2 π ) (cid:21) P t (1)+ (cid:20) −
32 ln(2) −
12 ln(2 π ) (cid:21) P t (2) . (34)Eq. (34) is used in order to evaluate the relative entropy during the contraction process,where P t ( k ) is obtained either from numerical integration of the master equation or fromcomputer simulations. V. CONVERGENCE OF THE RELATIVE ENTROPY
In each of the network contraction scenarios, the degree distribution P t ( k ) evolves intime according to the master equation [Eq. (10)]. As a result, the relative entropy S t of thenetwork also evolves as the network contracts. The time derivative of S t is given by ddt S t = ∞ X k =0 ln (cid:20) P t ( k ) π ( k |h K i t ) (cid:21) ddt P t ( k ) + ∞ X k =0 ddt P t ( k ) − ∞ X k =0 P t ( k ) π ( k |h K i t ) ddt π ( k |h K i t ) . (35)Replacing the order of the summation and the derivative in the second term on the righthand side of Eq. (35), we obtain ∞ X k =0 ddt P t ( k ) = ddt " ∞ X k =0 P t ( k ) = 0 . (36)Inserting the derivative dπ ( k |h K i t ) /dt from Eq. (16) into the third term on the right handside of Eq. (35), we obtain ∞ X k =0 P t ( k ) π ( k |h K i t ) ddt π ( k |h K i t ) = − d h K i t dt ∞ X k =0 (cid:18) − k h K i t (cid:19) P t ( k ) = 0 . (37)Since the second and third terms in Eq. (35) vanish, the time derivative of the relativeentropy is simply given by 14 dt S t = ∞ X k =0 ln (cid:20) P t ( k ) π ( k |h K i t ) (cid:21) ddt P t ( k ) . (38)This is a general equation that applies to any network contraction scenario in which thePoisson distribution π ( k |h K i t ) is a solution. The relative entropy satisfies S t ≥ P t ( k ). It vanishes if and only if P t ( k ) = π ( k |h K i t ). Therefore, in orderto prove the convergence of the degree distribution P t ( k ) towards a Poisson distribution ina given network contraction scenario, one needs to show that for this scenario dS t /dt < dP t /dt by the right hand sideof the master equation, Eq. (10).For the analysis below it is convenient to express the time evolution of the relative entropy,given by Eq. (38), in the form ddt S t = ∆ A ( t ) + ∆ B ( t ) , (39)where ∆ A ( t ) emanates from the F A ( t ) term (trickle-down term) of the master equationand ∆ B ( t ) emanates from the F B ( t ) term (redistribution term). The contribution of thetrickle-down term to dS t /dt is given by∆ A ( t ) = A t N t ∞ X k =0 ln (cid:20) P t ( k ) π ( k |h K i t ) (cid:21) [( k + 1) P t ( k + 1) − kP t ( k )] , (40)where A t is given by Eq. (13), and the contribution of the redistribution term is given by∆ B ( t ) = − BN t ∞ X k =0 ln (cid:20) P t ( k ) π t ( k |h K i t ) (cid:21) (cid:18) k h K i t − (cid:19) P t ( k ) , (41)where B = . (42)In order to show that the degree distribution of the contracting network converges towardsa Poisson distribution, one needs to show that during the contraction process ∆ A ( t ) +∆ B ( t ) <
0. Below we consider each one of these terms separately. We show that in allthe three network contraction scenarios and for any initial degree distribution P ( k ), the15rickle-down term satisfies ∆ A ( t ) < B ( t ) we obtain a necessary and sufficient condition on the instantaneousdegree distribution P t ( k ) under which ∆ B ( t ) <
0. The condition essentially states that∆ B ( t ) < A. Convergence due to the trickle-down term
To gain more insight on the structure of the ∆ A ( t ) term, given by Eq. (40), it is usefulto express it in the form∆ A ( t ) = A t N t ( ∞ X k =0 ln (cid:20) P t ( k ) π ( k |h K i t ) (cid:21) ( k + 1) P t ( k + 1) − ∞ X k =1 ln (cid:20) P t ( k ) π ( k |h K i t ) (cid:21) kP t ( k ) ) . (43)Taking a factor of h K i t out of the curly parentheses and multiplying the numerators anddenominators in the arguments of the logarithmic functions by k/ h K i t (for k ≥ A ( t ) = A t N t {h K i t + ln[ P t (0)] } P t (1) (44)+ A t h K i t N t ( ∞ X k =1 ln " e P t ( k ) π ( k − |h K i t ) P t ( k + 1) − ∞ X k =1 ln " e P t ( k ) π ( k − |h K i t ) P t ( k ) ) , where e P t ( k ) = k h K i t P t ( k ) , (45)is the degree distribution of nodes selected via a random edge in a random network withdegree distribution P t ( k ). Similarly, the distribution π ( k − |h K i t ) = k h K i t π ( k |h K i t ) (46)can be interpreted as the degree distribution of nodes selected via a random edge in an ERnetwork with a Poisson degree distribution of the form π ( k |h K i t ).16ewriting e P t ( k + 1) in the form [ e P t ( k + 1) / e P t ( k )] e P t ( k ), one can express the ∆ A ( t ) termas a covariance of the form∆ A ( t ) = A t N t (cid:26) h K i t P t (1) + ln[ P t (0)] P t (1) − P t (1) h K i t S [ e P t ( k ) || π ( k − |h K i t )] (47)+ e E t " e P t ( k + 1) e P t ( k ) ln e P t ( k ) π ( k − |h K i t ) ! − e E t " e P t ( k + 1) e P t ( k ) E t " ln e P t ( k ) π ( k − |h K i t ) ! , where e E t [ f ( k )] = P k f ( k ) e P t ( k ). In particular, e E t " e P t ( k + 1) e P t ( k ) = ∞ X k =1 e P t ( k + 1) e P t ( k ) ! e P t ( k ) = 1 − P t (1) h K i t . (48)In order that the covariance will be negative, in domains in which e P t ( k ) is an increasingfunction [namely, e P t ( k + 1) > e P t ( k )], it should be lower than the corresponding Poissondistribution [namely, e P t ( k ) < π ( k − |h K i t )], while in domains in which e P t ( k ) is a decreasingfunction it should be higher than the corresponding Poisson distribution.In order to prove that ∆ A ( t ) < P t ( k ) at all stages of thecontraction process we rewrite Eq. (40) in the form∆ A ( t ) = ∆ PA ( t ) − ∆ π A ( t ) , (49)where ∆ PA ( t ) = A t N t ∞ X k =0 ln [ P t ( k )] [( k + 1) P t ( k + 1) − kP t ( k )] , (50)and ∆ π A ( t ) = A t N t ∞ X k =0 ln [ π ( k |h K i t )] [( k + 1) P t ( k + 1) − kP t ( k )] . (51)Separating the sum in Eq. (50) into two sums and replacing k + 1 by k in the first sum, weobtain ∆ PA ( t ) = A t N t ( ∞ X k =1 ln [ P t ( k − kP t ( k ) − ∞ X k =1 ln [ P t ( k )] kP t ( k ) ) . (52)Expressing the degree distribution P t ( k ) in terms of e P t ( k ),17 PA ( t ) = A t h K i t N t ( ∞ X k =1 ln [ P t ( k − e P t ( k ) − ∞ X k =1 ln h e P t ( k ) i e P t ( k ) ) + A t N t ∞ X k =1 ln (cid:18) k h K i t (cid:19) kP t ( k ) . (53)Combining the first two terms in Eq. (53) and splitting the last term, we obtain∆ PA ( t ) = − A t h K i t N t ∞ X k =1 e P t ( k ) ln " e P t ( k ) P t ( k − + A t N t h K ln( K ) i t − A t N t h K i t ln( h K i t ) . (54)In order to evaluate ∆ π A we insertln[ π ( k |h K i t )] = −h K i t + k ln( h K i t ) − ln( k !) (55)into Eq. (51) and obtain∆ π A ( t ) = A t N t ∞ X k =0 [ −h K i t + k ln( h K i t ) − ln( k !)] [( k + 1) P t ( k + 1) − kP t ( k )] . (56)Carrying out the summation and using the identityln( k !) = ln[( k + 1)!] − ln( k + 1) , (57)we obtain ∆ π A ( t ) = A t N t h K ln( K ) i t − A t N t h K i t ln( h K i t ) . (58)Inserting the results for ∆ PA and ∆ π A , from Eqs. (54) and (58), respectively, into Eq. (49),we obtain ∆ A ( t ) = − A t h K i t N t S [ e P t ( k ) || P t ( k − S [ e P t ( k ) || P t ( k − ∞ X k =1 e P t ( k ) ln " e P t ( k ) P t ( k − (60)is the relative entropy of e P t ( k ) with respect to P t ( k − P t ( k − > k for which e P t ( k ) >
0. This means that the degree distribution18hould not have any gaps, namely, values of k ′ for which P t ( k ′ ) = 0 while P t ( k ) > k > k ′ . In practice, even if there are such gaps in the initial degree distribution P ( k ), theyare quickly filled up due to the trickle-down term F A ( t ) of the master equation, given byEq. (11).Since the relative entropy must be positive, we find that ∆ A ( t ) < P t ( k ) that differs from π ( k |h K i t ). Actually, since the only distribution forwhich S [ e P t ( k ) || P t ( k − A ( t ) term contributesto the time evolution of S t , while the ∆ B ( t ) term vanishes. This means that in the randomdeletion scenario the distance between P t ( k ) and the corresponding Poisson distribution π ( k |h K i t ) with the same mean degree h K i t decreases monotonically at any stage during thecontraction process. In the preferential deletion and the propagating deletion scenarios theconvergence also depends on the ∆ B ( t ) term, which is considered below. B. Convergence due to the redistribution term
In order to gain insight on the ∆ B ( t ) term, we rewrite Eq. (41) in the form∆ B ( t ) = − BN t ( ∞ X k =1 ln (cid:20) P t ( k ) π ( k |h K i t ) (cid:21) k h K i t P t ( k ) − ∞ X k =0 ln (cid:20) P t ( k ) π ( k |h K i t ) (cid:21) P t ( k ) ) . (61)Taking the factor of 1 / h K i t out of the curly brackets, we obtain∆ B ( t ) = − B h K i t N t ( ∞ X k =1 k ln (cid:20) P t ( k ) π ( k |h K i t ) (cid:21) P t ( k ) − h K i t ∞ X k =0 ln (cid:20) P t ( k ) π ( k |h K i t ) (cid:21) P t ( k ) ) . (62)The expression in the curly brackets is, in fact, equal to the covariance between k andln[ P t ( k ) /π ( k |h K i t )] under the distribution P t ( k ), namely∆ B ( t ) = − B h K i t N t (cid:26)(cid:28) k ln (cid:20) P t ( k ) π ( k |h K i t ) (cid:21)(cid:29) − h K i t (cid:28) ln (cid:20) P t ( k ) π ( k |h K i t ) (cid:21)(cid:29)(cid:27) . (63)Therefore, in the case of distributions for which the correlation between k andln[ P t ( k ) /π ( k |h K i t )] is positive, the term in the curly brackets is positive and ∆ B ( t ) < B ( t ) term contributes to the convergence of P t ( k ) towards a Poisson distri-bution. Such positive correlation essentially implies that for large values of k , P t ( k ) tends to19e larger than π ( k |h K i t ), namely, it has a heavier tail than the Poisson distribution with thesame mean value. Since network growth processes generically lead to fat tail distributionssuch as the power-law distributions of scale-free networks, it is expected that most empiricalnetworks will exhibit a positive correlation between k and ln[ P t ( k ) /π ( k |h K i t )].In those cases in which the correlation between k and ln[ P t ( k ) /π ( k |h K i t )] is negative, theterm in the curly brackets is negative and ∆ B ( t ) >
0. In this case the ∆ B ( t ) term worksagainst the convergence of P t ( k ) towards a Poisson distribution. However, comparing thecoefficients of ∆ A ( t ) and ∆ B ( t ) one finds that the coefficient of ∆ A ( t ) is effectively larger bya factor of h K i / h K i than the coefficient of ∆ B ( t ). Therefore, it is expected that the ∆ A ( t )term will be dominant and induce the convergence of P t ( k ) towards Poisson even in thosecases in which ∆ B ( t ) > B ( t ) from a different perspective, we use Eqs. (45)and (46) to express ∆ B ( t ) of Eq. (61) in the form∆ B ( t ) = − BN t ∞ X k =1 ln " e P t ( k ) π ( k − |h K i t ) P t ( k ) + BN t ∞ X k =0 ln (cid:20) P t ( k ) π ( k |h K i t ) (cid:21) P t ( k ) . (64)The first sum in Eq. (64) is the relative entropy of the degree distribution e P t ( k ) withrespect to the shifted Poisson distribution π ( k − |h K i t ). This is essentially a distancemeasure between the degree distribution of nodes selected preferentially in a network whosedegree distribution is P t ( k ) and the degree distribution of nodes selected preferentially fromthe corresponding Poisson distribution with the same mean degree. The second term inEq. (64) is the relative entropy of the degree distribution P t ( k ) with respect to the Poissondistribution π ( k |h K i t ), which is essentially a distance measure between P t ( k ) and π ( k |h K i t ).Thus, Eq. (64) can be written in the form∆ B ( t ) = − BN t n S [ e P t ( k ) || π ( k − |h K i t )] − S [ P t ( k ) || π ( k |h K i t )] o . (65)In the case that the degree distributions obtained for the preferential selection are fartherapart than the degree distributions obtained for random selection, ∆ B ( t ) <
0, while in theopposite case ∆ B ( t ) > P t ( k ) with respect to the Poisson distribution π ( k |h K i t ) with thesame mean degree h K i t . In contrast, the first term is the relative entropy of e P t ( k ) with20espect to the Poisson distribution π ( k − |h K i t ). The mean degree of e P t ( k ) is h e K i t = h K i t h K i t , (66)while the mean degree of π ( k − |h K i t ) is h K i t + 1. Therefore, Eq. (65) can be written inthe form ∆ B ( t ) = − BN t n S [ e P t ( k ) || π ( k − |h e K i t − − S [ P t ( k ) || π ( k |h K i t )] o − BN t δS ( h e K i t , h K i t + 1) , (67)where δS ( h e K i t , h K i t + 1) is given by Eq. (29). This implies that ∆ B ( t ) < S [ e P t ( k ) || π ( k − |h e K i t − > S [ P t ( k ) || π ( k |h K i t )] − δS ( h e K i t , h K i t + 1) . (68)Since δS ( h e K i t , h K i t + 1) is always positive and its value increases as P ( k ) becomes broader,this condition is expected to be satisfied for any degree distribution that exhibits a heavytail. From our experience, degree distributions for which ∆ B > A , which is always negative, as proven above,is much larger in absolute value than ∆ B . VI. CONTRACTION OF NETWORKS WITH GIVEN INITIAL DEGREE DIS-TRIBUTIONS
Here we apply the framework presented above to three examples of configuration modelnetworks, with a degenerate degree distribution (also known as random regular graphs), anexponential degree distribution and a power-law degree distribution (scale-free networks).
A. Random regular graphs
A random regular graph (RRG) is a configuration model network in which all the nodesare of the same degree, k = c , namely P ( k ) = δ k,c , (69)21here c is an integer. Here we consider the case of c ≥
3, in which the giant componentencompasses the whole network. In order to leave room for contraction into a non-trivialdegree distribution, we choose RRGs with c ≫
1. Since in node deletion processes thedegrees of nodes in the network are only reduced and never increase it is clear that therange of degrees of the contracted network will be limited to 0 ≤ k ≤ c . This means thatin the case that the initial network is an RRG the tail of the degree distribution of thecontracted network will be truncated above k = c . Thus, the convergence towards Poissonis expected to be relatively slow.To evaluate the relative entropy of the initial RRG network with respect to the corre-sponding Poisson distribution we insert the degenerate distribution of Eq. (69) into Eq.(21). We obtain the initial relative entropy S = ln (cid:20) π ( c | c ) (cid:21) . (70)Inserting the Poisson degree distribution into Eq. (70) we obtain S = c − c ln( c ) + ln( c !) . (71)Using the Stirling approximation to evaluate ln( c !), we obtain S = 12 ln( c ) + 12 ln(2 π ) . (72)Below we analyze the convergence of a configuration model network with a degeneratedegree distribution towards an ER graph structure upon contraction. In particular, wecalculate the time-dependent degree distribution P t ( k ) during contraction and examine itsconvergence towards π ( k |h K i t ). To this end we perform direct numerical integration ofthe master equation (10) and computer simulations, starting from a configuration modelnetwork with a degree distribution given by Eq. (69) and evaluate the time-dependentrelative entropy S t .In Fig. 2 we present the relative entropy S t as a function of time (represented by N t /N =1 − t/N ) for a random regular graph of size N = 10 with a degenerate degree distributionin which all the nodes are of degree c = 10, that contracts via: (a) random node deletion;(b) preferential node deletion; and (c) propagating node deletion. The results obtained fromnumerical integration of the master equation (solid lines) are in excellent agreement with22he results obtained from computer simulations, namely, direct simulations of contractingnetworks (circles). In all three cases the relative entropy quickly decays, which impliesthat the degree distribution P t ( k ) of the contracting network converges towards a Poissondistribution. The decay rate of S t is comparable in all the three scenarios. This implies thatfor extremely narrow degree distributions such as the degenerate distribution the preferentialand the propagating deletion scenarios do not exhibit faster convergence than the randomdeletion scenario.In Fig. 3(a) we present the degree distribution P ( k ) of a random regular graph (solid line)of size N = 10 with a degenerate degree distribution in which all the nodes are of degree c = 10. The corresponding Poisson distribution with the same mean degree h K i = c is also shown (dashed line). Clearly, it is highly dissimilar to the degenerate distribution.The random regular graph undergoes a network contraction process via the random nodedeletion scenario. In Fig. 3(b) we present the degree distribution P t ( k ) of the contractednetwork at time t = 8000, where the contracted network size is N t = 2000. The resultsobtained from the numerical integration of the master equation (solid line) are in excellentagreement with the results of computer simulations (circles). They are very well convergedtowards the corresponding Poisson distribution π ( k |h K i t ) with the same mean degree h K i t (dashed line). B. Configuration model networks with exponential degree distributions
Consider a configuration model network with an exponential degree distribution of theform P ( k ) ∼ e − αk , where k ≥ k min and k min is the lower cutoff of the initial degree distri-bution. It is convenient to parametrize the degree distribution using the mean degree h K i ,in the form P ( k ) = k < k min D (cid:16) h K i − k min h K i − k min +1 (cid:17) k k ≥ k min , (73)where D is the normalization constant, given by D = 1( h K i − k min ) + 1 (cid:18) h K i − k min h K i − k min + 1 (cid:19) − k min . (74)23 FIG. 2: (Color online) The relative entropy S t as a function of time for a random regular graphof initial size N = 10 and initial degree c = 10 that contracts via random deletion (a), preferen-tial deletion (b) and propagating deletion (c), obtained from numerical integration of the masterequation (solid lines). In all three cases the relative entropy quickly decays, which implies thatthe degree distribution of the contracting network converges towards a Poisson distribution. Themaster equation results are in excellent agreement with the results obtained from computer simu-lations (circles). Also, the initial value S ≃ .
08 is in perfect agreement with the result obtainedfrom Eq. (72). FIG. 3: (Color online) (a) The degree distribution P ( k ) of a random regular graph (solid line)in which all the nodes are of degree c = 10. The circles represent the degree sequence of asingle network instance of N = 10 nodes, which was used in the computer simulations. Thecorresponding Poisson distribution with the same mean degree is also shown (dashed line). Thenetwork contracts via random node deletion. (b) The degree distribution P t ( k ) of the contractednetwork at time t = 8000, when the network size is reduced to N t = 2000. The results obtainedfrom numerical integration of the master equation (solid line) are in excellent agreement with theresults obtained from computer simulations (circles). They are both very well converged towardsthe corresponding Poisson distribution π ( k |h K i t ) with the same mean degree h K i t (dashed line). Below we evaluate the relative entropy of an initial network with an exponential degree dis-tribution with respect to the corresponding Poisson distribution. Inserting the exponentialdegree distribution of Eq. (73) into Eq. (23) and carrying out the summation, we obtainthe Shannon entropy S [ P ( k )] = − ∞ X k = k min P ( k ) ln[ P ( k )]= − ( h K i − k min ) ln( h K i − k min )+ ( h K i − k min + 1) ln( h K i − k min + 1) . (75)In order to calculate the cross-entropy C [ P ( k ) || π ( k |h K i )], we insert the exponential dis-tribution P ( k ) of Eq. (73) into Eq. (33). We obtain25 [ P ( k ) || π ( k |h K i )] = −h K i ln( h K i ) + ∞ X k = k min (cid:18) k + 12 (cid:19) ln( k ) " D (cid:18) h K i − k min h K i − k min + 1 (cid:19) k + 12 ln(2 π ) −
12 ln(2 π ) P (0) + (cid:20) −
12 ln(2 π ) (cid:21) P (1)+ (cid:20) −
32 ln(2) −
12 ln(2 π ) (cid:21) P (2) . (76)Carrying out the summation, we obtain C [ P ( k ) || π ( k |h K i )] = −h K i ln( h K i ) − h K i − k min + 1) " ∂∂γ Φ (cid:18) h K i − k min h K i − k min + 1 , γ, k min (cid:19)(cid:12)(cid:12)(cid:12)(cid:12) γ = − + ∂∂γ Φ (cid:18) h K i − k min h K i − k min + 1 , γ, k min (cid:19)(cid:12)(cid:12)(cid:12)(cid:12) γ =0 + 12 ln(2 π ) −
12 ln(2 π ) P (0) + (cid:20) −
12 ln(2 π ) (cid:21) P (1)+ (cid:20) −
32 ln(2) −
12 ln(2 π ) (cid:21) P (2) , (77)where Φ( x, γ, k ) is the Lerch transcendent [48]. The relative entropy takes the form S = − S [ P ( k )] + C [ P ( k ) || π ( k |h K i )], where S [ P ( k )] is given by Eq. (75) and C [ P ( k ) || π ( k |h K i )] is given by Eq. (77).Below we analyze the convergence of a configuration model network with an exponentialdegree distribution towards an ER graph structure upon contraction. In particular, wecalculate the time dependent degree distribution P t ( k ) during contraction and examine itsconvergence towards π ( k |h K i t ). To this end we perform direct numerical integration ofthe master equation (10) and computer simulations, starting from a configuration modelnetwork with a degree distribution given by Eq. (73) and evaluate the time-dependentrelative entropy S t .In Fig. 4 we present the relative entropy S t as a function of time for a configuration modelnetwork of initial size N = 10 and initial mean degree h K i = 20 with an exponential degreedistribution that contracts via random deletion (a), preferential deletion (b) and propagatingdeletion (c), obtained from numerical integration of the master equation (solid lines). In allthree cases the relative entropy quickly decays, which implies that the degree distribution26 FIG. 4: (Color online) The relative entropy S t as a function of time for a configuration modelnetwork of initial size N = 10 and mean degree h K i = 20 with an exponential degree distri-bution in which k min = 10, that contracts via random deletion (a), preferential deletion (b) andpropagating deletion (c), obtained from numerical integration of the master equation (solid lines).In all three cases the relative entropy quickly decays, which implies that the degree distributionof the contracting network converges towards a Poisson distribution. The convergence is dramati-cally faster in the preferential and the propagating deletion scenarios compared to random deletionscenario. The master equation results are in very good agreement with the results obtained fromcomputer simulations (circles). Also, the initial value S ≃ .
32 is in perfect agreement with theresult obtained from Eqs. (75) and (77). FIG. 5: (Color online) (a) The degree distribution P ( k ) of a configuration model network withmean degree h K i = 20 and an exponential degree distribution, given by Eq. (73) with k min = 10(solid line). The circles represent the degree sequence of the N = 10 nodes in a single realizationof the initial network, which was used in the computer simulation. The corresponding Poissondistribution with the same mean degree is also shown (dashed line). The network contracts via thepreferential node deletion scenario. (b) The degree distribution P t ( k ) of the contracted network attime t = 7000, when the network size is reduced to N t = 3000, obtained from numerical integrationof the master equation (solid line). The master equation results are in excellent agreement withthe results obtained from computer simulations (circles). The corresponding Poisson distribution π ( k |h K i t ) with the same mean degree is also shown (dashed line). The master equation resultsand the computer simulation results are in very good agreement with the corresponding Poissondistribution with the same mean degree. of the contracting network converges towards a Poisson distribution. The convergence isdramatically faster in the preferential and the propagating deletion scenarios compared torandom deletion scenario. The master equation results are in very good agreement with theresults obtained from computer simulations (circles).In Fig. 5(a) we present the degree distribution P ( k ) of a configuration model networkof size N = 10 and an exponential degree distribution with mean degree h K i = 20(solid line). The corresponding Poisson distribution with the same mean degree is alsoshown (dashed line). The network contracts via preferential node deletion. In Fig. 5(b) wepresent the degree distribution P t ( k ) of the contracted network at time t = 7000, when the28 -3 FIG. 6: (Color online) The time derivative of the relative entropy, dS t /dt = ∆ A ( t ) + ∆ B ( t ), asa function of time, for a configuration model network of initial size N = 10 and exponentialdegree distribution with mean degree h K i = 20 and k min = 10, that contracts via preferentialnode deletion, obtained from numerical integration of the master equation (solid lines). The terms∆ A ( t ) (dashed line) and ∆ B ( t ) (dotted line), which sum up to the derivative dS t /dt are also shown.Note that both ∆ A ( t ) and ∆ B ( t ) are negative at all times during the contraction process. network size is reduced to N t = 3000, obtained from numerical integration of the masterequation (solid line) and from computer simulations (circles). The corresponding Poissondistribution π ( k |h K i t ) with the same mean degree is also shown (dashed line). The masterequation results, the computer simulation results and the corresponding Poisson distributionare found to be in very good agreement with each other.In Fig. 6 we present the time derivative of the relative entropy, dS t /dt = ∆ A ( t )+∆ B ( t ), asa function of time, for a configuration model network of initial size N = 10 and exponentialdegree distribution with mean degree h K i = 20 that contracts via preferential node deletion,obtained from numerical integration of the master equation (solid lines). The terms ∆ A ( t )(dashed line) and ∆ B ( t ) (dotted line), which sum up to the derivative dS t /dt are also shown.As expected, both ∆ A ( t ) and ∆ B ( t ) are negative at all times during the contraction process.29 . Configuration model networks with power-law degree distributions Consider a configuration model network with a power-law degree distribution of the form P ( k ) ∼ k − γ , where 1 ≤ k min ≤ k ≤ k max . Here we focus on the case of γ >
2, in which themean degree, h K i , is bounded even for k max → ∞ . Power-law distributions do not exhibita typical scale, and are therefore referred to as scale-free networks. The normalized degreedistribution is given by P ( k ) = k < k min D k − γ k min ≤ k ≤ k max k > k max , (78)where D is the normalization constant, given by D = D ( γ, k min , k max ) = 1 ζ ( γ, k min ) − ζ ( γ, k max + 1) , (79)and ζ ( γ, k ) is the Hurwitz zeta function [48]. For 2 < γ ≤ h K i , diverges in the limit of k max → ∞ . For γ > h K i = ζ ( γ − , k min ) − ζ ( γ − , k max + 1) ζ ( γ, k min ) − ζ ( γ, k max + 1) . (80)The second moment of the degree distribution, when finite, is h K i = ζ ( γ − , k min ) − ζ ( γ − , k max + 1) ζ ( γ, k min ) − ζ ( γ, k max + 1) . (81)Below we evaluate the relative entropy of an initial network with a power law degreedistribution with respect to the corresponding Poisson distribution. In order to calculatethe Shannon entropy S [ P ( k )] we insert the power-law distribution of Eq. (78) into Eq.(23). We obtain S [ P ( k )] = − ∞ X k = k min P ( k ) ln[ P ( k )] = − ln( D ) + γ ∞ X k = k min Dk − γ ln( k ) . (82)Since ln(1) = 0 the summation in Eq. (82) actually starts from the larger value between k = 2 and k min , denoted by k min = max { , k min } . We thus obtain30 [ P ( k )] = − ln( D ) + γ ∞ X k = k min Dk − γ ln( k ) . (83)Carrying out the summation, we obtain S [ P ( k )] = − ln( D ) + γD (cid:2) ζ ′ ( γ, k max + 1) − ζ ′ ( γ, k min ) (cid:3) , (84)where ζ ′ ( γ, k ) = ∂ζ ( γ, k ) /∂γ .In order to calculate the cross-entropy C [ P ( k ) || π ( k |h K i )], we insert the power-law dis-tribution P ( k ) into Eq. (33). We obtain C [ P ( k ) || π ( k |h K i )] = −h K i ln( h K i ) + ∞ X k = k min (cid:18) k + 12 (cid:19) ln( k ) Dk − γ + 12 ln(2 π ) + (cid:20) −
12 ln(2 π ) (cid:21) P (1)+ (cid:20) −
32 ln(2) −
12 ln(2 π ) (cid:21) P (2) . (85)Carrying out the summation, we obtain C [ P ( k ) || π ( k |h K i )] = −h K i ln( h K i ) + D (cid:2) ζ ′ ( γ − , k min ) − ζ ′ ( γ − , k max + 1) (cid:3) + D ζ ′ ( γ, k min ) − ζ ′ ( γ, k max + 1)]+ 12 ln(2 π ) + (cid:20) −
12 ln(2 π ) (cid:21) P (1)+ (cid:20) −
32 ln(2) −
12 ln(2 π ) (cid:21) P (2) . (86)The relative entropy of the initial network with a power-law degree distribution given byEq. (78) takes the form S = − S [ P ( k )] + C [ P ( k ) || π ( k |h K i )], where S [ P ( k )] is given byEq. (84) and C [ P ( k ) || π ( k |h K i )] is given by Eq. (86).Below we analyze the convergence of a configuration model network with a power-lawdegree distribution towards an ER graph structure upon contraction. In particular, wecalculate the time dependent degree distribution P t ( k ) during contraction and examine itsconvergence towards π ( k |h K i t ). To this end we perform direct numerical integration ofthe master equation (10) and computer simulations, starting from a configuration model31etwork with a degree distribution given by Eq. (78) and evaluate the time-dependentrelative entropy S t .In Fig. 7 we present the relative entropy S t as a function of time for a configurationmodel network with a power-law degree distribution, of initial size N = 10 and initialmean degree h K i = 20, where k min = 10, k max = 100 and γ = 2 .
65, that contracts viarandom deletion (a), preferential deletion (b) and propagating deletion (c), obtained fromnumerical integration of the master equation (solid lines). In all three cases the relativeentropy quickly decays, which implies that the degree distribution of the contracting networkconverges towards a Poisson distribution. The convergence is dramatically faster in thepreferential and the propagating deletion scenarios compared to random deletion scenario.The master equation results are in very good agreement with the results obtained fromcomputer simulations (circles).In Fig. 8(a) we present the degree distribution P ( k ) of a configuration model network ofsize N = 10 and a power-law degree distribution with mean degree h K i = 20 (solid line).The corresponding Poisson distribution with the same mean degree is also shown (dashedline). The network contracts via propagating node deletion. In Fig. 8(b) we present thedegree distribution P t ( k ) of the contracted network at t = 7000, when the network size isreduced to N t = 3000, obtained from numerical integration of the master equation (solid line)and from computer simulations (circles). The corresponding Poisson distribution π ( k |h K i t )with the same mean degree is also shown (dashed line). The master equation results, thecomputer simulation results and the corresponding Poisson distribution are found to be invery good agreement with each other.In Fig. 9 we present the time derivative of the relative entropy, dS t /dt as a functionof time, for a configuration model network of initial size N = 10 and a power-law degreedistribution with mean degree h K i = 20 that contracts via propagating node deletion,obtained from numerical integration of the master equation (solid lines). The terms ∆ A ( t )(dashed line) and ∆ B ( t ) (dotted line), which sum up to the derivative dS t /dt are also shown.As expected, both ∆ A ( t ) and ∆ B ( t ) are negative at all times during the contraction process.32 FIG. 7: (Color online) The relative entropy S t as a function of time for a configuration modelnetwork with a power-law degree distribution of initial size N = 10 and mean degree h K i = 20,where k min = 10, k max = 100 and γ = 2 .
65, that contracts via random deletion (a), preferentialdeletion (b) and propagating deletion (c), obtained from numerical integration of the master equa-tion (solid lines). In all three cases the relative entropy quickly decays, which implies that thedegree distribution of the contracting network converges towards a Poisson distribution. The con-vergence is dramatically faster in the preferential and the propagating deletion scenarios comparedto random deletion scenario. The master equation results are in very good agreement with theresults obtained from computer simulations (circles). Also, the initial value S ≃ .
59 is in perfectagreement with the result obtained from Eqs. (84) and (86).
10 20 35 50 70 1000.00050.00150.0050.0150.050.15 0 3 6 9 1200.050.10.150.20.250.3
FIG. 8: (Color online) (a) The degree distribution P ( k ) of a configuration model network witha power-law degree distribution, given by Eq. (78), and mean degree h K i = 20 (solid line),where k min = 10, k max = 100 and γ = 2 .
65, is shown on a log-log scale. The circles representthe degree sequence of the N = 10 nodes in a single realization of the initial network, whichwas used in the computer simulation. The corresponding Poisson distribution with the same meandegree is also shown (dashed line). The network contracts via the propagating node deletionscenario. (b) The degree distribution P t ( k ) of the contracted network at time t = 7000, when thenetwork size is reduced to N t = 3000, obtained from numerical integration of the master equationis shown on a linear scale (solid line). The master equation results are in excellent agreement withthe results obtained from computer simulations (circles). The corresponding Poisson distribution π ( k |h K i t ) with the same mean degree is also shown (dashed line). The master equation resultsand the computer simulation results are in very good agreement with the corresponding Poissondistribution with the same mean degree. VII. DISCUSSION
In Ref. [31] we used direct numerical integration of the master equation and computersimulations to show that the degree distributions of contracting networks converge towardsthe Poisson distribution. To this end, we used the relative entropy as a distance measurebetween the degree distribution P t ( k ) of the contracing network and the correspondingPoisson distribution π ( k |h K i t ), and showed that this distance decreases as the networkcontracts.A computer simulation of network contraction provides results for a single instance of the34 -3 FIG. 9: (Color online) The time derivative of the relative entropy dS t /dt as a function of time, fora configuration model network of initial size N = 10 and a power-law degree distribution withmean degree h K i = 20, where k min = 10, k max = 100 and γ = 2 .
65, that contracts via propagatingnode deletion, obtained from numerical integration of the master equation (solid lines). The terms∆ A ( t ) (dashed line) and ∆ B ( t ) (dotted line), which sum up to the derivative dS t /dt are also shown.Note that both ∆ A ( t ) and ∆ B ( t ) are negative at all times during the contraction process. initial network and a single stochastic path of the contraction process. In order to obtainstatistically significant results for a given ensemble of initial networks and given networkcontraction scenario one needs to combine the results of a large number of independentruns. The direct numerical integration of the master equation is advantageous in the sensethat a single run of the numerical integration process provides results for a whole ensembleof initial networks. However, a given network ensemble represents a single point in the highdimensional parameter space of possible network ensembles. Therefore, in order to explorethe general properties of network contraction processes one needs to repeatedly apply thedirect integration of the master equation to a large sample of distinct network ensembles.Our aim in this paper was to obtain rigorous analytical results for the convergence ofcontracting networks towards the ER network ensemble. To this end we devised a rigorousargument, which is based on the master equation that describes the temporal evolution ofthe degree distribution P t ( k ) and the relative entropy S t . Such an argument is advantageousover the direct numerical integration of the master equation or computer simulations in thesense that it is universally applicable to all possible degree distributions.35he relative entropy S [ P ( k ) || Q ( k )] of a distribution P ( k ) with respect to a distribution Q ( k ) is a special case of the R´enyi divergence S α [ P ( k ) || Q ( k )], with α = 1 [49]. The choice of α = 1 is advantageous in the sense that it has a natural information theoretic interpretation[35, 36]. The relative entropy is an asymmetric distance measure, or quasi-distance [50].Interestingly, the relative entropy is related to other distance measures between discreteprobability distributions. For example, the total variation distance between probabilitydistributions P ( k ) and Q ( k ) is given by T [ P ( k ) , Q ( k )] = P k | P ( k ) − Q ( k ) | , namely, the sumof the differences (in absolute value) between the probabilities assigned to all values of k bythe two distributions. Clearly, for any two distributions P ( k ) and Q ( k ), the total variationdistance satisfies 0 ≤ T [ P ( k ) , Q ( k )] ≤
2. The relative entropy provides an additional upperbound on the total variation distance via the Pinsker inequality, which takes the form [51–54] T [ P ( k ) , Q ( k )] ≤ r S [ P ( k ) || Q ( k )] . (87)This relation implies that whenever the relative entropy between P ( k ) and Q ( k ) vanishes,so does the total variation distance between them, meaning that the two distributions be-come identical in the L norm. This shows that when the relative entropy vanishes thedistributions become identical.In this paper we focused on the case of configuration model networks, which exhibita given degree distribution and no degree-degree correlations. The theoretical frameworkpresented here may provide the foundations for the study of network contraction processesin a much broader class of complex networks, which exhibit degree-degree correlations aswell as other structural correlations. This will require a more general formulation of therelative entropy, expressed in terms of the joint degree distributions of pairs or adjacentnodes, which take into account the correlations between their degrees.The theoretical framework presented here may be relevant in the broad context of neu-rodegeneration, which is the progressive loss of structure and function of neurons in thebrain. Such processes occur in normal aging [55] as well as in a large number of incurableneurodegenerative diseases such as Alzheimer, Parkinson, Huntington and AmylotrophicLateral Sclerosis, which result in a gradual loss of cognitive and motoric functions [56].These diseases differ in the specific brain regions or circuits in which the degeneration oc-curs. The characterization of the evolving structure using the relative entropy may provideuseful insight into the structural aspects of the loss of neurons and synapses in neurodegen-36rative processes [57].It is worth mentioning that there is another class of network dismantling processes thatinvolve optimized attacks, which maximize the damage to the network for a minimal set ofdeleted nodes [29, 30]. Such optimization is achieved by first decycling the network, namely,by selectively deleting nodes that reside on cycles, thus driving the giant component into atree structure. The branches of the tree are then trimmed such that the giant componentis quickly disintegrates. Clearly, these optimized dismantling processes do not convergetowards an ER structure. VIII. SUMMARY
In summary, we have analyzed the structural evolution of complex networks undergoingcontraction processes via generic node deletion scenarios, namely, random deletion, prefer-ential deletion and propagating deletion. Focusing on configuration model networks we haveshown using a rigorous argument that upon contraction the degree distributions of these net-works converge towards a Poisson distribution. In this analysis we used the relative entropy S t = S [ P t ( k ) || π ( k |h K i t )] of the degree distribution P t ( k ) of the contracting network at time t with respect to the corresponding Poisson distribution π ( k |h K i t ) with the same mean degree h K i t as a distance measure between P t ( k ) and Poisson. The relative entropy is suitable as adistance measure since it satisfies S t ≥ P t ( k ), while equality isobtained only for P t ( k ) = π ( k |h K i t ). We derived an equation for the time evolution of therelative entropy S t during network contraction and expressed its time derivative dS t /dt as asum of two terms, ∆ A ( t ) and ∆ B ( t ). We have shown that the first term satisfies ∆ A ( t ) < P t ( k ). This means that the ∆ A ( t ) term always pushes the relativeentropy down towards zero, driving the convergence of P t ( k ) towards Poisson. For the ∆ B ( t )term we provide a condition that can be used for any given degree distribution P t ( k ) to de-termine whether this term would accelerate the convergence to Poisson or slow it down. Thecondition implies that for degree distributions P t ( k ) whose tail falls more slowly than the tailof the corresponding Poisson distribution, the ∆ B ( t ) term would accelerate the convergenceto Poisson, while in the case that the tail falls more quickly than Poisson the ∆ B ( t ) termwhould slow down the convergence. We analyzed the convergence for configuration modelnetworks with degenerate degree distributions (random regular graphs), exponential degree37istributions and power-law degree distributions (scale-free networks) and showed that therelative entropy decreases monotonically to zero during the contraction process, reflectingthe convergence of the degree distribution towards a Poisson distribution. Since the con-tracting networks remain uncorrelated, this means that their structures converge towardsan Erd˝os-R´enyi (ER) graph structure, substantiating earlier results obtained using directintegration of the master equation and computer simulations [31].This work was supported by the Israel Science Foundation grant no. 1682/18. [1] S. Havlin and R. Cohen, Complex networks: structure, robustness and function (CambridgeUniversity Press, New York, 2010).[2] M.E.J. Newman,
Networks: an introduction (Oxford University Press, Oxford, 2010).[3] E. Estrada,
The structure of complex networks: theory and applications (Oxford UniversityPress, Oxford, 2011).[4] P. Erd˝os and A. R´enyi, On random graphs I,
Publ. Math. Debrecen , 290 (1959).[5] P. Erd˝os and A. R´enyi, On the evolution of random graphs, Publ. Math. Inst. Hungar. Acad.Sci. , 17 (1960).[6] P. Erd˝os and A. R´enyi, On the evolution of random graphs II, Bull. Int. Stat. Inst. , 343(1961).[7] B. Bollob´as, Random Graphs, Second Edition (Academic Press, London, 2001).[8] M. Bauer and D. Bernard, Maximal entropy random networks with given degree distribution,arXiv:cond-mat/0206150[9] L. Bogacz, Z. Burda and B. Wac law, Homogeneous complex networks,
Physica A , 587(2006).[10] G. Bianconi, The entropy of randomized network ensembles,
EPL , 28005 (2008).[11] G. Bianconi, Entropy of network ensembles, Phys. Rev. E , 036114 (2009).[12] R. Albert, H. Jeong and A.-L. Barab´asi, Diameter of the World-Wide Web, Nature , 130(1999).[13] S. Redner, How popular is your paper? An empirical study of the citation distribution,
Eur.Phys. J. B , 131 (1998).[14] R. Albert R and A.-L. Barab´asi, Statistical mechanics of complex networks, Rev. Mod. Phys. , 47 (2002).[15] A.-L. Barab´asi and R. Albert, Emergence of scaling in random networks, Science , 509(1999).[16] P.L. Krapivsky, S. Redner and F. Leyvraz, Connectivity of growing random networks,
Phys.Rev. Lett. , 4629 (2000).[17] S.N. Dorogovtsev, J.F.F. Mendes and A.N. Samukhin, Structure of growing networks withpreferential linking, Phys. Rev. Lett. , 4633 (2000).[18] A.-L. Barab´asi, Scale-free networks: a decade and beyond, Science , 412 (2009).[19] J. T¨or¨ok and J. Kert´esz, Cascading collapse of online social networks,
Scientific Reports ,16743 (2017).[20] L. L˝orincz, J. Koltai, A.F. Gy˝or and K. Tak´acs, Collapse of an online social network: Burningsocial capital to create it? Social Networks , 43 (2019).[21] L. Daqing, J. Yinan, K. Rui and S. Havlin, Spatial correlation analysis of cascading failures:congestions and blackouts, Scientific Reports , 5381 (2014).[22] B. Sch¨afer, D. Witthaut, M. Timme and V. Latora, Dynamically induced cascading failuresin power grids, Nature Communications , 1975 (2018).[23] R. Pastor-Satorras and A. Vespignani, Epidemic Spreading in Scale-Free Networks, Phys. Rev.Lett. , 3200 (2001).[24] R. Pastor-Satorras, C. Castellano, P. Van Mieghem and A. Vespignani, Epidemic processes incomplex networks, Rev. Mod. Phys. , 925 (2015).[25] R. Pastor-Satorras and A. Vespignani, Immunization of complex networks, Phys. Rev. E ,036104 (2002).[26] R. Albert, H. Jeong and A.-L. Barab´asi, Error and attack tolerance of complex networks, Nature , 378 (2000).[27] R. Cohen, K. Erez, D. ben-Avraham and D. Havlin, Resilience of the internet to randombreakdowns,
Phys. Rev. Lett. , 4626 (2000).[28] R. Cohen, K. Erez, D. ben-Avraham and S. Havlin, Breakdown of the internet under inten-tional attack, Phys. Rev. Lett. , 3682 (2001).[29] A. Braunstein, L. Dall’Asta, G. Semerjian and L. Zdeborov´a, Network dismantling, Proc.Natl. Acad. Sci. USA , 12368 (2016).[30] L. Zdeborov´a, P. Zhang and H.-J. Zhou, Fast and simple decycling and dismantling of net- orks, Scientific Reports , 37954 (2016).[31] I. Tishby, O. Biham and E. Katzav, Convergence towards an Erd˝os-R´enyi graph structure innetwork contraction processes, Phys. Rev. E , 032314 (2019).[32] M. Molloy and B. Reed, A critical point for random graphs with a given degree sequence,
Random Struct. Algorithms , 161 (1995).[33] B. Molloy and A. Reed, The size of the giant component of a random graph with a givendegree sequence, Combinatorics, Probability and Computing , 295 (1998).[34] M.E.J. Newman, S.H. Strogatz and D.J. Watts, Random graphs with arbitrary degree distri-butions and their applications, Phys. Rev. E , 026118 (2001).[35] A. Annibale, A.C.C. Coolen, L.P. Fernandes, F. Fraternali and J. Kleinjung, Tailored graphensembles as proxies or null models for real networks I: tools for quantifying structure, J.Phys. A , 485001 (2009).[36] E.S. Roberts, T. Schlitt and A.C.C Coolen, Tailored graph ensembles as proxies or null modelsfor real networks II: results on directed graphs, J. Phys. A , 275002 (2011).[37] A.C.C. Coolen, A. Annibale and E.S. Roberts, Generating Random Networks and Graphs(Oxford University Press, Oxford, 2017).[38] N.G. van Kampen, Stochastic processes in physics and chemistry, 3rd Edition (North Holland,2007).[39] C. Gardiner,
Handbook of stochastic methods: for physics, chemistry and the natural sciences,3rd edition , (Springer-Verlag, Berlin, 2004).[40] J.C. Butcher,
Numerical methods for ordinary differential equations (John Wiley & Sons,Chichester, UK, 2003).[41] Trickle down: diffuse downward through some hierarchical structure (The American HeritageDictionary of Phrasal Verbs, Houghton Mifflin Harcourt Publishing Company, New York,2005).[42] S. Kullback and R.A. Leibler, On information and sufficiency,
Annals of Mathematical Statis-tics , 79 (1951).[43] I. Csisz´ar and P.C. Shields, Information Theory and Statistics: A Tutorial, Foundations andTrends in Communications and Information Theory , Volume 1 Issue 4, Edited by S. Verd´u(Now Publishers Inc., Hanover, MA, USA, 2004).[44] S. Kullback,
Information theory and statistics (Dover Publications, Mineola, NY 1969).
45] T.M. Cover and J.A. Thomas, Elements of Information Theory, 2nd Edition (Wiley, Hoboken,New-Jersey, 2006).[46] C.E. Shannon, A Mathematical theory of communication
Bell System Technical Journal ,379 (1948).[47] J.E. Shore and R.W. Johnson, Axiomatic derivation of the principle of maximum entropy andthe principle of minimum cross-entropy, IEEE Trans. on Information Theory , 26 (1980).[48] F.W.J. Olver, D.M. Lozier, R.F. Boisvert and C.W. Clark, NIST handbook of mathematicalfunctions (Cambridge University Press, New York, 2010).[49] A. R´enyi, On measures of information and entropy, Proceedings of the fourth Berkeley sympo-sium on mathematics, statistics and probability, June 20-July 30, 1960, Edited by J. Neyman,Volume I, P. 547-561 (University of California Press, Berkeley, CA, 1961).[50] M.M. Deza and E. Deza,
Encyclopedia of Distances, 4th edition (Springer-Verlag, Berlin,Heidelberg 2016)[51] M.S. Pinsker,
Information and information stability of random variables and processes (Holden-Day, San Francisco, 1964).[52] I. Csisz´ar, Information-type measures of difference of probability distributions and indirectobservations,
Studia Sci. Math. Hungar. , 299 (1967).[53] S. Kullback, A Lower Bound for Discrimination Information in Terms of Variation, IEEETransactions on Information Theory , 126 (1967).[54] I. Vajda, Note on discrimination and variation, IEEE Transactions on Information Theory , 771 (1970).[55] J.H. Morrison and P.R. Hof, Life and death of neurons in the aging brain, Science , 412(1997).[56] M.-T. Heemels, Neurodegenerative diseases,
Nature , 179 (2016).[57] T. Arendt, M.K. Br¨uckner, M. Morawski, C. J¨ager and H.-J. Gertz, Early neurone loss inAlzheimers disease: cortical or subcortical?
Acta Neuropathologica Communications , 10(2015)., 10(2015).