The Cavity Approach to Parallel Dynamics of Ising Spins on a Graph
aa r X i v : . [ c ond - m a t . d i s - nn ] M a y The Cavity Approach to Parallel Dynamics of IsingSpins on a Graph
I. Neri and D. Boll´e
Instituut voor Theoretische Fysica, Katholieke Universiteit Leuven, Celestijnenlaan200D, B-3001 Leuven, BelgiumE-mail: [email protected], [email protected]
Abstract.
We use the cavity method to study parallel dynamics of disordered Isingmodels on a graph. In particular, we derive a set of recursive equations in single siteprobabilities of paths propagating along the edges of the graph. These equations areanalogous to the cavity equations for equilibrium models and are exact on a tree.On graphs with exclusively directed edges we find an exact expression for thestationary distribution of the spins. We present the phase diagrams for an Ising modelon an asymmetric Bethe lattice and for a neural network with Hebbian interactions onan asymmetric scale-free graph.For graphs with a nonzero fraction of symmetric edges the equations can be solvedfor a finite number of time steps. Theoretical predictions are confirmed by simulationresults.Using a heuristic method, the cavity equations are extended to a set of equationsthat determine the marginals of the stationary distribution of Ising models on graphswith a nonzero fraction of symmetric edges. The results of this method are discussedand compared with simulations.PACS numbers: 75.10.Nr, 75.10.Hk, 05.90.+m, 64.60.Cn
1. Introduction
Many problems in different research fields are based upon the interaction of unitsthrough some underlying graph. Some examples are: gene expressions in booleannetworks [1], agents competing for some limited resources [2, 3], interactions between thedecoding variables in low-density parity check codes [4], interactions between humanson a social network [5], the analysis of phase transitions of a spin glass [6].To calculate statistical quantities on a given graph instance, one can use thecavity method [7]. This method is based on the assumption that for sparse graphsthe neighbouring spins only depend on each other through their direct interactions.A similar method, known as the sum-product algorithm, is used in information theoryand artificial intelligence, see [8] for a tutorial paper. Examples of problems investigatedwith the cavity method are: the characterisation of the set of solutions of optimisation he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph
2. Dynamics on a given graph instance
We consider models defined on a given graph instance G = ( V, E ), with V and E respectively the set of vertices (or sites) and the set of edges. We limit ourselvesto simple directed graphs G determined by a connectivity matrix C , with elements[ C ] ij = c ij ∈ { , } . When c ij = 1 and c ji = 0 the graph has a directed edge from the i -th site to the j -th site. When c ij = c ji = 1 there is an undirected edge between i and j and when c ij = c ji = 0 there are no edges between them. We define the sets E , E d and E sym through: E ≡ { ( i, j ) ∈ V × V | c ij = 1 } , E d ≡ { ( i, j ) ∈ V × V | c ij = 1 , c ji = 0 } and E sym ≡ { ( i, j ) ∈ V × V | c ij = 1 , c ji = 1 } . We study the evolution of Ising like models of n -replicated Ising variables σ i ( t ) ∈ {− , } n , with i = 1 , . . . N and t the correspondingdiscrete time step. The dynamics in discrete time is defined by a transition probability W ( σ ( s ) | σ ( s − σ ( s −
1) = ( σ ( s − , σ ( s − , · · · , σ N ( s − s − σ ( s ) on the s -th time step. We consider transitionprobabilities W of the form: W [ σ ( s ) | σ ( s − θ ] = N Y i =1 W [ σ i ( s ) | σ ( s − θ i ( s )] = N Y i =1 W [ σ i ( s ) | h i ( s )] . (1)The n -dimensional local field h i ( s ) is defined through h i ( s ) = X j ∈ ∂ in i h j → i ( σ j ( s − θ i ( s ) , (2)where the field h j → i ( σ j ( s − j on thespin on site i and θ i ( s ) is an external field. We used ∂ in i for the neighbourhood ofall the vertices that influence i directly, i.e. ∂ in i ≡ { j ∈ V | c ji = 1 } . We will also use: ∂ out i ≡ { j ∈ V | c ij = 1 } , ∂ i ≡ ∂ in i ∪ ∂ out i and ∂ sym i ≡ ∂ in i ∩ ∂ out i . The probability to havethe path σ t ..t = ( σ ( t ) , · · · , σ ( t )), from time step t to time step t , is given by P t ..t (cid:0) σ t ..t | θ t +1 ..t (cid:1) = t Y s = t +1 W [ σ ( s ) | σ ( s − θ ( s )] ! P t ( σ ( t )) , (3)with P t ( σ ( t )) the probability distribution of the spins at time step t . Using the cavity method, see [7], it is possible to solve the parallel dynamics ongraphs. The cavity graph G ( i ) is the subgraph of G where the i -th vertex and all ofthe interactions with its neighbours are removed. We write the following relationshipbetween a path probability P ..t on the graph G and the probability P ( i )0 ..t on its relatedcavity graph G ( i ) P ..t (cid:0) σ ..t | θ ..t (cid:1) = P ( i )0 ..t (cid:0) σ ..t | θ ..t + ζ ( i ) , ..t (cid:1) t Y s =1 W [ σ i ( s ) | σ ( s − θ i ( s )] ! p ( σ i (0)) . (4) he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph ζ ( i ) j ( s ), representing the influence of the i -thspin on its neighbours j ∈ ∂ out i : ζ ( i ) j ( s ) = ǫ ij h i → j ( σ i ( s − . (5)The prefactor ǫ ij determines whether the edge is symmetric or not: ǫ ij = 1 for undirectededges and ǫ ij = 0 for directed edges. We took a factorised initial distribution P : P ( σ (0)) = Q Ni =1 p ( σ i (0)). The single site marginal P i, ..t is obtained by summing P ..t in (4) over all paths σ ..tj with j = i . In general, we will use the notations σ S = ( σ i , σ i , · · · , σ i | S | ) , (6) P S ( σ ..tS | θ ..t ) = X σ ..tj , j = S P ( σ ..t | θ ..t ) , (7)with S a set of indices: S = (cid:8) i , · · · , i | S | (cid:9) , where | S | denotes the size of the set S .Within this notation P ∂ i , ..t is the joint probability of the paths on the neighbours of i .When we sum over all paths σ ..tj , with j = i , on the left hand and right hand side of(4), we get P i, ..t (cid:0) σ ..ti | θ (cid:1) = X τ ..t∂i P ∂ i ∪ i, ..t − (cid:0) τ ..t∂ i , τ ..ti | θ ..t (cid:1) = X τ ..t∂i P ( i ) ∂ i , ..t (cid:0) τ ..t − ∂ i | θ ..t − + ζ ( i ) , ..t − (cid:1) t Y s =1 W [ σ i ( s ) | h i ( s )] p ( σ i (0)) . (8)In the sequel we drop the subscript i in the argument of P i, ..t . Now we make theBethe-Peierls approximation: i.e. we assume that the spins in the neighbourhood ∂ i of i become independent when we remove the i -th spin: P ( i ) ∂ i ,t (cid:16) τ ..tj , · · · , τ ..tj | ∂i | | θ ..t + ζ ( i ) , ..t (cid:17) = Y j ∈ ∂ i P ( i ) j, ..t (cid:16) τ ..tj | θ ..tj + ǫ ij ζ ( i ) , ..tj (cid:17) , (9)with ∂ i = (cid:8) j , · · · , j | ∂ i | (cid:9) . In (9) we took θ j = 0 when j / ∈ ∂i ∪ i . We substitute (9) in(8) to get the following set of recursive equations P ( ℓ ) i, ..t (cid:0) σ ..t | θ ..t (cid:1) = X σ ..t − ∂ in i \ ℓ Y j ∈ ∂ in i \ ℓ P ( i ) j, ..t − (cid:16) σ ..t − j | ǫ ij ζ ( i ) , ..t − j (cid:17) t Y s =1 W h σ ( s ) | h ( ℓ ) i ( s ) i p ( σ (0)) ! , (10)for the path probability P ( ℓ ) i, ..t on the graph G ( ℓ ) , with ℓ ∈ ∂i . To derive (10) we used P ( i,ℓ ) j, ..t = P ( i ) j, ..t . The set of | E | -equations (10) determines the | E | -probability distributions P ( ℓ ) i, ..t ( σ ..t | θ ..ti ) at time step t as a function of the | E | -probability distributions P ( ℓ ) i, ..t − ( σ ..t − | θ ..t − i ) at the previous time step t −
1. In equation (10) we only needto take the product over j ∈ ∂ in i because the fields h ( ℓ ) i ( s ) depend only on σ ∂ in i . Wecall the equations (10) the dynamical cavity equations analogous to the static equations he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph P i, ..t onthe original graph G from the cavity distributions, we need to combine equations (8)and (9): P i, ..t ( σ ..t | θ ..t ) = X σ ..t − ∂ in i Y j ∈ ∂ in i P ( i ) j, ..t − (cid:16) σ ..t − j | ǫ ij ζ ( i ) , ..t − j (cid:17) t Y s =1 W [ σ ( s ) | h i ( s )] p ( σ (0)) ! . (11)Equations (11) are the dynamical versions of the set of equations (B.15). The initialproblem of finding the single site marginals P i, ..t from the N -site probability P ..t hasa computational complexity O (2 N ). The set of equations (10) and (11) has a linearcomplexity O ( N ) in the system size and an exponential complexity O (2 t ) in time whichmakes the dynamics solvable for a finite number of time steps.The cavity equations simplify a lot when the graph is fully asymmetric. In thiscase we can set ǫ ij = 0 in equation (10). Therefore, the equations only have to besolved for θ ..t = 0 ..t , where 0 ..t is the null vector. Moreover, because ǫ ij = 0 theself-coupling disappears in (10). We can thus sum on the left and right hand side of(10) over ( σ i (0) , σ i (1) , · · · , σ i ( t − P ( ℓ ) i,t ( σ ) = X σ j , j ∈ ∂ in i Y j ∈ ∂ in i P ( i ) j,t − ( σ j ) W [ σ | h ] . (12)Equation (12) describes a Markovian dynamics.
3. The Ensemble Averaged Distribution of Paths
We calculate the average of equation (10) over all links in the graph, i.e. all a ∈ E . Thegraph is drawn from an ensemble of graphs G . We look at ensembles where the typicalgraphs have a local tree structure and the degrees on different sites are uncorrelated.An example is the Poissonian ensemble G p defined in (A.1) of Appendix A. The degreedistribution is defined through a histogram as p deg ( k in , k out , k sym ) ≡ P Ni =1 δ (cid:0) k in ; k in i (cid:1) δ ( k out ; k out i ) δ ( k sym ; k sym i ) N . (13)In equation (13) we use the following notations: the indegree k in i = | ∂ in i | , the outdegree k out i = | ∂ out i | and the symmetric degree k sym i = | ∂ in i ∩ ∂ out i | . For N → ∞ the dynamicsof Ising models on typical graphs drawn from such ensembles depends on the degreedistribution (13). We define P d as the average of the path probabilities P ( ℓ ) i,t over all he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph i, ℓ ) of E : P d ( σ ..t ) ≡ P ( i,ℓ ) ∈ E d P ( ℓ ) i, ..t ( σ ..t | ..t ) | E d | . (14)The average probability mass function P sym is defined as the average of P ( ℓ ) i,t over alllinks belonging to an undirected edge P sym (cid:0) σ ..t | θ ..t (cid:1) ≡ P ( i,ℓ ) ∈ E sym P ( ℓ ) i, ..t ( σ ..t | θ ..t ) | E sym | . (15)When we use the property that the spins in the neighbourhood of i are uncorrelated,we can write Y j ∈ ∂ i \ ℓ P ( i ) j, ..t − (cid:16) σ ..t − j | ζ ( i ) , ..t − j (cid:17) = Y j ∈ ∂ i \ ℓ P (cid:16) σ ..t − j | ζ ( i ) , ..t − j (cid:17) . (16)It is useful to focus on a specific example. We consider fields of the type h j → i ( s −
1) = J ji σ i ( s − J ij are i.i.d.r.v. drawn from a distribution R ( J ). When we take the average of the update equations (10) according to thedefinitions (14) and (15), and use (16) we find the recursive equations for the averagedprobability mass function of paths. These recursive equations are given by: P d (cid:0) σ ..t (cid:1) = ∞ X k out ≥ ∞ X k in ≥ k out , k in ) X k sym =0 p ( k in , k out , k sym )( k out − k sym ) c out − c sym k in Y ℓ = k sym +1 Z dJ ℓ R ( J ℓ ) X σ ..t − ℓ P d (cid:0) σ ..t − ℓ (cid:1) k sym Y ℓ =1 Z dJ ℓ R ( J ℓ ) X σ ..t − ℓ P sym (cid:0) σ ..t − ℓ | J ℓ σ ..t − (cid:1) p ( σ (0)) t − Y s ≥ W σ ( s + 1) | θ ( s ) + X <ℓ ′ ≤ k in J ℓ ′ σ ℓ ′ ( s ) , (17)and P sym (cid:0) σ ..t | θ ..t (cid:1) = ∞ X k in ≥ k in X k sym =0 p ( k in , k sym ) k sym c sym k in − Y ℓ = k sym Z dJ ℓ R ( J ℓ ) X σ ..t − ℓ P d (cid:0) σ ..t − ℓ (cid:1) k sym − Y ℓ =1 Z dJ ℓ R ( J ℓ ) X σ ..t − ℓ P sym (cid:0) σ ..t − ℓ | J ℓ σ ..t − (cid:1) p ( σ (0)) t − Y s ≥ W σ ( s + 1) | θ ( s + 1) + X <ℓ ′ ≤ k in − J ℓ ′ σ ℓ ′ ( s ) . (18)We introduced the average connectivities c sym ≡ P k in ,k out ,k sym p ( k in , k out , k sym ) k sym and c out ≡ P k in ,k out ,k sym p ( k in , k out , k sym ) k out .The averaged probability mass function P real ( σ ..t ) over the marginals P i ( σ ..t ), he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph P real ( σ ..t ) ≡ P i P i ( σ ..t ) /N , can be calculated from (11): P real (cid:0) σ ..t | θ ..t (cid:1) = ∞ X k in ≥ k in X k sym =0 p ( k in , k sym ) k in Y ℓ = k sym +1 Z dJ ℓ R ( J ℓ ) X σ ..t − ℓ P d (cid:0) σ ..t − ℓ (cid:1) k sym Y ℓ =1 Z dJ ℓ R ( J ℓ ) X σ ..t − ℓ P sym (cid:0) σ ..t − ℓ | J ℓ σ ..t − (cid:1) p ( σ (0)) t − Y s ≥ W σ ( s + 1) | θ ( s + 1) + X <ℓ ′ ≤ k in J ℓ ′ σ ℓ ′ ( s ) . (19)The Markovian dynamics of N spins defined in (1) is thus reduced to an effective non-Markovian dynamics of one single spin given by the recursive equations (17), (18) and(19). Equations analogous to (17) and (18) were derived in [16] in the context of LDGMchannel coding using the generating functional analysis.For fully asymmetric graphs, see (12), we remark that P ( ℓ ) i,t ( σ ) = P i,t ( σ ), but theaverages, P d t ( σ ) ≡ P ( ℓ ) i,t ( σ ) and P real t ( σ ) ≡ P i,t ( σ ), over, respectively, the links and thesites are different. Indeed: P d t ( σ ) = X k in p ( k in ) c ( k in ) c out Y <ℓ ≤ k in Z dJ ℓ R ( J ℓ ) X σ ℓ P d t − ( σ ℓ ) p ( σ (0)) t Y s =1 W [ σ ( s ) | h ( s )] , (20) P real t ( σ ) = X k in p ( k in ) Y <ℓ ≤ k in Z dJ ℓ R ( J ℓ ) X σ ℓ P d t − ( σ ℓ ) p ( σ (0)) t Y s =1 W [ σ ( s ) | h ( s )] , (21)with c ( k in ) = P k out p ( k out | k in ) k out and c out = P k out p ( k out ) k out .
4. Examples of Dynamics
In this section we define the type of dynamics we study by specifying the form of thetransition probabilities W [ σ | h ] used in equation (1). We consider Glauber dynamics for an Ising model with n = 1, i.e. σ i ∈ {− , } . Everyspin σ i ( t ) evolves under the influence of the field h i ( t −
1) with a transition probability W g ( σ i ( t ) | h i ( t )) defined through: W g [ σ | h ] ≡ exp ( βσh )2 cosh ( βh ) . (22)The parameter β is the inverse of the temperature T . It is possible to implement thedynamics defined by (22) and (1) with the heat-bath algorithm [19]. When the graph is he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph G p defined by the probability P p ( C ; c, ǫ ) of the connectivity matrix C : P p ( C ; c, ǫ ) = Y i
1) and τ ( t − σ i ( t ) , τ i ( t ))evolve according to W c [( σ i ( t ) , τ i ( t )); h i ( t ) , g i ( t )] : W c [( σ, τ ) | h, g ] ≡ δ ( σ ; − τ ) | r h − r g | he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph δ ( σ ; τ ) (1 − | r h − r g | )Θ( r h − r g ) (cid:20) δ ( σ ; 1) r g r g − r h + δ ( σ ; −
1) 1 − r h r g − r h (cid:21) + δ ( σ ; τ ) (1 − | r h − r g | )Θ( r g − r h ) (cid:20) δ ( σ ; 1) r h r h − r g + δ ( σ ; −
1) 1 − r g r h − r g (cid:21) , (26)where Θ is the Heaviside step function and the weights r h and r g are given by r h = exp ( βh )2 cosh ( βh ) , r g = exp ( βg )2 cosh ( βg ) . (27)Equation (26) can be simulated using a heat-bath algorithm where at each time step wechoose the same random numbers for both set of spins σ and τ . A more compact formof W c is: W c [( σ, τ ) | h, g ] = δ ( σ ; − τ ) | r h − r g | + δ ( σ ; τ ) Θ( r h − r g ) [ δ ( σ ; 1) r g + δ ( σ ; −
1) (1 − r h )]+ δ ( σ ; τ ) Θ( r g − r h ) [ δ ( σ ; 1) r h + δ ( σ ; −
1) (1 − r g )] . (28)When the thermal average of the distance between the paths σ ( t ) and τ ( t ) doesnot converge to zero for t → ∞ , even when the initial distance between σ (0) and τ (0)is very small, the system is in a chaotic phase. We use the transition probability W c to determine the phase transitions to this chaotic phase. Chaotic behaviour has beenstudied in [21] for spin glasses and in [22] for neural networks. The coupled dynamics(28) can not satisfy detailed balance.
5. The Path Entropy and the Distribution of the Probability Distributionsof Paths
The fluctuations of the path probabilities P ( ℓ ) i, ..t over all links are given by the distributionof the probabilities of the paths which we will call P . They determine quantities like theaverage path entropy S ( t ). On the basis of the recursive equations for the distributions P we discuss in section 9 the stationary solutions of the dynamics.The average path entropy is defined as S ( t ) ≡ − X σ ..t P ..t ( σ ..t ) log ( P ..t ( σ ..t )) , (29)where the bar denotes the average over the quenched variables. With the cavity method[7], we can write S ( t ) = N X i =1 ∆ S site i ( t ) − X a ∈ E sym ∆ S link a ( t ) − X a ∈ E d ∆ S link a ( t ) . (30)The quantity ∆ S site i ( t ) is the increment in the entropy S when the i -th site is added tothe graph G ( i ) :∆ S site i ( t ) = − X σ ..t X σ ..t∂i P ∂ i , ..t ( σ ..t∂ i ) t Y s =1 W [ σ ( s ) | h ( s )] p ( σ (0)) he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph P ∂i, ..t ( σ ..t∂ i ) t Y s =1 W [ σ ( s ) | h ( s )] p ( σ (0)) ! , (31)with P ∂ i , ..t ( σ ..t∂ i ) = Y ( j,i ) ∈ E d P ( i ) j, ..t ( σ ..tj ) Y ( j,i ) ∈ E sym P ( i ) j, ..t ( σ ..tj | J ij σ ..t − ) . (32)The quantity ∆ S link a ( t ) is minus the entropy difference when remove the link a from thegraph G ∆ S link( i,j ) ( t ) = − X σ ..t , τ ..t P i, ..t ( σ ..t ) P ( i ) j, ..t ( τ ..t | J ij σ ..t − )log (cid:16) P i, ..t ( σ ..t ) P ( i ) j, ..t ( τ ..t | J ij σ ..t − ) (cid:17) . (33)The summation over the sites in equation (30) can be done when we know thedistributions of the probabilities of paths P on the graph.We define the following distributions P d ( P ) ≡ P ( i,ℓ ) ∈ E d Q σ ..t δ (cid:16) P ( σ ..t ) − P ( ℓ ) i, ..t ( σ ..t ) (cid:17) | E d | , (34) P sym ( P ) ≡ P ( i,ℓ ) ∈ E sym Q σ ..t ,θ ..t δ (cid:16) P ( σ ..t | θ ..t ) − P ( ℓ ) i, ..t ( σ ..t | θ ..t ) (cid:17) | E sym | , (35) P real ( P ) ≡ P Ni =1 Q σ ..t δ ( P ( σ ..t ) − P i ( σ ..t )) N . (36)When the variables J ij are i.d.d.r.v. P d satisfies the recursive equation: P d ( P ) = ∞ X k out ≥ ∞ X k in ≥ k out , k in ) X k sym =0 p ( k in , k out , k sym )( k out − k sym ) c out − c sym k sym Y ℓ =1 Z dJ ℓ R ( J ℓ ) Z dP ℓ P sym ( P ℓ ) k in Y ℓ = k sym +1 Z dJ ℓ R ( J ℓ ) Z dP ℓ P out ( P ℓ ) Y σ ..t δ (cid:18) P ( σ ..t ) − F d (cid:0) σ ..t ; { P ℓ ′ , J ℓ ′ } ℓ ′ =1 ..k in (cid:1) (cid:19) , (37)with F d (cid:0) σ ..t ; { P ℓ , J ℓ } ℓ =1 ..k in (cid:1) = X σ ..t − , ··· ,σ ..t − k in k sym Y ℓ =1 P ℓ ( σ ..t − ℓ | J ℓ σ ..t − ) k in Y ℓ = k sym +1 P ℓ ( σ ..t − ℓ ) p ( σ (0)) t − Y s ≥ exp (cid:2) βσ ( s + 1) P <ℓ ′ ≤ k in J ℓ ′ σ ℓ ′ ( s ) (cid:3) (cid:2) β P <ℓ ′ ≤ k in J ℓ ′ σ ℓ ′ ( s ) (cid:3) ! . (38)The distribution along symmetric links is given by P sym P sym ( P ) = ∞ X k in ≥ k in X k sym =0 p ( k in , k sym ) k sym c sym he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph k sym − Y ℓ =1 Z dJ ℓ R ( J ℓ ) Z dP ℓ P sym ( P ℓ ) k in − Y ℓ = k sym Z dJ ℓ R ( J ℓ ) Z dP ℓ P out ( P ℓ ) Y σ ..t ,θ ..t δ (cid:18) P (cid:0) σ ..t | θ ..t (cid:1) − F sym (cid:0) σ ..t | θ ..t ; { P ℓ ′ , J ℓ ′ } ℓ ′ =1 ..k in − (cid:1) (cid:19) , (39)with F sym (cid:0) σ ..t | θ ..t ; { P ℓ , J ℓ } ℓ =1 ..k in − (cid:1) = X σ ..t − , ··· ,σ ..t − k in − k sym − Y ℓ =1 P ℓ ( σ ..t − ℓ | J ℓ σ ..t − ℓ ) k in − Y ℓ = k sym P ℓ ( σ ..t − ℓ ) p ( σ (0)) t − Y s ≥ exp (cid:2) βσ ( s + 1) (cid:0) θ ( s + 1) + P <ℓ ′ ≤ k in − J ℓ ′ σ ℓ ′ ( s ) (cid:1)(cid:3) (cid:2) βθ ( s + 1) + β P <ℓ ′ ≤ k in − J ℓ ′ σ ℓ ′ ( s ) (cid:3) ! . (40)The distribution of the single site marginals P i on the original graph is given by P ( P ) = ∞ X k in ≥ k in X k sym =0 p ( k in , k sym ) k sym Y ℓ =1 Z dJ ℓ R ( J ℓ ) Z dP ℓ P sym ( P ℓ ) k in Y ℓ = k sym +1 Z dJ ℓ R ( J ℓ ) Z dP ℓ P d ( P ℓ ) Y σ ..t δ (cid:18) P ( σ ..t ) − F real (cid:0) σ ..t ; { P ℓ ′ , J ℓ ′ } ℓ ′ =1 ..k in (cid:1) (cid:19) (41)with F real (cid:0) σ ..t ; { P ℓ , J ℓ } ℓ =1 ..k in (cid:1) = X σ ..t − , ··· ,σ ..t − k in k sym Y ℓ =1 P ℓ ( σ ..t − ℓ | J ℓ σ ..t − ℓ ) k in Y ℓ = k sym +1 P ℓ ( σ ..t − ℓ ) p ( σ (0)) t − Y s ≥ exp (cid:2) βσ ( s + 1) P <ℓ ′ ≤ k in J ℓ ′ σ ℓ ′ ( s ) (cid:3) (cid:2) β P <ℓ ′ ≤ k in J ℓ ′ σ ℓ ′ ( s ) (cid:3) ! . (42)In section 9 we use the equations (37), (39) and (41) to derive the stationary limit fromthe dynamics. When we compare the equations (37) and (39) with the density evolutionequations (B.12) we see a couple of differences. Since the graph is directed we have nowtwo distributions: one for the probabilities propagating along symmetric edges and onefor the probabilities propagating along directed edges. Since the equations (37) and (39)describe the dynamics of the model they are recursive equations. The computationalcomplexity of (37) and (39) scales exponentially in time. Equation (41) is the dynamicalequivalent of (B.13).
6. Comparison with Simulations
In this section we compare the magnetisation m ( t ) = P σ ..t σ ( t ) P real ( σ ..t | ..t ),predicted by equations (17), (18) and (19), with results from simulations. It is difficultto develop an Eisfeller-Opper scheme [23] for these equations because the probabilitydistributions of paths P sym depend on the fields θ ..t , such that it is necessary to solve he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph time m T/(J C)= 1.5T/(J C)= criticalT/(J C) = 1.1
Figure 1.
The magnetisation m as a function of discrete time for an Ising model ona symmetric Bethe lattice with connectivity C = 3. The interactions are drawn fromthe bimodal distribution (43) with ρ = 0 .
25. The exact enumeration of the recursiveequations (18) and (19) (lines) are compared with Monte Carlo simulations (markers).The red line is calculated at the critical temperature T c ≈ . time m C = 1C = 2C = 3
Figure 2.
The time evolution of the magnetisation m at T = 1 . k in = 3 and a given fixed outdegree k out = C . Theexact enumeration of the recursive equations (17), (18) and (19) (lines) are comparedwith Monte Carlo simulations (markers). (18) for all 2 t possible values of θ ..t . We calculate the first time steps through an exactenumeration of the equations (17), (18) and (19).In figures 1 and 2 we compare the magnetisation in the first time steps with the he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph J are drawn from the bimodal distribution R : R ( J ) = (cid:18) ρ (cid:19) δ ( J − J ) + (cid:18) − ρ (cid:19) δ ( J + J ) , (43)with ρ the bias in the couplings. In figure 2 we show results for the Ising model on aBethe lattice without bond disorder. Both, the exact enumeration and the simulationgive consistent results confirming the correctness of equations (17), (18) and (19).
7. The Ising Model on a Fully Asymmetric Bethe Lattice
The dynamical cavity equations (10) simplify to (12) for fully asymmetric graphs.We illustrate this for the Ising model on a graph with a degree distribution p B ( k in , k out , k sym ) = δ ( k in − C ) p ( k out ) δ ( k sym ). We call the graphs drawn from thisensemble asymmetric Bethe lattices. Because now P real t ( σ ) = P d t ( σ ), we only needto solve the recursive equation (20). We discuss this model for two typical dynamics. In this subsection we let the spins evolve through Glauber dynamics defined insection 4.1. The probability of σ i ( s ) is given by equation (22) with a field h i ( s ) = P j ∈ ∂ in i J ji σ j ( s −
1) .
We derive the equations that give the evolution over time of the following macroscopicobservables: the average magnetisation, the correlation function and the distribution ofmagnetisations.We define the magnetisation m ( t ) through the relation P t ( σ ) = ( m ( t ) σ + 1) / m ( t + 1) = Y <ℓ ≤ C "X σ ℓ (cid:18) σ ℓ m ( t )2 (cid:19) tanh " β θ ( t ) + X <ℓ ≤ C J ℓ σ ℓ ! J ,J , ··· ,J C . (44)To find the correlation function C ( t, t ′ ) between spins at time step t and t ′ , we sumequation (17) over all spins except σ ( t ) and σ ( t ′ ). We get the recursive equation for thetwo time marginal P t,t ′ ( σ ( t ) , σ ( t ′ )). When we define C ( t, t ′ ) through P t,t ′ ( σ, τ ) = 14 [1 + m ( t ) σ + τ m ( t ′ ) + C ( t, t ′ ) στ ] , (45)we obtain the recursive equation for the correlation function: C ( t + 1 , t ′ + 1) = Y <ℓ ≤ C X σ ℓ τ ℓ [1 + σ ℓ m ( t ) + τ ℓ m ( t ′ ) + σ ℓ τ ℓ C ( t, t ′ ]4 ! he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph (cid:28) tanh " β θ ( t ) + X <ℓ ≤ C J ℓ σ ℓ ! tanh " β θ ( t ′ ) + X <ℓ ≤ C J ℓ τ ℓ ! J ,J , ··· ,J C . (46)When we calculate the distribution of the marginals P ( ℓ ) i,t from equation (12), orequivalently the distribution W t ( m ) of the corresponding magnetisations, the followingrecursive equation appears W t ( m ) = Z C Y ℓ =1 ( dJ ℓ R ( J ℓ )) Z C Y ℓ =1 ( dm ℓ W t − ( m ℓ )) δ m − Y <ℓ ≤ C X σ ℓ (cid:18) σ ℓ m ℓ (cid:19) tanh " β θ ( t ) + X <ℓ ′ ≤ C J ℓ ′ σ ℓ ′ ! . (47)The time evolution determined by the equations (44), (46) and (47 is confirmed bynumerical simulations. Using the above equations (44), (46) and (47) we can find the stationary solutions. Weconsider the stationary solution m ( t + 1) = m ( t ) = m . Substitution of this ansatz in(44) shows that m is a solution of m = Y <ℓ ≤ C "X σ ℓ (cid:18) σ ℓ m (cid:19) tanh " β X <ℓ ′ ≤ C J ℓ ′ σ ℓ ! J ,J , ··· ,J C . (48)The model has a phase transition between a ferromagnetic phase (F-phase) with m > m = 0 at hightemperatures. Because this transition is continuous it is possible to determine the P toF phase transition line with an expansion of the right hand side of (48) around m = 0.The critical inverse temperature β ∗ between the P-phase and F-phase is the solution of:1 = ρ − C C X r =0 kr ! | r − k | tanh ( β ∗ J | r − k | ) . (49)Equation (49) holds for the bimodal distributions R ( J ) of the form (43). Using thestationary ansatz q = C ( t, t ′ ) = C ( t − n, t ′ − n ) in (46) we can try to find a phasetransition between a paramagnetic phase with q = 0 and a spin glass phase (SG-phase)with q >
0. Analogously to [20] we find that q = 0 for all temperatures and biases ρ . In figure 3 we show the P to F transitions (solid lines) for different values of theconnectivity C as a function of the temperature T and the bias ρ in the couplings . Although the spin model studied in this section has no SG-phase, it has a chaotic phase(CH-phase). In order to find this phase it is necessary to consider the dynamics of twoset of spins σ and τ that interact on the same graph with the same thermal noise witha slightly different initial configuration. The transition probability of the spins ( σ i , τ i )is given by (28) with h i ( s ) = P j ∈ ∂ in i J ji σ j ( s −
1) and g i ( s ) = P j ∈ ∂ in i J ji τ j ( s − he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph ρ T / ( J C ) C = 2C = 3C = 4C = 5C = 10C = 100C = inf
CH FP Figure 3.
The P to F transition lines (solid) for the Ising model on an asymmetricBethe graph are presented as a function of the rescaled temperature
T / ( J C ) andthe bias ρ in the couplings (see equation (43)). Phase transition lines for differentconnectivities C are shown. The dashed lines enclose the regions where the modelbehaves chaotically. As done in (7) it is possible to derive the recursive equations for the single timemarginals P d t ( σ, τ ). We define the magnetisation m t and the thermal average of theHamming distance d t between the sets σ and τ through P d t ( σ, τ ) = 14 (1 + m t σ + m t τ + (1 − d t ) στ ) . (50)In the case of a bimodal distribution R ( J ) we get for d t the recursive equation d t = C − X n =0 k in n ! ( d t − ) k in − n (1 − d t − ) n Z dxdy (cid:12)(cid:12)(cid:12)(cid:12) exp ( β ( x + | y | ))2 cosh ( β ( x + | y | )) − exp ( β ( x − | y | ))2 cosh ( β ( x − | y | )) (cid:12)(cid:12)(cid:12)(cid:12) n X v =0 nv ! (cid:18) ρm t − / (1 − d t − )2 (cid:19) v (cid:18) − ρm t − / (1 − d t − )2 (cid:19) n − v δ ( x − v + n ) k in − n X w =0 k in − nw ! n − k in δ (cid:0) y − w + k in − n (cid:1) . (51)The time evolution of the magnetisation m t is given by equation (44). In the case ofa Gaussian distribution of the couplings we find for d t the equation derived in [21].Starting from an initial configuration with d ≈
0, the system is said to be chaotic whenthe Hamming distance satisfies d t > t → ∞ . In the CH-phase twopaths that are initially close to each other diverge for t → ∞ .We consider a stationary ansatz d t = d and m t = m in equation (51). When d > he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph d = 0 solution. We find the following equation for theinverse transition temperature β ∗ to the CH-phase1 = ∞ X k out ≥ ∞ X k in ≥ k in min( k out , k in ) X u =0 p ( k in , k out , u )( k out − k sym ) c out − c sym Z dxdy (cid:12)(cid:12)(cid:12)(cid:12) exp ( β ∗ ( x + 1))2 cosh ( β ∗ ( x + 1)) − exp ( β ∗ ( x − β ∗ ( x − (cid:12)(cid:12)(cid:12)(cid:12) k in − X v =0 k in − v ! (cid:18) ρm (cid:19) v (cid:18) − ρm (cid:19) k in − − v δ (cid:0) x − v + k in − (cid:1) . (52)For C → ∞ equation (52) reduces to T = 4 e − β ∗ ρm (cid:0) e − β ∗ ρm (cid:1) − . In figure 3the different phase transitions are shown. For ρ large enough, and decreasing thetemperature starting from a large value we obtain subsequently the following phases:P-phase, CH-phase, the chaotic part of the F-phase and the non-chaotic part of theF-phase.
8. Neural Network on a Scale-Free Graph
The interactions between neurons in organisms are most of the time asymmetric.Introducing asymmetric couplings in models for neural networks increases the biologicalrealism of the models under study. That is why in [12, 22] the Hopfield model wasgeneralized to include asymmetric couplings.We add some more realism to the model by defining the neural network on a graphwith a given degree distribution. Many real-world networks have a degree distributionof the form p ( k ) = ak − γ , with a a normalisation constant. These are called scale-freegraphs. One example is the network of brain activity which has scale-free features [24].In [25] neural networks on scale-free graphs with only symmetric couplings were studied.We consider a neural network on a fully asymmetric graph with the followingdistribution of indegrees and outdegrees [26], p ( k in , k out ) = Aak − λ out δ (cid:0) k in , k out (cid:1) + (1 − A ) a k − λ in k − λ out . (53)The correlation factor A in (53) denotes the fraction of sites where the numberof connections entering and leaving the site are equal. Real-world networks havecorrelations between the indegrees and outdegrees [26]. This correlation between thedegrees will turn out to have much influence on the performance of a scale-free neuralnetwork. For fixed λ we will change the average number of interactions by increasingthe lower bound b : p ( k in , k out ) = 0 for k in < b and k out < b .We take the strengths of the interactions J ij according to the Hebb rule: J ij = 1 p p X µ =1 ξ µi ξ µj , (54)with the ξ µi ∈ {− , } , uncorrelated patterns drawn from the probability distribution q ( ξ µi ) = 12 ( δ ( ξ µi ; 1) + δ ( ξ µi ; − . (55) he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph p patterns ξ i = ( ξ i , ξ i , · · · , ξ pi ) on each site. Because the J ij are noti.i.d.r.v. variables we can not use the equations (17) and (18). We first derive the recursive equations for the marginal distributions when the variablesevolve through a Glauber dynamics. From these equations we determine the phasetransition from a P-phase to a retrieval phase (R-phase). In the R-phase the networkcan recover a stored pattern while in the P-phase the noise is too large to retrieve a storedpattern from a distorted signal. To calculate the mean of the cavity equations (10) overthe quenched variables, it is necessary to define the sublattices I ξ : I ξ ≡ { i ∈ V | ξ i = ξ } .The averaged path probabilities P d ξ ( σ ( t ) ) and P real ξ ( σ ( t ) ), on the sublattices I ξ are definedas P d ξ (cid:0) σ ..t (cid:1) ≡ P i ∈ I ξ P ( i,ℓ ) ∈ E d P ( ℓ ) i ( σ ..t | ..t ) P i ∈ I ξ P ( i,ℓ ) ∈ E d , (56) P real ξ ( σ ..t ) ≡ P i ∈ I ξ P i ( σ ..t | ..t ) P i ∈ I ξ . (57)When the graph is drawn from an ensemble defined by a degree distribution of the form p ( k in , k out , k sym ) = p ( k in , k out ) δ ( k sym ), such that there are no symmetric couplings, weget the following recursive equation for P d ξ P d ξ (cid:0) σ ..t (cid:1) = ∞ X k in = b p ( k in ) c out (cid:0) k in (cid:1) c out Y <ℓ ≤ k in X σ ..t − ℓ X ξ ℓ P d ξ ℓ ( σ ..t − ℓ )2 p p ( σ (0)) Y s ≥ exp h βσ ( s + 1) P <ℓ ′ ≤ k in ξ · ξ ′ ℓ p σ ℓ ′ ( s ) i h β P <ℓ ′ ≤ k in ξ · ξ ′ ℓ p σ ℓ ′ ( s ) i . (58)In the above the c out ( k in ) is the average number of directed edges leaving a site, giventhe indegree k in : c out ( k in ) = P k out p ( k out | k in ) k out . For the averaged path probability onthe original graph we obtain analogously P real ξ (cid:0) σ ..t (cid:1) = ∞ X k in = b p ( k in ) Y <ℓ ≤ k in X σ ..t − ℓ X ξ ℓ P d ξ ℓ ( σ ..t − ℓ )2 p p ( σ (0)) Y s ≥ exp h βσ ( s + 1) P <ℓ ′ ≤ k in ξ · ξ ℓ p σ ℓ ′ ( s ) i h β P <ℓ ′ ≤ k in ξ · ξ ℓ ′ p σ ℓ ′ ( s ) i . (59)We will use the notation P a ξ ( σ ( t )) = (cid:0) m a ξ ( t ) σ ( t ) (cid:1) with superscript a = d ora = real. The magnetisations m d ξ and m real ξ evolve in time according to m a ξ ( t ) = ∞ X k in = b p ( k in ) c a (cid:0) k in (cid:1) c out − pk in Y <ℓ ≤ k in X σ ℓ X ξ ℓ (cid:18) σ ℓ m a ξ ℓ ( t − (cid:19) tanh (cid:18) β P <ℓ ′ ≤ k in ξ · ξ ℓ ′ σ ℓ ′ p (cid:19) , (60) he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph c d ( k in ) = c out ( k in ) = (1 − A ) c out + A k in , (61) c real ( k in ) = c out . (62)We simplify the equations with the condensed ansatz m a ξ ( t ) = ξ m a ( t ). This ansatzassumes that the spins (= neurons) only have a finite overlap with the first pattern.The overlap m a ( t ) evolves according to m a ( t ) = ∞ X k in = b p ( k in ) c a ( k in ) c out M ( k in ) , (63)with M ( k in ) = 2 − ( p − k in k in ( p − X r =0 k in X s =0 k in ( p − r ! k in s !(cid:20) m d ( t − (cid:21) s (cid:20) − m d ( t − (cid:21) k in − s tanh (cid:18) β (2 s + 2 r − k in p ) p (cid:19) . (64)When calculating numerically the sum in the degrees k in in equation (63) we have tointroduce a cutoff K . We will bound m a by two values, m al < m a and m au > m a , with m al and m au defined through m al ( t ) ≡ K X k in = b p ( k in ) c a ( k in ) c out M ( k in ) , (65) m au ( t ) ≡ K X k in = b p ( k in ) c a ( k in ) c out M ( k in ) + sign (cid:0) m du ( t − ) (cid:1) ∞ X k in = K + p ( k in ) c a ( k in ) c out . (66)In equation (66) we used that M ( ∞ ) = sign (cid:0) m d ( t − ) (cid:1) . Because we have a power-lawdecay of the degree distribution (and not an exponential decay) it is important to takethe cutoff K into consideration when we want to know the asymptotic behaviour of theneural network for K → ∞ . The macroscopic observables will converge much slower tothe asymptotic value K = ∞ when the degree distribution is power law. For finite K the time evolution of equation (65) is confirmed by Monte Carlo simulations. When we consider a stationary state of the form, m ( t ) = m ( t − β R of the P to R transition1 = X k in p ( k in ) c (cid:0) k in (cid:1) c out A ( β R , k in ) , (67)with A ( β R , k in ) = 2 − pk in k in ( p − X r =0 k in X s =0 k in ( p − r ! k in s ! he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph s − k in ) tanh (cid:18) β R (2 s + 2 r − k in p ) p (cid:19) . (68)Below the critical temperature T R = 1 /β R , m real > p ( k in ) = ak − γ in with k in ∈ [ b, · · · , K ] and zero forother values of k in . We introduce the lower critical value β Rl , and the upper value β Ru through1 = K X k in = b p ( k in ) c (cid:0) k in (cid:1) c out A ( β Rl , k in ) , K X k in = b p ( k in ) c (cid:0) k in (cid:1) c out A ( β Ru , k in ) + r πp ∞ X k in = K +1 p ( k in ) √ k in c (cid:0) k in (cid:1) c out ! . (69)To derive equation (69) we used that A ( β, C ) has the following asymptotic behaviourfor C → ∞ A ( β, C ) → r πp √ C . (70)The asymptotic value of the Riemann sum in the second term of (69) can be calculatedusing a series that converges exponentially in K [27]. In figure 4 we compare how β Rl and β Ru converge to their asymptotic value for K → ∞ . The upper bound β Ru clearlyconverges fast to the asymptotic value. The lower bound β Rl on the other hand convergesvery slow to its asymptotic value. When we would have bounded the critical value β R from below with β Rl , which we do usually for Poissonian graphs, we would have obtaineda bad estimate of the critical temperature β R . Therefore, we estimate in figures 5 and 6the critical temperatures T R with the upper bounds T R u . There we plotted the criticaltemperatures T R as a function of, respectively, the exponent λ and the correlation factor A . The retrieval phase increases with the exponent λ , which is expected because themean connectivity of the graph also increases with λ . Increasing the correlation A between the indegrees and the outdegrees on scale-free graphs has a positive effect onthe performance of the network. The neural network becomes much more tolerant tonoise and can retrieve considerably more patterns when A increases. In this subsection we determine the CH-phase of the neural network. In [22] this wasdone on a Poissonian graph using annealed methods. We consider two systems on thesame graph undergoing the same thermal noise through the coupled dynamics of 4.2.We find the following Markovian process for the single time marginals P d t, ξ : P d t, ξ ( σ, τ ) = X k in p ( k in ) c ( k in ) c out Y <ℓ ≤ k in X σ ℓ ,τ ℓ X ξ ℓ P d t − , ξ ℓ ( σ ℓ , τ ℓ )2 p W c σ, τ | X <ℓ ′ ≤ k in ξ · ξ ℓ ′ p σ ℓ ′ , X <ℓ ′ ≤ k in ξ · ξ ℓ ′ p τ ℓ ′ , (71) he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph K β R CHR
Figure 4.
Neural network on a scale-free graphs: The bounds β Ru , β chu (upper lines)and β Rl , β chl (lower lines) on the inverse critical temperatures β R , β ch as a function ofthe cutoff K . The bounds on β R are calculated for the model parameters λ = 2, p = 3, b = 4. The bounds on β ch are calculated for λ = 2 . p = 3, b = 4. The upper boundssaturate much faster than the lower bounds. λ T p = 3p = 4p = 5 PCHR
Figure 5.
Neural network on a scale-free graphs: The critical temperatures T R (solidlines), T ch (upper dashed lines) and T m (lower dashed lines) as a function of theexponent λ (see equation (53)) for a different number of patterns p . The minimalindegree is b = 4 and the correlation factor A = 0. The R-phase is located below thesolid line. The neural network is chaotic between the dashed lines. where W c is the transition probability defined in (28). We parametrise the single timemarginals P d t, ξ with the magnetisations m ξ , ( t ), m ξ , ( t ) and the Hamming distance d ξ ( t ) he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph A T p = 3p = 4p = 5p = 7 P R
Figure 6.
The critical temperatures of the retrieval state T R of the neural network asa function of the correlation factor A of the scale-free graph for a different number ofpatters p are compared. The ensemble of scale-free graphs has the parameters λ = 2 . b = 4. The critical temperatures are estimated with T Ru for K = 2000. The R-phaseincreases considerably with the correlation factor A . at time step t : P d t, ξ ( σ, τ ) = 14 [1 + σ m ξ , ( t ) + τ m ξ , ( t ) + στ (1 − d ξ ( t ))] . (72)The Hamming distance evolves according to d ξ ( t ) = ∞ X k in = b p ( k in ) Y <ℓ ≤ k in X σ ℓ ,τ ℓ X ξ ℓ − p (cid:18) σ ℓ m ξ ℓ , ( t −
1) + τ ℓ m ξ ℓ , ( t −
1) + (1 − d ξ ℓ ( t − σ ℓ τ ℓ (cid:19)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) exp h β P ℓ ′ (cid:16) ξ · ξ ℓ ′ p (cid:17) σ ℓ ′ i h β P ℓ ′ (cid:16) ξ · ξ ℓ ′ p (cid:17) σ ℓ ′ i − exp h β P ℓ ′ (cid:16) ξ · ξ ℓ ′ p (cid:17) τ ℓ ′ i i β P ℓ ′ (cid:16) ξ · ξ ℓ ′ p τ ℓ ′ (cid:17)i (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) . (73)The magnetisations m ξ , ( t ) and m ξ , ( t ) behave according to equation (60). We use thecondensed initial conditions, m ξ , (0) = ξ m (0), m ξ , (0) = ξ m (0) and d ξ (0) = d (0).These initial conditions imply that the initial configurations have a finite overlap withonly the first pattern. From the time evolution (71) we find that for the condensedinitial conditions m ξ , ( t ) = ξ m ( t ), m ξ , ( t ) = ξ m ( t ) and d ξ ( t ) = d ( t ). The evolution ofthe overlap m ( t ) is given by equation (63). For d ( t ) we get d ( t ) = ∞ X k in = b p ( k in ) k in X n =0 k in n ! ( d ( t − k in − n (1 − d ( t − n he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph ( k in − n )( p − X R =0 n ( p − X R =0 f (cid:0) R , ( p − k in − n ) (cid:1) f ( R , ( p − n ) Z dxdy exp (cid:20) βp ( − n ( p −
1) + 2 R + x ) (cid:21)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) exp h βp (cid:0) | y | + 2 R − ( k in − n )( p − (cid:1)i h βp ( x + | y | − k in ( p −
1) + 2 R + 2 R ) i − exp h βp (cid:0) −| y | − R + ( k in − n )( p − (cid:1)i h βp ( x − | y | + ( k in − n )( p − − R + 2 R ) i (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) n X v =0 nv ! (cid:18) m ( t − / (1 − d ( t − (cid:19) v (cid:18) − m ( t − / (1 − d ( t − (cid:19) n − v δ ( x − v + n ) k in − n X w =0 k in − nw ! − n + k in δ (cid:0) y − w + k in − n (cid:1) . (74)In equation (74) we summed subsequently over the following variables: the indegrees k in , the number of neighbouring spins n with σ = τ , the number of neighbouring spins v with σ = τ and σ = 1 and the number of neighbouring spins w with σ = τ and σ = 1. The summation variables R and R are the number of non-condensed patterns ξ µℓ on the neighbouring spins with, respectively, σ = τ and σ = τ that are equal to thecorresponding pattern ξ µ on the original site. The complex function f ( R ; x ) used inequation (74) equals f ( R ; x ) = 12 π Z π dω exp [ iωR ] (cid:12)(cid:12)(cid:12) cos (cid:16) ω (cid:17)(cid:12)(cid:12)(cid:12) x exp (cid:20) − ix arctan (cid:18) sin ω ω (cid:19)(cid:21) . (75)It is possible to find an equation for the transition temperature β ch to a CH-phase withthe stationary value d > d ( t ) by expanding the left side of (74) around d ( t −
1) = 0:1 = X k in p ( k in ) k in c ( k in ) c out B ( β, k in , m ) , (76)with B ( β, k in , m ) = Z dx k in − X v =0 k in − v ! (cid:18) m (cid:19) v (cid:18) − m (cid:19) k in − − v δ (cid:0) x − v + k in − (cid:1) p − X R =0 ( k in − p − X R =0 f ( R , p − f (cid:0) R , ( p − k in − (cid:1) exp (cid:20) βp (cid:0) − ( k in − p −
1) + 2 R + x (cid:1)(cid:21) he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) exp h βp (1 + 2 R − ( p − i h βp (( x + 1) − k in ( p −
1) + 2 R + 2 R ) i − exp h βp ( − − R + ( p − i h βp (( x −
1) + ( − k in + 2)( p − − R + 2 R ) i (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) . (77)The asymptotic behaviour of B for C → ∞ is given by: B ( β, C, m ) ∼ exp ( C Ψ( m )) , (78)withΨ( m ) = log (1 − m ) + (1 + m ) exp (cid:16) xp (cid:17) + ( p −
1) log (cid:16) xp (cid:17) − x , (79)and x = p − mp + m + r(cid:16) − m + mp (cid:17) − ( m − m )1 + m . (80)When | m | > <
0, hence, the series converges exponentially for large C . When m = 0 we have that Ψ = 0. The asymptotic behaviour of B is then: B ( β, C, m ) ∼ √ C (cid:18)r pπ (cid:19) − p +1 p − X r =0 p − r ! | r − ( p − | = ζ √ C . (81)We define the upper bounds β chu and β chl on the inverse critical temperature β ch to theCH-phase for m = 0 as:1 = K X k in = b p ( k in ) k in c ( k in ) c out B ( β chl , k in , , (82)1 = K X k in = b p ( k in ) k in c ( k in ) c out B ( β chu , k in ,
0) + ζ ∞ X k in = K +1 p ( k in ) √ k in c ( k in ) c out . (83)The convergence of β chu and β chl to their asymptotic value β ch in function of K is plottedin figure 4. Because β chu saturates much faster we used this value in figure 5 to estimate β ch . The R-phase contains a part with d >
0. The bounds β mu and β ml on the inversecritical temperature β m of the transition from the chaotic part of the R-phase to thenon chaotic part of the R-phase are calculated with substitution of respectively m al and m au into equation (76). The value of β ml is a good approximation to β m . In figure 5the complete phase diagram of the neural network with the P-phase, the R-phase andthe CH-phase is presented. The chaotic region of the neural network is enclosed bythe dashed lines. This region is larger for odd values of p . The R-phase and CH-phasebecome smaller when p increases and the non chaotic part of the R-phase dissapearswhen γ increases. The CH-phase is larger for odd values of p than for even values of p . he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph
9. The Stationary Solution
In this section we develop an algorithm to calculate the marginals of the stationarysolution. This algorithm would be the equivalent of the belief propagation equations(B.14). Equations (10) constitute an algorithm with a linear computational complexity O ( N ). But because the algorithm scales as O (2 t ) in time t we can only follow thedynamics for a small number of time steps. We are interested in the marginals of thestationary state when t → ∞ . To solve this we introduce some assumptions on thedistribution P ( σ ..t ). The first simple approximation one can make is to neglect the correlations in time: P ℓ ( σ ..t | θ ..t ) = t Y s =1 P ∗ ℓ ( σ ( s ) | θ ( s )) = t Y s =1 exp [ βu ℓ ( θ ( s )) σ ( s )]2 cosh [ βu ℓ ( θ ( s ))] . (84)In general (see for example equations (37), (39) and (41)), when solving the dynamics,we have equations of the following type P ( σ ..t | θ ..t ) = F (cid:0) σ ..t | θ ..t ; { P ℓ , J ℓ } ℓ =1 ..k (cid:1) . (85)We close these equations with (84) using P ∗ ( σ ( t ) | θ ) = lim s →−∞ X σ s..t − P ( σ s..t | θ s +1 ..t ) , (86)with θ s..t the constant vector with components θ . When we insert equation (84) in theleft hand side of (86) we find P ∗ ( σ ( t ) | θ ) = lim s →−∞ X σ s..t − F (cid:0) σ s..t | θ s +1 ..t ; { P ∗ ℓ , J ℓ } ℓ =1 ..k (cid:1) . (87)This approximation leads to the correct stationary solution in the case of models definedon fully symmetric or fully asymmetric graphs. This makes us curious to see howgood this approximation would work for models defined on partially asymmetric graphs.Because we neglected correlations in time we can not expect a good description of thespin glass phase. We apply the approximation given by (84) and (86) to the equations (37), (39) and (41): P sym ( σ ..t | θ ..t ) = t Y s =1 exp [ βu sym ( θ ( s )) σ ( s )]2 cosh [ βu sym ( θ ( s ))] ! p (cid:0) σ (cid:1) , (88) P d ( σ ..t ) = t Y s =1 exp (cid:2) βu d σ ( s ) (cid:3) βu d ] ! p ( σ ) , (89) P real ( σ ..t ) = t Y s =1 exp [ βu σ ( s )]2 cosh [ βu ] ! p ( σ ) . (90) he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph F given by equations (38),(40) and (42) and close the equations through (86). We define the function u = U k in ,k sym ( { u ℓ , J ℓ } ; θ ) through the explicit solution u of the implicit equation u = 12 β X σ σ log X τ ,τ exp h βσ ( P k in ℓ =1 J ℓ τ ℓ + θ ) i h β (cid:16)P k in ℓ =1 J ℓ τ ℓ + θ (cid:17)i exp h β P k sym ℓ =1 τ ℓ u ℓ ( J ℓ τ ) + β P k in ℓ = k sym +1 τ ℓ u ℓ iQ k sym ℓ =1 cosh [ βu ℓ ( J ℓ τ )] exp [ βτ u ] . (91)In the sequel we use bimodal distributions of the form (43). We then have two type ofmessages propagating along symmetric links: u ( θ ) = u ( J σ ) = u σ . Using the one-timeapproximation in (37) we get for the density of the fields propagating along directededges W d ( u ) = ∞ X k out ≥ ∞ X k in ≥ k out , k in ) X k sym =0 p ( k in , k out , k sym )( k out − k sym ) c out − c sym k sym Y ℓ =1 Z dJ ℓ R ( J ℓ ) Z du + ℓ du − ℓ W sym ( u + ℓ , u − ℓ ) k in Y ℓ = k sym +1 Z dJ ℓ R ( J ℓ ) Z du ℓ W d ( u ℓ ) δ (cid:2) u − U k in ,k sym ( { u ℓ ′ , J ℓ ′ } ℓ ′ =1 ..k in ; 0) (cid:3) . (92)The density of the fields propagating along the symmetric edges follows from theequations (39) W sym ( u + , u − ) = ∞ X k in ≥ k in X k sym =0 p ( k in , k sym ) k sym c sym k in − Y ℓ = k sym Z dJ ℓ R ( J ℓ ) Z du ℓ W d ( u ℓ ) k sym − Y ℓ =1 Z dJ ℓ R ( J ℓ ) Z du + ℓ du − ℓ W sym ( u + ℓ , u − ℓ ) δ (cid:2) u − − U k in − ,k sym − (cid:0) { u ℓ ′ , J ℓ ′ } ℓ ′ =1 ..k in − ; − (cid:1)(cid:3) δ (cid:2) u + − U k in − ,k sym − (cid:0) { u ℓ ′ , J ℓ ′ } ℓ ′ =1 ..k in − ; 1 (cid:1)(cid:3) . (93)Substitution of the one-time approximation (90) in the equation for the distributionof the real marginals (41) gives W real ( u ) = ∞ X k in ≥ k in X u =0 p ( k in , k sym ) k sym Y ℓ =1 Z dJ ℓ R ( J ℓ ) Z du + ℓ du − ℓ W sym ( u + ℓ , u − ℓ ) k in Y ℓ = k sym +1 Z dJ ℓ R ( J ℓ ) Z du ℓ W d ( u ℓ ) δ (cid:2) u − U k in ,k sym ( { u ℓ ′ , J ℓ ′ } ℓ ′ =1 ..k in ; 0) (cid:3) (94)Because of the analogy with the equations (B.12) we call the equations (92) and (93) thedensity evolution equations in the one-time approximation. We see that now we havetwo densities instead of one: the density for the fields propagating along symmetric he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph U k in ,k sym , for partiallyasymmetric graphs, is not explicitly known. Instead we have to solve the implicitequation (91). The equations (92), (93) and (94) constitute an algorithm that generalizesbelief propagation to graphs with asymmetric bonds. For graphs with exclusively symmetric or asymmetric links, we determine the explicitform of U k in ,k sym . For a fully symmetric graph, the equation (91) admits the solution U k in − ,k sym − (cid:0) { u ℓ , J ℓ } ℓ =1 ..k sym − ; θ (cid:1) = θ + β − k sym − X ℓ =1 atanh [tanh ( βJ ℓ ) tanh ( βu ℓ )] , (95)where we used u ℓ ( J ℓ τ ) = J ℓ τ + u ℓ . For fully asymmetric models we have the solution U k in ,k sym ( { u ℓ , J ℓ } ℓ =1 ..k in ; 0) = 12 β X σ σ log X τ ,τ exp [ βσ P ℓ J ℓ τ ℓ ]2 cosh [ β ( P ℓ J ℓ τ ℓ )] exp β k in X ℓ =1 τ ℓ u ℓ . (96)From (95) and (96) it follows that the one-time approximation gives the correct resultsfor Glauber dynamics of the Ising model on fully symmetric and asymmetric graphs.Now we check how good the equations (92), (93) and (94) are for models on partiallyasymmetric graphs. We numerically solve the equations (92), (93) and (94) with a Monte Carlo integration.The unknown distributions W d , W sym and W real are represented as populations. Theprocedure is also known as population dynamics [7]. In figure 7 the magnetisation isplotted as a function of the temperature for an Ising model without bond disorder on aBethe lattice. The degree distribution is then: p ( k in , k out , k sym ) = δ ( k sym − C ) δ ( k in − D ) p ( k out ) . (97)Because there is no disorder the distributions W d ( u ) and W sym ( u + , u − ) are deltafunctions. Figure 7 tells us that the theory and the simulations are in good agreement.For C = 0 , C = 1 , ρ in the bonds. he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph T/(DJ) m C = 0C = 1C = 2C = 3 Figure 7.
The magnetisation m as a function of the rescaled temperature T / ( DJ )for a ferromagnet (i.e. ρ = 1) defined on a Bethe lattice. The indegree D of the graphequals 3 and each site is incident to C symmetric bonds. The simulations (markers)are mean values of 20 runs on graphs of sizes O (10 ). The theory (lines) follow fromsolving recursively the density evolution equations in the one-time approximation. T/(cJ) m ε = 1ε = 0.8ε = 0.5ε = 0.2ε = 0.1ε= 0 Figure 8.
The magnetisation m as a function of the temperature T / ( cJ ) for an Isingmodel without bond disorder on a Poissonian graph with mean connectivity c = 3 anddifferent fractions of symmetric edges ǫ . The lines are obtained by population dynamicsfrom the density evolution equations in the one-time approximation for populationsof sizes O (10 ). The markers are the average results from 20 runs with the heat-bathalgorithm on a graph instance of size O (10 ). he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph ρ m C = 3C = 2C = 1C = 0
Figure 9.
The magnetisation m as a function of the bias ρ in the couplings (seeequation (43)) at a temperature T / ( DJ ) = 0 .
5. The Ising model is defined on aBethe lattice with the degree distribution defined in (97). The indegree D equals 3and results are shown for various values of the symmetric degree C . The simulations(markers) are the mean values of 20 runs on graphs of sizes O (10 ). The theory (lines)are results from the density evolution equations in the one-time approximation usinga Monte Carlo calculation with populations of O (10 ) fields. We determine the P to F and the P to SG transition lines for Ising models with bimodaldistributions using a bifurcation analysis around the paramagnetic solution. First wenote that the equations (92) and (93) admit the solution: W d ( u ) = δ ( u ) , (98) W s ( u + , u − ) = Z dA W P ( A ) δ ( u + − A ) δ ( u − + A ) . (99)Indeed, when we insert (98) and (99) in (93) we get for W P ( A ): W P ( A ) = ∞ X k in ≥ k in X k sym =0 p ( k in , k sym ) k sym c sym Y <ℓ ≤ k sym − Z dA ℓ W P ( A ℓ ) δ (cid:2) A − A k sym − ,k in − (cid:0) { A ℓ ′ } ℓ ′ =1 ..k sym − (cid:1)(cid:3) (100)with A k sym − ,k in − the explicit solution of A = 12 β X θ = ± θ log X τ exp h θβJ ( P k in − ℓ =1 τ ℓ + 1) i cosh h βJ (cid:16)P k in − ℓ =1 τ ℓ + 1 (cid:17)i cosh " β k sym − X ℓ =1 τ ℓ A ℓ + βA . (101) he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph k in = k sym we have A k sym − ,k in − = J . For k sym = 0 we obtain A k sym − ,k in − = 12 β X θ = ± θ log X τ exp h θβJ ( P k in − ℓ =1 τ ℓ + 1) i cosh h βJ (cid:16)P k in − ℓ =1 τ ℓ + 1 (cid:17)i . (102)For partially asymmetric graphs W P ( A ) results in a more complicated form.To calculate the P to F and P to SG transitions, we expand the equations (92), (93)and (94) around the solution given by (98) and (99). We use an expansion in α ≪ Z W d ( u ) u n = m d n ∼ O ( α n ) , (103)and m s n ( σ ) = Z du + du − W ( u + , u − ) ( u ( σ ) − σA ) n ∼ O ( α n ) , (104) m snm = Z du + du − W ( u + , u − ) ( u (+) − A ) n ( u ( − ) + A ) m ∼ O ( α n + m ) . (105)Substitution of (103), (104) and (105) in (92), (93) and (94) gives, up to linear order in α , the equation m d1 m s1 ! = ρ M m d1 m s1 ! , (106)with M = M dd M ds M sd M ss ! . (107)The critical temperature T F of the P to F phase transition is given by the equation | λ | − ( T F ) = ρ with λ ( T ) the eigenvalue of M with the largest modulus. The elementsof the M matrix are M dd = ∞ X k out ≥ ∞ X k in ≥ k out , k in ) X k sym =0 p ( k in , k out , k sym )( k out − k sym ) c out − c sym (cid:0) k in − k sym (cid:1) , Z k sym Y ℓ =1 dA ℓ W ( A ℓ ) (cid:20) h τ k sym +1 i d − h τ i d (cid:21) , (108) M ds = ∞ X k out ≥ ∞ X k in ≥ k out , k in ) X k sym =0 p ( k in , k out , k sym )( k out − k sym ) c out − c sym k sym Z k sym Y ℓ =1 dA ℓ W ( A ℓ ) (cid:20) h τ i d − tanh ( βA ) h τ i d − h τ i d (cid:21) , (109) M ss = ∞ X k in ≥ k in X k sym =0 p ( k in , k sym ) k sym c sym ( k sym − he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph ρ T / ( J D ) C = 0C = 1C = 2C = 3 PSG F
Figure 10.
The phase transition lines between the P-phase, F-phase and SG-phase ona Bethe lattice with k in = 3 and k sym = C . The lines are calculated from the equationsfor the critical temperatures derived from a bifurcation analysis. The markers arecomputed with the population dynamics method on the density evolution equations inthe one-time approximation. Z k sym − Y ℓ =1 dA ℓ W ( A ℓ ) h τ i s − tanh ( βA ) h τ i s − h τ i s , (110) M sd = ∞ X k in ≥ k in X k sym =0 p ( k in , k sym ) k sym c sym ( k in − k sym ) Z k sym − Y ℓ =1 dA ℓ W ( A ℓ ) h τ k sym i s − h τ i s . (111)The averages h·i d and h·i s appearing in the matrix elements are given by h f ( τ , τ, A ) i d = X τ ,τ W k sym ,k in (cid:16) , τ, τ ; 0 , A (cid:16) { A ℓ } ,k sym (cid:17)(cid:17) f ( τ , τ, A ) , (112)and h f ( τ , τ, A ) i s = 2 − X τ ,τ,σ σ W k sym − ,k in − (cid:16) σ, τ, τ ; J , A (cid:16) { A ℓ } ,k sym − (cid:17)(cid:17) f ( τ , τ, A ) , (113)in the weight W k sym ,k in . The weight W k sym ,k in is expressed as W k sym ,k in ( σ, τ, τ ; θ, A ) ≡ exp » βσ „ J P k in ℓ =1 τ ℓ + θ «– h β “ J P k in ℓ =1 τ ℓ + θ ”i exp h βτ (cid:16)P k sym ℓ =1 A ℓ τ ℓ + sign ( θ ) A (cid:17)iP τ ,τ exp h βσ “ J P k in ℓ =1 τ ℓ + θ ”i h β “ J P k in ℓ =1 τ ℓ + θ ”i exp h βτ (cid:16)P k sym ℓ =1 A ℓ τ ℓ + sign ( θ ) A (cid:17)i . (114)For symmetric graphs the bifurcation condition | λ | − = ρ of (106) simplifies to he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph m s1 = ρM ss m s1 , which leads to the condition ρ tanh ( β F J ) ∞ X k in ≥ p ( k sym ) k sym ( k sym − c sym = 1 . (115)Equation (115) gives the P to F transition line on a symmetric graph [28]. For fullyasymmetric graphs we get m d1 = ρM dd m d1 : m d1 = ρm d1 ∞ X k in ≥ p ( k in )2 − k in X τ k in X ℓ =1 τ ℓ tanh β F J k in X ℓ =1 τ ℓ . (116)This is the same condition we found in (49).To determine the P to SG transition we expand up to order α with R duW d ( u ) u = 0and R du W s ( u + , u − )( u ( σ ) − σA ) = 0. We then arrive at m d2 m s2 ! = Q m d2 m s2 ! , Q = Q dd Q ds Q sd Q ss ! , (117)with Q dd = ∞ X k out ≥ ∞ X k in ≥ k out , k in ) X k sym =0 p ( k in , k out , k sym )( k out − k sym ) c out − c sym (cid:0) k in − k sym (cid:1)Z k sym Y ℓ =1 dA ℓ W ( A ℓ ) (cid:20) h τ k sym +1 i d − h τ i d (cid:21) , (118) Q ds = ∞ X k out ≥ ∞ X k in ≥ k out , k in ) X k sym =0 p ( k in , k out , k sym )( k out − k sym ) c out − c sym k sym Z k sym Y ℓ =1 dA ℓ W ( A ℓ ) (cid:20) h τ i d − tanh ( βA ) h τ i d − h τ i d (cid:21) , (119) Q ss = ∞ X k in ≥ k in X k sym =0 p ( k in , k sym ) k sym c sym ( k sym − Z k sym − Y ℓ =1 dA ℓ W ( A ℓ ) (cid:20) h τ i s − tanh ( βA ) h τ i s − h τ i s (cid:21) , (120) Q sd = ∞ X k in ≥ k in X k sym =0 p ( k in , k sym ) k sym c sym ( k in − k sym ) Z k sym − Y ℓ =1 dA ℓ W ( A ℓ ) (cid:20) h τ k sym i s − h τ i s (cid:21) . (121)Again we find the correct bifurcation conditiontanh ( β SG J ) ∞ X k in ≥ p ( k sym ) k sym ( k sym − c sym = 1 , (122)for symmetric lattices [28]. In figure 10 the phase transition lines between the P-phase,F-phase and SG-phase are shown for a bond-disordered Ising model with a bimodaldistribution. The P to F and P to SG lines are obtained using the bifurcation analysis he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph ε T / ( c J ) P to FP to SG Figure 11.
Critical temperatures computed from a bifurcation analysis (lines) withinthe one-time approximation are compared with simulations (markers) as a function ofthe fraction of symmetric edges ǫ . The results shown for the P to F transition are foran Ising model on a Poissonian graph without bond disorder. The results shown forthe P to SG transition are for an Ising model on a Poissonian graph for ρ = 0. while the F to SG transition is found through population dynamics. In figure 11 wepresent the P to F transition line for a ferromagnet ( ρ = 1) and the P to SG transitionline obtained through the bifurcation analysis on a Poissonian graph as a function of thefraction of symmetric edges ǫ . The markers from simulations are in agreement with thetheory. The simulations are performed through the method of Binder cumulants, see[19]. In Appendix C we give some details on how we derived these critical temperatures.We see that the SG-phase dissappears when the asymmetry in the graph increases.
10. Conclusion
In this paper we applied the cavity method to study the dynamics of spin models ona given graph instance. We derived a set of effective equations which describe thedynamics. Solving these recursive equations can be seen as the equivalent of the beliefpropagation algorithm known from inference problems or decoding algorithms. Justlike the latter, we expect these equations to be exact on a tree. The main differencewith statics is that path probabilities, instead of stationary probabilities of single spins,are propagated along the edges of the graph. We took the average over an ensembleof graphs to find the recursive equations describing the dynamics of Ising models ontypical graphs drawn from this ensemble. These equations generalize the result of[15] to graphs with arbitrary degree distributions. The macroscopic evolution of thesystem is given as a function of three mean values of path probabilities: the one of theprobabilities propagating along directed edges, the one of the probabilities propagating he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph
Appendix A. The Poissonian Ensemble
The Poissonian ensemble G p ( c, ǫ ) is defined through the following probability function P p ( C ; c, ǫ ) ≡ Y i
By using the tools of equilibrium statistical mechanics we determine the stationarybehaviour of a model evolving through Glauber dynamics determined by the equations(1) and (22). We consider the case where the field equals h i ( s ) = h i ( σ ( s − X j J ij c ij σ j ( s − , (B.1)with J ij = J ji and c ij = c ji . The stationary state is given by P st ( σ ) ∼ exp ( − βH ( σ )),with H ( σ ) equal to H ( σ ) = − β N X i =1 log (2 cosh ( βh i ( σ ))) . (B.2)This system has the same thermodynamic behaviour as a model with the two-spinHamiltonian H ( σ , τ ) = − X i,j J ij c ij σ i τ j . (B.3)Performing the standard replica method or the cavity method (see for example [30])we get the following set of selfconsistent equations for the distribution W of the cavityfields u , v and w , within the replica symmetric approximation: W c ( u, v, w ) = ∞ X k =0 p ( k ) kc Z k − Y ℓ =1 ( dJ ℓ R ( J ℓ )) Z k − Y ℓ =1 ( du ℓ dv ℓ dw ℓ W c ( u ℓ , v ℓ , w ℓ )) δ " u − k − X ℓ =1 U ( J ℓ , u ℓ , v ℓ , w ℓ ) δ " v − k − X ℓ =1 V ( J ℓ , u ℓ , v ℓ , w ℓ ) δ " w − k − X ℓ =1 W ( J ℓ , u ℓ , v ℓ , w ℓ ) , (B.4)with U ( J, u, v, w ) = 14 β X s,t s log X ˜ s, ˜ t exp (cid:2) β (cid:0) J s ˜ t + J ˜ st + u ˜ s + v ˜ t + w ˜ s ˜ t (cid:1)(cid:3) , (B.5) V ( J, u, v, w ) = 14 β X s,t t log X ˜ s, ˜ t exp (cid:2) β (cid:0) J s ˜ t + J ˜ st + u ˜ s + v ˜ t + w ˜ s ˜ t (cid:1)(cid:3) , (B.6) he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph W ( J, u, v, w ) = 14 β X s,t st log X ˜ s, ˜ t exp (cid:2) β (cid:0) J s ˜ t + J ˜ st + u ˜ s + v ˜ t + w ˜ s ˜ t (cid:1)(cid:3) . (B.7)The distribution W c determines the density of the fields ( u ( j ) i , v ( j ) i , w ( j ) i ) along the edgesof the graph. These fields correspond with the marginal probability distribution P ( j ) i onthe cavity graph G ( j ) : P ( j ) i ( σ, τ ) = exp h u ( j ) i σ + v ( j ) i τ + w ( j ) i στ iP σ,τ exp h u ( j ) i σ + v ( j ) i τ + w ( j ) i στ i . (B.8)On a given graph instance it is possible to calculate the fields ( u ( j ) i , v ( j ) i , w ( j ) i ) throughthe belief propagation algorithm. In the n -th time step of the algorithm, messages( u ( n ) i → j , v ( n ) i → j , w ( n ) i → j ) are propagated along the edges of the graph. The update equationsof these messages are given by u ( n +1) i → j = X ℓ ∈ ∂ i \ j U ( J ℓi , u ( n ) ℓ → i , v ( n ) ℓ → i , w ( n ) ℓ → i ) , (B.9) v ( n +1) i → j = X ℓ ∈ ∂ i \ j V ( J ℓi , u ( n ) ℓ → i , v ( n ) ℓ → i , w ( n ) ℓ → i ) , (B.10) w ( n +1) i → j = X ℓ ∈ ∂ i \ j W ( J ℓi , u ( n ) ℓ → i , v ( n ) ℓ → i , w ( n ) ℓ → i ) . (B.11)The fields ( u ( j ) i , v ( j ) i , w ( j ) i ) are given by the solutions of (B.9), (B.10) and (B.11) for n → ∞ . We call the equations (B.9), (B.10) and (B.11) the cavity or belief propagationequations. The density of the messages ( u ( n ) i → j , v ( n ) i → j , w ( n ) i → j ) can be found in equation(B.4). That is why (B.4) is also called the density evolution equation.When we choose W c ( u, v, w ) = δ ( w ) δ ( u − v ) W c ( u ) we find W c ( u ) = ∞ X k =0 p ( k ) kc Z k − Y ℓ =1 ( dJ ℓ R ( J ℓ )) Z k − Y ℓ =1 ( du ℓ W c ( u ℓ )) δ " u − β − k − X ℓ =1 atanh (tanh( βJ ℓ ) tanh( βu ℓ )) . (B.12)The distribution W r of the real fields u of the single site marginals P σ \ σ i P st ( σ ) ∼ exp [ u i σ ] are given by the selfconsistent equation W r ( u ) = ∞ X k =0 p ( k ) kc Z k Y ℓ =1 ( dJ ℓ R ( J ℓ )) Z k Y ℓ =1 ( du ℓ W c ( u ℓ )) δ " u − β − k X ℓ =1 atanh (tanh( βJ ℓ ) tanh( βu ℓ )) . (B.13)The solution W ( u, v, w ) = δ ( w ) δ ( u − v ) W c ( u ) corresponds with the solution w ( n +1) i → j = 0and u ( n +1) i → j = v ( n +1) i → j of the equations (B.9), (B.10) and (B.11). The cavity equations he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph u ( n +1) i → j = β − X ℓ ∈ ∂ i \ j atanh (cid:16) tanh ( βJ ℓi ) tanh (cid:16) βu ( n ) ℓ → i (cid:17)(cid:17) . (B.14)The single site marginals P i ( σ ) can be calculated through u i = β − lim n →∞ X ℓ ∈ ∂ i atanh (cid:16) tanh ( βJ ℓi ) tanh (cid:16) βu ( n ) ℓ → i (cid:17)(cid:17) . (B.15) Appendix C. Simulations
To determine the critical temperature from the P-phase to the F-phase of Ising modelson a graph of size N we calculate the Binder cumulant B N [19]: B N ( T ) ≡ − h m ih m i . (C.1)The brackets denote an average over the stationary distribution. The quantity m is themagnetisation P i σ i /N . In the F-phase B = 2 / B = 0. The point T F where the B N ( T ) lines for different values of N cross is the critical temperature. Infigure C1 we present B N as a function of T for a Poissonian graph with ǫ = 1 and c = 3.The value T F for finite system sizes is found to be higher then the theoretical result T F = ( c atanh(1 /c )) − valid for N → ∞ .The critical temperature for the P-SG transition is calculated through A N [31]: A N ( T ) ≡ (cid:18) − h q ih q i (cid:19) (C.2)The bar denotes the average over the quenched variables. Above the critical temperature T SG the lines for different system sizes join into one line. References [1] L. Correale, M. Leone, A. Pagnani, M. Weigt, and R. Zecchina, “The computational core and fixedpoint organization in boolean networks,”
J. Stat. Mech.: Theory Exp. , vol. 3, p. 03002, 2006.[2] A. C. C. Coolen,
The Mathematical Theory of Minority Games: Statistical Mechanics of InteractingAgents . Oxford University Press, USA, 2005.[3] M. Paczuski, K. E. Bassler, and A. Corral, “Self-organized networks of competing boolean agents,”
Phys. Rev. Lett. , vol. 84, p. 2185, 2000.[4] R. Vicente, D. Saad, and Y. Kabashima, “Finite-connectivity systems as error-correcting codes,”
Phys. Rev. E , vol. 60, p. 5352, 1999.[5] J. Kleinberg,
Cascading Behavior in Networks: Algorithmic and Economic Issues , ch. 24, pp. 613–633. Cambridge University Press, 2007.[6] L. Viana and A. J. Bray, “Phase diagrams for dilute spin glasses,”
J. Phys. C: Solid State Phys ,vol. 18, p. 3037, 1985.[7] M. M´ezard and G. Parisi, “The bethe lattice spin glass revisited,”
Eur. Phys. J. B , vol. 20, p. 217,2001.[8] F. R. Kschischang, B. J. Frey, and H. Loeliger, “Factor graphs and the sum-product algorithm,”
IEEE Transactions on Information Theory , vol. 47, pp. 498–519, 2001. he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph T B N = 1000N = 5000N = 10000N = 50000N = 100000
Figure C1.
The Binder cumulant B N for the P to F transition on a Poissonian graphwith ǫ = 1, c = 3 and different system sizes N . The vertical line indicates the criticalvalue T F = ( c atanh(1 /c )) − . We see that the Binder cumulant method overestimatesslightly the critical value for small system sizes.[9] F. Krzakala, A. Montanari, F. Ricc-Tersenghi, G. Semerjian, and L. Zdeborova, “Gibbs states andthe set of solutions of random constraint satisfaction problems,” Proc. Natl. Acad. Sci. , vol. 104,p. 10318, 2007.[10] T. Rogers, I. P. Castillo, R. K¨uhn, and K. Takeda, “Cavity approach to the spectral density ofsparse symmetric random matrices,”
Phys. Rev. E. , vol. 78, p. 031116, 2008.[11] M. Bayati, C. Borgs, A. Braunstein, J. Chayes, A. Ramezanpour, and R. Zecchina, “Statisticalmechanics of steiner trees,”
Phys. Rev. Lett. , vol. 101, p. 037208, 2008.[12] A. Crisanti and H. Sompolinsky, “Dynamics of spin systems with randomly asymmetric bonds:Ising spins and glauber dynamics,”
Phys. Rev. A , vol. 12, pp. 4865–4874, 1988.[13] C. D. Dominicis, “Dynamics as a substitute for replicas in systems with quenched randomimpurities,”
Phys. Rev. B , vol. 18, pp. 4913–4919, 1978.[14] K. Mimura and A. C. C. Coolen, “Parallel dynamics of disordered ising spin systems on finitelyconnected random graphs with arbitrary degree distributions.” in preparation.[15] J. P. L. Hatchett, I. P. Castillo, A. C. C. Coolen, and N. S. Skantzos, “Dynamical replica analysisof disordered ising spin systems on finitely connected random graph,”
Phys. Rev. Lett. , vol. 95,p. 117204, 2005.[16] K. Mimura and A. C. C. Coolen, “Generating functional analysi of ldgm channel coding withmany short loops.” to be published in Proceedings of the 2009 IEEE International Symposiumon Information Theory (ISIT2009, Seoul, Korea), 509-513.[17] A. Mozeika and A. C. C. Coolen, “Dynamical replica analysis of processes on finitely connectedrandom graphs i: vertex covering,”
J. Phys. A: Math. Theor. , vol. 41, p. 115003.[18] M. Kiemes and H. Horner, “Dynamics of an ising spin glass on the bethe lattice,”
J. Phys. A:Math. Gen. , vol. 41, p. 324017, 2008.[19] K. Binder and D. W. Heermann,
Monte Carlo Simulations in Statistical Physics . Springer, 2002.[20] J. P. L. Hatchett, B. Wemmenhove, I. P. Castillo, T. Nikoletopoulos, N. S. Skantzos, and A. C. C.Coolen, “Parallel dynamics of disordered ising spin systems on finitely connected randomgraphs,”
J Phys A: Math Gen , vol. 37, p. 6201, 2004.[21] B. Derrida, “Dynamical phase transition in non-symmetric spin glasses,”
J Phys A: Math Gen , he Cavity Approach to Parallel Dynamics of Ising Spins on a Graph vol. 20, pp. L721–L725, 1987.[22] B. Derrida, E. Gardner, and A. Zippelius, “An exactly solvable asymmetric neural network model,” Europhysics Letters , vol. 4, pp. 167–173, 1987.[23] H. Eissfeller and M. Opper, “New method for studying the dynamics of disordered spin systemswithout finite-size effects,”
Phys. Rev. Lett. , vol. 68, p. 2094, 1992.[24] V. M. Egu´ıluz, D. R. Chialvo, G. A. Cecchi, M. Baliki, and A. V. Apkarian, “Scale-free brainfunctional networks,”
Phys. Rev. Lett. , vol. 94, p. 018102, 2005.[25] I. P. Castillo, B. Wemmenhove, J. P. L. Hatchett, A. C. C. Coolen, N. S. Skantzos, andT. Nikoletopoulos, “Analytic solution of attractor neural networks on scale-free graphs,”
J.Phys. A: Math. Gen. , vol. 37, p. 8789, 2004.[26] N. Schwartz, R. Cohen, D. ben Avraham, A. L. Barab´asi, and S. Havlin, “Percolation in directedscale-free networks,”
Phys. Rev. E , p. 015104, 2002.[27] J. M. Borwein, D. M. Bradley, and R. E. Crandall, “Computational strategies for the riemann zetafunction,”
J. Comput. Appl. Math. , vol. 121, pp. 247–296.[28] A. K. Hartmann and M. Weigt,
Phase Transitions in Combinatorial Optimization Problems:Basics, Algorithms and Statistical Mechanics , ch. 5 Statistical mechanics of the Ising model.Wiley-VCH, 2005.[29] S. Morita, K. Oshio, Y. Osana, Y. Funabashi, K. Oka, and K. Kawamura, “Geometrical structureof the neuronal network of caenorhabditis elegans,”
Physica A , vol. 298, p. 553, 2001.[30] I. P. Castillo and N. S. Skantzos, “The little-hopfield model on a random graph,”
J Phys A , vol. 37,p. 9087, 2004.[31] R. N. Bhatt and A. P. Young, “Numerical studies of ising spin glasses in two, three, and fourdimensions,”