Effects of hidden nodes on the reconstruction of bidirectional networks
aa r X i v : . [ phy s i c s . d a t a - a n ] J a n Effects of hidden nodes on the reconstruction of bidirectional networks
Emily S.C. Ching ∗ and P.H. Tam Department of Physics, The Chinese University of Hong Kong, Shatin, Hong Kong (Dated: January 15, 2019)Much research effort has been devoted to developing methods for reconstructing the links ofa network from dynamics of its nodes. Many current methods require the measurements of thedynamics of all the nodes be known. In real-world problems, it is common that either some nodesof a network of interest are unknown or the measurements of some nodes are unavailable. Thesenodes, either unknown or whose measurements are unavailable, are called hidden nodes. In thispaper, we derive analytical results that explain the effects of hidden nodes on the reconstructionof bidirectional networks. These theoretical results and their implications are verified by numericalstudies.
PACS numbers: 89.75.Hc, 05.45.Tp, 05.45.Xt
I. INTRODUCTION
Many systems of interest in physics and biology arerepresented by complex networks of a large number ofelementary units or nodes that interact or link witheach other [1]. A substantial amount of data have beenobtained for various networks especially biological net-works, and a grand challenge is to reveal the structureof these networks, namely the links, their direction andrelative coupling strength, from the measured data. It isexpected that [2] the structure of a network controls itsdynamics and thus one might be able to uncover infor-mation about the structure of a network from its dynam-ics. Much research effort has been devoted to developingmethods for reconstructing a network from the dynamicsof the nodes (see e.g. [3, 4] for review). Counterintu-itively, it has been demonstrated that the presence ofnoise acting on the network can be beneficial for networkreconstruction as the noise induces a relation betweenmeasurable quantities from dynamics and the networkstructure [5]. Making use of different relations of thiskind, a number of methods [6–12] have been proposedfor reconstructing networks solely from the dynamics ofthe nodes. In all these methods, in order to calculatethe quantities that are related to the network structure,the measurements of the dynamics of all the nodes arerequired.In real-world problems, it is common that either somenodes of a network of interest are unknown or the mea-surements of some nodes are unavailable. These nodes,either unknown or whose measurements are unavailable,are called hidden nodes. It is thus important to study andunderstand the effects of hidden nodes on the reconstruc-tion of networks [13–19]. This task is highly challengingand, as of today, there is not yet a general and analyticalunderstanding of the effects of hidden nodes.A usual practice infers links from the correlation ofthe measurements, with a larger correlation coefficient ∗ [email protected] interpreted as a higher probability of link [20–22]. How-ever, correlation between measurements of two nodescannot be equated with direct interactions between thetwo nodes. In fact, it has been clearly shown that fornetworks of neurons, the spiking dynamics of neuronscan have weak pairwise correlations even though theyare strongly coupled [23]. This study further shows thatthe spiking dynamics of neurons are quantitatively cap-tured by the probability distribution P Ising (ˆ σ , . . . , ˆ σ N ) ∝ exp[ P Ni We consider weighted bidirectional networks of N nodes with nonlinear dynamics and diffusive-like cou-pling. The dynamics of each node is described by a vari-able x i ( t ), i = 1 , , . . . , N , and the time evolution of x i ( t )is given by dx i dt = f ( x i ) + X j = i g ij A ij h ( x j − x i ) + η i . (2)The adjacency matrix element A ij is 1 when node j islinked to node i by the diffusive-like coupling function h with coupling strength g ij ; otherwise A ij = g ij = 0.The coupling is bidirectional so A ij = A ji and g ij = g ji ,and the graph of the networks has no self-loops, that is, A ii ≡ 0. As discussed [7, 12], f describes the intrinsicdynamics that is generally nonlinear and identical for allthe nodes, and the diffusive-like coupling function h sat-isfies h ( − z ) = − h ( z ) and h ′ (0) > 0. Thus excitatoryor activating links have with g ij > g ij < 0. Here we take h ′ (0) = 1. Externalinfluences are modelled by a Gaussian white noise η withzero mean and variance σ n : η i ( t ) η j ( t ′ ) = σ n δ ij δ ( t − t ′ ) , (3)where the overbar is an ensemble average over differentrealizations of the noise.The weighted Laplacian matrix of the network, L , isgiven by L ij = s i δ ij − g ij A ij , s i ≡ N X k =1 g ik A ik (4)and contains connectivity information of the network.Here s i is the weighted degree or the strength of node i .For these networks, x i ( t )’s approach X with f ′ ( X ) < δx i ( t ) = x i ( t ) − X , thenthe linearized system around the noise-free steady stateis given by ddt δx i = − X j ( L ij + aδ ij ) δx j + η i (5)where a ≡ − f ′ ( X ) > 0. We consider systems that havestationary dynamics and this implies L + a I is positivedefinite [26]. Using Eq. (5), it has been derived [12] thatthe covariance matrix Σ , defined by Σ ≡ lim t →∞ [ x ( t ) − x ( t )] [ x ( t ) − x ( t )] T (6) is related to L by Σ − = 2 σ n ( L + a I N ) (7)where I N is the N × N identity matrix. Equation (7) is anexact result for the linearized system [Eq. (5)] and a goodapproximation for the original nonlinear network [gov-erned by Eq. (2)] when the noise is weak. All the theoret-ical results presented in this paper should be understoodin this manner. An important consequence of Eq. (7) is Σ − ij = − σ n g ij A ij , i = j (8)which indicates that the off-diagonal elements of Σ − ij would separate into two groups according to A ij = 0or 1. Making use of this result, a reconstruction methodof the adjacency matrix and thus the links of the networkby performing clustering analysis of the off-diagonal ele-ments of Σ − has been developed [12]. In this method, Σ is evaluated by approximating the ensemble averageby time average:Σ ij ≈ h [ x i ( t ) − h x i ( t ) i ][ x j ( t ) − h x j ( t ) i ] i (9)where h· · · i denotes a time average. To evaluate Σ − , themeasurements of x ( t ) from all the N nodes are required.We study the problem of network reconstruction whenthere are n h hidden nodes and only measurements from n = N − n h < N nodes are available. We call these n nodes the measured nodes and, for clarity, denote theirmeasured dynamics by y i ( t ), i = 1 , , . . . , n and the cor-responding covariance matrix of the measured data by Σ m : Σ m ≡ lim t →∞ [ y ( t ) − y ( t )] [ y ( t ) − y ( t )] T (10)where y ( t ) ≡ ( y , y , · · · , y n ) T . Similarly,(Σ m ) ij ≈ h [ y i ( t ) − h y i ( t ) i ][ y j ( t ) − h y j ( t ) i ] i (11)We would like to answer the following questions: Howwould the hidden nodes affect the reconstruction resultsbased on Σ − m ? Whether and when the links among the n measured nodes can be reconstructed from the measured y i ( t )’s? III. THEORETICAL RELATION FOR Σ − m Without loss of generality, we let y i ( t ) = x i ( t ), i =1 , , . . . , n . Then we partition Σ into four block matrices Σ = (cid:18) Σ m UU T Σ h (cid:19) (12)where the n × n block matrix is Σ m , the n h × n h blockmatrix Σ h is the covariance matrix of the hidden nodesgiven by(Σ h ) µν = lim t →∞ [ x µ + n ( t ) − x µ + n ( t )] [ x ν + n ( t ) − x ν + n ( t )](13)and the n × n h block matrix U measures the covariancebetween the measured and hidden nodes with U iµ = lim t →∞ [ y i ( t ) − y i ( t )] [ x µ + n ( t ) − x µ + n ( t )] (14)For clarity, we use Roman subscripts i, j, . . . for the mea-sured nodes and Greek subscripts µ, ν, . . . for the hiddennodes. We partition the weighted Laplacian matrix in asimilar fashion: L = (cid:18) L m EE T L h (cid:19) (15)The n × n block matrix L m and the n h × n h block matrix L h , with elements ( L m ) ij = L ij and ( L h ) µν = L µ + n,ν + n ,contain information of the connectivity among the mea-sured nodes and among the hidden nodes respectivelywhile the n × n h block matrix E contains information ofthe connectivity between the measured and hidden nodeswith elements E iµ = − g i,µ + n A i,µ + n (16)Using Eq. (7), we obtain I n = 2 σ n (cid:2) Σ m ( L m + a I n ) + U E T (cid:3) (17) = Σ m E + U ( L h + a I n h ) (18)which imply Σ − m = 2 σ n h L m + a I n − E ( L h + a I n h ) − E T i (19)As L + a I N is positive definite, L h + a I n h is also positivedefinite and is thus invertible. We define C ≡ E ( L h + a I n h ) − E T (20)then the off-diagonal elements of Eq. (19) can be writtenas (cid:0) Σ − m (cid:1) ij = − σ n ( g ij A ij + C ij ) , i = j (21)which shows that the hidden nodes affect the reconstruc-tion results based on Σ − m by introducing correctionsgiven by C . This can be seen directly by using Eq. (7)to rewrite Eq. (19) as (cid:0) Σ − m (cid:1) ij = Σ − ij − σ n C ij i, j = 1 , . . . , n (22)Equations (19) and (22) are our major theoretical resultsand we shall use them to answer the questions of interest. IV. CORRECTIONS DUE TO HIDDEN NODES From Eq. (20) and using Eq. (16), we immediately seethat C ij = 0 when g i,µ + n = 0 or g j,µ + n = 0 for all µ = 1 , , . . . , n h , that is when at least one of the measurednodes i and j is not connected to any hidden node. Welet M ≡ L h + a I n h ≡ S − W where S and W are definedby S µν = ( s µ + n + a ) δ µν (23) W µν = g µ + n,ν + n A µ + n,ν + n (24)for µ, ν = 1 , , . . . , n h . S is a diagonal matrix with thediagonal elements related to the strength of the hiddennodes and W is the weighted adjacency matrix of thehidden nodes. Then we obtain (see Appendix) C ij = n h X µ =1 F W iµ W µ j + n h X µ ,µ =1 F W iµ W µ µ W µ j + n h − X k =3 n h X µ , ··· ,µ k +1 =1 F k W iµ k Y j =1 W µ j µ j +1 W µ k +1 j (25)where F k generally depends on s µ l + n + a , l = 2 , . . . , k and the eigenvalues of M . We have also simi-larly defined W iµ ≡ g i,µ + n A i,µ + n for i = 1 , , . . . , n , µ = 1 , , . . . , n h and W µi = W iµ . The product W iµ W µ µ · · · W µ m − µ m W µ m j is nonzero only if there isa path connecting the measured node i and the measurednode j via hidden nodes µ , µ , . . . , µ m − , µ m (see Fig. 1).Thus C ij would be zero if there does not exist any pathconnecting the measured nodes i and j via hidden nodesonly. In general, we expect C ij to be nonzero for somepairs of measured nodes i and j . FIG. 1: A path connecting the measured nodes i and j (opencircles) via the hidden nodes µ , µ , . . . , µ m − , µ m (closed cir-cles). As indicated by Eq. (8), the distribution of the off-diagonal elements of Σ − ij for i, j = 1 , , . . . , n can bewritten as P (Σ − ij = x ) = (1 − r ) P ( x ) + rP ( x ) (26)where r is the fraction of connected pairs among the mea-sured nodes and is equal to the number of connected pairsof measured nodes divided by n ( n − / P and P arethe distributions of Σ − ij with A ij = 0 and A ij = 1 re-spectively. If the positive and negative coupling strengthof the links are described by two different distributions, P can further be a mixture of two distributions P and P − , which correspond to g ij > g ij < P ( x ) approaches δ ( x ). For a finite number ofdata points, numerical studies [12] show that P ( x ) iswell-approximated by a Gaussian distribution of mean m = 0 and standard deviation σ , and σ decreaseswhen the number of data points increases. Equation (8)implies that P ( x ) would have a mean m and standarddeviation σ given by m = − σ n h g i ; σ = 2 σ n σ g (27)where h g i and σ g are the average and standard deviationof the coupling strength g ij of the links. In the pres-ence of hidden nodes, the corrections C ij will modify thedistributions P and P to ˜ P and ˜ P : P ((Σ − m ) ij = x ) = (1 − r ) ˜ P ( x ) + r ˜ P ( x ) , i = j (28)The mean and standard deviation of ˜ P i , denoted by ˜ m i ,˜ σ i , i = 0 and 1, would be modified. Using Eq. (21) andthe results for P ( x ), we obtain˜ m = − σ n µ C, (29)˜ σ ≈ σ + 4 σ n σ C, (30)˜ m = m − σ n µ C, (31)˜ σ = σ + 4 σ n (cid:2) σ C, + K ( g ij , C ij ) (cid:3) (32)where µ C, ≡ h C ij | A ij = 0 i (33) µ C, ≡ h C ij | A ij = 1 i (34) σ C, ≡ h C ij | A ij = 0 i − h C ij | A ij = 0 i (35) σ C, ≡ h C ij | A ij = 1 i − h C ij | A ij = 1 i (36) K ( g ij , C ij ) ≡ h g ij C ij | A ij = 1 i − h g ih C ij | A ij = 1 i (37)Here µ C,i and σ C,i for i = 1 , C ij for A ij = 0 and A ij = 1 respectively, and K ( g ij , C ij ) measures the corre-lation of C ij with the coupling strength g ij for measurednodes i and j that are connected. Hence, the correc-tions C ij would shift the means, broaden and distort thedistributions of P and P to ˜ P and ˜ P .Suppose P and P are distinguishable. If ˜ P and ˜ P remain distinguishable even though with a larger extentof overlap, then the links among the measured nodes canstill be reconstructed amid with a larger error rate. Fora mixture of two general distributions, there is no simplecriterion on when the component distributions are distin-guishable. Nonetheless, the component distributions arelikely to be distinguishable when the absolute value ofthe difference between their means are larger than a cer-tain multiple of the sum of their standard deviations. Let m − m = γ ( σ + σ ) with | γ | > 1. Using Eqs. (29)-(32), we obtain˜ m − ˜ m = γ (cid:26) (˜ σ − σ n σ C, ) / + [˜ σ − σ n ( σ C, + K )] / (cid:27) − σ n ( µ C, − µ C, ) (38)Thus ˜ P and ˜ P are likely to remain distinguishable (with | ˜ m − ˜ m | / (˜ σ + ˜ σ ) > 1) if | µ C, − µ C, | , σ C, , σ C, and | K ( g ij , C ij ) | are sufficiently small.To shed further light on this, we use Eq. (20) to obtaina crude estimate of C ij . Substitute L h + a I n h = S − W = S ( I n h − S − W ) into Eq. (20), we have C = E ( I n h − S − W ) − S − E T (39)If the Neumann series P ∞ k =0 ( S − W ) k converges, it con-verges to ( I n h − S − W ) − . The necessary and suffi-cient condition for the Neumann series to converge isthe spectral radius of S − W , denoted by ρ ( S − W ),is less than 1 [27]. For networks with g ij ≥ 0, onecan easily show that the infinity norm || S − W || ∞ ≡ max µ { P ν | ( S − W ) µν |} < 1, and thus ρ ( S − W ) ≤|| S − W || ∞ < 1. For networks with both positiveand negative g ij , we have checked numerically that ρ ( S − W ) < I n h − S − W ) − ∼ I n h + S − W , to obtain acrude estimate of C ij : C ij ∼ N X µ = n +1 g iµ g jµ A iµ A jµ s µ + a + N X µ,ν = n +1 g iµ g µν g jν A iµ A µν A jν ( s µ + a )( s ν + a ) (40)Using Eq. (40), one sees that the magnitude of C ij de-pends on three factors: (1) the number of paths connect-ing the measured nodes i and j via the hidden nodeswhich determines the number of nonzero terms in thesums, (2) the strength of the hidden nodes and (3) thecoupling strength of the links in these paths. Regard-ing to the second factor, we note that hidden nodes withlarger strength actually give rise to smaller corrections incontrary to what one might have guessed. If these fac-tors do not differ much between the two groups of uncon-nected or connected measured nodes, then µ C, ∼ µ C, ;if these factors do not vary much among the measurednodes in each group, then σ C, and σ C, would be smalland if these factors do not correlate with the magnitudeof g ij for connected measured nodes, then | K ( g ij , C ij ) | would be small even when the magnitudes of the correc-tions C ij ’s themselves might be large. Such situationsare expected when the the hidden nodes are not prefer-entially linked to the measured nodes in any manner. Inthis case, ˜ P and ˜ P remain distinguishable and it is pos-sible to reconstruct the links among the measured nodesfrom (Σ − m ) ij , i = j . V. NUMERICAL RESULTS AND DISCUSSIONS We check our theoretical results using data from nu-merical simulations. We study five different networks,four of N = 100 each and one of N = 1000.(1) Network A: it consists of two random networks,each of 50 nodes and a connection probability of0.2, connected to each other by one link and g ij ’s,taken from a Gaussian distribution N (10 , ) ofmean 10 and standard deviation 2, are all posi-tive. We take all the 50 nodes of one of the randomnetwork as hidden nodes.(2) Network B: it is a random network of connectionprobability 0.2 and g ij ’s also taken from N (10 , )and are all positive. We choose the hidden nodesrandomly from the network with n h ≤ 70 such thatthe number of links among the measured nodes isat least of the order of 100.(3) Network C: it is similar to network B except that g ij of 80% of the links taken from N (10 , ) andthe remaining 20% taken from N ( − , ). As aresult, about 80% of the g ij ’s are positive and about20% are negative. The hidden nodes are chosenrandomly from the network.(4) Network D: it is a scale-free network of N =1000 [28] with degree distribution obeying a powerlaw and g ij ’s taken from N (10 , ) are all positive.The hidden nodes are chosen randomly from thenetwork.(5) Network E: it is constructed by linking 30 addi-tional nodes to a random network of 70 nodes andconnection probability 0.2 with the restriction thatevery one of the additional nodes is only com-monly connected to randomly selected pairs of un-connected nodes in the random network; and theadditional nodes are randomly connected amongthemselves with the same connection probability0.2. g ij ’s are taken from N (10 , ) and are all pos-itive. We take the 30 additional nodes as hiddennodes.For the dynamics, we mainly study nonlinear logisticfunction f ( x ) = 10 x (1 − x ) (41)and diffusive coupling function h ( y − x ) = y − x (42)and take σ n = 1 for the noise. To explore how generalour theoretical results are, we go beyond the descrip-tion by Eq. (2) and study two additional cases. In thefirst additional case, the nodes of network B have two-dimensional state variables ( x i ( t ) , y i ( t )) with nonlinear FitzHugh-Nagumo (FHN) dynamics [29]˙ x i = ( x i − x i / − y i ) /ǫ + X j = i g ij A ij ( x j − x i ) + η i (43)˙ y i = x i + α (44)where ǫ = 0 . 01 and α = 0 . 95. In the second additionalcase, the nodes of network B have three-dimensional statevariables ( x i ( t ) , y i ( t ) , z i ( t )) with nonlinear R¨ossler dy-namics [30] and nonlinear coupling [7]:˙ x i = − y i − z i + X j = i g ij A ij tanh( x j − x i ) + η i (45)˙ y i = x i + c y i + X j = i g ij A ij tanh( y j − y i ) (46)˙ z i = c + z i ( x i − c ) + X j = i g ij A ij tanh( z j − z i ) (47)where c = c = 0 . c = 9. In these two additionalcases, the system does not approach a steady state inthe absence of noise, and has chaotic dynamics when thenodes are decoupled in the second case with R¨ossler dy-namics. We integrate the equations of motion using theEuler-Maruyama method and record the time series x i ( t )with a sampling interval δt = 5 × − . For all the casesstudied, including the cases with FHN and R¨ossler dy-namics, we calculate Σ using x i ( t )’s with a time averageover N data = 2 × data points.For network A, since there is only one link connect-ing the hidden nodes and the measured nodes, there isno path connecting any pair of measured nodes via thehidden nodes thus C ij = 0 for all i = j . As a re-sult, Eq. (22) implies that (Σ − m ) ij = Σ − ij for i = j .We show the distributions of P (Σ − ij ) and P ((Σ − m ) ij )for i = j = 1 , , . . . , n = 50 in Fig. 2. As expected,the two distributions coincide with each other. More-over, P (Σ − ij ) is bimodal with the peak around zero cor-responding to P for unconnected nodes and the peakaround x m ≈ − 20 corresponding to P for connectednodes in accord with Eq. (26). Furthermore, the value of x m is in excellent agreement with the theoretical valueof µ = − h g i /σ n [see Eq. (27)]. Hence in this case, thelinks among the measured nodes can be reconstructedfrom (Σ − m ) ij with i = j , which can be calculated usingthe dynamics y i ( t ) of the measured nodes only.For network B with the hidden nodes randomly cho-sen, there are nonzero C ij ’s for some pairs of measurednodes i and j . We first consider the case with logisticdynamics and calculate C ij using Eq. (20) and togetherwith Σ − ij , we obtain the theoretical results for (Σ − m ) ij using Eq. (22). We compare these theoretical results with(Σ − m ) ij directly calculated from the measured dynamics y i ( t )’s in Fig. 3 and perfect agreement is found for all thevalues of n h studied. For FHN and R¨ossler dynamics,the system is not described by Eq. (2) thus a = − f ′ ( X )is not defined. We put a = 0 in Eq. (20) and obtainthe theoretical estimate for the off-diagonal (Σ − m ) ij as -30 -20 -10 0 x -3 -2 -1 P ( X = x ) FIG. 2: Comparison of the distributions P ( X = x ) of X = Σ − ij (circles) and X = (Σ − m ) ij (triangles) for i = j =1 , , . . . , n = 50 for network A. Σ − ij − (2 /σ n )( EL − h E T ) ij . Interestingly, these theoreti-cal estimates are in good agreement with the directly cal-culated (Σ − m ) ij ’s in most cases, as shown in Figs. 4 and 5.For FHN dynamics with larger n h , an improved theoreti-cal estimate is obtained by Σ − ij − (2 /σ n )[( EL − h E T ) ij + b ],where b is a constant. This indicates the general appli-cability of our theoretical results beyond the model classstudied. -30 -20 -10 0 x -3 -2 -1 P ( X = x ) (a) -30 -20 -10 0 x -3 -2 -1 P ( X = x ) (b) -40 -30 -20 -10 0 x -3 -2 -1 P ( X = x ) (c) -40 -30 -20 -10 0 x -3 -2 -1 P ( X = x ) (d) FIG. 3: Comparison of distributions P ( X = x ) of X =Σ − ij (circles) and X = (Σ − m ) ij (triangles) for network B withlogistic dynamics and different number of hidden nodes: (a) n h = 10, (b) n h = 30, (c) n h = 50, and (d) n h = 70. Thedashed lines are the theoretical results of X = Σ − ij − C ij /σ n with C ij calculated using Eq. (20). As shown in Figs. 3-5, the distribution P ((Σ − m ) ij ) isa mixture of the modified distributions ˜ P and ˜ P , inaccord with Eq. (28), which remain distinguishable asexpected since the hidden nodes are chosen randomly.Thus it is possible to reconstruct the links among themeasured nodes from (Σ − m ) ij with i = j . We note thatthis is true for all the three kinds of dynamics studiedand even when the hidden nodes outnumber the mea-sured nodes. In Table I, we compare the error rates ofthe reconstruction results obtained using k-means clus- -40 -30 -20 -10 0 x -3 -2 -1 P ( X = x ) (a) -40 -30 -20 -10 0 x -3 -2 -1 P ( X = x ) (b) -50 -40 -30 -20 -10 0 x -3 -2 -1 P ( X = x ) (c) -60 -50 -40 -30 -20 -10 0 x -3 -2 -1 P ( X = x ) (d) FIG. 4: Same as Fig. 3 for network B with FHN dy-namics but the dashed lines are now the theoretical esti-mates of X = Σ − ij − (2 /σ n )( EL − h E T ) ij . The dot-dashedlines are the improved theoretical estimates of X = Σ − ij − (2 /σ n )[( EL − h E T ) ij + b ] with b = 1 for (c) and b = 3 for (d). -30 -20 -10 0 x -3 -2 -1 P ( X = x ) (a) -30 -20 -10 0 x -3 -2 -1 P ( X = x ) (b) -40 -30 -20 -10 0 x -3 -2 -1 P ( X = x ) (c) -40 -30 -20 -10 0 x -3 -2 -1 P ( X = x ) (d) FIG. 5: Same as Fig. 4 for network B with R¨ossler dynamics. tering of (Σ − m ) ij from the measured dynamics only andof Σ − ij from the dynamics of the whole network. Wemeasure the error rates by the ratios of false negatives(FN) and false positives (FP) over the number of actuallinks N L among the measured nodes. These error ratesare related to the sensitivity and specificity usually usedfor a predictive test: sensitivity is given by 1 − FN /N L and specificity is given by 1 − (FP /N L ) ρ/ (1 − ρ ), where ρ = N L / [ n ( n − / 2] is the link density of the measurednodes. For networks with low link density ρ , the errorrates can be rather high even when specificity is close to1 so the error rates are better measures of the accuracyof the reconstruction results [9]. As can be seen, the ac-curacy of the reconstruction results using (Σ − m ) ij fromthe measured nodes only is comparable to that obtainedusing (Σ − ) ij from all the nodes. network dynamics n h ρ FN/ N L (%) FP/ N L (%)A logistic 50 0.202 0.81 (0.81) 0.00 (0.00)B logistic 10 0.198 0 . 76 (0 . 88) 0 . 00 (0 . . 05 (1 . 26) 0 . 00 (0 . . 62 (1 . 62) 0 . 00 (0 . . 05 (3 . 16) 0 . 00 (0 . . 63 (0 . 51) 0 . 00 (0 . . 05 (0 . 84) 0 . 00 (0 . . 62 (1 . 21) 0 . 00 (0 . . 16 (1 . 05) 1 . 05 (0 . . 01 (0 . 88) 0 . 00 (0 . . 10 (0 . 44) 0 . 00 (0 . . 87 (1 . 31) 0 . 00 (0 . . 57 (0 . 00) 5 . 95 (0 . . 96 (0 . 80) 0 . 00 (0 . . 85 (0 . 21) 0 . 00 (0 . . 85 (0 . 00) 1 . 14 (0 . . 42 (0 . 42) 29 . 88 (0 . − m ) ij for the various networks studied. N = 100 for networks A, B, C and E and N = 1000 fornetwork D. ρ = N L / [ n ( n − / N L is the numberof links among the measured nodes and n = N − n h is thenumber of measured nodes. Two clusters are used for allnetworks except network C where three clusters are used. Weshow also the error rates using Σ − ij in parentheses. For network C, the positive and negative g ij ’s followtwo different distributions so P can be further decom-posed into a weighted sum of P and P − , which cor-respond to g ij > g ij < P (Σ − ij = x )= (1 − r ) P ( x ) + r [ βP ( x ) + (1 − β ) P − ( x )] (48)where β is the fraction of the links among the mea-sured nodes with positive g ij ’s. Similarly, Eq. (28) isalso rewritten as P ((Σ − m ) ij = x )= (1 − r ) ˜ P ( x ) + r [ β ˜ P ( x ) + (1 − β ) ˜ P − ( x )] (49)As g ij can now be either positive or negative, the termsin the summations contributing to C ij [see Eq. (40)]could cancel one another. Thus we expect the mag-nitudes of µ C, , µ C, ≡ h C ij | A ij = 1 , g ij > i and µ C, − ≡ h C ij | A ij = 1 , g ij < i to be smaller than themagnitudes of µ C, and µ C, for network B with logisticdynamics. It can indeed be clearly seen that the shiftsof ˜ P , ˜ P and ˜ P − from P , P and P − in Fig. 6 aresmaller than the shifts of ˜ P and ˜ P from P and P inFig. 3. Moreover, ˜ P , ˜ P and ˜ P − are again only slightlybroadened as compared with P , P and P − becausethe hidden nodes are randomly chosen. Thus for net-work C, the links among the measured nodes can also beaccurately reconstructed from clustering of (Σ − m ) ij with i = j (see Table I). -40 -20 0 20 x -4 -3 -2 -1 P ( X = x ) (a) -40 -20 0 20 x -4 -3 -2 -1 P ( X = x ) (b) FIG. 6: Similar to Fig. 3 for network C with both positiveand negative g ij ’s for two different numbers of hidden nodes:(a) n h = 20 and (b) n h = 40. For the scale-free network D, as the link density ρ isvery small, most of the measured nodes are not linkedvia a path of hidden nodes and thus most C ij ’s vanish.But as the nodes have a power-law degree distribution,both the strength of the hidden nodes and the number ofpaths connecting measured nodes via hidden nodes couldhave a large variation leading to a large variation in themagnitude of C ij ’s. This implies a large σ C, /µ C, and σ C, /µ C, as compared to the case of random networkB. In particular, this results in a larger distortion from P to ˜ P as seen in Fig. 7. The effect is more evidentfor P because there are far more unconnected measurednodes than connected measured nodes due to the small ρ . Nonetheless, ˜ P and ˜ P remain distinguishable andthe error rates of the reconstruction of the links amongthe measured nodes remain low (see Table I). -30 -20 -10 0 x -6 -5 -4 -3 -2 -1 P ( X = x ) (a) -30 -25 -20 -15 -10 -5 0 5 x -5 -4 -3 -2 -1 P ( X = x ) (b) FIG. 7: Similar to Fig. 3 for scale-free network D with (a) n h = 100 and (b) n h = 700. In network E, every one of the n h = 30 hidden nodesis only commonly linked to randomly selected pairs ofunconnected measured nodes. We first randomly choosea pair of unconnected measured nodes and link all the n h hidden nodes to both of them. Then we link thehidden nodes to a second pair of unconnected measurednodes with the restriction that no hidden nodes are com-monly linked to connected measured nodes that mightexist among the first and second pairs of unconnectednodes. Therefore, the number of hidden nodes commonlylinked to the second pair can be less than n h . We repeatthe process for all the remaining pairs of unconnectedmeasured nodes. In this way, the number of hidden nodescommonly linked to a given pair of measured nodes i and j or the number of nonzero terms in the first sumin Eq. (40) is identically zero for i and j that are con-nected, and varies among i and j that are unconnected.This preferential connection of the hidden nodes to un-connected measured nodes gives rise to µ C, > µ C, and σ C, > σ C, . Thus the distortion of P is large lead-ing to a larger overlap of ˜ P and ˜ P as shown in Fig. 8.As expected, the error rates of the reconstruction resultsare larger with FP/ N L ≈ 30% (see Table I). However,we note that even with this error rate, the specificity isabove 90%. Moreover, the error rate FN/ N L remainsless than 1% and thus more than 99% of the actual linksamong the measured nodes are correctly reconstructed. -30 -20 -10 0 x -4 -3 -2 -1 P ( X = x ) FIG. 8: Comparison of distributions P ( X = x ) of X =Σ − ij (circles) and X = (Σ − m ) ij (triangles) for network E with n h = 30 hidden nodes that are preferentially linked to mea-sured nodes that are unconnected. The dashed line is thetheoretical result of X = Σ − ij − C ij /σ n with C ij calculatedusing Eq. (20). VI. CONCLUSIONS We have addressed the interesting question of how hid-den nodes affect reconstruction of bidirectional networks.By using a model class of bidirectional networks withnonlinear dynamics and diffusive-like coupling and sub-jected to a Gaussian white noise, as described by Eq. (2),we have derived analytical results, Eqs. (19) and Eq. (22),that allow us to answer this question precisely. Hiddennodes affect the reconstruction results by introducing cor-rections C ij . These corrections C ij are nonzero whenthe measured nodes i and j are connected via a pathof hidden nodes as depicted in Fig. 1. Our estimateof C ij , as shown in Eq. (40), shows that three factors determine C ij ’s, namely the number of paths of hiddennodes connecting the two measured nodes i and j , thecoupling strength of the links and the strength of thehidden nodes in these paths. Interestingly, hidden nodeswith larger strength give rise to smaller corrections whenthe other two factors remain the same. When the hid-den nodes are not preferentially linked to the measurednodes in any manner, these three factors would not dif-fer much between or among the two groups of connectedand unconnected measured nodes and, as a result, thehidden nodes would have little effects on the reconstruc-tion of the links among the measured nodes. This is trueeven when the hidden nodes outnumber the measurednodes. In the event that the hidden nodes are prefer-entially linked to the measured nodes such that one ormore of the above three factors vary significantly eitherbetween or among the two groups, the accuracy of thereconstruction results would deteriorate. Yet useful in-formation can still be uncovered. We have verified ourtheoretical results and their implications using numer-ical simulations and our numerical results indicate theapplicability of our results and analytical understandingbeyond the model class of bidirectional networks studied.Hence our work shows that the method based on theinverse of covariance is useful for reconstructing bidirec-tional networks even when there are hidden nodes. Mostnetworks of interest in real-world problems are directed.It is highly challenging to derive analytical results forthe effects of hidden nodes on the reconstruction of gen-eral directed networks. It would thus be interesting toinvestigate whether and how the present results and un-derstanding for bidirectional networks can be extendedto general directed networks. Acknowledgments The work of ESCC and PHT has been supported bythe Hong Kong Research Grants Council under grant no.CUHK 14304017. Appendix A: Derivation of Eq. (25) Denote the eigenvalues of M ≡ L h + a I n h by λ k , k =1 , . . . , n h . By Cayley-Hamilton theorem, M satisfies itsown characteristic equation. Therefore = N − n Y k =1 ( M − λ k I n h )= M n h + n h X m =1 ( − m e m M n h − m (A1)where M = I n h and e m ( λ , . . . , λ k ), 1 ≤ m ≤ n h , arethe elementary symmetric polynomials of λ k ’s. For ex-ample, e = P n h k =1 λ k and e n h = Q n h k =1 λ k . Multiply-ing M − to Eq. (A1) and rearranging terms, we express M − as a finite polynomial of M : M − = n h − X m =0 ( − m e n h − − m e n h M m (A2)Using M ≡ S − W with S and W defined in Eqs. (23)and (24), we obtain the elements of M m in terms of theelements of S and W . For m = 1 and 2: M µν = f (1)0 δ µν + f (1)1 W µν (A3)( M ) µν = f (2)0 δ µν + f (2)1 W µν + n h X α =1 f (2)2 W µα W αν (A4)and for 3 ≤ m ≤ n h − M m ) µν = f ( m )0 δ µν + f ( m )1 W µν + n h X α =1 f ( m )2 W µα W αν + m X k =3 n h X α , ··· ,α k − =1 f ( m ) k W µα k − Y j =1 W α j α j +1 W α k − ν (A5)Here f ( m )0 = ( s µ + n + a ) m and f ( m ) m = ( − m for m =1 , . . . , n h − 1. For m ≥ f ( m ) k , 1 ≤ k ≤ m − 1, gen-erally depends on ˆ s µ + n ≡ s µ + n + a , ˆ s ν + n and ˆ s α i + n , i = 1 , . . . , k − 1. Explicit results for f ( m ) k , 1 ≤ k ≤ m − m = 2 and 3 are f (2)1 = − (ˆ s µ + n + ˆ s ν + n ) (A6) f (3)1 = − [(ˆ s µ + n ) + (ˆ s ν + n ) + ˆ s µ + n ˆ s ν + n ] (A7) f (3)2 = ˆ s µ + n + ˆ s ν + n + ˆ s α + n (A8) Substituting Eqs. (A3)-(A5) into Eq. (A2), we obtain( M − ) µν = F δ µν + F W µν + n h X α =1 F W µα W αν + n h − X k =3 n h X α , ··· ,α k − =1 F k W µα k − Y j =1 W α j α j +1 W α k − ν (A9)where F k = n h − X m = k ( − m e n h − − m e n h f ( m ) k , ≤ k ≤ n h − , (A10)depends generally on ˆ s µ + n , ˆ s ν + n , ˆ s α i + n , i = 1 , . . . , k − λ k . Putting Eq. (A9) into Eq. (20), we thus obtain C ij = n h X µ,ν =1 E iµ ( M − ) µν E jν = n h X µ =1 F W iµ W µ j + n h X µ ,µ =1 F W iµ W µ µ W µ j + n h − X k =3 n h X µ , ··· ,µ k +1 =1 F k W iµ k Y j =1 W µ j µ j +1 W µ k +1 j (A11)which is just Eq. (25). [1] S.H. Strogatz, Exploring complex networks, Nature(London) , 268 (2001).[2] M. Timme, Does dynamics reflect topology in directednetworks?, Europhys. Lett. , 367 (2006).[3] M. Timme and J. Casadiego, Revealing networks fromdynamics: an introduction, J. Phys. A , 343001 (2014).[4] W.-X. Wang, Y.-C. Lai, C. Grebogi, Data based identifi-cation and prediction of nonlinear and complex dynami-cal systems, Phys. Reports , 1 (2016).[5] J. Ren, W.-X. Wang, B. Li, and Y.-C. Lai, Noise BridgesDynamical Correlation and Topology in Coupled Oscil-lator Networks, Phys. Rev. Lett. , 058701 (2010).[6] E.S.C. Ching, P.Y. Lai, and C.Y. Leung, Extracting con-nectivity from dynamics of networks with uniform bidi-rectional coupling, Phys. Rev. E , 042817 (2013); Er-ratum, Phys. Rev. E , 029901(E) (2014).[7] E.S.C. Ching, P.Y. Lai, and C.Y. Leung, Reconstruct-ing weighted networks from dynamics, Phys. Rev. E ,030801(R) (2015).[8] Z. Zhang, Z. Zheng, H. Niu, Y. Mi, S. Wu, and G. Hu,Solving the inverse problem of noise-driven dynamic net-works, Phys. Rev. E , 012814 (2015).[9] E.S.C. Ching and H.C. Tam, Reconstructing links in di- rected networks from noisy dynamics, Phys. Rev. E ,010301(R) (2017).[10] P.Y. Lai, Reconstructing Network topology and couplingstrengths in directed networks of discrete-time dynamics,Phys. Rev. E , 022311 (2017).[11] Y. Chen, S. Wang, Z. Zheng, Z. Zhang and G. Hu, De-picting network structures from variable data producedby unknown colored-noise driven dynamics, Europhys.Lett. , 18005 (2016).[12] H.C. Tam, E.S.C. Ching, and P.Y. Lai, Reconstructingnetworks from dynamics with correlated noise, PhysicaA , 106 (2018).[13] B. Dunn and Y. Roudi, Learning and inference in anonequilibrium Ising model with hidden nodes, Phys.Rev. E , 022127 (2013).[14] R.-Q. Su, Y.-C. Lai, X. Wang, and Y. Do, Uncoveringhidden nodes in complex networks in the presence ofnoise, Sci. Reports , 3944 (2014).[15] Y.H. Chang and C.J. Tomlin, Reconstruction of GeneRegulatory Networks with Hidden Nodes, Proc. Eur.Control Conf., 1492 (2014).[16] X. Han, Z. Shen, W.-X. Wang, and Z. Di, Robust Recon-struction of Complex Networks from Sparse Data, Phys. Rev. Lett. , 028701 (2015).[17] H. Huang, Effects of hidden nodes on network structureinference, J. Phys. A: Math. Theor. , 355002 (2015).[18] Y. Chen, C. Zhang, T.Y. Chen, S. Wang, and G. Hu,Reconstruction of noise-driven dynamic networks withsome hidden nodes, Sci. China Phys. Mech. Astron. ,070511 (2017).[19] J. Casadiego, M. Nitzan, S. Hallerberg, and M. Timme,Model-free inference of direct network interactions fromnonlinear collective dynamics, Nature Commun. , 2192(2017).[20] J.M. Stuart, E. Segal, D. Killer, and S.K. Jim, A gene-coexpression network for global discovery of conservedgenetic modules, Science , 249 (2003).[21] V. M. Egu´ıluz, D. R. Chialvo, G. A. Cecchi, M. Baliki,and A. V. Apkarian, Scale-Free Brain Functional Net-works, Phys. Rev. Lett. , 018102 (2005).[22] F. Emmert-Streib, G.V. Glazko, G. Altay, and R.deMatos Simoes, Statistical inference and reverse engineer-ing of gene regulatory networks from observational ex-pression data, Frontiers of Genetics , 1 (2012).[23] E. Schneidman, M.J. Berry, R. Segev, and W. Bialek, Weak pairwise correlations imply strongly correlated net-work states in a neural population, Nature , 1007(2006).[24] H. C. Naguyen, R. Zecchina, and J. Berg, Inverse statis-tical problems: from the inverse Ising problem to datascience, Advances in Physics ∼ kconrad/blurbs/[26] L. Arnold, Stochastic Differential Equations: Theory andApplications , Wiley-Interscience, New York (1974).[27] See, for example, page 618 of C.D. Meyer, Matrix Anal-ysis and Applied Linear Algebra , SIAM, Philadelphia(2000).[28] A.-L. Barab´asi and R. Albert, Emergence of scaling inrandom networks, Science , 509 (1999).[29] R. FitzHugh, Impulses and Physiological States in The-oretical Models of Nerve Membrane, Biophys. J. , 445(1961).[30] O. E. R¨ossler, An Equation For Continuous Chaos, Phys.Lett. A57