Another estimation of Laplacian spectrum of the Kronecker product of graphs
aa r X i v : . [ c s . S I] F e b Another estimation of Laplacian spectrum of the Kronecker productof graphs
Milan Baˇsi´c , Branko Arsi´c , and Zoran Obradovi´c Department of Computer Science, University of Niˇs, Serbia Department of Mathematics and Informatics, University of Kragujevac, Serbia Department of Computer and Information Sciences, Center for Data Analytics andBiomedical Informatics, Temple University, Philadelphia, PA, USA basic [email protected], [email protected], [email protected]
Abstract
The relationships between eigenvalues and eigenvectors of a product graph and those of itsfactor graphs have been known for the standard products, while characterization of Laplacianeigenvalues and eigenvectors of the Kronecker product of graphs using the Laplacian spectra andeigenvectors of the factors turned out to be quite challenging and has remained an open problemto date. Several approaches for the estimation of Laplacian spectrum of the Kronecker productof graphs have been proposed in recent years. However, it turns out that not all the methods arepractical to apply in network science models, particularly in the context of multilayer networks.Here we develop a practical and computationally efficient method to estimate Laplacian spectraof this graph product from spectral properties of their factor graphs which is more stable thanthe alternatives proposed in the literature. We emphasize that a median of the percentage errorsof our estimated Laplacian spectrum almost coincides with the x -axis, unlike the alternativeswhich have sudden jumps at the beginning followed by a gradual decrease for the percentageerrors. The percentage errors confined (confidence of the estimations) up to ±
10% for all consid-ered approximations, depending on a graph density. Moreover, we theoretically prove that thepercentage errors becomes smaller when the network grows or the edge density level increases.Additionally, some novel theoretical results considering the exact formulas and lower boundsrelated to the certain correlation coefficients corresponding to the estimated eigenvectors arepresented.
Keywords—
Kronecker product of graphs, Estimated Laplacian eigenvalues and eigenvectors of graphproduct
Many real-life interactions throughout nature and society, such as protein-protein interaction networks [1],connections among image pixels [2], Internet social networks [3], the evolution of a quantum system [4] etc.,could be naturally described and represented in the context of large networks. However, the properties ofsuch large networks can not be easily determined because of a large computational complexity of methodsand algorithms performed on their corresponding graph matrices. Fortunately, large networks are oftencomposed of several smaller pieces, for example motifs [5], communities [6], or layers [7]. In this case, byusing the properties of these smaller structures, we can determine the properties of large networks obtainedby using some operations [8, 9]. In graph theory there are three fundamental graph products which referto the large network’s construction from two or more small graphs: Cartesian product, Kronecker (direct) roduct, and strong product. In each case, the product of graphs G and H is a graph whose vertex set is theCartesian product V ( G ) × V ( H ) of sets, while each product has different rules for edge creation. Computerscience is one of the many fields (such as mathematics and engineering) in which graph products, with theirown set of applications and theoretical interpretations, are becoming commonplace. As one specific example,large networks such as the Internet graph, with several hundred million hosts, can be efficiently modeledby subgraphs of powers of small graphs with respect to the Kronecker product [10]. More recently, graphproducts have also began to appear in network science, where multiplication of graphs are often used as aformal way to describe certain types of multilayer network topologies [7][11][12]. Products of graphs thatmake use of spectral methods have also found important applications in interconnection networks, massivelyparallel computer architectures and diffusion schemes [13].It was recognized in about the last twenty years that graph spectra have many important applications invarious areas, especially in the fields of computer sciences (see, e.g., [14][15]), such as Internet technologies,computer vision, pattern recognition, data mining, multiprocessor systems, statistical databases and manyothers. One of the important questions to be addressed in this area, and which have been studied extensivelyby many researchers, is how to characterize spectral properties of a product graph using those of its factorgraphs. Relationships between spectral properties of a product graph and those of its factor graphs have beenknown for the spectra of degree and adjacency matrices for all of the three products, as well as the Laplacianspectra for Cartesian product [16]. Results describing the adjacency matrix and its spectra of the productgraphs can be also found in [17] and [18], while a complete characterization of the Laplacian spectrum of theCartesian product of two graphs has been done by Merris [19]. In the paper [20], the authors tried to exploitthe benefits of the Kronecker graph representation, which is used as a replacement for the multilayer network.However, they had to face an open problem, because the Laplacian spectrum of the Kronecker product oftwo graphs graphs can not be characterized by using the Laplacian spectra of the factors. In [21], the authorsgave the explicit complete characterization of the Laplacian spectrum of the Kronecker product of two graphsin some particular cases. Since it seems that an explicit formula can not be obtained for the general case, in[16] the authors developed empirical methods to estimate the Laplacian spectra of the Kronecker of graphsfrom spectral properties of their factor graphs.In this paper we develop an alternative practical method for an estimation of the the Laplacian spectrumand eigenvectors of the Kronecker (direct) product of two graphs. We noticed that estimated eigenvalues andeigenvectors of these approximations express different behavior depending on the type of network topology.The effectiveness of the proposed methods are evaluated through numerical experiments, where experimentsare performed on three types of graphs: Erd˝os-R´enyi, Barab´asi-Albert and Watts-Strogatz, while the edgedensity percentage is varied over 10%, 30%, and 65%. In order to see whether, our novel approximationor the one proposed by Sayama in [16], is more suitable for the original eigenvalues and eigenvectors, wecompare them in the following two ways. First, we give an empirical and some theoretical evidence that theKronecker product of eigenvectors of normalized Laplacian matrices of factor graphs can be also used as anapproximation for the eigenvectors of Laplacian matrix of Kronecker product of graphs. It can be done bycomparing the correlation coefficients that correspond to the approximated vectors for both approximationin regard to different types and edge density levels of graphs. Then, in order to test how close the estimatedto the original eigenvalues of Laplacian of the Kronecker product of graphs for both approximations are,the difference between them in terms of a distribution of percentage errors is reported. We show that adistribution of percentage errors between novel estimated and original spectra is more stable than the errorobtained for the Sayama’s spectrum and it is almost uniformly distributed around 0, all in the case of Erd˝os-R´enyi and Watts-Strogatz random networks. It is also noticed that both approximations produced reasonableestimations of Laplacian spectra with percentage errors confined within a ±
10% range for most eigenvalues,with a small variations depending on the type and edge density levels of random networks. Moreover, wetheoretically prove that the percentage errors become smaller when the network grows or the edge densitylevel increases for Erd˝os-R´enyi random networks. In the case of Barab´asi-Albert random networks, similarnumber of jumps in the graphs of percentage errors distribution is noticed for the both proposed estimatedspectra.The remainder of our paper is organized through the following sections. In Section 2 we will explainthe motivation and assumptions for our alternative approach developed for the estimation of the Laplacianeigenvalues and eigenvectors of the Kronecker product of graphs. Moreover, in subsection 2.1 we recallsome results and techniques used in [16] and provide a proof that all estimated eigenvalues proposed by ayama are nonnegative. In subsection 2.2 we introduce the Kronecker product of eigenvectors of normalizedLaplacian matrices of factor graphs as a potential approximation for the actual eigenvectors the Laplacianmatrix of Kronecker product of graphs and by using them we get the formula (5) for estimating the Laplacianspectra of Kronecker product of graphs. In Section 3 we report a behavior of the estimated eigenvalues andeigenvectors (for both approximations) compared to the original ones with regard to the different types ofgraphs and different edge density levels. The comparison between estimated and original spectra has beendone by calculating the percentage error, while the correlation coefficients are used to express the differencebetween eigenvectors. In subsection 3.1.2 we provide some new theoretical results related to the correlationcoefficients that correspond to the estimated vectors for both approximation and give certain explanationwhy the Kronecker product of eigenvectors of normalized Laplacian matrices of factor graphs can be usedas suitable approximation for the actual eigenvectors the Laplacian matrix of Kronecker product of graphs.In Theorem 1 and Theorem 2 we provide exact formulas for the certain correlation coefficients and theexpected values of the correlation coefficients, respectively, corresponding to the eigenvectors proposed in[16]. From the formulas for the correlation coefficients follow that they depend only on the degrees of oneof the factor graphs and hence they are mutually equal. According to the expected value of the previouscorrelation coefficients obtained by Theorem 2 and the inequality given by Theorem 4, we obtain that thecorrelation coefficients corresponding to our estimated vectors in some cases can be greater than the coefficientcorrelations corresponding to the eigenvectors proposed in [16]. Finally, using Theorem 5 we give a theoreticalexplanation of why the estimated eigenvalues for the random graphs become more accurate to the real valueswhen the network grows or the edge density level increases. The paper concludes with a summary of keypoints and directions for further work. We also point out that these approximations could have a veryimportant application in learning models based on multilayer networks. [20]. Before describing the proposed methods, we provide definitions for concepts used throughout the paper. By G = ( V G , E G ) we denote a simple connected graph (without loops and multiple edges), where V G is the setof vertices and E G ⊆ (cid:0) V G (cid:1) is a set of edges of G . The adjacency matrix A for a graph G with N vertices is an N × N matrix whose ( i, j ) entry is 1 if the i -th and j -th vertices are adjacent, and 0 if they are not. A numberof the vertices N of a graph G is called the order of a graph G . A vertex and an edge are called incident, ifthe vertex is one of the two vertices that the edge connects. The Laplacian matrix of the adjacency matrix A is defined as L = D − A where D is the degree matrix of A (degree matrix is a diagonal matrix whereeach entry ( i, i ) is equal to the number of edges incident to i -th vertex). The normalized Laplacian matrix isdefined as L = D − LD − = I − D − AD − . Let G = ( V G , E G ) and H = ( V H , E H ) be two simple connectedgraphs. Definition 1
The Kronecker product of graphs denoted by G ⊗ H is a graph defined on the set of vertices V G × V H such that two vertices ( g, h ) and ( g ′ , h ′ ) are adjacent if and only if ( g, g ′ ) ∈ E G and ( h, h ′ ) ∈ E H . The Kronecker product of an N × N matrix A and a M × M matrix B is the ( N M ) × ( N M ) matrix A ⊗ B with elements defined by ( A ⊗ B ) I,J = A i,j B k,l with I = M ( i −
1) + k and J = M ( j −
1) + l .In the rest of this section we discuss the spectral decomposition of the Laplacian of the Kronecker productof graphs from those of its factor graphs. Because it seems that such an explicit formula does not exist, weneed to apply some approximations in order to obtain the estimated eigenvalues and eigenvectors. In the following section we will explain the motivation and assumptions from [16] for the proposed approxi-mation and show some of their properties. The Laplacian of the Kronecker product of graphs is given by the ollowing L S ⊗ S = D S ⊗ S − A S ⊗ S = ( D S ⊗ D S ) − ( A S ⊗ A S )= D S ⊗ D S − ( D S − L S ) ⊗ ( D S − L S )= L S ⊗ D S + D S ⊗ L S − L S ⊗ L S , where A S and A S are the adjacency matrices and D S and D S are the degree matrices of graphs S and S , respectively, where | S | = n and | S | = n . The idea of the proposed approximation is to assume that w S i ⊗ w S j , where w S i and w S j are arbitrary eigenvectors of L S and L S respectively, could be used as asubstitute for the true eigenvectors of L S ⊗ S . A motivation for this assumption came from the fact thatthe Laplacian spectra of the Kronecker product of graphs resemble those of the Cartesian product of graphswhen either factor graph is regular [21]. Let W S and W S be n × n and n × n square matrices thatcontain all w S i and w S j as column vectors, respectively. By making (mathematically incorrect) assumptionthat D S W S ≈ W S D S and D S W S ≈ W S D S it can be obtained that L S ⊗ S ( W S ⊗ W S ) = L S W S ⊗ D S W S + D S W S ⊗ L S W S − L S W S ⊗ L S W S ≈ W S Λ S ⊗ W S D S + W S D S ⊗ W S Λ S − W S Λ S ⊗ W S Λ S = ( W S ⊗ W S ) (cid:16) Λ S ⊗ D S + D S ⊗ Λ S − Λ S ⊗ Λ S (cid:17) , (1)where Λ S and Λ S are diagonal matrices with eigenvalues µ S i of L S and µ S j of L S , respectively. From thelast equation, estimated Laplacian spectrum of S ⊗ S could be calculated as µ ij = { µ S i d S j + d S i µ S j − µ S i µ S j } . (2)where d S i and d S j are the diagonal entries of the degree matrices D S and D S , respectively.Here we note that the orderings of w S i and w S j (and hence µ S i and µ S j ) are independent of the vertexorderings in D S and D S , respectively. This can help in reducing the mathematical inaccuracy arising fromthe mentioned incorrect assumptions by finding optimal column permutations of W S and W S (influencingΛ S and Λ S ). Therefore, several types of ordering of eigenvalues ( µ S i and µ S j ) of factor graphs were tested[16], while the degree sequences are fixed in ascending order. It was obtained that the most effective heuristicmethod is when the eigenvalues are sorted in ascending order.From (2) it can be easily seen that the estimated spectrum always has an eigenvalue of 0, because if µ S i = 0 and µ S j = 0, then µ ij = 0. However, it is not commented in [16] whether all other estimatedeigenvalues µ ij are greater than or equal to 0. Notice that (2) can be rewritten as follows: µ S i ( d S j − µ S j µ S j ( d S i − µ S i ≤ i ≤ n , ≤ j ≤ n . If a graph is regular then the absolute values of the eigenvalues of its adjacency matrix are less thanor equal to the regularity of the graph (according to the Perron-Frobenius theorem, see [22], pp. 178) andit is clear from the definition of the Laplacian matrix that all Laplacian eigenvalues are less than or equalto the double value of the regularity. This implies that in the case when S and S are regular, we havethat d S j ≥ µ S j and d S i ≥ µ S i , and therefore µ ij ≥
0. In the following we prove that these eigenvalues arenonnegative in the general case.By applying Gershgorin circle theorem on Laplacian matrix we can obtain only the inequality d S n − µ S n ≥ µ S n ≤ d S n ). Indeed, as every eigenvalue of the n × n Laplacian matrix L = ( l i,j ) ≤ i,j ≤ n lies within the union of disks centered at l i,i = d S i with radius R i = d S i ( R i is the sum of the absolute valuesof the non-diagonal entries in the i -th row for 1 ≤ i ≤ n ), we can not conclude that every eigenvalue µ S i lies in the circle centered at d S i with radius d S i , i.e. µ S i ≤ d S i , 1 ≤ i ≤ n − µ S i ≤ d S i can be proved by using Courant-Fischer theorem for everyindex i . Namely, it is easy to see that the quadratic form x T Lx in respect to the Laplace matrix L and an rbitrary vector x = ( x , x , . . . , x n ) can be rewritten in the following way x T Lx = n X j =1 d S j x j − X ( i,j ) ∈ E ( S ) x i x j . Now, using the arithmetic-geometric mean inequality between x i and x j , | x i x j | ≤ x i + x j , it holds that − P ( i,j ) ∈ E ( S ) x i x j ≤ P n j =1 d S j x j and therefore x T Lx ≤ P n j =1 d S j x j . Furthermore, considering x ∈ R i × { } n − i ⊆ R n , we have in this case that x T Lx ≤ P ij =1 d S j x j ≤ d S i k x k . Finally, according toCourant-Fischer we have that µ S i ≤ max x ∈ R i ×{ } n − i x T Lx k x k ≤ d S i k x k k x k = 2 d S i (we have already mentionedthat the degree sequence is set in ascending order, that is d S ≤ . . . ≤ d S n ). d d d µ µ µ =0 Figure 1: Gershgorin disks for Laplacian matrix
The approximations from [16] are not derived from rigorous mathematical proofs, but from empiricalevidence and good behavior of estimated eigenvalues and eigenvectors has been noticed for some types ofrandom graphs. In the following subsection we propose an estimation of Laplacian spectral decomposition forthe Kronecker product of graphs by using the normalized Laplacian eigenvectors of factor graphs. We showsome differences side by side (both experimentally and analytically) between these approximations throughthe eigenvectors and eigenvalues analysis separately.
In this section we propose an alternative approach for estimating the Laplacian spectrum of the Kroneckerproduct of graphs. The idea comes from the fact that the normalized Laplacian matrix of the Kroneckerproduct of graphs can be represented in terms of normalized Laplacian matrices of factor graphs. Moreover,in some cases the Kronecker product of the eigenvectors of L S and L S gives better approximation foreigenvectors of L S ⊗ S than the Kronecker product of the eigenvectors of L S and L S . Now, we will explainthe motivation and assumptions for this novel approach in more detail.By the definition of the normalized Laplacian and the property ( A ⊗ B ) − = A − ⊗ B − , the normalizedLaplacian of the matrix S ⊗ S can be written in the following way L S ⊗ S = I n ⊗ I n − ( D − S ⊗ D − S )( S ⊗ S )( D − S ⊗ D − S ) . sing the property of the Kronecker product of matrices, ( A ⊗ B )( C ⊗ D ) = AC ⊗ BD , we further obtain: L S ⊗ S = I n ⊗ I n − ( D − S S D − S ) ⊗ ( D − S S D − S ) = I n ⊗ I n − ( I n − L S ) ⊗ ( I n − L S ) . Let { λ S i } and { λ S j } be the eigenvalues of the matrices L S and L S , with the corresponding orthonormaleigenvectors { v S i } and { v S j } , where i = 1 , , . . . , n and j = 1 , , . . . , n . Denote by Λ S and Λ S the diagonalmatrices whose diagonal elements are the values 1 − λ S i and 1 − λ S j , respectively. Also, V S and V S standfor the square matrices which contain v S i and v S j as column vectors. Using the spectral decomposition ofthe matrix ( I n − L S ) ⊗ ( I n − L S ), from the above equation it follows that L S ⊗ S = I n ⊗ I n − ( V S Λ S V TS ) ⊗ ( V S Λ S V TS )= I n ⊗ I n − ( V S ⊗ V S )(Λ S ⊗ Λ S )( V S ⊗ V S ) T = ( V S ⊗ V S )( I n ⊗ I n − Λ S ⊗ Λ S )( V S ⊗ V S ) T , (3)since ( V S ⊗ V S )( V S ⊗ V S ) T = I n n . This further implies that the normalized Laplacian matrix of theKronecker product of graphs have { − (1 − λ S i )(1 − λ S j ) } as eigenvalues and { v S i ⊗ v S j } as eigenvectors.Now, put Λ = I n ⊗ I n − Λ S ⊗ Λ S and D = D S ⊗ D S . It is well known that the normalized Laplaciancan be expressed in term of Laplacian matrix as L = D − LD − . Furthermore, as L S ⊗ S ( V S ⊗ V S ) = D L S ⊗ S D ( V S ⊗ V S ), using the similar assumption like in the previous subsection that D S V S ≈ V S D S and D S V S ≈ V S D S , from (3), we derive L S ⊗ S ( V S ⊗ V S ) ≈ D L S ⊗ S ( V S ⊗ V S ) D = D Λ( V S ⊗ V S ) D . Finally, applying the same assumption again we have the following formula L S ⊗ S ( V S ⊗ V S ) ≈ ( D Λ)( V S ⊗ V S ) . (4)Inside the first pair of parenthesis of the right-hand side of (4) is the diagonal matrix D Λ which leads us toa potential formula for estimating the Laplacian spectrum of the Kronecker product of graphs, while for thecorresponding eigenvectors we could use eigenvectors of the normalized Laplacian matrix of the Kroneckerproduct of graphs. Therefore, a potential formula for estimating the Laplacian spectra of the Kroneckerproduct of graphs µ ij = { (1 − (1 − λ S i )(1 − λ S j )) d S i d S j } = { ( λ S i + λ S j − λ S i λ S j ) d S i d S j } , (5)which are obviously nonnegative. Moreover, the first eigenvalue is always matched at 0 in both actual andestimated spectra, because (5) guarantees this.Similarly as in [16] this approximation shares the property that the orderings of v S i and v S j in V S and V S (and hence λ S i and λ S j ) are independent of vertex orderings in D S and D S , respectively, and it wouldbe impractical to try to find true optimal orderings. The following five heuristic methods that use onlydegrees and eigenvalues of factor graphs were tested: uncorrelated ordering , correlated ordering , correlatedordering with randomization , anti-correlated ordering and anti-correlated ordering with randomization . Ineach method, it is assumed that the degree sequences ( d S i and d S j ) are already sorted in ascending order,while the orders of eigenvalues ( λ S i and λ S j ) are altered differently. The most effective ordering methodsturned out to be correlated ordering ( λ S i and λ S j are sorted in ascending order), as it was obtained forapproximation spectrum [16]. In this section we report a behavior of the estimated eigenvalues and eigenvectors, from the presented ap-proximations, compared to the original ones with regard to the different types of graphs and different edgedensity levels. With these experiments we aim to address the following: We will show how close the estimated to the original eigenvectors of Laplacian of the Kronecker productof graphs are for these approximations. In order to do that we measure the distribution of vectorcorrelation coefficients between v S i ⊗ v S j and L S ⊗ S ( v S i ⊗ v S j ) as it was done for the eigenvectors w S i ⊗ w S j in [16]. In the rest of the section, we give an empirical and some theoretical evidence thatthe eigenvectors v S i ⊗ v S j can be also used as an approximation for the eigenvectors of L S ⊗ S . • We will show how close estimated to the original eigenvalues of Laplacian of the Kronecker product ofgraphs are for both approximations. Based on the corresponding estimated spectra (2) and (5), thedifference between estimated and original spectra is reported in terms of a distribution of percentageerrors between them. Both approximations produced reasonable estimations of Laplacian spectra withpercentage errors confined within a ±
10% range for most eigenvalues. This error value holds for thesparse graphs. It can be noticed that this error is even smaller for the denser graphs, i. e. about ± ±
2% when the edge density percentages are 30% and 65%, respectively. We also noticed that themedian of the percentage errors of our estimated Laplacian spectrum are more stable than in the caseof spectrum proposed by Sayama. Moreover, we give a theoretical explanation of why the percentageerrors of the approximated eigenvalues that correspond to v S i ⊗ v S j for the random graphs becomemore accurate to the real expected values when the network grows or the edge density level increases.Experiments are performed on three types of graphs: Erd˝os-R´enyi, Barab´asi-Albert and Watts-Strogatz,while the edge density percentage is varied over 10%, 30%, and 65%. For the orders of graphs G and H denoted by n and n , respectively, we conduct experiments three times depending on the orders of graphs( n , n ) ∈ { (30 , , (50 , , (100 , } . Here we describe the behavior of estimated eigenvectors and eigenvalues for both classes of graphs, Erd˝os-R´enyi and Watts-Strogatz, since their vector correlation coefficients and distributions of percentage errorsof the estimated eigenvalues behave similarly for the same experimental setup. We also noted a bit smallererrors in the case of Watts-Strogatz than for Erd˝os-R´enyi random networks. For both types of graphs we findthat the distribution of correlation coefficients between the vectors v S i ⊗ v S j and L S ⊗ S ( v S i ⊗ v S j ), and thevectors w S i ⊗ w S j and L S ⊗ S ( w S i ⊗ w S j ) behave very similar to the corresponding values of the eigenvectors v S i ⊗ v S j and w S i ⊗ w S j , when the edge density grows. Further in the paper by a term the correlationcoefficients corresponding to the arbitrary eigenvectors x Si of the graph S , we will mean the correlationcoefficients between the vectors x Si and L S ( x Si ). Also, we noticed that the shape of the percentage errordistribution across these two network topologies is more consistent (without sudden jumps) for the estimatedspectrum corresponding to the eigenvectors v S i ⊗ v S j , than for the estimated spectrum corresponding to theeigenvectors w S i ⊗ w S j . First, we present experimental and theoretical results for the estimated eigenvectorsand eigenvalues of Erd˝os-R´enyi random networks. It can be immediately seen that w S ⊗ w S coincides with an eigenvector of L S ⊗ S , where w S and w S are the eigenvectors of L S and L S , respectively, that correspond to the eigenvalue 0. Indeed, since it iswell-known that w S = 1 S , w S = 1 S , D S S = A S S and D S S = A S S we obtain that L S ⊗ S ( w S ⊗ w S ) = ( D S ⊗ D S − A S ⊗ A S )(1 S ⊗ S )= D S S ⊗ D S S − A S S ⊗ A S S = 0 . We can similarly show that L S · D S S = 0 and L S · D S S = 0. Indeed, we have that L S · D S S = ( I S − D − S A S D − S )( D S S )= D S S − D − S A S S = D S S − D − S D S S = 0 . orrelation coeff. between the predicted vectors v and L * v F r equen cy Correlation coeff. between the predicted vectors v and L * v F r equen cy Correlation coeff. between the predicted vectors v and L * v F r equen cy Figure 2: Smoothed probability density functions of vector correlation coefficients between w S i ⊗ w S j and L S ⊗ S ( w S i ⊗ w S j ) are represented by using a green solid line, while between v S i ⊗ v S j and L S ⊗ S ( v S i ⊗ v S j ) are represented using a blue solid line. Probability density functions are drawnfor each of the edge density level 10%, 30% and 65%, respectively, for the Erd˝os-R´enyi randomgraphs with 50 and 30 vertices. Therefore, for v S = D S S and v S = D S S it does not hold that v S ⊗ v S is an eigenvector of L S ⊗ S .Nevertheless, we omit the examination of the coefficient correlations that correspond to the vectors w S ⊗ w S and v S ⊗ v S in the following experimental setup, as we can explicitly calculate the first eigenvector of L S ⊗ S .Moreover, we can not claim in the general case (for example, when the graphs S and S are not regular)that any other approximation vector w S i ⊗ w S j or v S i ⊗ v S j coincides with the actual eigenvector of L S ⊗ S .The first set of experiments was performed for the eigenvectors comparison of two proposed approxima-tions on the sparse graphs, that is, we repeat the same experiment as in [16] where two Erd˝os-R´enyi randomnetworks have 50 vertices (100 edges) and 30 vertices (90 edges), respectively. It can be easily seen that theedge densities of these graphs are around 10%. In Figure 2 (left panel), one can see the smoothed probabilitydensity functions of vector correlation coefficients between the mentioned vectors drawn from five indepen-dent numerical results. Using the mentioned parameters on the estimated Laplacian eigenvectors w S i ⊗ w S j ,the correlation coefficients are above 0.8 in most of the cases, while the peaks are achieved above 0.9 (greensolid lines). For the same graphs, the correlation coefficients of the eigenvectors v S i ⊗ v S j are above 0.7 inmost of the cases, while the peaks are achieved between 0.8 and 0.9 (blue solid lines). Furthermore, it canbe seen in Figure 2 that the correlation coefficients for the eigenvectors v S i ⊗ v S j increase and their graphsshrink to the right (toward the value of 1) when the edge density level increases (middle and right panelsshow the graphics for the edge density levels of 30% and 65%, respectively).It can be noticed that the correlation coefficients corresponding to the eigenvectors v S i ⊗ v S j and w S i ⊗ w S j are symmetrically distributed around the peak and their smoothed probability density functions of vectorcorrelation coefficients look like a probability density function of the normal distribution. Indeed, according tothe Pearson’s chi-squared test (as a test of goodness of fit) we obtain that most of the correlation coefficientscorresponding to the eigenvectors v S i ⊗ v S j and w S i ⊗ w S j belong to a fitted normal distribution for the p -value of 0.05. When the edge density levels are 10% for both graphs, 1380 out of 1499 correlation coefficientscorresponding to the eigenvectors v S i ⊗ v S j belong to a fitted normal distribution. For the edge density levelsof 30% and 65%, 1471 and 1496 out of 1499 correlation coefficients belong to a fitted normal distribution,respectively. On the other hand, when the edge density levels are 10% for both graphs, 1488 out of 1499correlation coefficients corresponding to the eigenvectors w S i ⊗ w S j belong to a fitted normal distribution.For the edge density levels of 30% and 65%, 1476 and 1486 out of 1499 correlation coefficients belong toa fitted normal distribution, respectively. Moreover, a similar conclusion can be reached for both vectorproducts when the Erd˝os-R´enyi random networks have 50 and 100 vertices, as well as 100 and 200 vertices. .1.2 Theoretical results for eigenvectors estimation Given the performed experiments it can be noticed that some of the values of correlation coefficients thatcorrespond to the approximation vectors w S i ⊗ w S j are mutually equal. Indeed, we can explicitly determinethe values of correlation coefficients related to the vectors w S ⊗ w S j and w S i ⊗ w S , for 2 ≤ j ≤ n and2 ≤ i ≤ n , and show that they do not depend on the vectors w S j and w S i . In the following text, we provethat the correlation coefficients related to the vectors w S ⊗ w S j and w S i ⊗ w S only depend on the vertexdegrees of S and S , respectively. Theorem 1
The correlation coefficients r ,j ( ≤ j ≤ n ) corresponding to the vectors L S ⊗ S (1 S ⊗ w S j ) and S ⊗ w S j are equal to d S + ··· + d S n n r d S
11 2 + ··· + d S n n . Proof.
Using the fact that D S S = A S S = [ d S , . . . , d S n ] T , we show that the vectors L S ⊗ S ( w S ⊗ w S j )and [ d S , . . . , d S n ] T ⊗ w S j are colinear L S ⊗ S (1 S ⊗ w S j ) = ( D S ⊗ D S − A S ⊗ A S )(1 S ⊗ w S j )= ( D S S ) ⊗ ( D S w S j ) − ( A S S ) ⊗ ( A S w S j )= [ d S , . . . , d S n ] T ⊗ ( D S − A S ) w S j = µ S j [ d S , . . . , d S n ] T ⊗ w S j . (6)According to (6) we have the following chain of equalities r ,j = h L S ⊗ S (1 S ⊗ w S j ) , S ⊗ w S j ik µ S j [ d S , . . . , d S n ] T ⊗ w S j k · k S ⊗ w S j k = ( µ S j [ d S , . . . , d S n ] ⊗ w S j T ) · (1 S ⊗ w S j ) µ S j k [ d S , . . . , d S n ] k · k S k · k w S j k = ( µ S j [ d S , . . . , d S n ]1 S ) ⊗ k w S j k µ S j √ n k [ d S , . . . , d S n ] k · k w S j k = d S + ··· + d S n n r d S
11 2 + ··· + d S n n . (cid:3) We see that r ,j = 1 if and only if the arithmetic mean of the vertex degrees of S is equal to theroot mean square of the same elements and it is well-known that it is true if and only if S is a regulargraph. On the other hand, the values of r ,j can be very low in the cases where there is a large gap betweenthe lowest and highest vertex degrees in the graphic sequence of S , 1 ≤ d S ≤ . . . ≤ d S n ≤ n −
1. Forexample, considering the complete bipartite graph S = K ,n − and calculating the arithmetic mean andthe root mean square of the vertex degrees which are n − n and √ n −
1, respectively, we can deduce that r ,j = √ n − n →
0, when n → ∞ . However, if the sizes of the partition sets in a bipartite graph becomemore equal (tend to n /
2) then the coefficient r ,j → S = K ,n − andobtain that r ,j = √ n − n > √ n − n ). In addition, it can be noticed that the correlation coefficients do notdecrease with the increase in the number of different vertex degrees in S . Indeed, if we consider the graph S with an even order n = 2 k + 2 and the graphic sequence 1 , , . . . , k, k + 1 , k + 1 , k + 2 , . . . , k + 1, it an be determined that the arithmetic mean and the root mean square are k + 1 and q (2 k +1)(4 k +3)+3( k +1)6 ,respectively. Therefore, in this case we obtain high correlation coefficients r ,j = q − k +1) + k +1)2 → √ , k → ∞ .However, since we have obtained correlation coefficients r ,j using a certain number of synthetic networksproduced by the Erd˝os-R´enyi model, in the following text we theoretically discuss about the expected valuesof the correlation coefficients r ,j , using the following auxiliary result. Proposition 1 ([23] pp. 211)
Suppose that X n is AN ( µ, c n Σ) where Σ is a symmetric nonnegative definitematrix and c n → as n → ∞ . If g ( X ) = ( g ( X ) , . . . , g m ( X )) ′ is a mapping from R k into R m such thateach g i is continuously differentiable in a neighborhood of µ , and if D Σ D ′ has all of its diagonal elementsnon-zero, where D is the m × k matrix [( ∂g i ∂x j )( µ )] , then g ( X n ) is AN ( g ( µ ) , c n D Σ D ′ ) . Theorem 2 If S is Erd˝os-R´enyi graph model, then the expected value of the correlation coefficient r ,j corresponding to the vectors L S ⊗ S (1 S ⊗ w S j ) and S ⊗ w S j tends to s ( n − p − p + ( n − p , (7) as n → ∞ . Proof.
Since the distribution of the degree of any particular vertex of the Erd˝os-R´enyi graph S = G ( n , p )is binomial, that is P ( d S i = k ) = (cid:0) n − k (cid:1) p k (1 − p ) n − k − , and the fact that the expected value of any vertexdegree is equal to the expected value of the arithmetic mean of degrees Y = d S + ... + d S n n , we conclude that E ( Y ) = ( n − p . Furthermore, as n → ∞ , according to the central limit theorem Y has asymptoticnormal distribution AN ( µ , σ n ), where µ = E ( Y ) and σ = D ( d S i ) = ( n − p (1 − p ) ( E and D areusual notation for expected value and dispersion, respectively). Similarly, as d S i , 1 ≤ i ≤ n , have thesame distribution, we deduce that Y = d S
11 2 + ··· + d S n n has asymptotic normal distribution AN ( µ , σ n ), where µ = E ( Y ) = E ( d S i ) and σ = D ( d S i ). On the other hand, given that E ( d S i ) = D ( d S i ) + E ( d S i ) , wehave that µ = ( n − p (1 − p ) + ( n − p . Considering the two dimensional variable X = [ Y , Y ] it canbe concluded that its asymptotic normal distribution is AN ([ µ , µ ] ′ , c n Σ), where Σ represents a nonnegativedefinite symmetric matrix. Define g ( y , y ) = y √ y which is a continuously differentiable function. Finally,using Proposition 1 we conclude that g ( X ) has asymptotic normal distribution AN ( µ , c n D Σ D ′ ), where µ = g ( µ , µ ) = µ √ µ . Therefore, the expected value of the coefficient correlation r ,j is equal to E ( g ( X ))(according to Theorem 1) which tends to µ = q ( n − p − p +( n − p , as n → ∞ . (cid:3) If we rewrite (7) in the form r − p ( n − p +1 , we conclude that the expected value of r ,j tends to 1 when theorder of the graph increases, for the fixed edge density p . Therefore, we show that w S ⊗ w S j , 2 ≤ j ≤ n ,becomes a more stable approximation for the larger orders of graphs with constant edge level. Moreover, wereport higher coefficient correlations r i,j when the order of graphs are 50 and 100, respectively (see Fig. 3).The same conclusion can be obtained if the order of graphs are 100 and 200, respectively.Similarly, for a given order n of the graph S , by rewriting (7) in the form q n − p − n − , we concludethat r ,j →
1, if p tends to 1 and r ,j →
0, if p tends to 0. Notice that we have already obtained a moregeneral conclusion by performing three types of experiments in which the correlation coefficients increasein total as long as the edge density increases (for a fixed n ). In our experimental setup p can not tendto 0 since we deal with connected graphs. Namely, a sharp threshold for the connectedness of S is ln n n (more precisely if p > (1+ ǫ ) ln n n then the graph S will almost surely be connected). Since the parametersin the experimental setup satisfy the mentioned condition, we almost surely deal with connected graphs and orrelation coeff. between the predicted vectors v and L * v F r equen cy Smoothed density function
Product of Laplacian eigenvectorsProduct of Normalized Laplacian eigenvectors
Figure 3: Smoothed probability density functions of vector correlation coefficients between w S i ⊗ w S j and L S ⊗ S ( w S i ⊗ w S j ) are represented by using a solid green line, while between v S i ⊗ v S j and L S ⊗ S ( v S i ⊗ v S j ) are represented using a solid blue line. Probability density functions are drawnfor the edge density level of 10%, for the Erd˝os-R´enyi random graphs with 50 and 100 vertices. after applying the condition we obtain r ,j ≥ − n ǫ ) ln n +(1 − n ) . Therefore, r ,j →
1, as n → ∞ , whichtheoretically confirms our experimental results that the correlation coefficient r ,j grows as long as the orderof the connected Erd˝os-R´enyi graph grows.In the following text, we estimate the correlation coefficients related to the vectors v S ⊗ v S j and v S i ⊗ v S in terms of the vertex degrees of S and S , respectively. Moreover, we prove that the expected values ofthese coefficients r ′ ,j can exceed the expected value of r ,j given by (7), when n → ∞ and n → ∞ . Lemma 1
The scalar product of the vectors L S ⊗ S ( v S ⊗ v S j ) and v S ⊗ v S j is greater than or equal to ( d S + · · · + d S n ) v S j T L S v S j , for ≤ j ≤ n . The equality holds true if and only if S is regular. Proof.
Since v S = D S S we have the following chain of equalities h L S ⊗ S ( D S S ⊗ v S j ) , D S S ⊗ v S j i = ( D S S ⊗ v S j ) T L S ⊗ S ( D S S ⊗ v S j )= ( D S S ⊗ v S j ) T ( D S ⊗ D S − A S ⊗ A S )( D S S ⊗ v S j )= (1 TS D S ⊗ v S j T )( D S ⊗ D S )( D S S ⊗ v S j ) − (1 TS D S ⊗ v S j T )( A S ⊗ A S )( D S S ⊗ v S j )= (1 TS D S D S D S S ) ⊗ ( v S j T D S v S j ) − (1 TS D S A S D S S ) ⊗ ( v S j T A S v S j )= (1 TS D S S ) ⊗ ( v S j T D S v S j ) − ([ d S , . . . , d S n ] A S [ d S , . . . , d S n ] T ) ⊗ ( v S j T A S v S j ) . (8)Furthermore, the quadratic forms 1 TS D S S and [ d S , . . . , d S n ] A S [ d S , . . . , d S n ] T are equal to P n i =1 d S i and P { i,j }∈ E ( S ) d S i d S j , respectively. According to the inequality of arithmetic and geometric means itholds that P { i,j }∈ E ( S ) d S i d S j ≤ P { i,j }∈ E ( S ) d iS + d jS = P n i =1 d S i . The equality holds true if andonly if d S = . . . = d S n . Finally, we have that the term (8) is greater than or equal to P n i =1 d S i ( v S j T D S v S j ) − P n i =1 d S i ( v S j T A S v S j ) = P n i =1 d S i ( v S j T L S v S j ). (cid:3) emma 2 The norm of the vector L S ⊗ S ( v S ⊗ v S j ) is less than or equal to q d S + · · · d S n k L S v S j k , for ≤ j ≤ n . The equality holds true if and only if S is regular. Proof.
We have the following chain of equalities L S ⊗ S ( D S S ⊗ v S j ) = ( D S ⊗ D S − A S ⊗ A S )( D S S ⊗ v S j )= ( D S D S S ) ⊗ ( D S v S j ) − ( A S D S S ) ⊗ ( A S v S j )= [ d S , . . . , d S n ] T ⊗ ( D S v S j ) − [ X { ,i }∈ E ( S ) d i , . . . , X { n ,i }∈ E ( S ) d i ] T ⊗ ( A S v S j ) . Furthermore, since u ⊗ ( Av ) = ( u ⊗ A ) v , where u T ∈ R n , v T ∈ R n and A ∈ R n × n it holds that k L S ⊗ S ( D S S ⊗ v S j ) k = (9)= k ([ d S , . . . , d S n ] T ⊗ D S − [ X { ,i }∈ E ( S ) d i , . . . , X { n ,i }∈ E ( S ) d i ] T ⊗ A S ) v S j k . Now, if we denote B = [ d S , . . . , d S n ] T ⊗ D S − [ P { ,i }∈ E ( S ) d S i , . . . , P { n ,i }∈ E ( S ) d S i ] T ⊗ A S and A S = [ a i,j ], 1 ≤ i, j ≤ n , then we can easily conclude that B = B B ... B n , B k = d S k d S i , if i = j − X { k,l }∈ E ( S ) d S l a i,j , if i = j , ≤ k ≤ n . Therefore, we obtain that k Bv S j k = k [ d S , . . . , d S n ] T ⊗ ( D S v S j ) k + k [ X { ,i }∈ E ( S ) d S i , . . . , X { n ,i }∈ E ( S ) d S i ] T ⊗ ( − A S v S j ) k≤ k [ d S , . . . , d S n ] T k ⊗ k D S v S j k + k [ X { ,i }∈ E ( S ) d S i , . . . , X { n ,i }∈ E ( S ) d S i ] T k ⊗ k ( A S v S j ) k . (10)Furthermore, according to the inequality between the arithmetic mean and root mean square( P { k,i }∈ E ( S ) d S i ) ≤ d S k P { k,i }∈ E ( S ) d S i , for 1 ≤ k ≤ n , the following inequalities holds k [ X { ,i }∈ E ( S ) d S i , . . . , X { n ,i }∈ E ( S ) d S i ] T k ≤ X { i,j }∈ E ( S ) d S i d S j ≤ X { i,j }∈ E ( S ) d S i + d S j = n X i =1 d S i = k [ d S , . . . , d S n ] T k . (11)The equality holds true if and only if d S = . . . = d S n . Now, according to the inequalities (9), (10) and(11) we conclude that k L S ⊗ S ( D S S ⊗ v S j ) k = k Bv S j k ≤ n X i =1 q d S i ( k D S v S j k + k A S v S j k ) . rom the fact that k L S v S j k = k ( D S − A S ) v S j k = k D S v S j k + k A S v S j k we finally have that k L S ⊗ S ( D S S ⊗ v S j ) k ≤ vuut n X i =1 d S i k L S v S j k . (cid:3) Theorem 3
The correlation coefficients r ′ ,j ( ≤ j ≤ n ) corresponding to the vectors L S ⊗ S ( D S S ⊗ v S j ) and D S S ⊗ v S j are greater than or equal to d S + · · · + d S n q ( d S + · · · + d S n )( d S + · · · + d S n ) r S j , where r S j is the correlation coefficient corresponding to the vectors L S v S j and v S j . Proof.
According to Lemma 1 and Lemma 2 we obtain that r ′ ,j = h L S ⊗ S ( D S S ⊗ v S j ) , D S S ⊗ v S j ik L S ⊗ S ( D S S ⊗ v S j ) k · k D S S ⊗ v S j k≥ ( d S + · · · + d S n ) v S j T L S v S j q d S + · · · d S n k L S v S j k q d S + · · · d S n k v S j k = d S + · · · + d S n q d S + · · · d S n q d S + · · · d S n · v S j T L S v S j k L S v S j kk v S j k . (cid:3) Let us mention that the sum of cubes of vertex degrees of a graph G is known as the forgotten topologicalindex denoted by F ( G ) [24], while the sum of squares of vertex degrees of a graph G represents well knownfirst Zagreb index, denoted by M ( G ) [25]. In the following statement we actually prove that the expectedvalue of M ( G ) √ mF ( t ) is greater than or equal to the expected value of correlation coefficient r ,j for the randomgraphs in the asymptotic case. However, it can be shown that M ( G ) √ mF ( t ) ≥ r ,j does not always hold for anarbitrary graph and it would be nice to find the minimum of the function M ( G ) √ mF ( t ) r ,j . This would make amore elegant expression for the upper bound for F ( G ) than those that can be found in the literature [26]. Theorem 4
The asymptotic value of the expected value of the correlation coefficient r ,j is less than or equalto the asymptotic value of the expected value of d S + · · · + d S n q ( d S + · · · + d S n )( d S + · · · + d S n ) , as n → ∞ . Proof.
According to Theorem 2 we have that the asymptotic value of expected value of the correlationcoefficient r ,j is equal to q ( n − p − p +( n − p , as n → ∞ . On the other hand, as P ( d S i = k ) = (cid:0) n − k (cid:1) p k (1 − p ) n − k − we have that E ( Y ) = n ( n − p for Y = d S + . . . + d S n . Similarly, we can conclude that E ( Y ) = n (( n − p (1 − p ) + ( n − p ) for Y = d S + · · · + d S n and E ( Y ) = n (( n − n − n − p + 3 p ( n − n −
2) + ( n − p ) for Y = d S + · · · + d S n . Using Proposition 1 we can conduct he similar proof as we do in Theorem 2 and conclude that that asymptotic value of the expected value of d S
11 2 + ··· + d S n q ( d S
11 3 + ··· + d S n )( d S + ··· + d S n ) , as n → ∞ , is equal to E ( Y ) √ E ( Y ) E ( Y ) . It only remains to show that n (( n − p (1 − p ) + ( n − p ) p n ( n − p n (( n − n − n − p + 3 p ( n − n −
2) + ( n − p ) ≥ s ( n − p − p + ( n − p . After a short calculation, the inequality can be reduced to s (1 − p + ( n − p ) ( n − n − n − p + 3( n − n − p + ( n − p ≥ , which is equivalent to ( n − p − n − p + (2 n − p + 1 ≥
0. Furthermore, this can be rewritten inthe following way n ≥ p − p + 5 p − p − p + 2 p = 2 + p − p − p + 2 p = 2 + p − p ( p − p −
2) = 2 − p (2 − p ) . The arithmetic-geometric mean inequality implies that p (2 − p ) ≤ ( p +2 − p ) = 1 and therefore we get n ≥ ≥ − p (2 − p ) , which is obviously true. (cid:3) According to Theorem 3 and Theorem 4 we have the following chain of inequalities E ( r ′ ,j ) ≥ E ( d S + · · · + d S n q ( d S + · · · + d S n )( d S + · · · + d S n ) r S j ) ≥ E ( r ,j ) E ( r S j ) , as n → ∞ . Moreover, we see that the lower bound of r ′ ,j depends on the degrees of S and the correlationcoefficient r S j , while r ,j depends only on the degrees of S . Therefore, for the higher values r S it willbe more likely that the expected values of r ′ ,j is greater than the expected values of r ,j . In fact, if wechoose S to be the graph such that r S j is close to 1 (if S is regular then r S j = 1) we can conclude that E ( r ′ ,j ) ≥ E ( r ,j ), for every 1 ≤ j ≤ n , as n → ∞ . Furthermore, we show the distributions of percentage errors in estimated Laplacian spectra of the Kroneckerproduct of graphs compared to the actual spectrum. The error is calculated over one hundred independenttests for the Kronecker product of the Erd˝os-R´enyi random graphs with 50 and 30 vertices. The errors forthe estimated spectrum corresponding to the eigenvectors w S i ⊗ w S j are always drawn on the left hand side,while the errors for the estimated spectrum corresponding to the eigenvectors v S i ⊗ v S j are always drawnon the right hand side of Figure 4. Each row of the figure corresponds to one of the edge density levelsof 10%, 30%, and 65%, respectively. The solid black curve shows the median, and the shaded areas showranges from 5 to 95 percentiles. Notice that when the edge density increases, the percentage errors becomesmaller for both approximations. The characteristic shapes of error distributions for the estimated spectrumcorresponding to the eigenvectors w S i ⊗ w S j , seen in Figure 4 (left hand side) have sudden jumps at thebeginning followed by a gradual decrease and they are fairly consistent across various network density levelsthat we tested. There is no a sudden jump at the beginning, for the estimated spectrum corresponding tothe eigenvectors v S i ⊗ v S j , but there is a small error widening for the largest eigenvalues. In the case of theestimated spectrum corresponding to the eigenvectors w S i ⊗ w S j , the median takes positive values for theapproximately first half of eigenvalues and negative values for the second half. In the case of the estimatedspectrum corresponding to the eigenvectors v S i ⊗ v S j , the distribution of percentage errors becomes morestable, that is, the median is almost a straight line with value 0 for every eigenvalue (right hand side ofFigure 4). It could be seen that the error ranges are almost uniformly distributed around 0.
500 1000 1500−40−32−24−16−80816 eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r Figure 4: Distribution of percentage errors in estimated Laplacian spectra of the Kronecker productof Erd˝os-R´enyi random graphs (50 and 30 vertices) compared to original ones.
Left hand side isreserved for the spectrum of the vectors w S i ⊗ w S j and right hand side for the spectrum of thevectors v S i ⊗ v S j . Rows correspond to the edge density levels of 10%, 30% and 65%.15 ere we give a theoretical explanation of why the estimated eigenvalues corresponding to v S i ⊗ v S j forthe random graphs become more accurate to the real expected values when the network grows or the edgedensity level increases. Conducted experiments show that this approximation produces reasonable estimationof Laplacian spectra with percentage errors confined within a ± ±
5% and ±
2% range for most eigenvalueswhen the edge density percentages are 10%, 30% and 65%, respectively. We use the following statement inorder to show a theoretical justification for the above claim.
Theorem 5 [27] Let G be a random graph, where pr ( v i ∼ v j ) = p ij , and each edge is independent of eachother edge. Let A be the adjacency matrix of G , and ¯ A = E ( A ) , so A ij = p ij . Let D be the diagonal matrixwith D ii = deg ( v i ) , and ¯ D = E ( D ) . Let ¯ δ = ¯ δ ( G ) be the minimum expected degree of G , and L the normalizedLaplacian matrix for G . For any ǫ > , if there exists a constant k = k ( ǫ ) such that ¯ δ > k ln n , then withprobability at least − ǫ , the j -th eigenvalues of L and ¯ L satisfy | λ j ( L ) − λ j ( ¯ L ) | ≤ s ln ( nǫ )¯ δ for all ≤ j ≤ n , where ¯ L = I − ¯ D − ¯ A ¯ D − . Let G n,p be a random graph with order n and probability of creation of an edge p . Since in the experimentswe use the factor graphs with the same edge density percentage (denote these graphs by G n ,p and G n ,p ),without loss of generality, we may set p = p = p (an identical analysis can be conducted when p = p ). Forthe expected adjacency matrices of the random graphs G n ,p and G n ,p hold ¯ A n = p ( J n − I n ) and ¯ A n = p ( J n − I n ). By A n and A n we denote the adjacency matrices of G n ,p and G n ,p . By L ( G n ,p ⊗ G n ,p )we also denote the normalized Laplacian matrix for the graph G n ,p ⊗ G n ,p .First, we show that ¯ δ = ¯ δ ( G n ,p ⊗ G n ,p ) ∼ n n . Notice also that since the sum of each row of thematrix ¯ A n ⊗ ¯ A n is equal to p ( n − n − δ ( ¯ G n ,p ⊗ ¯ G n ,p ) = p ( n − n − Z = min { d i d k | ≤ i ≤ n ≤ k ≤ n } , where d i and d k are the degrees of the vertices in G n ,p and G n ,p , respectively. Therefore, we have that ¯ δ = E ( Z ). According to Jensen’s inequality, it holds that e − t ¯ δ ≤ E ( e − tZ ), for any positive real t . Furthermore, according to the definition of Z , we have the followingchain of relation e − t ¯ δ ≤ E ( e − tZ ) = E ( e − t min i,j { d i d k } ) = E (max i,j e − td i d k ) ≤ X i,j E ( e − td i d k ) = n n E ( e − td i d k ) , (12)for any 1 ≤ i ≤ n and 1 ≤ k ≤ n .As n , n → ∞ , according to the central limit theorem d i and d k have asymptotic normal distri-bution AN ( µ , σ ) and AN ( µ , σ ), respectively, where µ = n p , µ = n p , σ = √ n pq and σ = √ n pq . Considering the two dimensional variable X = [ d i , d k ] it can be concluded that it has asymp-totic normal distribution and since g ( x, y ) = e − txy is a continuously differentiable function we conclude that g ( X ) has an asymptotic normal distribution. Therefore, when n , n → ∞ , we have that E ( e − td i d k ) = πσ σ R ∞−∞ R ∞−∞ e − txy e ( x − µ σ ) e ( y − µ σ ) dxdy . After the substitutions x → x − µ σ , y → y − µ σ and certain num-ber of elementary algebraic transformations we obtain that E ( e − td i d k ) = e − tµ µ π Z ∞−∞ e − tµ σ y − y e t σ µ σ y )22 Z ∞−∞ e ( x + tσ µ σ y ))22 dxdy = e − tµ µ √ π Z ∞−∞ e − tµ σ y − y + t σ µ σ y )22 dy. The last integral can be rewritten in the following form √ π e − tµ µ e t σ µ R ∞−∞ e − y A − yB , where A = − t σ σ and B = − tµ σ + t σ µ σ . Finally, we have that E ( e − td i d k ) = 1 √ π e − tµ µ + t σ µ e B A Z ∞−∞ e − ( √ A ( y − BA ))22 dy = 1 √ π e − tµ µ + t σ µ + B A √ π √ A = e − tµ µ + t σ µ + ( − tµ σ t σ µ σ − t σ σ p − t σ σ = e − tµ µ µ σ µ σ t − t σ σ p − t σ σ . According to (12) we obtain¯ δ ≥ − ln ( n n ) t − − µ µ + ( µ σ + µ σ ) t − t σ σ ) + ln (1 − t σ σ )2 t , for every t >
0. Now, if we set t = µ σ , we can easily obtain that the leading summand of the right handside of the above inequality is µ µ , hence we further conclude that ¯ δ = Ω( n n ), when n , n → ∞ .Let Spectrum ( ¯ A n ), Spectrum ( ¯ A n ), Spectrum ( ¯ A n ⊗ ¯ A n ) and Spectrum ( L ( ¯ A n ⊗ ¯ A n )) be the mul-tisets of the eigenvalues of the matrices ¯ A n , ¯ A n , ¯ A n ⊗ ¯ A n and L ( ¯ A n ⊗ ¯ A n ), respectively. In order tocalculate Spectrum ( L ( ¯ A n ⊗ ¯ A n )), we need to determine the diagonal matrix D ( ¯ A n ⊗ ¯ A n ), Spectrum ( ¯ A n ), Spectrum ( ¯ A n ) and Spectrum ( ¯ A n ⊗ ¯ A n ), but for the sake of simplicity, these steps are skipped. So, thenormalized Laplacian spectrum of the expected adjacency matrix of the Kronecker product of two randomgraphs consists of (cid:18) n − n − n − n − n n − n n − − n − n − (cid:19) (13)where the second row represents the eigenvalues, while the first row represents the corresponding algebraicmultiplicities.Since δ = Ω( n n ) ≫ ln ( n n ), we can apply Theorem 5 by putting ǫ = √ n n and obtain | λ j ( L ( G n ,p ⊗ G n ,p )) − λ j ( L ( ¯ A n ⊗ ¯ A n )) | ≤ s ln ln ( n n )2 n n = o (1) , (14)with probability greater than or equal to 1 − √ n n = 1 − o (1).In the following, we estimate the difference between d i d k and ¯ δ by using Chebyshev’s inequality, i.e. P r ( | d i d k − ¯ δ | < ǫσ ( d i d k )) ≥ − ǫ , for any real ǫ >
0. Since d i and d k are independent, we have that σ ( d i d k ) = µ σ + µ σ + σ σ . Therefore, for ǫ = √ n n it can be concluded that | d i d k − ¯ δ | < q √ n n ( µ σ + µ σ + σ σ )with probability greater than or equal to 1 − √ n n = 1 − o (1). Furthermore, since 0 ≤ λ j ( L ( G n ,p ⊗ G n ,p )) ≤ λ j ( L ( ¯ A n ⊗ ¯ A n )) = ¯ δλ j ( L ( ¯ A n ⊗ ¯ A n )) = ¯ δO (1) = Ω( n n ) O (1), which follows from the formula L = D L D and the property that the graph ¯ G n ,p ⊗ ¯ G n ,p is regular, it holds that | d i d k − ¯ δ | λ j ( L ( G n ,p ⊗ G n ,p )) λ j ( L ( ¯ A n ⊗ ¯ A n )) < p √ n n ( µ σ + µ σ + σ σ )Ω( n n ) O (1) = o (1) . (15)On the other hand, by multiplying both hand sides of the inequality (14) with ¯ δ and dividing by λ j ( L ( ¯ A n ⊗ ¯ A n )), we obtain | ¯ δλ j ( L ( G n ,p ⊗ G n ,p )) − λ j ( L ( ¯ A n ⊗ ¯ A n )) | λ j ( L ( ¯ A n ⊗ ¯ A n )) ≤ ¯ δ o (1)¯ δO (1) = o (1) . (16) y adding the inequalities (15) and (16), we finally conclude that | d i d k λ j ( L ( G n ,p ⊗ G n ,p )) − λ j ( L ( ¯ A n ⊗ ¯ A n )) | λ j ( L ( ¯ A n ⊗ ¯ A n ))= | d i d k λ j ( L ( G n ,p ⊗ G n ,p )) − ¯ δλ j ( L ( G n ,p ⊗ G n ,p )) + ¯ δλ j ( L ( G n ,p ⊗ G n ,p )) − λ j ( L ( ¯ A n ⊗ ¯ A n )) | λ j ( L ( ¯ A n ⊗ ¯ A n )) ≤ | d i d k λ j ( L ( G n ,p ⊗ G n ,p )) − ¯ δλ j ( L ( G n ,p ⊗ G n ,p )) | + | ¯ δλ j ( L ( G n ,p ⊗ G n ,p )) − λ j ( L ( ¯ A n ⊗ ¯ A n )) | λ j ( L ( ¯ A n ⊗ ¯ A n )) = o (1) . In the previous formula we show that percentage error between the estimated spectra and the spectraof Laplacian of expected Kronecker random graph tends to zero, when n and n tend to infinity, whilein the performed experiments we calculate the percentage error between the estimated and actual spectra(estimated spectra is given by (5)). Therefore, in the rest of the section we give an asymptotic estimate ofthe percentage error between the estimated spectra and the mean of the eigenvalues of Laplacian matrix.Indeed, some empirical evidence indicate that the mean of the empirical distribution of the eigenvalues ofthe Laplacian matrix of G ( n, p ) is centered around np (see [28]). Similarly, if we denote mean of the empiricaldistribution of the eigenvalues of the Laplacian matrix of G ( n , p ) ⊗ G ( n , p ) by ¯ λ , we can conclude that¯ λ ∼ n n and therefore | d i d k λ j ( L ( G n ,p ⊗ G n ,p )) − ¯ λ ( L ( A n ⊗ A n )) | ¯ λ ( L ( A n ⊗ A n )) = o (1) . (17)Therefore, in that case we conclude that the formula (17) represents the percentage error of the estimatedspectrum d i d k λ j ( L ( G n ,p ⊗ G n ,p )) from (5), which tends to 0 when the order of the graph or its edge densitytends to infinity. Watts-Strogatz random graphs
Similarly, we apply the same experiments when two graphs are Watts-Strogatz graphs. By examining thespectral properties of the Kronecker product of graphs that are Watts-Strogatz graphs, we notice that thesituation is a bit different since even when the graphs are sparse (edge density level is 10%), the smoothedprobability density functions of the vector correlation coefficients are shrank toward the value of 1, for bothapproximations. For the same density, peaks for both approximations are located in the interval (0 . , v S i ⊗ v S j is almost uniformly distributed around 0 for each tested edgedensity, while the distribution of percentage errors of the estimated spectrum corresponding to the eigenvec-tors w S i ⊗ w S j always has a sudden jump at the beginning. In Figure 6, errors for the estimated spectrumfrom Subsection 2.1 are drawn on the left side, while errors for the estimated spectrum from Subsection 2.2on the right side are drawn. As in case of the Erd˝os-R´enyi random graphs, both approximations producedreasonable estimations of Laplacian spectra with percentage errors confined within a ±
10% and less as theedge density percentage becomes higher.
In this section we present a behavior of the eigenvectors and eigenvalues of the Kronecker product of twographs which are Barab´asi-Albert graphs. For this type of graph, the situation is not significantly differentcompared to the previous two types concerning correlation coefficients of the estimated eigenvalues. In allcases w S i ⊗ w S j eigenvectors express better properties, since their correlation coefficients are above 0.9 inmost of the cases, while the correlation coefficients of the v S i ⊗ v S j eigenvectors are in interval (0.7, 0.9) mostof cases (see Figure 7), for the edge density levels of 10%, 30%, and 65%. orrelation coeff. between the predicted vectors v and L * v F r equen cy Correlation coeff. between the predicted vectors v and L * v F r equen cy Correlation coeff. between the predicted vectors v and L * v0.5 0.6 0.7 0.8 0.9 1 F r equen cy Product of Laplacian eigenvectorsProduct of Normalized Laplacian eigenvectors
Figure 5: Smoothed probability density functions of vector correlation coefficients between w S i ⊗ w S j and L S ⊗ S ( w S i ⊗ w S j ) are represented using a solid green line, while between v S i ⊗ v S j and L S ⊗ S ( v S i ⊗ v S j ) are represented using a solid blue line. Watts-Strogatz random graphs have 50and 30 vertices. Probability density functions are drawn for each of the edge density level 10%, 30%and 65%, respectively. Also, we notice that the estimated eigenvalues corresponding to the eigenvectors w S i ⊗ w S j are morestable than the eigenvalues corresponding to the eigenvectors v S i ⊗ v S j . From Figure 8 it can be noticedthat the error ranges (which correspond to edge density levels of 10%, 30% and 65%) between the estimatedand original spectrum are less for the first approximation (left panels) than for the second one, which are atthe same time more distorted (right panels). The characteristic shape of error distribution for the estimatedspectrum corresponding to the eigenvectors w S i ⊗ w S j remains similar as in the previous subsection. Thisincludes a sudden jump at the beginning followed by a gradual decrease across various network density levelswe tested. Unlike the previous subsection, the estimated spectrum corresponding to the eigenvectors v S i ⊗ v S j has a sudden jump at the beginning and the error ranges are a little bit higher for all edge densities (rightpanels of Figure 8). When the edge density is from 50% to 65%, the characteristic shape for the secondapproximation (right panels) is a bit different than usual. It could be noticed sudden jump in the middle ofgraphs, but in the same time error narrowing for the largest eigenvalues. Although the relationships between spectral properties of a product graph and those of its factor graphshave been known for the standard products, characterization of Laplacian spectrum and eigenvectors of theKronecker product of graphs using the Laplacian spectra of the factors has remained an open problem todate. In this work we proposed a novel approximation method for estimating the Laplacian spectrum andthe corresponding eigenvectors of the Kronecker product of graphs knowing the eigenvalues and eigenvectorsof factor graphs. The estimated eigenvalues and eigenvectors were compared to the original ones with regardto different types of random networks and theirs edge density levels. Moreover, the properties of the novelapproximation were compared with the approximation proposed by Sayama. Although both approximationswere designed using a few mathematically incorrect assumptions, the obtained estimations of the spectra arevery close to the numerically calculated spectra with percentage errors constrained within a ±
10% range formost eigenvalues. Here, we give a theoretical explanation of why the estimated eigenvalues for the randomgraphs become more accurate to the real values when the network grows or the edge density level increases.This explains the fact that a distribution of percentage errors between estimated and original spectra becomesalmost uniformly distributed around 0. In this paper we also presented some novel theoretical results relatedto the certain correlation coefficients corresponding to the estimated and original vectors. Here, we providean exact formula of how some of these correlation coefficients can be explicitly calculated, as well as theirexpected values for some types of random networks.
500 1000 1500−50−40−30−20−100102030 eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r Figure 6: Distribution of percentage errors in estimated Laplacian spectra of the Kronecker prod-uct of Watts-Strogatz graphs (50 and 30 vertices) compared to original ones.
Left hand side isreserved for the spectrum of the eigenvectors w S i ⊗ w S j and right hand side for the spectrum of theeigenvectors v S i ⊗ v S j . Rows correspond to the edge density levels of 10%, 30% and 65%.20 orrelation coeff. between the predicted vectors v and L * v F r equen cy Correlation coeff. between the predicted vectors v and L * v F r equen cy Correlation coeff. between the predicted vectors v and L * v F r equen cy Figure 7: Smoothed probability density functions of vector correlation coefficients between w S i ⊗ w S j and L S ⊗ S ( w S i ⊗ w S j ) are represented using a solid green line, while between v S i ⊗ v S j and L S ⊗ S ( v S i ⊗ v S j ) are represented using a solid blue line. Barab´asi-Albert random graphs have 50and 30 vertices. Probability density functions are drawn for each of the edge density 10%, 30% and65%, respectively. As it was mentioned earlier, in this and Sayama’s paper, these approximations have many theoreticallimitations, because of the mathematically incorrect assumptions and there is no rigorous mathematical ex-planation of why and how the proposed methods work. That is why a design of spectral estimation algorithmswill be an important direction of future research, as well as their theoretical explanations. Moreover, it wouldbe very important to see how the estimated eigenvalues and eigenvectors are suitable for complete spectraldecomposition of the graph, where all eigenvalues and eigenvectors are included to replace original ones. Ac-cording to some preliminary results we have already obtained by incorporating these approximations in theGCRF model, a good behaviour of these approximations presented in this paper have been experimentallyconfirmed too. Moreover, we obtained that the presented estimations can be a good staring point for otherapplications and further improvements of Laplacian spectrum of the Kronecker product of graphs.
500 1000 1500−60−50−40−30−20−1001020 eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r eigenvalue ord. no. % e rr o r Figure 8: Distribution of percentage errors in estimated Laplacian spectra of the Kronecker prod-uct of Barab´asi-Albert graphs (50 and 30 vertices) compared to original ones.
Left hand side isreserved for the spectrum of the eigenvectors w S i ⊗ w S j and right hand side for the spectrum of theeigenvectors v S i ⊗ v S j . Rows correspond to the edge density levels of 10%, 30%, and 65%.22 eferences [1] Nataˇsa Prˇzulj. Protein-protein interactions: Making sense of networks via graph-theoretic modeling. Bioessays , 33(2):115–123, 2011.[2] Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation.
Departmental Papers (CIS) ,page 107, 2000.[3] Branko Arsi´c, Milan Baˇsi´c, Petar Spalevi´c, Miloˇs Ili´c, and Mladen Veinovi´c. Facebook profiles clustering.In
Proceedings of the 6th International Conference on Information Society and Technology (ICIST 2016) ,pages 154–158, Kopaonik, Serbia, 2016. Society for Information Systems and Computer Networks.[4] Milan Baˇsi´c. Which weighted circulant networks have perfect state transfer?
Information Sciences , 257:193–209, 2014.[5] Ron Milo, Shai Shen-Orr, Shalev Itzkovitz, Nadav Kashtan, Dmitri Chklovskii, and Uri Alon. Networkmotifs: simple building blocks of complex networks.
Science , 298(5594):824–827, 2002.[6] Michelle Girvan and Mark EJ Newman. Community structure in social and biological networks.
Pro-ceedings of the national academy of sciences , 99(12):7821–7826, 2002.[7] Manlio De Domenico, Albert Sol´e-Ribalta, Emanuele Cozzo, Mikko Kivel¨a, Yamir Moreno, Mason APorter, Sergio G´omez, and Alex Arenas. Mathematical formulation of multilayer networks.
PhysicalReview X , 3(4):041022, 2013.[8] Alex Arenas, Jordi Duch, Alberto Fern´andez, and Sergio G´omez. Size reduction of complex networkspreserving modularity.
New Journal of Physics , 9(6):176, 2007.[9] Per Sebastian Skardal and Juan G Restrepo. Hierarchical synchrony of phase oscillators in modularnetworks.
Physical Review E , 85(1):016208, 2012.[10] Jure Leskovec, Deepayan Chakrabarti, Jon Kleinberg, Christos Faloutsos, and Zoubin Ghahramani.Kronecker graphs: An approach to modeling networks.
Journal of Machine Learning Research , 11(Feb):985–1042, 2010.[11] Albert Sole-Ribalta, Manlio De Domenico, Nikos E Kouvaris, Albert Diaz-Guilera, Sergio Gomez, andAlex Arenas. Spectral properties of the laplacian of multiplex networks.
Physical Review E , 88(3):032807, 2013.[12] Mikko Kivel¨a, Alex Arenas, Marc Barthelemy, James P Gleeson, Yamir Moreno, and Mason A Porter.Multilayer networks.
Journal of complex networks , 2(3):203–271, 2014.[13] Robert Els¨asser, Burkhard Monien, Robert Preis, and Andreas Frommer. Optimal diffusion schemesand load balancing on product graphs.
Parallel Processing Letters , 14(01):61–73, 2004.[14] Dragoˇs M Cvetkovi´c and Ivan Gutman.
Selected topics on applications of graph spectra . Matematickiinstitut SANU, 2011.[15] Dragoˇs Cvetkovi´c and Slobodan Simi´c. Graph spectra in computer science.
Linear Algebra and itsApplications , 434(6):1545–1562, 2011.[16] Hiroki Sayama. Estimation of laplacian spectra of direct and strong product graphs.
Discrete AppliedMathematics , 205:160–170, 2016.[17] Andries E Brouwer and Willem H Haemers.
Spectra of graphs . Springer Science & Business Media, 2011.[18] Dragoˇs M Cvetkovi´c, Michael Doob, and Horst Sachs.
Spectra of graphs: theory and application , vol-ume 87. Academic Pr, 1980.
19] Russell Merris. Laplacian matrices of graphs: a survey.
Linear algebra and its applications , 197:143–176,1994.[20] Jesse Glass and Zoran Obradovic. Structured regression on multiscale networks.
IEEE IntelligentSystems , 32(2):23–30, 2017.[21] Sasmita Barik, Ravindra B Bapat, and S Pati. On the laplacian spectra of product graphs.
ApplicableAnalysis and Discrete Mathematics , pages 39–58, 2015.[22] Chris Godsil and Gordon F. Royle.
Algebraic Graph Theory . Graduate Texts in Mathematics. SpringerNew York, 2001. ISBN 9780387952413.[23] Peter J. Brockwell and Richard A. Davis.
Time Series: Theory and Methods . Graduate Texts inMathematics. Springer, 1991. ISBN 978-1-4419-0319-8.[24] Boris Furtula and Ivan Gutman. A forgotten topological index.
J. Math. Chem. , 53:1184–1190, 2015.[25] Ivan Gutman and Kinkar C. Das. The first zagreb index 30 years after.
MATCH Commun.Math.Comput. Chem. , 50:83–92, 2004.[26] Zhongyuan Che and Zhibo Chen. Lower and upper bounds of the forgotten topological index.
MATCHCommun. Math. Comput. Chem , 76(3):635–648, 2016.[27] Fan Chung and Mary Radcliffe. On the spectra of general random graphs.
The electronic journal ofcombinatorics , 18(1):P215, 2011.[28] Olivier Leveque. https://mathoverflow.net/questions/176689/limiting-empirical-spectral-distribution-of-the-laplacian-matrix-on-an-erdos-ren . Mathoverflow, 2014.. Mathoverflow, 2014.