Clique Gossiping
CClique Gossiping
Yang Liu, Bo Li, Brian D. O. Anderson, and Guodong Shi ∗ Abstract
This paper proposes and investigates a framework for clique gossip protocols. As complete subnet-works, the existence of cliques is ubiquitous in various social, computer, and engineering networks. Byclique gossiping, nodes interact with each other along a sequence of cliques. Clique-gossip protocols aredefined as arbitrary linear node interactions where node states are vectors evolving as linear dynamicalsystems. Such protocols become clique-gossip averaging algorithms when node states are scalars underaveraging rules. We generalize the classical notion of line graph to capture the essential node inter-action structure induced by both the underlying network and the specific clique sequence. We provea fundamental eigenvalue invariance principle for periodic clique-gossip protocols, which implies thatany permutation of the clique sequence leads to the same spectrum for the overall state transitionwhen the generalized line graph contains no cycle. We also prove that for a network with n nodes,cliques with smaller sizes determined by factors of n can always be constructed leading to finite-timeconvergent clique-gossip averaging algorithms, provided n is not a prime number. Particularly, suchfinite-time convergence can be achieved with cliques of equal size m if and only if n is divisible by m and they have exactly the same prime factors. A proven fastest finite-time convergent clique-gossipalgorithm is constructed for clique-gossiping using size- m cliques. Additionally, the acceleration effectsof clique-gossiping are illustrated via numerical examples. Gossip protocols provide a scalable and self-organized way of carrying out information disseminationover networks in the absence of centralized decision makers [1–6]. In a gossip process, a pair of nodes isselected randomly or deterministically at any given time, and then this pair of nodes gossip by exchanginginformation between each other as a fundamental resource allocation protocol for computer networks [7,8].Today, gossip processes are natural models for interpersonal interactions and opinion evolutions in socialnetworks [9]; distributed systems running gossip protocols have been developed to realize in-networkcontrol [10], filtering [11], signal processing [12], and computation [13]. ∗ Y. Liu, B. D. O. Anderson, and G. Shi are with the Research School of Engineering, The Australian National University,ACT 0200, Canberra, Australia. B. Li is with Key Lab of Mathematics Mechanization, Chinese Academy of Sciences, Beijing100190, China. Email: [email protected], [email protected], [email protected], [email protected]. a r X i v : . [ c s . D C ] J un articularly, gossip averaging algorithms serve as a basic model for gossip protocols, where during onegossip interaction the two involved nodes average their states, which are simply real numbers [14, 15]. Therate of convergence of such gossip averaging algorithms can represent the performance of gossip protocolsthat are built on top of that, and quantify efficiency and robustness of the underlying network structure.For random gossip algorithms, various results reveal that the network structure plays a critical role inshaping the convergence speed in the asymptotic sense [14,15]. With deterministic gossiping, scheduling ofthe gossiping pairs becomes equally influential [16]; indeed, even finite-time convergence can be achievedwith suitable number of nodes [17]. It is worth mentioning that in certain cases transitions can be madebetween deterministic and random gossip algorithms, where the Borel-Cantelli lemma provides immediateconnections [18].In this paper, we propose and investigate a framework involving clique gossip protocols, where simul-taneous node interactions can take place among cliques instead of being restricted to pairs. Cliques aresubnetworks that form a complete graph in their local topologies, whose existence is universal in social,computer, and engineering networks. In fact, the use of cliques for beamforming and clustering has beenemployed in wireless sensor networks [19, 20]. In a general model, each node holds a vector state at anygiven time, and clique-gossip protocols are arbitrary linear node dynamical interactions along a sequenceof cliques that forms a coverage of the underlying network. When the node state vector is one-dimensionaland the node interaction rules are simply averaging, the clique-gossip protocol is reduced to a clique-gossipaveraging algorithm. To facilitate the analysis of the network structure that governs the node interactions,we first generalize the classical notion of line graph in graph theory. Then our contributions are madethrough a few important convergence properties of clique gossiping: • We prove a fundamental eigenvalue invariance principle for scheduling periodic clique-gossip pro-tocols, valid for arbitrary clique-gossip protocols represented by linear dynamical systems. Such aninvariance principle implies that any permutation of the clique sequence leads to the same spectrumfor the overall state transition matrix if the generalized line graph associated with the clique-gossipalgorithm contains no cycle. • We prove that for a network with n nodes, there always exist ways of constructing cliques withsmaller sizes leading to finite-time convergent clique-gossip averaging algorithms, provided that n isnot a prime number. We also prove that such finite-time convergence can be achieved with cliquesof equal size m if and only if n is divisible by m and they have exactly the same prime factors.It is worth mentioning that for clique gossiping with equal size m cliques, we have constructed one of thefastest finite-time convergent clique-gossip algorithms, which is shown to reach the fundamental complexitylower bound by elementary number theory. Additionally, we illustrate how multi-clique gossiping can bebuilt over an existing clique-gossiping protocol, and the acceleration effects of clique-gossiping are shownvia numerical examples. 2he remainder of this paper is organized as follows. Section 2 presents the clique-gossip model. Sec-tion 3 studies periodic clique-gossip protocols by establishing the eigenvalue invariance principle andinvestigating the rate of convergence. Section 4 focuses on the possibilities of finite-time convergence forclique-gossip averaging algorithms. Finally Section 5 concludes the paper with a few remarks on potentialfuture directions. Consider a group of nodes whose interaction structure is described by a simple undirected graph G =(V , E), where V = { , , . . . , n } is the node set and an element ( i, j ) ∈ E is an unordered pair of twodistinct nodes i, j ∈ V. Define the neighbor node set N i of node i by N i = { j : ( i, j ) ∈ E } . Associatedwith a node subset S ⊂ V, its induced graph G[S] is defined as the graph with node set S and the edgeset containing all edges in E with both endpoints in S. A clique C is a subset of V whose induced graphG[C] is a complete graph. Let H G be the set containing all the cliques of G. We index the elements in H G as C , . . . , C D . We say that two cliques C i , C j ∈ H G with i (cid:54) = j are adjacent if C i (cid:84) C j (cid:54) = ∅ .Figure 1: A connected graph G. Here C = { , , } , C = { , } , C = { , , , } , C = { , , , } , C = { , , } , C = { , , } , C = { , } . Clearly { C , C , . . . , C } is a clique coverage. Links within thesame clique are marked with the same color and style. Definition 1.
A subset H ∗ G = { C µ , . . . , C µ d } ⊂ H G with d ≥ is called a clique coverage of G if (cid:83) C i ∈ H ∗ G C i = V and the union graph (cid:83) C i ∈ H ∗ G G[C i ] is connected. Note that every connected graph has a clique coverage. Let H ∗ G be the collection of pairs of the endpointsof all edges in G. Then H ∗ G is clearly a clique coverage.3 .2 Clique-gossip Protocols Let H ∗ G = { C µ , . . . , C µ d } be a clique coverage of G where µ k ∈ { , . . . , D } for k = 1 , . . . , d . Each node i ∈ V of G holds a vector x i ( k ) ∈ R b evolving at discretized time t = 0 , , , . . . . For each clique C µ l ∈ H ∗ G ,we assign a matrix A ij ( µ l ) ∈ R b × b to edge ( i, j ) ∈ E for all i, j ∈ C µ l and A ii ( µ l ) ∈ R b × b to each node i ∈ C µ l . Introduce a function σ ( · ) : Z ≥ → { µ , µ , . . . , µ d } . We define a clique gossip protocol over thegraph G as follows. Definition 2. (Clique-gossip Protocol) At each time t = 0 , , . . . , one clique C σ ( t ) ∈ H ∗ G is selected. Thenodes update their states by x i ( t + 1) = (cid:88) j ∈ C σ ( t ) A ij ( σ ( t )) x j ( t ) if i ∈ C σ ( t ) ; x i ( t ) if i / ∈ C σ ( t ) . Note that the signal σ ( · ) plays a role in selecting a clique gossip process which can be deterministic orrandom. After C σ ( t ) is determined at time t , the nodes within the clique C σ ( t ) interact with each other asspecified by the state transition matrices A ij ( µ l ). We remark that at this point we are not imposing anyconditions on the A ij ( µ l ), whose choices depend on the requirements for individual problems. For eachclique C µ l ∈ H ∗ G , we define a block matrix M µ l ∈ R nb × nb whose diagonal blocks equal I b except the ii thblock is A ii ( µ l ) for all i ∈ C µ l , and off-diagonal blocks equal b except the ij th block is A ij ( µ l ) and the ji th block is A ij ( µ l ) for all i, j ∈ C µ l , i (cid:54) = j . Then the above clique-gossip protocol can be put in vectorform equivalently x ( t + 1) = M σ ( t ) x ( t ) , (1)where x ( t ) = [ x ( t ) (cid:62) . . . x n ( t ) (cid:62) ] (cid:62) ∈ R nb .Therefore, by our definition a clique gossip protocol can be any linear dynamical system that runs overthe network G, under which the node interactions take place along a sequence of cliques. Practically ofcourse we would like the system (1) to asymptotically converge, preferably to some intrinsically nontriviallimits as solvers to certain network computation problems. This leads us to wonder how we can designthe A ij ( µ l ) to meet such a criterion in practice. We present the following example as a network linearequation solver [13]. Example 1.
Consider a linear algebraic equation with respect to the unknown variable y ∈ R m Hy = z (2)with H ∈ R n × m , z ∈ R n . Then (2) can be expressed in a system of linear equations h (cid:62) i y = z i , i = 1 , . . . , n ,where h (cid:62) i ∈ R m denotes the i -th row of H and z i ∈ R is the i -th component of z . Assume that (2) has aunique solution. 4onsider an n -node graph G = (V , E) with a clique coverage H ∗ G . We let each node i ∈ V hold a linearequation h (cid:62) i y = z i and be assigned a state x i ( t ) , t = 0 , , , . . . . Suppose each node i is only permitted toshare its state with its neighbors. Inspired by the distributed linear equation solver developed in [13] byusing the conventional gossip protocol, we apply the clique-gossip protocol to solve (2) in a distributedsense as follows. At each time t = 0 , , , . . . , we choose C σ ( t ) ∈ H ∗ G . Then for those nodes i / ∈ C σ ( t ) , x i ( t + 1) = x i ( t ). For i ∈ C σ ( t ) , the update rule is x i ( t + 1) = P i ( (cid:88) j ∈ C σ ( t ) x j ( t ) / | C σ ( t ) | − x i ( t )) + x i ( t )= ( I m − ( | C σ ( t ) | − / | C σ ( t ) | P i ) x i ( t ) + (cid:88) j ∈ C σ ( t ) ,j (cid:54) = i P i x j ( t ) / | C σ ( t ) | , (3)where P i = I m − h i h (cid:62) i / ( h (cid:62) i h i ) ∈ R m × m denotes the projection matrix to the kernel of h (cid:62) i . Now we seefrom (3) that the solver is an instance of the clique-gossip protocol by letting A ij ( σ ( t )) = I m − ( | C σ ( t ) | − / | C σ ( t ) | P i if i = j, i, j ∈ C σ ( t ) ; P i / | C σ ( t ) | if i (cid:54) = j, i, j ∈ C σ ( t ) , which in turn determines a particular M σ ( t ) . One can easily prove that the distributed linear equationsolver developed using the clique-gossip protocol drives all nodes of the network to asymptotically computethe solution of (2) if the sequence C σ (0) , C σ (1) , C σ (2) , . . . is periodic and the elements in its subsequenceover any one period form the clique coverage H ∗ G . One primary gossip protocol comes from the case where nodes simply average their current states duringtheir meetings, leading to the so-called random or deterministic gossip algorithms. Such gossip algorithmsserve as algorithmic descriptions of node interactions over time, and the simple structure of such gossipalgorithms enables clear investigation of the convergence rates related to the underlying network structure.Therefore, despite the fact that the exact node interactions can have various different forms in real-worldgossip protocols, the corresponding gossip algorithm is a good indicator to the performance of the protocols.In the same spirit now we define a clique-gossip averaging algorithm as follows.
Definition 3. (Clique-gossip Averaging Algorithm) Let x i ( t ) ∈ R . At time t , C σ ( t ) ∈ H ∗ G is selected. Thenodes update their states by x i ( t + 1) = (cid:88) j ∈ C σ ( t ) x j ( t ) / | C σ ( t ) | if i ∈ C σ ( t ) ; x i ( t ) if i / ∈ C σ ( t ) . We can see that the clique-gossip averaging algorithm is an instance of the clique-gossip protocol bysetting A ij ( σ ( t )) = 1 / | C σ ( t ) | for all i, j ∈ C σ ( t ) , which in turn determine M σ ( t ) .5 Clique Gossip Protocols
In this section, we investigate deterministic clique-gossip protocols with periodic clique selections. For thepurpose of guaranteeing the reaching of global agreement and the formulation of an eigenvalue invariancetheorem, we introduce the following assumption on the function σ ( t ). Assumption 1. (i) σ ( · ) : Z ≥ → { µ , µ , . . . , µ d } is a periodic function with period d ; (ii) σ ( t ) visits eachelement in { µ , µ , . . . , µ d } once and only once in any period. Recall that given a graph G, its conventional line graph K (G) is defined by the requirements (i) each nodeof K (G) represents an edge of G; (ii) two nodes of K (G) are linked if and only if the corresponding edgesof G share a common endpoint. In the following, we define the generalized line graph L ( H ∗ G ) for a graphG based on the clique coverage H ∗ G . Definition 4.
Let H ∗ G be a clique coverage of G . Its generalized line graph, L ( H ∗ G ) = ( V ( H ∗ G ) , E ( H ∗ G )) ,is an undirected graph defined by V ( H ∗ G ) = { C i : C i ∈ H ∗ G } and E ( H ∗ G ) = (cid:8) (C i , C j ) ∈ V ( H ∗ G ) × V ( H ∗ G ) :C i ∩ C j (cid:54) = ∅ , i (cid:54) = j (cid:9) . A sequence C i , C i , . . . , C i k is called a path of cliques if C i j , C i j +1 are adjacent for all j = 1 , . . . , k −
1. Acycle of L ( H ∗ G ) is a path of cliques C i , C i , . . . , C i k such that C i j ∈ H ∗ G for all j = 1 , . . . , k and C i = C i k .Figure 2: The generalized line graph L ( H ∗ G ) for G given in Figure 1.Note that the generalized line graph is equivalent to the conventional line graph if every clique in theclique coverage contains two nodes. An illustration of a connected graph G is shown in Figure 1, with itsgeneralized line graph given in Figure 2. Lemma 1.
Let G be a connected graph with a clique coverage H ∗ G . Then L ( H ∗ G ) is a connected graph. roof. For two arbitrary cliques C u , C v ∈ H ∗ G , we select two nodes k u ∈ C u , k v ∈ C v of G. Then thereexists a path in the union graph (cid:83) C i ∈ H ∗ G G[C i ], denoted as ( k u , k ) , ( k , k ) , . . . , ( k n − , k n ) , ( k n , k v ), thatconnects k u and k v . For the sake of convenience, we let k = k u , k n +1 = k v . As a result, there existsa clique C j m such that ( k m , k m +1 ) is an edge in G[C j m ] for each m = 0 , . . . , n . Therefore, the sequenceC u , C j , . . . , C j n , C v , which can have consecutive repeated elements, is a path of cliques that connects C u and C v . This completes the proof. Introduce F = M σ ( d ) . . . M σ (1) (Blocks A ij ( σ ( t )) , i, j ∈ σ ( t ) in M σ ( t ) , t = 1 , . . . , d are arbitrary). Then F is the state transition matrix for the periodic gossiping protocol defined by a periodic signal σ ( · ) withperiod d . Let π ( · ) be a permutation with order d , i.e., π ( · ) is a one-to-one mapping from { , . . . , d } to { , . . . , d } . Denote F π = M σ ( π ( d )) . . . M σ ( π (1)) . This represents the state transition matrix generated by apermuted order of clique selections. Let cp( Q ) denote the characteristic polynomial for a matrix Q .Define π s , s = 1 , . . . , d − { , . . . , d } with π s ( s ) = s + 1 , π s ( s + 1) = s ,and π s ( i ) = i, i (cid:54) = s, i (cid:54) = s + 1. In the following, we present a theorem regarding the eigenvalue invarianceof the state transition matrix under swapping permutations, generalizing the result of [13]. Theorem 1.
Let H ∗ G = { C µ , . . . , C µ d } be a clique coverage of G = (V , E) and let Assumption 1 hold. Thenalong any periodic clique-gossip protocol there holds cp( F ) = cp( F π s ) if s satisfies one of the followingconditions:(i) C σ ( s ) and C σ ( s +1) are not adjacent,(ii) C σ ( s ) and C σ ( s +1) are adjacent but neither of them is contained in any cycles of L ( H ∗ G ) .Proof. It is evident for Condition (i) that M σ ( s ) M σ ( s +1) = M σ ( s +1) M σ ( s ) if C σ ( s ) and C σ ( s +1) are notadjacent. Thus, in the rest of the proof, we focus on proving cp( F ) = cp( F π s ) for s satisfying Condition(ii), i.e., F = M σ ( d ) . . . M σ ( s +2) M σ ( s +1) M σ ( s ) . . . M σ (1) , F π s = M σ ( d ) . . . M σ ( s +2) M σ ( s ) M σ ( s +1) . . . M σ (1) . Now we take three steps to complete the proof.Step 1. Since cp( AB ) = cp( BA ) for any A , B ∈ R nb × nb (Theorem 1.3.22 [26]), we havecp( F ) = cp( M σ ( s +1) M σ ( s ) M σ ( s − . . . M σ (1) M σ ( d ) . . . M σ ( s +2) ) . (4)Step 2. In this step, we reorganize the terms in the product M σ ( s − . . . M σ (1) M σ ( d ) . . . M σ ( s +2) by repeat-edly interchanging the two consecutive commutable terms. DenoteA = { j : there exists a path of cliques between C σ ( j ) and C σ ( s ) that does not pass through C σ ( s +1) } , = { k : there exists a path of cliques between C σ ( k ) and C σ ( s +1) that does not pass through C σ ( s ) } . Based on Condition (ii), there hold(i) A (cid:83) A = { s − , . . . , , d, . . . , s + 2 } ;(ii) A (cid:84) A = ∅ ;(iii) C j (cid:84) C k = ∅ for all j ∈ A , k ∈ A .Therefore, M σ ( j ) M σ ( k ) = M σ ( k ) M σ ( j ) if j ∈ A , k ∈ A and j, k are two consecutive entries in the sequence s − , . . . , , d, . . . , s + 2. Then it follows M σ ( s − . . . M σ (1) M σ ( d ) . . . M σ ( s +2) = P P = P P , (5)where P = M σ ( j | A1 | ) . . . M σ ( j ) , P = M σ ( k | A2 | ) . . . M σ ( k ) , with j | A | , . . . , j ∈ A , k | A | , . . . , k ∈ A following the same order as they are in the sequence s − , . . . , ,d, . . . , s + 2. Plugging (5) into (4), we obtaincp( F ) = cp( M σ ( s +1) M σ ( s ) P P ) . (6)Step 3. In this step, we prove cp( F ) = cp( F π ) and complete the proof. We observe that M σ ( s ) P = P M σ ( s ) , due to the fact that C σ ( s ) and C σ ( k ) are not adjacent for all k ∈ A . Then (6) yieldscp( F ) = cp( M σ ( s +1) P M σ ( s ) P ) , (7)which in turn gives cp( F ) = cp( M σ ( s ) P M σ ( s +1) P ) . (8)Similarly, we also know M σ ( s +1) P = P M σ ( s +1) because C σ ( s +1) and C σ ( j ) are not adjacent for all j ∈ A .Finally, we have cp( F ) a ) = cp( M σ ( s ) M σ ( s +1) P P ) b ) = cp( M σ ( s ) M σ ( s +1) M σ ( s − . . . M σ (1) M σ ( d ) . . . M σ ( s +2) ) c ) = cp( M σ ( d ) . . . M σ ( s +2) M σ ( s ) M σ ( s +1) . . . M σ (1) )= cp( F π s ) , where a ) follows from (8), b ) is acquired based on (5), and c ) is again due to the fact that cp( AB ) = cp( BA )for any A , B ∈ R nb × nb . This completes the proof. Corollary 1.
Let Assumption 1 hold. Then along any clique-gossip protocol there holds there holds cp( F ) =cp( F π ) for any permutation π if the generalized line graph L ( H ∗ G ) contains no cycle. roof. From [22] we know that an arbitrary permutation from { , . . . , d } to { , . . . , d } can be generatedby d − π , . . . , π d − . Therefore, we only need to prove cp( F ) = cp( F π s ) for all s = 1 , . . . , d −
1. Based on the condition of the corollary that L ( H ∗ G ) contains no cycle, either of the twoconditions in Theorem 1 is met for all s . Hence it can be concluded that cp( F ) = cp( F π s ) for all s . Thiscompletes the proof. In this section, we analyze the convergence performance of periodic clique-gossip protocols. A precisedefinition of the convergence of a clique-gossip protocol is given below.
Definition 5.
A clique-gossip protocol is convergent if there holds lim t →∞ x ( t ) = ¯ y ( x (0)) for all x (0) ∈ R nb , where ¯ y ( x (0)) is a static state depending perhaps on x (0) . Let M σ ( t ) , t = 0 , , , . . . define a clique-gossip protocol in the form of (1). For a periodic clique-gossipprotocol with period d ∈ Z + , we term F d = M σ ( d − . . . M σ (0) , as a period-based state transition matrix in view of the recursion x (( q + 1) d ) = F d x ( qd ) , q = 0 , , , . . . . Let σ ( A ) and ρ ( A ) denote the spectrum and spectral radius of a matrix A , respectively. The followinglemma holds from the basic knowledge of linear systems. Lemma 2.
Let Assumption 1 hold. Let the periodic clique-gossip protocol admit a period-based statetransition matrix F d ∈ R nb × nb . The protocol is convergent if and only if the following conditions hold:(i) ρ ( F d ) ≤ ;(ii) If ∈ σ ( F d ) , then eigenvalue one has equal algebraic multiplicity and geometric multiplicity;(iii) If λ ∈ σ ( F d ) and | λ | = 1 , then λ = 1 ;(iv) There holds M σ ( k ) . . . M σ (0) β = β for all β ∈ I := { α ∈ R nb : F d α = α } . Next, we define precisely the convergence rates of standard gossiping and clique-gossiping protocols. Note that F = M σ ( d ) . . . M σ (1) and F d = M σ ( d − . . . M σ (0) have the same spectrum regardless of the underlying networkstructure and the generalized line graph of the clique coverage. efinition 6. For a convergent clique-gossip protocol, the rate of convergence is O ( ν t ) if there exists aunique ν ∈ (0 , such that < lim sup t →∞ (cid:107) x ( t ) − ¯ y (cid:107) ν t < + ∞ , ∀ x (0) / ∈ I , where ¯ y = lim t →∞ x ( t ) . Introduce the symbol | λ ( F ) | as the magnitude of the eigenvalues of F ∈ R nb × nb with the second largestmodulus of all its eigenvalues, i.e., | λ ( F ) | = max {| λ | : λ ∈ σ ( F ) , | λ | < ρ ( F ) } . Let (cid:100) x (cid:101) be the smallestinteger greater than or equal to x ∈ R and (cid:98) x (cid:99) be the largest integer less than or equal to x ∈ R . Nowwe present a proposition that reveals the relationship between | λ ( F d ) | and the convergence rate ν forclique-gossping. Proposition 1.
Let Assumption 1 hold and consider the resulting periodic clique-gossip protocol withperiod d ∈ Z + . Let F d ∈ R nb × nb be a period-based state transition matrix of the protocol and assume theprotocol is convergent. Suppose | λ ( F d ) | > . Then there holds ν = | λ ( F d ) | /d , i.e., the convergence rateof the protocol is O (cid:0) | λ ( F d ) | t/d (cid:1) .Proof. By the basic knowledge of the stability of linear systemslim sup t →∞ (cid:107) x ( (cid:98) td (cid:99) d ) − ¯ y (cid:107)| λ ( F d ) | (cid:98) td (cid:99) = lim sup t →∞ (cid:107) x ( (cid:100) td (cid:101) d ) − ¯ y (cid:107)| λ ( F d ) | (cid:100) td (cid:101) = C ( x (0)) . (9)with ¯ y = lim t →∞ x ( t ), where C ( x (0)) is a constant relying on the network initial value. Let M σ ( t ) , t ≥ σ ( t ) in the clique gossip sequence. By the convergenceof x ( t ), we have ¯ y = lim t →∞ x ( t ) = lim q →∞ x (( q + 1) d ) = lim t →∞ F d x ( qd ) = F d ¯ y , (10)which yields ¯ y ∈ I for all x (0). By (9), (10), and Lemma 2.(iv), we obtainlim sup t →∞ (cid:107) x ( t ) − ¯ y (cid:107)| λ ( F d ) | t/d = lim sup t →∞ (cid:107) M σ ( t − . . . M σ ( (cid:98) td (cid:99) d ) x ( (cid:98) td (cid:99) d ) − ¯ y (cid:107)| λ ( F d ) | t/d = lim sup t →∞ (cid:107) M σ ( t − . . . M σ ( (cid:98) td (cid:99) d ) ( x ( (cid:98) td (cid:99) d ) − ¯ y ) (cid:107)| λ ( F d ) | t/d ≤ (cid:107) M σ ( t − . . . M σ ( (cid:98) td (cid:99) d ) (cid:107) lim sup t →∞ (cid:107) x ( (cid:98) td (cid:99) d ) − ¯ y (cid:107)| λ ( F d ) | t/d ≤ (cid:107) M σ ( t − . . . M σ ( (cid:98) td (cid:99) d ) (cid:107) lim sup t →∞ (cid:107) x ( (cid:98) td (cid:99) d ) − ¯ y (cid:107)| λ ( F d ) | (cid:98) td (cid:99) | λ ( F d ) | (cid:98) td (cid:99)− td < | λ ( F d ) | − B C ( x (0)) , (11)with B = max {(cid:107) M σ ( k ) . . . M σ (0) (cid:107) : k = 0 , . . . , d − } . Similarly, noticinglim sup t →∞ (cid:107) M σ ( (cid:100) td (cid:101) d − . . . M σ ( t ) x ( t ) − ¯ y (cid:107)| λ ( F d ) | t/d = lim sup t →∞ (cid:107) x ( (cid:100) td (cid:101) d ) − ¯ y (cid:107)| λ ( F d ) | t/d , one also has lim sup t →∞ (cid:107) x ( t ) − ¯ y (cid:107)| λ ( F d ) | t/d > | λ ( F d ) | B − C ( x (0)) , (12)10ith B = min {(cid:107) M σ ( d − . . . M σ ( k ) (cid:107) : k = 0 , . . . , d − } . From (11) and (12), the desired characterizationto the rate of convergence follows.We note that the above discussions on convergence and convergence rate of clique-gossip protocols coverstandard gossip protocols since standard gossiping is a special case of clique-gossiping with all cliques beingnode pairs. Recall that in a clique-gossip protocol, one clique is selected at each time slot and the clique-gossipingoperation is undertaken among all nodes in this clique, while the other nodes maintain their states. For thepurpose of speeding up the information spreading over networks, we generalize the notion of multi-gossipin a standard gossip process [16] to a clique gossip. Based on the clique coverage H ∗ G = { C µ , . . . , C µ d } ,we define a multi-clique coverage M ( H ∗ G ) = { C , . . . , C ν } with each C k ⊂ H ∗ G , termed a clique class, beinga set of C µ l , where (cid:83) νk =1 C k = H ∗ G , and where any two cliques in one clique class C k are non-adjacent forall k = 1 , . . . , ν . We also define C ∗ i ( C k ) ∈ C k as the unique clique containing node i and belonging to C k .Introduce a function σ ( · ) : Z ≥ → { , . . . , ν } . Then a multi-clique-gossip protocol is defined as follows. Example 2.
Consider the graph G in Figure 1. Let cliques C = { , , } , C = { , } , C = { , , , } , C = { , , , } , C = { , , } , C = { , , } , C = { , } form its clique coverage H ∗ G . Then its general-ized line graph L ( H ∗ G ) is given in Figure 2. By observing L ( H ∗ G ), we can obtain that possible multi-cliquecoverages M ( H ∗ G ) include (cid:8) { C , C , C } , { C , C , C } , { C } (cid:9) , (cid:8) { C , C } , { C , C } , { C , C } , { C } (cid:9) . Definition 7. (Multi-clique-gossip Protocol) Select C σ ( t ) ∈ M ( H ∗ G ) at each time t = 0 , , . . . . Then thenode state update rule is described by x i ( t + 1) = (cid:88) j ∈ C ∗ i ( C σ ( t ) ) A ij ( µ l ) x j ( t ) if i ∈ (cid:91) C µl ∈ C σ ( t ) C µ l ; x i ( t ) if i / ∈ (cid:91) C µl ∈ C σ ( t ) C µ l . We can see that in contrast to the clique-gossip protocol, the multi-clique-gossip protocol allows multiple non-adjacent cliques to perform internal clique-gossip operations simultaneously. Evidently, the simulta-neous operations over non-adjacent cliques are not mutually influential because no node serves as theintermediary for information transmission. By direct intuition, we know that in order to speed up the con-vergence to a global agreement, one should arrange as many cliques as possible to perform gossiping oper-ation in every time slot, i.e., minimize | M ( H ∗ G ) | . Define the clique-class index by ρ ( H ∗ G ) = min {| M ( H ∗ G ) | : M ( H ∗ G ) is a multi-clique coverage induced by H ∗ G } . Let ∆( L ( H ∗ G )) denote the maximum node degree ofthe generalized line graph L ( H ∗ G ). Define α ( L ( H ∗ G )) = max {| C | : C ⊂ H ∗ G | C i ∩ C j = ∅ , ∀ C i , C j ∈ C } as theindependence number of L ( H ∗ G ). Then we have the following proposition.11 roposition 2. If L ( H ∗ G ) is neither a complete graph nor a cycle graph with an odd number of nodes,then | H ∗ G | α ( L ( H ∗ G )) ≤ ρ ( H ∗ G ) ≤ ∆( L ( H ∗ G )) . In particular, ρ ( H ∗ G ) = | H ∗ G | if L ( H ∗ G ) is a complete graph, and ρ ( H ∗ G ) = 3 if L ( H ∗ G ) is a cycle graph withan odd number of nodes.Proof. The left inequality naturally holds because of the definitions of α ( · ) and ρ ( · ). Now we prove theright inequality. Consider the vertex coloring problem of the generalized line graph L ( H ∗ G ), which is alabeling of the graph’s nodes with colors such that any two nodes which are the endpoints of some edgehave different colors. We denote the smallest number of colors needed to color the nodes of graph L ( H ∗ G ),namely its chromatic number, as χ ( L ( H ∗ G )). It can observed that based on the same clique coverage H ∗ G ,the minimum number of clique classes equals the chromatic number of its generalized line graph, i.e., ρ ( H ∗ G ) = χ ( L ( H ∗ G )). Then by Brooks’ Theorem [21], χ (G) ≤ ∆(G) if G is a simple connected graph butnot a complete graph or a cycle graph with odd nodes. Therefore, we have ρ ( H ∗ G ) ≤ ∆( L ( H ∗ G )) unless L ( H ∗ G ) is a complete graph or a cycle graph with odd nodes. The particular values of ρ ( H ∗ G ) for completegraphs and odd cycle graphs result easily from their specific structures.From the proof of Proposition 2, it can be seen that finding the clique classes of a graph is equivalentto finding a vertex coloring of its generalized line graph. It is known [23] that finding the number ofconventional multigossips of a graph is intrinsically obtaining an edge coloring of the graph, which is alabeling of the edges of the graph such that any two edges sharing the same endpoint have different colors.Then it follows that the edge coloring of the graph is equivalent to a vertex coloring of its conventionalline graph. Since the conventional line graph is a special case of the generalized line graph by letting everyclique in H ∗ G possess two nodes, we can conclude that the problem of finding the clique classes of a graphin this paper is consistent with the result regarding multigossips in [23].It is hard to find ρ ( H ∗ G ) of an arbitrary graph G. Inspired by the greedy algorithm in [24], however, wecan generate a multi-clique coverage M ( H ∗ G ) from the clique coverage H ∗ G by visiting every node of L ( H ∗ G )in order and assign it into the first available clique class, so that we can obtain a relatively small | M ( H ∗ G ) | . In this section, we provide a few numerical examples to illustrate the result in Theorem 1 and investigatethe performance of the clique-gossip averaging algorithm by comparing it to the standard gossip algorithm,and the multi-clique-gossiping in contrast to pure clique-gossiping.
The following example validates the result in Theorem 1.12 xample 3.
Consider the graph G in Figure 1 with the clique coverage H ∗ G = { C , . . . , C } , whereC , . . . , C are specified in Figure 1. In order to validate two conditions in Theorem 1, we compute thespectrum of F = M M M M M M M , F π = M M M M M M M , F π = M M M M M M M , where M µ , µ = 1 , . . . ,
7, corresponding to C µ , µ = 1 , . . . ,
7, are as defined as in Section 2.3. This impliesthat F , F π , F π are the state transition matrices for the clique-gossip averaging algorithm. Obviously π is the permutation that interchanges two non-adjacent cliques C , C , and π interchanges C , C ,neither of which is contained in any cycle of L ( H ∗ G ) plotted in Figure 2. As computed, σ ( F ) = σ ( F π ) = σ ( F π ) = { , . , . , . , . , . , , , , , , , } , i.e., cp( F ) = cp( F π ) = cp( F π ). Thisis consistent with Theorem 1. In the following example, we compare the convergence speed of the periodic clique-gossip averaging al-gorithm and the standard periodic gossip averaging algorithm and discuss the performance improvementwith the application of multi-clique-gossiping.
Example 4.
Consider the graph G in Figure 1. Let M µ , µ = 1 , . . . , x g ( t ) , x c ( t ) , t = 0 , , , . . . , respectively.Define F c = M M M M M M M as the period-based state transition matrix for the periodic clique-gossip averaging algorithm. Then we let(2 , , (2 , , (9 , , (9 , , (1 , , (1 , , (2 , , (3 , , (3 , , (11 , , (3 , , (8 , , (4 , , (5 , , (4 , , (6 , , (7 , , (4 , , (4 , , (6 ,
8) be a 20 edges gossip sequence for the standard periodic gossip averaging algorithm. De-note F g as the period-based state transition matrix corresponding to the standard gossip sequence. First wecompute that the second largest eigenvalues of F g , F c are 0 . , . √ . , √ . e ( t ) = (cid:80) i =1 ( x i ( t ) − ¯ x ) , with ¯ x = (cid:80) i =1 x i (0) /
7, for these twoalgorithms in Figure 3. It is known [16] that the second largest eigenvalue of the state transition matrixdetermines the convergence speed of its corresponding algorithm. Then we can conclude from the fact that √ . < √ . x faster. 13ased on the clique coverage H ∗ G = { C , . . . , C } and the generalized line graph L ( H ∗ G ) in Figure 2,we define a multi-clique coverage M ( H ∗ G ) = { C , C , C } with C = { C , C , C } , C = { C , C , C } , C = { C } . Let the multi-clique gossiping occur over the multi-clique coverage M ( H ∗ G ). Then the trajectory oferror e ( t ) for multi-clique-gossiping is plotted in Figure 3. It can be seen that multi-clique-gossiping yieldsmuch faster convergence speed than either clique-gossiping or standard gossiping. To make this conclusionnumerically clear, we calculate that the second largest eigenvalue of the state transition matrix for themulti-clique-gossiping √ . √ . time -30 -25 -20 -15 -10 -5 Trajectories of Error
GossipingClique-gossipingMulti-clique-gossiping
Figure 3: The trajectories of error e ( t ) = (cid:80) i =1 ( x i ( t ) − ¯ x ) with ¯ x = (cid:80) i =1 x i (0) / Example 5.
Consider the graphs G m with m = 3 , . . . ,
20 whose topologies are given in Figure 4. It can beseen that G m has a typical structure that the induced graphs G[ { , , . . . , m } ] and G[ { m +2 , m +3 , . . . , m +1 } ] are both ring graphs, which are linked by a complete induced graph G[ { , m +1 , m +2 } ]. Then it followsthat C = { , m + 1 , m + 2 } is a 3-node clique. Let e = (1 , m + 1) , e = (1 , m + 2) , e = ( m + 1 , m + 2).Define clique sequence S leftp = (1 , , (2 , , . . . , ( m, , S right = ( m + 2 , m + 3) , ( m + 3 , m + 4) , . . . , (2 m +1 , m + 2). Let S left , e , e , e , S right and S left , e , e , e , S right be two standard gossip averaging sequenceswith their period-based state transition matrix denoted by F g , F g , respectively. Correspondingly, wereplace (1 , m + 1) , ( m + 1 , m + 2) , (1 , m + 2) with C to form the clique-gossip averaging sequence, withits period-based state transition matrix denoted by F c . Note that the period length for clique-gossiping isshorter than that for standard gossiping. First we plot the values of | λ ( F g ) | , | λ ( F g ) | , | λ ( F c ) | varyingwith m = 3 , . . . ,
20 in Figure 5. As can be seen, | λ ( F g ) | < | λ ( F g ) | = | λ ( F c ) | for all m . This shows14hat the application of the clique gossiping does not necessarily reduce the second largest eigenvalue ofthe period-based state transition matrix. Based on Proposition 1, we next investigate the relationshipamong the convergence rate | λ ( F g ) | / (2 m +3) , | λ ( F g ) | / (2 m +3) and | λ ( F c ) | / (2 m +1) for all m = 3 , . . . , | λ ( F c ) | / (2 m +1) < | λ ( F g ) | / (2 m +3) < | λ ( F g ) | / (2 m +3) for all m = 3 , . . . ,
20, which indicates that clique-gossiping has faster convergence speed than standard gossiping,especially when the ring graphs G[ { , , . . . , m } ] and G[ { m + 2 , m + 3 , . . . , m + 1 } ] have small size.Figure 4: Graph G m , m = 3 , . . . , m | ( F )| Gossiping Clique-gossipingGossiping Figure 5: The values of | λ ( F g ) | , | λ ( F g ) | for standard gossiping and | λ ( F c ) | for clique-gossiping varyingwith m = 3 , . . . , Example 6.
Consider the 101-node graphs G k = (V , E k ) , k = 1 , . . . ,
99 with one such topology shownin Figure 7, which satisfy E k = { (1 , , (1 , , . . . , (1 , , ( l + 1 , l + 2) } , l = 1 , . . . , k . Note that G k has k l = { , l + 1 , l + 2 } , l = 1 , . . . , k . Let all 100 + k edges of G k be a standardgossip averaging sequence in a fixed but arbitrarily chosen order, whose period-based state transitionmatrix is denoted by F g . By replacing ( l + 1 , l + 2) with cliques C l , l = 1 , . . . , k, k = 1 , . . . ,
99, we obtain aclique-gossip averaging sequence with its period-based state transition matrix denoted by F c . Evidently,15 | ( F )| Gossiping Clique-gossipingGossiping Figure 6: The trajectories of | λ ( F g ) | / (2 m +3) , | λ ( F g ) | / (2 m +3) for standard gossiping and | λ ( F c ) | / (2 m +1) for clique-gossiping varying with m = 3 , . . . , | λ ( F g ) | and | λ ( F c ) | for values of k = 1 , . . . ,
99 in Figure 8. Since the period lengths for clique-gossiping and standardgossiping are equal, | λ ( F g ) | and | λ ( F c ) | embody their convergence speeds, respectively. We can seethat the convergence speed of clique-gossiping is observably faster than standard gossiping. Moreover, theperformance improvement becomes greater as the number of 3-node cliques involved increases.Figure 7: A 101-node graph G k , k = 2.It is implied from Proposition 1 that the convergence speed for periodic standard gossiping or clique-gossiping is determined by two factors: the period length and the second largest eigenvalue magnitude ofthe period-based state transition matrix. Clique-gossiping may provide faster convergence than standardgossiping by reducing the period length (as verified in Example 5) or decreasing the the second largest16 | ( F )| GossipingClique-gossiping
Figure 8: The values of | λ ( F g ) | and | λ ( F c ) | for standard gossiping and clique-gossiping varying with k = 1 , . . . ,
99. Since they have the same period length, we can conclude that clique-gossiping has fasterconvergence speed than standard gossiping.eigenvalue magnitude (as verified in Example 6). An intriguing phenomenon observed from these twoexamples lies in that the performance improvement becomes more pronounced when we replace moreedges in gossip sequence with cliques of size greater than two, and reduce the number of the nodes unableto be covered by cliques. Therefore, we conjecture that it is always encouraged to replace a pure gossipwith a clique-gossip for the graphs containing cliques, in order to speed up distributed computation andmake improvement in the algorithm performance. However, it is difficult to prove that clique-gossiping ismore efficient than standard gossiping in a general case, because making comparison among the secondlargest eigenvalues in magnitude of different period-based state transition matrices is a difficult problem.
In this section, we investigate the clique-gossip averaging algorithm introduced in Definition 3. Formallythe algorithm is written as x ( t + 1) = M σ ( t ) x ( t ) (13)where x ( t ) = ( x ( t ) . . . x n ( t )) (cid:62) and M σ ( t ) is induced by the matrices A ij ( σ ( t )) = 1 / | C σ ( t ) | . The asymptoticconvergence of this algorithm has been clear from Lemma 2. Interestingly enough for standard gossipalgorithms, finite-time convergence is possible providing a definitive solution within a finite time steps [17].Inspired by this we now study the finite-time convergence of clique-gossip algorithms. First we introducethe following definition. 17 efinition 8. A clique-gossip averaging algorithm achieves finite-time convergence with respect to initialvalue x (0) = c ∈ R n , if there exists a nonnegative integer T (which may depend on c ) such that x ( T ) ∈ span { } . Naturally, we say a clique-gossiping averaging algorithm achieves global finite-time convergence if finite-time convergence can be reached for any initial value in R n . A feasible process of producing global finite-time convergence is provided in the following example. Example 7.
Consider a node set V = { , , . . . , } shown in Figure 9. Let C = { , } , C = { , } , C = { , } , C = { , } , C = { , } , C = { , } , C = { , , . . . , } , C = { , , . . . , } . Suppose the nodeset of graph G = (cid:83) i =1 G[C i ] is V. Let H ∗ G = { C , C , . . . , C } be a clique coverage of G. It is evident that byperforming averaging operations on first C , C , . . . , C in an arbitrary order, then C , C in an arbitraryorder (or first C , C in an arbitrary order, then C , C , . . . , C in an arbitrary order), global finite-timeconvergence can be achieved over G.Figure 9: A 12-node complete graph G (only a subset of the edges are shown) with a clique coverage H ∗ G = { C , C , . . . , C } , where C = { , } , C = { , } , C = { , } , C = { , } , C = { , } , C = { , } , C = { , , . . . , } , C = { , , . . . , } .As can be seen in Example 7, the number of nodes n = 12 = 6 ×
2. As a result, global finite-timeconvergence can be achieved in 2 + 6 = 8 steps by constructing two cliques of size 6 and six cliques ofsize 2. Inspired by Example 7, we present a sufficient condition for finite-time convergence in the followingtheorem.
Theorem 2.
Consider a node set
V = { , , . . . , n } with n = r r for two integers r , r ≥ . Then thereexists a graph G with its node set being V and a clique coverage H ∗ G that consists of only cliques with sizes r or r leading to a globally finite-time convergent clique-gossip averaging algorithm. Furthermore, suchfinite-time convergence can be achieved in r + r steps.Proof. Define cliques C p = { r ( p − , r ( p − , . . . , r p } , p = 1 , . . . , r and Q q = { q, r + q, . . . , r ( r −
1) + q } , q = 1 , . . . , r . Let G = ( r (cid:83) i =1 G[C i ]) (cid:83) ( r (cid:83) i =1 G[Q i ]). Next we prove that along the r + r long sequenceof cliques C , . . . , C r , Q , . . . , Q r , the algorithm yields a global finite-time convergence. Note that thecliques C p , p = 1 , . . . , r (or Q q , q = 1 , . . . , r ) are mutually disjoint. Suppose every node i holds the initialstate x i (0). After undertaking averaging operations over C p s, we have the node i ∈ C p ’s state at time18 = r x i ( r ) = 1 r (cid:88) j ∈ C p x j (0) . (14)Then we perform averaging operations over Q q s and one has for node i ∈ Q q x i ( r + r ) = 1 r (cid:88) j ∈ Q q x j ( r ) . (15)It is worth noting that every node j , contained in the same Q q , belongs to a distinct C p . Thus by (14)and (15), and the fact that C p s are mutually disjoint x i ( r + r ) = 1 r r n (cid:88) j =1 x (0) , ∀ i = 1 , . . . , n. This completes the proof.
Remark 1.
Let us consider the case where n = r r . . . r k with integers r , r , . . . , r k ≥ . By recursivelyapplying Theorem 2, a clique sequence with the cliques’ sizes being r , r , . . . , r k can be constructed alongwhich finite-time convergent averaging algorithm is defined with convergence achieved in k (cid:88) i =1 k (cid:89) j =1 ,j (cid:54) = i r j steps. The intuition is that one can embed the n nodes into a k -dimensional lattice with the j ’th dimensioncontaining r j nodes. Then finite-time convergence can be built along each dimension. In particular, when n = 2 k , the clique coverage H ∗ G with all cliques being gossip edges can be found to produce finite-timeconvergence, as is known from [17]. Theorem 2 provides the method of constructing a clique sequence for finite-time convergence, on con-dition that the total number of a graph’s nodes is the product of two integers greater than one, which areexactly the size of the cliques to be constructed. In practical engineering problems, however, the numberof the nodes contained in each selected clique is required to be unchanged, for the convenience of synchro-nization, noise computation, delay elimination, etc. In order to analyze the finite-time convergence in thisbackground, we first provide the following definition.
Definition 9.
A clique coverage H ∗ G for a graph G is m -regular if every clique in H ∗ G possesses exactly m nodes. The resulting clique-gossip averaging algorithm is called an m -regular clique-gossip averagingalgorithm. It is obvious that not all connected graphs have an m -regular clique coverage if m ≥
3. For a completegraph with n nodes, there always exists an m -regular clique coverage of the graph for any m ≤ n . Now weare interested in the finite-time convergence of m -regular clique-gossip averaging algorithms. We presentthe following theorem. 19 heorem 3. Let
V = { , , . . . , n } . There exists a graph G with its node set being V such that one can findan m -regular clique coverage H ∗ G which can lead to a globally finite-time convergent clique-gossip averagingalgorithm if and only if n is divisible by m with the same prime factors as m . More precisely, the followingstatements hold.(i) If n is not a multiple of m , or n contains a different prime factor compared to m , then no m -regular clique-gossip averaging algorithm converges globally in finite time. In fact, in that case anygiven m -regular clique-gossip averaging algorithm fails to converge in finite time for almost all initialvalues.(ii) Suppose there exist factorizations m = p r · · · p r d d and n = p s · · · p s d d with p , . . . , p d being primenumbers and s i ≥ r i > for all ≤ i ≤ d . Then there exists a globally convergent m -regularclique-gossip averaging algorithm. Moreover, a fastest m -regular clique-gossip averaging algorithmconverges in n (cid:16) max ≤ i ≤ d (cid:24) s i r i (cid:25) (cid:17) /m steps. Theorem 3 is a generalization of the results on finite-time convergence with a standard gossip averagingalgorithm [17], which corresponds to the special case of m = 2. The sufficiency proof of the theorem isbased on a constructive algorithm, where clearly only a small fraction of edges in the complete graphhas been used. Therefore, the usefulness of this finite-time convergent result is not restricted only to thecomplete graph case. Finite-time convergence is also possible if we allow the A ij ( σ ( t )) to be genuinelytime-dependent, e.g., [25], which will result in a consensus algorithm with time-varying state transitions.Now we provide an example to illustrate the finite-time convergence in Theorem 3. Example 8.
Consider the 18-node complete graph G in Figure 10 (only a subset of the edges are shown).Let C = { , , , , , } , C = { , , , , , } , C = { , , , , , } , C = { , , , , , } , C = { , , , , , } , C = { , , , , , } and H ∗ G = { C , C , . . . , C } be a 6-regular clique coverage ofG. Also we plot its generalized line graph L ( H ∗ G ) in Figure 11. Note that L ( H ∗ G ) is a complete bipartitegraph. It can be seen that G with the clique coverage H ∗ G satisfies the finite-time convergence conditionin Theorem 3 and one can indentify that n = 18 = 2 × , m = 2 ×
3. By undertaking the averaging op-erations on first C , C , C in an arbitrary order, then C , C , C in an arbitrary order (or first C , C , C in an arbitrary order, then C , C , C in an arbitrary order), the clique-gossip averaging algorithm yieldsfinite-time convergence regardless of initial states. 20igure 10: A 18-node complete graph G(only a subset of the edges are shown) with a 6-regularclique coverage H ∗ G = { C , C , . . . , C } , where C = { , , , , , } , C = { , , , , , } , C = { , , , , , } , C = { , , , , , } , C = { , , , , , } , C = { , , , , , } .Figure 11: The generalized line graph L ( H ∗ G ) for G in Figure 10. It can be seen that L ( H ∗ G ) is a completebipartite graph. In this section, we prove that if n is divisible by m with the same prime factors as m , then there existsa graph G with its node set being V such that one can find an m -regular clique coverage H ∗ G which canlead to a globally finite-time convergent clique-gossip averaging algorithm.Let m = p r · · · p r d d and n = p s · · · p s d d with s i ≥ r i >
0. We introduce δ ( n, m ) := (cid:16) max ≤ i ≤ d (cid:24) s i r i (cid:25) (cid:17) . We denote by (Q . . . Q l ) a finite sequence of cliques of length l , where Q i ∈ H ∗ G , ≤ i ≤ l . Let(Q (cid:48) . . . Q (cid:48) l ) be another finite sequence of cliques of length l . We define the concatenation of (Q . . . Q l )and (Q (cid:48) . . . Q (cid:48) l ) as (Q . . . Q l ) ◦ (Q (cid:48) . . . Q (cid:48) l ) = (Q . . . Q l Q (cid:48) . . . Q (cid:48) l ) , l + l . We now present a recursive algorithm as a clique selectionprocess over the complete graph G = (V , E), the output of which is a finite sequence of cliques with m nodes. CliqueSelect (V , m ) Let n = nm . If n = 1, return V. Otherwise, let Q i = (cid:8) m ( i −
1) + 1 , . . . , mi (cid:9) for i = 1 , . . . , n . Let m = min { b ∈ N : m | n b } . Denote n = mm . Let Q ∗ ij = (cid:8) m ( i −
1) + m ( j −
1) + 1 , . . . , m ( i −
1) + m j (cid:9) for j = 1 , . . . , n , i = 1 , . . . , n . Let Q ∗ j = n (cid:83) i =1 Q ∗ ij , j = 1 , . . . , n . return (Q . . . Q n ) ◦ CliqueSelect (Q ∗ , m ) ◦ · · · ◦ CliqueSelect (Q ∗ n , m );We first show this algorithm is well defined. Note that the following mathematical notations are alldefined in the algorithm above. From the expressions m = p r · · · p r d d and n = p s · · · p s d d we know n = p s − r · · · p s d − r d d . By the definition of m , there holds m = p r (cid:48) · · · p r (cid:48) d d , where r (cid:48) = max { , r − ( s − r ) } ≤ r ,. . . . . .r (cid:48) d = max { , r d − ( s d − r d ) } ≤ r d . Therefore, m | m , which implies that n is a well defined integer. We further know that each Q ∗ ij contains m nodes, and each Q ∗ j contains m n nodes. The definition of m ensures m | m n with m n containingno distinct prime factor compared to m . That is to say, CliqueSelect (Q ∗ , m ) , . . . , CliqueSelect (Q ∗ n , m )can be reasonably recursively invoked.Next, we prove by an induction argument that the clique sequence produced by CliqueSelect (V , m )is of length δ ( n, m ) n/m , and the resulting clique-gossip algorithm converges in δ ( n, m ) n/m time steps.We complete the remainder of the proof in three steps.Step 1. There holds n = m if δ ( n, m ) = 1. The CliqueSelect (V , m ) returns one clique V, and obviouslythe resulting clique-gossip algorithm converge in one step. Now we assume Induction Hypothesis . For δ ( n, m ) ≤ K − K > CliqueSelect (V , m ) generates a sequenceof δ ( n, m ) n/m cliques, along which the resulting clique-gossip algorithm converges in δ ( n, m ) n/m timesteps.Step 2. Let δ ( n, m ) = K >
1. Note that every clique selected by
CliqueSelect (Q ∗ j , m ) contains m n nodes. By the definition of m and n , we can verify that m n = p s (cid:48) · · · p s (cid:48) d d , where s (cid:48) = max { r , s − r } , . . . , s (cid:48) d = max { r d , s d − r d } . δ ( n m , m ) = K −
1, and by our induction hypothesis, each
CliqueSelect (Q ∗ j , m ) producesa sequence of ( K − n m m cliques. Thus, the total length of the sequence CliqueSelect (V , m ) is n + ( K − n m m n = Knm .
This establishes the number of cliques generated by the algorithm
CliqueSelect (V , m ).Step 3. We finally prove finite-time convergence of the resulting clique-gossip algorithm along the cliquesequence CliqueSelect (V , m ). Fix the initial value at all nodes. Then after the first n steps, all nodesin Q i = (cid:8) v m ( i − , . . . , v mi (cid:9) hold a common value z i for i = 1 , . . . , n .Note that each Q i is decomposed as n disjoint subsets Q ∗ ij , j = 1 , . . . , n , where each Q ∗ ij contains m nodes. Therefore, at time n and for i = 1 , . . . , n , there are m nodes which hold value z i in Q ∗ j = n (cid:83) i =1 Q ∗ ij .Because the Q ∗ j are mutually disjoint, the clique-gossip algorithm given by CliqueSelect (Q ∗ j , m ) does notinfluence the values of nodes outside Q ∗ j . Again by our induction hypothesis, for any j = 1 , . . . , n , theclique-gossip algorithm given by CliqueSelect (Q ∗ j , m ) ensures that all nodes in Q ∗ j hold the same value1 n n (cid:88) i =1 z i = 1 n n (cid:88) j =1 x i (0) . Therefore, all nodes in V will hold the same value as the average of the network initial values along theclique-gossip algorithm generated by
CliqueSelect (V , m ) after δ ( n, m ) n/m time steps. (cid:3) Now we prove that if there exists a graph G with its node set being V such that one can find an m -regularclique coverage H ∗ G which can lead to a globally finite-time convergent clique-gossip averaging algorithm,then n is divisible by m with the same prime factors as m . We only need to find a particular initial value c ∈ R n that any deterministic clique-gossip algorithm will fail to converge in finite steps. We know that m can be written uniquely as p r · · · p r d k , where p < · · · < p d are prime numbers and r i > , ≤ i ≤ d .Given an arbitrary clique-gossip algorithm. We investigate two cases, respectively. • Let n have a prime factor p that m does not have. Choose the initial value c = (1 , , . . . , T . Forany t , it is easy to see that x i ( t ) = α i ( t ) β i ( t ) , where α i ( t ) and β i ( t ) are coprime integers with β i ( t ) having no prime factor that m does not have.That is to say, β i ( t ) does not contain the prime factor p , for any t . The limit of x i ( t ) however mustbe (cid:80) ni =1 x i (0) /n = 1 /n . Because n has p as its prime factor, such a value cannot be reached at anyfinite time steps. • Let n be represented by p s · · · p s d d , where s i ≥ , ≤ i ≤ d with some 1 ≤ a ≤ d that s a < r a . Again23e choose the initial value c = (1 , , . . . , T . Similarly, for any t we have x i ( t ) = α i ( t ) β i ( t ) , where α i ( t ) and β i ( t ) are coprime integers with β i ( t ) being some multiple of p r a a . Because p r a a cannotdivide n and the node state limit must be 1 /n , finite-time convergence is impossible. Note that Theorem 3.(i) asserts a stronger non-existence of claim in that any m -regular clique gossipalgorithm fails to reach agreement in finite steps for almost all initial values. Let M be a set consisting ofat most countable n × n real matrices. Define S M = (cid:8) c ∈ R n : ∃ t ≥ , M , . . . , M t ∈ M , s.t. M t · · · M c ∈ span { } (cid:9) . It is easy to verify that S M = ∞ (cid:91) t =0 (cid:91) M ,..., M t ∈ M S M ... M t , where S M ... M t = (cid:8) c ∈ R n : M t · · · M c ∈ span { } (cid:9) with M , . . . , M t ∈ M .Note that each S M ... M t is a linear subspace of R n , with a dimension no larger than n . If all S M ... M t are lower-dimensional subspaces of R n , S M ... M t has zero measure for any M , . . . , M t ∈ M . This in turntells us that S M is a zero-measure set for M is a union of countably many zero-measure sets. On the otherhand, if there exists M , . . . , M t ∈ M such that S M ... M t is n dimension, we have S M = R n . Therefore,either S M = R n or S M is a zero-measure set in R n . The desired almost everywhere impossibility conclusionholds immediately since we already proved non-existence of globally finite-time convergent m -regular cliquegossiping. Recall that m = p r · · · p r d d and n = p s · · · p s d d with s i ≥ r i >
0. We have provided an algorithm thatconverges in δ ( n, m ) n/m steps. Now we prove that it is indeed the fastest algorithm. Consider any m -regular clique gossip algorithm that converges globally in finite time. Then there must exist T ≥ M σ ( T ) · · · M σ (1) = 1 n T . Introduce N ( t ) = M σ ( t ) · · · M σ (1) and h i ( t ) = (cid:12)(cid:12)(cid:12)(cid:8) s : [ M σ ( s ) ] ii = m , ≤ s ≤ t (cid:9)(cid:12)(cid:12)(cid:12) . Note that h i ( t ) representsthe number of times at which i is in the selected cliques for the first t steps.24enote τ = arg max ≤ i ≤ d (cid:100) s i r i (cid:101) . Associated with the prime number p τ , we define a function P τ ( · ) overall natural numbers by P τ ( x ) = max { k : x = yp kτ , y ∈ N , k ∈ Z } . In other words, P τ ( x ) is the number of powers of the prime number p τ in the arithmetic decompositionof x .We can verify recursively that [ N ( t )] ii = γ ii ( t ) δ ii ( t ) , where γ ii ( t ) and δ ii ( t ) are coprime numbers with P τ ( δ ii ( t )) ≤ h i ( t ) r τ . Based the facts that [ N ( T )] = 1 /n and P τ ( n ) > (cid:0) δ ( n, m ) − (cid:1) r τ , we obtain (cid:0) δ ( n, m ) − (cid:1) r τ < P τ ( n ) = P τ ( δ ii ( T )) ≤ h i ( T ) r τ . This implies h i ( T ) ≥ δ ( n, m ). On the other hand, there must hold δ ( n, m ) n ≤ n (cid:88) i =1 h i ( T ) = T m.
We can now conclude T ≥ δ ( n, m ) n/m , and this is the fundamental lower bound that any m -regularclique gossip algorithm can reach in terms of convergence time. We have now proved the complexity claimin Theorem 3.(ii). We have presented a framework for clique gossip protocols where node interactions utilize cliques as com-plete subnetworks in gossip processes. Clique-gossip protocols and clique-gossip averaging algorithms havebeen defined as generalizations of standard gossip protocols and gossip averaging algorithms, respectively.A fundamental eigenvalue invariance principle for periodic clique-gossip protocols was established, andthe possibilities of realizing finite-time convergent clique-gossip averaging were thoroughly investigated.Numerical examples also revealed the acceleration effects of clique-gossiping compared to standard gossip-ing. Interesting future directions include concrete theoretical validations of how much improvement canbe gained via clique-gossiping in terms of efficiency, and self-organized or engineering mechanism thatproduces local cliques across a network.
References [1] P. T. Eugster, R. Guerraoui, S. B. Handurukande, and P. Kouznetsov, “Lightweight probabilisticbroadcast,”
ACM Trans. Computer Systems , 21(4): 341–374, 2003.[2] M. Jelasity, A. Montresor, and O. Babaoglu, “Gossip-based aggregation in large dynamic networks,”
ACM Trans. Computer Systems , 23(3): 291–252, 2005.253] D. Shah. Gossip algorithms.
Foundations and Trends in Networking , 3(1): 1–125, 2008.[4] V. Ravelomanana, “Optimal initialization and gossiping algorithms for random radio networks,”
IEEETransactions on Parallel and Distributed Systems , vol. 18, no. 1, 2007.[5] K. Hopkinson, K. Jenkins, K. Birman, J. Thorp, G. Toussaint, and M. Parashar, “Adaptive gravita-tional gossip: A gossip-based communication protocol with user-selectable rates,”
IEEE Transactionson Parallel and Distributed Systems , vol. 20, no. 12, pp. 1830–1843, 2009.[6] K. Iwanicki and M. van Steen, “Gossip-based self-management of a recursive area hierarchy for largewireless sensornets,”
IEEE Transactions on Parallel and Distributed Systems , vol. 21, no. 4, pp. 562–576, 2010.[7] A. Demers, D. Greene, C. Hauser, W. Irish, J. Larson, S. Shenker, H. Sturgis, D. Swinehart, andD. Terry, “Epidemic algorithms for replicated database maintenance,”
Proceedings of the 7th ACMSymposium on Operating Systems Principles . ACM, New York, 1–12, 1987.[8] D. Kempe, J. Kleinberg, and A. Demers, “Spatial gossip and resource location protocols,”
Journal ofACM , 51(6): 943–967, 2004.[9] B. Doerr, M. Fouz, and T. Friedrich, “Why rumors spread so quickly in social networks?”
Communi-cations of ACM , 55(6): 2012.[10] F. Bullo, R. Carli, and P. Frasca, “Gossip coverage control for robotic networks: dynamical systemson the space of partitions,”
SIAM J. Control Optim. , 50(1): 419–447, 2012.[11] A. G. Dimakis, S. Kar, J. M. F. Moura, M. G. Rabbat, and A. Scaglione, “Gossip algorithms fordistributed signal processing,”
Proceedings of IEEE , 98(11): 1847–1864, 2010.[12] K. I. Tsianos and M. G. Rabbat, “Consensus-based distributed online prediction and optimization,”
IEEE GlobalSIP Network Theory Symposium , pp. 807–810, 2013.[13] S. Mou and B. D. Anderson, “Eigenvalue invariance of inhomogeneous matrix products in distributedalgorithms,”
IEEE Control Systems Letters , vol. 1, pp. 8–13, 2017.[14] D. Kempe, A. Dobra, and J. Gehrke, “Gossip-based computation of aggregate information,”
Proc.Foundations of Computer Science , 482–491, 2003.[15] S. Boyd, A. Ghosh, B. Prabhakar and D. Shah, “Randomized gossip algorithms,”
IEEE Trans. In-formation Theory , 52(6): 2508–2530, 2006.[16] J. Liu, S. Mou, A. S. Morse, B. D. Anderson, and C. Yu, “Deterministic gossiping,”
Proceedings ofthe IEEE , vol. 99, no. 9, pp. 1505–1524, 2011. 2617] G. Shi, B. Li, M. Johansson, and K. H. Johansson, “Finite-time convergent gossiping,”
IEEE/ACMTransactions on Networking , vol. 24, no.4, pp. 2782–2794, 2016.[18] G. Shi, B. D. O. Anderson, and K. H. Johansson, “Consensus over random graphs: network Borel-Cantelli lemmas for almost sure convergence,”
IEEE Transactions on Information Theory , 61(10):5690-5707, 2015.[19] Y. Zeng, R. C. Hendriks, and R. Heusdens, “Clique-based distributed beamforming for speech en-hancement in wireless sensor networks,” in
Signal Processing Conference (EUSIPCO), 2013 Proceed-ings of the 21st European , pp. 1–5, 2013.[20] K. Biswas, V. Muthukkumarasamy, E. Sithirasenan, and M. Usman, “An energy efficient clique basedclustering and routing mechanism in wireless sensor networks,”
Wireless Communications and MobileComputing Conference , pp. 171–176, 2013.[21] R. L. Brooks, “On colouring the nodes of a network,” in
Mathematical Proceedings of the CambridgePhilosophical Society , vol. 37, pp. 194–197, Cambridge Univ Press, 1941.[22] J. B. Fraleigh,
A First Course in Abstract Algebra . Pearson Education India, 2003.[23] S. Mou, C. Yu, B. D. Anderson, and A. S. Morse, “Deterministic gossiping with a periodic protocol,”in
Decision and Control (CDC), 2010 49th IEEE Conference on , pp. 5787–5791, IEEE, 2010.[24] T. H. Cormen, C. E. Leiserson, R. L. Rivest, C. Stein, “Introduction to Algorithms,”
MIT Press ,vol. 44, pp. 97–138, 1990.[25] J. M. Hendrickx, G. Shi, and K. H. Johansson, “Finite-time consensus using stochastic matrices withpositive diagonals,”
IEEE Transactions on Automatic Control , vol. 60, no. 4, pp. 1070–1073, 2015.[26] R. A. Horn and C. R. Johnson,