Consensus over evolutionary graphs
CConsensus over evolutionary graphs
Michalis Smyrnakis, Nikolaos M. Freris and Hamidou Tembine
Abstract — We establish average consensus on graphs withdynamic topologies prescribed by evolutionary games amongstrategic agents. Each agent possesses a private reward functionand dynamically decides whether to create new links and/orwhether to delete existing ones in a selfish and decentralizedfashion, as indicated by a certain randomized mechanism. Thismodel incurs a time-varying and state-dependent graph topol-ogy for which traditional consensus analysis is not applicable.We prove asymptotic average consensus almost surely and inmean square for any initial condition and graph topology. Inaddition, we establish exponential convergence in expectation.Our results are validated via simulation studies on randomnetworks.
Index Terms — Consensus, Evolutionary Games, EvolutionaryGraphs, Distributed Algorithms, Randomized Algorithms.
I. I
NTRODUCTION
Evolutionary game theory has been established as a mod-eling tool for interactions between populations of strate-gic entities. In specific, evolutionary games describe thepopulation dynamics resulting from pairwise interactions. Ithas found numerous applications in various areas of multi-agent systems such as in wireless networks [1], [2], swarmrobotics [3], and dynamic routing protocols [4].
Evolutionary graphs arise as an application of evolutionarygame theory in modeling dynamic graph topologies. In suchcontext, a population is organized as a network (graph)with the nodes (vertices) representing atoms (agents) andlinks (edges) representing interactions among them. This iscaptured by a weighted graph with time-varying edge setdetermined by an evolutionary game. We will present a spe-cific randomized decentralized mechanism for determiningthe graph topology based on the individual fitness functionsof the agents. In our setting, each node maintains a localvariable and computes its fitness function using its own valuealong with the values from its neighbors (both active andinactive, cf. Sec. III). Subsequently, it dynamically readjustsits neighbor set by randomly adding or deleting links withprobabilities dictated by the resulting change of its fitnessfunction.
Consensus is a canonical example of in-network coordi-nation in multi-agent systems. Each agent maintains a localvalue and the goal is for the entire network to reach agree-ment to a common value in a distributed fashion, i.e., vialocal exchanges of messages between neighboring (adjacent)
The authors are with the Division of Engineering at New York UniversityAbu Dhabi, Saadiyat Island, P.O. Box 129188, Abu Dhabi, UAE. The lasttwo authors are also with NYU Tandon School of Engineering. E-mails: {ms10775, nf47, tembine}@nyu.edu . This work was supportedby the National Science Foundation (NSF) under grant CCF-1717207, andthe U.S. Air Force Office of Scientific Research (AFOSR) under grantFA9550-17-1-0259. agents. An archetypal problem is average consensus , wherethe goal is for each agent to asymptotically compute theaverage of all nodal values; cf. [5], [6], [7], [8], for alargely non-exhaustive list of references. This theme hasproven a prevalent design tool for distributed multi-agentoptimization [9], [10], signal processing [11], numericalanalysis [12], and estimation [13], [14].In this paper, we seek to bridge the gap between evolu-tionary game theory and distributed optimization by studyingaverage consensus over evolutionary graphs. In our setting,each agent has a local variable that represents its ‘strategy’:the strategies evolve following a consensus protocol overa time-varying graph capturing inter-agent cooperations. Ateach time instant, agents dynamically select the agents withwhich they cooperate (from a candidate set) based on theirutility (i.e., fitness) that depends on their own strategy andthe strategies of neighboring agents they intend to coop-erate with. Specifically, agents create or drop links (i.e.,cooperations) when they deem it beneficial for them, andthey do so via a randomized decentralized mechanism, ina selfish manner. Examples enlist social networks [15] andcoordination of robot swarms [3].We consider the problem of average consensus overnetworks of time-varying topology captured by a MarkovDecision Process (MDP). Unlike prior work on the sub-ject [8], [10], the topology depends on the agents’ values ,which renders previous analysis techniques inapplicable inour case. We proceed to establish average consensus a.s.(almost surely) and in m.s. (mean square) sense, usingstochastic Lyapunov techniques. Additionally, we prove thatthe convergence is exponential in expectation, and provide alower bound on the expected convergence rate. Finally, ourmethod was empirically assessed via numerical simulations.The remainder of the paper is organized as follows:Sec. II exposes preliminaries on graph theory, consensusand evolutionary graph theory. In Sec. III, we present theproblem formulation. The convergence analysis is presentedin Sec. IV. Sec. V illustrates simulation results, while Sec. VIconcludes the paper and discusses future research directions.II. P
RELIMINARIES
In this section, we recap essential background on graphtheory, consensus protocols and evolutionary games. In theremainder of the paper, we use boldface for vectors andcapital letters for matrices. Vectors are meant as columnvectors, and we use , to denote the vectors with all entriesequal to zero, one, respectively, and I to denote the identitymatrix (with the dimension made clear from the contextin all cases). Last, we use the terms ‘agent’, ‘vertex’ and a r X i v : . [ m a t h . O C ] M a r node’ interchangeably, and the same holds true for ‘edge’and ‘link,’ in what follows. A. Graph theory
Consider the case of n interacting agents which aim toachieve consensus over a quantity of interest, for instancecompute the average of their values. Such problem is an in-stance of computing on graphs , where each agent is modeledas a vertex of the graph and edges are drawn only betweeninteracting nodes: we assume that two nodes can interact witheach other (for instance, exchange private information) if andonly if they are connected, i.e., there is an edge between themin the graph.Formally, let a graph be denoted by G = ( V , E ) , where V is the non-empty set of vertices (nodes) and E is the setof edges. In this paper, we restrict attention to undirected graphs, that is to say the edge set E consists of unorderedpairs: ( i, j ) ∈ E implies that agents i, j can interact in asymmetric fashion, i.e., cooperate . We further assume thatthe graph does not contain self-loops (i.e., ( i, i ) / ∈ E for all i ∈ V ); this is without any loss in generality, since edgescapture inter-agent interactions in our framework. We set n := |V| , m = |E| , to denote the number of nodes, edgesrespectively. We say that the graph G is connected if thereis a path between any two nodes i, j ∈ V ; otherwise, we saythat the graph is disconnected.The adjacency matrix A ∈ R n × n of the graph capturesconnections between the nodes: for any two nodes i, j ∈ V , a ij is defined as: a ij = (cid:26) , if ( i, j ) ∈ E , , otherwise.The definition can be extended to weighted graphs, in whichcase a ij can take an arbitrary value if ( i, j ) ∈ E . For anundirected graph, a ij = a ji , for all i, j ∈ V , i.e., A issymmetric. Besides, A has a zero-diagonal ( a ii = 0 for all i ),since G is assumed to have no self-loops. The neighborhood of a node i contains all nodes that the node has a connectionwith, and is denoted by N i := { j : a ij (cid:54) = 0 } . The degree of node i is defined as the number of its neighbors, i.e., d i := |N i | = (cid:80) j ∈V a ij . Let D be the diagonal degree-matrix (i.e., its ( i, i ) − th entry equals d i , and off-diagonalentries are zero). We define := D − A , the Laplacian ofthe graph. Clearly, ∈ R n × n is symmetric. Additionally, itcan be shown that is positive semidefinite [16]. It is well-known [16] that rank = n − k , where k is the number ofconnected components of the graph. In particular, a graph isconnected if and only if rank = n − . By its very definition,the Laplacian has a zero eigenvalue with correspondingeigenvector the all-one vector . In fact, the multiplicityof the zero eigenvalue equals the number of connectedcomponents of the graph. The second smallest eigenvalueof the Laplacian is called the Fiedler value or algebraic The extension of our methods to directed graphs that model asymmetricinteractions (e.g., asymmetric reward functions) will be the focal point offuture work. connectivity of the graph, and is denoted as λ ( G ) := λ () .It is positive if and only if the graph is connected. B. Consensus
Each agent i ∈ V maintains a local scalar value x i . We callthe state of the network the vector x obtained by stackingall nodal values { x i } i ∈V . We use x i,t to denote the value ofnode i at time t , and, correspondingly, x t for the networkstate at time t .A widely studied method which describes the evolution of x i is linear consensus [5], [8]. The dynamics for x i can bewritten (up to a multiplicative constant) as ˙ x i,t = (cid:88) j ∈N i a i,j ( x j,t − x i,t ) , which can further be written compactly in matrix form: ˙ x t = − x t . For an undirected and connected graph, the network reaches average consensus , i.e., x t −→ t →∞ Ave ( x ) , where Ave ( x ) := n (cid:62) x is the average of the values at theinitial time 0 [8]. Besides, the convergence rate is exponentialwith rate lower-bounded by the Fiedler value [8].In this paper, we will study consensus over time-varying graphs as abstracted by a time-varying Laplacian t , dictatedby agents’ randomized decisions; that is to say, t = ( x t , E t − ) is a random matrix that depends on both the state at time t and the topology ‘right before’ time t ; consequently, theanalysis in existing literature [5], [6], [7], [8], [10] does notdirectly carry through. C. Evolutionary graphs An evolutionary graph is a graph whose topology isspecified by an evolutionary game [17], [18] of a singlepopulation with finite number of players, n , placed on agraph G . The interactions of players are captured by theedge set of G . Each player i has a set of actions s i ∈ S i and receives a certain pay-off according to its utility (orfitness) function which is a mapping from the joint actionspace S := S × S × . . . S n to the real numbers, f i : S → R .The evolution of graph topology follows a stochastic pro-cess administered through pairwise interactions. In particular,two players (that are allowed to interact) are randomlyselected and a “copy” of the player with the higher fitness,takes the place of the player with the lower one. As a result,the strategy of the player with the smaller fitness is replacedby the strategy of the player with the higher one.III. P ROBLEM FORMULATION
In a cyberphysical system, such as a wireless sensornetwork [19], [20], it is common that agents may opt todynamically create new links with other agents or dropexisting ones during the coordination process; this behaviorresults in time-varying graphs. In order to properly describehis process, it is necessary to formulate a dynamic graph,whose structure depends on time, topology and network state.In what follows, we define a dynamic graph as a graphwith fixed predetermined vertex set V of agents, in which theedge set E varies over time, in a state-dependent randomized fashion. In particular, the edge set E = E ( x t , t, ω ) ⊆ V × V is a state-/time-dependent random set in a probability space (Ω , F , P ) (where Ω is the sample space, F is the σ − algebraon Ω , and P is the probability measure); correspondingly,we define the adjacency matrix A = A ( x t , t, ω ) and theLaplacian matrix = ( x t , t, ω ) . In this paper, we focus ontime-varying graphs with topology at time t depending onboth the state at time t and the topology ‘right before’ time t , i.e., = ( x t , E t − , ω ) . We use the shorthand notation E t , A t , t and G t = ( V , E t ) to emphasize the type-varying aspect.For each node i ∈ V , we denote by N i the set of all feasible neighbors , i.e., the set of all nodes that node i can potentially create a connection with. Since we focuson undirected graphs, we assume that j ∈ N i implies that i ∈ N j . At each time t , for any given node i ∈ V , wedenote the set of active neighbors , i.e., the set of nodes withwhich i is connected, using N (1) i,t ⊆ N i . Furthermore, we let N (2) i,t := N i \ N (1) i,t , the set of inactive neighbors , i.e., theset of nodes that i is not connected with, but may decideto connect with based on the evolutionary game. The degreeof a node i at time t , (cid:12)(cid:12)(cid:12) N (1) i,t (cid:12)(cid:12)(cid:12) , is the total number of activeneighbors of node i at time t . For the subsequent treatment,we make the assumption that the graph obtained by takingthe union of all feasible neighbor sets is connected: Assumption 1:
The graph G (cid:48) = ( V , E (cid:48) ) , with E (cid:48) := ∪ i ∈V ∪ j ∈N i { ( i, j ) } is connected.This condition is necessary for consensus to be achieved: oth-erwise, the graph will be disconnected at each time regardlessof the agents’ decisions, with no possible exchange ofinformation across its connected components, which impliesthat consensus is infeasible. We will establish the sufficiencyof the condition for the dynamic topology instructed by theevolutionary game we propose.Let x t = (cid:8) x ,t , . . . , x n,t (cid:9) denote the coordination levels of the agents, where we will restrict attention (without anyloss in generality) to the case that ≤ x i,t ≤ , ∀ i ∈V , t ≥ . For instance, the evolution of the coordinationlevels may be considered as a resource allocation process, inwhich agents decide to share a percentage of a resource theyown with their neighbors. Similarly, in an opinion dynamicssetup, the coordination levels may reflect the beliefs of theagents, e.g., the information state of vehicles in a robot team.The agents adjust their coordination levels based on inter-actions with their neighbors, according also to their tendencyto create a link with other agents or drop an existing one. Theevolution of an agent’s coordination level may be describedby the following dynamic consensus protocol: ˙ x i,t = (cid:80) j ∈N (1) i,t − χ mij,t ( x j,t − x i,t )+ (cid:80) k ∈N (2) i,t − χ cik,t ( x k,t − x i,t ) , (1)where χ mij,t , χ cik,t are − variables that respectively indicate whether to: a) maintain an existing link (i.e., a link thatis active ‘right before’ time t , equivalently ( i, j ) with j ∈N (1) i,t − ), if χ mij,t = 1 ( χ mij,t = 0 means that the link isdropped); and b) create a new link ( i, k ) with k ∈ N (2) i,t − ,if χ cik,t = 1 . Clearly, we set χ mji,t ≡ χ mij,t , χ cki,t ≡ χ cik,t .The decisions are Bernoulli random variables with respective‘success’ probabilities (the probability of the value ) givenby ≤ w mij,t ≤ , and ≤ w cik,t ≤ .In this paper, the decision rules are state-dependent andtime-invariant, i.e., χ mij,t , χ cik,t are independent Bernoullirandom variables with success probabilities that dependon the coordination levels of the two neighbors: w mij,t = w mij,t ( x i,t , x j,t ) , w cik,t = w cik,t ( x i,t , x k,t ) , cf. (6), (7) for theirexact definition.The following remark underlines the inapplicability ofprevious analysis [8] in our setting. Remark 1:
The graph corresponding to the Laplacian ma-trix t may be disconnected.Indeed, since decisions are probabilistic, there is a positiveprobability that the resulting graph is disconnected (even theevent of an empty edge set has positive probability) at eachgiven time instant.We may stack the decisions { χ mij,t , χ cij,t } into a cor-responding Laplacian matrix = ( x t , E t − , ω ) ≡ t withentries { l ij } i,j ∈V (dropping time dependency for notationalsimplicity) defined by: l ij = l ji := − (cid:16) { j ∈N (1) i } χ mij + 1 { j ∈N (2) i } χ cij (cid:17) ,l ii := − (cid:80) j (cid:54) = i l ij , where {·} is the − indicator function (1 if the event holdsand 0 else). We proceed to write the update rule in matrixform as follows: ˙ x t = − t x t . (2)We call this the state evolution equation; note that, by itsvery definition, it constitutes a continuous Markov DecisionProcess (MDP).The following proposition shows that all coordinationlevels are guaranteed to remain in the interval [0 , if theyare initialized in [0 , , i.e., it establishes that the set [0 , n is forward invariant . Proposition 1:
Suppose x ∈ [0 , n . Then, under stateevolution (2), x t ∈ [0 , n for all t > , i.e., [0 , n isforward-invariant. Proof:
The proof considers two cases: the first caseconsiders a nodal value reaching the upper bound (1), andthe second the lower bound (0).Case 1: Assume that for some t ≥ , there exists i ∈ V with x i,t = 1 and that x s ∈ [0 , n for all s ≤ t . Then, giventhat x j,t ∈ [0 , for all j (cid:54) = i it follows that ˙ x i = (cid:88) j ∈V\{ i } − l ij ( x j,t − ≤ , because x j,t ≤ and − l ij ≥ . Therefore x i can neverexceed the value 1.ase 2: Assume that for some t ≥ , there exists i ∈ V with x i,t = 0 and that x s ∈ [0 , n for all s ≤ t . Since x j,t ∈ [0 , for all j (cid:54) = i it follows that ˙ x i = (cid:88) j ∈V\{ i } − l ij x j,t ≥ , therefore x i can never go below 0. Remark 2:
The forthcoming analysis applies irrespec-tively of the assumption that the values x t ∈ [0 , n , i.e., forarbitrary initial conditions x . This assumption is adoptedsolely for the sake of interpretability in the context ofevolutionary games. A. Evolutionary game
In this section, we provide a rule for selecting the weights(i.e., probabilities) w mij,t , w cij,t based on a particular evolu-tionary game. We use Continuous Actions Iterative Pris-oner’s Dilemma (CAIPD) [21] to define the fitness functionof a given node and illustrate how the weights are selected.In CAIPD, there are n agents that choose their coordina-tion levels given their neighbors’ decisions. Each agent i hasto pay a fee that is related to its coordination level and gains areward related to the coordination levels of its neighbors: thehigher the coordination level of agent i and the coordinationlevels of its neighbors are, the higher the cost and rewardare, respectively.Formally, the reward of agent i when it sets its coordina-tion level to x i is defined using the following fitness function(where we drop dependency from time t henceforth, sincethe definition of fitness in CAIPD is time-independent): f i ( x ) = b (cid:88) j ∈N (1) i x j − c (cid:12)(cid:12)(cid:12) N (1) i (cid:12)(cid:12)(cid:12) x i , (3)where b > c > are constants (i.e., we assume that the gain–per unit of coordination–from cooperating with another agent b is higher than the per-unit loss c ).The change in the fitness function of agent i when itcreates or drops a link, denoted by ˜ f cij (for j ∈ N (2) i ) and ˜ f dij (for j ∈ N (1) i ) respectively, is determined by: ˜ f cij ( x ) = b (cid:88) k ∈N (1) i ∪{ j } x k − c (cid:18)(cid:12)(cid:12)(cid:12) N (1) i (cid:12)(cid:12)(cid:12) + 1 (cid:19) x i − b (cid:88) k ∈N (1) i x k − c (cid:12)(cid:12)(cid:12) N (1) i (cid:12)(cid:12)(cid:12) x i = bx j − cx i , (4) ˜ f dij ( x ) = b (cid:88) k ∈N (1) i \{ j } x k − c (cid:18)(cid:12)(cid:12)(cid:12) N (1) i (cid:12)(cid:12)(cid:12) − (cid:19) x i − b (cid:88) k ∈N (1) i x k − c (cid:12)(cid:12)(cid:12) N (1) i (cid:12)(cid:12)(cid:12) x i = cx i − bx j . (5) In the evolutionary game, a link may be created/droppedif both agents desire to coordinate or not based on thecorresponding increment (or decrement) of their individualfitness functions. Essentially, if both agents benefit frommaintaining/creating a link, the corresponding probabilitymust be higher than the case where only one node benefitsor when the fitness of both agents is decreased. In [22] asigmoid function was used to determine the weights w mij,t and w cij,t . In our formulation, the weights correspond to theprobabilities that agent i maintains (one minus the probabil-ity that it drops) or creates link ( i, j ) , respectively. Followinga similar approach, the weights are selected as (where weonce again drop time dependency since the weight-rule isstate-dependent but time-invariant): w mij = − tanh (cid:16) ˜ f dij ( x ) + ˜ f dji ( x ) (cid:17) = − tanh (cid:0) ( c − b )( x i + x j ) (cid:1) , (6) w cij = + tanh (cid:16) ˜ f cij ( x ) + ˜ f cji ( x ) (cid:17) = + tanh (cid:0) ( b − c )( x i + x j ) (cid:1) . (7)Note that, by definition, the two values are equal andlower-bounded by (in light of the fact that x t ∈ [0 , n , forall t ≥ ; cf. Proposition 1). We define the weighted Lapla-cian matrix W with entries (again dropping time dependencyfor notational simplicity) given by: w ij = w ji := − (cid:16) { j ∈N (1) i } w mij + 1 { j ∈N (2) i } w cij (cid:17) , (8) w ii := − (cid:80) j (cid:54) = i w ij , IV. C
ONVERGENCE ANALYSIS
Formally, for t > , t is an F t − -measurable randommatrix where the σ − algebra F t − is defined by F t − := σ ( ∪ s Under state evolu-tion (2), the mean value follows the differential equation: ˙¯ x t = − E [ W t x t ] . (11) Proof: Taking expectation in (10) yields ¯ x t = ¯ x − E [ (cid:90) t s x s ds ]= ¯ x − (cid:90) t E [ s x s ] ds = ¯ x − (cid:90) t E [ E s [ s x s ]] ds = ¯ x − (cid:90) t E [ W s x s ] ds The first equality uses the definition of ¯ x t . The secondone invokes Fubini’s theorem [24] (since x t ∈ [0 , n andis a finite dimensional matrix with bounded entries). Thethird equality uses the towering property of expectation [23].The fourth uses the fact that x s is F s − − measurable alongwith (9).The next theorem establishes the convergence of ourscheme: Theorem 1 (Average consensus): Under Assumption 1and state evolution (2) the system reaches average consensus: lim t →∞ x t = Ave ( x ) a.s. and in m.s. , for any x ∈ [0 , n , where Ave ( x ) := n (cid:62) x is theaverage of the initial nodal values. Furthermore, the m.s.convergence is exponential in expectation, with the expectedrate lower-bounded by λ ( G (cid:48) ) > . Proof: Since is symmetric and = , pre-multiplying (10) by (cid:62) gives (cid:62) x t = (cid:62) x for all t ≥ ,i.e., the sum (and therefore the average) of entries is constantover time. We define the disagreement vector e t := x t − Ave ( x ) : it follows that e t ⊥ , i.e., (cid:62) e t = for all t ≥ . Consider the Lyapunov function V ( e ) = (cid:107) e (cid:107) = e (cid:62) e .Under (2) it follows that: ˙ V ( e t ) = − e (cid:62) t t e t , where we have used the chain rule and the property that t = . Note that the drift satisfies − e (cid:62) t t e t ≤ since t ispositive semidefinite. Using the exact same line of analysisas in Lemma 1 we get: E [ V ( e t )] = V ( e ) − (cid:90) t E [ e (cid:62) s W s e s ] ds, or more generally: E s [ V ( e t )] = V ( e s ) − (cid:90) ts E s [ e (cid:62) τ W τ e τ ] dτ, Therefore V ( e t ) is a bounded (cf. Proposition 1) (Ω , F t − , P ) − supermartingale, and therefore converges a.s.by the supermartingale convergence theorem [23]. Denotethe limit by e ∞ ( ω ) ; we will establish that e ∞ = a.s. Notethat W ( x ) is a (state-dependent) weighted Laplacian on thegraph G (cid:48) = ( V , E (cid:48) ) which is connected (cf. Assumption 1), therefore λ ( G (cid:48) ) > . Furthermore, the edge weights arepositive and bounded away from zero uniformly over x ; tosee this note that (6), (7) and the fact that x ∈ [0 , n implythat min( w mij , w cij ) ≥ . This also shows that λ ( W ) ≥ λ ( G (cid:48) ) > . Since e t ⊥ for all t ≥ , and by the definition of V ( · ) , we have: E [ V ( e t )] ≤ V ( e ) − λ ( G (cid:48) ) (cid:90) t E [ V ( e s )] ds, consequently E [ V ( e t )] ≤ V ( e ) e − λ ( G (cid:48) ) t , i.e., lim t →∞ E [ V ( e t )] = 0 . This establishes m.s. convergence to , with expectedexponential convergence with rate lower-bounded by λ ( G (cid:48) ) ;a.s.-convergence follows by the supermartingale convergencetheorem and Fatou’s lemma [24]. Corollary 1 (Convergence in expectation): UnderAssumption 1 and state evolution (2): lim t →∞ E [ x t ] = Ave ( x ) . Proof: By Jensen’s inequality, (cid:107) E [ e t ] (cid:107) ≤ E [ (cid:107) e t (cid:107) ] ,therefore lim t →∞ E [ e t ] = , and the result follows by the definition of e t .V. E XPERIMENTS In this section, we present simulation studies that attestour convergence results. We have employed the small worldnetwork model [25], i.e., a Bernoulli random graph in whichany two agents are allowed to interact with a fixed probability p ; this process generates the neighborhood sets {N i } andtherefore the graph G (cid:48) = ( V , E (cid:48) ) , cf. Assumption 1. Wetook the network size n = 1000 in our experiments andset p = 0 . ; we repeated the experiment until a connectedgraph G (cid:48) was obtained as required by Assumption 1. Forinitialization, we chose x uniformly distributed on [0 , n and selected the active neighbor sets N (1) i as follows: foreach i , neighbors in N i were selected to be active withprobability p (independently from one another); we took p = 0 . . Last, we set b = 5 , c = 4 in (6), (7).For numerical simulation of the state evolution we haveperformed uniform discretization of (2) with a step-size ∆ ,i.e., we set t = ∆ k where k is the discrete iterate counterand run: x k +1 = x k − ∆ k x k . We chose the step-size ∆ = n which guarantees that thespectral radius of ( I − ∆ k ) is less than or equal to 1 (since theeigenvalues of the Laplacian are upper bounded by n [16]).Figure 1 depicts the evolution of the coordination levels(for a single small world network and initialization of x ): itis evident that all coordination levels converge to the average ig. 1. Time evolution of coordination levels of 1000 agents.Fig. 2. Normalized disagreement vector norm over time. value. Figure 2 illustrates a logarithmic plot of the evolutionof the normalized norm of the disagreement vector (cid:107) e t (cid:107) (cid:107) e (cid:107) ,referred to as relative error, averaged over 1000 experiments(random topologies G (cid:48) and initializations of x ); it is evidentthat the convergence is exponential, in full alliance withTheorem 1.VI. C ONCLUSIONS AND FUTURE WORK We have proposed and analyzed average consensus on evo-lutionary graphs. Linear consensus iterations are performedon a dynamic graph, where the topology is determined byan evolutionary game in which agents can randomly createnew links or drop existing ones in a selfish manner basedon their fitness function. We have established a.s. and m.s.average consensus with expected exponential rate regardlessof the initial topology and agents’ values.Our future work will focus on devising distributed methodsfor multi-agent optimization over evolutionary graphs, aswell as on extending the analysis to directed graphs anddiscrete-time iterations.R EFERENCES[1] H. Tembine, E. Altman, R. El-Azouzi, and Y. Hayel, “Evolutionarygames in wireless networks,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) , vol. 40, no. 3, pp. 634–646,2010.[2] E. Altman, R. El-Azouzi, Y. Hayel, and H. Tembine, “The evolutionof transport protocols: An evolutionary game perspective,” ComputerNetworks , vol. 53, no. 10, pp. 1751–1759, 2009.[3] C. Sun and H. Duan, “Markov decision evolutionary game theoreticlearning for cooperative sensing of unmanned aerial vehicles,” SciChina Tech Sci , vol. 58, pp. 1392–1400, 2015.[4] H. Tembine and A. P. Azad, “Dynamic routing games: An evolutionarygame theoretic approach,” in , 2011, pp.4516–4521.[5] J. Tsitsiklis and M. Athans, “Convergence and asymptotic agreementin distributed decision problems,” IEEE Transactions on AutomaticControl , vol. 29, no. 1, pp. 42–50, 1984.[6] L. Xiao and S. Boyd, “Fast linear iterations for distributed averaging,” Systems & Control Letters , vol. 53, no. 1, pp. 65–78, 2004.[7] S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, “Randomized gossipalgorithms,” IEEE/ACM Transactions on Networking , vol. 14, pp.2508–2530, 2006.[8] R. Olfati, Saber and R. Murray, “Agreement problems in networkswith directed graphs and switching topology,” in Proceedings of the42nd IEEE Conference on Decision and Control (CDC) , 2003, pp.4126–4132.[9] A. Nediç and A. Ozdaglar, “Distributed subgradient methods for multi-agent optimization,” IEEE Transactions on Automatic Control , vol. 54,no. 1, pp. 48–61, 2009.[10] A. Nediç, A. Olshevsky, and W. Shi, “Achieving geometric conver-gence for distributed optimization over time-varying graphs,” arXivpreprint arXiv:1607.03218 , 2016.[11] A. G. Dimakis, S. Kar, J. M. Moura, M. G. Rabbat, and A. Scaglione,“Gossip algorithms for distributed signal processing,” Proceedings ofthe IEEE , vol. 98, no. 11, pp. 1847–1864, 2010.[12] A. Zouzias and N. Freris, “Randomized Extended Kaczmarz for solv-ing least squares,” SIAM Journal on Matrix Analysis and Applications ,vol. 34, no. 2, pp. 773–793, 2013.[13] N. Freris and A. Zouzias, “Fast distributed smoothing of relativemeasurements,” in Proceedings of the 51st IEEE Conference onDecision and Control (CDC) , 2012, pp. 1411–1416.[14] A. Zouzias and N. Freris, “Randomized gossip algorithms for solvingLaplacian systems,” in Proceedings of the 14th IEEE EuropeanControl Conference (ECC) , 2015, pp. 1920–1925.[15] T. Y. Berger-Wolf and J. Saia, “A framework for analysis of dynamicsocial networks,” in Proceedings of the 12th ACM SIGKDD Interna-tional Conference on Knowledge Discovery and Data Mining , 2006,pp. 523–528.[16] D. West, Introduction to graph theory . Prentice Hall Upper SaddleRiver, 2001, vol. 2.[17] E. Lieberman, C. Hauert, and M. A. Nowak, “Evolutionary dynamicson graphs,” Nature , vol. 433, no. 7023, pp. 312–316, 2005.[18] G. Szabó and G. Fath, “Evolutionary games on graphs,” Physicsreports , vol. 446, no. 4, pp. 97–216, 2007.[19] R. McCabe, N. Freris, and P. R. Kumar, “Controlled random accessMAC for network utility maximization in wireless networks,” in Proceedings of the 47th IEEE Conference on Decision and Control(CDC) , 2008, pp. 2350–2355.[20] N. Freris, H. Kowshik, and P. R. Kumar, “Fundamentals of largesensor networks: Connectivity, capacity, clocks, and computation,” Proceedings of the IEEE , vol. 98, no. 11, pp. 1828–1846, 2010.[21] B. Ranjbar-Sahraei, H. Bou Ammar, D. Bloembergen, K. Tuyls, andG. Weiss, “Evolution of cooperation in arbitrary complex networks,”in Proceedings of the 2014 International Conference on AutonomousAgents and Multiagent Systems , 2014, pp. 677–684.[22] M. Smyrnakis, D. Bauso, P. A. Trodden, and S. M. Veres, “Learningof cooperative behaviour in robot populations,” in Proceedings of the15th IEEE European Control Conference (ECC) , 2016, pp. 184–189.[23] D. Stroock, Probability theory . Cambridge University Press, 1994.[24] H. L. Royden, Real analysis , 3rd ed. Macmillan New York, 1989.[25] D. Watts,