Attack-Resilient Distributed Algorithms for Exponential Nash Equilibrium Seeking
aa r X i v : . [ ee ss . S Y ] S e p Attack-Resilient Distributed Algorithms for Exponential Nash Equilibrium Seeking
Zhi Feng and Guoqiang Hu
Abstract —This paper investigates a resilient distributed Nash equi-librium (NE) seeking problem on a directed communication networksubject to malicious cyber-attacks. The considered attacks, named asDenial-of-Service (DoS) attacks, are allowed to occur aperiodically,which refers to interruptions of communication channels carried outby intelligent adversaries. In such an insecure network environment,the existence of cyber-attacks may result in undesirable performancedegradations or even the failures of distributed algorithm to seek theNE of noncooperative games. Hence, the aforementioned setting canimprove the practical relevance of the problem to be addressed andmeanwhile, it poses some technical challenges to the distributed algo-rithm design and exponential convergence analysis. In contrast to theexisting distributed NE seeking results over a prefect communicationnetwork, an attack-resilient distributed algorithm is presented suchthat the NE can be exactly reached with an exponential convergencerate in the presence of DoS attacks. Inspired by the previous works in[21]–[26], an explicit analysis of the attack frequency and duration isinvestigated to enable exponential NE seeking with resilience againstattacks. Examples and numerical simulation results are given to showthe effectiveness of the proposed design.
Index Terms —Distributed algorithm, NE seeking, Directed graph,Exponential convergence, Cyber-attack, Resilience.
I. I
NTRODUCTION
Recently, distributed Nash equilibrium (NE) seeking of non-cooperative games has been attracting increasingly attention dueto its broad applications in multi-robot systems [1], mobile sensornetworks [3], smart grids [5], and so on. In contrast to early works(e.g., [2], [4], [6]) with a complete information setting, players indistributed NE seeking have limited local information, i.e., eachplayer needs to make the decision based on the local or relativeinformation, e.g., information from its neighbors, to optimize itsown objective function. Thus, the players involved in the gameare required to communicate with each other over the network toestimate other players’ actions.
Literature review: various distributed continuous- and discrete-time algorithms are developed in [8]–[18] to solve distributed NEseeking problems in games, in which each player cannot observeall other players’ actions, but can exchange information betweenneighbors over an undirected graph or weight-balanced digraph.Gradient-based NE seeking algorithms are popular techniques tofind the NE of games with differentiable objective functions whereeach player modifies its current action based on the gradient withrespect to its own action. The distributed NE seeking issue withcontinuous-time agent dynamics is studied in [8]–[14]. In partic-ular, distributed NE seeking strategies are proposed in [8] and [9]by combining the leader-follower based consensus algorithms andgradient-play strategies over an undirected and connected graph.The authors in [11] exploit some incremental passivity propertiesof pseudo-gradients to illustrate that the estimates of the proposedaugmented gradient dynamics converge to the NE exponentially
This work was supported by Singapore Ministry of Education AcademicResearch Fund Tier 1 RG180/17(2017-T1-002-158). Z. Feng and G. Hu arewith the School of Electrical and Electronic Engineering, Nanyang TechnologicalUniversity, Singapore 639798 (E-mail: [email protected]; [email protected]). under the graph coupling condition. To remove this condition, thetwo time-scale singular perturbation analysis is further developed.Distributed NE seeking of aggregative games has been studied in[12]–[14], where each player’ objective function relies on its ownaction and an aggregate of all players’ actions. On the other hand,discrete-time consensus algorithms are presented in [15] to searchfor the NE of congestion games, where each player has the linearcost function. In [16], the gossip-based algorithm is developed toseek the NE. An alternating direction method of multipliers’ algo-rithm with the constant step-size is proposed in [17] by exploitingthe convex and smooth properties of pseudogradients. Recently,[18] adopts consensus-based gradient-free NE seeking algorithmsfor limited cost function knowledge. Distributed algorithms viadiminishing and constant step-sizes are studied, where the formerensures an almost sure NE convergence, while the latter providesconvergence to the neighborhood of the NE.One observation in distributed NE seeking problems is that allplayers require to find the NE via information exchange betweenneighbors through a communication network. Unfortunately, dueto malicious cyber-attacks such as DoS attacks, deception attacks(false data injection, replay attack), disclosure attacks [20]–[30],the secure network environment is hardly guaranteed in practice.The network security plays a fundamental yet critical role in suc-cessful information transmission. The malicious attacks interrupt,incorrect, or tamper transmitting information so that efficiency ofdistributed NE seeking algorithms is degraded significantly, and itmight even lead to the failure of seeking the NE under maliciousattacks. In light of wide applications of distributed algorithms incyber-physical systems (safety-critical), and inspired by studies ofsecurity issues in many existing works (e.g., see [20]–[30]), it ishighly desirable to determine how resilient distributed NE seekingalgorithms will be against malicious attacks based on the fact thatdistributed algorithms are easily disrupted by malicious behaviors.
Thus, the main objective of this work is to address attack-resilientproblems of distributed NE seeking so as to provide certain safetyand resilience performances against attacks. So far, few efforts aremade on resilient distributed NE seeking.
This paper focuses on the attack-resilient research of distributedNE seeking of a non-cooperative game. In particular, an attack-resilient distributed algorithm is proposed to solve the NE seekingproblem in the multi-player non-cooperative game over a directedcommunication network under malicious DoS attacks. The majorcontributions of this paper are as follows. Firstly, to the best ofour knowledge, this paper is the first work to solve this issue. Theproposed distributed NE seeking algorithm is capable of exactlyseeking the NE in the presence of DoS attacks. The exponentialconvergence of the proposed algorithm is rigorously guaranteed,provided that the frequency and duration of attacks satisfy certainbounded conditions. Moreover, compared with existing NE seek-ing works requiring ideal communication in [8]–[18], we developattack-resilient distributed NE seeking algorithms to search for the
NE in adversarial network environments. The algorithms employconsensus-based pesudo-gradient strategies with a hybrid systemmethod to constrain DoS attacks. The explicit analysis is providedaccording to the Lyapunov stability. In addition, different from thedistributed convex optimization works in [31]–[33] that considerfaults on nodes and require the removal of their states to be priorknown, this paper investigates the more practical DoS attack oncommunication network. The attacks are time-sequence based andallowed to occur aperiodically. Another contribution of this workis that unlike results in [8]–[18] and [31]–[33] over an undirectedgraph or weight-balanced digraph, the strongly connected directedgraph is allowed here, and in the presence of malicious attacks,this directed graph can be disrupted or totally paralyzed.The paper is organized as follows. Section II gave mathematicalpreliminaries. In Section III, the non-cooperative game and DoSattack model are described, and the main objective is presented.The attack-resilient distributed algorithm is proposed in SectionIV, where the NE seeking results with an exponential convergenceanalysis are provided. Examples and numerical simulation resultsare given in Section V, followed by the conclusion in Section VI.II. P
RELIMINARIES
A. Notation
Denote R , R n , and R n × m as the sets of the real numbers, real n -dimensional vectors and real n × m matrices, respectively. De-note R ≥ as the set of nonnegative numbers, while N + representsthe set of positive integers. Let n ( n ) be the n × vector withall zeros (ones) and I n be the identity matrix. Let col ( x , ..., x n ) and diag { a , ..., a n } be a column vector with entries x i and adiagonal matrix with entries a i , i = 1 , , · · · , n , respectively.The symbols ⊗ and k·k represent the Kronecker product and theEuclidean norm, respectively. Given a real symmetric matrix M ,let M > ( M ≥ ) denote that M is positive (or positive semi-definite), and λ min ( M ) , λ max ( M ) are its minimum and maximumeigenvalues, respectively. For a function f , it is said to be C m if itis m th continuously differentiable. For two sets X and Y , Y \ X denotes the set of elements belonging to Y, but not to X. B. Convex analysis
A function f : R n → R is convex if f ( ax +(1 − a ) y ) ≤ af ( x )+(1 − a ) f ( y ) for any scalar a ∈ [0 , and vectors x, y ∈ R n . f islocally Lipschitz on R n if it is locally Lipschitz at x for ∀ x ∈ R n .If f is a differentiable function, ▽ f denotes the gradient of f . Avector-valued function (or mapping) F : R n → R n is said to be ι F -Lipschitz continuous if, for any x, y ∈ R n , k F ( x ) − F ( y ) k ≤ ι F k x − y k . Function F : R n → R n is (strictly) monotone if, forany x, y ∈ R n , ( x − y ) T ( F ( x ) − F ( y ))( > ) ≥ . Further, F isa ε -strongly monotone, if for any scalar ε > , and x, y ∈ R n , ( x − y ) T ( F ( x ) − F ( y )) > ε k x − y k . C. Graph Theory
Let ¯ G = (cid:8) V , ¯ E (cid:9) denote a directed graph, where V ∈ { , ..., N } is a set of nodes and ¯ E ⊆ V × V is a set of edges. An edge ( i, j ) ∈ ¯ E denotes that i th agent receives the information from j th agent,but not vice versa. ¯ N i = (cid:8) j ∈ V | ( j, i ) ∈ ¯ E (cid:9) is a neighborhoodset of the agent i . A directed graph is strongly connected if thereexists a directed path connecting every pair of nodes. The matrix ¯ A = [¯ a ij ] denotes the adjacency matrix of ¯ G , where ¯ a ij > ifand only if ( j, i ) ∈ ¯ E , else ¯ a ij = 0 . A matrix ¯ L = [¯ l ij ] ∈ R N × N is called the Laplacian matrix of ¯ G , where ¯ l ii = P Nj =1 ¯ a ij and ¯ l ij = − ¯ a ij , i = j . Denote ¯ L , ¯ D − ¯ A , where ¯ D = (cid:2) ¯ d ii (cid:3) is thediagonal matrix with ¯ d ii = P Nj =1 ¯ a ij . Similarly, let G denote aweighted digraph, where a ij = ω i ¯ a ij for a scalar ω i > . Withthe same definition of ¯ L , the Laplacian matrix of this new digraphbecomes L = W ¯ L , where W = diag { ω , · · · , ω N } . Assumption 1:
The directed graph ¯ G is strongly connected. Lemma 1: [37] By Assumption 1, the Laplacian matrix ¯ L of ¯ G has the following properties.1) ¯ L has a simple zero eigenvalue associated with right eigen-vector N , and nonzero eigenvalues have positive real parts;2) ω = col ( ω , · · · , ω N ) with ω T N = 1 is a left eigenvectorof ¯ L associated with the zero eigenvalue. Then, ω T ¯ L = 0 TN and min ζ T x =0 , x =0 x T ˆ L xx T x > λ ( ˆ L ) N , where ˆ L = ( L + L T ) and ζ is any vector with positive entries. Moreover, ω = 1 N if and only if ¯ G is strongly connected and weight-balanced.III. P ROBLEM F ORMULATION
A. Non-cooperative Game over Networks
In this paper, we consider a multi-agent network consisting of N players, which form a N-player non-cooperative game definedas follows. For each agent i ∈ V , the i th player aims to minimizeits cost function J i ( x i , x − i ) : R n i → R by choosing its strategy x i ∈ R n i , and x − i = col ( x , · · · , x i − , · · · , x N ) is the strategyprofile of the whole strategy profile except for player i . Let x =( x i , x − i ) represent all players’ action profile. Alternatively, let x = col ( x , · · · , x N ) ∈ R n , n = P i ∈V n i .The concept of Nash equilibrium is given below. Definition 1: (Nash equilibrium)
A strategy profile x ∗ = ( x ∗ i ,x ∗− i ) ∈ R n is said to be an Nash equilibrium of the game if J i ( x ∗ i , x ∗− i ) ≤ J i ( x i , x ∗− i ) , for ∀ x i ∈ R n i , i ∈ V . (1)Condition (1) means that all players simultaneously take theirown best (feasible) responses at the NE x ∗ , where no player canunilaterally decrease its cost by changing its strategy. Assumption 2:
For each player i , J i ( x i , x − i ) is C , strictlyconvex, and radially unbounded in x i for each x − i .Under Assumption 2, it follows from [35] that an NE x ∗ exists,and satisfies ▽ i J i ( x ∗ i , x ∗− i ) = 0 n i , and ▽ i J i ( x i , x − i ) = ∂J i ( x i ,x − i ) /∂x i ∈ R n i represents the partial gradient of player i ’s costwith respect to its own action x i . We define F ( x ) , col ( ▽ J ( x , x − ) , · · · , ▽ N J N ( x N , x − N )) , (2)where F ( x ) ∈ R n denotes the pseudo-gradient (the stacked vectorof all players’ partial gradient). Thus, we have F ( x ∗ ) = 0 n . Assumption 3:
The pesudogradient F is ε -strongly monotoneand ι F -Lipschitz continuous for certain constants ε, ι F > . Remark 1:
Assumptions 2 and 3 were widely used in existingworks (e.g., [10]–[18]) to guarantee the unique NE x ∗ .In a game with perfect information (a complete communicationgraph) requiring the global knowledge on all players’ actions, agradient-play algorithm ˙ x i = − ▽ i J i ( x i , x − i ) can be used to seekthe NE asymptotically (Lemma 2, [11]). However, this design isimpractical if the communication network is not complete. Then,a distributed algorithm is desirable to broadcast the local informa-tion among players. Unfortunately, when distributed computationsare required and executed over insure communication networksunder cyber-attacks, the exact NE may not be found. Fig. 1. An illustration of DoS attacks on communication channels ( i, j ) occurringat s, s, a ijm s with their durations being s, s, τ ijm s, respectively. Thus, we obtain Ξ ( i,j ) a (3 ,
10) = [3 , ∪ [9 , for just an example. B. Malicious DoS Attack Model
As studied in the preliminary results [21]–[26], the well-knownDoS attack refers to a class of malicious cyber-attacks where anattacker aims to corrupt and interrupt certain or all componentsof communication channels between the players. Without loss ofgenerality, we suppose that each adversary follows an independentattacking strategy with a varying active period. Due to the limitedenergy, then it terminates attacking activities and shifts to a sleepperiod to supply its energy for next attacks.To be specific, suppose that there exists an m ∈ N and denote { a ijm } m ∈ N as an attack sequence when a DoS attack is lunched atchannel ( i, j ) . Let the duration of this attack be τ ijm ≥ . Then, the m -th attack strategy is generated with A ( i,j ) m = a ijm ∪ [ a ijm , a ijm + τ ijm ) where a ijm +1 > a ijm + τ ijm for m ∈ N + . Given t ≥ t , the setsof time instants where communication between ( i, j ) is denied,are described in the following form [24]–[26]: Ξ ( i,j ) a ( t , t ) = ( ∪ ∞ m =1 A ( i,j ) m ) ∩ [ t , t ] , m ∈ N + , (3)which implies that the sets where communication between ( i, j ) isallowed, are: Ξ ( i,j ) s ( t , t ) = [ t , t ] \ Ξ ( i,j ) a ( t , t ) . Then, | Ξ a ( t , t ) | = |∪ ( i,j ) ∈E Ξ ( i,j ) a ( t , t ) | is the total length of attacker being activeover [ t , t ] , while | Ξ s ( t , t ) | = [ t , t ] \ | Ξ a ( t , t ) | denotes the totallength of attacker being sleeping over [ t , t ] . Definition 2: (Attack Frequency)
For any T > T ≥ t , let N a ( T , T ) denote the total number of DoS attacks over [ T , T ) .Then, F a ( T , T ) = N a ( T ,T ) T − T denotes the attack frequency over [ T , T ) for ∀ T > T ≥ t , where there exists scalars N , T f > so that N a ( T , T ) ≤ N + ( T − T ) /T f . Definition 3: (Attack Duration)
For any T > T ≥ t , denote | Ξ a ( T , T ) | as the total time interval under attacks over [ T , T ) . The attack duration over [ T , T ) is defined as: there exist scalars T ≥ , T a > so that | Ξ a ( T , T ) | ≤ T + ( T − T ) /T a . Remark 2:
As investigated in the pioneer works [21]–[26] formulti-agent systems under attacks, Definitions 2 and 3 are firstlyintroduced in [21] to specify attack signals in terms of frequencyand time-ratio constraints. The m -th DoS attacks occurring at a ijm between communication channels ( i, j ) with τ ijm are allowed tooccur aperiodically and can interrupt any communication channelssynchronously or asynchronously. Fig. 1 provides an example formore details. In Definition 2, /T f provides an upper bound onthe average DoS frequency, while /T a in Definition 3 providesan upper bound on the average DoS duration. It requires attacksto neither occur at an infinitely fast rate or be always activated. C. Main Objective
This work aims to study an attack-resilient issue of NE seekingunder an insecure communication network as follows.
Problem 1: ( Attack-Resilient Distributed NE Seeking )Consider a non-cooperative game consisting of N players com-municating over an insecure communication network induced byDoS attacks. Design an attack-resilient distributed NE algorithmso that all players can exactly reach the NE x ∗ with an exponentialconvergence rate and a resilient feature against attacks.minimize J i ( x i ( t ) , x − i ( t )) , x i ( t ) ∈ R n i , i = 1 , · · · , N, subject to: ˙ x i ( t ) = u i ( t ) , t ∈ Ξ ( i,j ) s ( t , t ) ∪ Ξ ( i,j ) a ( t , t ) . (4) Remark 3:
In contrast to existing works in [8]–[18], solvingProblem 1 is much more challenging at least from the followingaspects: (1) Player communication network: the games involvedin an insecure communication network may lead to the interrup-tion of communication transmission caused by DoS attacks, whichmakes existing NE seeking algorithms [8]–[18] inapplicable. (2)Assumption: in the absence of attacks, the graph is directed ratherthan being an undirected graph or weight-balanced digraph in [8]–[18]. Further, this digraph under DoS attacks can be disconnectedor totally paralyzed, which brings nontrivial convergence analysis. (3) Design requirement: propose an attack-resilient distributed NEseeking scheme with the exponential convergence and resiliencefeatures against attacks. Due to aforementioned challenges, exist-ing NE seeking algorithms cannot be directly applied.
D. Motivating Example
In order to illustrate the influence of DoS attacks on distributedNE seeking, the following motivation example is provided. Weconsider a classic example in economy, namely the Nash-Cournotgame (e.g., [11], [38]). This game includes some firms involved inthe production of a homogeneous commodity, where the quantityproduced by firm i ∈ V is denoted by x i , and the overall costfunction of each firm i is described by J i ( x i , x − i ) = g i ( x i ) − x i f ( x ) , i ∈ V , (5)where g i ( x i ) = a i + b i ( x i − c i ) + d i x i is the production cost with a i , b i , c i , d i describing the characteristics of firm i , and f ( x ) = f − f P Nj =1 x j is the commodity price with constants f , f . Asstudied in [11], these parameters are chosen as a i = c i = d i = 0 , b i = 10 + 4( i − , f = 720 , and f = 1 . By certain calculations,the NE is x ∗ = col (110 , , , , , .Next, we consider the following different cases:i) in the absence of attacks, the existing distributed NE seekingalgorithm in [11] is performed, and Fig. 2 shows that all players’strategies can reach consensus and converge to the NE. Time [sec] S t r a t eg i e s o f a ll p l a y e r s x *1 x *2 x *3 x *4 x *5 x *6 Fig. 2. Simulated results generated by the existing algorithm [11] in the absenceof DoS attacks. Left: plot of players’ strategies and the NE x ∗ i ; Right: plot of anundirected graph in [11] and its associated Laplacian matrix. ii) in the presence of attacks, this algorithm is further performedunder graphs in Fig. 3(a) and the simulation result is shown in Fig.3(b). As we see, all players’ strategies neither reach consensus orconverge to the NE. In contrast, the simulation result generatedby the proposed attack-resilient algorithm is shown in 3(c), whereall players’ strategies reach consensus and converge to the NE. (a) Various graphs under DoS attacks on communication channels Time [sec] S t r a t eg i e s o f a ll p l a y e r s x *1 x *2 x *3 x *4 x *5 x *6 (b) Algorithm in [11] Time [sec] S t r a t eg i e s o f a ll p l a y e r s x *1 x *2 x *3 x *4 x *5 x *6 (c) Proposed attack-resilient algorithmFig. 3. Simulated results generated by different algorithms under DoS attacks. IV. A
TTACK -R ESILIENT D ISTRIBUTED
NE S
EEKING
In distributed NE seeking games, each player i has no accessto the full information of all players’ strategies. Then, each agent i shall estimate all other players’ strategies. Inspired by [11], leteach player combine its gradient-play dynamics with an auxiliarydynamics, i.e., implement the following dynamics: (cid:26) ˙ x ii = u ii , u ii = − ▽ i J i ( x ii , x i − i ) + e ii , i ∈ V , ˙ x ij = u ij , u ij = e ij , ∀ j ∈ V , j = i, (6)where player i maintains an estimate vector x i = col ( x i , · · · , x ii , · · · , x iN ) in which x ij is player i ’s estimate of player j ’s action, x ii = x i is the player i ’s actual action, x i − i is the player i ’ estimatevector without its own action, u ii = u i is the player i ’s actualinput, u ij is the other players’ input, and e ii , e ij are to be developed.In (6), each player i updates x ii to reduce its own cost function andupdates x ij to reach consensus with the other players. In addition,each player i relies on its local estimated action x i − i .For each player i , (6) can be rewritten in a compact form ˙ x i = −R Ti ▽ i J i ( x i ) + e i , i ∈ V , (7)where e i = col ( e i , · · · , e ii , · · · , e iN ) ∈ R n is a relative estimatederror to be designed, and R i ∈ R n i × n used to align the gradientto the action component, is a matrix given by R i = (cid:2) n i × n · · · n i × n i − I n i × n i n i × n i +1 · · · n i × n N (cid:3) . (8) A. Attack-Resilient Distributed NE Seeking Algorithm Design
From the DoS attack model in the subsection III-B, we considersequence-based attacks where the m -th attack is lunched over acommunication channel ( i, j ) . Without loss of generality, supposethat there exists an infinite sequence k = 0 , , , · · · for intervals [ t k , t k +1) ) such that in the absence of DoS attacks, each player i updates its input e i based on an original communication networkduring [ t k , t k +1 ) , while for each communication channel ( i, j ) under DoS attacks during [ t k +1 , t k +2 ) , the attackers interruptits information transmission to make original connected commu-nication network disrupted or totally paralyzed.To analyze the influence of DoS attacks, we propose an attack-resilient distributed NE seeking algorithm as ˙ x i = −R Ti ▽ i J i ( x i ) + e i , t ∈ [ t k , t k +2 ) , k = 0 , , · · · , (9)where e i denotes the relative estimated errors under attacks e i = ( − κ P Nj =1 a ij ( x i − x j ) , t ∈ [ t k , t k +1 ) , P Nj =1 a Ψ( t ) ij ( x j − x i ) , t ∈ [ t k +1 , t k +2 ) , (10)where κ is a positive constant gain to be specified later, and the a Ψ( t ) ij , t ∈ [ t k +1 , t k +2 ) , dependent on an attack flag ψ ( i, j, t ) , isdefined as a Ψ( t ) ij = 0 if ψ ( i, j, t ) = 1 or − ; otherwise a Ψ( t ) ij = 1 if ψ ( i, j, t ) = 0 . The expression of ψ ( i, j, t ) is presented below.Set the initial value ψ ( i, j, t ) = 0 for ∀ ( i, j ) ∈ ¯ E , and then eachplayer i updates the attack flag ψ ( j, i, t ) as follows1) if player i can receive information from player j at t , then ψ ( i, j, t ) = 0 and it sends its information to player j ;2) if player i cannot receive information from player j at t ,then ψ ( i, j, t ) = 1 and it sends ψ ( i, j, t ) = 1 to player j ;3) if player i receives ψ ( i, j, t ) = 1 which means it knows theattacking of channel ( i, j ) , then denote ψ ( i, j, t ) = − andsend its information to player j . Remark 4:
To facilitate understandings of (10), Fig. 4 showsthe schematic of time sequences with and without DoS attacks.Intuitively, not all communication networks are secured anytimein practice, while it is reasonable to secure some original network.In the presence of DoS attacks, e i in (10) relies on ψ ( i, j, t ) . Eachplayer updates this attack flag once an attack signal over commu-nication channels is detected by certain devices or mechanisms.The attack detection design is beyond the scope of this work. Remark 5:
In the absence of DoS attacks, e i in (10) becomes e i = − κ P Nj =1 a ij ( x i − x j ) , which is a modified version of thedesign in [11] that requires a restrictive graph coupling conditionunder an undirected graph. In contrast, an adjustable proportionalcontrol gain κ is introduced to enable a natural trade-off betweenthe control effort and graph connectivity under Assumption 1. Fig. 4. Schematic of time sequences with and without DoS attacks.
Next, denote the following stacked vectors and matrices x = col ( x , · · · , x N ) , R = diag {R , · · · , R N } , (11) e = col ( e , · · · , e N ) , F(x) = col ( ▽ J ( x ) , · · · , ▽ N J N ( x N )) . Combining (9)-(11) gives rise to the closed-loop system underattacks in the sense of a compact form ˙ x = (cid:26) −R T F ( x ) − κ ( L ⊗ I n ) x , t ∈ [ t k , t k +1 ) , −R T F ( x ) − ( L Ψ( t ) ⊗ I n ) x , t ∈ [ t k +1 , t k +2 ) , (12)where L is the Laplacian matrix of the original strongly connecteddigraph, while L Ψ( t ) is the Laplacian matrix of various potentialgraphs under attacks, and its zero eigenvalues may not be simpleas those graphs can be unconnected under attacks. B. Stability Analysis with An Exponential Convergence Rate
Before presenting the main result, we show that in the absenceof attacks, the equilibrium of the system occurs when all playerscan reach consensus at the NE.
Proposition 1:
Consider the game over the directed commu-nication graph ¯ G . Then, under Assumptions 1-3, ˜ x = 1 N ⊗ x ∗ isthe NE of the game on networks in the absence of DoS attacks if F (˜ x ) = 0 n (or ▽ i J i (˜ x i ) = 0 n i ). At the NE, estimated vectors ofall players reach consensus and equal to the NE x ∗ . Thus, players’action components coincide with optimal actions ( ˜ x ii = x ∗ i ). Proof:
In the absence of DoS attacks, let ˜ x be an equilibriumof the system. Then, we have Nn = −R T F (˜ x ) − κ ( L ⊗ I n )˜ x , which implies that multiplying both sides by TN ⊗ I n yields n = − (1 TN ⊗ I n ) R T F (˜ x ) − κ (1 TN ⊗ I n )( L ⊗ I n )˜ x . (13)Since TN L = 0 TNn under Assumption 1, we obtain n = (1 TN ⊗ I n ) R T F (˜ x ) . Then, it follows from the notations of R and F in(11) that F (˜ x ) = 0 n . Then, submitting it into (13) gives rise to ( L ⊗ I n )˜ x = 0 Nn . Hence, there exists certain θ ∈ R n such that ˜ x = 1 N ⊗ θ under Assumption 1. Then, it has F (1 N ⊗ θ ) = 0 n for each player i ∈ V . Thus, ▽ i J i ( θ i , θ − i ) = 0 n i . That is, θ is aunique NE of the game and θ = x ∗ . Thus, ˜ x = 1 N ⊗ x ∗ and for i, j ∈ V , we have ˜ x i = ˜ x j = x ∗ (NE of the game).Notice that in the presence of DoS attacks, the system becomes ˙ x = −R T F ( x ) − ( L Ψ( t ) ⊗ I n ) x . Due to the fact that the existenceand uniqueness of the NE x ∗ are guaranteed under Assumptions2 and 3, then, following a similar analysis above can give rise to n = (1 TN ⊗ I n ) R T F (˜ x ) − (1 TN ⊗ I n )( L Ψ( t ) ⊗ I n )˜ x . However,the presence of DoS attacks will make TN L Ψ( t ) = 0 TNn not holdand then, consensus estimates on the NE cannot be reached underattacks because there may not have correct information exchangeamong all players. Under such a situation, x ∗ may not be the NE.Next, the main task is an explicit analysis of the frequency andduration of attacks to guarantee ˜ x i = ˜ x j = x ∗ .Next, we present the main result on the resilient distributed NEseeking on networks under DoS attacks. Theorem 1:
Under Assumptions 1-3, Problem 1 can be solv-able for any x i (0) under the proposed resilient distributed opti-mization algorithm in (9)-(10) provided that for κ > λ ( ˆ L ) ( ι ε + ι ) and positive scalars λ a , λ b , u to be determined later, the followingtwo attack-related conditions are satisfied:(1). There exists constants η ∗ ∈ (0 , λ a ) and µ > so that T f in the attack frequency Definition 2 satisfies the condition: T f > T ∗ f = 2 ln( µ ) /η ∗ , (14)(2). There exist constants λ a , λ b > such that T a in the attackduration Definition 3 that satisfies the condition: T a > T ∗ a = ( λ a + λ b ) / ( λ a − η ∗ ) . (15)Moreover, the estimated states can converge to the NE with anexponential convergence rate, i.e., || x ( t ) − ˜ x || ≤ ςe − η ( t − t ) || x ( t ) − ˜ x || , ∀ t ≥ , (16)where ς is a positive scalar and η = λ a − ( λ a + λ b ) /T a − η ∗ > . Proof:
The proof includes four steps:
Step i): when communication networks do not suffer from DoS attacks during [ t k , t k +1 ) , we first show that the NE seeking canbe achieved exponentially under a strongly connected digraph.Now, we first make a coordinate transformation as −→ x = (1 N ⊗ S ) x ∈ R Nn , S = 1 N (1 TN ⊗ I n ) , (17) ←− x = ( T ⊗ I n ) x ∈ R Nn , T = I N − N (1 N TN ) . (18)Then, it follows from (17) that the average estimate of x i canbe described by ¯ x = N P Ni =1 x i = N (1 TN ⊗ I n ) x = S x . Further,for any x ∈ R Nn , it can be decomposed as x = −→ x + ←− x with ( −→ x ) T ←− x = 0 and ( L ⊗ I n ) −→ x = 0 Nn under Assumption 1.For stability analysis, we select the Lyapunov function as V a ( x ) = α x − ˜ x ) T ( x − ˜ x ) = α −→ x + ←− x − ˜ x ) T ( −→ x + ←− x − ˜ x )= 12 (cid:20) −→ x − ˜ x ←− x (cid:21) T P a (cid:20) −→ x − ˜ x ←− x (cid:21) , (19)where P a = diag { αI Nn , αI Nn } , α > is an adjustable constant,and the fact that ( −→ x ) T ←− x = 0 and TN T = 0 TN are used.The time derivative of V a ( x ) is expressed as ˙ V a ( x ) = − α ( x − ˜ x ) T [ R T F ( x ) + κ ( L ⊗ I n ) x ] . (20)As Nn = −R T F (˜ x ) − κ ( L ⊗ I n )˜ x , (20) can be expressed as ˙ V a ( x ) = − α ( x − ˜ x ) T [ R T ( F ( x ) − F (˜ x ))+ κ ( L⊗ I n )( x − ˜ x )] . (21)In light of ˜ x = 1 N ⊗ x ∗ , x = −→ x + ←− x , and ( L ⊗ I n ) −→ x = 0 Nn under Assumption 1, then the first term in (21) becomes − ( x − ˜ x ) T R T [ F ( x ) − F (˜ x )] (22) = − ( ←− x ) T R T [ F ( x ) − F ( −→ x )] − ( ←− x ) T R T [ F ( −→ x ) − F (˜ x )] − ( −→ x − ˜ x ) T R T [ F ( x ) − F ( −→ x )] − ( −→ x − ˜ x ) T R T [ F ( −→ x ) − F (˜ x )] . It follows from Assumption 3 that according to the ι F -Lipschitzcontinuity of F , it yields that k F ( x ) − F ( −→ x ) k ≤ ι F k←− x k . Further, k F ( x ) − F ( −→ x ) k ≤ ι F k←− x k for certain scalar ι F > . In addition,since kR T k = 1 , F ( −→ x ) = F (¯ x ) , and F (˜ x ) = F ( x ∗ ) = 0 , − ( ←− x ) T R T [ F ( −→ x ) − F (˜ x )] = − ( ←− x ) T R T ( F (¯ x ) − F ( x ∗ )) ≤ ι F k←− x kk ¯ x − x ∗ k , (23) − ( −→ x − ˜ x ) T R T [ F ( x ) − F ( −→ x )] = − (¯ x − x ∗ ) T ( F ( x ) − F ( −→ x )) ≤ ι F k ¯ x − x ∗ kk←− x k , (24)where the fact that R−→ x = ¯ x and R ˜ x = x ∗ is used, and exploitingthe ε -strong monotonicity of F , we can obtain − ( −→ x − ˜ x ) T R T [ F ( −→ x ) − F (˜ x )]= − (¯ x − x ∗ ) T ( F (¯ x ) − F ( x ∗ )) ≤ − ε k ¯ x − x ∗ k . (25)In addition, the second term in (21) can be rewritten as − ( x − ˜ x ) T ( L ⊗ I n )( x − ˜ x ) = − ( −→ x + ←− x ) T ( L ⊗ I n )( −→ x + ←− x )= − ←− x T [( L T + L ) ⊗ I n ] ←− x ≤ − λ ( ˆ L ) k←− x k , (26)where λ ( ˆ L ) is a minimal positive eigenvalue of ˆ L = ( L T + L ) .Let ι = max { ι F , ι F } . Substituting (23)-(26) into (21) gives ˙ V a ( x ) ≤ − α { ( κλ ( ˆ L ) − ι ) k←− x k + ε k ¯ x − x ∗ k − ι k←− x kk ¯ x − x ∗ k} = − α (cid:20) k ¯ x − x ∗ kk←− x k (cid:21) T (cid:20) ε − ι − ι κλ ( ˆ L ) − ι (cid:21) (cid:20) k ¯ x − x ∗ kk←− x k (cid:21) , = − (cid:20) k−→ x − ˜ x kk←− x k (cid:21) T Q a (cid:20) k−→ x − ˜ x kk←− x k (cid:21) , (27) where the fact that k ¯ x − x ∗ k = √ N k−→ x − ˜ x k is used and Q a = 2 α " εN − ι √ N − ι √ N κλ ( ˆ L ) − ι > if κ > λ ( ˆ L ) ( ι ε + ι ) .Thus, it concludes that there exists positive scalars α, ǫ, ι , anda matrix P a , so that for the Lyapunov function in (19), V a ( x ) = 12 χ T P a χ ⇒ ˙ V a ( x ) ≤ − λ a V a ( x ) , t ∈ [ t k , t k +1 ) , (28)where χ = col ( −→ x − ˜ x , ←− x ) and λ a = λ min ( Q a ) /λ max ( P a ) > . Step ii): when considering the presence of DoS attacks during [ t k +1 , t k +1) ) , we select the Lyapunov function as V b ( x ) = β x − ˜ x ) T ( x − ˜ x ) = 12 χ T P b χ, (29)where P b = diag { βI Nn , βI Nn } , β = α is a positive constant.In the presence of DoS attacks, ˙ x = −R T F ( x ) − ( L Ψ( t ) ⊗ I n ) x by (12). Since x ∗ is the unique NE, Nn = −R T F (˜ x ) − ( L Ψ( t ) ⊗ I n )˜ x . By adding this equation, it follows from (22)-(27) that thetime derivative of V b ( x ) can be described by ˙ V b ( x ) = − β ( x − ˜ x ) T [ R T ( F ( x ) − F (˜ x )) + ( L Ψ( t ) ⊗ I n )( x − ˜ x )] ≤ − β (cid:20) k ¯ x − x ∗ kk←− x k (cid:21) T (cid:20) ε − ι − ι − ι (cid:21) (cid:20) k ¯ x − x ∗ kk←− x k (cid:21) − β ( −→ x − ˜ x + ←− x ) T ( L Ψ( t ) ⊗ I n )( −→ x − ˜ x + ←− x ) ≤ β (cid:20) k−→ x − ˜ x kk←− x k (cid:21) T " − εN ι √ Nι √ N ι k−→ x − ˜ x kk←− x k (cid:21) + β kL Ψ( t ) k (cid:20) k−→ x − ˜ x kk←− x k (cid:21) T (cid:20) k−→ x − ˜ x kk←− x k (cid:21) = 12 (cid:20) k−→ x − ˜ x kk←− x k (cid:21) T Q b (cid:20) k−→ x − ˜ x kk←− x k (cid:21) (30)where Q b = 2 β " c − εN c + ι √ N c + ι √ N c + ι with c = kL Ψ( t ) k , mayhave both positive and negative eigenvalues.Thus, it concludes that there exists positive scalars β, ǫ, ι , anda matrix P b , so that for the Lyapunov function in (29), V b ( x ) = 12 χ T P b χ ⇒ ˙ V b ( x ) ≤ λ b V b ( x ) , t ∈ [ t k +1 , t k +2 ) , (31)where λ a = σ max ( Q b ) /λ min ( P b ) > and σ max ( Q b ) denotes themaximum singular value of Q b . Step iii): we analyze the exponential convergence of the closed-loop system from a switching perspective [21]–[25].Let δ ( t ) ∈ { a, b } be a switching signal. Then, we can choose V ( t ) = (cid:26) V a ( x ) , if t ∈ [ t k , t k +1 ) ,V b ( x ) , if t ∈ [ t k +1 , t k +2 ) , (32)where V a ( x ) and V b ( x ) are defined in (19) and (29), respectively.Suppose that V a is activated in [ t k , t k +1 ) , while V b is acti-vated in [ t k +1 , t k +2 ) . Then, by (28) and (31), we have V ( t ) ≤ (cid:26) e − λ a ( t − t k ) V a ( t k ) , if t ∈ [ t k , t k +1 ) ,e λ b ( t − t k +1 ) V b ( t k +1 ) , if t ∈ [ t k +1 , t k +2 ) . (33)The closed-loop system is switched at t = t +2 k or t = t +2 k +1 .Let µ = max { λ max ( P a ) /λ min ( P b ) , λ max ( P b ) /λ min ( P a ) } > , and next, we discuss the following two cases:Case a): If t ∈ [ t k , t k +1 ) , it follows from (33) that V ( t ) ≤ e − λ a ( t − t k ) V a ( t k ) ≤ µe − λ a ( t − t k ) V b ( t − k ) ≤ µe − λ a ( t − t k ) [ e λ b ( t k − t k − ) V b ( t k − )] ≤ µe − λ a ( t − t k ) e λ b ( t k − t k − ) [ µV a ( t − k − )]= µ e − λ a ( t − t k ) e λ b ( t k − t k − ) V a ( t − k − ) ≤ µ e − λ a ( t − t k ) e λ b ( t k − t k − ) [ e − λ a ( t k − − t k − ) V a ( t k − )] ≤ · · ·≤ µ k e − λ a | Ξ s ( t ,t ) | e λ b | Ξ a ( t ,t ) | V a ( t ) . (34)Case b): If t ∈ [ t k +1 , t k +1) ) , it follows from (33) that V ( t ) ≤ e λ b ( t − t k +1 ) V b ( t k +1 ) ≤ µe λ b ( t − t k +1 ) V a ( t − k +1 ) ≤ µe λ b ( t − t k +1 ) [ e − λ a ( t k +1 − t k ) V a ( t k ) ≤ µe λ b ( t − t k +1 ) e − λ a ( t k +1 − t k ) [ µV b ( t − k − )] ≤ µ e λ b ( t − t k +1 ) e − λ a ( t k +1 − t k ) [ e λ b ( t k − − t k − ) V b ( t k − )] ≤ · · ·≤ µ k +1 e − λ a | Ξ s ( t ,t ) | e λ b | Ξ a ( t ,t ) | V a ( t ) . (35) Step iv): we consider bounds on attack frequency and duration.Notice that N a ( t , t ) = k for t ∈ [ t k , t k +1 ) and k + 1 for t ∈ [ t k +1 , t k +1) ) . Thus, for ∀ t ≥ t , by (34) and (35), V ( t ) ≤ µ N a ( t ,t ) e − λ a | Ξ s ( t ,t ) | e λ b | Ξ a ( t ,t ) | V ( t ) . (36)Notice that for all t ≥ t , | Ξ s ( t , t ) | = t − t − | Ξ a ( t , t ) | and | Ξ a ( t , t ) | ≤ T + ( t − t ) /T a by Definition 3. Thus, we have − λ a ( t − t − | Ξ a ( t , t ) | ) + λ b | Ξ a ( t , t ) | = − λ a ( t − t ) + ( λ a + λ b ) | Ξ a ( t , t ) |≤ − λ a ( t − t ) + ( λ a + λ b )[ T + ( t − t ) /T a ] . (37)Next, substituting (37) into (36) yields V ( t ) ≤ µ N a ( t ,t ) e − λ a ( t − t −| Ξ a ( t ,t ) | ) e λ b | Ξ a ( t ,t ) | V ( t ) (38) ≤ e ( λ a + λ b ) T e − λ a ( t − t ) e ( λa + λb ) τa ( t − t ) e µ ) N a ( t ,t ) V ( t ) . By exploiting the attack condition in (14), we can have µ ) N a ( t , t ) ≤ µ ) N + η ∗ ( t − t ) . (39)Let η = λ a − ( λ a + λ b ) /T a − η ∗ > . Based on another attackcondition in (15), and using (39), we can rewrite (38) as V ( t ) ≤ e ( λ a + λ b ) T +2 ln( µ ) N e − η ( t − t ) V ( t ) . (40)Further, it follows from (19), (29), and (40) that || χ ( t ) || ≤ ςe − η ( t − t ) || χ ( t ) || , (41)where ς = e ( λ a + λ b ) T +2 ln( µ ) ς a /ς b , ς a = max { λ max ( P a ) , λ max ( P b ) } , and ς b = min { λ min ( P a ) , λ min ( P b ) } .Therefore, it follows from (41) that all estimate states −→ x − ˜ x and ←− x are bounded, and converge to zero exponentially. Furthermore, lim t →∞ ( −→ x − ˜ x ) = 0 Nn and lim t →∞ ←− x = 0 Nn . Then, accordingto the coordinate transformation −→ x , ←− x in (17) and (18), and byusing the fact that x = −→ x + ←− x , we can obtain that x exponentiallyconverges to ˜ x = 1 ⊗ x ∗ . That is, Problem 1 is solved. Remark 6:
Theorem 1 presented the main resilient distributedNE seeking result with an exponential convergence rate η = λ a − ( λ a + λ b ) /T a − η ∗ . Here, (1 − T a ) λ a is mainly used to measurethe average rate of exponential decay of the stable subsystems,while λ b /T a isy used to measure the exponential growth rate ofunstable subsystems. The η ∗ explains the exponential growth dueto switchings. Note that the convergence rate is not only affectedby λ a and λ b that rely on the communication topology, number ofplayers, and control gains, but the attack frequency and duration.Moreover, the larger values of attack frequency and duration are,the more active that those attacks are allowed to be.Notice that in the absence of DoS attacks, Theorem 1 can bereduced to the following corollary: Corollary 1:
Under Assumptions 1-3, the following distributedNE seeking algorithm enables all players’ estimated strategies toexponentially converge to the NE provided κ > ( ι ε + ι ) /λ ( ˆ L ) . ˙ x i = −R Ti ▽ i J i ( x i ) + e i , e i = − κ N X j =1 a ij ( e i − e j ) , i ∈ V . Remark 7:
The corollary can cover some existing results (e.g.,[8], [11]) as special cases. Moreover, it can avoid restrictive graphcoupling conditions in [11] by adding a proportional gain κ , andremove the use of two-timescale singular perturbation that yieldssemi-global convergence in [8], [11]. The design does not requireany initial requirements and allow for a general directed graph.V. N UMERICAL S IMULATIONS
In this section, numerical examples are presented to verify theeffectiveness of the proposed NE seeking design. The communi-cation graph for all players in examples is depicted in Fig. 5, inwhich the original strongly connected digraph and the paralyzedgraphs induced by attacks are presented, respectively.
Fig. 5. Communication graph for the players in the examples: (a) original stronglyconnected digraph; and (b)-(d) paralyzed graphs under various DoS attacks.
Example 1: (Energy Consumption Game)In this example, we consider an energy consumption game of N players for Heating Ventilation and Air Conditioning (HVAC)system (see [8]), where the cost function of each player i can bemodeled by the following function: J i ( x i , x − i ) = a i ( x i − b i ) + c N X j =1 x j + d x i , i ∈ V , where a i > , c > , b i and d are constants for i ∈ V . It canbe verified that Assumptions 2 and 3 are satisfied. Throughoutthis simulation, let a i = 1 , c = 0 . , d = 10 for each player. Inthe following simulation, we investigate the effectiveness of the D o S a tt a c k s S t r a t e g i e s o f p l a y e r s x i j ( t ) x *1 x *2 x *3 x *4 x *5 Time [sec] || x ( t ) − x ∗ || / || x ∗ || (a)(b)(c) Fig. 6. Simulated results of the proposed attack-resilient NE seeking algorithmin (9)-(10): (a) DoS attacks on various communication channels; (b) all players’estimated strategies x ij ( t ) , i, j ∈ V ; and (c) relative errors of all players’ actions. proposed resilient distributed NE seeking algorithm in (9)-(10)from the perspective of network under DoS attacks with severalcomparisons with the existing algorithm, controller gain, attackfrequency/duration, network topology and number of players. A. Resilient Algorithm for Exponential Distributed NE Seeking
We consider five players ( N = 5 ) in the game over a stronglyconnected digraph in the absence of attacks and three types ofdisconnected digraphs caused by DoS attacks as shown in Fig. 5.Constants b i for i = 1 , · · · , , are set to , , , , and ,respectively. By certain calculation based on those parameters, theNE is x ∗ = col (2 . , . , . , . , . [8].The initial states are given by x ii (0) = col ( − , − , − , − , − and x ij (0) = col (15 , , , , ∀ i = j , which are not close to x ∗ .The control gain of algorithm in (10) is set as κ = 10 and thevariable to balance the weight is ω = col ( , , , , ) .Next, we perform the proposed resilient NE seeking algorithmin (9)-(10), and simulation results are provided as shown in Fig.6. In particular, Fig. 6(a) shows the occurrence of DoS attacks,where attack frequency and duration conditions in Theorem 1 aresatisfied. Fig. 6(b) illustrates all players’ estimate strategies on theNE x ∗ , while the relative errors of all players’ actions k x − x ∗ k / k x ∗ k are depicted in Fig. 6(c). As observed, all players’ estimatestrategies reach consensus and converge to the NE exponentially. B. Algorithm Comparison
In order to make some comparisons, we perform the algorithmin [11] to further illustrate the proposed algorithm’s effectivenessunder DoS attacks. All simulation environments are set the sameas those in the subsection V-A. It can be observed from Fig. 7 (a)that in the presence of attacks, the design in [11] cannot guaranteethe exact convergence of all players’ estimates to the NE x ∗ . Incontrast, Fig. 7 (b) shows the performance of the proposed attack-resilient algorithm, which can verify the design’s effectiveness. Time [sec] T he s t r a t eg i e s o f a ll p l a y e r s x (t) x *1 x (t) x *2 x (t) x *3 x (t) x *4 x (t) x *5 (a) Algorithm in [11] Time [sec] S t r a t eg i e s o f p l a y e r s x i ( t ) x (t) x *1 x (t) x *2 x (t) x *3 x (t) x *4 x (t) x *5 (b) Algorithm in (9)-(10)Fig. 7. The plot of players’ strategies x i ( t ) produced by the proposed algorithmand the algorithm in [16] in the presence of DoS attacks. C. Performance Analysis of Algorithm
1) Controller gain: we show the influence of control gain κ onthe performance of algorithm, we conduct the proposed algorithmwith the same simulation setting in the subsection V-A, but undertwo different gains κ = 1 and κ = 5 , respectively. Fig. 8 showsthe plots of players’ strategies and relative errors under differentcontrol gains, which implies that the larger the gain κ , the betterthe convergence performance is, which is as analyzed.
2) Number of Player: we increase the number of players to N = 3 , , and analyze its influence on the performance of theproposed algorithm. We set b i = 5 i + 5 for each i = 1 , , · · · , N ,and select a cycle directed graph as the original communicationgraph. Fig. 9 depicts the performance of the proposed algorithmunder the different number of players. The algorithm is scalable tovarious number of players, and the smaller the number of players,the better the performance is as expected.
3) Network Topology: in this part, we investigate the influenceof the different original communication topologies on the perfor-mance of algorithm. As analyzed in the theorem, the larger the λ a , the better convergence performance is. Intuitively, the largerthe nonzero eigenvalue of the Laplacian matrix, the larger the λ a .Fig. 10 shows the logarithmic curve of relative errors. As we see,the performance is better for a graph with more links.
4) Attack Frequency and Duration: we illustrate the influenceof attack frequency and duration on the performance of algorithm.According to the theorem, the frequency and duration of attackshave to be constrained to guarantee the convergence of all players’estimates to the NE. Simulated result is shown in Fig. 11, wherea comparison, when the attack frequency and duration conditionsin (14) and (15) hold and do not not, is provided. As can be seen,the result is consistent with the analysis in Remark 6.
Example 2: (General Non-Quadratic Game)In this example, we investigate a more general non-quadraticgame, where the cost functions for each player i are given by J ( x , x − ) = x x X j =2 x j , J ( x , x − ) = e x x x ,J ( x , x − ) = x x , J ( x , x − ) = ln( e x ) + x + x ,J ( x , x − ) = x − x + x x + x x , i = 1 , · · · , . (42)Next, we perform the proposed algorithm for this non-quadraticgame with the same simulation setting in the subsection V-A. Bythe calculation, the NE is x ∗ = col ( − . , . , , − , . .The simulation result is shown in Fig. 12, and as we can see, under P l a y e r s ’ s t r a t eg i e s x i ( t ) Time [sec] || x ( t ) − x ∗ || / || x ∗ || (a) κ = 1 P l a y e r s ’ s t r a t eg i e s x i ( t ) Time [sec] || x ( t ) − x ∗ || / || x ∗ || (b) κ = 5 Fig. 8. The plot of players’ strategies x i and their relative errors k x − x ∗ k / k x ∗ k produced by the proposed algorithm in (9)-(10) with different controller gains.(a) || x ( t ) − x ∗ || / || x ∗ || Time [sec] N =3 N =5 N =10 (b) Time [sec] l n ( || x ( t ) − x ∗ || / || x ∗ || ) N =3 N =5 N =10 (c)Fig. 9. Performance of the proposed algorithm (9)-(10) with different players:(a) the cycle digraph; (b) the relative error; and (c) its logarithmic curve. l n ( || x ( t ) − x ∗ || / || x ∗ || ) Time [sec] G G G (a) (b)Fig. 10. The logarithmic curve of the relative error produced by the proposedalgorithm (9)-(10) with different network graphs. Time [sec] S t r a t eg i e s o f p l a y e r s x i ( t ) x (t) x *1 x (t) x *2 x (t) x *3 x (t) x *4 x (t) x *5 (a) Small attack requency and duration Time [sec] S t r a t eg i e s o f p l a y e r s x i ( t ) x (t) x *1 x (t) x *2 x (t) x *3 x (t) x *4 x (t) x *5 (b) Large attack requency and durationFig. 11. The plot of players’ strategies x i ( t ) produced by the proposed algorithm:(a) conditions (14)-(15) hold; (b) conditions (14)-(15) are not satisfied. attacks, all players’ estimates can reach consensus and convergeto the NE exponentially. D o S a tt a c k s S t r a t e g i e s o f p l a y e r s x i j ( t ) x *2 x *5 x *3 x *4 x *1 Time [sec] || x ( t ) − x ∗ || / || x ∗ || (c)(b)(a) Fig. 12. Simulated results of the proposed attack-resilient NE seeking algorithmin (9)-(10): (a) DoS attacks on various communication channels; (b) all players’estimated strategies x ij ( t ) , i, j ∈ V ; and (c) relative errors of all players’ actions. VI. C
ONCLUSION
In this paper, an attack-resilient distributed algorithm has beenpresented for exponential NE seeking of non-cooperative games,where all players’ strategies have been updated through a directedcommunication network subject to malicious DoS attacks. Undersuch an adversary network environment, the exponential conver-gence of the proposed distributed algorithm has been establishedthrough the explicit analysis of the attack frequency and duration.Moreover, in the absence of DoS attacks, the corollary has beenprovided, which can cover many existing results as special cases.The effectiveness of the developed approach has been illustratedby the numerical examples. Further work may consider distributedNE seeking problems for aggregative games with constraints.R
EFERENCES[1] S. Li, T. Basar, “Distributed algorithms for the computation of noncooper-ative equilibria,”
Automatica , 23, 523–533, 1987.[2] J. S. Shamma, G. Arslan, “Dynamic fictitious play, dynamic gradientplay, and distributed convergence to Nash equilibria,”
IEEE Trans. Autom.Control , 50(3): 312–327, 2005.[3] M. S. Stankovic, K. H. Johansson, D. M. Stipanovic, “Distributed seeking ofNash equilibria with applications to mobile sensor networks,”
IEEE Trans.Autom. Control , 57(4): 904–919, 2012.[4] P. Frihauf, M. Krstic, T. Basar, “Nash equilibrium seeking in noncooperativegames,”
IEEE Trans. Autom. Control , 57(5): 1192–1207, 2012.[5] S. Grammatico, “Dynamic control of agents playing aggregative games withcoupling constraints,”
IEEE Trans. Autom. Control , 62(9): 4537–4548, 2017.[6] G. Scutari, F. Facchinei, J. Pang, D. Pallomar, “Real and complex monotonecommunication games,”
IEEE Trans. Inf. Theory , 60(7): 400–409, 2014.[7] J. R. Marden, “Learning in large-scale games and cooperative control,”
Ph.D.dissertation, University of California , Los Angeles, CA, USA, 2007.[8] M. Ye, G. Hu, “Distributed Nash equilibrium seeking by a consensus basedapproach,”
IEEE Trans. Autom. Control , 62(9): 4811–4818, 2017.[9] C. Sun, G. Hu, “Distributed Nash equilibrium seeking for a generalizedconvex game with nonsmooth objective functions and certain nonsmoothconstraints,”
International Conference on Information and Automation , pp.77–82, Fujian, China, Aug. 11–14, 2018. [10] K. Lu, G. Jing, L. Wang, “Distributed algorithms for searching generalizedNash equilibrium of noncooperative Games,”
IEEE Trans. Cybern.,
IEEE Trans. Autom. Control , 64: 1077–1092, 2019.[12] S. Liang, P. Yi, Y. Hong, “Distributed Nash equilibrium seeking for aggrega-tive games with coupled constraints,”
Automatica , 85: 179–185, 2017.[13] Z. Deng, X. Nian, “Distributed generalized Nash equilibrium seekingalgorithm design for aggregative games over weight-balanced digraphs,”
IEEE Trans. Neur. Net. Lear. , 30(3): 695–706, 2019.[14] Y. Zhang, X. Wang, H. Ji, “Distributed Nash equilibrium seeking in aggrega-tive game with disturbance rejection,” the 15th International Conference onControl and Automation , Scotland, July 16-19, pp. 1091–1095, 2019.[15] X. Wang, N. Xiao, T. Wongpiromsarn, L. Xie, E. Frazzoli, D. Rus,“Distributed consensus in noncooperative congestion games: an applicationto road pricing,” in IEEE Int. Conf. Control Autom. , pp. 16681673, 2013.[16] F. Salehisadaghiani, L. Pavel, “Distributed Nash equilibrium seeking: agossip-based algorithm,”
Automatica , 72(1): 209–216, 2016.[17] F. Salehisadaghiani, W. Shi, L. Pavel, “Distributed Nash equilibrium seekingunder partial-decision information via the alternating direction method ofmultipliers,”
Automatica , 103: 27–35, 2019.[18] Y. Pang, G. Hu, “Distributed Nash equilibrium seeking with limited costfunction knowledge via a consensus-based gradient-free method,”
IEEETrans. Autom. Control , DOI: 10.1109/TAC.2020.2995666, 2020.[19] Y. Pang, G. Hu, “Randomized gradient-free distributed optimization methodsfor a multi-agent system with unknown cost function,”
IEEE Trans. Autom.Control , 65(1), pp. 333–340, 2020.[20] Y. Mo, J. P. Hespanha, B. Sinopoli, “Resilient detection in the presence ofintegrity attacks,”
IEEE Trans. Signal Process . 62(1): 31–43, 2014.[21] Z. Feng, G. Hu, “Distributed coordinated control for multi-agent systemsunder two types of attacks with an application to power system,” the 19 th IFAC World Congress,
August 24-29, South Africa, pages, 124-130, 2014.[22] Z. Feng, G. Hu, G. Wen, “Distributed consensus tracking for multi-agentsystems under two types of attacks,”
Int. J. Robust. Nonlinear Control , 26(5): 896–918, 2015.[23] Z. Feng, G. Wen, G. Hu, “Distributed secure control for multi-agent systemsunder strategic attacks,”
IEEE Trans. Cybern.,
American Control Conference,
May 24-26, Seattle, WA, USA, 2017.[25] Z. Feng, G. Hu, “Secure cooperative event-triggered control of linear multi-agent systems under DoS attacks,”
IEEE Trans. Contr. Syst. Tech. , 28(3):741–752 2020.[26] W. Xu, G. Hu, D. W. C. Ho, Z. Feng, “Distributed secure cooperativecontrol under Denial-of-Service attacks from multiple adversaries,”
IEEETrans. Cybern.,
Automatica , 79: 42–51, 2017.[28] A. Lu, G. Yang, “Input-to-state stabilizing control for cyber-physical systemswith multiple transmission channels under denial of service,”
IEEE Trans.Autom. Control , 63(6): 1813–1820, 2018.[29] B. Chen, D. Ho, W. Zhang, L. Yu, “Distributed dimensionality reductionfusion estimation for cyber-physical systems under DoS attacks,”
IEEETrans. Syst., Man, Cybern., Syst. , 49(2): 455–468, 2019.[30] D. Zhang, L. Liu, G. Feng, “Consensus of heterogeneous linear multi-agentsystems subject to aperiodic sampled-data and DoS attack,”
IEEE Trans.Cybern. , 49(4): 1501–1511, 2019.[31] L. Su, N. Vaidya, “Multi-agent optimization in the presence of Byzantineadversaries: Fundamental limits,”
American Control Conference , July 6-8,Boston, MA, USA, pages 7183-7188, 2016.[32] S. Sundaram, B. Gharesifard, “Distributed optimization under adversarialnodes,”
IEEE Trans. Autom. Control,
IEEE Trans. Autom. Control,
DOI:10.1109/TAC. 2019.2954363, 2020.[34] Z. Feng, G. Hu, C. G. Cassandras, “Finite-time distributed convex optimiza-tion for continuous-time multi-agent systems with disturbance rejection,”
IEEE Trans. Control Netw. Syst.,