A Neural-Network based Approach for Nash Equilibrium Seeking in Mixed-order Multi-player Games
aa r X i v : . [ m a t h . O C ] S e p A Neural-Network based Approach for Nash Equilibrium Seeking inMixed-order Multi-player Games
Maojiao Ye and Jizhao Yin
Abstract — Noticing that agents with different dynamics maywork together, this paper considers Nash equilibrium compu-tation for a class of games in which first-order integrator-typeplayers and second-order integrator-type players interact ina distributed network. To deal with this situation, we firstlyexploit a centralized method for full information games. In theconsidered scenario, the players can employ its own gradientinformation, though it may rely on all players’ actions. Basedon the proposed centralized algorithm, we further develop adistributed counterpart. Different from the centralized one,the players are assumed to have limited access into the otherplayers’ actions. In addition, noticing that unmodeled dynamicsand disturbances are inevitable for practical engineering sys-tems, the paper further considers games in which the players’dynamics are suffering from unmodeled dynamics and time-varying disturbances. In this situation, an adaptive neuralnetwork is utilized to approximate the unmodeled dynamics anddisturbances, based on which a centralized Nash equilibriumseeking algorithm and a distributed Nash equilibrium seekingalgorithm are established successively. Appropriate Lyapunovfunctions are constructed to investigate the effectiveness of theproposed methods analytically. It is shown that if the consideredmixed-order game is free of unmodeled dynamics and distur-bances, the proposed method would drive the players’ actionsto the Nash equilibrium exponentially. Moreover, if unmodeleddynamics and disturbances are considered, the players’ actionswould converge to arbitrarily small neighborhood of the Nashequilibrium. Lastly, the theoretical results are numericallyverified by simulation examples.
Index Terms — Nash equilibrium seeking; mixed-order dy-namics; games; distributed network.
I. I
NTRODUCTION
Game theory is a powerful tool for analyzing decision-making processes in which multiple rational decision-makersinteract with each other. For example, optimal charging ofplug-in electric vehicles [1], economic dispatch [2], coor-dinative control in mobile sensor networks [3], formationcontrol [4], energy consumption control in smart grids [8][9],to mention just a few, are representatives that fall into thegame theoretic framework. Inspired by the broad applicationsof game theoretic approaches, many researchers devotedthemselves to the development of Nash equilibrium seekingstrategies and quite a few Nash equilibrium seeking strategieshave been reported in the existing literature. For example,games with first-order dynamic players were investigated in[5]-[7] and [27]-[29]. Nash equilibrium seeking protocols for
M. Ye and J. Yin are with the School of Automation, Nanjing Uni-versity of Science and Technology, 210094, P.R. China (Email-address:[email protected], [email protected]).This work is supported by the National Natural Science Foundationof China (NSFC), No. 61803202 and the Natural Science Foundation ofJiangsu Province, No. BK20180455. second-order dynamic players and linear time-invariant dy-namic players were studied in [10][11] and [12], respectively.Nevertheless, only a few works reported results on Nashequilibrium seeking for heterogeneous multi-player games.In [13], distributed Nash equilibrium seeking algorithmswere developed for heterogeneous Euler-Lagrange systems.The work in [14] considered distributed Nash equilibriumcomputation for games with first-order continuous-time play-ers and discrete-time players. However, due to the differentcomputation capabilities of distinct computing units, varioushardware environments and diversities of the agents’ dy-namics, multi-agent systems show remarkable and versatileheterogeneities. Inspired by the above observations, hetero-geneous multi-agent systems have been widely explored.For example, linear heterogeneous multi-agent systems withdistinct constant matrices in the agents’ dynamics were in-vestigated for formation control, output regulation problemsand distributed optimal coordination problems in [16]-[18]and [24], respectively. Nonlinear heterogeneous multi-agentsystems, in which both the state dynamics and dimensionscan be different were addressed in [19]. Heterogeneousmulti-agent systems consisting of first-order continuous-time agents and first-order discrete-time agents were studiedin [20]. Besides, second-order heterogeneous multi-agentsystems, in which the agents’ inertias and control gainswere time-varying, were investigated in [21].
In particular,as velocity-actuated vehicles and acceleration-actuatedvehicles might work together, heterogeneous multi-agentsystems composed of first-order agents and second-orderagents occupy an important position [22].
Consensus of heterogeneous multi-agent systems com-posed of first-order agents and second-order agents withoututilizing velocity measurements was investigated in [23].Average consensus tracking for sensor networks in whichvelocity-actuated sensors and force-actuated sensors existsimultaneously was investigated in [22]. Two stationaryconsensus algorithms were designed for discrete-time hetero-geneous multi-agent systems composed of first-order agentsand second-order agents in [15] with bounded communica-tion delays considered.
However, Nash equilibrium com-putation for mixed-order multi-player games consistingof first-order players and second-order players hasn’tbeen addressed yet though it is a problem of greatimportance.
Inspired by the above observation, this papertries to accommodate Nash equilibrium computation forgames with mixed-order integrator-type dynamics. Moreover,noticing that in many practical situations, e.g., physicalhydraulic systems [32], air hybrid vehicles [33] and marineurface vessels [34], external disturbances and unmodeleddynamics are inevitable due to complex working environmentof engineering actuators and limited knowledge about theexplicit system model, this paper further addresses Nashequilibrium computation for mixed-order games in which theplayers’ dynamics are suffering from unmodeled dynamicsand disturbances. Noticing that radial basis function neuralnetwork (RBFNN) has been shown to be capable of approx-imating unknown continuous functions over a compact set(see, e.g., [31][35]-[38]). This paper takes the benefits ofthe RRBFNN to establish robust Nash equilibrium seekingstrategies for the considered mixed-order multi-player games.Compared with the existing works, the main contributions ofthe paper are summarized as follows.1) Nash equilibrium seeking for mixed-order multi-playergames, in which first-order integrator-type players andsecond-order integrator-type players coexist is investi-gated in this paper. To the best of the authors’ knowl-edge, mixed-order multi-player games have rarely beeninvestigated by the existing works. The exploration ofthis paper would broaden the applicable fields of gametheoretic approaches for distributed games with mixed-order players.2) Games with ideal mixed-order players are investigated,followed by the case in which the players’ dynamicsare suffering from unmodeled dynamics and externaldisturbances. For both situations, a centralized algo-rithm and a distributed algorithm are proposed. Inparticular, the unmodeled dynamics are accommodatedby neural networks. Compared with the RISE-basedmethod in [26], the conditions on the unmodeleddynamics and disturbances are relaxed to some extent.3) The convergence results of the proposed algorithms areanalytically investigated by utilizing Lyapunov stabilityanalysis. It is theoretically proven that the players’actions would be driven to the Nash equilibrium ifthere is no unmodeled dynamics and disturbances.Moreover, with unmodeled dynamics and disturbances,the players’ actions can be driven to arbitrarily smallneighborhood of the Nash equilibrium.The rest of this paper is organized as follows. Notationsand preliminaries are given in Section II. The problemformulation is given in Section III and the main resultsare presented in Sections IV-V. For games with mixed-orderplayers, a centralized algorithm and a distributed algorithmwill be given successively. Moreover, the cases in which thegame is subject to unmodeled dynamics and disturbances willbe considered following the ideal disturbance-free situation.The numerical examples are presented in Section VI and theconclusions are given in Section VII.II. N
OTATIONS AND P RELIMINARIES
Notations:
In this paper, we use R to denote the set ofreal numbers. The notation max { ¯ l i } (min { ¯ l i } ) defines themaximum (minimum) value of ¯ l i for i ∈ { , , · · · , N } . A = [ a ij ] defines a matrix whose entry on the i th row and j th column is a ij . For a symmetric matrix Q ∈ R N × N , λ min ( Q ) and λ max ( Q ) are the minimum and maximumeigenvalues of Q , respectively. Moreover, diag { a ij } for i, j ∈ { , , · · · , N } denotes a diagonal matrix whosediagonal elements are a , a , · · · , a N , a , · · · , a NN and ⊗ is the Kronecker product. Graph theory:
For a graph defined as G = ( ℵ , E ) , where ℵ = { , , · · · , N } is the set of vertices and E ⊆ ℵ × ℵ is the set of edges. The network is undirected if for every ( i, j ) ∈ E , we have ( j, i ) ∈ E . In addition, the undirectedgraph is connected if there is a path between any pair ofdistinct vertices. The adjacency matrix associated with graph G is defined as A = [ a ij ] , where a ij = 1 if node ( j, i ) ∈ E ,else, a ij = 0 ( a ii = 0 ). Furthermore, the Laplacian matrixassociated with G is defined as L = D − A where D is adiagonal matrix whose i th diagonal entry is equal to the outdegree of node i , represented by P Nj =1 a ij [5]. Radial basis function neural networks:
A continuousfunction l ( z ) : R N → R N can be approximated on a compactset z ∈ Ω z ⊂ R N by l NN ( z ) = W T S ( z ) , (1)where W ∈ R q × N is an adjustable weight matrix, q is thenumber of neuron, and S ( z ) = [ s ( z ) , s ( z ) , · · · , s q ( z )] T isthe activation function given by s i ( z ) = exp (cid:20) − ( z − µ i ) T ( z − µ i ) ρ i (cid:21) , i = 1 , , · · · , q, (2)where µ i = [ µ i , µ i , · · · , µ iN ] T is the center of the recep-tive field, and ρ i denotes the width of the Gaussian function[31]. Lemma 1: [31] For any arbitrary small positive constant ¯ ε and z ∈ Ω z , there exists a weight matrix W ∗ ∈ R q × N suchthat l ( z ) = W ∗ T S ( z ) + ε, (3)where ε is the approximation error that satisfies | ε | ≤ ¯ ε. Lemma 2: [31] Let V ( t ) ≥ be a continuous functiondefined for all ∀ t ≥ . Suppose that there are positiveconstants a, b such that ˙ V ( t ) ≤ − aV ( t ) + b, (4)then V ( t ) ≤ V (0) e − at + ba (1 − e − at ) . (5) Lemma 3: [30] For any ǫ > and η ∈ R , ≤ | η |− η tanh ( ηǫ ) ≤ K ǫ, (6)where K is a constant that satisfies K = e −K +1 .III. P ROBLEM FORMULATION
Consider a game with N players whose dynamics aregoverned by ˙ x f = u f + Υ( g f ( x ) + d f ( t )) , f ∈ ν f ¨ x s = u s + Υ( g s ( x ) + d s ( t )) , s ∈ ν s , (7)where x f , x s ∈ R , u f , u s ∈ R are the actions and the controlinputs of players f and s, respectively. Furthermore, ν f = , , · · · , n } and ν s = { n + 1 , n + 2 , · · · , N } ( N > n ). It’sworth mentioning that following the above definitions, we getthat ℵ = ν f S ν s . Note that g f ( x ) , g s ( x ) and d f ( t ) , d s ( t ) are the unmodeled terms and external disturbances whoseexplicit expressions are unknown. In addition, Υ is a variablethat is equal to or and these two cases will be investigatedsuccessively in the rest. This paper aims to design controllaws to seek the Nash equilibrium x ∗ = ( x ∗ i , x ∗− i ) on which F i ( x ∗ i , x ∗− i ) ≤ F i ( x i , x ∗− i ) , (8)for x i ∈ R , i ∈ ℵ , and x − i =[ x , x , · · · , x i − , x i +1 , · · · , x N ] T . In addition, F i ( x ) , where x = [ x , x , · · · , x N ] T , is the cost function of player i . For notational convenience, define ∇ i F i ( x ) = ∂F i ( x ) ∂x i , and ∇ ij F i ( x ) = ∂ F i ( x ) ∂x i ∂x j . The following assumptions will beutilized in the upcoming analysis.
Assumption 1:
For each i ∈ ℵ , F i ( x ) is twice-continuously differentiable and ∇ i F i ( x ) is globally Lipschitzfor x ∈ R N . Remark 1:
By Assumption 1, there exists a positive con-stant ¯ l i such that k∇ i F i ( x ) − ∇ i F i ( z ) k ≤ ¯ l i k x − z k for x , z ∈ R N , i ∈ ℵ . Moreover, (cid:13)(cid:13) ¯ P ( x ) − ¯ P ( z ) (cid:13)(cid:13) ≤ l k x − z k , (9)where ¯ P ( x ) = [ ∇ F ( x ) , ∇ F ( x ) , · · · , ∇ N F N ( x )] T , and l = √ N max { ¯ l i } . Assumption 2:
The players can communicate with eachother by utilizing an undirected and connected graph G . Remark 2:
By Assumption 2, − ( L ⊗ I N × N + A ) , where A = diag { a ij } and I N × N is an N × N dimensional identitymatrix, is Hurwitz and there exist symmetric positive definitematrices P and Q of compatible dimensions such that P ( L⊗ I N × N + A ) + ( L ⊗ I N × N + A ) P = Q [5]. Assumption 3:
For all x , z ∈ R N , ( x − z ) T ( ¯ P ( x ) − ¯ P ( z )) ≥ m || x − z || , (10)where m is a positive constant. Assumption 4:
The elements in H ( x ) defined as H ( x ) =[ ∇ i,j F n +1 ( x )] , where i ∈ ν s , j ∈ ℵ are bounded for x ∈ R N . Remark 3:
By Assumption 4, there exists a positive con-stant h such that sup x ∈ R N k H ( x ) k≤ h. IV.
MIXED - ORDER MULTI - PLAYER GAMES
In this section, we suppose that
Υ = 0 in (7). If this isthe case, the dynamics of the players in the considered gameare ˙ x f = u f , f ∈ ν f ¨ x s = u s , s ∈ ν s . (11)In the subsequent subsections, a centralized seeking algo-rithm and a distributed seeking algorithm will be establishedone by one. A. A centralized algorithm for mixed-order multi-playergames
Suppose that each player can access its own gradient value,then, the control input can be designed as u f = − k ∇ f F f ( x ) , f ∈ ν f u s = − k v s − k k ∇ s F s ( x ) , s ∈ ν s , (12)where k , k , are positive control gains.Recalling that the players’ dynamics are governed by (11),we get that ˙ x f = − k ∇ f F f ( x ) , f ∈ ν f , ˙ x s = v s , ˙ v s = − k v s − k k ∇ s F s ( x ) , s ∈ ν s , (13)where v s is the velocity of player s, s ∈ ν s . Writing (13) in its concatenated-vector form gives ˙ x f = − k ¯ P f ( x )˙ x s = v s ˙ v s = − k v s − k k ¯ P s ( x ) , (14)where ¯ P f ( x ) = [ ∇ F ( x ) , ∇ F ( x ) , · · · , ∇ n F n ( x )] T , ¯ P s ( x ) = [ ∇ n +1 F n +1 ( x ) , ∇ n +2 F n +2 ( x ) , · · · , ∇ N F N ( x )] T , x f = [ x , x · · · , x n ] T , x s = [ x n +1 , x n +2 · · · , x N ] T , and v s = [ v n +1 , v n +2 · · · , v N ] T . Note that by the above defini-tions, [ ¯ P f ( x ) T , ¯ P s ( x ) T ] T = ¯ P ( x ) , [ x Tf , x Ts ] T = x .The following theorem shows the stability of the systemin (13). Theorem 1:
Suppose that Assumptions 1,3-4 are satisfiedand k > ( k hl + 1) k m + k h. (15)Then, the Nash equilibrium is globally exponentially stableunder (13). Proof:
Let v s = ¯ v s − k ¯ P s ( x ) . (16)Then, by (14) and (16), we get that ˙ v s = − k ( v s + k ¯ P s ( x )) = − k ¯ v s . (17)Moreover, ˙ x s =¯ v s − k ¯ P s ( x ) ˙¯v s = ˙ v s + k H ( x ) ˙x = − k ¯ v s + k H ( x ) ˙x . (18)Define the Lyapunov candidate function as V = 12 ( x − x ∗ ) T ( x − x ∗ ) + 12 ¯v Ts ¯v s . (19)hen, ˙ V = ( x − x ∗ ) T ˙ x + ¯v Ts ˙¯ v s = (cid:0) x f − x ∗ f (cid:1) T ˙ x f + ( x s − x ∗ s ) T ˙ x s + ¯v Ts ( − k ¯ v s + k H ( x ) ˙x )= − (cid:0) x f − x ∗ f (cid:1) T k ¯ P f ( x )+ ( x s − x ∗ s ) T (¯ v s − k ¯ P s ( x )) − k k ¯v s k + k ¯v Ts H ( x ) ˙ x = − k ( x − x ∗ ) T ¯ P ( x ) + ( x s − x s ∗ ) T ¯ v s − k k ¯v s k + k ¯v Ts H ( x ) ¯ φ = − k ( x − x ∗ ) T ¯ P ( x ) + ( x s − x s ∗ ) T ¯ v s − k k ¯v s k − k ¯v Ts H ( x ) ¯ P ( x ) + k ¯v Ts H ( x ) ¯ φ , (20)where ¯ φ = (cid:2) − k [ ¯ P f ( x )] T v Ts (cid:3) T = (cid:2) − k [ ¯ P f ( x )] T (¯ v s − k ¯ P s ( x )) T (cid:3) T and ¯ φ = (cid:2) Tn ¯ v Ts (cid:3) T .By Assumption 1, k ¯ P ( x ) k = (cid:13)(cid:13) ¯ P ( x ) − ¯ P ( x ∗ ) (cid:13)(cid:13) ≤ l k x − x ∗ k . Hence, k ¯v Ts H ( x ) ¯ P ( x ) ≤ k l h || x − x ∗ |||| ¯v s || byAssumptions 1 and 4. Therefore, ˙ V ≤ − k m k x − x ∗ k − ( k − k h ) k ¯ v s k + ( k l h + 1) k x − x ∗ kk ¯ v s k , (21)by further utilizing Assumption 3.Define A = " k m − k hl +12 − k hl +12 k − k h and let k > ( k hl +1) k m + k h . Then, A is symmetric positive definite and ˙ V ≤ − λ min ( A ) k E k , (22)where λ min ( A ) > , and E = [( x − x ∗ ) T , ¯v Ts ] T ,Recalling the definition of the Lyapunov candidate func-tion, it is clear that V = 12 || E || . (23)Hence, ˙ V ≤ − λ min ( A ) V, (24)which further indicates that k E ( t ) k≤ e − λ min ( A ) t k E (0) k . (25)Define E r ( t ) = [( x − x ∗ ) T , v Ts ] T . Then, k E r ( t ) k≤ k E ( t ) k + k k ¯ P s ( x ) k≤ (1 + l k ) k E ( t ) k≤ (1 + l k ) e − λ min ( A ) t k E (0) k≤ (1 + l k ) e − λ min ( A ) t ( k E r (0) k + k k ¯ P s (0) k ) ≤ (1 + l k ) e − λ min ( A ) t k E r (0) k , thus arrivingat the conclusion. (cid:3) In Section IV-A, the players are supposed to have accessto their own gradient values. However, the players’ gradientvalues rely on all the players’ actions, which can hardlybe obtained in many practical situations. Hence, in thefollowing, a distributed seeking algorithm will be established.
B. A distributed algorithm for mixed-order multi-playergames
To establish a distributed seeking algorithm, the controlinputs are designed as u f = − k ∇ f F f ( y f ) , f ∈ ν f u s = − k v s − k k ∇ s F s ( y s ) , s ∈ ν s , (26)where k , k are positive parameters, y i =[ y i , y i , · · · y iN ] T stands for player i ’s local estimateson x and y ij is player i ’s estimate on x j . In addition, ∇ i F i ( y i ) = ∇ i F i ( x ) | x = y i . Moreover, motivated by [5], y ij is generated by ˙ y ij = − k N X k =1 a ik ( y ij − y kj ) + a ij ( y ij − x j ) ! , (27)for all i, j ∈ ℵ , where k is a positive control gain.Recalling that the players’ dynamics are governed by (11),it can be obtained that ˙ x f = − k ∇ f F f ( y f ) , f ∈ ν f ˙ x s = v s ˙ v s = − k v s − k k ∇ s F s ( y s ) , s ∈ ν s ˙ y ij = − k N X k =1 a ik ( y ij − y kj ) + a ij ( y ij − x j ) ! , (28)where in the last equation, i, j ∈ ℵ .Writing (28) in its concatenated-vector form yields ˙ x f = − k ¯ P f (¯ y f )˙ x s = v s ˙ v s = − k v s − k k ¯ P s (¯ y s ) ˙y = − k ( L ⊗ I N × N + A )( y − N ⊗ x ) , (29)where y = [ y T , y T , · · · , y TN ] T , ¯ y f = [ y T , y T , · · · , y Tn ] , ¯ y s = [ y Tn +1 , y Tn +2 , · · · , y TN ] , ¯ P (¯ y f ) =[ ∇ F ( y ) , ∇ F ( y ) , · · · , ∇ n F n ( y n )] T , and ¯ P (¯ y s ) =[ ∇ n +1 F n +1 ( y n +1 ) , ∇ n +2 F n +2 ( y n +2 ) , · · · , ∇ N F N ( y N )] T .Moreover, for notational convenience,let ¯ P ( y ) = [ ¯ P (¯ y f ) T , ¯ P (¯ y s ) T ] T and H (¯ y s ) ∈ R ( N − n ) × N whose j th row is [ TN ( j − , ∇ n + j, F n + j ( y n + j ) , ∇ n + j, F n + j ( y n + j ) , · · · , ∇ n + j,N F n + j ( y n + j ) , TN ( N − j ) ] . Then, the following resultcan be obtained.
Theorem 2:
Suppose that Assumptions 1-4 are satisfiedand k > m + (max i ∈ℵ { ¯ l i } + 2 √ N kPk l ) σ m (30a) k > k kPk l + k σ + √ N kPk + σ b λ min ( Q ) (30b) k >
12 + √ N kPk + k k b σ , (30c)in which, b = sup ¯ y s ∈ R N ( N − n ) k H (¯ y s ) kkL ⊗ I N × N + Ak and σ and σ are positive constants that can be arbitrarilychosen. Then, the Nash equilibrium is globally exponentiallystable under (28). roof: Let v s = ¯ v s − k ¯ P s (¯ y s ) . (31)Then, by (29), we have ˙ v s = − k ¯ v s . (32)Hence, ˙ x s =¯ v s − k ¯ P s (¯ y s ) ˙¯v s = ˙ v s + k H (¯ y s ) ˙¯ y s = − k ¯ v s + k H (¯ y s ) ˙¯ y s . (33)Define the Lyapunov candidate function as V = 12 ( x − x ∗ ) T ( x − x ∗ ) + 12 ¯v Ts ¯v s + ( y − N ⊗ x ) T P ( y − N ⊗ x ) . (34)Then, it can be easily concluded that ¯ d k E k ≤ V ≤ ¯ d k E k , where ¯ d = min { , λ min ( P ) } , ¯ d = max { , λ max ( P ) } and E = [( x − x ∗ ) T , ¯ v Ts , ( y − N ⊗ x ) T ] T . Moreover, ˙ V = ( x − x ∗ ) T ˙ x + ¯v Ts ˙¯ v s + ( ˙y − N ⊗ ˙x ) T P ( y − N ⊗ x )+ ( y − N ⊗ x ) T P ( ˙y − N ⊗ ˙x ) , (35)in which ( x − x ∗ ) T ˙ x = − (cid:0) x f − x ∗ f (cid:1) T k ¯ P f (¯ y f )+ ( x s − x ∗ s ) T (¯ v s − k ¯ P s (¯ y s ))= − k ( x − x ∗ ) T ¯ P ( y ) + ( x s − x s ∗ ) T ¯ v s = − k ( x − x ∗ ) T ¯ P ( x )+ k ( x − x ∗ ) T ( ¯ P ( x ) − ¯ P ( y ))+ ( x s − x s ∗ ) T ¯ v s . (36)By Assumption 3, − ( x − x ∗ ) T ¯ P ( x ) ≤ − m k x − x ∗ k .Moreover, by Assumption 1, we get that (cid:13)(cid:13) ¯ P ( x ) − ¯ P ( y ) (cid:13)(cid:13) ≤ max i ∈ℵ { ¯ l i }k y − N ⊗ x k . Hence, ( x − x ∗ ) T ˙ x ≤ − k m k x − x ∗ k + k x − x ∗ kk ¯ v s k + k max i ∈ℵ { ¯ l i }k x − x ∗ kk y − N ⊗ x k . (37)Moreover, ¯v Ts ˙¯ v s = ¯v Ts ( − k ¯v s + k H (¯ y s ) ˙¯ y s ) . (38)Recalling the definition of ˙¯ y s , we get that ¯v Ts ˙¯ v s ≤ − k k ¯ v s k + k k b k ¯ v s kk y − N ⊗ x k , (39)where b = sup ¯ y s ∈ R N ( N − n ) k H (¯ y s ) kkL ⊗ I N × N + Ak . Furthermore, ( ˙y − N ⊗ ˙x ) T P ( y − N ⊗ x )+ ( y − N ⊗ x ) T P ( ˙y − N ⊗ ˙x )=( − k ( L ⊗ I N × N + A )( y − N ⊗ x ) − N ⊗ ¯ φ ) T P ( y − N ⊗ x )+ ( y − N ⊗ x ) T P ( − k ( L ⊗ I N × N + A )( y − N ⊗ x ) − N ⊗ ¯ φ )= − k ( y − N ⊗ x ) T ( L ⊗ I N × N + A ) P ( y − N ⊗ x ) − k ( y − N ⊗ x ) T P ( L ⊗ I N × N + A )( y − N ⊗ x ) − y − N ⊗ x ) T P N ⊗ ¯ φ in which ¯ φ = (cid:2) − k [ ¯ P f (¯ y f )] T v Ts (cid:3) T = h − k [ ¯ P f (¯ y f )] T ¯ v Ts − k ¯ P s (¯ y s ) T i T .By Assumption 1, it can be obtained that y − N ⊗ x ) T P N ⊗ ¯ φ ≤ k ||P|| l || y − N ⊗ x || + 2 k √ N ||P|| l || y − N ⊗ x |||| x − x ∗ || + 2 √ N ||P|||| y − N ⊗ x |||| ¯ v s || . (40)Therefore, ( ˙y − N ⊗ ˙x ) T P ( y − N ⊗ x )+ ( y − N ⊗ x ) T P ( ˙y − N ⊗ ˙x ) ≤ − k λ min ( Q ) k y − N ⊗ x k + 2 k kPk l k y − N ⊗ x k + 2 k √ N kPk l k y − N ⊗ x kk x − x ∗ k + 2 √ N kPkk y − N ⊗ x kk ¯ v s k . (41)Hence, ˙ V ≤ − k m k x − x ∗ k − k k v s k + k v s kk x − x ∗ k− ( k λ min ( Q ) − k kPk l ) k y − N ⊗ x k + k (max i ∈ℵ { ¯ l i } + 2 √ N kPk l ) k y − N ⊗ x kk x − x ∗ k + (2 √ N kPk + k k b ) k v s kk y − N ⊗ x k≤ − Ψ k x − x ∗ k − ( k − − √ N kPk − k k b σ ) k v s k − Ψ k y − N ⊗ x k , (42)where σ and σ are positive constants that can be arbitrarilychosen and Ψ = k m − − (max i ∈ℵ { ¯ l i } +2 √ N kPk l ) σ , Ψ = k λ min ( Q ) − k kPk l − k σ − √ N kPk − σ b .Therefore, ˙ V ≤ − K k E k , (43)where K = min { Ψ , k − − √ N kPk − k k b σ , Ψ } > bychoosing k according to 30a, followed by choosing k and k according to and , successively for fixed σ , σ .Hence, k E ( t ) k≤ s ¯ d ¯ d e − K d t k E (0) k , (44)y using the Comparison Lemma [25].Furthermore, define E r ( t ) = [( x − x ∗ ) T , v Ts , ( y − N ⊗ x ) T ] T . Then, k E r ( t ) k≤ k E ( t ) k + k k ¯ P s (¯ y s ) k≤ (1 + k √ N − n max j ∈ ν s { ¯ l j } ) k E ( t ) k≤ (1 + k √ N − n max j ∈ ν s { ¯ l j } ) s ¯ d ¯ d e − K d t k E (0) k≤ (1 + k √ N − n max j ∈ ν s { ¯ l j } ) s ¯ d ¯ d e − K d t × ( k E r (0) k + k k ¯ P s (0) k ) ≤ (1 + k √ N − n max j ∈ ν s { ¯ l j } ) s ¯ d ¯ d e − K d t k E r (0) k , (45)thus arriving at the conclusion. (cid:3) In this section, we consider that the mixed-order gameis free of unmodeled dynamics and external disturbances.However, due to limited knowledge about explicit systemmodel and complicated working environments of actuatorsand sensors, unmodeled dynamics and disturbances are in-evitable in practice. Hence, in the following section, weconsider mixed-order games with unmodeled dynamics anddisturbances.V. M
IXED - ORDER MULTI - PLAYER GAMES WITHUNMODELED DYNAMICS AND THE EXTERNALDISTURBANCE
In this section, we consider that the players’ dynamics aregiven by ˙ x f = u f + g f ( x ) + d f ( t ) , f ∈ ν f ¨ x s = u s + g s ( x ) + d s ( t ) , s ∈ ν s . (46)In the following, a centralized seeking method and a dis-tributed seeking method will be presented, successively. A. A centralized algorithm for mixed-order multi-playergames
In this section, we consider that the players’ gradientvalues are accessible. Moreover, a RBFNN is adopted todeal with the unmodeled dynamics and external disturbancesbased on the following condition.
Assumption 5:
For each i ∈ ℵ , g i ( x ) is globally Lipschitzand d i ( t ) is bounded. Remark 4:
Note that in [26], it is required that the un-modeled dynamics g i ( x ) is sufficiently smooth with itsfirst two partial derivatives being bounded given that x isbounded. Similarly, the disturbance d i ( t ) was supposed to besufficiently smooth with its first two time derivatives beingbounded in [26]. From Assumption 5, we see that theseconditions are relaxed to some extent in this paper.Based on the RBFNN, the control inputs are designed as u f = − k ( x f − z f ) − ˆ W Tf S f (¯ x ) − φ f ˙ z f = − k ∇ f F f (¯ x ) , f ∈ ν f u s = − k v s − k k ∇ s F s (¯ x ) − ˆ W Ts S s (¯ x ) − φ s , s ∈ ν s (47) where k , k , k are positive control gains, z f is anauxiliary variable generated by player f and ¯ x =[ z , z , · · · , z n , x n +1 , x n +2 , · · · , x N ] T . Moreover, ˆ W i ∈ R q × , in which q is the number of neurons, defines theweight matrix of the RBFNN. Motivated by [31], we updatethe weight matrices ˆ W f and ˆ W s by ˙ˆ W f = βS f (¯ x )( x f − z f ) , if Tr ( ˆ W Tf ˆ W f ) < W max or Tr ( ˆ W Tf ˆ W f ) = W max and ( x f − z f ) ˆ W Tf S f (¯ x ) < βS f (¯ x )( x f − z f ) − β ( x f − z f ) ˆ W Tf S f (¯ x ) Tr ( ˆ W Tf ˆ W f ) ˆ W f , if Tr ( ˆ W Tf ˆ W f ) = W max and ( x f − z f ) ˆ W Tf S f (¯ x ) ≥ and ˙ˆ W s = βS s (¯ x )¯ v s , if Tr ( ˆ W Ts ˆ W s ) < W max or Tr ( ˆ W Ts ˆ W s ) = W max and ¯ v s ˆ W Ts S s (¯ x ) < βS s (¯ x )¯ v s − β ¯ v s ˆ W Ts S s (¯ x ) Tr ( ˆ W Ts ˆ W s ) ˆ W s , if Tr ( ˆ W Ts ˆ W s ) = W max and ¯ v s ˆ W Ts S s (¯ x ) ≥ , (48)where β , W max are positive constants, ¯ v s = v s + k ∇ s F s (¯ x ) and Tr ( ˆ W Ti (0) ˆ W i (0)) ≤ W max . Remark 5:
Note that if Tr ( ˆ W Ti (0) ˆ W i (0)) ≤ W max , then,Tr ( ˆ W Ti ( t ) ˆ W i ( t )) ≤ W max and k ˜ W i k F = k ˆ W i − W ∗ i k F ≤ √ W max [31].Furthermore, φ f = δ tanh ( K δ ( x f − z f ) ǫ ) , φ s = δ tanh ( K δ ¯ v s ǫ ) , (49)in which ǫ > is a constant, and δ is a constant that satisfies | δ | ≥ k ε k + k d ( t ) k , where ε = [ ε , ε , · · · , ε N ] T and d ( t ) =[ d ( t ) , d ( t ) , · · · , d N ( t )] T .Recalling the players’ dynamics in (46), the centralizedNash equilibrium seeking strategy is given as ˙ x f = − k ( x f − z f ) − ˆ W Tf S f (¯ x ) − φ f + g f ( x ) + d f ( t )˙ z f = − k ∇ f F f (¯ x ) , f ∈ ν f ˙ x s = v s ˙ v s = − k v s − k k ∇ s F s (¯ x ) − ˆ W Ts S s (¯ x ) − φ s + g s ( x ) + d s ( t ) , s ∈ ν s . (50)Writing (50) in its concatenated-vector form gives ˙ x f = − k ( x f − z f ) − ˆ W Tf S f (¯ x ) − φ f + g f ( x ) + d f ( t )˙ z f = − k ¯ P f (¯ x )˙ x s = v s ˙ v s = − k v s − k k ¯ P s (¯ x ) − ˆ W Ts S s (¯ x ) − φ s + g s ( x ) + d s ( t ) , (51)where z f = [ z , z , · · · , z n ] T , ˆ W Tf S f ( x ) =[( ˆ W T S ( x )) T , · · · , ( ˆ W Tn S n ( x )) T ] T , ˆ W Ts S s ( x ) =[( ˆ W Tn +1 S n +1 ( x )) T , · · · , ( ˆ W TN S N ( x )) T ] T , φ f =[ φ T , · · · , φ Tn ] T , φ s = [ φ Tn +1 , · · · , φ TN ] T , g f ( x ) =[ g T ( x ) , · · · , g Tn ( x )] T , g s ( x ) = [ g Tn +1 ( x ) , · · · , g TN ( x )] T , f ( t ) = [ d T ( t ) , · · · , d Tn ( t )] T , and d s ( t ) =[ d Tn +1 ( t ) , · · · , d TN ( t )] T .The following lemma is given to support the stabilityanalysis. Lemma 4:
Suppose that Assumptions 1,3-5 are satisfiedand k > ( k hl + 1) k m + k h, k > Φ λ min ( A ) + √ n max i ∈ ν f { η i } , (52)where A = " k m − k l h +12 − k l h +12 k − k h , Φ = k l + √ n max i ∈ ν f { η i } + √ N − n max i ∈ ν s { η i } and η i is theLipschitz constant of g i ( x ) . Then, v s ( t ) , x ( t ) and z f ( t ) generated by (50) are bounded given that their initial valuesare bounded. Proof:
Let v s = ¯ v s − k ¯ P s (¯ x ) . (53)Then, ˙ x s =¯ v s − k ¯ P s (¯ x ) ˙¯v s = − k ¯ v s + k H (¯ x ) ˙¯x − ˆ W Ts S s (¯ x ) − φ s + g s ( x ) + d s ( t ) . (54)Define the Lyapunov candidate function as V = 12 (¯ x − x ∗ ) T (¯ x − x ∗ ) + 12 ¯ v Ts ¯ v s + 12 ( x f − z f ) T ( x f − z f ) . (55)Then, (¯ x − x ∗ ) T ˙¯ x = (¯ x − x ∗ )[ ˙ z Tf , ˙ x Ts ] T = − k (¯ x − x ∗ ) T ¯ P (¯ x ) + (¯ x − x ∗ ) T [ Tn , ¯ v Ts ] T ≤ − k m k ¯ x − x ∗ k + k ¯ x − x ∗ kk ¯ v s k . (56)Moreover, ¯ v Ts ˙¯ v s = ¯ v Ts ( − k ¯ v s + k H (¯ x ) ˙¯x )+ ¯ v Ts ( − ˆ W Ts S s (¯ x ) + d s ( t ) − φ s + g s ( x ))= − k k ¯ v s k − k ¯ v Ts H (¯ x ) ¯ P (¯ x )+ k ¯ v Ts H (¯ x )[ Tn , ¯ v Ts ] T + ¯ v Ts ( g s ( x ) − g s ( x ∗ ))+ ¯ v Ts ( − ˆ W Ts S s (¯ x ) + d s ( t ) − φ s + g s ( x ∗ )) ≤ − ( k − k h ) k ¯ v s k + k hl k ¯ x − x ∗ kk ¯ v s k + √ N − n max i ∈ ν s { η i }|| ¯ v s |||| x f − z f || + a s k ¯ v s k + √ N − n max i ∈ ν s { η i }|| ¯ v s |||| ¯ x − x ∗ || , (57)in which a s = √ N − n ( √ q √ W max + δ + d + g ) , d, g arepositive constants that satisfy | d i ( t ) | < d, | g i ( x ∗ ) | < g and η i is the Lipschitz constant of g i ( x ) . Moreover, the lastinequality is obtained by utilizing k ¯ P (¯ x ) k = k ¯ P (¯ x ) − ¯ P ( x ∗ ) k≤ l k ¯ x − x ∗ k based on Assumption 1. Furthermore, ( x f − z f ) T ( ˙ x f − ˙ z f )=( x f − z f ) T ( − k ( x f − z f ) + k ¯ P f (¯ x ))+ ( x f − z f ) T ( − ˆ W Tf S f (¯ x ) − φ f + g f ( x ∗ ) + d f ( t ))+ ( x f − z f ) T ( g f ( x ) − g f ( x ∗ )) ≤ − ( k − √ n max i ∈ ν f { η i } ) k x f − z f k + a f k x f − z f k + ( k l + √ n max i ∈ ν f { η i } ) k x f − z f kk ¯ x − x ∗ k (58)in which a f = √ n ( √ q √ W max + δ + d + g ) . Hence, ˙ V ≤ − k m k ¯ x − x ∗ k − ( k − k h ) k ¯ v s k − ( k − √ n max i ∈ ν f { η i } ) k x f − z f k + a f k x f − z f k + ( k hl + 1) k ¯ x − x ∗ kk ¯ v s k + ( k l + √ n max i ∈ ν f { η i } ) k x f − z f kk ¯ x − x ∗ k + a s k ¯ v s k + √ N − n max i ∈ ν s { η i }|| ¯ v s |||| x f − z f || + √ N − n max i ∈ ν s { η i }|| ¯ v s |||| ¯ x − x ∗ || . (59)Define A = (cid:20) k m − Φ − Φ k − k h (cid:21) , where Φ = k l h + 1 + √ N − n max i ∈ ν s { η i } , and let k > ( k hl +1+ √ N − n max i ∈ νs { η i } ) k m + k h . Then, A is symmetricpositive definite and ˙ V ≤ − λ min ( A ) k E k − ( k − √ n max i ∈ ν f { η i } ) k x f − z f k + ( k l + √ n max i ∈ ν f { η i } ) k x f − z f kk ¯ x − x ∗ k + a f k x f − z f k + a s k ¯ v s k , + √ N − n max i ∈ ν s { η i }|| ¯ v s |||| x f − z f || (60)in which λ min ( A ) > , E = [(¯ x − x ∗ ) T , ¯v Ts ] T . Define A = (cid:20) λ min ( A ) − Φ − Φ k − √ n max i ∈ ν f { η i } (cid:21) ,where Φ = k l + √ n max i ∈ ν f { η i } + √ N − n max i ∈ ν s { η i } , and let k > Φ λ min ( A ) + √ n max i ∈ ν f { η i } . Then, A is symmetric positive matrix.Hence ˙ V ≤ − λ min ( A ) k E k + ( a f + a s ) k E k = − ( λ min ( A ) − θ ) k E k + ( a f + a s ) k E k − θ k E k , (61)in which λ min ( A ) > , E = [(¯ x − x ∗ ) T , ¯v Ts , ( x f − z f ) T ] T and < θ < λ min ( A ) .Hence, ˙ V ≤ − ( λ min ( A ) − θ ) k E k , (62)for ∀k E k ≥ a f + a s θ .Therefore, according to Theorem 4.18 in [25], it canbe obtained that k E k is bounded given that the states areinitialized to be bounded. Recalling the definition of ¯ v s , theconclusion can be obtained.y Lemma 4, the trajectories generated by the proposedmethod belong to a compact set given that they are initializedto be bounded. Hence, by Lemma 1, it is clear that g i ( x ) can be approximated as g i ( x ) = W ∗ i T S i ( x ) + ε i , where W ∗ i ∈ R q × is the optimal weight matrix and ε i ∈ R is the approximation error. Therefore, the centralized Nashequilibrium seeking strategy can be written as ˙ x f = − k ( x f − z f ) − ˜ W Tf S f (¯ x )+ d f ( t ) + ε f − φ f + g f ( x ) − g f (¯ x )˙ z f = − k ∇ f F f (¯ x ) , f ∈ ν f ˙ x s = v s ˙ v s = − k v s − k k ∇ s F s (¯ x ) − ˜ W Ts S s (¯ x )+ d s ( t ) + ε s − φ s + g s ( x ) − g s (¯ x ) , s ∈ ν s . (63)Moreover, the concatenated-vector form of (63) is ˙ x f = − k ( x f − z f ) − ˜ W Tf S f (¯ x )+ ( d f ( t ) + ε f − φ f ) + g f ( x ) − g f (¯ x )˙ z f = − k ¯ P f (¯ x )˙ x s = v s ˙ v s = − k v s − k k ¯ P s (¯ x ) − ˜ W Ts S s (¯ x )+ ( d s ( t ) + ε s − φ s ) + g s ( x ) − g s (¯ x ) . (64)The following theorem establishes the stability of (63). Theorem 3:
Suppose that Assumptions 1,3-5 are satisfied.Then, for any pair of positive constants Λ and Ξ , there existsa positive constant k ∗ (Λ , Ξ) such that for each fixed k > k ∗ , there exist positive constants k ∗ , k ∗ , β ∗ such that for each k > k ∗ , k > k ∗ , β > β ∗ , there exists a positive constant T such that k x ( t ) − x ∗ k + k v s ( t ) k ≤ Ξ , ∀ t > T, (65)given that k [(¯ x (0) − x ∗ ) T , v s (0) T , ( x f (0) − z f (0)) T ] T k + P Ni =1 T r ( ˜ W i (0) T ˜ W i (0)) ≤ Λ and Tr ( W ∗ i T W ∗ i ) ≤ W max . Proof:
Define the Lyapunov candidate function as V = V + V + V + V , (66)in which V = 12 (¯ x − x ∗ ) T (¯ x − x ∗ ) , V = 12 ¯v Ts ¯v s V = 12 ( x f − z f ) T ( x f − z f ) , V = 12 β N X i =1 T r ( ˜ W Ti ˜ W i ) . (67)Then, by (56), ˙ V ≤ − k m k ¯ x − x ∗ k + k ¯ x − x ∗ kk ¯ v s k . (68) and ˙ V =¯ v Ts ˙¯ v s = ¯ v Ts ( − k ¯ v s + k H (¯ x ) ˙¯x + g s ( x ) − g s (¯ x )) − ¯ v Ts ˜ W Ts S s (¯ x ) + ¯ v Ts ( d s ( t ) − φ s + ε s )= − k k ¯ v s k − k ¯ v Ts H (¯ x ) ¯ P (¯ x )+ k ¯ v Ts H (¯ x )[ Tn , ¯ v Ts ] T + ¯ v Ts ( g s ( x ) − g s (¯ x )) − ¯ v Ts ˜ W Ts S s (¯ x ) + ¯ v Ts ( d s ( t ) − φ s + ε s ) ≤ − ( k − k h ) k ¯ v s k + ( N − n ) ǫ + k hl k ¯ x − x ∗ kk ¯ v s k − ¯ v Ts ˜ W Ts S s (¯ x )+ √ N − n max i ∈ ν s { η i }|| ¯ v s |||| x f − z f || , (69)in which we have utilized that ¯v Ts ( d s ( t ) + ε s − φ s )) ≤ ( N − n ) ǫ , which can be easily proved by Lemma 3.Moreover, ˙ V =( x f − z f ) T ( ˙ x f − ˙ z f )=( x f − z f ) T ( − k ( x f − z f ) + k ¯ P f (¯ x )) − ( x f − z f ) T ( ˜ W Tf S f (¯ x ) + g s ( x ) − g s (¯ x ))+ ( x f − z f ) T ( − φ f + ε f + d f ( t )) ≤ − k k x f − z f k + nǫ + k l k x f − z f kk ¯ x − x ∗ k− ( x f − z f ) T ˜ W Tf S f (¯ x ) + √ n max i ∈ ν f { η i }|| x f − z f || , (70)in which ( x f − z f ) T ( − φ f + ε f + d f ( t )) ≤ nǫ can be easilyproved by Lemma 3.Furthermore, ˙ V = N X i =1 T r ( ˜ W Ti ˙˜ W i ) β = n X f =1 T r ( ˜ W Tf ˙ˆ W f ) β + N X s = n +1 T r ( ˜ W Ts ˙ˆ W s ) β . (71)Hence, ˙ V ≤ − k m k ¯ x − x ∗ k − ( k − k h ) k ¯ v s k − k k x f − z f k + N ǫ + ( k hl + 1) k ¯ x − x ∗ kk ¯ v s k + k l k x f − z f kk ¯ x − x ∗ k + √ N − n max i ∈ ν s { η i }|| ¯ v s |||| x f − z f || + √ n max i ∈ ν f { η i }|| x f − z f || + n X f =1 T r ˜ W Tf ˙ˆ W f β − S f (¯ x )( x f − z f ) !! + N X s = n +1 T r ˜ W Ts ˙ˆ W s β − S s (¯ x )¯ v s !! ≤ − k m k ¯ x − x ∗ k − ( k − k h ) k ¯ v s k (72) k k x f − z f k + N ǫ + ( k hl + 1) k ¯ x − x ∗ kk ¯ v s k + k l k x f − z f kk ¯ x − x ∗ k − N X i =1 T r ( ˜ W Ti ˜ W i )+ √ N − n max i ∈ ν s { η i }|| ¯ v s |||| x f − z f || + √ n max i ∈ ν f { η i }|| x f − z f || + 4 N W max , in which P nf =1 T r (cid:18) ˜ W Tf (cid:18) ˙ˆ W f β − S f (¯ x )( x f − z f ) (cid:19)(cid:19) ≤ and P Ns = n +1 T r (cid:16) ˜ W Ts (cid:16) ˙ˆ W s β − S s (¯ x )¯ v s (cid:17)(cid:17) ≤ , which canbe obtained by following the proof of Theorem 2 in [31].Hence, ˙ V ≤ − (cid:18) k m − ρ (cid:19) k ¯ x − x ∗ k − Ψ k ¯ v s k + N ǫ − ( k − √ n max i ∈ ν f { η i } − √ N − n max i ∈ ν s { η i } k x f − z f k + k l k x f − z f kk ¯ x − x ∗ k − N X i =1 T r ( ˜ W Ti ˜ W i ) + 4 N W max , (73)where ρ is a positive constant that can be arbitrarily chosenand Ψ = k − k h − ρ ( k hl +1) + √ N − n max i ∈ νs { η i } .In addition, ˙ V ≤ − Ψ k ¯ x − x ∗ k − Ψ k ¯ v s k − Ψ k x f − z f k − N X i =1 T r ( ˜ W Ti ˜ W i ) + 4 N W max + N ǫ, (74)where ρ is a positive constant that can be arbitrarilychosen, Ψ = k m − ρ − l ρ and Ψ = k − ρ l k −√ n max i ∈ ν f { η i } − √ N − n max i ∈ νs { η i } . Therefore, ˙ V ≤ − KV + ∆ , (75)where K = 2 min { Ψ , Ψ , Ψ , β } and ∆ = N ǫ + 4
N W max . Hence, V ≤ V (0) e − Kt + ∆ K (1 − e − Kt ) , (76)by Lemma 2.Recalling the definition of the Lyapunov candidate func-tion, it can be obtained that k ¯ x ( t ) − x ∗ k + k ¯ v s ( t ) k + k x f ( t ) − z f ( t ) k ≤ V (0) e − Kt + 2 ∆ K . (77)In addition, with fixed ρ and ρ , we can choose k to besufficiently large such that Ψ > and is sufficiently large.Moreover, with fixed k , ρ and ρ , we can choose k , k and β to be sufficiently large such that K is sufficiently large.Therefore, we arrive at the conclusion. (cid:3) In this section, we consider mixed-order games in whichthe players’ dynamics are subject to unmodeled dynamicsand external disturbance by designing a centralized method.In the following, we propose a distributed counterpart for theconsidered problem.
B. A distributed algorithm for mixed-order multi-playergames
To achieve disturbance rejection for the mixed-ordergames in (46), the distributed Nash equilibrium seekingstrategy is designed as ˙ x f = − k ( x f − z f ) − ˆ W Tf S f ( y f ) − φ f + g f ( x ) + d f ( t )˙ z f = − k ∇ f F f ( y f ) , f ∈ ν f ˙ x s = v s ˙ v s = − k v s − k k ∇ s F s ( y s ) − ˆ W Ts S s ( y s ) − φ s + g s ( x ) + d s ( t ) , s ∈ ν s ˙ y ij = − k N X k =1 a ik ( y ij − y kj ) + a ij ( y ij − ¯ x j ) ! i, j ∈ ℵ , (78)where k , k , k , k are positive constants, ¯ x j = z j for j ∈ ν f and ¯ x j = x j for j ∈ ν s . In addition, the weight matrices ˆ W f and ˆ W s are updated according to ˙ˆ W f = βS f ( y f )( x f − z f ) , if Tr ( ˆ W Tf ˆ W f ) < W max or Tr ( ˆ W Tf ˆ W f ) = W max and ( x f − z f ) ˆ W Tf S f ( y f ) < βS f ( y f )( x f − z f ) − β ( x f − z f ) ˆ W Tf S f ( y f ) Tr ( ˆ W Tf ˆ W f ) ˆ W f , if Tr ( ˆ W Tf ˆ W f ) = W max and ( x f − z f ) ˆ W Tf S f ( y f ) ≥ and ˙ˆ W s = βS s ( y s )¯ v s , if Tr ( ˆ W Ts ˆ W s ) < W max or Tr ( ˆ W Ts ˆ W s ) = W max and ¯ v s ˆ W Ts S s ( y s ) < βS s ( y s )¯ v s − β ¯ v s ˆ W Ts S s ( y s ) Tr ( ˆ W Ts ˆ W s ) ˆ W s , if Tr ( ˆ W Ts ˆ W s ) = W max and ¯ v s ˆ W Ts S s ( y s ) ≥ . (79)Moreover, the concatenated-vector form of (78) is ˙ x f = − k ( x f − z f ) − ˆ W Tf S f (¯ y f ) − φ f + g f ( x ) + d f ( t )˙ z f = − k ¯ P f (¯ y f )˙ x s = v s ˙ v s = − k v s − k k ¯ P s (¯ y s ) − ˆ W Ts S s (¯ y s ) − φ s + g s ( x ) + d s ( t ) ˙y = − k ( L ⊗ I N × N + A )( y − N ⊗ ¯ x ) . (80)The following lemma is given to support the upcomingstability analysis. Lemma 5:
Suppose that Assumptions 1-5 are satisfied.Then, there exists a positive constant k ∗ such that for each k > k ∗ , there exist positive constants k ∗ , k ∗ such that for k > k ∗ , k > k ∗ , there exists a positive constant k ∗ suchthat for k > k ∗ , x ( t ) , z f ( t ) , v s ( t ) and y ( t ) generated bythe proposed method in (78) stay bounded given that theirinitial values are bounded. Proof:
Let v s = ¯ v s − k ¯ P s (¯ y s ) . (81)hen, ˙ x s =¯ v s − k ¯ P s (¯ y s ) ˙¯v s = ˙ v s + k H (¯ y s ) ˙¯ y s = − k ¯ v s + k H (¯ y s ) ˙¯ y s − ˆ W Ts S s (¯ y s ) − φ s + g s ( x ) + d s ( t ) . (82)Define the Lyapunov candidate function as V = V + V + V + V , (83)where V = 12 (¯ x − x ∗ ) T (¯ x − x ∗ ) , V = 12 ¯ v Ts ¯ v s V = 12 ( x f − z f ) T ( x f − z f ) V =( y − N ⊗ ¯ x ) T P ( y − N ⊗ ¯ x ) . (84)Then, ˙ V =(¯ x − x ∗ ) T ˙¯ x = (¯ x − x ∗ )[ ˙ z Tf , ˙ x Ts ] T = − k (¯ x − x ∗ ) T ¯ P ( y ) + (¯ x − x ∗ ) T [ Tn , ¯ v Ts ] T = − k (¯ x − x ∗ ) T ( ¯ P (¯ x ) − ¯ P ( x ∗ ))+ k (¯ x − x ∗ ) T ( ¯ P (¯ x ) − ¯ P ( y ))+ (¯ x − x ∗ ) T [ Tn , ¯ v Ts ] T ≤ − k m k ¯ x − x ∗ k + k ¯ x − x ∗ kk ¯ v s k + k max i ∈ℵ { ¯ l i }k ¯ x − x ∗ kk y − N ⊗ ¯ x k , (85)and ˙ V =¯ v Ts ( − k ¯ v s + k H ( y s ) ˙ y s )+ ¯ v Ts ( − ˆ W Ts S s ( y s ) + d s ( t ) − φ s + g s ( x ))= − k k ¯ v s k + k ¯ v Ts H (¯ y s ) ˙¯ y s + ¯ v Ts ( − ˆ W Ts S s (¯ y s ) + d s ( t ) − φ s + g s ( x )) ≤ − k k ¯ v s k + a s k ¯ v s k + ¯ v Ts ( g s ( x ) − g s ( x ∗ ))+ k k b k y − N ⊗ ¯ x kk ¯ v s k , (86)in which b = sup ¯ y s ∈ R N k H (¯ y s ) kkL ⊗ I N × N + Ak and a s is defined in the proof of Lemma 4.Moreover, ˙ V =( x f − z f ) T ( ˙ x f − ˙ z f )=( x f − z f ) T ( − k ( x f − z f ) + k ¯ P f (¯ y f ))+( x f − z f ) T ( − ˆ W Tf S f (¯ y f ) − φ f + g f ( x ) + d f ( t )) ≤ − k k x f − z f k + a f k x f − z f k + k l k x f − z f kk ¯ x − x ∗ k + k max i ∈ℵ { ¯ l i }k x f − z f kk y − N ⊗ ¯ x k + ( x f − z f ) T ( g f ( x ) − g f ( x ∗ )) (87)where a f is defined in the proof of Lemma 4 and we haveutilized that k ¯ P ( y ) k = k ¯ P ( y ) − ¯ P (¯ x ) + ¯ P (¯ x ) − ¯ P ( x ∗ ) k ≤ max i ∈ℵ { ¯ l i }k y − N ⊗ x k + l k x − x ∗ k based on Assumption1. Furthermore, ˙ V =( ˙y − N ⊗ ˙¯ x ) T P ( y − N ⊗ ¯ x )+ ( y − N ⊗ ¯ x ) T P ( ˙y − N ⊗ ˙¯ x )= − k ( y − N ⊗ ¯ x ) T ( L ⊗ I N × N + A ) P ( y − N ⊗ ¯ x ) − k ( y − N ⊗ ¯ x ) T P ( L ⊗ I N × N + A )( y − N ⊗ ¯ x ) − y − N ⊗ x ) T P N ⊗ ˙¯ x ≤ − k λ min ( Q ) k y − N ⊗ ¯ x k + 2 k ( y − N ⊗ ¯ x ) T P N ⊗ ¯ P ( y ) − y − N ⊗ ¯ x ) T P N ⊗ [ Tn , ¯ v Ts ] T ≤ − k λ min ( Q ) k y − N ⊗ ¯ x k + 2 k l kPkk y − N ⊗ ¯ x k + 2 k l √ N kPkk y − N ⊗ ¯ x kk ¯ x − x ∗ k + 2 √ N kPkk y − N ⊗ ¯ x kk ¯ v s k . (88)Hence, ˙ V ≤ − k m k ¯ x − x ∗ k − k k x f − z f k − k k ¯ v s k − ( k λ min ( Q ) − k l kPk ) k y − N ⊗ ¯ x k + (1 + √ N − n max i ∈ ν s { η i } ) k ¯ x − x ∗ kk ¯ v s k + a s k ¯ v s k + (2 k l √ N kPk + k max i ∈ℵ { ¯ l i } ) k y − N ⊗ ¯ x kk ¯ x − x ∗ k + (2 √ N kPk + k k b ) k y − N ⊗ ¯ x kk ¯ v s k + ( k l + √ n max i ∈ ν f { η i } ) k x f − z f kk ¯ x − x ∗ k + ( k max i ∈ℵ { ¯ l i } ) k x f − z f kk y − N ⊗ ¯ x k + a f k x f − z f k + √ N − n max i ∈ ν s { η i }|| ¯ v s |||| x f − z f || + √ n max i ∈ ν f { η i }|| x f − z f || . (89)Therefore, ˙ V ≤ − ¯Ψ || ¯ x − x ∗ || − ¯Ψ || x f − z f || − ¯Ψ || ¯ v s || − ¯Ψ || y − N ⊗ ¯ x || + a f || x f − z f || + a s || ¯ v s || , (90)where ρ i , i ∈ { , , , } are positive constants that can be ar-bitrarily chosen, ¯Ψ = k m − √ N − n max i ∈ νs { η i } − ρ + ρ , ¯Ψ = k − √ n max i ∈ ν f { η i } − ( k l + √ n max i ∈ νf { η i } ) ρ − ( k max i ∈ℵ { ¯ l i } ) ρ − √ N − n max i ∈ νs { η i } , ¯Ψ = k − √ N − n max i ∈ νs { η i } − ρ (2 √ N kPk + k k b ) − √ N − n max i ∈ νs { η i } and ¯Ψ = k λ min ( Q ) − k l kPk − ρ − ρ − (2 k l √ N kPk + k max i ∈ℵ { ¯ l i } ) ρ .Hence, by choosing k to be sufficiently large, ¯Ψ > .Then, for fixed k , we can choose k and k to be sufficientlylarge such that ¯Ψ > and ¯Ψ > . Then, for fixed k , k , k , we can choose k to be sufficiently large suchthat ¯Ψ > . By such a tuning rule, there exists a positiveconstant ¯ θ such that ˙ V ≤ − ¯ θV + a f || x f − z f || + a s || ¯ v s || , (91)from which the conclusion can be easily concluded.y Lemma 5, we can conclude that g i ( y i ) = W ∗ i T S i ( y i ) + ε i by Lemma 1. Hence, we can obtain that ˙ x f = − k ( x f − z f ) − ˜ W Tf S f ( y f )+ g f ( x ) − g f ( y f ) + d f ( t ) + ε f − φ f ˙ z f = − k ∇ f F f ( y f ) , f ∈ ν f ˙ x s = v s ˙ v s = − k v s − k k ∇ s F s ( y s ) − ˜ W Ts S s ( y s )+ g s ( x ) − g s ( y f ) + d s ( t ) + ε s − φ s , s ∈ ν s ˙ y ij = − k N X k =1 a ik ( y ij − y kj ) + a ij ( y ij − ¯ x j ) ! i, j ∈ ℵ , (92)Moreover, the concatenated-vector form of (92) is ˙ x f = − k ( x f − z f ) − ˜ W Tf S f (¯ y f )+ g f ( x ) − g f (¯ y f ) + ( d f ( t ) + ε f − φ f )˙ z f = − k ¯ P f (¯ y f )˙ x s = v s ˙ v s = − k v s − k k ¯ P s (¯ y s ) − ˜ W Ts S s (¯ y s )+ g s ( x ) − g s (¯ y f ) + ( d s ( t ) + ε s − φ s ) ˙y = − k ( L ⊗ I N × N + A )( y − N ⊗ ¯ x ) . (93) Theorem 4:
Suppose that Assumptions 1-5 are satisfied.Then, for any pair of positive constants Λ and Ξ , there existsa positive constant β ∗ and k ∗ such that for β > β ∗ and k >k ∗ such that there exist positive constants k ∗ and k ∗ suchthat for k > k ∗ , k > k ∗ , there exists a positive constant k ∗ such that for k > k ∗ , there exists a positive constant T such that k x ( t ) − x ∗ k + k v ( t ) k ≤ Ξ , ∀ t > T, (94)given that k [(¯ x (0) − x ∗ ) T , v s (0) T , ( y (0) − N ⊗ ¯ x (0)) T , ( x f (0) − z f (0)) T ] T k + P Ni =1 T r ( ˜ W i (0) T ˜ W i (0)) ≤ Λ . Proof:
Define the Lyapunov candidate function as V = V + V + V + V + V , (95)where V = 12 (¯ x − x ∗ ) T (¯ x − x ∗ ) , V = 12 ¯v Ts ¯v s V = 12 ( x f − z f ) T ( x f − z f ) V =( y − N ⊗ ¯ x ) T P ( y − N ⊗ ¯ x ) V = 12 β N X i =1 T r ( ˜ W Ti ˜ W i ) (96)Then, ˙ V ≤ − k m k ¯ x − x ∗ k + k ¯ x − x ∗ kk ¯ v s k + k max i ∈ℵ { ¯ l i }k ¯ x − x ∗ kk y − N ⊗ ¯ x k , (97) and ˙ V =¯ v Ts ˙¯ v s = ¯ v Ts ( − k ¯ v s + k H (¯ y s ) ˙¯ y s )+ ¯ v Ts ( − ˜ W Ts S s (¯ y s ) + d s ( t ) − φ s + ε s )+ ¯ v Ts ( g s ( x ) − g s (¯ y s ))= − k k ¯ v s k + k ¯ v Ts H (¯ y s ) ˙¯ y s + ¯ v Ts ( − ˜ W Ts S s (¯ y s ) + d s ( t ) − φ s + ε s )+ ¯ v Ts ( g s ( x ) − g s (¯ y s )) ≤ − k k ¯ v s k + k k b k y − N ⊗ ¯ x kk ¯ v s k− ¯ v Ts ˜ W Ts S s (¯ y s ) + ( N − n ) ǫ + ¯ v Ts ( g s ( x ) − g s (¯ y s )) . (98)Moreover, ˙ V =( x f − z f ) T ( ˙ x f − ˙ z f )=( x f − z f ) T ( − k ( x f − z f ) + k ¯ P f ( y f ))+ ( x f − z f ) T ( − ˜ W Tf S f (¯ y f ) − φ f + ε f + d f ( t ))+ ( x f − z f ) T ( g f ( x ) − g f (¯ y f )) ≤ − k k x f − z f k + nǫ + k l k x f − z f kk ¯ x − x ∗ k + k max i ∈ℵ { ¯ l i }k x f − z f kk y − N ⊗ ¯ x k− ( x f − z f ) T ˜ W Tf S f (¯ y f )+ ( x f − z f ) T ( g f ( x ) − g f (¯ y f )) , (99)and ˙ V ≤ − k λ min ( Q ) k y − N ⊗ ¯ x k + 2 k l kPkk y − N ⊗ ¯ x k + 2 k l √ N kPkk y − N ⊗ ¯ x kk ¯ x − x ∗ k + 2 √ N kPkk y − N ⊗ ¯ x kk ¯ v s k . (100)Furthermore, ˙ V = N X i =1 T r ( ˜ W Ti ˙˜ W i ) β = n X f =1 T r ( ˜ W Tf ˙ˆ W f ) β + N X s = n +1 T r ( ˜ W Ts ˙ˆ W s ) β . (101)ence, ˙ V ≤ − k m k ¯ x − x ∗ k − k k x f − z f k − k k ¯ v s k − ( k λ min ( Q ) − k l kPk ) k y − N ⊗ ¯ x k + k ¯ x − x ∗ kk ¯ v s k + N ǫ + (2 k l √ N kPk + k max i ∈ℵ { ¯ l i } ) k y − N ⊗ ¯ x kk ¯ x − x ∗ k + (2 √ N kPk + k k b ) k y − N ⊗ ¯ x kk ¯ v s k + k l k x f − z f kk ¯ x − x ∗ k + k max i ∈ℵ { ¯ l i }k x f − z f kk y − N ⊗ ¯ x k + n X f =1 T r ˜ W Tf ˙ˆ W f β − S f (¯ y f )( x f − z f ) !! + N X s = n +1 T r ˜ W Ts ˙ˆ W s β − S s (¯ y s )¯ v s !! + ( x f − z f ) T ( g f ( x ) − g f (¯ y f )) + ¯ v Ts ( g s ( x ) − g s (¯ y s )) ≤ − k m k ¯ x − x ∗ k − k k x f − z f k − k k ¯ v s k − ( k λ min ( Q ) − k l kPk ) k y − N ⊗ ¯ x k + (2 k l √ N kPk + k max i ∈ℵ { ¯ l i } ) k y − N ⊗ ¯ x kk ¯ x − x ∗ k + (2 √ N kPk + k k b ) k y − N ⊗ ¯ x kk ¯ v s k + k l k x f − z f kk ¯ x − x ∗ k + k ¯ x − x ∗ kk ¯ v s k + k max i ∈ℵ { ¯ l i }k x f − z f kk y − N ⊗ ¯ x k− N X i =1 T r ( ˜ W Ti ˜ W i ) + N ǫ + 4
N W max + ( x f − z f ) T ( g f ( x ) − g f (¯ y s )) + ¯ v Ts ( g f ( x ) − g f (¯ y s )) . (102)Noticing that k ¯ x − x ∗ kk ¯ v s k ≤ ρ || ¯ x − x ∗ || + ρ || ¯ v s || , (103)where ρ is a positive constant that can be arbitrarily chosenand (2 k l √ N ||P|| + k max i ∈ℵ { ¯ l i } ) || y − N ⊗ ¯ x |||| ¯ x − x ∗ ||≤ (2 l √ N ||P|| + max i ∈ℵ { ¯ l i } ) ρ || x − x ∗ || + ρ k || y − N ⊗ ¯ x || , (104)where ρ is a positive constant that can be arbitrarily chosen.In addition, (2 √ N ||P|| + k k b ) || y − N ⊗ ¯ x |||| ¯ v s ||≤ √ N ||P|| ρ + k b ρ ! || ¯ v s || + (cid:18) √ N ||P|| ρ + k bρ (cid:19) || y − N ⊗ ¯ x || , (105)where ρ is a positive constant that can be arbitrarily chosen. Furthermore, k l || x f − z f |||| ¯ x − x ∗ || ≤ l ρ || ¯ x − x ∗ || + ρ l k || x f − z f || (106)and k l || x f − z f |||| y − N ⊗ ¯ x ||≤ l ρ || y − N ⊗ ¯ x || + ρ l k || x f − z f || , (107)where ρ , ρ are positive constants that can be arbitrarilychosen.Hence, ˙ V ≤ − ¯Φ || ¯ x − x ∗ || − ( k − ρ − √ N ||P|| ρ − k b ρ ) || ¯ v s || − (cid:18) k − ρ l k − k l ρ (cid:19) || x f − z f || − (cid:18) k λ min ( Q ) − k l ||P|| − ρ k − √ N ||P|| ρ − k bρ − l ρ (cid:19) || y − N ⊗ ¯ x || + N ǫ + 4
N W max − N X i =1 T r ( ˜ W Ti ˜ W i ) + ( x f − z f ) T ( g f ( x ) − g f (¯ y s ))+ ¯ v Ts ( g f ( x ) − g f (¯ y s )) , (108)where ¯Φ = k m − ρ − (2 l √ N ||P|| +max i ∈ℵ { ¯ l i } ) ρ − l ρ . By further noticing that ( x f − z f ) T ( g f ( x ) − g f (¯ y f )) ≤√ n max i ∈ ν f { η i }|| x f − z f || + max i ∈ ν f { η i }|| x f − z f |||| y − N ⊗ ¯ x || , (109)and similarly, ¯ v Ts ( g s ( x ) − g s (¯ y s )) ≤√ N − n max i ∈ ν s { η i }|| ¯ v s |||| x f − z f || + max i ∈ ν s { η i }|| ¯ v s |||| y − N ⊗ ¯ x || . (110)Let ¯Φ = k − ρ − √ N ||P|| ρ − k b ρ − √ N − n max i ∈ νs { η i } − max i ∈ νs { η i } , ¯Φ = k − ρ l k − k l ρ −√ n max i ∈ ν f { η i } − max i ∈ νf { η i } − √ N − n max i ∈ νs { η i } , and ¯Φ = λ min ( P )( k λ min ( Q ) − k l ||P|| − ρ k −√ N ||P|| ρ − k bρ − l ρ − max i ∈ νf { η i } − max i ∈ νs { η i } ) , then, ˙ V ≤ − KV + N ǫ + 4
N W max , (111)where K = 2 min { ¯Φ , ¯Φ , ¯Φ , ¯Φ , β } / max { , λ max ( P ) } . Hence, by Lemma 2, we can obtain that V ( t ) ≤ V (0) e − Kt + N ǫ + 4
N W max
K , (112)where K can be arbitrarily large by the following tuning rule:choose k to be sufficiently large such that ¯Φ is sufficientlylarge. Then, for fixed k , choose k , k such that ¯Φ and Φ are sufficiently large. Then, for fixed k , choose k tobe sufficiently large such that for fixed k , choose k to besufficiently large such that ¯Φ is sufficiently large. If thisis the case, K is sufficiently large with sufficiently large β , indicating that V ( t ) can converge to an arbitrarily smallneighborhood of zero by such a tuning rule. Recalling thedefinitions of the Lyapunov candidate function and ¯ v s , theconclusion can be obtained. (cid:3) VI. S
IMULATION STUDIES
In this section, we consider the connectivity control ofa network of sensors considered in [26] in which theobjective function of sensor i is given as F i ( x ) = h i ( x i ) + l i ( x ) , (113)where x i = [ x i , x i ] T ∈ R and h i ( x i ) = x Ti m ii x i + x Ti m i + i, (114)in which m ii = (cid:20) i i (cid:21) , m i = [ i, i ] T . Moreover, l ( x ) = k x − x k , l ( x ) = k x − x k , l ( x ) = k x − x k ,l ( x ) = k x − x k + k x − x k and l ( x ) = k x − x k .It was calculated in [26] that on the Nash equilibrium ofthe game, x ∗ ij = − for i ∈ { , , , , } , j ∈ { , } . Inthe following simulations, we suppose that sensors - arefirst-order integrator-type sensors and sensors - are second-order integrator-type sensors. A. Nash equilibrium seeking for mixed-order integrator-typegames
In this section, we consider that the players’ dynamics aregiven by ˙ x f = u f , f ∈ ν f ¨ x s = u s , s ∈ ν s . (115)In the subsequent simulations, the proposed methods in (13)and (28) will be numerically verified one by one.
1) Centralized Nash equilibrium seeking:
Let x (0) =[ − , , − , − , − , , , − , − , T , v s (0) =[0 , , , T .Then, the simulation results generated by the proposedmethod in (13) are shown in Figs. 1-2, in which Fig. 1 showsthe evolutions of the players’ positions and Fig. 2 illustrates v s ( t ) . From the simulation results, we see that the players’actions would converge to the Nash equilibrium.
2) Distributed Nash equilibrium seeking:
In thissection, we suppose that x (0) = [ − , , − , − , , , , − , − , T , and v s (0) =[ − , − , , T .Moreover, y (0) is initialized at zero. With thecommunication graph given in Fig. 3, the simulationresults generated by (28) are shown in Figs. 4-5, in whichFig. 4 shows the evolutions of the players’ positions.In addition, Fig. 5 plots the trajectories of v s ( t ) . Thesimulation results demonstrate that the players’ actionswould converge to the Nash equilibrium, thus verifyingTheorem 2. -8 -7 -6 -5 -4 -3 -2 -1 0 1x i1 -8-6-4-20246810 x i (-8,5) (-2,-4)(-5,7) (1,-8)(-1,9)(-0.5,-0.5)-0.5002 -0.5-0.504-0.502-0.5-0.498-0.496 Fig. 1: The trajectories of players’ positions generated by(13). Th e t r a j ec t o r i es o f v . Fig. 2: The trajectories of v s ( t ) generated by (13). Fig. 3: The communication graph among the players. -5 -4 -3 -2 -1 0 1x i1 -10-8-6-4-20246810 x i (-5,3) (-4,-6) (1,8)(0,-8)(-1,10)(-0.5,-0.5)-0.8 -0.7 -0.6 -0.5-1.5-1-0.500.5 Fig. 4: The trajectories of players’ positions generated by(28). Th e t r a j ec t o r i es o f v Fig. 5: v s ( t ) generated by (28). B. Mixed-order games with unmodeled dynamics and distur-bances
In the section, we consider that the players’ actions aregiven by ˙ x f = u f + g f ( x ) + d f ( t ) , f ∈ ν f ¨ x s = u s + g s ( x ) + d s ( t ) , s ∈ ν s . (116)In addition, g i ( x ) + d i ( t ) in the players’ dynamics are x + sin ( t ) , x + sin ( t ) , x + x + 2 sin (2 t ) , x + 2 sin (2 t ) , x + 3 sin (3 t ) , x + 3 sin (3 t ) , x + 4 sin (4 t ) , x +4 sin (4 t ) , x + 5 sin (5 t ) , x + 5 sin (5 t ) , respectively.In the simulation, the number of neurons of the RBFNN ischosen as and the centers of RBFNN activation functionare -2.5, -2, -1.5, -1, -0.5, 0, 0.5, 1, 1.5, 2, 2.5, respectively.Furthermore, the variances are all set as √ . By setting W max = 500 , β = 100 , δ = 10 , ǫ = 0 . and ˆ W i (0) as a zeromatrix, the centralized algorithm in (50) and the distributedalgorithm in (78) will be simulated, successively.
1) Centralized Nash equilibrium seeking:
With x (0) =[ − , , − , − , − , , , − , − , T , v s (0) =[0 , , , T ,the simulation results produced by (50) are plotted in Figs.6-7. Fig. 6 plots the evolutions of the players’ positions andFig. 7 illustrates v s ( t ) generated by the proposed method.From the simulation results, it can be concluded that theplayers’ actions would be driven to the Nash equilibrium bythe proposed methods.
2) Distributed Nash equilibrium seeking:
With x (0) =[ − , , − , − , , , , − , − , T , v s (0) = [0 , , , T ,the simulation results produced by (78) are given in Figs.8-9 by utilizing the communication graph in Fig. 3. Figs. 8-9 illustrate the evolutions of the players’ positions and v s ( t ) , from which we see that the players’ actions would convergeto the Nash equilibrium.VII. C ONCLUSIONS
This paper considers Nash equilibrium seeking for mixed-order multi-player games consisting of first-order integrator-type players and second-order integrator-type players. A cen-tralized algorithm is designed based on the gradient search,followed by a distributed seeking strategy. Considering that -8 -7 -6 -5 -4 -3 -2 -1 0 1x i1 -8-6-4-20246810 x i (-8,5) (-2,-4)(-5,7) (1,-8)(-1,9)(-0.5,-0.5)-0.5 -0.45 -0.4-1-0.50 Fig. 6: The evolutions of players’ positions generated by(50). Th e t r a j ec t o r i es o f v . Fig. 7: v s ( t ) generated by (50).unmodeled dynamics and external disturbances are inevitablein many practical situations, we further address mixed-order games in which the players are subject to unmodeleddynamics and time-varying disturbances. In this situation, acentralized algorithm and a distributed algorithm are pro-posed one by one. The convergence of the proposed seekingstrategies are investigated analytically based on Lyapunovstability analysis. R EFERENCES[1] L. Deori, K. Margellos, M. Prandini, “Price of anarchy in electricvehicle charging control games: When Nash equilibria achieve socialwelfare,”
Automatica, vol. 96, pp. 150-158, 2018.[2] A. Cherukuri, J. Cort´es, “Decentralized Nash equilibrium learningby strategic generators for economic dispatch,”
American ControlConference, pp. 1082-1087, 2016.[3] M. Stankovic, K. Johansson, D. Stipanovic, “Distributed seeking ofNash equilibria with applications to mobile sensor networks,”
IEEETransactions on Automatic Control, vol. 57, no. 4, pp. 904-919, 2011.[4] W. Lin, C. Li, Z. Qu, M. Simaan, “Distributed formation control withopen-loop Nash strategy,”
Automatica, vol. 106, pp. 266-273, 2019.[5] M. Ye, G. Hu, “Distributed Nash equilibrium seeking by a consensusbased approach,”
IEEE Transactions on Automatic Control, vol. 62,no. 9, pp. 4811-4818, 2017.[6] M. Ye, G. Hu, “Distributed Nash equilibrium seeking in multiagentgames under switching communication topologies,”
IEEE Transac-tions on Cybernetics, vol. 48, no. 11, pp. 3208 - 3217, 2017. i1 -10-8-6-4-20246810 x i (-5,8) (-4,-6) (1,8)(0,-8)(-1,10)(-0.5,-0.5)-0.5 -0.4 -0.3-202 Fig. 8: The evolutions of players’ positions generated by(78). Th e t r a j ec t o r i es o f v Fig. 9: v s ( t ) generated by (78). [7] C. Persis, S. Grammatico, “Distributed averaging integral Nashequilibrium seeking on networks,” Automatica, published online,DOI:10.1016/j.automatica.2019.108548.[8] M. Ye, G. Hu, “Game design and analysis for price-based demandresponse: an aggregate game approach,”
IEEE Transactions on Cy-bernetics, vol. 47, no. 3, pp. 720 - 730, 2017.[9] H. Hua, Y. Qin, C. Hao, and J. Cao, “Optimal energy managementstrategies for energy Internet via deep reinforcement learning ap-proach,”
Applied Energy, vol. 239, pp. 598-609, 2019.[10] A. Ibrahim, T. Hayakawa, “Nash equilibrium seeking with second-order dynamic agents,”
IEEE Conference on Decision and Control, pp. 2514-2518, 2018.[11] M. Ye, “Distributed Nash equilibrium seeking for games in systemswith bounded control inputs,” arXiv preprint arXiv:1901.09333, 2019.[12] A. Ibrahim, T. Hayakawa, “Nash equilibrium seeking with linear time-invariant dynamic agents,”
American Control Conference, pp. 1202-1207, 2019.[13] Z. Deng, S. Liang, “Distributed algorithms for aggregative games ofmultiple heterogeneous Euler-Lagrange systems,”
Automatica, vol. 99,pp. 246-252, 2019.[14] M. Ye, “Nash equilibrium seeking for games in hybrid systems,”
International Conference on Control, Automation, Robotics and Vision, pp. 140-145, 2018.[15] C. Liu, F. Liu, “Stationary consensus of heterogeneous multi-agentsystems with bounded communication delays,”
Automatica, vol.47,no.9, pp.2130-2133, 2011.[16] W. Jiang, G. Wen, Z. Peng et al, “Fully distributed formation-containment control of heterogeneous linear multi-agent systems,”
IEEE Transactions on Automatic Control, vol. 64, no. 9, pp. 3889-3896, 2018.[17] Y. Hua, X. Dong, Q. Li, and Z. Ren, “Distributed time-varying formation robust tracking for general linear multiagent systems withparameter uncertainties and external disturbances”
IEEE Transactionson Cybernectics, vol. 48, no. 7, pp. 1959-1969, 2017.[18] F. Yaghmaie, F. Lewis, R. Su, “Output regulation of linear heteroge-neous multi-agent systems via output and state feedback,”
Automatica, vol. 67, pp. 157-164, 2016.[19] A. Bidram, F. Lewis, A. Davoudi, S. Ge, “Adaptive and distributedcontrol of nonlinear and heterogeneous multi-agent systems,”
IEEEConference on Decision and Control, pp. 6238-6243, 2013.[20] J. Ma, M. Ye, Y. Zheng, Y. Zhu, “Consensus analysis of hybrid mul-tiagent systems: A game-theoretic approach,”
International Journal ofRobust and Nonlinear Control, vol. 29, no. 6, pp. 1840-1853, 2019.[21] J. Mei, W. Ren, J. Chen, “Distributed consensus of second-order multi-agent systems with heterogeneous unknown inertias and control gainsunder a directed graph,”
IEEE Transactions on Automatic Control, vol.61, no. 8, pp.2019-2034, 2015.[22] M. Zheng, C. Liu, F. Liu, “Average-consensus tracking of sensornetwork via distributed coordination control of heterogeneous multi-agent systems,”
IEEE Control Systems Letters, vol. 3, no. 1, pp. 132-137, 2018.[23] Y. Zheng, L. Wang, “Consensus of heterogeneous multi-agent systemswithout velocity measurements,”
International Journal of Control, vol.85, no. 7, pp. 906-914, 2012.[24] Z. Li, Z. Wu, Z. Li, Z. Ding, “Distributed optimal coordinationfor heterogeneous linear multi-agent systems with event-triggeredmechanisms,”
IEEE Transactions on Automatic Control, publishedonline, DOI:10.1109/TAC.2019.2937500.[25] H. Khailil,
Nonlinear Systems,
Upper Saddle River, NJ: Prentice Hall,2002.[26] M. Ye, “A RISE-based distributed robust Nash equilibrium seekingstrategy for networked games”,
IEEE Conference on Decision andControl,
Automatica, vol. 95, pp. 266-272, 2018.[28] M. Ye, G. Hu, F. Lewis, L. Xie, “A unified strategy for solution seekingin graphical N-coalition noncooperative games,”
IEEE Transactionson Automatic Control, vol. 64, no. 11, pp. 4645-4652, 2019.[29] M. Ye, G. Hu and S. Xu, “An extremum seeking-based approachfor Nash equilibrium seeking in N-cluster noncooperative games,”
Automatica, vol. 114, 108815, 2020.[30] M. Polycarpou, “Stable adaptive neural control scheme for nonlinearsystems,”
IEEE Transactions on Automatic Control, vol. 41, no. 3, pp.447-451, 1996.[31] Z. Hou, L. Cheng, M. Tan, “Decentralized robust adaptive controlfor the multiagent system consensus problem using neural networks,”
IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cyber-netics), vol. 39, no. 3, pp. 636-647, 2009.[32] J. Yao, W. Deng, Z. Jiao, “RISE-based adaptive control of hydraulicsystems with asymptotic tracking,”
IEEE Transactions on AutomationScience and Engineering, vol. 14, no. 3, pp. 1524-1531, 2017.[33] A. Fazeli, M. Zeinali, A. Khajepour, “Application of adaptive slidingmode control for regenerative braking torque control,”
IEEE/ASMETransactions On Mechatronics, vol. 17, no. 4, pp. 745-755, 2012.[34] S. He, S. Dai, F. Luo, “Asymptotic trajectory tracking control withguaranteed transient behavior for MSV With uncertain dynamics andexternal disturbances,”
IEEE Transactions on Industrial Electronics, vol. 66, no. 5, pp. 3712-3720, 2019.[35] T. Poggio, F. Girosi, “Networks for approximation and learning,”
Proceedings of the IEEE, vol. 78, no. 9, pp. 1481-1497, 1990.[36] L. Cheng, Z. Hou, M. Tan, Y. Lin, W. Zhang, “Neural-network-based adaptive leader-following control for multiagent systems withuncertainties,”
IEEE Transactions on Neural Networks, vol. 21, no. 8,pp. 1351-1358, 2010.[37] C. Chen, G. Wen, Y. Liu, F. Wang, “Adaptive consensus controlfor a class of nonlinear multiagent time-delay systems using neuralnetworks,”
IEEE Transactions on Neural Networks and LearningSystems, vol. 25, no. 6, pp. 1217-1226, 2014.[38] G. Wen, C. Chen, Y. Liu, Z. Liu, “Neural-network-based adaptiveleader-following consensus control for second-order non-linear multi-agent systems,”