Adaptive set-point regulation of discrete-time nonlinear systems
aa r X i v : . [ m a t h . O C ] M a y Adaptive set-point regulation of discrete-timenonlinear systems ∗ Shigeru Hanba † October 15, 2018
Abstract
In this paper, adaptive set-point regulation controllers for discrete-time nonlinear systems are constructed. The system to be controlled isassumed to have a parametric uncertainty, and an excitation signal is usedin order to obtain the parameter estimate. The proposed controller be-longs to the category of indirect adaptive controllers, and its constructionis based on the policy of calculating the control input rather than that ofobtaining a control law. The proposed method solves the adaptive set-point regulation problem under the (possibly minimal) assumption thatthe target state is reachable provided that the parameter is known. Ad-ditional feature of the proposed method is that Lyapunov-like functionshave not been used in the construction of the controllers. keywords adaptive set-point regulation, discrete-time nonlinear systems, per-sistent excitation
For decades, adaptive control of nonlinear systems has been an active area ofresearch, and several design methods have been established for both continuous-time and discrete-time systems [1–16]. A majority of these design methodsfirst assume that the systems are described in some canonical forms and thatparametric Lyapunov-like functions are known; they then construct controllerstogether with tuners of specific forms to obtain sufficient conditions for thestability of the closed-loop system. In other words, their sufficient conditions ∗ This work was supported by the Japan Society for the Promotion of Science under Grant-in-Aid for Scientific Research (C) 23560535. This manuscript is a former version of themanuscript the author has submitted to International Journal of Adaptive Control and SignalProcessing. The manuscript was rejected, and a revision is in preparation, but this versiondoes not reflect the comments of the referees to the rejected version. † Department of Electrical and Electronics Engineering, University of the Ryukyus, 1Senbaru Nishihara, Nakagami-gun, Okinawa 903-0213, Japan; email: [email protected]
Consider a discrete-time nonlinear system with parametric uncertainty of theform x ( t + 1) = f ( x ( t ) , u ( t ) , θ ) , (1)where x ( t ) ∈ R n is the state, u ( t ) ∈ R n u is the control input, and θ ∈ R n θ is theparameter to be estimated. The parameter θ is assumed to be inside a compactand convex set Ω θ , and the function f is assumed to be C with respect to allarguments.Henceforth, we use the following notations. We denote the sequence of inputs( u ( t ) , . . . , u ( t )) by u [ t , t ]. The sequence of the state ( x ( t ) , . . . , x ( t )) isdenoted as X [ t , t ; u, θ ], where the symbols u, θ have been added to emphasizetheir effect. Although u [ t , t ] and X [ t , t ; u, θ ] are sequences of vectors withlength t − t + 1, we sometimes identify them with vectors in R n u ( t − t +1) and R n ( t − t +1) , respectively.The solution of (1) at t = t initialized at t = t with x ( t ) is denotedby ϕ ( t , t , x ( t ); u, θ ). The symbols B ( x, ρ ) and B ( x, ρ ) denote the open andclosed balls centered at x with radius ρ . For a sequence u [ t , t ], B ∞ ( u [ t , t ] , ρ )denotes the set { v [ t , t ] : ∀ t ∈ { t , . . . , t t } , v ( t ) ∈ B ( u ( t ) , ρ ) } . The symbol N denotes the set of nonnegative integers.Let the target state be x ∗ . Henceforth, we assume that the target state isfinite-time reachable in the following sense, which is a parametric counterpartof those given in [17]. Assumption 1 ∀ x , ∀ θ ∈ Ω θ , ∃ N > , ∃ u [0 , N − ,1. ϕ ( N, , x ; u, θ ) = x ∗ ,2. rank ∂ϕ ( N, ,x ; u,θ ) ∂u [0 ,N − = n . Assumption 2 ∃ N , ∃ u [0 , N − ∀ x , ∀ θ ∈ Ω θ , rank ∂X [0 ,N ; u,θ ] ∂θ = n θ .In what follows, we construct a controller with structure similar to blockmodel predictive controllers [17,20] together with parameter estimators based onnonlinear equation solvers. Although model predictive control is an application-oriented method, the use of this ‘block model predictive control’ structure in ouradaptive controller has completely different objective. It is used as a theoreticaltool to show that Assumption 2 serves as a ‘persistent excitation condition’for a nonlinear system of the form (1). Basically, our control strategy is asfollows. Partition the time interval N into blocks of finite length (the lengthof each block is determined adaptively. ) Let t = T k be the beginning of the k -th block. At this time instant, update the parameter estimate θ ( T k ) usingthe entire sequence of past states. Then, obtain N k and u [ T k , T k + N k −
1] suchthat ϕ ( T k + N k , T k , x ( T k ); u, θ ( T k )) = x ∗ or ϕ ( T k + N k , T k , x ( T k ); u, θ ( T k )) ∈N ( x ∗ ), where N ( x ∗ ) is a neighborhood of x ∗ . Next, apply the input sequence u [ T k , T k + N k −
1] to the system (1) until t = T k + N k − θ ( T k ) is rewritten as θ k . For the parameter estimation,the entire sequence of states up to T k , X [1 , T k ; u, θ ] will be used, but whatreally matters is the dependence on θ only; we rewrite this expression in thecolumn vector form and let g k ( θ ) = (( x (1)) T , . . . , ( x ( T k )) T ) T , omitting unneces-sary variables to avoid confusion. Similarly, we rewrite ϕ ( T k + N k , , x (0); u, θ )as h k ( θ ). We first consider the ideal case where solutions of nonlinear equations are avail-able exactly, and we then consider the case where numerical errors to the solu-tions of nonlinear equations do exist. It is to be emphasized, however, that theassumption that an exact solution of the nonlinear equation of the parameterestimate is available, that is, g k ( θ k ) = g k ( θ ), does not always imply that θ k = θ because g k ( θ ) is not always a global injection — Assumption 2 merely assuresthat it is a local injection.The first algorithm of adaptive set-point regulation is as follows. Algorithm 1(Initialization)
Given x (0), let u [0 , N −
1] be the excitation signal, T = 0, T = N , k = 1; apply u [0 , N − x ( T ). (Loop) f x ( T k ) = x ∗ thenbreak ; else Obtain θ k ∈ Ω θ that satisfy g k ( θ k ) = g k ( θ );Obtain u [ T k , T k + N k −
1] that satisfy h k ( θ k ) = x ∗ ;Let T k +1 = T k + N k ;Apply u [ T k , T k +1 −
1] to the system (1) to obtain x ( T k +1 ).Let k = k + 1; Theorem 1
Under the first condition of Assumption 1 and Assumption 2, Al-gorithm 1 terminates after finitely many iterations, and the state of (1) reachesto the target state x ∗ provided that exact solutions of nonlinear equations areavailable. Proof
First, note that our assumptions permit that Algorithm 1 is always fea-sible.We prove our assertion by contradiction. Suppose that Algorithm 1 neverterminates after finitely many iterations. Then, the resulting sequence of theparameter estimate ( θ k ) k ∈ N is an infinite sequence in the compact set Ω θ andhence, it has at least one limit point. Let θ ♯ be one of its limit points.We first prove that ∀ j, g j ( θ ♯ ) = g j ( θ ). Because we have assumed that exactsolutions to nonlinear equations are available, ∀ j , g j ( θ j ) = g j ( θ ). Moreover, for k < k , the relation between g k ( θ ) and g k ( θ ) are given by g k ( θ ) = g k ( θ ) x ( T k + 1)... x ( T k ) . Due to this structure, we call that g k ( θ ) is an initial segment of g k ( θ ). Because θ ♯ is a limit point, there is a subsequence ( θ k l ) l ∈ N of ( θ k ) k ∈ N that convergesto θ ♯ . For any j > g j ( θ ) is a continuous function of θ , and ∀ l such that j ≤ k l , g j ( θ k l ) = g j ( θ ) because g j ( θ k l ) is an initial segment of g k l ( θ ). Sincelim l →∞ θ k l = θ ♯ and g j is continuous, g j ( θ ♯ ) = g j ( θ ).Next, we prove by contradiction that ∃ l , ∀ l ≥ l , θ k l = θ ♯ . Suppose that ∀ l , ∃ l ≥ l , θ k l = θ ♯ . Then, ∀ ε > ∃ θ k l , θ k l ∈ B ( θ ♯ , ε ) \ { θ ♯ } . Because g (the firstsegment of g k l ) is C and ∂g ∂θ is of full rank, ∃ c > , ∃ ε > , ( k θ k l − θ ♯ k < ε ⇒ k g ( θ k l ) − g ( θ ♯ ) k ≥ c k θ k l − θ ♯ k ) . (2)This contradicts the assumption that g ( θ k l ) = g ( θ ) = g ( θ ♯ ). Hence, ( θ k l ) l ∈ N converges to θ ♯ after finitely many iterations, and ∀ l ≥ l , θ k l = θ ♯ . This alsoimplies that there are infinitely many k such that θ k = θ ♯ .Let k , k ′ be such that k < k ′ and θ k = θ k ′ = θ ♯ . We conclude our analysisby showing that x ( T k +1 ) = x ∗ . To see this, we recall our parameter tuning andcontrol mechanism. At the beginning of the k -th block, the parameter estimate4s updated from θ k − to θ k to satisfy g k ( θ k ) = g k ( θ ♯ ) = g k ( θ ). The predictedtrajectory based on θ k is g k +1 ( θ k ), and the control input is determined to makethe last n components of g k +1 ( θ k ), ϕ ( T k +1 , T k , x ( T k ); u, θ k ), identical to thetarget state x ∗ . At this state, superficially, it is not assured that g k +1 ( θ k ) = g k +1 ( θ ), where g k +1 ( θ ) corresponds to the actual trajectory. However, g k ′ ( θ k ′ ) = g k ′ ( θ ♯ ) = g k ′ ( θ ), and g k +1 ( θ k ′ ) = g k +1 ( θ ♯ ) = g k +1 ( θ ) is its initial segment.Therefore, ϕ ( T k +1 , T k , x ( T k ); u, θ k ) = ϕ ( T k +1 , T k , x ( T k ); u, θ ) = x ∗ . This is acontradiction because we have supposed that the algorithm never terminatesafter finitely many iterations. (cid:3) Next, we consider the case where solutions to nonlinear equations may con-tain numerical errors, that is, a numerical solution to a nonlinear equation g ( θ ) = 0 (we temporally denote it by b θ ) satisfies k g ( b θ ) − g ( θ ) k ≤ ε for some ε >
0, but the size of ε may be arbitrarily specified by a numerical nonlinearequation solver — generally, such specification is possible by adequately tuningthe termination condition of the solver, as far as the CPU time permits it.In Algorithm 1, where we have assumed exact solutions to nonlinear equa-tions, there has been no limitation on the length of the blocks and the amplitudeof the inputs. They may be arbitrary, and the “exact solution” assumption ab-sorbs all of their effect. In contrast, for inexact solution cases, they should beupper-bounded by some constant. The existence of the upper bound (and hence,feasibility) is assured by the following lemma, which is a variant of Lemma 2in [17]. Lemma 1
Under Assumption 1, for a fixed x , the length of the control blockand the amplitude of the control inputs that drive x into the target state x ∗ are uniformly bounded for all admissible parameters in Ω θ in the followingsense: ∃ N x > , ∃ ρ x > , ∀ θ ∈ Ω θ , ∃ N ≤ N x , ∃ u [0 , N − ∈ B ∞ (0 , ρ x ) , ϕ ( N, , x ; u, θ ) = x ∗ . Proof
The proof is similar to that of Lemma 2 of [17] and hence, it is omitted. (cid:3)
Henceforth, we assume the following.
Assumption 3
For each x , N x and ρ x are known a priori. Our algorithm based on inexact numerical solution also applies the excitationsignal of Assumption 2 to the system (1) at the beginning of the first controlblock. We have not yet described the algorithm itself, but the function g ( θ ) ofAlgorithm 1 is independent of the algorithm and hence is already determined.In the proof of Theorem 1, we have used the fact that for a fixed θ ♯ , (2) holdsbecause rank ∂g ∂θ = n θ . Our inexact numerical solution counterpart requires its“uniform counterpart. ” Lemma 2 ∃ ε g > , ∃ c g > , ∀ θ , θ ∈ Ω θ , k θ − θ k ≤ ε g ⇒ k g ( θ ) − g ( θ ) k ≥ c g k θ − θ k . roof The proof is by contradiction. Suppose that ∀ ε > ∀ c > ∃ θ , θ ∈ Ω θ , k θ − θ k < ε and k g ( θ ) − g ( θ ) k < c k θ − θ k . (3)Let Jg = ∂g ∂θ and λ min = min {k ( Jg )( θ ) v k : v ∈ R n θ , k v k = 1; θ ∈ Ω θ } .Because Jg is continuous and of full rank, λ min >
0. Let c = λ min , andlet ( θ ( k ) , θ ( k )) be the pair in Ω θ that satisfies (3) for ε = 1 /k . Because(3) does not include equality, θ ( k ) = θ ( k ). Because Ω θ × Ω θ is compact,( θ ( k ) , θ ( k )) k ∈ N has an accumulation point ( θ ♯ , θ ♯ ). By Taylor’s formula andthe assumption that Ω θ is convex, g ( θ ) − g ( θ ) = ( Rg )( θ , θ , p )( θ − θ ),where p = ( p , . . . , p n θ ) ∈ Q n θ [0 ,
1] and( Rg )( θ , θ , p ) = ∂g , ∂θ (cid:12)(cid:12)(cid:12) p θ +(1 − p ) θ . . . ∂g ,nθ ∂θ (cid:12)(cid:12)(cid:12) p nθ θ +(1 − p nθ ) θ . Because Rg is continuous and its domain is compact, it is uniformly continuous,and ( Rg )( θ ♯ , θ ♯ , p ) = ( Jg )( θ ♯ ). Therefore, ∀ ε > ∃ δ > i =1 , {k θ i − θ ♯ k} < δ ⇒ k ( Rg )( θ , θ , p ) − ( Jg )( θ ♯ ) k < ε. (4)Let ( θ ( k ) , θ ( k )) be the pair that satisfies (4) for ε < λ min . Then, since g ( θ ( k )) − g ( θ ( k )) = ( Jg )( θ ♯ )( θ ( k ) − θ ( k ))+ (( Rg )( θ , θ , p ) − ( Jg )( θ ♯ ))( θ ( k ) − θ ( k )) , it follows that k g ( θ ( k )) − g ( θ ( k )) k ≥ λ min k θ ( k ) − θ ( k ) k , contradicting (3). (cid:3) Now, we describe the algorithm. In our algorithm, the numerical error ofthe solutions of nonlinear equations are treated by a method that is similar tothe trust-region method of nonlinear programming [21].
Algorithm 2(Initialization)
Given x (0), choose a constant β (0 < β < µ > κ > ε fin >
0. Let u [0 , N −
1] be the excitation signal, T = 0, T = N , k = 1;Apply u [0 , N −
1] to the system (1) to obtain x ( T ). (Loop)if k x ( T k ) k < ε fin thenbreak ; else µ = βµ k − ; κ k = κ k − β ; while do Obtain θ k ∈ Ω θ that satisfy k g k ( θ k ) − g k ( θ ) k < µ ;Obtain u [ T k , T k + N k −
1] that satisfy:6 N k ≤ N x ( T k ) , • u [ T k , T k + N k − ∈ B ∞ (0 , ρ x ( T k ) ), • k h k ( θ k ) − x ∗ k < ε fin ; if h k ( B ( θ k , κ k µ )) ⊂ B ( h k ( θ k ) , ε fin ) then µ k = µ ; break ; else µ = βµ ; T k +1 = T k + N k ;Apply u [ T k , T k + N k −
1] to system (1) to obtain x ( T k +1 ). k = k + 1; Theorem 2
Under Assumptions 1, 2, and 3, Algorithm 2 terminates af-ter finitely many iterations, and the state of (1) reaches to the neighborhood B ( x ∗ , ε fin ) of the target state x ∗ . Proof
We first prove that Algorithm 2 is feasible. Assumptions 1 and 3 makeall steps inside the while loop feasible, except for the condition h k ( B ( θ k , κ k µ )) ⊂ B ( h k ( θ k ) , ε fin . (5)The analysis of h k ( θ k ) needs some care, because it is the abbreviation of thefunction ϕ ( T k + N k , T k , x ( T k ); u, θ k ). However, since N k is bounded by N x ( T k ) ,the amplitude of u is bounded by ρ x ( T k ) , and ϕ is C , for a positive constant c ( x ( N k )) that depends on x ( N k ), k h k ( θ ) − h k ( θ ′ ) k ≤ c ( x ( N k )) k θ − θ ′ k for all θ, θ ′ ∈ Ω θ . If (5) fails, the minor loop of the while loop of Algorithm 2 makes µ = βµ , 0 < β <
1. Thus, residually, µ < ε fin κ k max { ,c ( x ( N k )) } , and (5) is fulfilled.Next, we prove by contradiction that Algorithm 2 terminates after finitelymany steps. Suppose that the termination condition of the (Loop) part ofAlgorithm 2 is never fulfilled. Then, an infinite sequence of the parameterestimate ( θ k ) k ∈ N is obtained. In this case, the fourth line of (Loop) makes µ k ≤ βµ k − (in fact, with the iteration of the while loop, µ k = β d k − µ k − forsome d k − ≥
1. ) Thus, the sequence ( µ k ) k ∈ N converges to zero. Contrary,by the execution of the fifth line of (Loop), κ k = κ k − β , and κ >
0; hence thesequence ( κ k ) k ∈ N diverges to infinity. Because θ k ∈ Ω θ for each k and Ω θ iscompact, the sequence ( θ k ) k ∈ N has accumulation points in Ω θ . Let L be the setof all accumulation points of ( θ k ) k ∈ N . For q ∈ L , there is a subsequence ( θ k l ) l ∈ N that converges to q . For all j , ∃ k l ≥ j , and because g j is an initial segment of g k l for j ≤ k l , k g j ( θ k l ) − g j ( θ )) k ≤ k g k l ( θ k l ) − g k l ( θ ) k ≤ µ k l . Because µ k l converges to zero and g j is continuous, lim l →∞ k g j ( θ k l ) − g j ( θ )) k = k g j ( q ) − g j ( θ ) k = 0. Hence, ∀ q ∈ L, ∀ j, g j ( q ) = g j ( θ ) . (6)7ext, let L ( ε ) = ∪ q ∈ L B ( q, ε ) for some ε >
0. Then, we can show that ∀ ε, ∃ k a , ∀ k ≥ k a , θ k ∈ L ( ε ) . (7)To see this, let us suppose contrary: ∃ ε, ∀ k a , ∃ k ≥ k a , θ k L ( ε ). Then, ( θ k ) k ∈ N has an accumulation point in Ω θ \ L ( ε ), contradicting the assumption that L is the set of all accumulation points. Let ε g and c g be constants definedin Lemma 2. Choose a k a that satisfies (7) for ε = ε g . Because ( κ k ) k ∈ N diverges to infinity, ∃ k b , ∀ k ≥ k b , κ k ≥ c g . Let k ≥ max { k a , k b } . Let q k =arg min q ∈ L k θ k − q k . Then, k θ k − q k k < ε g . Hence, by Lemma 2, k θ k − q k k ≤ c g k g ( θ k ) − g ( q k ) k , and because κ k ≥ c g , k θ k − q k k ≤ κ k k g ( θ k ) − g ( q k ) k .Moreover, by (6), g ( q k ) = g ( θ ), and with the first execution of the statementinside the while loop of Algorithm 2, k g ( θ k ) − g ( θ ) k ≤ k g k ( θ k ) − g k ( θ ) k ≤ µ k .Thus, k θ k − q k k ≤ κ k µ k , and hence, h k ( q k ) ⊂ B ( h k ( θ k ) , ε fin ). Because h k ( θ k ) ∈ B ( x ∗ , ε fin ), h k ( q k ) ∈ B ( x ∗ , ε fin ), which shows that the termination condition of(Loop) has already been fulfilled at the k + 1-th step; a contradiction. (cid:3) In this paper, the finite-time adaptive set-point regulation problem for discrete-time nonlinear systems with parametric uncertainty has been solved under theassumption that the target state is reachable provided that the parameter isknown and an excitation signal is available.The proposed controller has a pathological structure that all history of thepast state is preserved until the state reaches to the target state. Moreover,in Algorithm2, it is not easy to obtain estimates of N x ( T k ) and ρ x ( T k ) . Hence,the proposed algorithms are computationally extremely demanding and by nomeans practical. They should be regarded as being of purely theoretical andconceptual nature. On the other hand, in order for the proposed algorithms to beapplicable, except for the reachability to the target state, no additional conditionis required. This contrasts to the majority of existing methods of nonlinearadaptive control, where many structural conditions are required in order forthose methods to be applicable. Thus, the implication of this paper is to showthe potential of the concept of nonlinear adaptive control in the sense that noextra condition other than the reachability to the target state is required in orderto construct a stable nonlinear adaptive controller provided that sufficiently fastand reliable nonlinear minimizer or nonlinear equation solver is available. In thisrespect, nonlinear adaptive control problem is reduced to nonlinear optimizationproblem. An extra bonus of the proposed methodology is that it is completely“Lyapunov-free.”From a practical point of view, it is desirable to develop a more down-to-earth algorithm that has less computational complexity but does not necessitateextra conditions other than the reachability to the target state. It is also to benoted that the proposed algorithms have the drawback that they are not robustagainst disturbances. To overcoming these problems is left for further research.8 eferenceseferences