Parameter Estimation in Adaptive Control of Time-Varying Systems Under a Range of Excitation Conditions
Joseph E. Gaudio, Anuradha M. Annaswamy, Eugene Lavretsky, Michael A. Bolender
11 Parameter Estimation in Adaptive Control of Time-Varying SystemsUnder a Range of Excitation Conditions
Joseph E. Gaudio, Anuradha M. Annaswamy, Eugene Lavretsky, and Michael A. Bolender
Abstract — This paper presents a new parameter estimation al-gorithm for the adaptive control of a class of time-varying plants.The main feature of this algorithm is a matrix of time-varying learn-ing rates, which enables parameter estimation error trajectories totend exponentially fast towards a compact set whenever excitationconditions are satisfied. This algorithm is employed in a largeclass of problems where unknown parameters are present andare time-varying. It is shown that this algorithm guarantees globalboundedness of the state and parameter errors of the system,and avoids an often used filtering approach for constructing keyregressor signals. In addition, intervals of time over which theseerrors tend exponentially fast toward a compact set are provided,both in the presence of finite and persistent excitation. A projectionoperator is used to ensure the boundedness of the learning ratematrix, as compared to a time-varying forgetting factor. Numericalsimulations are provided to complement the theoretical analysis.
Index Terms — Adaptive control, time-varying learningrates, finite excitation, parameter convergence.
I. I
NTRODUCTION
Adaptive control is a well established sub-field of control whichcompensates for parametric uncertainties that occur online so as tolead to regulation and tracking [1]–[5]. This is accomplished byconstructing estimates of the uncertainties in real-time and ensuringthat the closed loop system is well behaved even when these un-certainties are learned imperfectly. Both in the adaptive control andsystem identification literature, numerous tools for ensuring that theparameter estimates converge to their true values have been derivedover the past four decades [6]–[9]. While most of the current literaturein these two topics makes an assumption that the unknown parametersare constants, the desired problem statement involves plants wherethe unknown parameters are varying with time. This paper proposesa new algorithm for such plants.Parameter convergence in adaptive systems requires a necessaryand sufficient condition, denoted as persistent excitation, whichensures that the convergence is uniform in time [10]–[12]. If instead,a weaker condition is enforced where the excitation holds only over afinite interval, then parameter errors decrease only over a finite time.It is therefore of interest to achieve a fast rate of convergence byleveraging any excitation that may be available so that the parameterestimation error remains as small as possible, even in the presence oftime-variations. The algorithm proposed in this paper will be shown tolead to such a fast convergence under a range of excitation conditions.
This work was supported by the Air Force Research Laboratory, Col-laborative Research and Development for Innovative Aerospace Lead-ership (CRDInAL), Thrust 3 - Control Automation and Mechanizationgrant FA 8650-16-C-2642 and the Boeing Strategic University Initiative. (Corresponding author: Joseph E. Gaudio.)
J.E. Gaudio and A.M. Annaswamy are with the Department ofMechanical Engineering, Massachusetts Institute of Technology, Cam-bridge, MA, 02139 USA (email: [email protected] and [email protected]).E. Lavretsky is with The Boeing Company, Huntington Beach, CA,92647 USA (email: [email protected]).M.A. Bolender is with the Air Force Research Laboratory, WPAFB,OH, 45433 USA (email: [email protected]).
The underlying structure in many of the adaptive identification andcontrol problems consists of a linear regression relation between twodominant errors in the system [4], [13], [14]. Examples include adap-tive observers [15]–[19] and certain classes of adaptive controllers [1].The underlying algebraic relation is often leveraged in order to lead toa fast convergence through the introduction of a time-varying learningrate in the parameter estimation algorithm, which leads to the well-known recursive least squares algorithm [20]. Together with the use ofan outer product of the underlying regressor, a matrix of time-varyinglearning rates is often adjusted to enable fast convergence [15], [18],[21]–[25]. In many cases, however, additional dynamics are presentin the underlying error model that relates the two dominant errors,which prevents the derivation of the corresponding algorithm andtherefore a fast convergence of the parameter estimates. To overcomethis roadblock, filtering has been proposed in the literature [21]–[23],[26]–[30]. This in turn leads to an algebraic regression, using whichcorresponding adaptive algorithms are derived in [15], [18], [21]–[25]with time-varying learning rates become applicable. In [26], [27],it is shown that parameter convergence can occur even with finiteexcitation for a class of adaptive control architectures considered in[28]–[30]. In all of these papers, the underlying unknown parametersare assumed to be constants. The disadvantage of such a filteringapproach is that the convergence properties cannot be easily extendedto the case when the unknown parameters are time-varying, as thefiltering renders the problem intractable. The algorithm that wepropose in this paper introduces no filtering of system dynamics,and is directly applied to the original error model with the dynamicsintact. As such, we are able to establish conditions for fast decreasesof errors, even in the presence of time-varying parameters undervaried properties of excitation.In [31], [32], the problem of parameter estimation has beentackled in the presence of time-varying parameters using a concurrentlearning approach. It is assumed in these papers however, that statederivatives from previous time instances are available. In [33], thisassumption is removed and an integration based method is usedtogether with the same filtering approach mentioned above. Theunderlying parameters are assumed to be constant, and thus theapproach in [33] becomes intractable when these parameters vary.This paper focuses on the ultimate goal of all adaptive control andidentification problems, which is to provide a tractable parameterestimation algorithm for problems where the unknown parametersare time-varying. We will derive such an algorithm that guarantees,in the presence of time-varying parameters, 1) exponentially fasttending of parameter errors and tracking errors to a compact set for arange of excitation conditions, 2) does not require filtering of systemdynamics, and 3) is applicable to a large class of adaptive systems.An error model approach as in [1], [34], [35] is adopted to describethe underlying class. The algorithm consists of time-varying learningrates in order to guarantee fast parameter convergence. Rather thanuse a forgetting factor, continuous projection algorithms are employedin order to ensure that the learning rates are bounded. Additionally,fewer number of integrations are required to implement the algorithmas compared to the existing literature [26], [27], [31]–[33]. a r X i v : . [ m a t h . O C ] J a n This paper proceeds as follows: Section II presents mathematicalpreliminaries regarding continuous projection-based operators anddefinitions of persistent and finite excitation. The underlying problemis introduced in Section III. The main algorithm with time-varyinglearning rates is presented in Section IV. Stability and convergenceproperties of this algorithm are established for a range of excitationconditions in Section V. Computational comparisons with existingadaptive controllers are provided alongside numerical simulations inSection VI. Concluding remarks follow in Section VII.
II. P
RELIMINARIES
In this paper we use (cid:107)·(cid:107) to represent the 2-norm. Definitions, keylemmas, and properties of the Projection Operator (c.f. [4], [5], [36]–[39]) are all presented in this section. Proofs of all lemmas can befound in the appendix, and omitted where it is straightforward.We begin with a few definitions and properties of convex sets andconvex, coercive functions.
Definition 1 ([36]):
A set E ⊂ R N is convex if λx +(1 − λ ) y ∈ E for all x ∈ E , y ∈ E , and ≤ λ ≤ . Definition 2 ([36]):
A function f : R N → R is convex if f ( λx + (1 − λy )) ≤ λf ( x ) + (1 − λ ) f ( y ) for all ≤ λ ≤ . Definition 3 ([36]):
A function f ( x ) : R N → R is said to becoercive if for all sequences { x k } , k ∈ N with (cid:107) x k (cid:107) → ∞ then lim k →∞ f ( x k ) = ∞ . Lemma 1 ([39]):
For a convex function f ( x ) : R N → R and anyconstant δ > , the subset Ξ δ = { x ∈ R N | f ( x ) ≤ δ } is convex. Lemma 2:
For a coercive function f ( x ) : R N → R and anyconstant δ > , any nonempty subset Ξ δ = { x ∈ R N | f ( x ) ≤ δ } is bounded. Corollary 1:
For a coercive, convex function f ( x ) : R N → R anda constant δ > , any nonempty subset Ξ δ = { x ∈ R N | f ( x ) ≤ δ } is convex and bounded. Remark 1:
Definitions 1, 2, 3, Lemmas 1, 2, and Corollary 1 holdby simple extension to functions of matrices i.e., f : R N × N → R . Lemma 3 ([39]):
For a continuously differentiable convex func-tion f ( x ) : R N → R and any constant δ > , let θ ∗ be aninterior point of the subset Ξ δ = { x ∈ R N | f ( x ) ≤ δ } , i.e. f ( θ ∗ ) < δ , and let a boundary point θ be such that f ( θ ) = δ .Then ( θ − θ ∗ ) T ∇ f ( θ ) ≥ .We now present properties related to projection operators. Whilesome of these properties have been previously reported (c.f. [4], [5],[37]–[39]), they are included here for the sake of completeness andto help discuss the main result of this paper. Definition 4 ([39]):
The Γ -projection operator for general matri-ces is defined as,Proj Γ ( θ, Y, F ) = (cid:2) Proj Γ ( θ , y , f ) · · · Proj Γ ( θ m , y m , f m ) (cid:3) (1)where θ = [ θ , . . . , θ m ] ∈ R N × m , Y = [ y , . . . , y m ] ∈ R N × m , F = [ f , . . . , f m ] T ∈ R m , f j : R N → R are convex continuouslydifferentiable functions, < Γ = Γ T ∈ R N × N is a symmetricpositive definite matrix and ∀ j ∈ , . . . , m ,Proj Γ ( θ j , y j , f j ) = Γ y j − Γ ∇ f j ( θ j )( ∇ f j ( θ j )) T ( ∇ f j ( θ j )) T Γ ∇ f j ( θ j ) Γ y j f j ( θ j ) ,f j ( θ j ) > ∧ y Tj Γ ∇ f j ( θ j ) > y j , otherwise (2) Definition 5:
The projection operator for positive definite matricesis defined as,Proj (Γ , Y , F ) = (cid:40) Y − F (Γ) Y , F (Γ) > ∧ T r (cid:104) Y T ∇F (Γ) (cid:105) > Y , otherwise (3)where < Γ = Γ T ∈ R N × N , Y ∈ R N × N and F : R N × N → R is a convex continuously differentiable function. ∇ f ( θ ) y Proj ( θ, y, f ) θ ∗ { θ | f ( θ ) = 1 }{ θ | f ( θ ) = 0 } Ξ Fig. 1 : (adapted from [39]) Γ -Projection operator in R with Γ = I .Uncertain parameter θ ∗ ( t ) ∈ Ξ = { θ ∗ ∈ R | f ( θ ∗ ) ≤ } . Remark 2:
The projection operator in (3) may be expressed morecompactly as Proj (Γ , Y , F ) = ρ ( t ) Y where ρ ( t ) = (cid:40) (1 − F (Γ)) , F (Γ) > ∧ T r (cid:104) Y T ∇F (Γ) (cid:105) > , otherwise (4)From (3), (4), and as displayed in Figure 1, it can be seen that ρ ( t ) =1 on the inside of the projection boundary and ρ ( t ) = 0 on the outsideedge of the boundary if T r (cid:104) Y T ∇F (Γ) (cid:105) > . Remark 3:
An example of a coercive, continuously differentiableconvex function commonly used in projection for adaptive control isgiven by [39] f ( θ ) = (cid:107) θ (cid:107) − θ ∗ max εθ ∗ max + ε (5)where θ ∗ max and ε are positive scalars. It is easy to see that f ( θ ) = 0 when (cid:107) θ (cid:107) = θ ∗ max and f ( θ ) = 1 when (cid:107) θ (cid:107) = θ ∗ max + ε . Thisfunction is commonly used in a projection-based parameter updatelaw to result in a bounded parameter estimate (proven in this paperin Lemma 8). It should be noted that numerous choices other thanthe one in (5) exist for f . Lemma 4:
Let θ = [ θ , . . . , θ m ] ∈ R N × m , θ ∗ = [ θ ∗ , . . . , θ ∗ m ] ∈ R N × m , Y = [ y , . . . , y m ] ∈ R N × m , F = [ f , . . . , f m ] T ∈ R m , where f j : R N → R are convex continuously differentiablefunctions, < Γ = Γ T ∈ R N × N is a symmetric positive definitematrix, and θ ∗ j ∈ Ξ ,j = { θ ∗ j ∈ R N | f j ( θ ∗ j ) ≤ } ∀ j ∈ , . . . , m ,then T r (cid:104) ( θ − θ ∗ ) T Γ − ( Proj Γ ( θ, Y, F ) − Γ Y ) (cid:105) ≤ . The following lemma lists two key properties related to matrixinversion in the presence of time-variations.
Lemma 5:
For a matrix < Γ( t ) = Γ T ( t ) ∈ R N × N , thefollowing identities hold: • ˙Γ( t ) = − Γ( t ) (cid:104) ddt (cid:16) Γ − ( t ) (cid:17)(cid:105) Γ( t ) • ddt (cid:16) Γ − ( t ) (cid:17) = − Γ − ( t ) ˙Γ( t )Γ − ( t ) A central component of this paper is with regards to excitation ofa regressor for which two definitions are provided.
Definition 6 ([1]):
A bounded function φ : [ t , ∞ ) → R N ispersistently exciting (PE) if there exists T > and α > such that (cid:90) t + Tt φ ( τ ) φ T ( τ ) dτ ≥ αI, ∀ t ≥ t . Definition 7 (adapted from [26], [27]):
A bounded function φ :[ t , ∞ ) → R N is finitely exciting (FE) on an interval [ t , t + T ] if there exists t ≥ t , T > , and α > such that (cid:90) t + Tt φ ( τ ) φ T ( τ ) dτ ≥ αI. In both Definitions 6 and 7, the degree of excitation is given by α . It can be noted that the PE condition in Definition 6 pertains to TABLE I : Adaptive Control Systems with a Common Structure
Name Error Model Y State Feedback MRAC [1] ˙ e = Ae + B ˜ θ T φ − φe T P Be y = e Output Feedback MRAC ˙ e = Ae + B ˜ θ T φ − φe y W ( s ) A.S, SPR [1] e y = Ce Output Feedback MRAC (cid:15) = ˜ θ T ζ − ζ(cid:15)W ( s ) A.S., not SPR [1]Nonlinear Adaptive ˙ e = A z ( e, θ, t ) e − φe Backstepping [3] + ˜ θ T φ ( e, θ, t ) Relative Degree ≤ e y = e a property over a moving window for all t ≥ t , whereas the FEcondition in Definition 7 pertains to a single interval [ t , t + T ] . III. A
DAPTIVE C ONTROL OF A C LASS OF P LANTS WITH T IME -V ARYING P ARAMETERS
Large classes of problems in adaptive identification and controlcan be represented in the form of differential equations containingtwo errors, e ( t ) ∈ R n and ˜ θ ( t ) ∈ R N × m . The first is an errorthat represents an identification error or tracking error. The secondis the underlying parameter error, either in estimation of the plantparameter or the control parameter. The parameter error is commonlyexpressed as the difference between a parameter estimate θ and thetrue unknown value θ ∗ as ˜ θ ( t ) = θ ( t ) − θ ∗ ( t ) . The differentialequations which govern the evolution of e ( t ) with ˜ θ ( t ) are referredto as error models [1], [34], [35], and provide insight into how stableadaptive laws for adjusting the parameter error can be designed for alarge class of adaptive systems. The class of error models we focuson in this paper is of the form ˙ e ( t ) = g ( e ( t ) , φ ( t ) , θ ( t ) , θ ∗ ( t )) e y ( t ) = g ( e ( t ) , φ ( t ) , θ ( t ) , θ ∗ ( t )) (6)where the regressor φ ( t ) ∈ R N and e y ( t ) ∈ R p is a measurableerror at each t . The corresponding adaptive law for adjusting ˜ θ isassumed to be of the form ˙˜ θ ( t ) = Γ Y ( e y ( t ) , φ ( t ) , θ ( t )) , (7)where Y is a known function that is implementable at each t and Γ ∈ R N × N is a symmetric positive definite matrix referred to asthe learning rate. In addition, for a given g and g , Y is chosenso that e ( t ) = 0 , ˜ θ ( t ) = 0 is an equilibrium point of the system.Assuming that θ ∗ is a constant, the law in (7) can be written as ˙ θ ( t ) = Γ Y ( e y ( t ) , φ ( t ) , θ ( t )) . (8)We consider all classes of adaptive systems that can be expressed inthe form of (6) and (8) where g , g , Y , and Γ are such that allsolutions are bounded, and where lim t →∞ e ( t ) = 0 . In particular,we assume that g , g , and Y are such that a quadratic Lyapunovfunction candidate V ( t ) = e T ( t ) P e ( t ) + T r (cid:104) ˜ θ T ( t )Γ − ˜ θ ( t ) (cid:105) , (9)yields a derivative for the case of constant θ ∗ as ˙ V ( t ) ≤ − e T ( t ) Qe ( t ) − T r (cid:104) ˜ θ T ( t ) Y ( t ) (cid:105) + 2 T r (cid:104) ˜ θ T ( t )Γ − ˙ θ ( t ) (cid:105) , (10)where P and Q are symmetric positive definite matrices. Due tothe choice of the adaptive law in (8), it follows therefore ˙ V ( t ) ≤ − e T ( t ) Qe ( t ) . Further conditions on g , g , and Y guarantee that e ( t ) → as t → ∞ . We formalize this assumption below: Assumption 1 (Class of adaptive systems):
For the case of a con-stant unknown parameter ( ˙ θ ∗ ( t ) = 0 ), the error model in (6) and theadaptive law in (8) are such that they admit a Lyapunov function V as in (9) which satisfies the inequality in (10).Several adaptive systems that have been discussed in the litera-ture satisfy Assumption 1, some examples of which are shown inTable I. They include plants where state feedback is possible andcertain matching conditions are satisfied, and where only outputs areaccessible and a strictly positive real transfer function W ( s ) canbe shown to exist. For a SISO plant that is minimum phase, thesame assumption can be shown to hold as well. Finally, for a classof nonlinear plants, where the underlying relative degree does notexceed two, Assumption 1 once again can be shown to be satisfied. A. Problem Formulation
The class of error models we consider is of the form (6), where θ ∗ ( t ) , the time-varying unknown parameter, is such that if θ ( t ) ≡ θ ∗ ( t ) , then the solutions of (6) are globally bounded, with e ( t ) = 0 remaining an equilibrium point. This is formalized in the followingassumption: Assumption 2 (Uncertainty variation):
The uncertainty, θ ∗ ( t ) , in(6) is such that || θ ∗ ( t ) || ≤ θ ∗ max , ∀ t ≥ t . In addition, its timederivative, ˙ θ ∗ ( t ) , is assumed to be bounded, i.e. (cid:107) ˙ θ ∗ ( t ) (cid:107) ≤ θ ∗ d,max , ∀ t ≥ t . Furthermore, if θ ( t ) ≡ θ ∗ ( t ) , then the solutions of (6) areglobally bounded, with an equilibrium point at e ( t ) = 0 .The problem that we address in this paper is the determinationof an adaptive law similar to (8) for all error models of the form(6) where Assumptions 1 and 2 hold. Our goal is to ensure globalboundedness of solutions of (6) and exponentially fast tending ofboth e ( t ) and ˜ θ ( t ) to a compact set with finite excitation. IV. A
DAPTIVE L AW WITH A T IME -V ARYING L EARNING R ATE
The adaptive law that we propose is a modification of (8) with a time-varying learning rate Γ( t ) as ˙ θ ( t ) = Γ( t ) Y ( e y ( t ) , φ ( t ) , θ ( t )) .To ensure a bounded Γ( t ) , we include a projection operator in thisadaptive law which is stated compactly as ˙ θ ( t ) = Proj Γ( t ) ( θ ( t ) , Y ( t ) , F ) , θ ( t ) ∈ Ξ . (11)where Proj Γ( t ) ( · , · , · ) is defined as in Definition 4. The Γ -projectionoperator in (11) uses F = [ f , . . . , f m ] T ∈ R m , where f j ( θ j ) : R N → R are coercive, continuously differentiable convex functions.Define the subsets Ξ δ,j = { θ j ∈ R N | f j ( θ j ) ≤ δ } , ∀ j ∈ , . . . , m ,and Ξ δ = { M ∈ R N × m | M j ∈ Ξ δ,j , ∀ j ∈ , . . . , m } . ViaAssumption 2, each f j are chosen such that || θ ∗ ( t ) || ≤ θ ∗ max and δ = 0 corresponds to θ ∗ j ( t ) ∈ Ξ ,j , ∀ j ∈ , . . . , m , ∀ t ≥ t .The time-varying learning rate Γ( t ) is adjusted using the projectionoperator for positive definite matrices (see Definition 5) as ˙Γ( t ) = λ Γ Proj (Γ( t ) , Y ( t ) , F ) , Γ( t ) = Γ t , Y ( t ) = Γ( t ) − κ Γ( t )Ω( t )Γ( t ) , (12)where λ Γ , κ are positive scalars and Ω( t ) ∈ R N × N . Γ t is asymmetric positive definite constant matrix chosen so that Γ t ∈ Υ = { Γ ∈ R N × N | F (Γ) ≤ } , where F (Γ) : R N × N → R is a coercive, continuously differentiable convex function. Lemma 2implies there exists a constant Γ max > such that (cid:107) Γ (cid:107) ≤ Γ max for all Γ ∈ Υ . We assume that F is chosen so that F (Γ) = 1 forall (cid:107) Γ (cid:107) = Γ max . It should be noted that a large Ω( t ) contributes toa decrease in Γ( t ) . TABLE II : Norm of Signals in Phases of Excitation Propagation t ∈ [ t , t ] t [ t , t ] [ t , t ] (cid:107) φ ( t ) φ T ( t ) (cid:107) ≥ (cid:107) (cid:82) t t φ ( τ ) φ T ( τ ) dτ (cid:107) ≥ α (cid:107) Ω( t ) (cid:107) ≥ Ω FE (cid:107) Γ( t ) (cid:107) ≤ Γ FE ρ ( t ) ≥ ρ Finally the matrix Ω( t ) is adjusted as ˙Ω( t ) = − λ Ω Ω( t ) + λ Ω φ ( t ) φ T ( t )1 + φ T ( t ) φ ( t ) , Ω( t ) = Ω (13)where Ω is a symmetric positive semi-definite matrix with ≤ Ω ≤ I and denotes a filtered normalized regressor matrix. λ Γ and λ Ω are arbitrary positive scalars and κ is chosen so that κ > Γ − max .These scalars represent various weights of the proposed algorithm.The main contribution of this paper is the adaptive law in (11),(12), and (13), which will be shown to result in bounded solutionsin Section V which tend exponentially fast to a compact set if φ ( t ) is finitely exciting. If in addition, φ ( t ) is persistently exciting,exponentially fast convergence to a compact set will occur ∀ t ≥ t . Remark 4:
It should be noted that while different aspects of thealgorithm in (11), (12), and (13) have been explored in the literature,a combined algorithm as presented and analyzed here has not beenreported thus far. For example, filtered regressor outer products areconsidered in [25]–[27], but parameters are assumed to be constant.Projection-based update laws have additionally been considered in[1]–[5]. It is the fact that we have the use of Γ in (12) which isadjusted with Ω , the fact that we are using projections to contain Γ within a bounded set, and that we are using a filtered version of φφ T together with normalization to adjust Ω as in (13) that enables ourproposed algorithm to have desirable convergence properties, over arange of excitation conditions. Remark 5:
The projection operator employed in (12) is onemethod to bound the time-varying learning rate. Instead of (12),one can also use a time-varying forgetting factor to provide for (cid:107) Γ( t ) (cid:107) ≤ Γ max of the form ˙Γ( t ) = λ Γ (cid:18) − (cid:107) Γ( t ) (cid:107) Γ max (cid:19) [Γ( t ) − κ Γ( t )Ω( t )Γ( t )] . (14)While the time-varying forgetting factor, (1 − (cid:107) Γ( t ) (cid:107) / Γ max ) , alsoachieves a bounded (cid:107) Γ( t ) (cid:107) , it is more conservative than the projectionoperator in (12) as it is always active. In comparison, the projectionoperator as in (12) only provides limiting action if Γ( t ) is in aspecified boundary region and the direction of evolution of Γ( t ) causes F (Γ) to increase. An equivalence between time-varyingforgetting factors and projection operators may be drawn using thesquare root of the function in (5) with θ ∗ max = 0 , ε = Γ max , andthe limiting action always remaining active. Remark 6:
It can be noted that while the regressor normalizationin (13) is optional for linear regression systems [21], [22], it isrequired for general adaptive control problems in the presence ofsystem dynamics as the regressor cannot be assumed to be bounded.
Remark 7:
From a stability standpoint, the filtering in (13) forlinear time-invariant error dynamics is optional, i.e. φφ T / (1 + φ T φ ) may be used in place of Ω in (12). The inclusion of (13) howeverprovides for a more smooth adjustment of Γ in the presence ofsharp changes in φ ( t ) and enhances finite excitation properties byrestricting all directions of increase of Γ( t ) after a finite excitationand no additional excitation. V. S
TABILITY AND C ONVERGENCE A NALYSIS
We now state and prove the main result. The following assumptionis needed for discussion of a finite excitation. We define an excitationlevel α on an interval [ t , t ] as α = k Ω dκ Γ max ρ Ω λ Ω exp( − λ Ω ( t − t )) (15)where ρ Ω ∈ (0 , , k Ω > , and d = max τ ∈ [ t ,t ] { (cid:107) φ ( τ ) (cid:107) } . Assumption 3 (Finite excitation):
There exists a time t ≥ t anda time t > t such that the regressor φ ( t ) in (6) is finitely excitingover [ t , t ] , with excitation level α ≥ α . A. Propagation of Excitation and Boundedness of InformationMatrix, Time-Varying Learning Rate
We first prove a few important properties of Ω( t ) and Γ( t ) underdifferent excitation conditions. Lemma 6:
For the algorithm in (13), it follows that for any φ ( t ) ,1) Ω( t ) ≥ , ∀ t ≥ t ,2) Ω( t ) ≤ I , ∀ t ≥ t .If in addition φ is finitely exciting as in Assumption 3, then3) Ω( t ) ≥ Ω F E
I > (1 / ( κ Γ max )) I , ∀ t ∈ [ t , t ] ,where Ω F E = ( k Ω / ( κ Γ max )) and t = t − (ln ρ Ω ) /λ Ω . If inaddition φ is persistently exciting ∀ t ≥ t ≥ t (see Definition 6),with interval T and level α ≥ α (cid:48) , t (cid:48) = t + T , then4) Ω( t ) ≥ Ω F E
I > (1 / ( κ Γ max )) I , ∀ t ≥ t (cid:48) , α (cid:48) = α exp( − λ Ω ( t − t (cid:48) )) d (cid:48) /d , d (cid:48) = max τ ≥ t { (cid:107) φ ( τ ) (cid:107) } . Lemma 7:
The solutions of (12) and (4) satisfy the following:1) Γ( t ) ≤ Γ max I , Γ − ( t ) ≥ Γ − max I > , ∀ t ≥ t ,2) ρ ( t ) ∈ [0 , , ∀ t ≥ t ,3) Γ( t ) ≥ Γ min I > , Γ − ( t ) ≤ Γ − min I , ∀ t ≥ t ,where Γ min = 1 / (max eig (Γ − t ) + κ ) . If in addition φ is finitelyexciting as in Assumption 3, then there exists a ρ ∈ (0 , such that4) Γ( t ) ≤ Γ F E
I < Γ max I , Γ − ( t ) ≥ Γ − F E
I > , ∀ t ∈ [ t , t ] ,5) ρ ( t ) ≥ ρ > , ∀ t ∈ [ t , t ] ,where Γ F E = ρ − Γ t , ρ Γ ∈ ((Γ t / Γ max ) , , Γ t = Γ( t ) < Γ max , and t = t − (ln ρ Γ ) /λ Γ . If in addition φ is persistentlyexciting ∀ t ≥ t ≥ t (see Definition 6), with interval T and level α ≥ α (cid:48) , then there exists a ρ ∈ (0 , , t (cid:48) > t (cid:48) , and Γ( t (cid:48) ) ≤ Γ P E < Γ max such that6) Γ( t ) ≤ Γ P E
I < Γ max I , Γ − ( t ) ≥ Γ − P E
I > , ∀ t ≥ t (cid:48) ,7) ρ ( t ) ≥ ρ > , ∀ t ≥ t (cid:48) .The properties of Ω and Γ for a persistently exciting φ are relativelywell known. For a finitely exciting φ , it should be noted that aftera certain time elapses, the lower bound for Ω is realized. Thispropagation is illustrated in Table II.The choice of the finite excitation level α in Assumption 3enables a fast convergence rate as follows: The denominator κ Γ max in α ensures that the update in (12) pushes Γ( t ) away from Γ max , ρ Ω provides for a bound for Ω away from a minimum value, and λ Ω exp( − λ Ω ( t − t )) accounts for excitation propagation through(13). The numerator scaling d accounts for the normalization in (13),and k Ω provides for a bound away from a minimum excitation level. B. Stability and Convergence Analysis
With the properties of the learning rate and filtered regressor above,we now proceed to the main theorem. The following lemma andcorollary state important properties of the parameter estimate θ . Lemma 8:
The update for θ ( t ) in (11) guarantees that there existsa θ max such that (cid:107) θ ( t ) (cid:107) ≤ θ max , ∀ t ≥ t . Corollary 2:
Under Assumption 2, the update for θ ( t ) in (11)provides for a constant ˜ θ max such that (cid:107) ˜ θ ( t ) (cid:107) ≤ ˜ θ max , ∀ t ≥ t .The following definitions are useful for stating the main result inTheorem 1. Define scalars υ ( t ) and η ( t ) as υ ( t ) = λ Γ ρ ( t ) κ (cid:107) Ω( t ) (cid:107)(cid:107) ˜ θ ( t ) (cid:107) + 2Γ − min (cid:107) ˜ θ ( t ) (cid:107)(cid:107) ˙ θ ∗ ( t ) (cid:107) , (16) η ( t ) = min (cid:110) q , λ Γ ρ ( t )Γ − max (cid:111) max (cid:110) p max , Γ − min (cid:111) . (17)It is easy to see that ≤ η ( t ) and ≤ υ ( t ) ≤ υ max , where υ max = λ Γ κ ˜ θ max + 2Γ − min ˜ θ max θ ∗ d,max . (18)Define η as η = min (cid:110) q , λ Γ ρ Γ − max (cid:111) max (cid:110) p max , Γ − min (cid:111) , (19)where ρ ∈ (0 , . Define a compact set D as D = (cid:110)(cid:16) e, ˜ θ (cid:17) (cid:12)(cid:12)(cid:12) η (cid:104) p min (cid:107) e (cid:107) + Γ − max (cid:107) ˜ θ (cid:107) (cid:105) ≤ υ (cid:111) , (20)alongside a corresponding set D max , defined as D max = (cid:110)(cid:16) e, ˜ θ (cid:17) (cid:12)(cid:12)(cid:12) η (cid:104) p min (cid:107) e (cid:107) + Γ − max (cid:107) ˜ θ (cid:107) (cid:105) ≤ υ max (cid:111) . (21)We now state the main theorem of stability and convergence. Theorem 1:
Under Assumptions 1 and 2, the update laws in (11),(12), and (13) for the error model in (6) guarantee for any φ ( t ) ,A) boundedness of the trajectories of e ( t ) and ˜ θ ( t ) , ∀ t ≥ t .If in addition φ is finitely exciting as in Assumption 3, thenB) the trajectories of e ( t ) , ˜ θ ( t ) tend exponentially fast towards acompact set D ⊂ D max , ∀ t ∈ [ t , t ] .If in addition φ is persistently exciting ∀ t ≥ t ≥ t as in Definition6 with level α ≥ α (cid:48) and interval T , thenC) exponential convergence of the trajectories follows, of e ( t ) , ˜ θ ( t ) towards a compact set D ⊂ D max , ∀ t ≥ t (cid:48) . Proof:
Let q = min eig ( Q ) , p min = min eig ( P ) , p max =max eig ( P ) . Consider a candidate Lyapunov function of the form V ( t ) = e T ( t ) P e ( t ) + T r (cid:104) ˜ θ T ( t )Γ − ( t )˜ θ ( t ) (cid:105) . (22)It follows that ˙ V ( t ) ≤ − e T ( t ) Qe ( t ) − T r (cid:104) ˜ θ T ( t ) Y ( t ) (cid:105) + 2 T r (cid:104) ˜ θ T ( t )Γ − ( t ) ˙ θ ( t ) (cid:105)(cid:124) (cid:123)(cid:122) (cid:125) due to Assumption 1 + T r (cid:20) ˜ θ T ( t ) (cid:26) ddt (cid:16) Γ − ( t ) (cid:17)(cid:27) ˜ θ ( t ) (cid:21)(cid:124) (cid:123)(cid:122) (cid:125) due to time-varying Γ( t ) − T r (cid:104) ˜ θ T ( t )Γ − ( t ) ˙ θ ∗ ( t ) (cid:105)(cid:124) (cid:123)(cid:122) (cid:125) due to time-varying θ ∗ ( t ) . Using (11), (12), and Lemma 5, ˙ V ( t ) may be simplified as ˙ V ( t ) ≤ − e T ( t ) Qe ( t ) − T r (cid:104) ˜ θ T ( t )Γ − ( t ) ˙ θ ∗ ( t ) (cid:105) + 2 T r (cid:104) ˜ θ T ( t )Γ − ( t ) (cid:16) Proj Γ( t ) ( θ ( t ) , Y ( t ) , F ) − Γ( t ) Y ( t ) (cid:17)(cid:105) − λ Γ T r (cid:104) ˜ θ T ( t )Γ − ( t ) Proj (Γ( t ) , Y ( t ) , F ) Γ − ( t )˜ θ ( t ) (cid:105) . Using Lemma 4 and ρ ( t ) in (4), we obtain that ˙ V ( t ) ≤ − e T ( t ) Qe ( t ) − T r (cid:104) ˜ θ T ( t )Γ − ( t ) ˙ θ ∗ ( t ) (cid:105) − λ Γ ρ ( t ) T r (cid:104) ˜ θ T ( t ) (cid:110) Γ − ( t ) − κ Ω( t ) (cid:111) ˜ θ ( t ) (cid:105) . (23) Using (16), (17), Corollary 2, and Assumption 2, the inequalitybecomes ˙ V ( t ) ≤ (cid:26) − q (cid:107) e ( t ) (cid:107) + 2Γ − min ˜ θ max θ ∗ d,max , if ρ ( t ) = 0 − η ( t ) V ( t ) + υ ( t ) , if ρ ( t ) ∈ (0 , (24)From the first case of (24) it can be seen that ˙ V ( t ) ≤ for (cid:107) e ( t ) (cid:107) ≥ (cid:113) − min ˜ θ max θ ∗ d,max /q . From Lemmas 6, 7, 8, andCorollary 2, each of Ω( t ) , Γ( t ) , θ ( t ) , θ ∗ ( t ) and ˜ θ ( t ) are bounded.Thus the trajectories of the closed loop system remain bounded. Thisproves Theorem 1-A).From (22) and (24), it can be noted that ˙ V < in D c , where thecompact set D is defined in (20). Applying the Comparison Lemma(see [40], Lemma 3.4) for the second case of (24), we obtain that V ( t ) ≤ Φ( t, t ) V ( t ) + (cid:90) tt Φ( t, τ ) υ ( τ ) dτ, ∀ t ∈ [ t , t ] , (25)with transition function Φ( t, τ ) = exp (cid:104) − (cid:82) tτ η ( τ ) dτ (cid:105) . It can benoted that from Lemma 7 it was shown that ρ ( t ) ≥ ρ , ∀ t ∈ [ t , t ] ,and thus η ( t ) ≥ η , ∀ t ∈ [ t , t ] , which follows from (17), (19).Thus (25) is simplified using (18) as V ( t ) ≤ exp [ − η ( t − t )] (cid:18) V ( t ) − υ max η (cid:19) + υ max η , ∀ t ∈ [ t , t ] . (26)Furthermore given that η ( t ) ≥ η ∀ t ∈ [ t , t ] and ≤ υ ( t ) ≤ υ max , it can be noted that D ⊂ D max , ∀ t ∈ [ t , t ] . Thereforeit can be seen that over the interval of time t ∈ [ t , t ] , the stateerror e ( t ) and parameter error ˜ θ ( t ) tend exponentially fast towardsthe bounded set D ⊂ D max . This proves Theorem 1-B).If φ is persistently exciting with level α ≥ α (cid:48) , and interval T , thenit follows from Lemma 7-7) that (25) and (26) hold for all t ≥ t (cid:48) ,which proves Theorem 1-C). Remark 8: η ( t ) denotes the convergence rate of V ( t ) . This in turnfollows if ρ ( t ) > , i.e. if Γ( t ) is bounded away from Γ max . Thelatter follows from Lemma 6-3) and 6-4) if φ ( t ) is either finitelyexciting or persistently exciting, with an exponentially fast trajectoryof V ( t ) towards a compact set occurring over a finite interval orfor all t ≥ t , respectively. This convergence rate however is upperbounded by q /p max . Remark 9:
Theorem 1-C) guarantees convergence of V ( t ) to acompact set D , while Theorem 1-B) guarantees that V ( t ) approaches D . This set scales with the signal υ ( t ) in (16), which contains contri-butions both from Ω( t ) and ˙ θ ∗ ( t ) . For static parameters ( θ ∗ ( t ) ≡ )and low excitation (i.e., ρ ( t ) (cid:54) = 0 and (cid:107) Ω( t ) (cid:107) < (1 / ( κ Γ max )) ), from(23) it can be shown that the trajectories of ( e, ˜ θ ) , tend towards theorigin, i.e. the set ( e, ˜ θ ) = (0 , . Remark 10:
Since we did not introduce any filtering of the un-derlying signals, the bound θ ∗ d,max on the uncertain parametersis explicit in the compact set D max . It can be seen that D max directly scales with θ ∗ d,max from (18). Such an explicit bound cannotbe derived using existing approaches in the literature which filterdynamics. Remark 11:
The dependence of υ ( t ) on ˙ θ ∗ ( t ) is reasonable. Asthe time-variations in the uncertain parameters grow, it should beexpected that the residue will increase as well. The dependence of υ ( t ) on the filtered regressor Ω( t ) is introduced due to the structureof our algorithm in (11), (12), and (13). As a result, even withpersistent excitation, we can only conclude convergence of V ( t ) toa compact set as opposed to convergence to the origin. This compactset will be present even in the absence of time-variations in θ ∗ .This disadvantage, however, is offset by the property of exponentialconvergence to the compact set, which is virtue of the fact that wehave a time-varying Γ( t ) . A closer examination of the convergence properties of the proposedalgorithm is worth carrying out for the case of constant parameters. Itis clear from (23) that the negative contributions to ˙ V ( t ) come fromthe first term, while any positive contribution comes if κ Ω( t ) > Γ − ( t ) . That is, if there is a large enough excitation, then the thirdterm can be positive. This in turn is conservatively reflected in themagnitude of υ ( t ) . It should however be noted that a large Ω( t ) with persistent excitation, leads to a large e ( t ) , which implies thatas the third term in (23) becomes positive, it leads to a first termthat is proportionately large and negative as well, thereby resultingin a net contribution that is negative. An analytical demonstration ofthis effect, however, is difficult to obtain. For this reason, the natureof our main result is convergence to a bounded set rather than tozero, in the presence of persistent excitation. Finally, we note that inour simulation studies, ˙ V ( t ) remained negative for almost the entireperiod of interest, resulting in a steady convergence of the parameterestimation error to zero. Remark 12:
The bounds for Ω( t ) and Γ( t ) derived in Lemmas 6and 7 were crucial in establishing the lower bound for η ( t ) . That thisoccurs over an interval of time results in exponential and not justasymptotic properties. This has obvious implications of robustness(c.f. [40] §9.1). VI. A
LGORITHM D ISCUSSION AND S IMULATIONS
In this section we analyze the memory and computation require-ments of our proposed algorithm and provide numerical simulationsto demonstrate the algorithm in an illustrative application.
A. Memory Requirement and Computation
The standard parameter update in (8) requires N × m integrationsto adjust the N × m parameters θ . Given that the updates for both Γ and Ω result in symmetric matrices, an additional N ( N + 1) / integrations are required for each update for a total increase of N ( N + 1) integrations.In comparison, the composite approach with finite excitation analy-sis presented in [27] results in an additional n integrations to filter theerror dynamics, N integrations to filter the regressor, N ( N + 1) / integrations to compute a symmetric information matrix, and N integrations to compute an auxiliary matrix; for a total increase of n + 3 N ( N + 1) / integrations. In order to avoid the knowledge ofstate derivatives used in the concurrent learning approach, estimatesof past state derivatives is proposed using smoothing techniqueswith a forward and backward Kalman filter [31], [32]. This howeversignificantly increases the memory and computational requirementscompared to the proposed algorithm. B. Numerical Simulations
In this section we present numerical simulation results for lin-earized F-16 longitudinal dynamics trimmed at a straight and levelflying condition with a velocity of ft/s and an altitude of , ft. We present results for the case of a constant unknown parameter inorder to demonstrate the exponential convergence properties towardsthe origin with finite excitations. We include integral tracking ofcommands thus resulting in an extended state plant model given by ˙ x ( t )˙ x ( t )˙ x ( t ) (cid:124) (cid:123)(cid:122) (cid:125) ˙ x ( t ) = − . . − . − . (cid:124) (cid:123)(cid:122) (cid:125) A x ( t ) x ( t ) x ( t ) (cid:124) (cid:123)(cid:122) (cid:125) x ( t ) + − . − . (cid:124) (cid:123)(cid:122) (cid:125) B u ( t ) + − (cid:124) (cid:123)(cid:122) (cid:125) B z z cmd ( t ) where z cmd is the pitch rate command (dps), u is an elevatordeflection (deg), and the state variables x , x , x are the angleof attack (deg), pitch rate (dps), and integrated pitch rate tracking -50 -50 (a) State and control time histories. -10 -5 -5 -50 (b) Lyapunov function, tracking error norm, parameter error norm andtime-varying learning rate time histories. Fig. 2 : Time histories of numerical simulation.error (deg), respectively. A reference model, representing the desireddynamics, is designed as ˙ˆ x ( t ) = A m ˆ x ( t ) + B z z cmd ( t ) , where A m = A − BK T is Hurwitz, with K = [0 . , − . , − T .Therefore the tracking error dynamics may be expressed in the formof (6), with error e ( t ) = ˆ x ( t ) − x ( t ) , e y ( t ) = e ( t ) , φ ( t ) = x ( t ) ,and the control input selected as u ( t ) = − K T x ( t ) − θ T ( t ) φ ( t ) . Theadaptive parameter estimate θ is initialized at zero, to estimate theunknown parameter θ ∗ = [0 . , − . , T , which representsuncertainty as a function of angle of attack and pitch rate. With theparameter update argument selected as Y ( t ) = − φ ( t ) e T ( t ) P B (asin Table I), Assumption 1 may be verified.For the algorithm in (11), (12), (13), we set λ Γ = 0 . , κ = 0 . , λ Ω = 10 , and find a matrix P which solves A Tm P + P A m = − I .The time-varying learning rate is initialized as Γ( t ) = Γ = 10 I .For the projection algorithms in (11) and (12), we use the convex,coercive continuous function in (5) where the 2-norm is used for the θ update and the Frobenius norm is used for the Γ update.The numerical simulation results in Figure 2a demonstrate com-mand tracking of pitch rate step responses using the standard staticlearning rate update (MRAC) in (8) and the time-varying learningrate update (TR-MRAC) in (11), (12), (13). It can be noted thatwhile step responses contain some frequency content, the regressor φ ( t ) does not satisfy the PE condition in Definition 6 with a large α .This can be further seen in Figure 2b, where the update law with astatic learning rate does not demonstrate significant decreases in anyof the performance variables after the initial transient. The limitedspectral content of the step responses is captured in our algorithmin (13) and held onto for a period of time. The resulting excitationleads to a Γ( t ) as shown in Figure 2b. It can be seen that Γ( t ) has off-diagonal entries, signifying a rotation in learning direction.Every time there is a step in the regressor, Γ( t ) entries decrease,which is immediately followed by an exponentially fast decrease in V ( t ) and its sub-components, as displayed in log-scale in Figure 2b.Furthermore, a reduction in oscillations in the control rate ˙ u can beseen throughout the time history as displayed in Figure 2a, whereasoscillations remain when the learning rate is constant. VII. C
ONCLUDING R EMARKS
In this paper we presented a new parameter estimation algorithmfor the adaptive control of a class of time-varying plants. The mainfeature of this algorithm is a matrix of time-varying learning rates,which enables exponentially fast trajectories of parameter estimationerrors towards a compact set whenever excitation conditions aresatisfied. It is shown that even in the presence of time-varyingparameters, this algorithm guarantees global boundedness of the stateand parameter errors of the system. In addition, it is shown thatthese errors tend exponentially fast towards a compact set over aninterval of time in the presence of finite excitation. In the presenceof persistent excitation, exponential convergence to a compact set isshown. The learning rate matrix is ensured to be bounded throughthe use of a projection operator. Since no filtering is employed andthe original dynamic structure of the system is preserved, the boundsderived are tractable and are clearly related to the bounds on thetime-variations of the unknown parameters as well as the excitationproperties. Numerical simulations were provided to complement thetheoretical analysis. Future work will focus on connecting these time-varying learning rates to accelerated learning in machine learningproblems. P ROOFS OF L EMMAS
Proof of Lemma 1:
Let x , x ∈ Ξ δ and thus f ( x ) ≤ δ and f ( x ) ≤ δ . From the convexity of f , for any ≤ λ ≤ : f ( λx +(1 − λ ) x ) ≤ λf ( x )+(1 − λ ) f ( x ) ≤ λδ +(1 − λ ) δ = δ .Therefore for all x = λx + (1 − λ ) x : f ( x ) ≤ δ , and thus x ∈ Ξ δ .Therefore Ξ δ is a convex set. Proof of Lemma 2:
Suppose there exists a constant δ > suchthat the subset Ξ δ = { x ∈ R N | f ( x ) ≤ δ } is nonempty andunbounded. Thus there exists a sequence { x k | k ∈ N } ∈ Ξ δ suchthat (cid:107) x k (cid:107) → ∞ . From Definition 3, given that f is coercive then lim k →∞ f ( x k ) = ∞ . This contradicts f ( x ) ≤ δ, ∀ x ∈ Ξ δ . Proof of Lemma 3: f ( θ ) is convex, thus for any < λ ≤ : f ( λθ ∗ + (1 − λ ) θ ) = f ( θ + λ ( θ ∗ − θ )) ≤ f ( θ ) + λ ( f ( θ ∗ ) − f ( θ )) .Thus ( θ − θ ∗ ) T ∇ f ( θ ) = lim λ → ( f ( θ ) − f ( θ + λ ( θ ∗ − θ ))) /λ ≥ f ( θ ) − f ( θ ∗ ) ≥ δ − δ = 0 . Proof of Lemma 4:
For each j ∈ , . . . , m , if f j ( θ j ) > ∧ y Tj Γ ∇ f j ( θ j ) > , then using (2) and Lemma 3, T r (cid:104) ( θ − θ ∗ ) T Γ − ( Proj Γ ( θ, Y, F ) − Γ Y ) (cid:105) = m (cid:88) j =1 ( θ j − θ ∗ j ) T Γ − (cid:0) Proj Γ ( θ j , y j , f j ) − Γ y j (cid:1) = − m (cid:88) j =1 ( θ j − θ ∗ j ) T ∇ f j ( θ j ) (cid:124) (cid:123)(cid:122) (cid:125) ≥ ( ∇ f j ( θ j )) T Γ y j (cid:124) (cid:123)(cid:122) (cid:125) > ( ∇ f j ( θ j )) T Γ ∇ f j ( θ j ) (cid:124) (cid:123)(cid:122) (cid:125) > f j ( θ j ) (cid:124) (cid:123)(cid:122) (cid:125) > ≤ otherwise ( θ j − θ ∗ j ) T Γ − (cid:0) Proj Γ ( θ j , y j , f j ) − Γ y j (cid:1) = 0 . Proof of Lemma 5:
It can be noticed that ddt ( I ) = ddt (cid:16) Γ( t )Γ − ( t ) (cid:17) = ˙Γ( t )Γ − ( t ) + Γ( t ) (cid:104) ddt (cid:16) Γ − ( t ) (cid:17)(cid:105) . Lemma 5follows by solving for ˙Γ( t ) and ddt (cid:16) Γ − ( t ) (cid:17) respectively. Proof of Lemma 6:
Let v ∈ R N . Given the initial condition for(13), it can be noted that ≤ v T Ω( t ) v ≤ (cid:107) v (cid:107) . Furthermore ≤ | v T φ ( t ) | (cid:107) φ ( t ) (cid:107) ≤ (cid:107) v (cid:107) , ∀ v, t ≥ t (27)as multiplying through by (cid:107) φ ( t ) (cid:107) ≥ , the lower and upperbounds may be shown as ≤ | v T φ ( t ) | , and | v T φ ( t ) | ≤(cid:107) v (cid:107) (cid:107) φ ( t ) (cid:107) ≤ (cid:107) v (cid:107) + (cid:107) v (cid:107) (cid:107) φ ( t ) (cid:107) , ∀ v, t ≥ t . 1) From the integral update in (13) we obtain v T Ω( t ) v = e − λ Ω ( t − t ) v T Ω( t ) v + (cid:90) tt e − λ Ω ( t − τ ) λ Ω | v T φ ( τ ) | (cid:107) φ ( τ ) (cid:107) dτ. (28)Given that λ Ω > , using (27), all terms in (28) are non-negative,therefore v T Ω( t ) v ≥ , ∀ v, t ≥ t and thus Ω( t ) ≥ , ∀ t ≥ t .2) From the integral update in (13) we obtain v T ( I − Ω( t )) v = v T (cid:16) I − e − λ Ω ( t − t ) Ω( t ) (cid:17) v − (cid:90) tt e − λ Ω ( t − τ ) λ Ω | v T φ ( τ ) | (cid:107) φ ( τ ) (cid:107) dτ. (29)Thus equation (29) may be bounded using λ Ω > and (27) to resultin v T ( I − Ω( t )) v ≥ , ∀ v, t ≥ t and thus Ω( t ) ≤ I , ∀ t ≥ t .3) From (28) we obtain v T Ω( t ) v ≥ e − λ Ω ( t − t ) λ Ω v T (cid:90) t t φ ( τ ) φ T ( τ )1 + (cid:107) φ ( τ ) (cid:107) dτ v. (30)Thus using the finite excitation condition in Assumption 3 (seeDefinition 7), v T Ω( t ) v ≥ v T ( λ Ω α/d ) exp ( − λ Ω ( t − t )) Iv ≥ ( k Ω / ( ρ Ω κ Γ max )) v T Iv . Furthermore from (13): v T Ω( t ) v ≥ exp ( − λ Ω ( t − t )) v T Ω( t ) v , ∀ v, t ≥ t . Thus v T Ω( t ) v ≥ exp ( − λ Ω ( t − t ))( k Ω / ( ρ Ω κ Γ max )) v T Iv , ∀ v, t ≥ t . Therefore Ω( t ) ≥ ( k Ω / ( κ Γ max )) I > (1 / ( κ Γ max )) , ∀ t ∈ [ t , t ] .4) Immediate from extension of the proof of case 3) ∀ t ≥ t (cid:48) . Proof of Lemma 7:
1) The time derivative for F (Γ) in (12) is ˙ F (Γ) = T r [( ∇F (Γ)) T ˙Γ] = λ Γ T r [( Proj (Γ , Y , F )) T ∇F (Γ)] . Withthe projection equation in (3), ˙ F (Γ) = (cid:26) λ Γ T r [ Y T ∇F (Γ)](1 − F (Γ)) , F (Γ) > ∧ T r [ Y T ∇F (Γ)] > λ Γ T r [ Y T ∇F (Γ)] , otherwise Therefore within the limiting region F (Γ) > , ˙ F (Γ) > , < F (Γ) < ∧ T r [ Y T ∇F (Γ)] > F (Γ) = 0 , F (Γ) = 1 ∧ T r [ Y T ∇F (Γ)] > F (Γ) ≤ , T r [ Y T ∇F (Γ)] ≤ (31)Thus given the initial condition for (12), F (Γ( t )) ≤ and therefore F (Γ( t )) ≤ for all t ≥ t . Therefore from Lemma 2, there exists aconstant denoted Γ max such that (cid:107) Γ (cid:107) ≤ Γ max for all Γ ∈ Υ , i.e.all F (Γ) ≤ , ∀ t ≥ t , which implies Γ( t ) ≤ Γ max I , ∀ t ≥ t .2) From (31) and Remark 2, ρ ( t ) ∈ [0 , , ∀ t ≥ t .From (3), (12), Remark 2 and Lemma 5, the inverse of the time-varying learning rate may be expressed as ddt (cid:16) Γ − ( t ) (cid:17) = − λ Γ ρ ( t )Γ − ( t ) + λ Γ ρ ( t ) κ Ω( t ) . (32)Let v ∈ R N . From (32) we obtain v T Γ − ( t ) v = e − λ Γ (cid:82) tt ρ ( τ ) dτ v T Γ − ( t ) v + (cid:90) tt e − λ Γ (cid:82) tτ ρ ( ν ) dν λ Γ ρ ( τ ) κv T Ω( τ ) vdτ. (33)3) Using Ω( t ) ≤ I , ∀ t ≥ t from Lemma 6 and ρ ( t ) ∈ [0 , , ∀ t ≥ t , with equation (33) we obtain v T (cid:16) Γ − min I − Γ − ( t ) (cid:17) v ≥ , ∀ v, t ≥ t and thus Γ( t ) ≥ Γ min I > , ∀ t ≥ t .4) From the excitation lower bound: Ω( t ) ≥ ( k Ω / ( κ Γ max )) I > (1 / ( κ Γ max )) I , ∀ t ∈ [ t , t ] from Lemma 6, it can be noted withequation (12) that Γ max − κ Γ max Ω( t )Γ max ≤ − Γ max ( k Ω − < , ∀ t ∈ [ t , t ] , as k Ω > . Thus even if Γ( t ) were to beat the limit Γ max , the evolution of Γ( t ) is bounded away from Γ max (towards Γ max /k Ω ). Therefore there exists a ρ t ∈ (0 , ,such that ρ ( t ) ≥ ρ t , ∀ t ∈ [ t , t ] , and a Γ t < Γ max , such that Γ( t ) ≤ Γ t , ∀ t ∈ [ t , t ] . Furthermore from (33): v T Γ − ( t ) v ≥ exp ( − λ Γ (cid:82) tt ρ ( τ ) dτ ) v T Γ − ( t ) v , ∀ v, t ≥ t . Thus v T Γ − ( t ) v ≥ exp ( − λ Γ ( t − t ))Γ − t v T Iv , ∀ v, t ≥ t . Therefore Γ − ( t ) ≥ Γ − F E I , ∀ t ∈ [ t , t ] , thus Γ( t ) ≤ Γ F E
I < Γ max , ∀ t ∈ [ t , t ] .5) Given that Γ( t ) ≤ Γ F E
I < Γ max , ∀ t ∈ [ t , t ] , it can beseen that ∃ ρ ∈ (0 , such that ρ ( t ) ≥ ρ > , ∀ t ∈ [ t , t ] .6) Immediate from extension of the proof of case 4) ∀ t ≥ t (cid:48) .7) Immediate from extension of the proof of case 5) ∀ t ≥ t (cid:48) . Proof of Lemma 8:
For each j ∈ , . . . , m , the time derivativefor f j ( θ j ) in (11) may be expressed as ˙ f j ( θ j ) = ( ∇ f j ( θ j )) T ˙ θ j = (cid:0) Proj Γ (cid:0) θ j , y j , f j (cid:1)(cid:1) T ∇ f j ( θ j ) . With the projection equation in (2), ˙ f j ( θ j ) = (cid:40) y Tj Γ ∇ f j ( θ j )(1 − f j ( θ j )) , f j ( θ j ) > ∧ y Tj Γ ∇ f j ( θ j ) > y Tj Γ ∇ f j ( θ j ) , otherwise Therefore within the limiting region f j ( θ j ) > , ˙ f j ( θ j ) > , < f j ( θ j ) < ∧ y Tj Γ ∇ f j ( θ j ) > f j ( θ j ) = 0 , f j ( θ j ) = 1 ∧ y Tj Γ ∇ f j ( θ j ) > f j ( θ j ) ≤ , y Tj Γ ∇ f j ( θ j ) ≤ Thus given the initial condition for (11), f j ( θ j ( t )) ≤ and therefore f j ( θ j ( t )) ≤ for all t ≥ t . Therefore from Lemma 2, there existsconstants denoted θ j,max such that (cid:107) θ j ( t ) (cid:107) ≤ θ j,max for all θ j ( t ) ∈ Ξ ,j , i.e. all f j ( θ j ) ≤ , ∀ t ≥ t , as proven. Thus there exists aconstant denoted θ max such that (cid:107) θ ( t ) (cid:107) ≤ θ max , ∀ t ≥ t . R EFERENCES [1] K. S. Narendra and A. M. Annaswamy,
Stable Adaptive Systems . Dover,2005.[2] ——,
Stable Adaptive Systems . NJ: Prentice-Hall, Inc., 1989, (out ofprint).[3] M. Krsti´c, I. Kanellakopoulos, and P. Kokotovi´c,
Nonlinear and AdaptiveControl Design . Wiley, 1995.[4] P. A. Ioannou and J. Sun,
Robust Adaptive Control . PTR Prentice-Hall,1996.[5] E. Lavretsky and K. A. Wise,
Robust and Adaptive Control withAerospace Applications . Springer London, 2013.[6] A. P. Morgan and K. S. Narendra, “On the stability of nonautonomousdifferential equations ˙ x = [ A + B ( t )] x , with skew symmetric matrix B ( t ) ,” SIAM Journal on Control and Optimization , vol. 15, no. 1, pp.163–176, jan 1977.[7] ——, “On the uniform asymptotic stability of certain linear nonau-tonomous differential equations,”
SIAM Journal on Control and Opti-mization , vol. 15, no. 1, pp. 5–24, jan 1977.[8] B. D. Anderson and C. Johnson, “Exponential convergence of adaptiveidentification and control algorithms,”
Automatica , vol. 18, no. 1, pp.1–13, jan 1982.[9] L. Ljung,
System Identification: Theory for the User . Prentice-Hall,1987.[10] S. Boyd and S. Sastry, “On parameter convergence in adaptive control,”
Systems & Control Letters , vol. 3, no. 6, pp. 311–319, dec 1983.[11] S. Boyd and S. S. Sastry, “Necessary and sufficient conditions forparameter convergence in adaptive control,”
Automatica , vol. 22, no. 6,pp. 629–639, nov 1986.[12] K. S. Narendra and A. M. Annaswamy, “Persistent excitation in adaptivesystems,”
International Journal of Control , vol. 45, no. 1, pp. 127–160,jan 1987.[13] S. Aranovskiy, A. Bobtsov, R. Ortega, and A. Pyrkin, “Parameters esti-mation via dynamic regressor extension and mixing,” in . IEEE, jul 2016.[14] S. Aranovskiy, A. Belov, R. Ortega, N. Barabanov, and A. Bobtsov,“Parameter identification of linear time-invariant systems using dynamicregressor extension and mixing,”
International Journal of AdaptiveControl and Signal Processing , vol. 33, no. 6, pp. 1016–1030, may 2019.[15] P. M. Lion, “Rapid identification of linear and nonlinear systems,”
AIAAJournal , vol. 5, no. 10, pp. 1835–1842, oct 1967.[16] R. L. Carroll and D. P. Lindorff, “An adaptive observer for single-inputsingle-output linear systems,”
IEEE Transactions on Automatic Control ,vol. 18, no. 5, pp. 428–435, oct 1973. [17] G. Luders and K. S. Narendra, “An adaptive observer and identifier for alinear system,”
IEEE Transactions on Automatic Control , vol. 18, no. 5,pp. 496–499, oct 1973.[18] G. Kreisselmeier, “Adaptive observers with exponential rate of conver-gence,”
IEEE Transactions on Automatic Control , vol. 22, no. 1, pp.2–8, feb 1977.[19] B. Jenkins, A. Krupadanam, and A. M. Annaswamy, “Fast adaptiveobservers for battery management systems,”
IEEE Transactions onControl Systems Technology , 2019.[20] C. F. Gauss,
Theoria Combinationis Observationum Erroribus MinimisObnoxiae . Henricum Dieterich, 1825.[21] J.-J. E. Slotine and W. Li, “Composite adaptive control of robotmanipulators,”
Automatica , vol. 25, no. 4, pp. 509–519, jul 1989.[22] W. Li, “Adaptive control of robot motion,” Ph.D. dissertation, MIT, 1990.[23] J.-J. E. Slotine and W. Li,
Applied Nonlinear Control . Prentice-Hall,1991.[24] G. Goodwin and D. Mayne, “A parameter estimation perspective ofcontinuous time model reference adaptive control,”
Automatica , vol. 23,no. 1, pp. 57–70, jan 1987.[25] M. Krsti´c and P. V. Kokotovi´c, “Adaptive nonlinear design withcontroller-identifier separation and swapping,”
IEEE Transactions onAutomatic Control , vol. 40, no. 3, pp. 426–440, mar 1995.[26] S. B. Roy, S. Bhasin, and I. N. Kar, “Combined MRAC for unknownMIMO LTI systems with parameter convergence,”
IEEE Transactionson Automatic Control , vol. 63, no. 1, pp. 283–290, jan 2018.[27] N. Cho, H.-S. Shin, Y. Kim, and A. Tsourdos, “Composite modelreference adaptive control with parameter convergence under finiteexcitation,”
IEEE Transactions on Automatic Control , vol. 63, no. 3,pp. 811–818, mar 2018.[28] M. A. Duarte and K. S. Narendra, “Combined direct and indirectapproach to adaptive control,”
IEEE Transactions on Automatic Control ,vol. 34, no. 10, pp. 1071–1075, 1989.[29] ——, “A new approach to model reference adaptive control,”
Interna-tional Journal of Adaptive Control and Signal Processing , vol. 3, no. 1,pp. 53–73, mar 1989.[30] E. Lavretsky, “Combined/composite model reference adaptive control,”
IEEE Transactions on Automatic Control , vol. 54, no. 11, pp. 2692–2697, nov 2009.[31] G. Chowdhary, T. Yucelen, M. M¨uhlegg, and E. N. Johnson, “Concurrentlearning adaptive control of linear systems with exponentially convergentbounds,”
International Journal of Adaptive Control and Signal Process-ing , vol. 27, no. 4, pp. 280–301, may 2013.[32] G. Chowdhary, M. M¨uhlegg, and E. Johnson, “Exponential parameterand tracking error convergence guarantees for adaptive controllers with-out persistency of excitation,”
International Journal of Control , vol. 87,no. 8, pp. 1583–1603, may 2014.[33] A. Parikh, R. Kamalapurkar, and W. E. Dixon, “Integral concurrentlearning: Adaptive control with parameter convergence using finite exci-tation,”
International Journal of Adaptive Control and Signal Processing ,oct 2018.[34] K. S. Narendra, I. H. Khalifa, and A. M. Annaswamy, “Error modelsfor stable hybrid adaptive systems,”
IEEE Transactions on AutomaticControl , vol. 30, no. 4, pp. 339–347, apr 1985.[35] A.-P. Loh, A. M. Annaswamy, and F. P. Skantze, “Adaptation inthe presence of a general nonlinear parameterization: An error modelapproach,”
IEEE Transactions on Automatic Control , vol. 44, no. 9, pp.1634–1652, 1999.[36] D. P. Bertsekas,
Convex Optimization Theory . Athena Scientific, 2009.[37] L. Praly, G. Bastin, J.-B. Pomet, and Z. P. Jiang, “Adaptive stabilizationof nonlinear systems,” in
Foundations of Adaptive Control . Springer-Verlag, 1991, pp. 347–433.[38] J.-B. Pomet and L. Praly, “Adaptive nonlinear regulation: Estimationfrom the lyapunov equation,”
IEEE Transactions on Automatic Control ,vol. 37, no. 6, pp. 729–740, jun 1992.[39] E. Lavretsky, T. E. Gibson, and A. M. Annaswamy, “Projection operatorin adaptive systems,” arXiv preprint arXiv:1112.4232 , 2012.[40] H. K. Khalil,