The transversality conditions in infinite horizon problems and the stability of adjoint variable
aa r X i v : . [ m a t h . O C ] A ug THE TRANSVERSALITY CONDITIONS IN INFINITEHORIZON PROBLEMS AND THE STABILITY OFADJOINT VARIABLE
Dmitry Khlopin
Received: 10.08.2011 / Accepted: date
Abstract
This paper investigates the necessary conditions of optimality for uni-formly overtaking optimal control on infinite horizon with free right endpoint.Clarke’s form of the Pontryagin Maximum Principle is proved without the as-sumption on boundedness of total variation of adjoint variable. The transversalitycondition for adjoint variable is shown to become necessary if the adjoint variableis partially Lyapunov stable. The modifications of this condition are proposed forthe case of unbounded adjoint variable. The Cauchy-type formula for the adjointvariable proposed by S. M. Aseev and A. V. Kryazhimskii in [1],[2] is shown tocomplement relations of the Pontryagin Maximum Principle up to the completeset of necessary conditions of optimality if the improper integral in the formulaconverges conditionally and continuously depends on the original position. Theresults are extended to an unbounded objective functional (described by a non-convergent improper integral), unbounded constraint on the control, and uniformlysporadically catching up optimal control.
Keywords
Optimal control · infinite horizon problem · transversality conditionfor infinity · necessary conditions · Lyapunov stability · uniformly overtakingoptimal Mathematics Subject Classification (2000) · · · Introduction
The Pontryagin Maximum Principle for infinite horizon problems has already beenformulated in the monograph [26], but without the transversality condition theobtained relations were incomplete and in general, selected a much too broadfamily of potentially extremal trajectories. A significant number [20, 4, 8, 25, 31,
Our grant could be hereD. KhlopinInstitute of Mathematics and Mechanics, Ural Branch, Russian Academy of Sciences, Yekater-inburg, RussiaE-mail: [email protected] Dmitry Khlopin
10, 22] of such conditions has been proposed; however, as it was noted in, forexample, [20, 25, 30],[4, Sect. 6],[28, Example 10.2], these conditions may be eitherinconsistent with the relations of the Pontryagin Maximum Principle, or followfrom them. Hence the need to investigate the applicability of a transversalitycondition (see [4, 8, 22, 25, 31, 36, 28, 29, 27]) and the need to separately check if itis necessary for a specific optimization problem. The first aim of this paper isto offer a common approach to selecting a necessary transversality condition onthe adjoint variable for this problem (Subsect. 4.3–4.4). However, the necessity ofa condition does not imply its nontriviality on solutions of the relations of theMaximum Principle. Hence the need to find a condition that would select a singlesolution of the relations of the Maximum Principle for any uniformly overtakingoptimal control. In the papers [1, 2, 3, 4, 5], Aseev and Kryazhimskii develop andinvestigate the Cauchy-type formula for the adjoint variable that possesses sucha property. The second aim is to maximize the applicability of the approach [4](Sect. 5).First of all, we construct the bicompact extension (see [34]) for the space ofadmissible controls in the form of the inverse limit of the sequence of correspond-ing finite horizon extensions. It is shown that there exists a uniformly overtakingoptimal generalized control for the case of a conditionally convergent objectivefunctional that converges uniformly with respect to all trajectories; this general-izes some results [9],[12],[16]. Without this assumption, for uniformly overtakingoptimal control for problems with free right endpoint, the necessity of the Pon-tryagin Maximum Principle in Clarke’s form for the more general conditions thanin [4, 5, 25],[36, Theorem 2.1] is shown; the obtained result is not a part of theresults [7, 20, 27].In Subsect. 3.3 for a free right endpoint problem, the convergence of transver-sality conditions on the adjoint variable is provided by the integral partial stabilityof the adjoint variable as a component of the Maximum Principle system. Thuswe obtain the assumptions that guarantee the necessity of such condition, whichare more general than the assumptions in [28],[36, Corollary 2.1],[7, Theorem 1](It seems that the first implementation of the approach that employs the notionof stability to obtain transversality conditions was in [31]). If the the objectivefunction and the right-hand side of the equation of dynamics are smooth (in thephase variable), then, instead of integral partial stability we can check the simplercondition of partial Lyapunov stability for the variable ψ as a component of solu-tions of the system of the Maximum Principle. For example, we can check if allLyapunov exponents are negative for this variable (see Subsect. 4.3).We propose the new transversality condition: the product of the adjoint vari-able and a matrix function of time must be vanishingly small at infinity. Thiscondition becomes necessary if the product is stable. The stability can be pro-vided by the correct choice of the matrix function; the choice may also reflecta priori information on stability and asymptotic estimates, which may allow topinpoint the single extremal (see Subsect. 4.4).If the above matrix function is the fundamental matrix of linearized systemalong the optimal trajectory, then, the corresponding transversality condition auto-matically yields the formula that was proposed for affine systems in [1],[2], generalcase of which was examined in [4, Theorems 11.1, 12.1], [5, Theorem 2],[7, Theo-rem 1]. As it was shown in [4, Sect. 16], the results of [8, 36] are the corollaries of[4, Theorem 12.1],[5, Theorem 2]. ECESSARY CONDITIONS IN CASE OF STABLE ADJOINT VARIABLE 3
Such choice of matrix function allows us to reduce the question of necessity ofthe corresponding condition to not just the question of the stability of the prod-uct, but even to the issue of checking if the improper integral from the formula [4,(12.8)], [5, Theorem 2],[7, Theorem 1] converges conditionally and continuously de-pends on the initial position of the original problem. This yields the Cauchy-typeformula for the adjoint variable and the “normal” Pontryagin Maximum Principleunder the assumptions weaker than in [4, Theorem 12.1]. This result also general-izes [27, Theorem 3.2],[36, Theorem 2.1] and [28, Theorems 3.1 and 8.1] (as far asthe necessary conditions for problems with free right endpoint are concerned).For the case of monotonous system, we also demonstrated certain estimatesfor the adjoint variable. In particular, we obtained the nonnegativity of adjointvariable under the weaker assumptions than in [6, Theorem 1], [35, Theorem 1],[4, Theorem 10.1].In the last part of the paper, we extend the obtained results to the the cases of σ -compact constraints on controls and to uniformly sporadically catching up opti-mal controls. Important breakthroughs for these problems were recently achievedin [7].A part of the results of this paper has been shown and announced in paper [24]. We consider the time interval T △ = R ≥ . The phase space of the original con-trol system is the certain finite-dimensional metric space X △ = R m . The unit ballof this space is denoted by D . Let L denote the linear space of all m × m ma-trices. For the sake of definiteness, let us equip L with the operator norm. Thesymbol E (which may be equipped with some indices) denotes various auxiliaryfinite-dimensional Euclidean spaces, and the symbol B ( E ) denotes the σ -algebrasof their Borel subsets.For a subset A of a topological space, cl A denotes the closure of this subset.On the sets of all functions that are continuous on the whole T , we considerthe topology of uniform convergence on T and the compact-open topology; forexample, C ( T , E ) and C loc ( T , E ). The first one is considered to be equipped withthe norm || · || C of the uniform convergence topology. Ω denotes the family offunctions ω ∈ C ( T , T ) such that lim t →∞ ω ( t ) = 0 . Here and below, for each summable function a of time, the integral R T a ( t ) dt isthe limit R [0 ,T ] a ( t ) dt as T → ∞ . The integral over an infinite interval, for example,over [ T, ∞i , is interpreted in the same sense.Let us also consider a finite-dimensional Euclidean space U and a set-valuedmap U : T U . The set U of admissible controls is understood as the set ofall Borel measurable selectors of the multi-valued map U . The topology on U isdefined by virtue of the inclusion U ⊂ L loc ( T , U ) . A function a : T × E ′ × U → E ′′ is said to1) satisfy the Carath´eodory conditions if a) the function a ( · , y, u ) : T → E ′′ ismeasurable for all ( y, t, u ) ∈ X × GrU, b) the function a ( t, · , · ) : E ′ × U ( t ) → E ′′ is continuous for all t ∈ T . Dmitry Khlopin
2) be locally Lipshitz continuous if for each compact K ∈ ( comp )( T × E ) thereexists a function L aK ∈ L loc ( T , T ) such that for all ( t, x ′ ) , ( t, x ′′ ) ∈ K, u ∈ U ( t ),the inequality || a ( t, x ′ , u ) − a ( t, x ′′ , u ) || E ′′ ≤ L aK ( t ) || x ′ − x ′′ || E ′ holds.3) be integrally bounded (on each compact subset of T × E ) if for each compact K ∈ ( comp )( T × E ) there exists a function M aK ∈ L loc ( T , T ) such that for all( t, x ) ∈ K, u ∈ U ( t ) we have || a ( t, x, u ) || E ′′ ≤ M aK ( t ).4) satisfy the continuability condition on T if it satisfies the sublinear growthcondition, i.e., if the function f is Lipshitz continuous such that the function L aK is independent of K and is integrally bounded (on each compact subset);see [33, 1.4.6].Here and below, we assume the following conditions hold: Condition ( u ) : U is a compact-valued map such that it is integrally bounded(on each compact subset of T ) and Gr U ∈ B ( T × U ). Condition ( fg ) : the mappings f : T × X × U → X , g : T × X × U → R arelocally Lipshitz continuous Carath´eodory mappings that are integrally bounded(on each compact subset) and f satisfies the continuability condition.Let us consider the control system˙ x = f ( t, x, u ) , x (0) = 0 , t ∈ T , x ∈ X , u ∈ U ( t ) . (1a)Now we can assign the solution (1a) to every u ∈ U . The solution is unique and itcan be extended to the whole T . Let us denote it by ϕ [ u ]. The mapping ϕ : U → C loc ( T , X ) is continuous.In what follows, we examine the problem of maximizing the objective functionallim T →∞ J T ( u ) → max; (1b) J T ( u ) △ = Z T g (cid:0) t, ϕ [ u ]( t ) , u ( t ) (cid:1) dt. If there is no limit in (1b), the optimality may be defined in diverse ways (fordetails, see [13],[11],[32]), generally, we will us the following one:
Definition 1
A control u ∈ U is called uniformly overtaking optimal if for each ε ∈ R > there exists T ∈ R > such that J t ( u ) ≥ J t ( u ) − ε holds for all u ∈ U , t ∈ [ T, ∞i .Note that in paper [32], this definition is referred to as uniformly catching upoptimal control. In [21, Theorem 3.1], it is shown that the uniformly overtakingoptimality is equivalent to the conditionlim t → + ∞ (cid:16) J t ( u ) − sup u ∈ U J t ( u ) (cid:17) = 0which, in terms of [21], says that u is strongly agreeable.Note that for each uniformly overtaking optimal control u ∈ U there exists afunction ω ∈ Ω such that J t ( u ) ≥ J t ( u ) − ω ( T ) ∀ u ∈ U , T ∈ T , t ∈ [ T, ∞i . (2) ECESSARY CONDITIONS IN CASE OF STABLE ADJOINT VARIABLE 5
To complete the main objective of this section, we need the following assumption:
Condition ( e ) : there exists a function ω ∈ Ω such that Z τT g (cid:0) t, ϕ [ u ]( t ) , u ( t ) (cid:1) dt ≤ ω ( T ) ∀ u ∈ U , T, τ ∈ T , T < τ. Note that to the best of author’s knowledge, a one-sided condition like ( e ) wasfirst proposed in paper [16, ( Π e ), it is enough to assume, for example, the stronger condition g (cid:0) t, ϕ [ u ]( t ) , u ( t ) (cid:1) ≤ l ( t ) ∀ u ∈ U , T ∈ T for some summable on T mapping l ∈ L ( T , R ) . e U of generalized controlsFor each u ∈ U , the symbol e δ ( u ) denotes the probability measure concentrated atthe point u . Let e U n denote the family of all weakly measurable mappings µ from[0 , n ] to the set of Radon probability measures over U such that R U ( t ) η ( t )( du ) = 1for a.a. t ∈ [0 , n ]. Let us equip this set with the topology of *-weak convergence.Then, the obtained topological space is a compact [34, IV.3.11], and the set U n △ = { u | [0 ,n ] | u ∈ U } is everywhere densely included in e U n [34, IV.3.10] by the mapping u → e δ ◦ u .Now, let us introduce the set of all maps η from T into the set of Radonprobability measures over U such that η | [0 ,n ] ∈ e U n for every n ∈ N ; and letus denote it by e U . To each n ∈ N let projections e π n : e U → e U n be given by e π n ( η ) △ = η | [0 ,n ] for all η ∈ e U . Let us equip e U with the weakest topology such thatall projections are continuous. The set e U is called the set of generalized controls.Let us assume that for the certain Euclidean space E a mapping a : T × E × U → ( comp )( E ) is given and the following condition is satisfied: Condition ( a ) : the mapping a : T × E × U → ( comp )( E ) is a locally Lipshitzcontinuous integrally bounded Carath´eodory mapping that satisfies the continua-bility condition.Let us fix the set Ξ ⊂ E of initial values and the system for u ∈ U :˙ y = a ( t, y ( t ) , u ( t )) , y (0) = ξ ∈ Ξ, t ∈ T , u ∈ U . (3)It can also be generalized for η ∈ e U :˙ y = Z U ( t ) a ( t, y ( t ) , u ) η ( t )( du ) , y (0) ∈ Ξ, t ∈ T , η ∈ e U . (4)Each its local solution can be extended to the whole T . For every η ∈ e U , let usdenote the family of all solutions y ∈ C loc ( T , E ) of system (4) by e A [ η ]. Dmitry Khlopin e ϕ [ η ] ∈ C loc ( T , X ) of the Cauchy problem˙ x = Z U ( t ) f ( τ, x ( τ ) , u ) η ( t )( du ) , x (0) = 0 ∀ η ∈ e U , (5a)the function T e J T ( η ) △ = R [0 ,T ] R U ( t ) g ( t, e ϕ [ η ]( t ) , u ) η ( t )( du ) dt ; and the problem ofmaximizing the functional lim T →∞ e J T ( u ) → max . (5b) Proposition 1
Assume ( u ) . Then,1) the space e U is a compact, and e δ ( U ) is everywhere dense in it;2) If ( a ) holds, then for a compact Ξ ∈ ( comp )( E ) the map e A : e U → C loc ( T , E ) iscontinuous, and e A [ e δ ◦ U ] is everywhere dense in e A [ e U ] ∈ ( comp )( C loc ( T , E ));
3) If ( fg ) hold, then e ϕ, e J ∈ C loc ( T × e U , R ) ;4) If ( fg ) , ( e ) hold, then there is a uniformly overtaking optimal control e u ∈ e U forthe relaxed problem ( ) – ( ) such that lim T →∞ sup u ∈ U Z T g ( t, ϕ [ u ]( t ) , u ( t )) dt = lim T →∞ max η ∈ e U e J T ( η ) = max η ∈ e U lim T →∞ e J T ( η ) = (6)= lim T →∞ e J T ( e u ) = Z T Z U ( t ) g ( t, e ϕ [ e u ]( t ) , u ) e u ( du ) dt, and all limits in ( ) exist, although they can equal −∞ Proof.
For the sake of brevity, let us denote e Π △ = Q n ∈ N e U n , and let us equip it withTikhonov topology. Let e ∆ : e U → e Π be given by e ∆ ( η ) △ = (cid:0)e π n ( η ) (cid:1) n ∈ N for all η ∈ e U . It is a homeomorphism by continuity of the maps e π n and e π n ◦ e ∆ − .Let n, k ∈ N , ( n > k ) . Then, the space e U n is included in e U k by the mapping e π nk ( η ) △ = η | [0 ,k ] for all η ∈ e U n . By e π nk ◦ e π ki = e π ni for all n, k, i ∈ N , ( n > k > i ) , wehave the projective sequence of the topological spaces { e U n , e π nk } ; and we can definethe inverse limit [18, III.1.5], [17, 2.5.1]. In our notation, we can write it in theform lim ← { e U n , e π nk } △ = e ∆ ( e U ) ⊂ e Π. As shown above, e ∆ is a homeomorphism; hence, e U is homeomorphous to e ∆ ( e U ). Now, by Kurosh Theorem [18, III.1.13], the inverselimit e ∆ ( e U ) of compacts e U n is compact, and e U is a compact too. Similarly, from[17, 4.2.5] and [34, IV.3.11] it follows that e U is also metrizable.Repeating the reasonings without e or referring to [17, 3.4.11] and [17, 2.5.6]yields U ∼ = lim ← { U n , π nk } △ = ∆ ( U ) ⊂ Π. For each n ∈ N , let the mapping e n : U n → e U n be given by e n ( u )( t ) △ = ( e δ ◦ u )( t ) = e δ u ( t ) for all t ∈ [0 , n ] , u ∈ U n . Since for all n, k ∈ N , n > k it holds that e k ◦ π nk = e n , we have the projective system { e n , π nk } . Passing to the inverse limit, we obtain themapping e ∆ : ∆ ( U ) → e ∆ ( e U ); from e n ◦ π n = e π n ◦ e δ we have e ∆ ◦ ∆ = e ∆ ◦ e δ , and ECESSARY CONDITIONS IN CASE OF STABLE ADJOINT VARIABLE 7 from e U n = cle n ( U n ) ([34]) we have e ∆ ( e U ) = cle ∆ (cid:0) ∆ ( U ) (cid:1) = cl ( e ∆ ◦ e δ )( U ); now, bycontinuity of e ∆ − , we obtain e U = cl e δ ( U ) . The mapping e A [ η ] is continuous by virtue of, for example, [33, Theorem 3.5.6];the set e A [ η ]( e U ) is compact as a continuous image of a compact. In what follows,is sufficient to use e U = cl e δ ( U ) . Replacing a and the compact Ξ with the mapping { ( f, g ) } and the compact { (0 X , R ) } , we obtain the continuous dependence on η for the maps e ϕ, e J . Now, byvirtue of clJ t ( U ) = cl e J t ( e δ ◦ U ) = e J t ( e U ), the condition ( e ) holds for η ∈ e U too, i.e.,it holds that e J t ( η ) ≤ J T ( η ) + ω ( T ) ∀ η ∈ e U , T ∈ T , t ∈ [ T, ∞i , (7)then, lim sup t →∞ e J t ( η ) ≤ J T ( η ) + ω ( T ) ∀ η ∈ e U , T ∈ T , passing to the lower limit as T → ∞ , we obtain, for arbitrary η ∈ e U , the existenceof the limit lim t →∞ e J t ( η ) (possibly infinite).Then, for every t ∈ T , there exists an η t ∈ e U such that R t △ = max η ∈ e U e J t ( η ) = e J t ( η t ) ∀ t ∈ T . (8)Since ( η t ) t ∈ T is in the compact, for the certain unbounded increasing sequence( t k ) k ∈ N ∈ T and the certain e u ∈ e U , it is η t k → e u . Let us also define R △ = lim sup t →∞ R t , R △ = lim inf t →∞ R t , R ∗ △ = sup η ∈ e U lim t →∞ e J t ( η ) , R △ = lim t →∞ e J t ( e u ) . (9)Now, R t k = e J t k ( η t k ) ( ) ≤ e J t i ( η t k ) + ω ( t i ) ∀ i, k ∈ N ( i < k ) , passing to the upper limit as k → ∞ and then as i → ∞ , we obtain R ≤ e J t i ( e u ) + ω ( t i ) and R ≤ R . Thus, for all T ∈ T R ≤ R ) = lim t →∞ e J t ( e u ) ≤ sup η ∈ e U lim t →∞ e J t ( η ) ( ) = R ∗ ( ) ≤ sup η ∈ e U e J T ( η ) + ω ( T ) ( ) = R T + ω ( T ) . Passing to the lower limit as T → ∞ , we obtain R ≤ R ≤ R ∗ ≤ R , it remains tonote that by virtue of clJ t ( U ) = cl e J t ( e δ ◦ U ) = e J t ( e U ), it holds thatlim t →∞ sup u ∈ U J t ( u ) = lim t →∞ max η ∈ e U e J t ( η ) = lim t →∞ R t = R = R ∗ . ⊓⊔ Remark 1
As it was shown in 1), for each generalized control there exists thesequence of controls from U that converges (in the topology e U ) to it. Remark 2 (turnpike property)
Item 4) actually shows more. It shows that the uni-formly overtaking optimal control e u can be obtained as a limitary point of thesets arg max η ∈ e U e J t ( η ) ∈ ( comp )( e U ) as t → ∞ . Dmitry Khlopin
Remark 3
As it was shown in 4), the limit Z T Z U ( t ) g ( t, e ϕ [ η ]( t ) , u () η ( du ) dt △ = lim T →∞ e J T ( η )is defined (though it may be infinite) for all η ∈ e U .Note that not only did the paper [16] prove the theorem of existence of anoptimal solution based on the condition ( e ) but it also discussed the proof of suchtheorems based on the inverse limit. To the best of author’s knowledge, there isonly one paper [23] besides the previous one in the control theory that explicitlyemploys the notion of inverse limit.There are many existence theorems, for example, [9], [11],[13],[12]. The resultsobtained in Proposition 1 have much in common with paper [12] (in terms of [12],the obtained e u is strongly optimal). Note that if the initial set U does not containa uniformly overtaking optimal control, we may pass to Gamkrelidze controls byincreasing the dimension of the set U in m +1 times. (For details of such bicompactextension, see [19], [12]). These controls also form a compact and the items 1)-3) of Proposition 1 hold from them; therefore, there always exists a uniformlyovertaking optimal control among such finite-dimensional controls.As a corollary, we assume the uniformly overtaking optimal control u to existamong the elements of U , and denote the trajectory that corresponds to u by x .We also keep the denotation e u △ = e δ ◦ u . We are also interested in the degree of closeness of various generalized controlsfor large t . Let w : T × U → T be an integrally bounded Carath´eodory map. Forall τ ∈ T and η ∈ e U , let us introduce L w [ η ]( τ ) △ = Z τ Z U ( t ) w ( t, u ) η ( t )( du ) dt. Let us denote by (
F in )( u ) the family of η ∈ e U such that η | [ T, ∞i = e u | [ T, ∞i forthe certain T ∈ T . Let us assume that L w [ e u ] ≡
0, and for every η ∈ ( F in )( u )from L w [ η ]( τ ) = 0 for all τ ∈ T it follows that η equals e u a.e. on [0 , τ ]. The set ofsuch w is denoted by ( Null )( u ). H : X × Gr U × T × X → R be given by H ( x, t, u, λ, ψ ) △ = ψf (cid:0) t, x, u (cid:1) + λg (cid:0) t, x, u (cid:1) . Let us introduce the relations ˙ x ( t ) = f (cid:0) t, x ( t ) , u ( t ) (cid:1) ; (10a)˙ ψ ( t ) ∈ − ∂ x H (cid:0) x ( t ) , t, u ( t ) , λ, ψ ( t ) (cid:1) ; (10b)sup p ∈ U ( t ) H (cid:0) x ( t ) , t, p, λ, ψ ( t ) (cid:1) = H (cid:0) x ( t ) , t, u ( t ) , λ, ψ ( t ) (cid:1) ; (10c) x (0) = 0 , || ψ (0) || X + λ = 1 . (10d) ECESSARY CONDITIONS IN CASE OF STABLE ADJOINT VARIABLE 9
It is easily seen that for each u ∈ U , for each initial condition, system (10a)–(10b) has a local solution, and each solution of these relations can be extended tothe whole T . Let us denote by Y the family of all solutions ( x, u, λ, ψ ) ∈ C loc ( T , X ) × U × [0 , × C loc ( T , X ) of system (10a)–(10b),(10d) on T , and let us denote by Z theset of solutions from Y for which (10c) also holds a.e. on T .Let us introduce such conditions for generalized controls; namely, under initialcondition (10d) let us consider˙ x ( t ) = Z U ( t ) f ( t, x ( t ) , u ) η ( t )( du ); (11a)˙ ψ ( t ) ∈ − Z U ( t ) ∂ x H (cid:0) x ( t ) , t, u, λ, ψ ( t ) (cid:1) η ( t )( du ); (11b)sup p ∈ U ( t ) H (cid:0) x ( t ) , t, p, λ, ψ ( t ) (cid:1) = Z U ( t ) H (cid:0) x ( t ) , t, u, λ, ψ ( t ) (cid:1) η ( t )( du ) . (11c)Similarly, for each η ∈ e U for each initial condition, system (11a)–(11b) has a localsolution that can be extended to the whole T .Let us denote by e Y the family of all solutions ( x, η, λ, ψ ) ∈ C loc ( T , X ) × e U × [0 , × C loc ( T , X ) of system (10d)–(11b). Let us also introduce e Z , the family of( x, η, λ, ψ ) ∈ e Y such that (11c) also holds a.e. on T .Let us note that for every η ∈ e U , the family of all solutions ( x, η, λ, ψ ) ∈ e Y of system (10d)–(11a) on T for given control η is compact by virtue of [33,Theorem 3.4.2]. Moreover, this compact-valued map is upper semicontinuous in η .Indeed, the right-hand side of (11a)–(11b) is convex and integrally bounded, uppersemicontinuous in η , and it is measurable for each fixed x, ψ ; therefore, it has ameasurable selector ([33, Lemm 2.3.11]); moreover, all local solutions of (11a)–(11b) can be extended to the whole T . Since all the conditions of [33, Theorem3.5.6] are satisfied, the mapping is upper semicontinuous. Therefore, e Y and e Z arecompact, as the graphs of this mapping on the compact subdomain of its domain.Note that by [14, Theorem 2.7.5] always holds the inclusion: ∂ x Z U ( t ) H (cid:0) x, t, u, λ, ψ (cid:1) η ( t )( du ) ⊂ Z U ( t ) ∂ x H (cid:0) x, t, u, λ, ψ (cid:1) η ( t )( du ) . (12)3.2 The necessity of the Maximum Principle Theorem 1
Assume conditions ( u ) , ( fg ) . For each uniformly overtaking optimalpair ( x , u ) ∈ C ( T , X ) × U for problem ( ) – ( ) , there exist λ ∈ [0 , , ψ ∈ C ( T , X ) such that the relations of the Maximum Principle ( ) – ( ) hold; i.e., ( x , u , λ , ψ ) ∈ Z .Proof. Let us fix a certain unbounded monotonically increasing sequence ( τ n ) n ∈ N ∈ T N . Let us also consider an arbitrary sequence ( γ n ) n ∈ N ∈ T N that converges tozero with the property ω ( τ n ) /γ n →
0, where the function ω was taken from (2).For example, γ n △ = p ω ( τ n ) will suffice. Fix a w ∈ ( Null )( u ). For each n ∈ N let us consider the problem J τ n ( η ) − γ n L w [ η ]( τ n ) = Z τ n Z U ( t ) g ( t, e ϕ [ η ]( t ) , u ) η ( t )( du ) dt − γ n L w [ η ]( τ n ) → max . Here, the functional is bounded from above by the number J τ n ( u ) + ω ( τ n ),therefore, it has the supremum. Every summand continuously depends on η , whichcovers the compact e U ; therefore, there is an optimal solution for this problem in e U ;let us denote one of them by ( x n , η n ).Let the function H τ n : X × Gr U × T × X → R be given by H τ n ( x, t, u, λ, ψ ) △ = H ( x, t, u, λ, ψ ) − γ n w ( t, u ) . Then, by the Clarke form [14, Theorem 5.2.1] of the Pontryagin Maximum Prin-ciple, there exists ( λ n , ψ n ) ∈ T × C ([0 , n ] , X ) such that relation (10d) and thetransversality condition at the free endpoint ψ n ( τ n ) = 0 hold, andsup p ∈ U ( t ) H τ n (cid:0) x n ( t ) , t, p, λ n , ψ n ( t ) (cid:1) = Z U ( t ) H τ n (cid:0) x n ( t ) , t, u, λ n , ψ n ( t ) (cid:1) η n ( t )( du ) , (13)˙ ψ n ( t ) ∈ − ∂ x Z U ( t ) H τ n (cid:0) x n ( t ) , t, u, λ n , ψ n ( t ) (cid:1) η n ( t )( du )also hold for a.a. t ∈ [0 , τ n ]. By (12), ( x n , η n , λ n , ψ n ) ∈ T × C ([0 , n ] , X ) satisfy therelations (10d)–(11b), (13) a. e. on the [0 , τ n ] . Let us extend the ( x n , η n , λ n , ψ n ) to [ τ n , ∞i by the generalized control e u | [ τ n , ∞i .Then, η n ∈ ( F in )( u ). Let us denote by Z n the set of ( x, u, λ, ψ ) that satisfyrelations (10d)–(11b) a. e. on T , satisfy relation (13) a. e. on [0 , τ n i , and possessthe property e u | [ τ n , ∞i = η n | [ τ n , ∞i . Now we have ( x n , η n , λ n , ψ n ) ∈ Z n for every n ∈ N .Let us note that all Z n are closed and, since these sets are contained in thecompact e Y , these sets are also compact. Hence, the sequence ( x n , η n , λ n , ψ n ) n ∈ N has the limit point ( x , η , λ , ψ ) ∈ e Y . Passing, if necessary, to a subsequence,we may assume that it is the limit of the sequence itself.For a fixed x , the set of u ∈ U ( t ) that realize the maximum in (13) has ameasurable selector by virtue of [15, Theorem 3.7]. By [33, Lemm 2.3.11], it existsif we put an arbitrary continuous function x into H . Besides, since relation (13) alsodepends on x, ψ and on the parameters γ and λ upper semicontinuously, and all therelations are integrally bounded on bounded sets; by virtue of [33, Theorem 3.5.6],on each finite interval for the funnels of solutions of (10a)–(10b) that satisfy (13),we have upper semicontinuity by γ, λ . In particular, for γ n → λ n → λ , weobtain the fact that the upper limit of the compacts Z n is included in e Z . Hence,( x , η , λ , ψ ) ∈ e Z .On the other side, by w ∈ ( Null )( u ) and by optimality of η n , u for theirproblems, we obtain e J τ n ( η n ) − γ n L w [ η n ]( τ n ) ≥ J τ n ( u ) ( ) ≥ e J τ n ( η n ) − ω ( τ n )therefore, we have γ n L w [ η n ]( τ n ) ≤ ω ( τ n ) . By virtue of e u | [ τ n , ∞i = η n | [ τ n , ∞i , weobtain L w [ η n ]( τ ) ≤ ω ( τ n ) /γ n ∀ τ ∈ T . (14) ECESSARY CONDITIONS IN CASE OF STABLE ADJOINT VARIABLE 11
For each τ ∈ T , passing to the limit as n → ∞ , we obtain that L w [ η ] ≤
0; i.e., L w [ η ]( τ ) = 0 for all τ ∈ T . Since w ∈ ( Null )( u ), we have η = e u a.e. on T , hence x = x and ( x , u , λ , ψ ) ∈ Z . Moreover, from (14), we have || L w [ η n ] || C → ⊓⊔ We have additionally proved that
Remark 4
Under conditions ( u ) , ( fg ), for each optimal pair ( x , u ) ∈ X × U forproblem (1b), for each weight w ∈ ( Null )( u ) , for each unbounded increasingsequence ( τ n ) n ∈ N ∈ T N , we have constructed the sequence ( x n , η n , λ n , ψ n ) n ∈ N ∈ e Y N that possesses the following properties:1) This sequence (as a sequence from C loc ( T , X ) × e U × T × C loc ( T , X )) convergesto the certain ( x , e u , λ , ψ ) ∈ Z ;2) || L w ( η n ) || C → e J t n ( η ) − J t n ( u ) →
0, and ψ n ( t n ) = 0 for each n ∈ N , where ( t n ) n ∈ N is a certainsubsequence of ( τ n ) n ∈ N ∈ T N . t →∞ ψ ( t ) = 0 . (15a)Let us formulate the propositions in terms of the stability of ψ such that acondition would be necessary. Condition ( ψ ): There exists a weight w ∈ ( Null )( u ) such that for every so-lution ( x , u , λ , ψ ) ∈ Z , the Lagrange multiplier ψ is stable under L w − smallperturbations of system (10a)–(10b); i.e., for every ε ∈ R > , there exist a number δ ∈ R > and a neighborhood Υ ⊂ C loc ( T , X ) × e U × [0 , × C loc ( T , X ) of the solution( x , e u , λ , ψ ) such that for every solution ( x, η, λ, ψ ) ∈ Υ ∩ e Y from || L w [ η ] || C < δ it follows that || ψ − ψ || C < ε . Proposition 2
Assume conditions ( u ) , ( fg ) hold. For each uniformly overtaking opti-mal pair ( x , u ) ∈ C ( T , X ) × U satisfying ( ψ ) , for each unbounded increasing sequence, ( τ n ) n ∈ N ∈ T N there exists ( x , u , λ , ψ ) ∈ Z such that lim inf n →∞ || ψ ( τ n ) || X = 0 (15b) holds.Proof. Let us choose the certain ε ∈ R > , and let us take Υ ⊂ e Y and δ ∈ R > fromcondition ( ψ ); by Remark 4, there exists N ∈ N such that for n ∈ N , n > N , itis ( x n , η n , λ n , ψ n ) ∈ Υ , || L w [ η n ] || C < δ ; now, condition ( ψ ) also yields || ψ n ( τ n ) − ψ ( τ n ) || X < ε ; but ψ n ( τ n ) = 0; whence || ψ ( τ n ) || X < ε for all n ∈ N , n > N . Since ε ∈ R > was arbitrary, we have shown (15b). ⊓⊔ Note that by linearity of (10b), the stability of the variable ψ implies its bound-edness. Therefore, the proved proposition is useless for unbounded adjoint vari-able ψ .Note that, as it follows from [32, Example 5.1], for a uniformly overtakingoptimal control, there can be no ( x , u , λ , ψ ) ∈ Z that satisfies stronger condi-tion (15a) instead of (15b). On the other side, Remark 5
Assume the functions L fK , L gK are independent of a compact K , andthe mapping T L gK ( T ) e R [0 ,T ] L fK ( t ) dt is summable on T ([27, Hypotesis 3.1 (iv)]);therefore, the total variation of ψ is a fortiori bounded. Then, ( ψ ) holds and,moreover, (15b) implies (15a).The even more strong conditions used for proving the Maximum Principlecan be seen, for example, in [36, (A3)] (the Lipshitz constants were required todecrease exponentially with time). Naturally, the propositions proved there for thecondition are also covered by proposition 2.One of the most general conditions on (15a) was shown in [28]. For a controlproblem without phase restrictions, the transversality condition from [28, Theorem6.1] follows from Proposition 2 and [28, Lemm 3.1], or from Remark 5 and condition[28, (C3)]. The Remark 4 automatically yields [28, Theorem 8.1]. The objective at hand is to choose the weight w ∈ ( Null )( u ) such that condi-tion ( ψ ) would follow from a variety of (nonasymptotic) Lyapunov stability of ψ .4.1 On weight w Assume conditions ( u ) , ( a ) hold. In what follows, assume Ξ △ = E . Then, for everyposition ( τ ∗ , y ∗ ) ∈ T × E there exists the unique solution y of the equation˙ y = a ( t, y ( t ) , u ( t )) , y ( τ ∗ ) = y ∗ , τ ∗ ∈ T (16)that can be extended to the whole time interval T . It (as an element of e A ( e u ) ⊂ C loc ( T , E )) continuously depends on ( τ ∗ , y ∗ ) ∈ T × E . Let us denote its initialposition y (0) by κ ( τ ∗ , y ∗ ). Proposition 3
Assume ( u ) , ( a ) hold. Let the compact-valued map G : T E bebounded on each compact set, and let Gr G be closed.Then, there exists w ∈ ( Null )( u ) , such that for arbitrary η ∈ e U , T ∈ T for every y ∈ e A [ η ] from Gr y | [0 ,T ] ⊂ Gr G it follows that || κ ( τ, y ( τ )) − y (0) || E ≤ L w [ η ]( τ ) ∀ τ ∈ [0 , T ] . Proof.
ECESSARY CONDITIONS IN CASE OF STABLE ADJOINT VARIABLE 13
Fix an n ∈ N . By continuability, for each ( τ ∗ , y ∗ ) ∈ Gr G | [0 ,n ] , there exists theposition κ ( τ ∗ , y ∗ ); by virtue of the theorem of continuous dependence on initialconditions, this mapping is continuous; hence, the image¯ G n △ = n e ∈ Gr y | [0 ,n ] (cid:12)(cid:12)(cid:12) ∀ y ∈ e A [ e u ] , ( τ ∗ , y ( τ ∗ )) ∈ Gr G | [0 ,n ] o is closed; by the continuability, this set is bounded and, therefore, compact. There-fore, on this set, the function a ( t, y, u ( t )) is Lipshitz continuous with respect to y for the certain Lipshitz constant L n △ = L a ¯ G n ∈ L loc ( T , T ). For all t ∈ [0 , n ], de-fine M n ( t ) △ = R [0 ,t ] L n ( τ ) dτ . Note that this function is absolutely continuous andmonotonically nondecreasing.Fix n ∈ N ; for all t ∈ [ n − , n i , u ∈ U , let us consider a number R ( t, u ) △ = sup y ∈ ¯ G n (cid:12)(cid:12)(cid:12)(cid:12) a ( t, y, u ) − a ( t, y, u ( t ) (cid:1)(cid:12)(cid:12)(cid:12)(cid:12) E . Note that the norm inside is a mapping that is continuous with respect to y and u , and y assumes values from the compact set; now, for every u ∈ U by [15, Theorem 3.7] the supremum reaches the maximum for the certain function y max [ u ] ∈ L ([ n, n − i , ¯ G n ). Hence, R ( t, u ) is measurable with respect to t for each u ∈ U .Fix a t ∈ [ n − , n i ; for each sufficiently small neighborhood Υ ⊂ U ( t ), bycontinuity of a ( t, · , · ) on compact ¯ G n × clΥ , there exists a function ω t ∈ Ω , forwhich (cid:12)(cid:12)(cid:12) (cid:12)(cid:12)(cid:12)(cid:12) a ( t, y, u ′ ) − a ( t, y, u ( t ) (cid:1)(cid:12)(cid:12)(cid:12)(cid:12) − (cid:12)(cid:12)(cid:12)(cid:12) a ( t, y, u ′′ ) − a (cid:0) t, y, u ( t ) (cid:1)(cid:12)(cid:12)(cid:12)(cid:12) (cid:12)(cid:12)(cid:12) < ω t (cid:0) || u ′ − u ′′ || (cid:1) (17)holds for every y ∈ ¯ G n , u ′ , u ′′ ∈ Υ ( u ′ = u ′′ ). Without loss of generality, as-sume R ( t, u ′ ) ≤ R ( t, u ′′ ). Now, by definition, R ( t, u ′ ) ≥ (cid:12)(cid:12)(cid:12)(cid:12) a ( t, y, u ′ ) − a (cid:0) t, y, u ( t ) (cid:1)(cid:12)(cid:12)(cid:12)(cid:12) ,and, substituting y △ = y max ( u ′′ )( t ) into (17), we obtain 0 ≤ R ( t, u ′′ ) − R ( t, u ′ ) ≤ ω t (1 / || u ′ − u ′′ || ); i.e., R is continuous with respect to the variable u on each suffi-ciently small neighborhood Υ ⊂ U ( t ); therefore on U ( t ) and GrU | [ n − ,n i too. Thus,the function R : GrU | [ n − ,n i → T is a Carath´eodory function.Let us note that by considering all n ∈ N , we define the Carath´eodory func-tion R on the whole GrU . Moreover, by construction, R ( t, u ( t )) ≡
0. Hence, it iscorrect to define w ∈ ( Null )( u ) by the rule w ( t, u ) △ = || u − u ( t ) || + e M n ( t ) R ( t, u ) ∀ n ∈ N , ( t, u ) ∈ Gr U | [ n − ,n i . Consider arbitrary n ∈ N , τ ∗ ∈ [0 , n ], and ( τ ∗ , y ∗ ) , ( τ, y ∗ ) ∈ ¯ G n . For the solu-tions y , y ∈ e A [ e u ] of equation (16), for the initial conditions y i ( τ ∗ ) = y ∗ i , we have Gr y i | [0 ,n ] ⊂ ¯ G n . Let us introduce functions r ( t ) △ = y ( t ) − y ( t ) , W + ( t ) △ = e M n ( t ) || r ( t ) || E ∀ t ∈ [0 , n ] . By Lipshitz continuity of the right-hand side of (16) we obtain || ˙ r ( t ) || E ≥− L n ( t ) || r ( t ) || E , and dW ( t ) dt = 2 L n ( t ) W ( t ) + 2 e M n ( t ) r ( t ) ˙ r ( t ) ≥ L n ( t ) W ( t ) − L n ( t ) W ( t ) = 0 . Thus, the function W + is nondecreasing, and finally for all ( τ, y ∗ ) , ( τ, y ∗ ) ∈ ¯ G n wehave || κ ( τ, y ∗ ) − κ ( τ, y ∗ ) || E = W + (0) ≤ W + ( τ ) = e M n ( τ ) || y ∗ − y ∗ || E . (18)Assume the η ∈ e U , y ∈ e A [ η ] , T ∈ T satisfy Gr y | [0 ,T ] ⊂ Gr G . Fix arbitrary n ∈ N and τ , τ ∈ [0 , T ] ∩ [ n − , n i , τ < τ . There exists the solution y ∈ e A [ e u ]that satisfies the condition y ( τ ) = y ( τ ); let us also define r △ = y ( t ) − y ( t ) , W − ( t ) △ = e − M n ( t ) || r ( t ) || E ∀ t ∈ [ τ , τ ] . By construction of ¯ G n , we have Gr y | [ τ ,τ ] , Gry | [ τ ,τ ] ⊂ ¯ G n . Now, dW − ( t ) dt = 2 e − M n ( t ) r ( t ) ˙ r ( t ) − L n ( t ) W − ( t ) =2 e − M n ( t ) r ( t ) (cid:0) ˙ y ( t ) − a ( t, y ( t ) , u ( t )) + a ( t, y ( t ) , u ( t )) − ˙ y ( t ) (cid:1) − L n ( t ) W − ( t ) ≤ e − M n ( t ) || r ( t ) || E Z U ( t ) R ( t, u ) η ( t )( du ) + 2 L n ( t ) W − ( t ) − L n ( t ) W − ( t ) ≤ e − M n ( t ) W − ( t ) Z U ( t ) R ( t, u ) η ( t )( du ) ≤ e − M n ( t ) W − ( t ) d L w [ η ]( t ) dt . Since function W − is nonnegative, for a. a. t ∈ { t ∈ [ τ , τ ] | W − ( t ) = 0 } we obtain dW − ( t ) dt ≤ e − M n ( t ) d L w [ η ]( t ) dt ≤ e − M n ( τ ) d L w [ η ]( t ) dt . (19)This inequality is trivial for [ τ , τ ] ∋ t < sup { t ∈ [ τ , τ ] | W − ( t ) = 0 } ; whence, || κ ( τ , y ( τ )) − κ ( τ , y ( τ )) || E ( ) ≤ e M n ( τ ) || y ( τ ) − y ( τ ) || E = e M n ( τ ) W − ( τ ) ( ) ≤ e M n ( τ ) − M n ( τ ) (cid:0) L w [ η ]( τ ) − L w [ η ]( τ ) (cid:1) . But κ ( τ , y ( τ )) = y (0) = κ ( τ , y ( τ )) = κ ( τ , y ( τ )) , hence, we have || κ ( τ , y ( τ )) − κ ( τ , y ( τ )) || E ≤ e M n ( τ ) − M n ( τ ) (cid:0) L w [ η ]( τ ) − L w [ η ]( τ ) (cid:1) . (20)Fix arbitrary t ∈ [0 , T ]. For each ε ∈ R > we can split interval [0 , t i into theintervals of the form [ τ ′ , τ ′′ i such that M n ( τ ′′ ) − M n ( τ ′ ) = R [ τ ′ ,τ ′′ i L n ( t ) dt < ε and[ τ ′ , τ ′′ i ⊂ [ n − , n i for the certain n ∈ N . But, (20) holds for every interval, i.e., || κ ( τ ′′ , y ( τ ′′ )) − κ ( τ ′ , y ( τ ′ )) || E ≤ e ε (cid:0) L w [ η ]( τ ′′ ) − L w [ η ]( τ ′ ) (cid:1) . Summing for all intervals, by κ (0 , y (0)) = y (0) and by the triangle inequality,we obtain || κ ( t, y ( t )) − y (0) || E ≤ e ε L w [ η ]( t ) for every t ∈ [0 , T ]. Arbitrariness of ε ∈ R > completes the proof of the proposition. ⊓⊔ ECESSARY CONDITIONS IN CASE OF STABLE ADJOINT VARIABLE 15 E can be represented in the form E = E p × E q for some finite-dimensionalEuclidean subspaces E p and E q . Let us denote the projections of the map a to thesubspaces E p and E q by b and c , respectively. Now, the system (3) can be writtenin the form˙ p = b ( t, p, q, u ) , ˙ q = c ( t, p, q, u ) , ( p, q )(0) = ξ ∈ E, u ∈ U ( t ); (21)Then, it is possible to say that for all η ∈ e U , the set e A [ η ] contains pairs of functions( p, q ) ∈ C loc ( T , E p ) × C loc ( T , E q ). For every ξ ∈ E , let us denote by y ξ △ = ( p ξ , q ξ ) ∈ e A [ e u ] the unique solution of (16) for τ ∗ = 0 , y ∗ = ξ. Definition 2
Consider a closed set G ⊂ E and ξ ∈ G . We say that the solution y ξ of equation ( ) has Lyapunov stable component p ξ in domain G if for each ε ∈ R > there exists δ ( ε, y ) ∈ R > such that for each ξ ′ ∈ G from || ξ ′ − ξ || E < δ ( ε, y ) itfollows that || p ξ ′ ( s ) − p ξ ( s ) || E < ε for all s ∈ T . Proposition 4
Assume ( u ) , ( a ) holds. Suppose there is a closed set G ⊂ E and acompact K ∈ ( comp )( G ) such that for each ξ ∈ K the solution y ξ of equation ( ) has Lyapunov stable component p ξ in G .Then, for each ε ∈ R > , there exists a number δ ∈ R > such that for all η ∈ e U , y =( p, q ) ∈ e A [ η ] from y (0) ∈ K , || L w [ η ] || C < δ , and κ ( t, y ( t )) ∈ G for all t ∈ T , itfollows that || p − p y (0) || C < ε. Proof.
Consider a compact K > △ = { ξ ∈ G | ∃ ξ ∈ K || ξ − ξ || E ≤ } . Toeach t ∈ T , let us assign the set G ( t ) △ = { y ( t ) | η ∈ e U , y ∈ e A [ η ] , y (0) ∈ K > } . Theobtained map G is compact-valued and continuous; in particular, its graph isclosed. Now we can use Proposition 3 for the multi-valued map G and fix theweight w ∈ ( Null )( u ) which exists by this Proposition.Define M ( ξ ′ , ξ ′′ ) △ = sup t ∈ T || p ξ ′ ( t ) − p ξ ′′ ( t ) || E p ∈ T ∪ { + ∞} ∀ ξ ′ , ξ ′′ ∈ K > . For all ξ ∈ K , the stability of the component p ξ implies that the map M is finiteand continuous at the point ( ξ, ξ ) ∈ K > × K > . Fix an ε ∈ R > ; choose for every ξ ∈ K its δ ( ε/ , y ξ ) ∈ h , / δ ( ε/ , y ξ ) − neighborhood of the point ( ξ, ξ ) (in K > × K > ). Fromthe obtained cover of the diagonal ∆ of the set K × K , let us select a finitesubcover; it induces certain open neighborhood Υ of the diagonal ∆ . Let δ ( K ) bethe minimum distance from the diagonal ∆ to the boundary of the neighborhood Υ .Now, for all ξ ′ ∈ K > , ξ ∈ K from || ξ ′ − ξ || E < δ ( K ) it follows that ( ξ ′ , ξ ) ∈ Υ ; i.e.,for some ξ ′′ ∈ K we have M ( ξ, ξ ′′ ) , M ( ξ ′′ , ξ ′ ) < ε/
2, whence M ( ξ, ξ ′ ) < ε . Thus,( || ξ ′ − ξ || E < δ ( K )) ⇒ ( || p ξ ′ − p ξ ) || C < ε ) ∀ ξ ∈ K , ξ ′ ∈ K > . (22)Suppose the η ∈ e U , y = ( p, q ) ∈ e A [ η ] satisfy L w [ η ]( t ) < δ, ξ ( t ) △ = κ ( t, y ( t )) ∈ G for all t ∈ T . For K ⊂ K > = G (0), the definition T △ = sup { T ∈ T | Gr ξ | [0 ,t ] ⊂ K > ∀ t ∈ [0 , T i} ∈ T ∪ { + ∞} is correct, although T can be infinite. Hence, we have Gr y | [0 ,t ] ⊂ Gr G for all t ∈ [0 , T i . Now, from Proposition 3, we obtain || ξ ( t ) − y (0) || E = || κ ( t, y ( t )) − y (0) || E ≤ L w [ η ]( t ) < δ ( K ) ∀ t ∈ [0 , T i . (23)For every t ∈ [0 , T i , let us substitute ξ = y (0) , ξ ′ △ = ξ ( t ) ∈ K > in (22); from theequality p ξ ( t ) ( t ) = p ( t ) we obtain || p ( t ) − p y (0) ( t ) || E < ε for all [0 , T i . To concludethe proof, it remains to prove that T = ∞ . Suppose T ∈ T ; by construction of T , for each τ ∈ h T , ∞i , we have Gr ξ | h T ,τ ] Gr K > ; but Gr ξ ⊂ G . Then, ρ ( ξ ( T ) , G \ K > ) = 0, and, in partic-ular, by construction of K > , we have || ξ ( T ) − y (0) || E ≥
1. However, passing to thelimit in (23) yields || ξ ( T ) − y (0) || E ≤ δ ( K ) ≤ /
2. The acquired contradictionproves that T = ∞ . ⊓⊔ Condition ( ∂ ) : for the maps ( t, x ) ∈ T × X × U → f ( t, x, u ) ∈ X and ( t, x ) ∈ T × X × U → g ( t, x, u ) ∈ R on their respective domains, there exist partial derivativesin x that are integrally bounded (on each compact) locally Lipshitz continuousCarath´eodory maps.Under this condition, the set ∂ x H ( x ( t ) , t, u ( t ) , λ, ψ ( t )) is also a single-elementset, therefore system (10a)–(10b) can be rewritten for u = u in the form˙ ψ ( t ) = − ∂ H ∂x ( x ( t ) , t, u ( t ) , λ, ψ ( t )) , (24a)˙ x ( t ) = f ( t, x ( t ) , u ( t )) , (24b)˙ λ = 0 . (24c) Corollary 1
Assume conditions ( u ) , ( fg ) , ( ∂ ) hold. Let the pair ( x , u ) ∈ C loc ( T , X ) × U be uniformly overtaking optimal for problem ( ) – ( ) . If for eachsolution ( ψ , x , λ ) of system ( ) – ( ) with initial conditions from K △ = D × [0 , × { X } the component ψ is partially Lyapunov stable in G △ = X × [0 , × X .Then, the result of Proposition 2 holds.Proof. In (21), it is sufficient to define E p △ = X , E q △ = X × R and to take ψ and ( x, λ ) for p and q = ( q , q ), and to take for b and c the right-hand sidesof (24a) and (24b)–(24c), respectively. Now Proposition 4 guarantees ( ψ ), i.e., allconditions of Proposition 2 are met. This proves Corollary 1. ⊓⊔ For G , we can take the image { κ ( t, ψ ( t ) , x ( t ) , λ ) | ( x, u, λ, ψ ) ∈ e Y , t ∈ T } . Using Proposition 3 and Remark 4 instead of Proposition 4 and Corollary 1 inthis proof, we obtain
Remark 6
Under conditions ( u ) , ( fg ) , ( ∂ ), for each uniformly overtaking optimalpair ( x , u ) ∈ C loc ( T , X ) × U for problem (1b), for each unbounded increasingsequence ( τ n ) n ∈ N ∈ T N , we have constructed the sequence ( x n , η n , λ n , ψ n ) n ∈ N ∈ e Y N such that ECESSARY CONDITIONS IN CASE OF STABLE ADJOINT VARIABLE 17
1) this sequence (as a sequence from C loc ( T , X ) × e U × T × C loc ( T , X )) convergesto a certain ( x , e u , λ , ψ ) ∈ Z ;2) the graphs Gr ( x n , λ n , ψ n ) of its elements are contained within the thinningfunnels of solutions of system (24a)–(24c); i.e., for a sequence ( δ n ) n ∈ N ∈ R N ≥ that tends to 0, we have κ ( t, ( ψ n , x n , λ n )) ∈ ( ψ (0) , , λ ) + δ n D × δ n D × [ − δ n , δ n ] ∀ t ∈ T , n ∈ N . e J t n ( η ) − J t n ( u ) →
0, and ψ n ( t n ) = 0 for each n ∈ N , where ( t n ) n ∈ N is a certainsubsequence of ( τ n ) n ∈ N ∈ T N . ψ is not stable, but we know thatcertain components of the vector variable ψ are stable, or we know the rate of itsgrowth. Then we can try to select the mapping A ∗ : T → L , may help to modifycondition (15b), and use the conditionlim inf t →∞ || ψ ( t ) A ∗ ( t ) || X = 0 (25a)for certain map A ∗ : T L . Here are the examples of such maps A ∗ : one that maps the unity matrix A ∗ ( t ) ≡ L ; some “scalar” multiplier A ∗ ( t ) ≡ r ( t )1 L ; a mapping A ∗ ( · ) ≡ D withthe diagonal matrix D ; the condition ψ ( t ) x ( t ) →
0, which is often used as thesufficient condition, can also be reduced to this form.Let us assume that for all η ∈ e U , ξ ∈ X , we have chosen the measurable mapping A ηξ : T → L . Assume A ηξ (0) = 1 L for all η ∈ e U , ξ ∈ X . Define A ∗ △ = A e u X . Condition ( ψA ): There exists a weight w ∈ ( Null )( u ) such that for everysolution ( x , u , λ , ψ ) ∈ Z for each ε ∈ R > there exist a number δ ∈ R > and aneighborhood Υ ⊂ C loc ( T , X ) × e U × [0 , × C loc ( T , X ) of the solution ( x , e u , λ , ψ )such that for every solutions z △ = ( x, η, λ, ψ ) ∈ Υ ∩ e Y , from || L w [ η ] || C < δ it followsthat || ψA ηx (0) − ψ A e u X || C < ε . Proposition 5
Assume conditions ( u ) , ( fg ) hold. For each uniformly overtaking op-timal pair ( x , u ) ∈ C ( T , X ) × U satisfying ( ψ A ) , for each unbounded increasingsequence ( τ n ) n ∈ N ∈ T N there exists ( x , u , λ , ψ ) ∈ Z such that lim inf n →∞ || ψ ( τ n ) A ∗ ( τ n ) || X = 0 . (25b) hold. The only differences between the proof of this Proposition and Proposition 2are the facts that the references to ( ψ ) are replaced with references to ( ψA ) andthe factors A ηξ , A ∗ are added to the inequalities of the last strings.Similarly, we can formulate an analogue to Corollary 1 for this condition: if itis possible to choose matrix maps such that the product ψA uz (0) is the solution ofan equation dpdt = b ( t, p ( t ) , ψ ( t ) , x ( t ) , λ, u ( t )) (26) for each u ∈ U for each solution z = ( ψ, x, λ ) of system (10b),(10a),(24c) withinitial conditions z (0) ∈ E q , then the corresponding stability of this component p in system (26),(24a)–(24c) implies the result of Corollary 1 (see [24]).The simplest way to account for the a priori information on stability or forasymptotic estimates of ψ and its components is to take A ∗ ( t ) △ = e − λt L , where λ is greater than or equal to all Lyapunov’s exponents of the variable ψ . In particular,in [28, Example 10.2], the use of A ∗ ( t ) △ = e − t L in such condition (in contrast tothe standard condition) selects the single extremal. In the papers [1, 2, 3, 4, 5], Aseev and Kryazhimskii have proposed and proven theanalytic expression for the values of the adjoint variables. This version of thenormal form of the Maximum Principle holds with the explicitly specified adjointvariable providing a complete set of necessary optimality conditions; moreover, thesolution of this form of Maximum Principle is uniquely determined by the optimalcontrol. This approach generalizes (see [4, Sect. 16], [7]) a number of transversalityconditions; in particular, it is more general than the conditions that were obtainedfor linear systems in [8].It turns out that if the function A ∗ is fundamental matrix of linearized sys-tem along the optimal trajectory, then, condition (25a) automatically yields thisexplicit representation for the adjoint variable.Let us simplify Proposition 5 for such A ∗ to weaken the requirements of [4,Theorem 12.1],[7, Theorem 1],[5, Theorem 2], and their corollaries.5.1 The case of dominating discountLet the pair ( x , u ) ∈ C loc ( T , X ) × U be uniformly overtaking optimal for prob-lem (1a)–(1b). Along with it, let us consider the solution A ∗ of the Cauchy problem dA ∗ ( t ) dt = ∂f ( t, x ( t ) , u ( t )) ∂x A ∗ ( t ) , A ∗ (0) = 1 L . Likewise, for each ξ ∈ X , let us denote by x ξ the solution of (10a) for the initialcondition x ξ (0) = ξ ∈ X ; let us also consider A ξ , the solution of the matrix Cauchyproblem dA ξ ( t ) dt = ∂f ( t, x ξ ( t ) , u ( t )) ∂x A ξ ( t ) , A ξ (0) = 1 L ∀ ξ ∈ X . For each T ∈ T , let us consider I ξ ( T ) △ = Z T ∂g ( t, x ξ ( t ) , u ( t )) ∂x A ξ ( t ) dt. Proposition 6
Assume conditions ( u ) , ( fg ) , ( ∂ ) . Let the pair ( x , u ) ∈ C loc ( T , X ) × U be uniformly overtaking optimal for problem ( ) – ( ) . Let the map I be boundedand let lim ξ → || I ξ − I || C = 0 . ECESSARY CONDITIONS IN CASE OF STABLE ADJOINT VARIABLE 19
Let I ∗ ∈ X be a partial limit (the limit of a subsequence) of I ( τ ) as τ → ∞ .Then, there exists a solution ( x , u , λ , ψ ) ∈ Z of all relations of the MaximumPrinciple ( ) – ( ) satisfying the transversality condition ( ) . Moreover, λ △ =11 + || I ∗ || X > , and ψ ∈ C loc ( T , X ) defined by the following rule: ψ ( T ) △ = λ (cid:16) I ∗ − Z T ∂g ( t, x ( t ) , u ( t )) ∂x A ∗ ( t ) dt (cid:17) A − ∗ ( T ) ∀ T ∈ T . (27) Proof.
For each u ∈ U and z △ = ϕ [ u ], let us introduce a matrix function A u thatis the solution of the matrix equation˙ A u ( t ) = ∂f ( t, z ( t ) , u ( t )) ∂x A u ( t ) , A u (0) = 1 L . (28)Now, for each solution ( z, u, λ u , ψ u ) ∈ Y from (11b) it follows that ddt (cid:0) ψ u A u (cid:1) ( t ) = − λ u ∂g ( t, z ( t ) , u ( t )) ∂x A u ( t ) . (29)For E p △ = X , b ( t, p, ( q , q , q ) , u ) △ = − q ∂g ( t, q , u ) ∂x q , (30) E q △ = L × X × R , c ( t, p, ( q , q , q ) , u ) △ = (cid:16) ∂f ( t, q , u ) ∂x q , f ( t, q , u ) , (cid:17) the system (21) becomes system (29),(28),(10a),(24c); now, for u = u ,˙ p = − r ∂g ( t, z, u ( t )) ∂x B, ˙ B = ∂f ( t, z, u ( t )) ∂x B, ˙ z = f ( t, z, u ( t )) , ˙ r = 0 . (31)Solving this system, we obtain r ( t ) = r (0) , z ( t ) = x z (0) ( t ) , B ( t ) = A z (0) ( t ) B (0) ,p ( t ) = p (0) − r (0) I z (0) ( t ) B (0) . (32)Let G △ = X × L × X × [0 , p in G for B (0) = 1 L , z (0) = 0 , r (0) ∈ [0 , , p (0) ∈ D . Indeed, by (32)it remains to verify that I ξ is continuous at the point ξ = 0, and I is bounded;both hold by assumptions. Therefore, by Proposition 4, for the certain weight w ∈ ( Null )( u ), the component p is stable for L w − small perturbations of thecontrol u . This provides condition ( ψA ) for A u that were defined as we did.By condition, I ∗ is a partial limit; hence, there exists an unbounded increasingsequence ( τ n ) n ∈ N ∈ T N with property I ( τ n ) → I ∗ . Now, by Proposition 5, thereexists ( x , u , λ , ψ ) ∈ Z with properties (25b) and (25a).Substituting z (0) = 0, r (0) = λ , A (0) = 1 L , and p (0) = ψ (0) into (32) yields (cid:0) ψ A ∗ (cid:1) ( T ) = (cid:0) ψ A (cid:1) ( T ) = p ( T ) = ψ (0) − λ I ( T ) ∀ T ∈ T . (33)Now, substituting T = τ n and passing to the lower limit, from (25a), we obtain0 = ψ (0) − λ I ∗ ; therefore, from (33) and (10d) respectively, we have ψ ( T ) A ∗ ( T ) = λ (cid:0) I ∗ − I ( T ) (cid:1) , λ = 11 + || I ∗ || X > . Using the inverse matrix for A ∗ ( T ), we obtain (27). ⊓⊔ Let us note that if I ∗ is independent of choice of the subsequence ( τ n ) n ∈ N , weautomatically obtain the stronger transversality conditionlim t →∞ ψ ( t ) A ∗ ( t ) = 0 . (34)Moreover, since for different ( x , u , λ ), solutions of (29) differ by a constant, for all( x , u , λ, ψ ) ∈ Y , the products ψA tend to a finite limit as t → ∞ . If (27) holds,then this limit is equal to zero. Hence, to every ( x , u , λ ) there corresponds at mostone ψ , for which relations (10a)–(10c), (34) hold; now, from (10d) and (32) wecan reconstruct λ uniquely. Thus there exists the unique solution ( x , u , λ, ψ ) ∈ Z that satisfies condition (34), and the following theorem is proved. Theorem 2
Assume conditions ( u ) , ( fg ) , ( ∂ ) hold. Let the pair ( x , u ) ∈ C loc ( T , X ) × U be uniformly overtaking optimal for problem ( ) – ( ) , and let thelimit lim t →∞ ,ξ → X I ξ ( t ) = Z T ∂g ( t, x ( t ) , u ( t )) ∂x A ∗ ( t ) dt ∈ R be well-defined and finite.Then, there exists the unique solution ( x , u , λ , ψ ) ∈ Z of all relations of theMaximum Principle ( ) – ( ) satisfying the transversality condition ( ) . Moreover,accurate to the positive factor, we can assume λ △ = 1 , ψ ( T ) △ = Z [ T, ∞i ∂g ( t, x ( t ) , u ( t )) ∂x A ∗ ( t ) dt A − ∗ ( T ) ∀ T ∈ T . (35)From conditions of [5, Theorem 2],[4, Theorem 12.1],[3, Theorem 1],[7, Theo-rem 1] it follows that for some α, β ∈ R > and for all admissible controls u , alltrajectories x , and all fundamental matrices A u , the inequality (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) ∂g ( t, x ( t ) , u ( t )) ∂x (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) || A u ( t ) || ≤ βe − αt ∀ t ∈ T (36)holds. This is stronger than the conditions of Theorem 2. Informally, the require-ments of [3, Theorem 1],[5, Theorem 2],[4, Theorem 12.1], [7, Theorem 1] boildown to the need for uniform exponential Lyapunov stability of the product ψA along all trajectories of the system (1), while Lyapunov stability of the product ψ A along the optimal solution of the initial control problem is sufficient for The-orem 2. On the other side, the condition (36) can be verified by calculating theLyapunov exponents of the system of the Maximum Principle, see [4, Sect. 12],[5,Sect. 3],[7, Sect. 5].5.2 The general caseLet us base on the start of the proof of Proposition 6, and let us use not Corollary 1but Remark 4. Proposition 7
Assume conditions ( u ) , ( fg ) , ( ∂ ) hold. Let the pair ( x , u ) ∈ C loc ( T , X ) × U be uniformly overtaking optimal for problem ( ) – ( ) .Now, for an unbounded increasing sequence of times ( τ n ) n ∈ N ∈ T N , there exist: ECESSARY CONDITIONS IN CASE OF STABLE ADJOINT VARIABLE 21
1) its subsequence ( t n ) n ∈ N ∈ T N ;2) the sequence of initial conditions ( ζ n ) n ∈ N ∈ X N that converges to X ;3) the sequence ( λ n ) n ∈ N ∈ [0 , N that converges to some λ ∈ [0 , ;such that if ψ ∈ C ( T , X ) is defined for every t ∈ T by the rule ψ ( T ) = lim n →∞ λ n Z t n T ∂g ( t, x ζ n ( t ) , u ( t )) ∂x A ζ n ( t ) dt A − ∗ ( T ) , then, the limit would be uniform on every compact, and ( x , u , λ , ψ ) n ∈ N ∈ Z wouldsatisfy all relations of the Maximum Principle ( ) – ( ) . Moreover, ψ ( T ) = lim n →∞ λ n Z t n T ∂g ( t, x ζ n ( t ) , u ( t )) ∂x A ζ n ( t ) dt A − ζ n ( T ) ∀ T ∈ T . (37) Proof.
Indeed, consider the control system ( ˙ p, ˙ q ) = ( b, c ) = a from (30). It featuresthe set of controls U , however, as a system of form (3), it defines the controlsystem of form (4) that is controlled by the elements of e U . For such a system, fixthe weight w from the formulation of Proposition 3.By Remark 4, for every ( τ n ) n ∈ N ∈ T N , there exist its subsequence ( t n ) n ∈ N ∈ T N and the sequence ( x n , η n , λ n , ψ n ) n ∈ N ∈ e Y of solutions of system (10a)–(10b),converging to the certain solution ( x , e u , λ , ψ ) ∈ e Z of all relations of the Maxi-mum Principle.Now, for every n ∈ N , we can find B n ∈ C ( T , L ) and p n ∈ C ( T , X ) such that a n △ = ( p n , B n , x n , λ n ) ∈ e A [ η n ] , B n (0) = 1 L , p n (0) = ψ n (0) . (38)On the other side, differentiating ψ n B n (as in (29)), we check that( ψ n B n , B n , x n , λ n ) ∈ e A [ η n ]. Comparing the initial conditions, we see that p n ≡ ψ n B n . For each n ∈ N , for each t ∈ T , there exists the solution a n,t ∈ C ( T , E ) of(31) for the initial conditions a n,t (0) = κ ( t, a n ( t )) . Note that the last componentsof a n,t and a n are independent of t ; thus, they correspond with λ n . Now we cancorrectly define the components of the map t κ ( t, a n ( t )) by the rule κ ( t, a n ( t )) = (cid:0) ν n ( t ) , µ n ( t ) , ξ n ( t ) , λ n (cid:1) ∀ t ∈ T , n ∈ N . Substituting these initial conditions into (32), by virtue of equalities (38) and a n ( t ) = a n,t ( t ) for all n ∈ N , t ∈ T , we obtain (cid:0) ψ n ( t ) B n ( t ) , B n ( t ) , x n ( t ) (cid:1) = a n,t ( t ) = (cid:0) ν n ( t ) − λ n I ξ n ( t ) µ n ( t ) , A ξ n ( t ) µ n ( t ) , x ξ n ( t ) (cid:1) . Specifically, ψ n ( t ) A ξ n ( t ) ( t ) = ψ n ( t ) B n ( t ) µ − n ( t ) = ν n ( t ) µ − n ( t ) − λ n I ξ n ( t ) ( t ) . Remark 4 provides ψ n ( t n ) = 0; substituting t = t n , we obtain0 = ψ n ( t n ) A ξ n ( t n ) ( t n ) = ν n ( t n ) µ − n ( t n ) − λ n ( t n ) I ξ n ( t n ) ( t n ) . Substracting one from another yields ψ n ( t ) A ξ n ( t ) ( t ) = ν n ( t ) µ − n ( t ) − λ n I ξ n ( t ) ( t ) − ν n ( t n ) µ − n ( t n ) + λ n I ξ n ( t n ) ( t n ) . (39) By Remark 4, we have a n (0) ( ) = ( ψ n (0) , L , X , λ n ) → ( ψ (0) , L , X , λ ) , and || L w ( η n ) || C → n → ∞ ; moreover, Proposition 3 yields for all t ∈ T max t ∈ T || a t,n (0) − a n (0) || E = max t ∈ T || κ ( t, a n ( t )) − κ (0 , a n (0)) || E ≤ || L w ( η n ) || C → . Hence uniformly on the whole T as n → ∞ , it holds that a t,n (0) = (cid:0) ν n ( t ) , µ n ( t ) , ξ n ( t ) , λ n (cid:1) → ( ψ (0) , L , X , λ ) ∀ t ∈ T. (40)Whence the theorem of continuous dependence on initial conditions yields theuniformity of the limitslim n →∞ ν n ( τ ) µ − n ( τ ) = ψ (0) , lim n →∞ λ n I ξ n ( τ ) ( t ) = λ I ( t ) ∀ t ∈ K, τ ∈ T . as n → ∞ for each compact K ∈ ( comp )( T ). Putting here τ = t, τ = t n , let usconsider the limit of both sides of (39) as n → ∞ ; thus, ψ ( t ) A ( t ) = lim n →∞ (cid:16) ψ (0) − λ I ( t ) − ψ (0) + λ n I ξ n ( t n ) ( t n ) (cid:17) =lim n →∞ (cid:16) − λ n I ξ n ( t n ) ( t ) + λ n I ξ n ( t n ) ( t n ) (cid:1) = lim n →∞ λ n (cid:0) I ξ n ( t n ) ( t n ) − I ξ n ( t n ) ( t ) (cid:1) . Multiplying on the right by A − ( t ) = A − ∗ ( t ) and A ( t ) A ξ n ( t n ) ( t ) we obtain ourproposition for ζ n △ = ξ n ( t n ). All necessary convergences are provided by uniformityof limits in (40). ⊓⊔ We say an optimal pair ( x , u ) ∈ C loc ( T , X ) × U for problem (1a)–(1b) is abnormal if every solution ( x , u , λ , ψ ) ∈ Z of all relations of the MaximumPrinciple (10a)–(10d) satisfies λ = 0 . Remark 7
Assume conditions ( u ) , ( fg ) , ( ∂ ). Let the pair ( x , u ) ∈ C loc ( T , X ) × U be uniformly overtaking optimal pair for problem (1a)–(1b) and let this pair beabnormal. Then,lim sup τ →∞ ,ξ → || I ξ ( τ ) || E = lim sup τ →∞ ,ξ → (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) Z τ ∂g ( t, x ξ ( t ) , u ( t )) ∂x A ξ ( t ) dt (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) E = ∞ . Indeed, if it is wrong, then, the right-hand side of (37) equals zero for T = 0, i.e., ψ (0) = 0 X , which contradicts the relation (10d) for λ = 0.5.3 Monotonous caseConsider the case of when both the right-hand side of the dynamics equation andthe objective function are monotonous. This case frequently arises in economicalapplications while monotonicity simplifies its examination. It seems that the firstto note the peculiarities of this case and to investigate it were Aseev, Kryazhimskii,and Taras’ev in their paper [6]. These were followed by papers [35],[1],[2], and themost general case was considered in [4].In Euclidean space E ′ , let us define binary relations < and ≻ by the rules( α < β ) ⇔ ( α − β ∈ T dimE ) , ( α ≻ β ) ⇔ ( α − β ∈ R dimE> ) ∀ α, β ∈ E ′ . This allows us to use the symbols < and ≻ to compare vectors and matrices, andvector and matrix functions. For the latter two, < and ≻ allow us to discuss theirmonotonicity. ECESSARY CONDITIONS IN CASE OF STABLE ADJOINT VARIABLE 23
Proposition 8
Assume conditions ( u ) , ( fg ) , ( ∂ ) hold. Let the pair ( x , u ) ∈ C loc ( T , X ) × U be uniformly overtaking optimal for problem ( ) – ( ) . Assume forall x ∈ X and for a.a. t ∈ T there exists a number d ( t, x ) ∈ R such that the followingrelation holds: ∂g ( t, x, u ( t )) ∂x < L , ∂f ( t, x, u ( t )) ∂x < d ( t, x )1 L . Then, there exists a solution ( x , u , λ , ψ ) ∈ Z of all relations of the MaximumPrinciple ( ) – ( ) satisfying ( ) , and ψ < X . If at the same time the pair ( x , u ) is normal, then λ lim sup t →∞ ,ξ → I ξ ( t ) < ψ (0) < λ lim t →∞ I ( t ) < X (41) hold, and all limits in ( ) well-defined and finite. Corollary 2
Assume conditions ( u ) , ( fg ) , ( ∂ ) hold. Let the pair ( x , u ) ∈ C loc ( T , X ) × U be uniformly overtaking optimal for problem ( ) – ( ) , and let this pairbe normal. Assume for all x ∈ X and for a.a. t ∈ T there exists a number d ( t, x ) ∈ R such that the following relation holds: ∂g ( t, x, u ( t )) ∂x ≻ L , ∂f ( t, x, u ( t )) ∂x ≻ d ( t, x )1 L . Then, there exists a solution ( x , u , λ , ψ ) ∈ Z of all relations of the MaximumPrinciple ( ) – ( ) satisfying ( ) , ( ) , and ψ ≻ X . Proof.
Below, in the proof of Proposition 8, we understand the symbol ⊲ as < , andin the proof of Corollary 2, we understand it as ≻ .Fix arbitrary ξ ∈ X , T ∈ R > , τ ∈ h T, ∞i ; let us show that A ξ ( τ ) A − ξ ( T ) ⊲ L .Denote by F ξ ( t ) the matrix ∂f ( t,x ξ ( t ) ,u ( t )) ∂x for all t ∈ [ T, τ ]. The diagonal of themap F ξ is dominated by a function M △ = M F ξ [ T,τ ] ∈ L loc ( T , T ); then, by condition, F ξ + m ( t )1 L | h T,τ ] ⊲ L . Now, let us consider a solution P ( t ) of the equation˙ P = ( F ξ ( t ) + M ( t )1 L ) P, P ( T ) = 1 L , t ≥ T ;for it, it holds that P ( t ) ⊲ L for all t ∈ h T, τ ] . Since A ξ and 1 L commute, thesolution P is the product of two solutions of the equations ˙ Q = F ξ ( t ) Q, Q ( T ) = 1 L ,and ˙ R = M ( t )1 L R, R ( T ) = 1 L . Thus, P ( τ ) = Q ( τ ) R ( τ ) = Q ( τ ) e R τT M ( t ) dt L = A ξ ( τ ) A − ξ ( T ) e R τT M ( t ) dt , and P ( τ ) ⊲ L implies A ξ ( τ ) A − ξ ( T ) ⊲ L for all τ ∈ h T, ∞i . Now, by monotonicityof matrix product, we obtain dI ξ ( t ) dt A − ξ ( T ) = ∂g ( t, x ξ ( t ) , u ( t )) ∂x A ξ ( t ) A − ξ ( T ) ⊲ X ∀ t ∈ h T, ∞i (42)for all ξ ∈ X , T ∈ T ; specifically, for T = 0 we have dI ξ ( t ) dt ⊲ X , hence the functions I ξ , I ξ A − ξ ( T ) are monotonically increasing for all ξ ∈ X , T ∈ T . By Proposition 7, there exists the solution ( x , u , ψ , λ ) of relations of theMaximum Principle satisfying of formula (37) for certain sequences λ n and ξ n .However, the expression into the limit of (37) lies in L < by (42). Passing to thelimit as n → ∞ , we obtain ψ < X . Suppose the pair ( x , u ) is normal; then λ >
0. Since the function I ξ ismonotonically increasing and, by Remark 7, uniformly bounded in the certainneighborhood 0 X , the Lebesgue theorem yields the existence of the finite limitsin (41). Hence, λ lim sup t →∞ ,ξ → I ξ ( t ) < λ lim n →∞ I ζ n ( t n ) ( ) = ψ (0) . On the other side, monotonicity of I ξ A − ξ ( T ) yields1 λ ψ ( T ) ( ) = lim n →∞ (cid:0) I ζ n ( t n ) − I ζ n ( T ) (cid:1) A − ζ n ( T ) ( ) ⊲ lim n →∞ (cid:0) I ζ n ( t ) − I ζ n ( T ) (cid:1) A − ζ n ( T )= (cid:0) I ( t ) − I ( T ) (cid:1) A − ( T ) ( ) ⊲ X ∀ T ∈ T , t ∈ h T, ∞i , i.e. ψ ⊲ X . Moreover, substituting T = 0 and passing to the limit as t → ∞ , weobtain the lower estimate from (41). ⊓⊔ Note that in [6, Theorem 1], [4, Theorem 10.1] the estimate ψ < X ( ψ ≻ X )is made for autonomous systems under less general assumptions; in [4, Theorem10.1], the lower estimate from (41) was made too (see [4, (10.17)]). However, inthese papers, the condition λ > ∂g∂x ([3, Theorem 1],[4, Theorem 11.1]). It seems, this result is not adirect consequence of Theorem 2, which was proved in that paper. In the paper, the left endpoint is considered to be fixed. It seems this conditionmay be easily discarded, since to do it, it is sufficient to equip the finite horizonoptimization problems from the proof of Theorem 1 with the same condition forthe left endpoint and to provide the boundedness of x (0).6.1 Case of σ -compact-valued map U The condition ( u ) implies that at every time t ∈ T , the controls are chosen fromthe compact U ( t ). Let us weaken this assumption to the following: Condition ( u σ ) : U is a σ -compact-valued map such that Gr U ∈ B ( T × U ).We shall still assume the conditions ( a ) , ( fg ) to hold, and we shall not changethe definition of U . Then, we can assume there exists a nondecreasing sequence( U ( r ) ) r ∈ N of integrally bounded (on each compact subset of T ) compact-valuedmaps such that U ≡ ∪ r ∈ N U ( r ) . Let us assume the uniformly overtaking optimalcontrol u exists. Then, we may safely assume that Gru ⊂ GrU (1) . ECESSARY CONDITIONS IN CASE OF STABLE ADJOINT VARIABLE 25
Repeating the reasonings of Sect 2, for each r ∈ N , we can construct sets U ( r ) , e U ( r ) and their images for the restriction: U ( r ) n △ = π n ( U ( r ) ) , e U ( r ) n △ = e π n ( e U ( r ) ) . Let us introduce the set e U of all maps η from T into the set of Radon probabilitymeasures over U such that for every n ∈ N there exists r = r ( η, n ) ∈ N such that π n ( η ) = η | [0 ,n ] ∈ e U ( r ) n . The topology of this set is of no use to us, thus we assumeit is indiscrete. Note that under our definition, e δ ( U ) e U , but u ∈ e δ ( U ( r ) ) ⊂ e δ ( U )for all r ∈ N Note that, for all η ∈ e U , the set e A [ η ] is still compact. Indeed, for each n ∈ N ,it holds that e A [ η ] | [0 ,n ] ⊂ e A [ e U ( r ( η,n )) ] | [0 ,n ] ∈ ( comp )( C ([0 , n ] , E )); all that remainsis to use the definition of compact-open topology. For each r ∈ N , denote by Z ( r ) the pairs ( ψ, λ ) ∈ C ( T , X ) such that ( x , u , ψ r , λ r ) satisfies relations (10a)–(10b),(10d), and for a.a. t ∈ T instead of (10c), there holds the weaker relationsup p ∈ U ( r ) ( t ) H (cid:0) x ( t ) , t, p, λ, ψ ( t ) (cid:1) = H (cid:0) x ( t ) , t, u ( t ) , λ, ψ ( t ) (cid:1) . (43)Note that this set is compact (it follows from compactness of e A [ η ]).By Theorem 1, Z ( r ) is not empty for all r ∈ N . It is easily seen that Z ( r ′ ) ⊂ Z ( r ′′ ) for any r ′ , r ′′ ∈ N , ( r ′ ⊂ r ′′ ). Then, there exists ( ψ , λ ) ∈ ∩ r ∈ N Z ( r ) . Therefore, for it, (43)holds for all r ∈ N ; thus, (10c) holds too; hence, ( x , u , ψ , λ ) satisfies all relations(10a)–(10d) of the Maximum Principle.For each r ∈ N , consider the sequences ( x nr , η nr , λ nr , ψ nr ) n ∈ N , ( t n,r ) n ∈ N from Re-mark 4. Then, for the sequence ( x nn , η nn , λ nn , ψ nn ) n ∈ N , by uniformity of estimate (14),we have pointwise convergence of η n to e u ; moreover, for each k ∈ N in the interval[0 , k ] for this sequence, the convergences from Remark 4 hold (it is sufficient toconsider there topologies with respect to C ([0 , k ] , X ), e U ( k ) ), specifically, it wouldhold that1) ( x n , η n , λ nn , ψ nn ) → ( x , e u , λ , ψ ) n ∈ N ∈ C loc ( T , X ) × e U × T × C loc ( T , X );2) || L w ( η n ) || C → e J t n ( η ) − J t n ( u ) →
0, and ψ nn ( t n ) = 0 for each n ∈ N , where t n △ = t n,n .The verbatim repetition of the proof of Proposition 2 yields Proposition 9
Assume conditions ( u σ ) , ( fg ) . For each uniformly overtaking optimalpair ( x , u ) ∈ C loc ( T , X ) × U satisfying ( ψ ) for each unbounded increasing sequence ( τ n ) n ∈ N ∈ T N there exists ( x , u , λ , ψ ) such that the relations of the MaximumPrinciple ( ) – ( ) , and the transversality condition ( ) hold. Since our case is more general, starting with Sect. 4, the references to ( u ) oughtto be replaced with ( u σ ), and the results of Sect. 3 ought to be replaced with theirrespectful analogues.6.2 On uniformly sporadically catching up controls. Definition 3
We say that a control u ∈ U is uniformly sporadically catching upoptimal if for every ε, T ∈ R > there exists t ∈ [ T, ∞i such that J t ( u ) ≥ J t ( u ) − ε holds for all u ∈ U . Note that for each uniformly sporadically catching up optimal control, there existsan unbounded monotonically increasing sequence ( τ n ) n ∈ N ∈ T N and a function ω ∈ Ω such that J τ n ( u ) ≥ J τ n ( u ) − ω ( τ k ) ∀ u ∈ U , k, n ∈ N , k < n. We call such control a τ -sporadically catching up optimal.Now, if we consider the sequence ( τ n ) n ∈ N everywhere defined and understandthe optimality in the above sense, then all statements, starting with Theorem 1,hold. In particular, we can rewrite Proposition 6 and Theorem 2 in the followingway: Theorem 3
Assume conditions ( u σ ) , ( fg ) , ( ∂ ) hold. Let the pair ( x , u ) ∈ C loc ( T , X ) × U be τ -sporadically catching up optimal for problem ( ) – ( ) . Let I ∗ ∈ X be the limit of I ξ ( τ n ) as n → ∞ , ξ → X . Then, there exists the unique solution ( x , u , λ , ψ ) of all relations of the Maxi-mum Principle ( ) – ( ) and the transversality condition ( ) . Moreover, accurateto the positive factor, we can assume λ △ = 1 , ψ ( T ) △ = (cid:16) I ∗ − Z T ∂g ( t, x ( t ) , u ( t )) ∂x A ∗ ( t ) dt (cid:17) A − ∗ ( T ) ∀ T ∈ T . Acknowledgements
I would like to express my gratitude to A. G. Chentsov, A. M. Tarasyev,N. Yu. Lukoyanov, and Yu. V. Averboukh for valuable discussion in course of writing thisarticle. Special thanks to Ya. Salii for the translation.
References S. M. Aseev, A. V. Kryazhimskii,
The Pontryagin Maximum Principle for Infinite-HorizonOptimal Controls, IIASA Interim Rept. IR-03-013 (Laxenburg, Austria), 2003.2.
S. M. Aseev, A. V. Kryazhimskii,
The Pontryagin Maximum Principle and transversalityconditions for a class of optimal control problems with infinite time horizons, SIAM J.Control Optim. 43, 1094-1119, 2004.3.
S. M. Aseev, A. V. Kryazhimskii,
The Pontryagin Maximum Principle for an OptimalControl Problem with a Functional Specified by an Improper Integral, Dokl. Math. 69, 1,89-91, 2004.4.
S. M. Aseev, A. V. Kryazhimskii,
The Pontryagin Maximum Principle and problems ofoptimal economic growth, Proc. Steklov Inst. Math. 257, no. 1, 1-255, 2007.5.
S. M. Aseev, A. V. Kryazhimskii,
Shadow prices in infinite-horizon optimal control prob-lems with dominating discounts, Applied Mathematics and Computation V.204, 2, 519-531, 20086.
S. M. Aseev, A. V. Kryazhimskii, A. M. Tarasyev,
The Pontryagin Maximum Principleand Transversality Conditions for an Optimal Control Problem with Infinite Time Interval,Proc. Steklov Inst. Math. 233, 64-80, 2001.7.
S. M. Aseev, V. M. Veliov,
Maximum Principle for infinite-horizon optimal control; prob-lems with dominating discount, Dynamics of Continuous, Discrete and Impulsive Systems(in appear?)8.
J. P. Aubin, F. H. Clarke,
Shadow Prices and Duality for a Class of Optimal ControlProblems, SIAM J. Control Optim. 17, 567-586, 1979.9.
E. J. Balder,
An existence result for optimal economic growth problems, J. of Math.Anal.95, 1, 195-213, 1983.10.
L. M. Benveniste, J. A. Scheinkman,
Duality theory for dynamic optimization modelsof economics: The continuous tim e case, Journal of Economic Theory, 27, 1-19, 1982.ECESSARY CONDITIONS IN CASE OF STABLE ADJOINT VARIABLE 2711.
D. A. Carlson,
Uniformly overtaking and weakly overtaking optimal solutions in infinite-horizon optimal control: when optimal solutions are agreeable, J. Optim. Theory Appl.,64, 1, 55-69. 1990.12.
D. A. Carlson,
Nonconvex and Relaxed Infinite-Horizon Optimal Control Problems, J.Optim. Theory Appl. 78, 3, 465-491, 1993.13.
D. A. Carlson, A. B. Haurie, A. Leizarowitz,
Infinite Horizon Optimal Control. Deter-ministic and Stochastic Systems. Springer, Berlin, 1991.14.
F. H. Clarke,
Optimization and Nonsmooth Analysis. Wiley, New York, 1983.15.
H. W. Daniel,
Survey of measurable selection theorems: an update // Lect.Notes Math.,794, Springer, 176-219, 1980.16.
A. V. Dmitruk, N. V. Kuz’kina,
An existence theorem in an optimal control problem onan infinite time interval, Math. Notes 78, 3-4, 466-480, 2005.17.
R. Engelking,
General Topology, PWN, Warszawa, 1983.18.
V. V. Fedorchuk, V. V. Filippov,
General topology. Basic Constructions. (in Russian)Fizmatlit, Moscow, 2006.19.
R. V. Gamkrelidze,
Principles of Optimal Control Theory. Plenum Press, New York, 1978.20.
H. Halkin,
Necessary Conditions for Optimal Control Problems with Infinite Horizons,Econometrica 42, 267-272, 1974.21.
P. J. Hammond, J. Kennan,
Uniformly optimal infinite-horizon plans, International Eco-nomic Review, 20, 283-296, 1979.22.
T. Kamihigashi,
Necessity of transversality conditions for infinite horizon problems,Econometrica, 69, 995-1012, 2001.23.
D. V. Khlopin
On extension of conflict control problems on infinite horizon (In Russian),Vestn. Udmurtsk. Univ. Mat. Mekh. Comp. Nauki, No 1, 105-112, 2011.24.
D. V. Khlopin
Necessary conditions of optimality in the case of stable adjoint variable forinfinite horizon problems with free right endpoint (In Russian), Vestn. Udmurtsk. Univ.Mat. Mekh. Comp. Nauki, (in appear?) 201125.
P. Michel
On the transversality condition in infinite horizon optimal problems, Econo-metrica, 50, 975-984, 1982.26.
L. S. Pontryagin et al.,
The Mathematical Theory of Optimal Processes, Fizmatgiz,Moscow, 1961.27.
N. Sagara,
Value functions and transversality conditions for infinite-horizon optimal con-trol problems // Set-Valued Var. Anal. 18, 1, 1-28, 2010.28.
A. Seierstad,
Necessary conditions for nonsmooth, infinite-horizon optimal control prob-lems, J. Optim. Theory Appl., 103, 1, 201-230, 1999.29.
A. Seierstad, K. Sydsæter,
Conditions implying the vanishing of the Hamiltonian at in-finity in optimal control problems, Optim. Lett., 3, 507-512, 2009.30.
K. Shell,
Applications of Pontryagins maximum principle to economics, MathematicalSystems Theory and Economics 1, Lect. Notes Oper. Res. Math. Econ. vol. 11, Springer,Berlin, 241-292, 1969.31.
G. V. Smirnov,
Transversality condition for infinite-horizon problems, J. Optim. TheoryAppl. 88, 3, 671-688, 1996.32.
L. E. Stern,
Criteria of optimality in the infinite-time optimal control problem, J. Optim.Theory Appl. 44, 3, 497-508, 1984.33.
A. A. Tolstonogov,
Differential inclusions in a Banach space. Mathematics and its Appli-cations, 524. Kluwer Academic Publishers, Dordrecht, 2000.34.
J. Warga,
Optimal Control of Differential and Functional Equations. Academic Press,New York, 1972.35.
T. Weber,
An infinite-horizon maximum principle with bounds on the adjoint variable,Journal of Economic Dynamics & Control, 30, 229-241, 2006.36.