Ergodic Optimal Quadratic Control for an Affine Equation with Stochastic and Stationary Coefficients
aa r X i v : . [ m a t h . P R ] A p r ERGODIC OPTIMAL QUADRATIC CONTROL FOR AN AFFINE EQUATIONWITH STOCHASTIC AND STATIONARY COEFFICIENTS
GIUSEPPINA GUATTERI AND FEDERICA MASIERO
Abstract.
We study ergodic quadratic optimal stochastic control problems for an affine stateequation with state and control dependent noise and with stochastic coefficients. We assumestationarity of the coefficients and a finite cost condition. We first treat the stationary case andwe show that the optimal cost corresponding to this ergodic control problem coincides with theone corresponding to a suitable stationary control problem and we provide a full characterizationof the ergodic optimal cost and control.
Key words.
Linear and affine quadratic optimal stochastic control, random and stationarycoefficients, ergodic control, Backward Stochastic Riccati Equation.
AMS subject classifications.
Introduction
In this paper we study an ergodic quadratic control problem for a linear affine equation with bothstate and control dependent noise, and the coefficients of the state equation, allowed to be random,are assumed to be stationary. We continue our previous work [4], where the infinite horizon caseand the ergodic case are studied but no characterization of the ergodic limit was given. The mainresult of the present paper is to obtain the characterization of the ergodic limit, see Theorem 3.5,when the coefficients are stationary in a suitable sense, see [11] and section 2 below.The main tool will be Backward Stochastic Riccati Equations (BSREs): such equations arenaturally linked with stochastic optimal control problems with stochastic coefficients. The firstexistence and uniqueness result for such a kind of equations has been given by Bismut in [2], butthen several works, see e. g. [3], [6], [7], [8], [9] followed. Only very recently Tang in [10] solved thegeneral non singular case corresponding to the linear quadratic problem with random coefficientsand control dependent noise. In [4], we have studied the infinite horizon case and the ergodic casenamely, we have considered a cost functional depending only on the asymptotic behaviour of thestate (ergodic control).Starting from this point, in this paper we first consider the stationary problem: minimize over alladmissible controls the cost functional J ♮ ( u, X ) = E Z [ | p S s X s | + | u s | ] ds. The control u is stationary and X is the corresponding solution of the state equation dX t = A t X t dt + B t u t dt + d X i =1 C it X t dW it + d X i =1 D it u t dW it + f t dt. (1.1)We denote the optimal cost for the stationary problem by J ♮ .The main technical point of this paper is to prove that the closed loop equation for the stationarycontrol problem, admits a unique stationary solution, see proposition 2.10.In order to study the ergodic control problem, we first consider the discounted cost functional J α (0 , x, u ) = E Z + ∞ e − αs [ (cid:10) S s X ,x,us , X ,x,us (cid:11) + | u s | ] ds, (1.2) here X is solution to equation dX s = ( A s X s + B s u s ) ds + d X i =1 (cid:0) C is X s + D is u s (cid:1) dW is + f s ds, s ≥ X = x. (1.3) A , B , C and D are bounded random and stationary processes and f ∈ L ∞P (Ω × [0 , + ∞ ) , R n ),moreover we assume suitable finite cost conditions. It is proved in [4] that in general, withoutstationarity assumptions,lim α → αJ α ( x ) = lim α → α E Z + ∞ h r αs , f αs i ds − lim α → α E Z + ∞ | ( I + d X i =1 (cid:0) D is (cid:1) ∗ P αs D is ) − ( B ∗ s r αs + d X i =1 (cid:0) D is (cid:1) ∗ g α,is ) | ds. Starting from this point, we show here that in the stationary caselim α → αJ α ( x ) = J ♮ Then we consider the “true” ergodic optimal cost, we minimize the following functional b J ( x, u ) = lim α → αJ ( x, u )over all u ∈ b U , see (3.5) for the definition of b U . We are able to prove thatinf u ∈ b U b J ( x, u ) = J ♮ ( u ) . and to the characterize the optimal ergodic control, see lemma 3.4 and theorem 3.5.2. Linear Quadratic optimal control in the stationary case
Let (Ω , F , P ) be a probability space and assume that W : ( −∞ , + ∞ ) → R is a d -dimensionalbrownian motion defined on the whole real axis. Let {F t } t ∈ ( −∞ , + ∞ ) its natural filtration completed.For all s, t ∈ R with t ≥ s we denote by G st the σ -field generated by { W τ − W s , s ≤ τ ≤ t } . Noticethat for all s ∈ R , {G st } t ≥ s is a filtration in (Ω , F ). Finally we assume that for all s < G s ⊆ F .Next we set a stationary framework: we introduce the semigroup ( θ t ) t ∈ R of measurable mappings θ t : (Ω , E ) → (Ω , E ) verifying(1) θ = Id, θ t ◦ θ s = θ t + s , for all t, s ∈ R (2) θ t is measurable: (Ω , F t ) → (Ω , F ) and {{ θ t ∈ A } : A ∈ F } = F t (3) P { θ t ∈ A } = P ( A ) for all A ∈ F (4) W t ◦ θ s = W t + s − W s According to this framework we introduce the definition of stationary stochastic process.
Definition 2.1.
We say that a stochastic process X : [0 , ∞ [ × Ω → R m , is stationary if for all s ∈ R X t ◦ θ s = X t + s P -a.s. for a.e. t ≥ A , B , C , D and S to be stationary stochastic processes. Namelyon the coefficients we make the following assumptions: Hypothesis 2.2.
A1) A : [0 , + ∞ ) × Ω → R n × n , B : [0 , + ∞ ) × Ω → R n × k , C i : [0 , + ∞ ) × Ω → R n × n , i = 1 , ..., d and D i : [0 , + ∞ ) × Ω → R n × k , i = 1 , ..., d , are uniformly bounded process adapted to thefiltration {F t } t ≥ . A2) S : [0 , + ∞ ) × Ω → R n × n is uniformly bounded and adapted to the filtration {F t } t ≥ and itis almost surely and almost everywhere symmetric and nonnegative. Moreover we assumethat there exists β > such that S ≥ βI . A3) A , B , C , D and S are stationary processes. In this case we immediately get: emma 2.3. Fix
T > and let hypothesis 2.2 holds true. Let ( P, Q ) be the solution of the finitehorizon BSRE − dP t = G ( A t , B t , C t , D t ; S t ; P t , Q t ) dt + d X i =1 Q it dW it , t ∈ [0 , T ] P T = P T . (2.1) For fixed s > we define b P ( t + s ) = P ( t ) θ s , b Q ( t + s ) = Q ( t ) θ s then ( b P , b Q ) is the unique solution in [ s, T + s ] of the equation − d b P t = G (cid:16) A t , B t , C t , D t ; S t ; b P t , b Q t (cid:17) dt + d X i =1 b Q it dW it , t ∈ [ s, T + s ] b P T = P T ◦ θ s . (2.2)In the stationary assumptions the backward stochastic Riccati equation dP t = − " A ∗ t P t + P t A t + S t + d X i =1 (cid:16)(cid:0) C it (cid:1) ∗ P t C it + (cid:0) C it (cid:1) ∗ Q t + Q t C it (cid:17) dt + d X i =1 Q it dW it + (2.3) " P t B t + d X i =1 (cid:16)(cid:0) C it (cid:1) ∗ P t D it + Q i D it (cid:17) I + d X i =1 (cid:0) D it (cid:1) ∗ P t D it − " P t B t + d X i =1 (cid:16)(cid:0) C it (cid:1) ∗ P t D it + Q it D it (cid:17) ∗ dt, (2.4)admits a minimal solution ( P , Q ), in the sense that whenever another couple (
P, Q ) is a solution tothe Riccati equation then P − P is a non-negative matrix, see also Corollary 3.3 in [5] and definiton3.2 in [4]. This minimal solution ( P , Q ) turns out to be stationary.
Proposition 2.4.
Assume hypothesis 2.2, then the minimal solution ( P , Q ) of the infinite horizonstochastic Riccati equation (2.3) is stationary. Proof.
For all ρ > P ρ the solution of equation (2.1) in [0 , ρ ] with final condition P ρ ( ρ ) = 0. Denoting by ⌊ ρ ⌋ the integer part of ρ , we have, following Proposition 3.2 in [5] that forall N for all t ∈ [0 , ⌊ N + s ⌋ ], P ⌊ N + s ⌋ t ≤ P N + st ≤ P ⌊ N + s ⌋ +1 t , P -a.s.. Thus we can conclude noticingthat by lemma 2.2 P N + st + s = P Nt ◦ θ s . Thus letting N → + ∞ we obtain that for all t ≥
0, and s > P (cid:8) P t + s = P t ◦ θ s (cid:9) = 1 . Now P T + s = P T ◦ θ s = P T so if one consider (2.1) in the intervall [ s, T + s ] with final data P T + s and (2.2) with final data P T ◦ θ s , by the uniqueness of the solution it follows that Q r =ˆ Q r , P − a.s. and for almost all r ∈ [ s, T + s ].We notice that in the BSRDE (2.3) the final condition has been replaced by the stationaritycondition on the solution process ( P, Q ).Next we give some definitions.
Definition 2.5.
We say that (
A, B, C, D ) is stabilizable relatively to the observations √ S (or √ S -stabilizable) if there exists a control u ∈ L P ([0 , + ∞ ) × Ω; R k ) such that for all t ≥ x ∈ R n E F t Z + ∞ t [ (cid:10) S s X t,x,us , X t,x,us (cid:11) + | u s | ] ds < M t,x . (2.5)for some positive constant M t,x where X t,x,u is the solution of the linear equation dX s = ( A s X s + B s u s ) ds + d X i =1 (cid:0) C is X s + D is u s (cid:1) dW is s ≥ X = x. (2.6) his kind of stabilizability condition, also called finite cost condition, has been introduced in [5].This condition has been proved to be equivalent to the existence of a minimal solution ( ¯ P , ¯ Q ) of theRiccati equation (2.3). Moreover whenever the first component ¯ P is uniformly bounded in time itfollows that the constant M t,x appearing in (2.5) can be chosen independent of time. Definition 2.6.
Let P be a solution to equation (2.3). We say that P stabilizes ( A, B, C, D )relatively to the identity I if for every t > x ∈ R n there exists a positive constant M ,independent of t , such that E F t Z + ∞ t | X t,x ( r ) | dr ≤ M P − a.s., (2.7)where X t,x is a mild solution to: dX t = AX t − B t I + d X i =1 (cid:0) D it (cid:1) ∗ P t D it ! − P t B t + d X i =1 (cid:16) Q it D it + (cid:0) C it (cid:1) ∗ P t D it (cid:17)! ∗ X t dt + d X i =1 C is X t − D is I + d X i =1 (cid:0) D it (cid:1) ∗ P t D it ! − P t B t + d X i =1 (cid:16) Q it D it + (cid:0) C it (cid:1) ∗ P t D it (cid:17)! ∗ X t dW t ,X = x (2.8)From now on we assume that Hypothesis 2.7. (i) (
A, B, C, D ) is √ S - stabilizable; (ii) the process P is uniformly bounded in time; (iii) the minimal solution ¯ P stabilizes ( A, B, C, D ) with respect to the identity I . We refer to [4] for cases when P stabilizes ( A, B, C, D ) relatively to the identity I . Notice that,thanks to the stationarity assumptions the stabilizability condition can be simplified, see Remark5.7 of [5].Next we study the dual (costate) equation in the stationary case. We denote byΛ (cid:0) t, P t , Q t (cid:1) = − I + d X i =1 (cid:0) D it (cid:1) ∗ P t D it ! − P t B t + d X i =1 (cid:16) Q it D it + (cid:0) C it (cid:1) ∗ P t D it (cid:17)! ∗ ,H t = A t + B t Λ (cid:0) t, P t , Q t (cid:1) ,K it = C it + D it Λ (cid:0) t, P t , Q t (cid:1) . (2.9)Thanks to Proposition 2.4, all the coefficients that appear in equation dr t = − H ∗ t r t dt − ¯ P t f t dt − d X i =1 (cid:0) K it (cid:1) ∗ g it dt + d X i =1 g it dW it , t ∈ [0 , T ] r T = 0 . (2.10)are stationary so exactly as before we deduce that for the solution ( r T , g T ) the following holds: Lemma 2.8.
Let A , B , C , D and S satisfy hypothesis 2.2 and let f ∈ L ∞P (Ω × [0 , + ∞ )) be astationary process. Fix T > and r T ∈ L ∞P (Ω , F T ; R n ) . Let ( r, g ) a solution to equation dr t = − H ∗ t r t dt − ¯ P t f t dt − d X i =1 (cid:0) K it (cid:1) ∗ g it dt + d X i =1 g it dW it , t ∈ [0 , T ] r T = r T . (2.11) For fixed s > we define b r t + s = r t ◦ θ s , b g t + s = g t ◦ θ s then ( b r, b g ) is the unique solution in [ s, T + s ] of the equation d b r t = − H ∗ t b r t dt − ¯ P t f t dt − d X i =1 (cid:0) K it (cid:1) ∗ b g it dt + d X i =1 b g it dW it , t ∈ [ s, T + s ] b r T = r T ◦ θ s . (2.12) ence arguing as for the first component P , we get that the solution of the infinite horizon dualequation is stationary, as stated in the following proposition: Proposition 2.9.
Assume hypothesis 2.2 and hypothesis 2.7, then the solution ( r ♮ , g ♮ ) of dr t = − H ∗ t r t dt − P t f t dt − d X i =1 (cid:0) K it (cid:1) ∗ g it dt + d X i =1 g it dW it , (2.13) obtained as the pointwise limit of the solution to equation (2.10) is stationary. Moreover (cid:0) r ♮ , g ♮ (cid:1) ∈ L ∞P (Ω × [0 , , R n ) × L P (cid:0) Ω × [0 , , R n × d (cid:1) . Proof.
The proof follows from an argument similar to the one in Proposition 4.5 in [4]. Stationarityof the solution ( r ♮ , g ♮ ) follows from the previous lemma.Again we notice that in the dual BSDE (2.13) the final condition has been replaced by thestationarity condition on the solution process ( r ♮ , g ♮ ).We need to show that in the stationary assumptions, the solution of the closed loop equation isstationary. By using notation (2.9), we consider the following stochastic differential equation, whichwill turn out to be the closed loop equation: dX s = H s X s ds + d X i =1 K is X s dW is + B s ( B ∗ s r ♮s + d X i =1 D is g ♮,is ) ds + f s ds + d X i =1 D is ( B ∗ s r ♮s + d X i =1 D is g ♮,is ) dW is , (2.14)where ( r ♮ , g ♮ ) is the solution of the dual (costate) equation (2.13). Proposition 2.10.
Assume hypothesis 2.2 and hypothesis 2.7 holds true then there exists a uniquestationary solution of equation (2.14) . Proof.
We set f s = f s + B s ( B ∗ s r ♮s + d X i =1 D is g ♮,is ) and f ,js = D js ( B ∗ s r ♮s + d X i =1 D is g ♮,is ), j = 1 , ..., d . Wecan extend f , f for negative times letting for all t ∈ [0 , f i − N + t = f it ◦ θ − N , i = 1 , N ∈ N .We notice that f i | [ − N, + ∞ ) is predictable with respect to the filtration ( G − Nt ) t ≥− N . Therefore forall N ∈ N equation dX − Ns = H s X − Ns ds + + P di =1 K is X − Ns dW is + f s ds + d X i =1 f ,is dW is ,X − N − N = 0 , admits a solution ( X − N, t ) t , defined for t ≥ − N and predictable with respect to the filtration( G − Nt ) t ≥− N . We extend X − N, to the whole real axis by setting X − N, t = 0 for t < − N . We wantto prove that, fixed t ∈ R , ( X − N, t ) N is a Cauchy sequence in L (Ω). In order to do this we noticethat for t ≥ − N + 1, X − N, t − X − N +1 , t solves the following (linear) stochastic differential equation X − N, t − X − N +1 , t = X − N, − N +1 + Z t − N +1 H s ( X − N, s − X − N +1 , s )+ d X i =1 Z t − N +1 K is ( X − N, s − X − N +1 , s ) dW is . By the Datko theorem, see e.g. [4] and [5], there exist constants a, c > E | X − N, t − X − N +1 , t | ) / ≤ Ce − a ( t + N − ( E | X − N, − N +1 | ) / . So, fixed t ∈ R and M, N ∈ N , M > N sufficiently large such that − N ≤ t ,( E | X − N, t − X − M, t | ) / ≤ M − X k = N ( E | X − k, t − X − k +1 , t | ) / ≤ C M − X k = N e − a ( t + k − ( E | X − k, − k +1 | ) / . (2.15)Next we look for a uniform estimate with respect to k of E | X − k, − k +1 | . For s ∈ [ − k, − k + 1], X − k, s = Z s − k A r X − k, r dr + Z s − k B r ¯ u r dr + d X i =1 Z s − k C ir X − k, r dW ir + d X i =1 Z s − k D ir ¯ u r dW ir + Z s − k f r dr, (2.16) here ¯ u is the optimal control that minimizes the cost J ( − k, , u ) = E Z − k +1 − k [ | p S s X s | + | u s | ] ds. By computing d [ h P s X − k, s , X − k, s i + 2 h ¯ r ♮s , X − k, s i ] we get, for every T > E Z − k +1 − k [ | p S s X s | + | ¯ u s | ] ds = − E h P − k +1 X − k, − k +1 , X − k, − k +1 i − E Z − k +1 − k h r ♮s , f s i ds − E Z − k +1 − k | I + d X i =1 (cid:0) D is (cid:1) ∗ P s D is ! − ( B ∗ s r ♮s + d X i =1 (cid:0) D is (cid:1) ∗ g ♮,is ) | ds ≤ | E Z − k +1 − k h r ♮s , f s i ds | ≤ A, where A is a constant independent on k . By (2.16) we getsup − k ≤ s ≤− k +1 E | X − k, s | ≤ C Z s − k sup − k ≤ r ≤ s E | X − k, r | dr + C E Z − k +1 − k | ¯ u r | dr + E Z − k +1 − k | f r | dr, and so by applying the Gronwall lemma, we getsup − k ≤ s ≤− k +1 E | X − k, s | ≤ Ce C ( A + E Z − k +1 − k | f r | dr ) . Since f is stationary, we can conclude thatsup − k ≤ s ≤− k +1 E | X − k, s | ≤ C, where C is a constant independent on k . By (2.15), we get( E | X − N, t − X − M, t | ) / ≤ C M − X k = N e − a ( t + k − . So we can conclude that, fixed t ∈ R , ( X − N, t ) N is a Cauchy sequence in L (Ω), and so it convergesin L (Ω) to a random variable denoted by ζ ♮t . Notice that for every t ∈ R we can define ζ ♮t , and weprove that ζ ♮ is a stationary process. Let t ∈ R , − N < t and s >
0: since the shift θ is measurepreserving, lim N →∞ E | X − N, t ◦ θ s − ζ ♮t ◦ θ s | = 0 , moreover X − N, t ◦ θ s = X − N + s, t + s andlim N →∞ E | X − N + s, t + s − ζ ♮t + s | = 0 . By uniqueness of the limit we conclude that ζ ♮t ◦ θ s = ζ ♮t + s . Notice that since N ∈ N and F ⊃ G − N , ζ ♮ is F -measurable. Let us consider the value of the solution of equation (2.14) starting from X = ζ ♮ . By stationarity of the coefficients and of ζ ♮ , we get that X is a stationary solution ofequation (2.14), that we denote by X ♮ . In order to show the uniqueness of the periodic solution itis enough to notice that if f j = 0, j = 1 ,
2, and X ♮ is a periodic solution of (2.14), then E | X ♮ | = E | X ♮N | ≤ Ce − aN E | ζ ♮ | . Therefore X ♮ = 0 and this concludes the proof.We can now treat the following optimal control problem for a stationary cost functional: minimizeover all admissible controls u ∈ U ♮ the cost functional J ♮ ( u, X ) = E Z [ | p S s X s | + | u s | ] ds, ( u, X ) ∈ U ♮ , (2.17)where U ♮ = (cid:8) ( u, X ) ∈ L P (Ω × [0 , × C ([0 , , L P (Ω)) : X s = X ◦ θ s , ∀ s ∈ R (cid:9) (2.18) nd X is the solution of equation dX t = A t X t dt + B t u t dt + d X i =1 C it X t dW it + d X i =1 D it u t dW it + f t dt, (2.19)relative to u . Theorem 2.11.
Let X ♮ ∈ C ([0 , , L ( P Ω)) be the unique stationary solution of equation (2.14) andlet u ♮t = − I + d X i =1 (cid:0) D it (cid:1) ∗ P t D it ! − P t B t + d X i =1 (cid:16) Q it D it + (cid:0) C it (cid:1) ∗ Q t D it (cid:17)! ∗ X ♮t + B ∗ t r ♮t + d X i =1 ( D it ) ∗ g ♮,it . (2.20) Then ( u ♮ , X ♮ ) ∈ U ♮ and it is the unique optimal couple for the cost (2.17) , that is J ♮ ( u ♮ , X ♮ ) = inf ( u,X ) ∈U ♮ J ♮ ( u, X ) . The optimal cost is given by J ♮ = J ♮ ( u ♮ , X ♮ ) = 2 E Z h r ♮s , f s i ds − E Z | ( I + d X i =1 (cid:0) D it (cid:1) ∗ P t D it ) − ( B ∗ t r ♮t + d X i =1 (cid:0) D it (cid:1) ∗ g ♮,it ) | ds. (2.21) Proof.
By computing d h P s X s , X s i + 2 h r ♮s , X s i we get E Z [ h S s X s , X s i + | u s | ] ds = E h P X , X i − E h P X , X i + 2 E h r ♮ , X i − E h r ♮ , X i − E Z h r ♮s , f s i ds + E Z | I + d X i =1 (cid:0) D is (cid:1) ∗ P s D is ! / u s + ( I + d X i =1 (cid:0) D is (cid:1) ∗ P s D is ) − ∗∗ P s B s + d X i =1 (cid:16) Q is D is + (cid:0) C is (cid:1) ∗ P s D is (cid:17)! ∗ X s + B ∗ s r ♮s + d X i =1 D is (¯ g ♮,is ) ∗ ! | ds − E Z | I + d X i =1 (cid:0) D is (cid:1) ∗ P s D is ! − ( B ∗ s r ♮s + d X i =1 (cid:0) D is (cid:1) ∗ ¯ g ♮,is ) | ds. Since by Propositions 2.4, 2.9, and 2.10 ( u, X ) ∈ U ♮ , we get E Z [ h S s X s , X s i + | u s | ] ds = − E Z h r ♮s , f s i ds + E Z | I + d X i =1 (cid:0) D is (cid:1) ∗ P s D is ! / ×× u s + ( I + d X i =1 (cid:0) D is (cid:1) ∗ P s D is ) − P s B s + d X i =1 (cid:16) Q is D is + (cid:0) C is (cid:1) ∗ P s D is (cid:17)! ∗ X s + B ∗ s r ♮s + d X i =1 D is (¯ g ♮,is ) ∗ ! | ds − E Z | I + d X i =1 (cid:0) D is (cid:1) ∗ P s D is ! − ( B ∗ s r ♮s + d X i =1 (cid:0) D is (cid:1) ∗ ¯ g ♮,is ) | ds. So u ♮t = − I + d X i =1 (cid:0) D it (cid:1) ∗ P t D it ! − P t B t + d X i =1 (cid:16) Q it D it + (cid:0) C it (cid:1) ∗ Q t D it (cid:17)! ∗ X ♮t + B ∗ t r ♮t + d X i =1 ( D it ) ∗ g ♮,it (2.22)is the optimal cost: u ♮ minimizes the cost (2.21), and the corresponding state X ♮ is stationary byProposition 2.10, so that ( u ♮ , X ♮ ) ∈ U ♮ . . Ergodic control
In this section we consider cost functionals depending only on the asymptotic behaviour of thestate (ergodic control). Throughout this section we assume the following:
Hypothesis 3.1.
The coefficient satisfy hypothesis 2.2, and moreover • S ≥ ǫI , for some ǫ > . • ( A, B, C, D ) is stabilizable relatively to S . • The first component of the minimal solution P is bounded in time. Notice that these conditions implies that (
P, Q ) stabilize (
A, B, C, D ) relatively to the identity.We first consider discounted cost functional and then we compute a suitable limit of the discountedcost. Namely, we consider the discounted cost functional J α (0 , x, u ) = E Z + ∞ e − αs [ (cid:10) S s X ,x,us , X ,x,us (cid:11) + | u s | ] ds, (3.1)where X is solution to equation dX s = ( A s X s + B s u s ) ds + d X i =1 (cid:0) C is X s + D is u s (cid:1) dW is + f s ds s ≥ tX t = x.A , B , C and D satisfy hypothesis 2.2 and f ∈ L ∞P (Ω × [0 , + ∞ )) and is a stationary process. Whenthe coefficients are deterministic the problem has been extensively studied, see e.g. [1] and [11].Our purpose is to minimize the discounted cost functional with respect to every admissible control u . We define the set of admissible controls as U α = (cid:26) u ∈ L (Ω × [0 , + ∞ )) : E Z + ∞ e − αs [ (cid:10) S s X ,x,us , X ,x,us (cid:11) + | u s | ] ds < + ∞ (cid:27) . Fixed α >
0, we define X αs = e − αs X s and u αs = e − αs u s . Moreover we set A αs = A s − αI and f αs = e − αs f s , and f α ∈ L P (Ω × [0 , + ∞ )) ∩ L ∞P (Ω × [0 , + ∞ )). X αs is solution to equation dX αs = ( A αs X αs + B s u αs ) ds + d X i =1 (cid:0) C is X αs + D is u αs (cid:1) dW is + f αs ds s ≥ X α = x, (3.2)By the definition of X α , we note that if ( A, B, C, D ) is stabilizable with respect to the identity, then( A α , B, C, D ) also is. We also denote by ( P α , Q α ) the minimal solution of a stationary backwardRiccati equation (2.3) with A α in the place of A . Since, for 0 < α < A α is uniformly boundedin α , also P α is uniformly bounded in α . Arguing as in Proposition 2.4, ( P α , Q α ) is a stationaryprocess.Let us denote by ( r α , g α ) the solution of the infinite horizon BSDE dr αt = − ( H αt ) ∗ r αt dt − P αt f αt dt − d X i =1 (cid:16) K α,it (cid:17) ∗ g α,it dt + d X i =1 g α,it dW it , t ≥ , (3.3)where H α and K α are defined as in (2.9), with A α , P α and Q α respectively in the place of A , P and Q . By [4], section 4, we get that equation (3.3) admits a solution ( r α , g α ) ∈ L P (Ω × [0 , + ∞ )) ∩ L P (Ω × [0 , + ∞ )) × L ∞P (Ω × [0 , T ]), for every fixed T > α → α inf u α ∈U α J α (0 , x, u α ) =lim α → [ α Z + ∞ E h r αs , f αs i ds − α E Z + ∞ | ( I + d X i =1 (cid:0) D is (cid:1) ∗ P αs D is ) − ( B ∗ s r αs + d X i =1 (cid:0) D is (cid:1) ∗ g α,is ) | ds ] . We can also prove the following convergence result for ( r α , g α ). emma 3.2. For all fixed
T > , r α | [0 ,T ] → r ♮ | [0 ,T ] in L P (Ω × [0 , T ]) . Moreover, for every fixed T > , as α → : E Z T | ( I + d X i =1 (cid:0) D is (cid:1) ∗ P αs D is ) − ( B ∗ s r αs + d X i =1 (cid:0) D is (cid:1) ∗ g α,is ) | ds → E Z T | ( I + d X i =1 (cid:0) D is (cid:1) ∗ P s D is ) − ( B ∗ s r ♮s + d X i =1 (cid:0) D is (cid:1) ∗ g ♮,is ) | ds Proof.
The first assertion follows from lemma 6.6 in [4]. Notice that stationarity of the coefficientsin the limit equation gives stationarity of the solution, and so it allows to identify the limit with thestationary solution of the dual BSDE. For the second assertion for the optimal couple ( X α , u α ) forthe optimal control problem on the time interval [0 , T ]: Z T [ | p S s X αs | + | u αs | ds = h P α x, x i + 2 h r α , x i + 2 E Z T h r αs , f αs i ds E h P αT X αT , X αT i + 2 E h r αT , X αT i − E Z T | ( I + d X i =1 (cid:0) D it (cid:1) ∗ P αt D it ) − ( B ∗ t r αt + d X i =1 (cid:0) D it (cid:1) ∗ g α,it ) | ds. (3.4)Since, as α →
0, in (3.4) all the terms but the last one converge to the corresponding stationary term,and since by [4] ( r α , g α ) is uniformly, with respect to α , bounded in L P (Ω × [0 , T ]) × L P (Ω × [0 , T ]),then ( r α | [0 ,T ] , g α | [0 ,T ] ) ⇀ ( r ♮ | [0 ,T ] , g ♮ | [0 ,T ] ) in L P (Ω × [0 , T ]) × L P (Ω × [0 , T ]), we get the desiredconvergence.This is enough to characterize the ergodic limit. Indeed we have that: Theorem 3.3.
We get the following characterization of the optimal cost: lim α → α inf u ∈U α J α ( x, u ) = E " h f (0) , r ♮ (0) i − | ( I + d X i =1 (cid:0) D i (cid:1) ∗ P D i ) − ( B ∗ r ♮ + d X i =1 (cid:0) D i (cid:1) ∗ g ♮,i ) | . Proof.
Let us define e r αt = e αt r αt , e g αt = e αt g αt . ( e r αt , e g αt ) is the solution to d e r αt = − ( H αt ) ∗ e r αt dt + αI e r αt dt − P αt f t dt − d X i =1 (cid:16) K α,it (cid:17) ∗ e g α,it dt + d X i =1 e g α,it dW it , t ≥ , and so, arguing as in lemma 2.9, ( e r αt , e g αt ) are stationary processes. Now we computelim α → α inf u α ∈U α J α (0 , x, u α ) =lim α → (cid:20) α Z + ∞ e − αs E h e r αs , f s i ds − α Z + ∞ e − αs E | ( I + d X i =1 (cid:0) D is (cid:1) ∗ P αs D is ) − ( B ∗ s e r αs + d X i =1 (cid:0) D is (cid:1) ∗ e g α,is ) | ds = lim α → " α ∞ X k =1 e − αk Z e − αs E h e r αs , f s i ds − α ∞ X k =1 e − αk Z e − αs E | ( I + d X i =1 (cid:0) D is (cid:1) ∗ P αs D is ) − ( B ∗ s e r αs + d X i =1 (cid:0) D is (cid:1) ∗ e g α,is ) | ds = lim α → " α ∞ X k =1 e − αk Z E h r αs , f αs i ds − α ∞ X k =1 e − αk Z E | ( I + d X i =1 (cid:0) D is (cid:1) ∗ P αs D is ) − ( B ∗ s r αs + d X i =1 (cid:0) D is (cid:1) ∗ g α,is ) | ds . ince ( r αs , g αs ) → ( r ♮s , g ♮s ) in L P (Ω × [0 , × L P (Ω × [0 , α → α inf u α ∈U α J α (0 , x, u α ) = 2 E Z h r ♮s , f s i ds − E Z | ( I + d X i =1 (cid:0) D is (cid:1) ∗ P αs D is ) − ( B ∗ s r ♮s + d X i =1 (cid:0) D is (cid:1) ∗ g ♮,is ) | ds = 2 E h r ♮ , f i − E | ( I + d X i =1 (cid:0) D i (cid:1) ∗ P D i ) − ( B ∗ r ♮ + d X i =1 (cid:0) D i (cid:1) ∗ g α,i ) | , where the first equality holds also in the periodic case and the second equality holds only in thestationary case.The next step is to minimize b J ( x, u ) = lim α → αJ ( x, u )over all u ∈ b U , where b U = (cid:26) u ∈ L loc : E Z + ∞ e − αs [ (cid:10) S s X ,x,us , X ,x,us (cid:11) + | u s | ] ds < + ∞ , ∀ α > . (cid:27) (3.5)We will prove that inf u ∈ b U b J ( x, u ) = J ♮ ( u ) . Let b X be solution of d b X xs = H s b X xs ds + d X i =1 K is b X xs dW is + B s ( B ∗ s r ♮s + d X i =1 D is g ♮,is ) ds + f s ds + d X i =1 D is ( B ∗ s r ♮s + d X i =1 D is g ♮,is ) dW is b X x = x, and let b u xs = − Λ( s, P s , Q s ) b X xs + ( B ∗ s r ♮s + d X i =1 D is g ♮,is ) . Notice that by proposition 2.10 if x = ζ ♮ , then b X ζ ♮ is stationary and ( b u ζ ♮ , b X ζ ♮ ) is the optimal couple( u ♮ , X ♮ ). Lemma 3.4.
For all x ∈ L (Ω) , b u x ∈ b U and b J ( b u x , x ) does not depend on x . Proof.
Let us consider X s,xt the solution of equation dX s,xt = H t X s,xt dt + d X i =1 K it X s,xt dW it X s,xs = x, starting from x at time s . We denote, for every 0 ≤ s ≤ t , U ( t, s ) x := X s,xt . We notice that b X ,xt − b X ,ζ ♮ t = x − ζ ♮ + Z s H s ( b X ,xs − b X ,ζ ♮ s ) ds + d X i =1 Z s K is ( b X ,xs − b X ,ζ ♮ s ) dW is = U ( t, x − ζ ♮ ) . So by the Datko theorem, see e.g. [4] and [5], there exist constants a, C > E | b X xt − b X ζ ♮ t | ≤ Ce − at E | x − ζ ♮ | . So E | b X x | ≤ Ce − at E | x − ζ ♮ | + E | b X ζ ♮ | ≤ C, where in the last passage we use that b X ζ ♮ = X ♮ and it is stationary.Again by applying the Datko theorem we obtainlim α → α E Z ∞ e − αs (2 h SX ♮s , U ( s, x − ζ ♮ ) i + |√ SU ( s, x − ζ ♮ )) | ) ds = 0 . Moreover b u t = u ♮t − Λ( t, P t , Q t ) U (0 , t )( x − ζ ♮ ) t is clear that u ♮ belongs to the space of admissible control space b U .The term ˜ u t = Λ( t, P t , Q t ) U (0 , t )( x − ζ ♮ ), t ∈ (0 + ∞ ) can be proved to be the optimal controlfor the infinite horizon problem with f = 0 and random initial data x − ζ ♮ :inf u ∈ L P ((0 , + ∞ ); R k ) E Z + ∞ ( |√ S s X us | + | u ( s ) | ) ds. Hence Theorem 5.2 of [4] can be extended without any difficulty to get that: J (0 , x − ζ ♮ , ˜ u ) = E h P ( x − ζ ♮ ) , x − ζ ♮ i + 2 E h r , x − ζ ♮ i− E Z ∞ | ( I + d X i =1 (cid:0) D it (cid:1) ∗ P t D it ) − ( B ∗ t r t + d X i =1 (cid:0) D it (cid:1) ∗ g it ) | ds. Therefore E Z ∞ e − αs | ˜ u ( s ) | ds ≤ E Z ∞ | ˜ u ( s ) | ds ≤ C. This proves that b u is an admissible control since it follows thatlim α → α E Z ∞ e − αs ( | u ♮s | − | b u xs | ) ds = 0 . We can now conclude as follows:
Theorem 3.5.
For all x ∈ L (Ω) the couple ( b X x , b u x ) is optimal that is b J ( b u x , x ) = min { b J ( u, x ) : u ∈ b U} Moreover the optimal cost, that does not depend on the initial state x , is equal to the optimal costfor the periodic (respectively stationary) problem, i.e. b J ( b u x , x ) = J ♮ . Proof.
We denote inf u ∈U α J α ( x, u ) := J ∗ α ( x, u ). If u ∈ b U , then for every α > u ∈ U α . Conse-quently for every α > αJ α ( u, x ) ≥ αJ ∗ α . By taking the limit on both sides we get b J ( x, u ) = lim α → αJ α ≥ lim α → αJ ∗ α = J ♮ . By the previous lemma b J ( x, b u x ) is independent on x so we let x = ζ ♮ , which implies that b u x = u ♮ and b X x = X ♮ . Then b J ( ζ ♮ , u ♮ ) = lim α → α Z ∞ e − αt [ |√ S t X ♮t | + | u ♮t | ] dt = lim α → α ( + ∞ X k =1 e − kα ) Z e − αt [ |√ S t X ♮t | + | u ♮t | ] dt = J ♮ , and this concludes the proof. References [1] A. Bensoussan and J. Frehse. On Bellman Equations of Ergodic Control in R b . J. reine angew. Math.
SIAM J. Contr. Optim.
S´eminairede Probabilit´es, XII . Lecture Notes in Math., 649, Springer, Berlin, 1978.[4] G. Guatteri and F. Masiero. Infinite Horizon and Ergodic Optimal Quadratic Control for an Affine Equationwith Stochastic Coefficients, arxiv:math/0707.0606,
Submitted
5] G. Guatteri and G. Tessitore. Backward Stochastic Riccati Equations and Infinite Horizon L-Q Optimal ControlProblems with Stochastic Coefficients
Applied Mathematics and Optimization,
57 (2008), no.2, pp.207-235[6] M. Kohlmann and S. Tang. New developments in backward stochastic Riccati equations and their applications.In
Mathematical finance (Konstanz, 2000) , Trends Math., Birkhser, Basel, 2001.[7] M. Kohlmann and S. Tang. Global adapted solution of one-dimensional backward stochastic Riccati equations,with application to the mean-variance hedging.
Stochastic Process. Appl.
97 (2002), 1255–288.[8] M. Kohlmann and S. Tang. Multidimensional backward stochastic Riccati equations and applications.
SIAM J.Contr. Optim.
41 (2003), 1696–1721.[9] S. Peng. Stochastic Hamilton-Jacobi-Bellman Equations.
SIAM J. Contr. Optim.
30 (1992), 284–304.[10] S. Tang. General linear quadratic optimal control problems with random coefficients: linear stochastic Hamiltonsystems and backward stochastic Riccati equations. SIAM J. Control Optim. 42 (2003), no. 1, 53–75[11] G.Tessitore. Infinite Horizon, Ergodic and Periodic Control for a Stochastic Infinite Dimensional Affine Equation.
Journal of Mathematical Systems, Estimation, and Control. 8 n.4 (1998), 1-28