Mean-Field Stochastic Linear Quadratic Optimal Control Problems: Open-Loop Solvabilities
aa r X i v : . [ m a t h . O C ] S e p Mean-Field Stochastic Linear Quadratic OptimalControl Problems: Open-Loop Solvabilities
Jingrui Sun ∗ August 15, 2018
Abstract:
This paper is concerned with a mean-field linear quadratic (LQ, for short) optimal controlproblem with deterministic coefficients. It is shown that convexity of the cost functional is necessary for thefiniteness of the mean-field LQ problem, whereas uniform convexity of the cost functional is sufficient forthe open-loop solvability of the problem. By considering a family of uniformly convex cost functionals, acharacterization of the finiteness of the problem is derived and a minimizing sequence, whose convergence isequivalent to the open-loop solvability of the problem, is constructed. Then, it is proved that the uniformconvexity of the cost functional is equivalent to the solvability of two coupled differential Riccati equationsand the unique open-loop optimal control admits a state feedback representation in the case that the costfunctional is uniformly convex. Finally, some examples are presented to illustrate the theory developed.
Key words: mean-field stochastic differential equation, linear quadratic optimal control, Riccati equation,finiteness, open-loop solvability, feedback representation
AMS subject classifications.
Let (Ω , F , F , P ) be a complete filtered probability space on which a standard one-dimensional Brownianmotion W = { W ( t ); 0 t < ∞} is defined, where F = {F t } t > is the natural filtration of W augmented byall the P -null sets in F . Consider the following controlled linear stochastic differential equation (SDE, forshort) on a finite horizon [ t, T ]:(1.1) dX ( s ) = n A ( s ) X ( s ) + ¯ A ( s ) E [ X ( s )] + B ( s ) u ( s ) + ¯ B ( s ) E [ u ( s )] + b ( s ) o ds + n C ( s ) X ( s ) + ¯ C ( s ) E [ X ( s )] + D ( s ) u ( s ) + ¯ D ( s ) E [ u ( s )] + σ ( s ) o dW ( s ) , s ∈ [ t, T ] ,X ( t ) = ξ, where A ( · ), ¯ A ( · ), B ( · ), ¯ B ( · ), C ( · ), ¯ C ( · ), D ( · ), ¯ D ( · ) are given deterministic matrix-valued functions; b ( · ), σ ( · )are vector-valued F -progressively measurable processes and ξ is an F t -measurable random vector. In theabove, u ( · ) is the control process and X ( · ) is the corresponding state process with initial pair ( t, ξ ). For any t ∈ [0 , T ), we define U [ t, T ] = ( u : [ t, T ] × Ω → R m (cid:12)(cid:12) u ( · ) is F -progressively measurable, E Z Tt | u ( s ) | ds < ∞ ) . ∗ Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China ([email protected]). u ( · ) ∈ U [ t, T ] is called an admissible control (on [ t, T ]). Under some mild conditions, for any initialpair ( t, ξ ) with ξ being square-integrable and any admissible control u ( · ) ∈ U [ t, T ], (1.1) admits a uniquesquare-integrable solution X ( · ) ≡ X ( · ; t, ξ, u ( · )). Now we introduce the following cost functional:(1.2) J ( t, ξ ; u ( · )) , E ( h GX ( T ) , X ( T ) i + 2 h g, X ( T ) i + h ¯ G E [ X ( T )] , E [ X ( T )] i + 2 h ¯ g, E [ X ( T )] i + Z Tt "* Q ( s ) S ( s ) ⊤ S ( s ) R ( s ) ! X ( s ) u ( s ) ! , X ( s ) u ( s ) !+ + 2 * q ( s ) ρ ( s ) ! , X ( s ) u ( s ) !+ ds + Z Tt "* ¯ Q ( s ) ¯ S ( s ) ⊤ ¯ S ( s ) ¯ R ( s ) ! E [ X ( s )] E [ u ( s )] ! , E [ X ( s )] E [ u ( s )] !+ + 2 * ¯ q ( s )¯ ρ ( s ) ! , E [ X ( s )] E [ u ( s )] !+ ds ) , where G , ¯ G are symmetric matrices and Q ( · ), ¯ Q ( · ), S ( · ), ¯ S ( · ), R ( · ), ¯ R ( · ) are deterministic matrix-valuedfunctions with Q ( · ) ⊤ = Q ( · ), ¯ Q ( · ) ⊤ = ¯ Q ( · ), R ( · ) ⊤ = R ( · ), ¯ R ( · ) ⊤ = ¯ R ( · ); g is an F T -measurable randomvector and ¯ g is a (deterministic) vector; q ( · ), ρ ( · ) are vector-valued F -progressively measurable processes and¯ q ( · ), ¯ ρ ( · ) are deterministic vector-valued functions. Our mean-field stochastic LQ optimal control problemcan be stated as follows: Problem (MF-LQ).
For any given initial pair ( t, ξ ) ∈ [0 , T ) × L F t (Ω; R n ), find a u ∗ ( · ) ∈ U [ t, T ] suchthat(1.3) J ( t, ξ ; u ∗ ( · )) = inf u ( · ) ∈U [ t,T ] J ( t, ξ ; u ( · )) , V ( t, ξ ) . In the above, L F t (Ω; R n ) is the space of all F t -measurable, R n -valued random vectors ξ with E | ξ | < ∞ .Any u ∗ ( · ) ∈ U [ t, T ] satisfying (1.3) is called an ( open-loop ) optimal control of Problem (MF-LQ) for the initialpair ( t, ξ ), and the corresponding X ∗ ( · ) ≡ X ( · ; t, ξ, u ∗ ( · )) is called an optimal state process . The function V ( · , · ) is called the value function of Problem (MF-LQ). In the special case of b ( · ), σ ( · ), g ( · ), ¯ g ( · ), q ( · ), ¯ q ( · ), ρ ( · ), ¯ ρ ( · ) = 0, we denote by J ( t, ξ ; u ( · )), V ( t, ξ ) and Problem (MF-LQ) the corresponding cost functional,value function and Problem (MF-LQ), respectively.Comparing with the classical stochastic LQ optimal control problem, a new feature of Problem (MF-LQ) is that both the state equation and the cost functional involve the states and the controls as well astheir expectations. In this case, we call (1.1) a controlled mean-field (forward) SDE (MF-SDE, for short).The history of MF-SDEs can be traced back to the work of Kac [18] in 1956 and McKean [21] in 1966.Since then, many researchers have made contributions to such kind of equations and applications; see, forexample, Dawson [12], Dawson–G¨artner [13], Scheutzow [24], G¨artner [14], Graham [15], Chan [9], Chiang[10] and Ahmed–Ding [2]. For recent development of MF-SDEs, readers may refer to Huang–Malham´e–Caines[17], Veretennikov [27], Mahmudov–McKibben [20], Buckdahn–Djehiche–Li–Peng [7], Buckdahn–Li–Peng[8], Borkar–Kumar [5], Crisan–Xiong [11], Kotelenez–Kurtz [19] and the references cited therein. Controlproblems of MF-SDEs were studied by Ahmed–Ding [3], Ahmed [1], Park–Balasubramaniam–Kang [23],Buckdahn–Djehiche–Li [6], Andersson–Djehiche [4], Meyer-Brandis–Øksendal–Zhou [22], and so on. Morerecently, Yong [28] investigated an LQ problem for MF-SDEs in finite horizons and gave some interestingmotivation for the control problem with E [ X ( · )] and E [ u ( · )] being included in the cost functional. Later,Huang–Li–Yong [16] generalized the results in [28] to the case with an infinite time horizon.In [28], two coupled differential Riccati equations are derived by decoupling the optimality system. It isshown that under certain conditions, the two Riccati equations are uniquely solvable and Problem (MF-LQ)admits a unique optimal control which has a state feedback representation. To be precise, if(1.4) ( G, G + ¯ G > , Q ( s ) , Q ( s ) + ¯ Q ( s ) > ,S ( s ) = ¯ S ( s ) = 0 , R ( s ) , R ( s ) + ¯ R ( s ) > δI, a.e. s ∈ [0 , T ] , δ >
0, then the unique solvability of the two Riccati equations can be obtained from the classicalresult [29, Theorem 7.2]. However, examples show that the two Riccati equations might still be solvableeven if both R ( · ) and ¯ R ( · ) are negative semi-definite (see Example 6.1). On the other hand, it may happenthat Problem (MF-LQ) is open-loop solvable, while the optimal control cannot be obtained by solving thecorresponding Riccati equations due to the possible singularities of the terms R + D ⊤ P D and R + ¯ R +( D + ¯ D ) ⊤ P ( D + ¯ D ) (see Example 6.2). Thus, some questions arise naturally: (a) What is the relationshipbetween Problem (MF-LQ) and the solvability of the two Riccati equations? (b) How can we characterizethe open-loop solvability of Problem (MF-LQ)? (c) How can we find an optimal control in general? Thepurpose of this paper is to study Problem (MF-LQ) from an open-loop point of view and to address theabove issues. Closed-loop mean-field LQ problems will be investigated in a forthcoming paper.Our main idea and results of this paper can be informally described as follows. By a representation of thecost functional, we first show that for the open-loop solvability of Problem (MF-LQ), a necessary condition isthe convexity of the cost functional and a sufficient condition is the uniform convexity of the cost functional.Under the convexity condition, by adding ε E R Tt | u ( s ) | ds ( ε >
0) to the original cost functional, we get afamily of uniformly convex functionals. The corresponding mean-field LQ problems admit unique optimalcontrols u ∗ ε ( · ) , ε >
0, which form a minimizing sequence of Problem (MF-LQ). Then the open-loop solvabilityof Problem (MF-LQ) is characterized by the convergence of the sequence, whose limit is an optimal controlof Problem (MF-LQ). To construct u ∗ ε ( · ) explicitly, we further investigate Problem (MF-LQ) with uniformlyconvex cost functionals. Since the uniform convexity condition is much weaker than (1.4), the result in [28]fails to apply to this case. To overcome this difficulty, we reduce Problem (MF-LQ) to a classical stochasticLQ problem and a deterministic LQ problem. By making use of a result found in [25], we establish theequivalence between the uniform convexity of the cost functional and the solvability of the two Riccatiequations. Then by the completion of squares technique, we obtain a state feedback representation of theoptimal control via the solutions of the two Riccati equations.The rest of the paper is organized as follows. Section 2 gives some preliminaries. In Section 3, westudy Problem (MF-LQ) from a Hilbert space viewpoint and derive necessary and sufficient conditions forthe finiteness and open-loop solvability of the problem by considering a family of uniformly convex costfunctionals. Section 4 shows that the solvability of two coupled Riccati equations is necessary for theuniform convexity of the cost functional. In Section 5, we further prove that the solvability of the twocoupled Riccati equations is also sufficient for the uniform convexity of the cost functional. Moreover, astate feedback representation is obtained for the optimal control. Some illustrative examples are presentedin Section 6. Throughout this paper, we denote by R n × m the Euclidean space of all n × m real matrices, and by S n the space of all symmetric n × n real matrices. Recall that the inner product h· , ·i on R n × m is given by h M, N i 7→ tr ( M ⊤ N ), where the superscript ⊤ denotes the transpose of vectors or matrices, and the inducednorm is given by | M | = p tr ( M ⊤ M ). When there is no confusion, we shall use h· , ·i for inner products inpossibly different Hilbert spaces, and denote by | · | the norm induced by h· , ·i . For a matrix M ∈ R n × m ,we denote by R ( M ) the range of M , and if M ∈ S n , we use the notation M > >
0) to indicate that M is positive (semi-) definite. For a bounded linear operator A form a Banach X space into another Banachspace Y , we denote by A ∗ the adjoint operator of A . Let T > t ∈ [0 , T ]and Euclidean space H , we let L p ( t, T ; H ) (1 p ∞ ) be the space of all H -valued functions that are3 p -integrable on [ t, T ] and C ([ t, T ]; H ) be the space of all H -valued continuous functions on [ t, T ]. Next, weintroduce the following spaces: L F t (Ω; H ) = n ξ : Ω → H (cid:12)(cid:12) ξ is F t -measurable, E | ξ | < ∞ o ,L F ( t, T ; H ) = ( ϕ : [ t, T ] × Ω → H (cid:12)(cid:12) ϕ ( · ) is F -progressively measurable, E Z Tt | ϕ ( s ) | ds < ∞ ) ,L F (Ω; C ([ t, T ]; H )) = ( ϕ : [ t, T ] × Ω → H (cid:12)(cid:12) ϕ ( · ) is F -adapted, continuous, E sup s ∈ [ t,T ] | ϕ ( s ) | ! < ∞ ) ,L F (Ω; L ( t, T ; H )) = ϕ : [ t, T ] × Ω → H (cid:12)(cid:12) ϕ ( · ) is F -progressively measurable, E Z Tt | ϕ ( s ) | ds ! < ∞ . Further, we introduce the following notation: For any S n -valued measurable function F on [ t, T ], F > ⇐⇒ F ( s ) > , a.e. s ∈ [ t, T ] ,F > ⇐⇒ F ( s ) > , a.e. s ∈ [ t, T ] ,F ≫ ⇐⇒ F ( s ) > δI, a.e. s ∈ [ t, T ] , for some δ > . The following assumptions will be in force throughout this paper. (H1)
The coefficients of the state equation satisfy the following: ( A ( · ) , ¯ A ( · ) ∈ L (0 , T ; R n × n ) , B ( · ) , ¯ B ( · ) ∈ L (0 , T ; R n × m ) , b ( · ) ∈ L F (Ω; L (0 , T ; R n )) ,C ( · ) , ¯ C ( · ) ∈ L (0 , T ; R n × n ) , D ( · ) , ¯ D ( · ) ∈ L ∞ (0 , T ; R n × m ) , σ ( · ) ∈ L F (0 , T ; R n ) . (H2) The weighting coefficients in the cost functional satisfy the following: Q ( · ) , ¯ Q ( · ) ∈ L (0 , T ; S n ) , S ( · ) , ¯ S ( · ) ∈ L (0 , T ; R m × n ) , R ( · ) , ¯ R ( · ) ∈ L ∞ (0 , T ; S m ) ,g ∈ L F T (Ω; R n ) , q ( · ) ∈ L F (Ω; L (0 , T ; R n )) , ρ ( · ) ∈ L F (0 , T ; R m ) , ¯ g ∈ R n , ¯ q ( · ) ∈ L (0 , T ; R n ) , ¯ ρ ( · ) ∈ L (0 , T ; R m ) , G, ¯ G ∈ S n . By a standard argument using contraction mapping theorem, one can show that under (H1), for any( t, ξ ) ∈ [0 , T ) × L F t (Ω; R n ) and any u ( · ) ∈ U [ t, T ], (1.1) admits a unique solution X ( · ) ≡ X ( · ; t, ξ, u ( · )) ∈ L F (Ω; C ([ t, T ]; R n )). Hence, under (H1)–(H2), the cost functional (1.2) is well-defined, and Problem (MF-LQ) makes sense. Now we introduce the following definition. Definition 2.1. (i) Problem (MF-LQ) is said to be finite at initial pair ( t, ξ ) ∈ [0 , T ] × L F t (Ω; R n ) if(2.1) V ( t, ξ ) > −∞ . Problem (MF-LQ) is said to be finite at t ∈ [0 , T ] if (2.1) holds for all ξ ∈ L F t (Ω; R n ), and Problem (MF-LQ)is said to be finite if it is finite at all t ∈ [0 , T ].(ii) Problem (MF-LQ) is said to be ( uniquely ) open-loop solvable at initial pair ( t, ξ ) ∈ [0 , T ] × L F t (Ω; R n )if there exists a (unique) u ∗ ( · ) ∈ U [ t, T ] satisfying (1.3). Problem (MF-LQ) is said to be ( uniquely ) open-loopsolvable at t if for any ξ ∈ L F t (Ω; R n ), there exists a (unique) u ∗ ( · ) ∈ U [ t, T ] satisfying (1.3), and Problem(MF-LQ) is said to be ( uniquely ) open-loop solvable ( on [0 , T )) if it is (uniquely) open-loop solvable at all t ∈ [0 , T ). 4ext, we introduce the following mean-field backward SDE (MF-BSDE, for short) associated with thestate process X ( · ) ≡ X ( · ; t, ξ, u ( · )):(2.2) dY ( s ) = − n A ⊤ Y + ¯ A ⊤ E [ Y ] + C ⊤ Z + ¯ C ⊤ E [ Z ] + QX + ¯ Q E [ X ]+ S ⊤ u + ¯ S ⊤ E [ u ] + q + ¯ q o ds + ZdW ( s ) , s ∈ [ t, T ] ,Y ( T ) = GX ( T ) + ¯ G E [ X ( T )] + g + ¯ g. The following result is concerned with the differentiability of the map u ( · ) J ( t, ξ ; u ( · )). Proposition 2.2.
Let (H1)–(H2) hold and t ∈ [0 , T ) be given. For any ξ ∈ L F t (Ω; R n ) , λ ∈ R and u ( · ) , v ( · ) ∈ U [ t, T ] , the following holds: (2.3) J ( t, ξ ; u ( · ) + λv ( · )) − J ( t, ξ ; u ( · ))= λ J ( t, v ( · )) + 2 λ E Z Tt (cid:10) B ⊤ Y + ¯ B ⊤ E [ Y ] + D ⊤ Z + ¯ D ⊤ E [ Z ]+ SX + ¯ S E [ X ] + Ru + ¯ R E [ u ] + ρ + ¯ ρ, v (cid:11) ds, where X ( · ) = X ( · ; t, ξ, u ( · )) and ( Y ( · ) , Z ( · )) is the adapted solution to the MF-BSDE (2.2) associated with X ( · ) . Consequently, the map u ( · ) J ( t, ξ ; u ( · )) is Fr´echet differentiable with the Fr´echet derivative givenby (2.4) D J ( t, ξ ; u ( · ))( s ) = 2 n B ( s ) ⊤ Y ( s ) + ¯ B ( s ) ⊤ E [ Y ( s )] + D ( s ) ⊤ Z ( s ) + ¯ D ( s ) ⊤ E [ Z ( s )] + S ( s ) X ( s )+ ¯ S ( s ) E [ X ( s )] + R ( s ) u ( s ) + ¯ R ( s ) E [ u ( s )] + ρ ( s ) + ¯ ρ ( s ) (cid:3)o , s ∈ [ t, T ] . Proof.
Let b X ( · ) = X ( · ; t, ξ, u ( · ) + λv ( · )) and X ( · ) be the solution to the following MF-SDE: dX ( s ) = n A ( s ) X ( s ) + ¯ A ( s ) E [ X ( s )] + B ( s ) v ( s ) + ¯ B ( s ) E [ v ( s )] o ds + n C ( s ) X ( s ) + ¯ C ( s ) E [ X ( s )] + D ( s ) v ( s ) + ¯ D ( s ) E [ v ( s )] o dW ( s ) , s ∈ [ t, T ] ,X ( t ) = 0 , By the linearity of the state equation, b X ( · ) = X ( · ) + λX ( · ). Hence, J ( t, ξ ; u ( · ) + λv ( · )) − J ( t, ξ ; u ( · ))= λ E ((cid:10) G (cid:2) X ( T ) + λX ( T ) (cid:3) , X ( T ) (cid:11) + 2 h g, X ( T ) i + Z Tt "* Q S ⊤ S R ! X + λX u + λv ! , X v !+ + 2 * qρ ! , X v !+ ds ) + λ ( D ¯ G (cid:16) E [ X ( T )] + λ E [ X ( T )] (cid:17) , E [ X ( T )] E + 2 h ¯ g, E [ X ( T )] i + Z Tt "* ¯ Q ¯ S ⊤ ¯ S ¯ R ! E [ X ] + λ E [ X ]2 E [ u ] + λ E [ v ] ! , E [ X ] E [ v ] !+ + 2 * ¯ q ¯ ρ ! , E [ X ] E [ v ] !+ ds ) = 2 λ E ( h GX ( T ) + g, X ( T ) i + Z Tt h h QX + S ⊤ u + q, X i + h SX + Ru + ρ, v i i ds ) + λ E ( h GX ( T ) , X ( T ) i + Z Tt * Q S ⊤ S R ! X v ! , X v !+ ds ) λ ( h ¯ G E [ X ( T )]+¯ g, E [ X ( T )] i + Z Tt h h ¯ Q E [ X ]+ ¯ S ⊤ E [ u ]+ ¯ q, E [ X ] i + h ¯ S E [ X ]+ ¯ R E [ u ]+ ¯ ρ, E [ v ] i i ds ) + λ ( h ¯ G E [ X ( T )] , E [ X ( T )] i + Z Tt * ¯ Q ¯ S ⊤ ¯ S ¯ R ! E [ X ] E [ v ] ! , E [ X ] E [ v ] !+ ds ) = 2 λ E ( h GX ( T ) + ¯ G E [ X ( T )] + g + ¯ g, X ( T ) i + Z Tt h(cid:10) QX + ¯ Q E [ X ]+ S ⊤ u + ¯ S ⊤ E [ u ]+ q + ¯ q, X (cid:11) + (cid:10) SX + ¯ S E [ X ]+ Ru + ¯ R E [ u ]+ ρ + ¯ ρ, v (cid:11)i ds ) + λ J ( t, v ( · )) . Now applying Itˆo’s formula to s
7→ h Y ( s ) , X ( s ) i , we have E h GX ( T ) + ¯ G E [ X ( T )] + g + ¯ g, X ( T ) i = E Z Tt n − h A ⊤ Y + ¯ A ⊤ E [ Y ] + C ⊤ Z + ¯ C ⊤ E [ Z ] + QX + ¯ Q E [ X ] + S ⊤ u + ¯ S ⊤ E [ u ] + q + ¯ q, X i + h AX + ¯ A E [ X ] + Bv + ¯ B E [ v ] , Y i + h CX + ¯ C E [ X ] + Dv + ¯ D E [ v ] , Z i o ds = E Z Tt n h B ⊤ Y + ¯ B ⊤ E [ Y ] + D ⊤ Z + ¯ D ⊤ E [ Z ] , v i − h QX + ¯ Q E [ X ] + S ⊤ u + ¯ S ⊤ E [ u ] + q + ¯ q, X i o ds. Combining the above equalities, we obtain (2.3).From the above, we have the following result, which gives a characterization for the optimal controls ofProblem (MF-LQ).
Theorem 2.3.
Let (H1)–(H2) hold and ( t, ξ ) ∈ [0 , T ) × L F t (Ω; R n ) be given. Let u ∗ ( · ) ∈ U [ t, T ] and ( X ∗ ( · ) , Y ∗ ( · ) , Z ∗ ( · )) be the adapted solution to the following (decoupled) mean-field forward-backwardstochastic differential equation (MF-FBSDE, for short): (2.5) dX ∗ ( s ) = n AX ∗ + ¯ A E [ X ∗ ] + Bu ∗ + ¯ B E [ u ∗ ] + b o ds + n CX ∗ + ¯ C E [ X ∗ ] + Du ∗ + ¯ D E [ u ∗ ] + σ o dW ( s ) , s ∈ [ t, T ] ,dY ∗ ( s ) = − n A ⊤ Y ∗ + ¯ A ⊤ E [ Y ∗ ] + C ⊤ Z ∗ + ¯ C ⊤ E [ Z ∗ ] + QX ∗ + ¯ Q E [ X ∗ ]+ S ⊤ u ∗ + ¯ S ⊤ E [ u ∗ ] + q + ¯ q o ds + Z ∗ dW ( s ) , s ∈ [ t, T ] ,X ∗ ( t ) = ξ, Y ∗ ( T ) = GX ∗ ( T ) + ¯ G E [ X ∗ ( T )] + g + ¯ g. Then u ∗ ( · ) is an optimal control of Problem (MF-LQ) for the initial pair ( t, ξ ) if and only if (2.6) J ( t, u ( · )) > , ∀ u ( · ) ∈ U [ t, T ] , and the following stationarity condition holds: (2.7) D J ( t, ξ ; u ∗ ( · )) = 2 n B ⊤ Y ∗ + D ⊤ Z ∗ + SX ∗ + Ru ∗ + ρ + ¯ B ⊤ E [ Y ∗ ] + ¯ D ⊤ E [ Z ∗ ] + ¯ S E [ X ∗ ] + ¯ R E [ u ∗ ] + ¯ ρ o = 0 , a.e. a.s. Proof.
By (2.3), we see that u ∗ ( · ) is an optimal control of Problem (MF-LQ) for the initial pair ( t, ξ ) ifand only if λ J ( t, u ( · )) + λ E Z Tt hD J ( t, ξ ; u ∗ ( · ))( s ) , u ( s ) i ds = J ( t, ξ ; u ∗ ( · ) + λu ( · )) − J ( t, ξ ; u ∗ ( · )) > , ∀ λ ∈ R , ∀ u ( · ) ∈ U [ t, T ] , E Z Tt hD J ( t, ξ ; u ∗ ( · ))( s ) , u ( s ) i ds , ∀ u ( · ) ∈ U [ t, T ] . Note that the above inequality holds for all u ( · ) ∈ U [ t, T ] if and only if D J ( t, ξ ; u ∗ ( · ))( · ) = 0. The resulttherefore follows. We begin with a representation of the cost functional. For any u ( · ) ∈ U [ t, T ], let X u ( · ) be the solution of(3.1) dX u ( s ) = n A ( s ) X u ( s ) + ¯ A ( s ) E [ X u ( s )] + B ( s ) u ( s ) + ¯ B ( s ) E [ u ( s )] o ds + n C ( s ) X u ( s ) + ¯ C ( s ) E [ X u ( s )] + D ( s ) u ( s ) + ¯ D ( s ) E [ u ( s )] o dW ( s ) , s ∈ [ t, T ] ,X u ( t ) = 0 . By the linearity of (3.1), we can define bounded linear operators L t : U [ t, T ] → L F ( t, T ; R n ) and b L t : U [ t, T ] → L F T (Ω; R n ) by u ( · ) X u ( · ) and u ( · ) X u ( T ), respectively, via the MF-SDE (3.1). Then J ( t, u ( · )) = E ( h GX u ( T ) , X u ( T ) i + h ¯ G E [ X u ( T )] , E [ X u ( T )] i + Z Tt * Q ( s ) S ( s ) ⊤ S ( s ) R ( s ) ! X u ( s ) u ( s ) ! , X u ( s ) u ( s ) !+ ds + Z Tt * ¯ Q ( s ) ¯ S ( s ) ⊤ ¯ S ( s ) ¯ R ( s ) ! E [ X u ( s )] E [ u ( s )] ! , E [ X u ( s )] E [ u ( s )] !+ ds ) = (cid:10) G b L t u, b L t u (cid:11) + (cid:10) ¯ G E [ b L t u ] , E [ b L t u ] (cid:11) + (cid:10) Q L t u, L t u (cid:11) + 2 (cid:10) S L t u, u (cid:11) + (cid:10) Ru, u (cid:11) + (cid:10) ¯ Q E [ L t u ] , E [ L t u ] (cid:11) + 2 (cid:10) ¯ S E [ L t u ] , E [ u ] (cid:11) + (cid:10) ¯ R E [ u ] , E [ u ] (cid:11) = (cid:10)(cid:2) b L ∗ t ( G + E ∗ ¯ G E ) b L t + L ∗ t ( Q + E ∗ ¯ Q E ) L t + ( S + E ∗ ¯ S E ) L t + L ∗ t ( S ⊤ + E ∗ ¯ S ⊤ E ) + ( R + E ∗ ¯ R E ) (cid:3) u, u (cid:11) . Denote(3.2) M t , b L ∗ t ( G + E ∗ ¯ G E ) b L t + L ∗ t ( Q + E ∗ ¯ Q E ) L t + ( S + E ∗ ¯ S E ) L t + L ∗ t ( S ⊤ + E ∗ ¯ S ⊤ E ) + ( R + E ∗ ¯ R E ) , which is a bounded self-adjoint linear operator on U [ t, T ]. Then by Proposition 2.2, the cost functional J ( t, ξ ; u ( · )) can be written as(3.3) J ( t, ξ ; u ( · )) = hM t u, u i + hD J ( t, ξ ; 0) , u i + J ( t, ξ ; 0) , ∀ ( t, ξ ) ∈ [0 , T ] × L F t (Ω; R n ) , ∀ u ( · ) ∈ U [ t, T ] . Now let us introduce the following conditions. (H3)
The following holds:(3.4) J ( t, u ( · )) > , ∀ u ( · ) ∈ U [ t, T ] . (H4) There exists a constant δ > J ( t, u ( · )) > δ E Z Tt | u ( s ) | ds, ∀ u ( · ) ∈ U [ t, T ] . u ( · ) J ( t, ξ ; u ( · )) is convex if and only if(3.6) M t > , which is also equivalent to (H3), and u ( · ) J ( t, ξ ; u ( · )) is uniformly convex if and only if(3.7) M t > δI, for some δ > , which is also equivalent to (H4). The following result tells us that (H3) is necessary for the finiteness(and open-loop solvability) of Problem (MF-LQ) at t , and (H4) is sufficient for the open-loop solvability ofProblem (MF-LQ) at t . Proposition 3.1.
Let (H1)–(H2) hold and t ∈ [0 , T ) be given. We have the following: (i) If Problem (MF-LQ) is finite at t , then (H3) must hold. (ii) Suppose (H4) holds. Then Problem (MF-LQ) is uniquely open-loop solvable at t , and the uniqueoptimal control for the initial pair ( t, ξ ) is given by (3.8) u ∗ ( · ) = − M − t D J ( t, ξ ; 0)( · ) . Moreover, (3.9) V ( t, ξ ) = J ( t, ξ ; 0) − (cid:12)(cid:12)(cid:12) M − t D J ( t, ξ ; 0) (cid:12)(cid:12)(cid:12) . Proof. (i) We prove the result by contradiction. Suppose that J ( t, u ( · )) < u ( · ) ∈ U [ t, T ].By Proposition 2.2, we have J ( t, ξ ; λu ( · )) = J ( t, ξ ; 0) + λ J ( t, u ( · )) + λ E Z Tt hD J ( t, ξ ; 0)( s ) , u ( s ) i ds, ∀ λ ∈ R . Letting λ → ∞ , we obtain that V ( t, ξ ) lim λ →∞ J ( t, ξ ; λu ( · )) = −∞ , which is a contradiction.(ii) Suppose (H4) holds. Then the operator M t is invertible, and J ( t, ξ ; u ( · )) = (cid:12)(cid:12)(cid:12)(cid:12) M t u + 12 M − t D J ( t, ξ ; 0) (cid:12)(cid:12)(cid:12)(cid:12) + J ( t, ξ ; 0) − (cid:12)(cid:12)(cid:12) M − t D J ( t, ξ ; 0) (cid:12)(cid:12)(cid:12) , > J ( t, ξ ; 0) − (cid:12)(cid:12)(cid:12) M − t D J ( t, ξ ; 0) (cid:12)(cid:12)(cid:12) , ∀ ξ ∈ L F t (Ω; R n ) , ∀ u ( · ) ∈ U [ t, T ] . Note that the equality in the above holds if and only if u = − M − t D J ( t, ξ ; 0) . The result therefore follows.Due to the necessity of (H3) for the finiteness of Problem (MF-LQ), we will assume (H3) holds in therest of this paper. Now for any ε >
0, consider state equation (1.1) and the following cost functional:(3.10) J ε ( t, ξ ; u ( · )) , J ( t, ξ ; u ( · )) + ε E Z Tt | u ( s ) | ds = h ( M t + εI ) u, u i + hD J ( t, ξ ; 0) , u i + J ( t, ξ ; 0) . ε and V ε ( · , · ),respectively. By Proposition 3.1, part (ii), for any ξ ∈ L F t (Ω; R n ), Problem (MF-LQ) ε admits a uniqueoptimal control(3.11) u ∗ ε ( · ) = −
12 ( M t + εI ) − D J ( t, ξ ; 0)( · ) , and the value function is given by(3.12) V ε ( t, ξ ) = J ( t, ξ ; 0) − (cid:12)(cid:12)(cid:12) ( M t + εI ) − D J ( t, ξ ; 0) (cid:12)(cid:12)(cid:12) . Now, we are ready to state the main result of this section.
Theorem 3.2.
Let (H1)–(H3) hold and ξ ∈ L F t (Ω; R n ) . We have the following: (i) lim ε → V ε ( t, ξ ) = V ( t, ξ ) . In particular, Problem (MF-LQ) is finite at ( t, ξ ) if and only if { V ε ( t, ξ ) } ε> is bounded from below. (ii) The sequence { u ∗ ε ( · ) } ε> defined by (3.11) is a minimizing sequence of u ( · ) J ( t, ξ ; u ( · )) : (3.13) lim ε → J ( t, ξ ; u ∗ ε ( · )) = inf u ( · ) ∈U [ t,T ] J ( t, ξ ; u ( · )) = V ( t, ξ ) . (iii) The following statements are equivalent: (a)
Problem (MF-LQ) is open-loop solvable at ( t, ξ ) ; (b) The sequence { u ∗ ε ( · ) } ε> is bounded in U [ t, T ] ; (c) The sequence { u ∗ ε ( · ) } ε> admits a weakly convergent subsequence; (d) The sequence { u ∗ ε ( · ) } ε> admits a strongly convergent subsequence.In this case, the weak ( strong ) limit of any weakly ( strongly ) convergent subsequence of { u ∗ ε ( · ) } ε> is anoptimal control of Problem (MF-LQ) at ( t, ξ ) . To prove Theorem 3.2, we need the following lemma.
Lemma 3.3.
Let H be a Hilbert space with norm | · | and θ, θ n ∈ H , n = 1 , , · · · . (i) If θ n → θ weakly, then | θ | lim n →∞ | θ n | . (ii) θ n → θ strongly if and only if | θ n | → | θ | and θ n → θ weakly . Proof of Theorem ε > ε >
0, we have J ε ( t, ξ ; u ( · )) > J ε ( t, ξ ; u ( · )) > J ( t, ξ ; u ( · )) , ∀ u ( · ) ∈ U [ t, T ] , which implies that(3.14) V ε ( t, ξ ) > V ε ( t, ξ ) > V ( t, ξ ) , ∀ ε > ε > . Thus, the limit lim ε → V ε ( t, ξ ) exists and(3.15) ¯ V ( t, ξ ) ≡ lim ε → V ε ( t, ξ ) > V ( t, ξ ) .
9n the other hand, for any
K, δ >
0, we can find a u δ ( · ) ∈ U [ t, T ], such that V ε ( t, ξ ) J ( t, ξ ; u δ ( · )) + ε E Z Tt | u δ ( s ) | ds max { V ( t, ξ ) , − K } + δ + ε E Z Tt | u δ ( s ) | ds. Letting ε →
0, we obtain that ¯ V ( t, ξ ) max { V ( t, ξ ) , − K } + δ, ∀ K, δ > , from which we see that(3.16) ¯ V ( t, ξ ) V ( t, ξ ) . Combining (3.15)–(3.16), we obtain the desired result.(ii) If V ( t, ξ ) > −∞ , then by (i), we have ε E Z Tt | u ∗ ε ( s ) | ds = J ε ( t, ξ ; u ∗ ε ( · )) − J ( t, ξ ; u ∗ ε ( · )) = V ε ( t, ξ ) − J ( t, ξ ; u ∗ ε ( · )) V ε ( t, ξ ) − V ( t, ξ ) → ε → . Hence, lim ε → J ( t, ξ ; u ∗ ε ( · )) = lim ε → (cid:20) V ε ( t, ξ ) − ε E Z Tt | u ∗ ε ( s ) | ds (cid:21) = V ( t, ξ ) . If V ( t, ξ ) = −∞ , then by (i), we have J ( t, ξ ; u ∗ ε ( · )) J ε ( t, ξ ; u ∗ ε ( · )) = V ε ( t, ξ ) → −∞ as ε → , and (3.13) still holds.(iii) (b) ⇒ (c) and (d) ⇒ (c) are obvious. We next prove (c) ⇒ (a). Let { u ∗ ε k ( · ) } k > be a weaklyconvergent subsequence of { u ∗ ε ( · ) } ε> with weak limit u ∗ ( · ). Then { u ∗ ε k ( · ) } k > is bounded in U [ t, T ]. Forany u ( · ) ∈ U [ t, T ], we have(3.17) J ( t, ξ ; u ∗ ε k ( · )) + ε k E Z Tt | u ∗ ε k ( s ) | ds = V ε k ( t, ξ ) J ( t, ξ ; u ( · )) + ε k E Z Tt | u ( s ) | ds. Note that u ( · ) J ( t, ξ ; u ( · )) is sequentially weakly lower semi-continuous. Letting k → ∞ in (3.17), weobtain J ( t, ξ ; u ∗ ( · )) lim k →∞ J ( t, ξ ; u ∗ ε k ( · )) J ( t, ξ ; u ( · )) , ∀ u ( · ) ∈ U [ t, T ] . Hence, u ∗ ( · ) is an optimal control of Problem (MF-LQ) at ( t, ξ ). Now it remains to show (a) ⇒ (b) and (a) ⇒ (d). Suppose v ∗ ( · ) is an optimal control of Problem (MF-LQ) at ( t, ξ ). Then for any ε >
0, we have V ε ( t, ξ ) = J ε ( t, ξ ; u ∗ ε ( · )) > V ( t, ξ ) + ε E Z Tt | u ∗ ε ( s ) | ds,V ε ( t, ξ ) J ε ( t, ξ ; v ∗ ( · )) = V ( t, ξ ) + ε E Z Tt | v ∗ ( s ) | ds, from which we see that(3.18) E Z Tt | u ∗ ε ( s ) | ds V ε ( t, ξ ) − V ( t, ξ ) ε E Z Tt | v ∗ ( s ) | ds, ∀ ε > . { u ∗ ε ( · ) } ε> is bounded in the Hilbert space U [ t, T ] and hence admits a weakly convergent subsequence { u ∗ ε k ( · ) } k > . Let u ∗ ( · ) be the weak limit of { u ∗ ε k ( · ) } k > . By the proof of (c) ⇒ (a), we see that u ∗ ( · ) is alsoan optimal control of Problem (MF-LQ) at ( t, ξ ). Replacing v ∗ ( · ) with u ∗ ( · ) in (3.18), we have(3.19) E Z Tt | u ∗ ε ( s ) | ds E Z Tt | u ∗ ( s ) | ds, ∀ ε > . Also, by Lemma 3.3, part (i),(3.20) E Z Tt | u ∗ ( s ) | ds lim k →∞ E Z Tt | u ∗ ε k ( s ) | ds. Combining (3.19)–(3.20), we have E Z Tt | u ∗ ( s ) | ds = lim k →∞ E Z Tt | u ∗ ε k ( s ) | ds. Then it follows from Lemma 3.3, part (ii), that { u ∗ ε k ( · ) } k > converges to u ∗ ( · ) strongly. Theorem 3.2 tells us that in order to solve Problem (MF-LQ), we need only solve mean-filed LQ problems withuniformly convex cost functionals and then pass to the limit. By Proposition 3.1, under the uniform convexitycondition (H4), the unique optimal control u ∗ ( · ) for the initial pair ( t, ξ ) is determined by (3.8). However,such a representation is not easy to compute, since M − t is in an abstract form and very complicated. Thus,we would like to find some more explicit form of the optimal control. In this section we shall investigateuniform convexity of the cost functional and show the necessity of solvability of two Riccati equations forthe uniform convexity of the cost functional.First, we present the following result concerning the value function of Problem (MF-LQ) . Proposition 4.1.
Let (H1)–(H2) and (H4) hold. Then there exists a constant α ∈ R such that (4.1) V ( s, ξ ) > α E (cid:2) | ξ | (cid:3) , ∀ ( s, ξ ) ∈ [ t, T ] × L F s (Ω; R n ) with E [ ξ ] = 0 . Proof.
For any s ∈ [ t, T ] and any u ( · ) ∈ U [ s, T ], we define the zero-extension of u ( · ) as follows:(4.2) [ 0 I [ t,s ) ⊕ u ( · )]( r ) = ( , r ∈ [ t, s ) ,u ( r ) , r ∈ [ s, T ] . Then v ( · ) ≡ I [ t,s ) ⊕ u ( · ) ∈ U [ t, T ], and due to the initial state being 0, the solution X v ( · ) of dX v ( r ) = (cid:8) A ( r ) X v ( r ) + ¯ A ( r ) E [ X v ( r )] + B ( r ) v ( r ) + ¯ B ( r ) E [ v ( r )] (cid:9) dr + (cid:8) C ( r ) X v ( r ) + ¯ C ( r ) E [ X v ( r )] + D ( r ) v ( r ) + ¯ D ( r ) E [ v ( r )] (cid:9) dW ( r ) , r ∈ [ t, T ] ,X v ( t ) = 0 , satisfies X v ( r ) = 0 , r ∈ [ t, s ]. Hence,(4.3) J ( s, u ( · )) = J ( t,
0; 0 I [ t,s ) ⊕ u ( · )) > δ E Z Tt (cid:12)(cid:12) [0 I [ t,s ) ⊕ u ( · )]( r ) (cid:12)(cid:12) dr = δ E Z Ts | u ( r ) | dr. X ( · ) , Y ( · ) , Z ( · )) be the solution of the following (decoupled) MF-FBSDE:(4.4) dX ( r ) = (cid:8) AX + ¯ A E [ X ] (cid:9) dr + (cid:8) CX + ¯ C E [ X ] (cid:9) dW ( r ) , r ∈ [ s, T ] ,dY ( r ) = − (cid:8) A ⊤ Y + ¯ A ⊤ E [ Y ] + C ⊤ Z + ¯ C ⊤ E [ Z ] + QX + ¯ Q E [ X ] (cid:9) dr + ZdW ( r ) , r ∈ [ s, T ] ,X ( s ) = ξ, Y ( T ) = GX ( T ) + ¯ G E [ X ( T )] . By Proposition 2.2 and (4.3), we have(4.5) J ( s, ξ ; u ( · )) − J ( s, ξ ; 0)= J ( s, u ( · )) + 2 E Z Ts (cid:10) B ⊤ Y + ¯ B ⊤ E [ Y ]+ D ⊤ Z + ¯ D ⊤ E [ Z ]+ SX + ¯ S E [ X ] , u (cid:11)i dr > J ( s, u ( · )) − δ E Z Ts | u ( r ) | dr − δ E Z Ts (cid:12)(cid:12) B ⊤ Y + ¯ B ⊤ E [ Y ]+ D ⊤ Z + ¯ D ⊤ E [ Z ]+ SX + ¯ S E [ X ] (cid:12)(cid:12) dr > − δ E Z Ts (cid:12)(cid:12) B ⊤ Y + ¯ B ⊤ E [ Y ]+ D ⊤ Z + ¯ D ⊤ E [ Z ]+ SX + ¯ S E [ X ] (cid:12)(cid:12) dr. If E [ ξ ] = 0, then E [ X ( · )] ≡
0, and one can verify that(4.6) X ( r ) = X ( r ) X ( s ) − ξ, Y ( r ) = Y ( r ) X ( s ) − ξ, Z ( r ) = Z ( r ) X ( s ) − ξ, r ∈ [ s, T ] , where X ( · ) is the solution to the following R n × n -valued SDE:(4.7) ( d X ( r ) = A ( r ) X ( r ) dr + C ( r ) X ( r ) dW ( r ) , r ∈ [0 , T ] , X (0) = I, and ( Y ( · ) , Z ( · )) is the adapted solution to the following R n × n -valued backward SDE (BSDE, for short):(4.8) ( d Y ( r ) = − (cid:2) A ( r ) ⊤ Y ( r ) + C ( r ) ⊤ Z ( r ) + Q ( r ) X ( r ) (cid:3) dr + Z ( r ) dW ( r ) , r ∈ [0 , T ] , Y ( T ) = G X ( T ) . Note that X ( r ) X ( s ) − , Y ( r ) X ( s ) − and Z ( r ) X ( s ) − are independent of F s . Thus, E [ X ( · )] = E [ Y ( · )] = E [ Z ( · )] = 0 and (noting (4.5)) J ( s, ξ ; u ( · )) > J ( s, ξ ; 0) − δ E Z Ts (cid:12)(cid:12) B ( r ) ⊤ Y ( r ) + D ( r ) ⊤ Z ( r ) + S ( r ) X ( r ) (cid:12)(cid:12) dr = E ( h GX ( T ) , X ( T ) i + Z Ts h Q ( r ) X ( r ) , X ( r ) i dr ) − δ E Z Ts (cid:12)(cid:12) B ( r ) ⊤ Y ( r ) + D ( r ) ⊤ Z ( r ) + S ( r ) X ( r ) (cid:12)(cid:12) dr = E ( ξ ⊤ (cid:2) X ( s ) − (cid:3) ⊤ X ( T ) ⊤ G X ( T ) X ( s ) − + Z Ts (cid:2) X ( s ) − (cid:3) ⊤ X ( r ) ⊤ Q ( r ) X ( r ) X ( s ) − dr ! ξ ) − δ E Z Ts ξ ⊤ (cid:2) X ( s ) − (cid:3) ⊤ (cid:2) B ( r ) ⊤ Y ( r ) + D ( r ) ⊤ Z ( r ) + S ( r ) X ( r ) (cid:3) ⊤ · (cid:2) B ( r ) ⊤ Y ( r ) + D ( r ) ⊤ Z ( r ) + S ( r ) X ( r ) (cid:3) X ( s ) − ξdr = E ( ξ ⊤ E (cid:2) X ( s ) − (cid:3) ⊤ X ( T ) ⊤ G X ( T ) X ( s ) − + Z Ts (cid:2) X ( s ) − (cid:3) ⊤ X ( r ) ⊤ Q ( r ) X ( r ) X ( s ) − dr − δ Z Ts (cid:2) X ( s ) − (cid:3) ⊤ (cid:2) B ( r ) ⊤ Y ( r ) + D ( r ) ⊤ Z ( r ) + S ( r ) X ( r ) (cid:3) ⊤ · (cid:2) B ( r ) ⊤ Y ( r ) + D ( r ) ⊤ Z ( r ) + S ( r ) X ( r ) (cid:3) X ( s ) − dr ! ξ ) ≡ E (cid:2) ξ ⊤ M ( s ) ξ (cid:3) . M ( · ) : [ t, T ] → S n is continuous. The result therefore follows.Now, let us introduce the following Riccati equation:(4.9) ˙ P + P A + A ⊤ P + C ⊤ P C + Q − (cid:0) P B + C ⊤ P D + S ⊤ (cid:1) (cid:0) R + D ⊤ P D (cid:1) − (cid:0) B ⊤ P + D ⊤ P C + S (cid:1) = 0 , a.e. s ∈ [ t, T ] ,P ( T ) = G. A solution P ( · ) of (4.9) is said to be strongly regular if(4.10) R ( s ) + D ( s ) ⊤ P ( s ) D ( s ) > δI, a.e. s ∈ [ t, T ] , for some δ >
0. The Riccati equation (4.9) is said to be strongly regularly solvable , if it admits a stronglyregular solution. By a standard argument using Gronwall’s inequality, one can show that if the regularsolution of (4.9) exists, it must be unique. Compared with the strongly regular solution, the notion of regular solution , which is closely related to the closed-loop strategy, was introduced in [26]. The interestedreader is referred to [25] for further information.The following result shows that the strongly regular solvability of the Riccati equation (4.9) is necessaryfor the uniform convexity of the cost functional.
Theorem 4.2.
Let (H1)–(H2) and (H4) hold. Then the Riccati equation (4.9) is strongly regularlysolvable.
To prove the above result, we need the following lemma, whose proof can be found in [25].
Lemma 4.3.
Let (H1)–(H2) hold. For any Θ( · ) ∈ L ( t, T ; R m × n ) , let P Θ ( · ) ∈ C ([ t, T ]; S n ) be thesolution to the following Lyapunov equation: (4.11) ˙ P Θ + P Θ ( A + B Θ) + ( A + B Θ) ⊤ P Θ + ( C + D Θ) ⊤ P Θ ( C + D Θ)+ Θ ⊤ R Θ + S ⊤ Θ + Θ ⊤ S + Q = 0 , a.e. s ∈ [ t, T ] ,P Θ ( T ) = G. If there exists a constant β > such that for all Θ( · ) ∈ L ( t, T ; R m × n ) , (4.12) P Θ ( s ) , R ( s ) + D ( s ) ⊤ P Θ ( s ) D ( s ) > βI a.e. s ∈ [ t, T ] , then the Riccati equation (4.9) is strongly regularly solvable. Proof of Theorem . We only need to show that the condition stated in Lemma 4.3 holds. To this end,let Θ( · ) ∈ L ( t, T ; R m × n ) and P ( · ) ≡ P Θ ( · ) be the corresponding solution of (4.11). For any deterministic u ( · ) ∈ L ( t, T ; R m ), let X u ( · ) be the solution of(4.13) ( dX u ( s ) = (cid:2) ( A + B Θ) X u + BuW (cid:3) ds + (cid:2) ( C + D Θ) X u + DuW (cid:3) dW, s ∈ [ t, T ] ,X u ( t ) = 0 , and set v ( · ) , Θ( · ) X u ( · ) + u ( · ) W ( · ) ∈ U [ t, T ] . Clearly,(4.14) E [ X u ( s )] = 0 , E [ v ( s )] = 0 , s ∈ [ t, T ] .
13y the uniqueness of solutions, X u ( · ) also solves(4.15) ( dX u ( s ) = (cid:8) AX u + ¯ A E [ X u ]+ Bv + ¯ B E [ v ] (cid:9) ds + (cid:8) CX u + ¯ C E [ X u ]+ Dv + ¯ D E [ v ] (cid:9) dW, s ∈ [ t, T ] ,X u ( t ) = 0 . Thus, by applying Itˆo’s formula to s → h P ( s ) X u ( s ) , X u ( s ) i , we have (noting (H4) and (4.14)) δ E Z Tt | Θ( s ) X u ( s ) + u ( s ) W ( s ) | ds = δ E Z Tt | v ( s ) | ds J ( t, v ( · ))= E ( h GX u ( T ) , X u ( T ) i + Z Tt h h QX u , X u i + 2 h SX u , v i + h Rv, v i i ds ) = E Z Tt n(cid:10) ˙ P X u , X u (cid:11) + (cid:10) P (cid:2) ( A + B Θ) X u + BuW (cid:3) , X u (cid:11) + (cid:10) P X u , ( A + B Θ) X u + BuW (cid:11) + (cid:10) P (cid:2) ( C + D Θ) X u + DuW (cid:3) , ( C + D Θ) X u + DuW (cid:11) + (cid:10) QX u , X u (cid:11) + 2 (cid:10) SX u , Θ X u + uW (cid:11) + (cid:10) R (Θ X u + uW ) , Θ X u + uW (cid:11)o ds = E Z Tt n (cid:10)(cid:2) B ⊤ P + D ⊤ P C + S + ( R + D ⊤ P D )Θ (cid:3) X u , uW (cid:11) + (cid:10) ( R + D ⊤ P D ) uW, uW (cid:11)o ds. Hence, for any u ( · ) ∈ L ( t, T ; R m ), the following holds:(4.16) E Z Tt n (cid:10)(cid:2) B ⊤ P + D ⊤ P C + S + ( R + D ⊤ P D − δI )Θ (cid:3) W X u , u (cid:11) + W (cid:10) ( R + D ⊤ P D − δI ) u, u (cid:11)o ds = δ E Z Tt | Θ( s ) X u ( s ) | ds > . Now, applying Itˆo’s formula again, we have d E (cid:2) W ( s ) X u ( s ) (cid:3) = n(cid:2) A ( s ) + B ( s )Θ( s ) (cid:3) E (cid:2) W ( s ) X u ( s ) (cid:3) + sB ( s ) u ( s ) o ds, s ∈ [ t, T ] , E (cid:2) W ( t ) X u ( t ) (cid:3) = 0 . Fix any u ∈ R m , take u ( s ) = u [ t ′ ,t ′ + h ] ( s ), with t t ′ < t ′ + h T . Then E (cid:2) W ( s ) X u ( s ) (cid:3) = , s ∈ [ t, t ′ ] , Φ( s ) Z s ∧ ( t ′ + h ) t Φ( r ) − B ( r ) ru dr, s ∈ [ t ′ , T ] , where Φ( · ) is the solution of the following R n × n -valued ordinary differential equation (ODE, for short): ( ˙Φ( s ) = (cid:2) A ( s ) + B ( s )Θ( s ) (cid:3) Φ( s ) , s ∈ [0 , T ] , Φ(0) = I. Consequently, (4.16) becomes Z t ′ + ht ′ n (cid:10)(cid:2) B ⊤ P + D ⊤ P C + S + ( R + D ⊤ P D − δI )Θ (cid:3) Φ( s ) Z st Φ( r ) − B ( r ) ru dr, u (cid:11) + s (cid:10) ( R + D ⊤ P D − δI ) u , u (cid:11)o ds > . Dividing both sides by h and letting h →
0, we obtain t ′ (cid:10)(cid:2) R ( t ′ ) + D ( t ′ ) ⊤ P ( t ′ ) D ( t ′ ) − δI (cid:3) u , u (cid:11) > , ∀ u ∈ R m , a.e. t ′ ∈ [ t, T ] , R ( s ) + D ( s ) ⊤ P ( s ) D ( s ) > δI, a.e. s ∈ [ t, T ] . Next, for any ( s, x ) ∈ [ t, T ] × R n , let X ( · ) be the solution of ( dX ( r ) = (cid:2) A ( r ) + B ( r )Θ( r ) (cid:3) X ( r ) dr + (cid:2) C ( r ) + D ( r )Θ( r ) (cid:3) X ( r ) dW ( r ) , r ∈ [ s, T ] ,X ( s ) = W ( s ) x, and set w ( · ) , Θ( · ) X ( · ) ∈ U [ s, T ] . Similar to the previous argument, by applying Itˆo’s formula to r → h P ( r ) X ( r ) , X ( r ) i , we can derive that(4.18) J ( s, W ( s ) x ; w ( · )) = E h P ( s ) W ( s ) x, W ( s ) x i = s h P ( s ) x, x i . By Proposition 4.1, we have s h P ( s ) x, x i = J ( s, W ( s ) x ; w ( · )) > α E (cid:2) | W ( s ) x | (cid:3) = sα | x | , ∀ ( s, x ) ∈ [ t, T ] × R n , which implies that P ( s ) > αI, ∀ s ∈ [ t, T ]. The proof is completed.From Theorem 4.2, we see that the Riccati equation (4.9) is strongly regularly solvable under the uniformconvexity condition (H4). With the strongly regular solution P ( · ) of (4.9), we may further introduce thefollowing deterministic LQ optimal control problem.Consider the state equation(4.19) ( ˙ y ( s ) = (cid:2) A ( s ) + ¯ A ( s ) (cid:3) y ( s ) + (cid:2) B ( s ) + ¯ B ( s ) (cid:3) v ( s ) , s ∈ [ t, T ] ,y ( t ) = x, and cost functional(4.20) ¯ J ( t, x ; v ( · )) , (cid:10) ( G + ¯ G ) y ( T ) , y ( T ) (cid:11) + Z Tt h h Υ y, y i + 2 h Γ y, v i + h Σ v, v i i ds, where(4.21) Υ = Q + ¯ Q + ( C + ¯ C ) ⊤ P ( C + ¯ C ) , Γ = ( D + ¯ D ) ⊤ P ( C + ¯ C ) + S + ¯ S, Σ = R + ¯ R + ( D + ¯ D ) ⊤ P ( D + ¯ D ) . We pose the following deterministic LQ problem.
Problem (DLQ).
For any given ( t, x ) ∈ [0 , T ) × R n , find a v ∗ ( · ) ∈ L ( t, T ; R m ), such that(4.22) ¯ J ( t, x ; v ∗ ( · )) = inf v ( · ) ∈ L ( t,T ; R m ) ¯ J ( t, x ; v ( · )) . Note that the Riccati equation associated with Problem (DLQ) is(4.23) ˙Π + Π( A + ¯ A ) + ( A + ¯ A ) ⊤ Π + Q + ¯ Q + ( C + ¯ C ) ⊤ P ( C + ¯ C ) − (cid:2) Π( B + ¯ B ) + ( C + ¯ C ) ⊤ P ( D + ¯ D ) + ( S + ¯ S ) ⊤ (cid:3)(cid:2) R + ¯ R + ( D + ¯ D ) ⊤ P ( D + ¯ D ) (cid:3) − · (cid:2) ( B + ¯ B ) ⊤ Π + ( D + ¯ D ) ⊤ P ( C + ¯ C ) + ( S + ¯ S ) (cid:3) = 0 , a.e. s ∈ [ t, T ] , Π( T ) = G + ¯ G. We have the following result. 15 heorem 4.4.
Let (H1)–(H2) and (H4) hold. Then the map v ( · ) ¯ J ( t, v ( · )) is uniformly convex,i.e., there exists a λ > such that (4.24) ¯ J ( t, v ( · )) > λ Z Tt | v ( s ) | ds, ∀ v ( · ) ∈ L ( t, T ; R m ) . Consequently, the strongly regular solution P ( · ) of the Riccati equation (4.9) satisfies (4.25) Σ = R + ¯ R + ( D + ¯ D ) ⊤ P ( D + ¯ D ) ≫ , and the Riccati equation (4.23) admits a unique solution Π( · ) ∈ C ([ t, T ]; S n ) . Proof.
Let P ( · ) be the strongly regular solution of the Riccati equation (4.9) and setΘ = − ( R + D ⊤ P D ) − ( B ⊤ P + D ⊤ P C + S ) ∈ L ( t, T ; R m × n ) . We claim that(4.26) J ( t,
0; Θ( · ) X ( · ) + v ( · )) = ¯ J ( t,
0; Θ( · ) y ( · ) + v ( · )) , ∀ v ( · ) ∈ L ( t, T ; R m ) . To prove (4.26), take any v ( · ) ∈ L ( t, T ; R m ), let y ( · ) be the solution of(4.27) ( ˙ y ( s ) = (cid:2) A ( s ) + ¯ A ( s ) (cid:3) y ( s ) + (cid:2) B ( s ) + ¯ B ( s ) (cid:3)(cid:2) Θ( s ) y ( s ) + v ( s ) (cid:3) , s ∈ [ t, T ] ,y ( t ) = 0 , and X ( · ) be the solution of(4.28) dX ( s ) = n AX + ¯ A E [ X ] + B (Θ X + v ) + ¯ B E [Θ X + v ] o ds + n CX + ¯ C E [ X ] + D (Θ X + v ) + ¯ D E [Θ X + v ] o dW ( s ) , s ∈ [ t, T ] ,X ( t ) = 0 . Note that v ( · ) is deterministic. Then d E [ X ( s )] = n(cid:0) A + ¯ A (cid:1) E [ X ] + (cid:0) B + ¯ B (cid:1)(cid:0) Θ E [ X ] + v (cid:1)o ds, s ∈ [ t, T ] , E [ X ( t )] = 0 . By the uniqueness of solutions, we see that(4.29) E [ X ( s )] = y ( s ) , s ∈ [ t, T ] . Now let z ( · ) = X ( · ) − E [ X ( · )]. Then(4.30) ( dz ( s ) = ( A + B Θ) zds + (cid:8) ( C + D Θ) z + ( C + ¯ C ) y + ( D + ¯ D )(Θ y + v ) (cid:9) dW ( s ) , s ∈ [ t, T ] ,z ( t ) = 0 . Keep in mind that v ( · ) is deterministic and note that(4.31) 0 = ˙ P + P ( A + B Θ) + ( A + B Θ) ⊤ P + ( C + D Θ) ⊤ P ( C + D Θ)+ Θ ⊤ R Θ + S ⊤ Θ + Θ ⊤ S + Q.
16y applying Itˆo’s formula to s
7→ h P ( s ) z ( s ) , z ( s ) i , we have (also, noting E [ z ] = 0) J ( t,
0; Θ( · ) X ( · ) + v ( · ))= E (cid:26) h GX ( T ) , X ( T ) i + (cid:10) ¯ G E [ X ( T )] , E [ X ( T )] (cid:11) + Z Tt h h QX, X i + (cid:10) ¯ Q E [ X ] , E [ X ] (cid:11) + 2 h SX, Θ X + v i + 2 (cid:10) ¯ S E [ X ] , E [Θ X + v ] (cid:11) + h R (Θ X + v ) , (Θ X + v ) i + (cid:10) ¯ R E [Θ X + v ] , E [Θ X + v ] (cid:11) i ds (cid:27) = E (cid:26) h Gz ( T ) z ( T ) i + Z Tt h h Qz, z i + 2 h Sz, Θ z i + h R Θ z, Θ z i i ds (cid:27) + (cid:10) ( G + ¯ G ) y ( T ) , y ( T ) (cid:11) + Z Tt h (cid:10) ( Q + ¯ Q ) y, y (cid:11) + 2 (cid:10) ( S + ¯ S ) y, Θ y + v (cid:11) + (cid:10) ( R + ¯ R )(Θ y + v ) , Θ y + v (cid:11) i ds = E Z Tt n(cid:10) ˙ P z, z (cid:11) + (cid:10) P ( A + B Θ) z, z (cid:11) + (cid:10) P z, ( A + B Θ) z (cid:11) + (cid:10) P (cid:2) ( C + D Θ) z + ( C + ¯ C ) y + ( D + ¯ D )(Θ y + v ) (cid:3) , ( C + D Θ) z + ( C + ¯ C ) y + ( D + ¯ D )(Θ y + v ) (cid:11) + (cid:10)(cid:0) Q + S ⊤ Θ + Θ ⊤ S + Θ ⊤ R Θ (cid:1) z, z (cid:11) o ds + (cid:10) ( G + ¯ G ) y ( T ) , y ( T ) (cid:11) + Z Tt h (cid:10) ( Q + ¯ Q ) y, y (cid:11) + 2 (cid:10) ( S + ¯ S ) y, Θ y + v (cid:11) + (cid:10) ( R + ¯ R )(Θ y + v ) , Θ y + v (cid:11) i ds = Z Tt (cid:10) P (cid:2) ( C + ¯ C ) y + ( D + ¯ D )(Θ y + v ) (cid:3) , ( C + ¯ C ) y + ( D + ¯ D )(Θ y + v ) (cid:11) ds + (cid:10) ( G + ¯ G ) y ( T ) , y ( T ) (cid:11) + Z Tt h (cid:10) ( Q + ¯ Q ) y, y (cid:11) + 2 (cid:10) ( S + ¯ S ) y, Θ y + v (cid:11) + (cid:10) ( R + ¯ R )(Θ y + v ) , Θ y + v (cid:11) i ds = (cid:10) ( G + ¯ G ) y ( T ) , y ( T ) (cid:11) + Z Tt n (cid:10)(cid:2) Q + ¯ Q + ( C + ¯ C ) ⊤ P ( C + ¯ C ) (cid:3) y, y (cid:11) + 2 (cid:10)(cid:2) ( D + ¯ D ) ⊤ P ( C + ¯ C ) + S + ¯ S (cid:3) y, Θ y + v (cid:11) + (cid:10)(cid:2) R + ¯ R + ( D + ¯ D ) ⊤ P ( D + ¯ D ) (cid:3) (Θ y + v ) , Θ y + v (cid:11) o ds = ¯ J ( t,
0; Θ( · ) y ( · ) + v ( · )) . Thus, (4.26) holds. Consequently, by (H4), we have¯ J ( t,
0; Θ( · ) y ( · ) + v ( · )) = J ( t,
0; Θ( · ) X ( · ) + v ( · )) > δ E Z Tt | Θ( s ) X ( s ) + v ( s ) | ds > δ Z Tt (cid:12)(cid:12) E [Θ( s ) X ( s ) + v ( s )] (cid:12)(cid:12) ds = δ Z Tt | Θ( s ) y ( s ) + v ( s ) | ds, ∀ v ( · ) ∈ L ( t, T ; R m ) , which implies the uniform convexity of v ( · ) ¯ J ( t, v ( · )). The rest of the theorem follows now immediatelyfrom [25, Theorem 4.6]. 17 Sufficiency of the Riccati equations
In the previous section, we proved that the solvability of the Riccati equations (4.9) and (4.23) is necessaryfor the uniform convexity of the cost functional. In this section, we shall show that it is also sufficient.Moreover, under the uniform convexity condition, the optimal control can be represented explicitly as astate feedback form via the solutions of the Riccati equations.First we need the following lemma.
Lemma 5.1.
Let (H1)–(H2) hold. For any u ( · ) ∈ U [ t, T ] , let X u ( · ) be the solution of (5.1) dX u ( s ) = n A ( s ) X u ( s ) + ¯ A ( s ) E [ X u ( s )] + B ( s ) u ( s ) + ¯ B ( s ) E [ u ( s )] o ds + n C ( s ) X u ( s ) + ¯ C ( s ) E [ X u ( s )] + D ( s ) u ( s ) + ¯ D ( s ) E [ u ( s )] o dW ( s ) , s ∈ [ t, T ] ,X u ( t ) = 0 . Then for any Θ( · ) , ¯Θ( · ) ∈ L ( t, T ; R m × n ) , there exists a constant γ > such that (5.2) E Z Tt (cid:12)(cid:12) u ( s ) − Θ( s ) (cid:0) X u ( s ) − E [ X u ( s )] (cid:1)(cid:12)(cid:12) ds > γ E Z Tt | u ( s ) | ds, ∀ u ( · ) ∈ U [ t, T ] , Z Tt (cid:12)(cid:12) E [ u ( s )] − ¯Θ( s ) E [ X u ( s )] (cid:12)(cid:12) ds > γ Z Tt | E [ u ( s )] | ds, ∀ u ( · ) ∈ U [ t, T ] . Proof.
Let Θ( · ) ∈ L ( t, T ; R m × n ). Define a bounded linear operator A : U [ t, T ] → U [ t, T ] by A u = u − Θ( X u − E [ X u ]) . Then A is bijective and its inverse A − is given by A − u = u + Θ (cid:16) e X u − E (cid:2) e X u (cid:3)(cid:17) , where e X u ( · ) is the solution of d e X u ( s ) = n ( A + B Θ) e X u + ( ¯ A − B Θ) E (cid:2) e X u (cid:3) + Bu + ¯ B E [ u ] o ds + n ( C + D Θ) e X u + ( ¯ C − D Θ) E (cid:2) e X u (cid:3) + Du + D E [ u ] o dW ( s ) , s ∈ [ t, T ] , e X u ( t ) = 0 . By the bounded inverse theorem, A − is bounded with norm kA − k >
0. Thus, E Z Tt | u ( s ) | ds = E Z Tt | ( A − A u )( s ) | ds kA − k E Z Tt | ( A u )( s ) | ds = kA − k E Z Tt (cid:12)(cid:12) u ( s ) − Θ( s ) (cid:0) X u ( s ) − E [ X u ( s )] (cid:1)(cid:12)(cid:12) ds, ∀ u ( · ) ∈ U [ t, T ] , which implies the first inequality in (5.2) with γ = kA − k − .To prove the second, for any v ( · ) ∈ L ( t, T ; R m × n ), let y v ( · ) be the solution to the following ODE:(5.3) ( ˙ y v ( s ) = (cid:2) A ( s ) + ¯ A ( s ) (cid:3) y v ( s ) + (cid:2) B ( s ) + ¯ B ( s ) (cid:3) v ( s ) , s ∈ [ t, T ] ,y v ( t ) = 0 . For ¯Θ( · ) ∈ L ( t, T ; R m × n ), we define a bounded linear operator B : L ( t, T ; R m ) → L ( t, T ; R m ) by B v = v − ¯Θ y v . B is invertible and Z Tt (cid:12)(cid:12) v ( s ) − ¯Θ( s ) y v ( s ) (cid:12)(cid:12) ds > kB − k Z Tt | v ( s ) | ds, ∀ v ( · ) ∈ L ( t, T ; R m × n ) . Observe that E [ X u ( · )] satisfies (5.3) with v ( · ) = E [ u ( · )]. The result therefore follows.Now we present the main result of this section, which gives a characterization for the uniform convexityof the cost functional as well as a feedback representation of the optimal control. Theorem 5.2.
Let (H1)–(H2) hold. Then the map u ( · ) J ( t, u ( · )) is uniformly convex if and onlyif the Riccati equation (4.9) admits a strongly regular solution P ( · ) such that (5.4) Σ ≡ R + ¯ R + ( D + ¯ D ) ⊤ P ( D + ¯ D ) ≫ , and the corresponding Riccati equation (4.23) admits a solution Π( · ) . In this case, the unique optimal u ∗ ( · ) of Problem (MF-LQ) at ( t, ξ ) is given by (5.5) u ∗ = Θ (cid:0) X ∗ − E [ X ∗ ] (cid:1) + ¯Θ E [ X ∗ ] + ϕ − E [ ϕ ] + ¯ ϕ, where (5.6) Θ = − ( R + D ⊤ P D ) − ( B ⊤ P + D ⊤ P C + S ) , ¯Θ = − Σ − (cid:2) ( B + ¯ B ) ⊤ Π + ( D + ¯ D ) ⊤ P ( C + ¯ C ) + ( S + ¯ S ) (cid:3) ,ϕ = − ( R + D ⊤ P D ) − (cid:2) B ⊤ η + D ⊤ ( ζ + P σ ) + ρ (cid:3) , ¯ ϕ = − Σ − (cid:8) ( B + ¯ B ) ⊤ ¯ η + ( D + ¯ D ) ⊤ (cid:0) E [ ζ ] + P E [ σ ] (cid:1) + E [ ρ ] + ¯ ρ (cid:9) , with ( η ( · ) , ζ ( · )) and ¯ η ( · ) being the ( adapted ) solutions to the following BSDE(5.7) dη ( s ) = − (cid:2) ( A + B Θ) ⊤ η + ( C + D Θ) ⊤ ζ + ( C + D Θ) ⊤ P σ + Θ ⊤ ρ + P b + q (cid:3) ds + ζdW ( s ) , s ∈ [ t, T ] ,η ( T ) = g, and ordinary differential equation (5.8) ˙¯ η + (cid:2) ( A + ¯ A ) + ( B + ¯ B ) ¯Θ (cid:3) ⊤ ¯ η + ¯Θ ⊤ n ( D + ¯ D ) ⊤ (cid:0) P E [ σ ] + E [ ζ ] (cid:1) + E [ ρ ] + ¯ ρ o + ( C + ¯ C ) ⊤ (cid:0) P E [ σ ] + E [ ζ ] (cid:1) + E [ q ] + ¯ q + Π E [ b ] = 0 , a.e. s ∈ [ t, T ] , ¯ η ( T ) = E [ g ] + ¯ g, respectively, and X ∗ ( · ) is the solution of the closed-loop system (5.9) dX ∗ ( s ) = n ( A + B Θ) (cid:0) X ∗ − E [ X ∗ ] (cid:1) + (cid:2) ( A + ¯ A ) + ( B + ¯ B ) ¯Θ (cid:3) E [ X ∗ ] + b o ds + n ( C + D Θ) (cid:0) X ∗ − E [ X ∗ ] (cid:1) + (cid:2) ( C + ¯ C ) + ( D + ¯ D ) ¯Θ (cid:3) E [ X ∗ ] + σ o dW ( s ) , s ∈ [ t, T ] ,X ∗ ( t ) = ξ. Moreover, the value V ( t, ξ ) is given by (5.10) V ( t, ξ ) = E h P ( t )( ξ − E [ ξ ]) + 2 η ( t ) , ξ − E [ ξ ] i + h Π( t ) E [ ξ ] + 2¯ η ( t ) , E [ ξ ] i + E Z Tt n h P σ, σ i + 2 h η, b − E [ b ] i + 2 h ζ, σ i + 2 h ¯ η, E [ b ] i− h Σ ( ϕ − E [ ϕ ]) , ϕ − E [ ϕ ] i − h Σ ¯ ϕ, ¯ ϕ i o ds, where Σ = R + D ⊤ P D . roof. The “only if ” part has been proved in Section 4. Let us now show the “if ” part. For any ξ ∈ L F t (Ω; R n ) and u ( · ) ∈ U [ t, T ], let X ( · ) ≡ X ( · ; t, ξ, u ( · )) be the corresponding solution of (1.1). Set z ( · ) = X ( · ) − E [ X ( · )] , v ( · ) = u ( · ) − E [ u ( · )] , y ( · ) = E [ X ( · )] . Then(5.11) dz ( s ) = n Az + Bv + b − E [ b ] o ds + n Cz + Dv + σ + ( C + ¯ C ) y + ( D + ¯ D ) E [ u ] o dW ( s ) , s ∈ [ t, T ] ,z ( t ) = ξ − E [ ξ ] , and(5.12) ( ˙ y = ( A + ¯ A ) y + ( B + ¯ B ) E [ u ] + E [ b ] , s ∈ [ t, T ] ,y ( t ) = E [ ξ ] . Now we rewrite the cost functional as follows:(5.13) J ( t, ξ ; u ( · )) = E ( h Gz ( T ) + 2 g, z ( T ) i + Z Tt "* Q S ⊤ S R ! zv ! , zv !+ + 2 * qρ ! , zv !+ ds ) + (cid:10) ( G + ¯ G ) y ( T ) + 2 ( E [ g ] + ¯ g ) , y ( T ) (cid:11) + Z Tt "* Q + ¯ Q ( S + ¯ S ) ⊤ S + ¯ S R + ¯ R ! y E [ u ] ! , y E [ u ] !+ + 2 * E [ q ]+ ¯ q E [ ρ ]+ ¯ ρ ! , y E [ u ] !+ ds. Applying Itˆo’s formula to s
7→ h P ( s ) z ( s ) + 2 η ( s ) , z ( s ) i , we have (noting E [ z ] ≡ , E [ v ] ≡ E h Gz ( T ) + 2 g, z ( T ) i − E h P ( t )( ξ − E [ ξ ]) + 2 η ( t ) , ξ − E [ ξ ] i + E Z Tt h h Qz, z i + 2 h Sz, v i + h Rv, v i + 2 h q, z i + 2 h ρ, v i i ds = E Z Tt h(cid:10) ˙ P z, z (cid:11) + (cid:10) P ( Az + Bv + b − E [ b ]) , z (cid:11) + (cid:10) P z, ( Az + Bv + b − E [ b ]) (cid:11) + (cid:10) P (cid:8) Cz + Dv + σ + ( C + ¯ C ) y + ( D + ¯ D ) E [ u ] (cid:9) ,Cz + Dv + σ + ( C + ¯ C ) y + ( D + ¯ D ) E [ u ] (cid:11) − (cid:10) ( A + B Θ) ⊤ η + ( C + D Θ) ⊤ ζ + ( C + D Θ) ⊤ P σ + Θ ⊤ ρ + P b + q, z (cid:11) + 2 (cid:10) η, Az + Bv + b − E [ b ] (cid:11) + 2 (cid:10) ζ, Cz + Dv + σ + ( C + ¯ C ) y + ( D + ¯ D ) E [ u ] (cid:11)i ds + E Z Tt h h Qz, z i + 2 h Sz, v i + h Rv, v i + 2 h q, z i + 2 h ρ, v i i ds = E Z Tt h(cid:10)(cid:0) ˙ P + P A + A ⊤ P + C ⊤ P C + Q (cid:1) z, z (cid:11) + 2 (cid:10)(cid:0) P B + C ⊤ P D + S ⊤ (cid:1) v, z (cid:11) + (cid:10) ( R + D ⊤ P D ) v, v (cid:11) + 2 (cid:10) B ⊤ η + D ⊤ ζ + D ⊤ P σ + ρ, v − Θ z (cid:11) + 2 (cid:10) P E [ σ ] + E [ ζ ] , ( C + ¯ C ) y + ( D + ¯ D ) E [ u ] (cid:11) + (cid:10) P (cid:8) ( C + ¯ C ) y + ( D + ¯ D ) E [ u ] (cid:9) , ( C + ¯ C ) y + ( D + ¯ D ) E [ u ] (cid:11) + h P σ, σ i + 2 h η, b − E [ b ] i + 2 h ζ, σ i i ds = E Z Tt h(cid:10) Θ ⊤ Σ Θ z, z (cid:11) − (cid:10) Θ ⊤ Σ v, z (cid:11) + h Σ v, v i − h Σ ϕ, v − Θ z i + (cid:10) ( C + ¯ C ) ⊤ P ( C + ¯ C ) y, y (cid:11) + 2 (cid:10) ( C + ¯ C ) ⊤ P ( D + ¯ D ) E [ u ] , y (cid:11) (cid:10) ( D + ¯ D ) ⊤ P ( D + ¯ D ) E [ u ] , E [ u ] (cid:11) + 2 (cid:10) ( C + ¯ C ) ⊤ (cid:0) P E [ σ ] + E [ ζ ] (cid:1) , y (cid:11) + 2 (cid:10) ( D + ¯ D ) ⊤ (cid:0) P E [ σ ] + E [ ζ ] (cid:1) , E [ u ] (cid:11) + h P σ, σ i + 2 h η, b − E [ b ] i + 2 h ζ, σ i i ds = E Z Tt h h Σ ( v − Θ z − ϕ ) , v − Θ z − ϕ i − h Σ ϕ, ϕ i + (cid:10) ( C + ¯ C ) ⊤ P ( C + ¯ C ) y, y (cid:11) + 2 (cid:10) ( C + ¯ C ) ⊤ P ( D + ¯ D ) E [ u ] , y (cid:11) + (cid:10) ( D + ¯ D ) ⊤ P ( D + ¯ D ) E [ u ] , E [ u ] (cid:11) + 2 (cid:10) ( C + ¯ C ) ⊤ (cid:0) P E [ σ ] + E [ ζ ] (cid:1) , y (cid:11) + 2 (cid:10) ( D + ¯ D ) ⊤ (cid:0) P E [ σ ] + E [ ζ ] (cid:1) , E [ u ] (cid:11) + h P σ, σ i + 2 h η, b − E [ b ] i + 2 h ζ, σ i i ds. Applying the integration by parts formula to s
7→ h Π( s ) y ( s ) + 2¯ η ( s ) , y ( s ) i , we have(5.15) (cid:10) ( G + ¯ G ) y ( T ) + 2( E [ g ] + ¯ g ) , y ( T ) (cid:11) − (cid:10) Π( t ) E [ ξ ] + 2¯ η ( t ) , E [ ξ ] (cid:11) + Z Tt "* Q + ¯ Q ( S + ¯ S ) ⊤ S + ¯ S R + ¯ R ! y E [ u ] ! , y E [ u ] !+ + 2 * E [ q ] + ¯ q E [ ρ ] + ¯ ρ ! , y E [ u ] !+ ds = Z Tt h(cid:10) ˙Π y, y (cid:11) + (cid:10) Π (cid:8) ( A + ¯ A ) y + ( B + ¯ B ) E [ u ] + E [ b ] (cid:9) , y (cid:11) + (cid:10) Π y, ( A + ¯ A ) y + ( B + ¯ B ) E [ u ] + E [ b ] (cid:11) + 2 (cid:10) ˙¯ η, y (cid:11) + 2 (cid:10) ¯ η, ( A + ¯ A ) y + ( B + ¯ B ) E [ u ] + E [ b ] (cid:11)i ds + Z Tt h h ( Q + ¯ Q ) y, y i +2 h ( S + ¯ S ) y, E [ u ] i + h ( R + ¯ R ) E [ u ] , E [ u ] i +2 h E [ q ]+ ¯ q, y i +2 h E [ ρ ]+ ¯ ρ, E [ u ] i i ds = Z Tt n(cid:10)(cid:2) ˙Π + Π( A + ¯ A ) + ( A + ¯ A ) ⊤ Π + Q + ¯ Q (cid:3) y, y (cid:11) + 2 (cid:10)(cid:2) Π( B + ¯ B ) + ( S + ¯ S ) ⊤ (cid:3) E [ u ] , y (cid:11) + 2 (cid:10) ˙¯ η + ( A + ¯ A ) ⊤ ¯ η + E [ q ] + ¯ q + Π E [ b ] , y (cid:11) + 2 (cid:10) ( B + ¯ B ) ⊤ ¯ η + E [ ρ ] + ¯ ρ, E [ u ] (cid:11) + h ( R + ¯ R ) E [ u ] , E [ u ] i + 2 h ¯ η, E [ b ] i o ds. Adding (5.14) and (5.15) together and noting (5.13), we obtain J ( t, ξ ; u ( · )) − E h P ( t )( ξ − E [ ξ ]) + 2 η ( t ) , ξ − E [ ξ ] i − h Π( t ) E [ ξ ] + 2¯ η ( t ) , E [ ξ ] i = E Z Tt n h Σ ( v − Θ z − ϕ ) , v − Θ z − ϕ i − h Σ ϕ, ϕ i + h P σ, σ i + 2 h η, b − E [ b ] i + 2 h ζ, σ i + 2 h ¯ η, E [ b ] i o ds + Z Tt n(cid:10)(cid:2) ˙Π + Π( A + ¯ A ) + ( A + ¯ A ) ⊤ Π + Q + ¯ Q + ( C + ¯ C ) ⊤ P ( C + ¯ C ) (cid:3) y, y (cid:11) + 2 (cid:10)(cid:2) Π( B + ¯ B ) + ( C + ¯ C ) ⊤ P ( D + ¯ D ) + ( S + ¯ S ) ⊤ (cid:3) E [ u ] , y (cid:11) + (cid:10)(cid:2) R + ¯ R + ( D + ¯ D ) ⊤ P ( D + ¯ D ) (cid:3) E [ u ] , E [ u ] (cid:11) + 2 (cid:10) ˙¯ η + ( A + ¯ A ) ⊤ ¯ η + ( C + ¯ C ) ⊤ (cid:0) P E [ σ ] + E [ ζ ] (cid:1) + E [ q ] + ¯ q + Π E [ b ] , y (cid:11) + 2 (cid:10) ( B + ¯ B ) ⊤ ¯ η + ( D + ¯ D ) ⊤ (cid:0) P E [ σ ] + E [ ζ ] (cid:1) + E [ ρ ] + ¯ ρ, E [ u ] (cid:11)o ds = E Z Tt n h Σ ( v − Θ z − ϕ ) , v − Θ z − ϕ i − h Σ ϕ, ϕ i + h P σ, σ i + 2 h η, b − E [ b ] i + 2 h ζ, σ i + 2 h ¯ η, E [ b ] i o ds + Z Tt n(cid:10) ¯Θ ⊤ Σ ¯Θ y, y (cid:11) − (cid:10) ¯Θ ⊤ Σ E [ u ] , y (cid:11) + h Σ E [ u ] , E [ u ] i − h Σ ¯ ϕ, E [ u ] − ¯Θ y i o ds = E Z Tt n h Σ ( v − Θ z − ϕ ) , v − Θ z − ϕ i − h Σ ϕ, ϕ i (cid:10) Σ (cid:0) E [ u ] − ¯Θ y − ¯ ϕ (cid:1) , E [ u ] − ¯Θ y − ¯ ϕ (cid:11) − h Σ ¯ ϕ, ¯ ϕ i + h P σ, σ i + 2 h η, b − E [ b ] i + 2 h ζ, σ i + 2 h ¯ η, E [ b ] i o ds = E Z Tt n h P σ, σ i + 2 h η, b − E [ b ] i + 2 h ζ, σ i + 2 h ¯ η, E [ b ] i − h Σ ¯ ϕ, ¯ ϕ i− h Σ ( ϕ − E [ ϕ ]) , ϕ − E [ ϕ ] i + h Σ ( v − Θ z − ϕ + E [ ϕ ]) , v − Θ z − ϕ + E [ ϕ ] i + (cid:10) Σ (cid:0) E [ u ] − ¯Θ y − ¯ ϕ (cid:1) , E [ u ] − ¯Θ y − ¯ ϕ (cid:11)o ds. Since Σ , Σ ≫
0, (5.16) implies that(5.17) J ( t, ξ ; u ( · )) > E h P ( t )( ξ − E [ ξ ]) + 2 η ( t ) , ξ − E [ ξ ] i + h Π( t ) E [ ξ ] + 2¯ η ( t ) , E [ ξ ] i + E Z Tt n h P σ, σ i + 2 h η, b − E [ b ] i + 2 h ζ, σ i + 2 h ¯ η, E [ b ] i− h Σ ( ϕ − E [ ϕ ]) , ϕ − E [ ϕ ] i − h Σ ¯ ϕ, ¯ ϕ i o ds, with the equality holding if and only if ( u − E [ u ] = v = Θ z + ϕ − E [ ϕ ] = Θ (cid:0) X − E [ X ] (cid:1) + ϕ − E [ ϕ ] , E [ u ] = ¯Θ y + ¯ ϕ = ¯Θ E [ X ] + ¯ ϕ, which is also equivalent to(5.18) u = Θ (cid:0) X − E [ X ] (cid:1) + ¯Θ E [ X ] + ϕ − E [ ϕ ] + ¯ ϕ. In particular, when b ( · ) , σ ( · ) , g ( · ) , ¯ g ( · ) , q ( · ) , ¯ q ( · ) , ρ ( · ) , ¯ ρ ( · ) = 0, we have( η ( · ) , ζ ( · )) = (0 , , ¯ η ( · ) = 0 , ϕ ( · ) = ¯ ϕ ( · ) = 0 . Take ξ = 0. Then X ( · ) satisfies(5.19) dX ( s ) = n A ( s ) X ( s ) + ¯ A ( s ) E [ X ( s )] + B ( s ) u ( s ) + ¯ B ( s ) E [ u ( s )] o ds + n C ( s ) X ( s ) + ¯ C ( s ) E [ X ( s )] + D ( s ) u ( s ) + ¯ D ( s ) E [ u ( s )] o dW ( s ) , s ∈ [ t, T ] ,X ( t ) = 0 , and (5.16) becomes(5.20) J ( t, u ( · )) = E Z Tt n(cid:10) Σ (cid:2) u − E [ u ] − Θ( X − E [ X ]) (cid:3) , u − E [ u ] − Θ( X − E [ X ]) (cid:11) + (cid:10) Σ (cid:0) E [ u ] − ¯Θ E [ X ] (cid:1) , E [ u ] − ¯Θ E [ X ] (cid:11)o ds. Noting that Σ , Σ > δI for some δ > J ( t, u ( · )) > δ E Z Tt n | u − E [ u ] − Θ( X − E [ X ]) | + (cid:12)(cid:12) E [ u ] − ¯Θ E [ X ] (cid:12)(cid:12) o ds > δ E Z Tt | u − Θ( X − E [ X ]) | − h u − Θ( X − E [ X ]) , E [ u ] i + (1 + γ ) | E [ u ] | ds > δγ γ E Z Tt | u − Θ( X − E [ X ]) | ds > δγ γ E Z Tt | u ( s ) | ds, ∀ u ( · ) ∈ U [ t, T ] , for some γ >
0. The uniform convexity of u ( · ) J ( t, u ( · )) follows immediately.22ote that for Problem (MF-LQ) (where b ( · ) , σ ( · ) , g ( · ) , ¯ g ( · ) , q ( · ) , ¯ q ( · ) , ρ ( · ) , ¯ ρ ( · ) = 0), under the uniformconvexity condition (H4), the value at ( t, ξ ) is given by(5.22) V ( t, ξ ) = E h P ( t )( ξ − E [ ξ ]) , ξ − E [ ξ ] i + h Π( t ) E [ ξ ] , E [ ξ ] i , where P ( · ) and Π( · ) are the solutions to the Riccati equations (4.9) and (4.23), respectively. The uniqueoptimal u ∗ ( · ) is given by(5.23) u ∗ = Θ (cid:0) X ∗ − E [ X ∗ ] (cid:1) + ¯Θ E [ X ∗ ] , where Θ( · ) , ¯Θ( · ) are defined by (5.6) and X ∗ ( · ) is the solution of(5.24) dX ∗ ( s ) = n ( A + B Θ) (cid:0) X ∗ − E [ X ∗ ] (cid:1) + (cid:2) ( A + ¯ A ) + ( B + ¯ B ) ¯Θ (cid:3) E [ X ∗ ] o ds + n ( C + D Θ) (cid:0) X ∗ − E [ X ∗ ] (cid:1) + (cid:2) ( C + ¯ C ) + ( D + ¯ D ) ¯Θ (cid:3) E [ X ∗ ] o dW ( s ) , s ∈ [ t, T ] ,X ∗ ( t ) = ξ. To conclude this section, we present a sufficient condition for the uniform convexity of the cost functional.From the following result, we will see that (1.4) implies the uniform convexity condition (H4). However, theconverse fails. A counterexample will be present in the next section (see Example 6.1).
Proposition 5.3.
Let (H1)–(H2) hold and t ∈ [0 , T ) be given. If there exists a constant δ > such that (5.25) ( G, G + ¯ G > , R ( s ) , R ( s ) + ¯ R ( s ) > δI, Q ( s ) − S ( s ) ⊤ R ( s ) − S ( s ) > ,Q ( s ) + ¯ Q ( s ) − (cid:2) S ( s ) + ¯ S ( s ) (cid:3) ⊤ (cid:2) R ( s ) + ¯ R ( s ) (cid:3) − (cid:2) S ( s ) + ¯ S ( s ) (cid:3) > , a.e. s ∈ [ t, T ] , then the map u ( · ) J ( t, u ( · )) is uniformly convex. Proof.
For any u ( · ) ∈ U [ t, T ], let X u ( · ) be the solution of (5.1). Then J ( t, u ( · )) = E ( (cid:10) G (cid:0) X u ( T ) − E [ X u ( T )] (cid:1) , X u ( T ) − E [ X u ( T )] (cid:11) + Z Tt * Q S ⊤ S R ! X u − E [ X u ] u − E [ u ] ! , X u − E [ X u ] u − E [ u ] !+ ds ) + (cid:10) ( G + ¯ G ) E [ X u ( T )] , E [ X u ( T )] (cid:11) + Z Tt * Q + ¯ Q ( S + ¯ S ) ⊤ S + ¯ S R + ¯ R ! E [ X u ] E [ u ] ! , E [ X u ] E [ u ] !+ ds > E Z Tt n (cid:10) Q (cid:0) X u − E [ X u ] (cid:1) , X u − E [ X u ] (cid:11) + 2 (cid:10) S (cid:0) X u − E [ X u ] (cid:1) , u − E [ u ] (cid:11) + (cid:10) R (cid:0) u − E [ u ] (cid:1) , u − E [ u ] (cid:11) o ds + Z Tt n (cid:10)(cid:0) Q + ¯ Q (cid:1) E [ X u ] , E [ X u ] (cid:11) + 2 (cid:10)(cid:0) S + ¯ S (cid:1) E [ X u ] , E [ u ] (cid:11) + (cid:10)(cid:0) R + ¯ R (cid:1) E [ u ] , E [ u ] (cid:11) o ds = E Z Tt n (cid:10)(cid:0) Q − S ⊤ R − S (cid:1)(cid:0) X u − E [ X u ] (cid:1) , X u − E [ X u ] (cid:11) + (cid:10) R (cid:2) u − E [ u ] + R − S (cid:0) X u − E [ X u ] (cid:1)(cid:3) , u − E [ u ] + R − S (cid:0) X u − E [ X u ] (cid:1)(cid:11) o ds + Z Tt n (cid:10)(cid:2) Q + ¯ Q − ( S + ¯ S ) ⊤ ( R + ¯ R ) − ( S + ¯ S ) (cid:3) E [ X u ] , E [ X u ] (cid:11) + D(cid:0) R + ¯ R (cid:1)(cid:16) E [ u ] + ( R + ¯ R ) − ( S + ¯ S ) E [ X u ] (cid:17) , E [ u ] + ( R + ¯ R ) − ( S + ¯ S ) E [ X u ] E o ds > δ E Z Tt (cid:12)(cid:12) u − E [ u ] + R − S (cid:0) X u − E [ X u ] (cid:1)(cid:12)(cid:12) ds + δ Z Tt (cid:12)(cid:12) E [ u ] + ( R + ¯ R ) − ( S + ¯ S ) E [ X u ] (cid:12)(cid:12) ds. − R − S and ¯Θ = − ( R + ¯ R ) − ( S + ¯ S )), we have(5.26) J ( t, u ( · )) > δ E Z Tt n (cid:12)(cid:12) u − E [ u ] + R − S (cid:0) X u − E [ X u ] (cid:1)(cid:12)(cid:12) + γ | E [ u ] | o ds > δγ γ E Z Tt (cid:12)(cid:12) u + R − S (cid:0) X u − E [ X u ] (cid:1)(cid:12)(cid:12) ds > δγ γ E Z Tt | u ( s ) | ds, ∀ u ( · ) ∈ U [ t, T ] , for some γ >
0. This completes the proof.
In this section we present two illustrative examples. In the first example, the condition (1.4) does not hold,but the corresponding Riccati equations are still solvable. Thus, by Theorem 5.2, the cost functional isuniformly convex. This example shows that the uniform convexity condition (H4) is indeed weaker than(1.4).
Example 6.1.
Consider the following Problem (MF-LQ) with one-dimensional state equation(6.1) ( dX ( s ) = (cid:8) E [ X ( s )] + u ( s ) + E [ u ( s )] (cid:9) ds + √ u ( s ) dW ( s ) , s ∈ [ t, ,X ( t ) = ξ, and cost functional(6.2) J ( t, ξ ; u ( · )) = E (cid:26) G | X (1) | + ¯ G | E [ X (1)] | + Z t (cid:16) R ( s ) | u ( s ) | + ¯ R ( s ) | E [ u ( s )] | (cid:17) ds (cid:27) , where ( G = 8 , ¯ G = − α − < α < e − ,R ( s ) = ( s + 1) − s + 1) , ¯ R ( s ) = 1 − ( s + 1) , s ∈ [0 , . The Riccati equations for the above problem are ˙ P ( s ) − P ( s ) R ( s ) + 2 P ( s ) = 0 ,P (1) = 8 , and ˙Π( s ) + 2Π( s ) − s ) R ( s ) + ¯ R ( s ) + 2 P ( s ) = 0 , Π(1) = − α. Clearly, G + ¯ G = − α < , R ( s ) = ( s + 1) ( s − − , R ( s ) + ¯ R ( s ) = 1 − s + 1) − , s ∈ [0 , . Hence, the condition (1.4) does not hold. However, one can verify that the above Riccati equations aresolvable on the whole interval [0 ,
1] with solutions given by P ( s ) = 2( s + 1) , Π( s ) = αe − s ) α [ e − s ) − − < , s ∈ [0 , . ( R ( s ) + D ( s ) ⊤ P ( s ) D ( s ) = ( s + 1) > ,R ( s ) + ¯ R ( s ) + (cid:2) D ( s ) + ¯ D ( s ) (cid:3) ⊤ P ( s ) (cid:2) D ( s ) + ¯ D ( s ) (cid:3) = 1 , s ∈ [0 , . By Theorem 5.2, the cost functional J ( t, ξ ; u ( · )) is uniformly convex in u ( · ), and for any initial pair ( t, ξ ) ∈ [0 , × L F t (Ω; R ), the problem admits a unique optimal control u ∗ ( · ) given by u ∗ ( s ) = − s + 1 X ∗ ( s ) + (cid:20) s + 1 − s ) (cid:21) E [ X ∗ ( s )] , s ∈ [ t, X ∗ ( · ) being the solution to the following closed-loop system: dX ∗ ( s ) = (cid:26) − s + 1 X ∗ ( s ) + (cid:20) s + 3 s + 2 − s ) (cid:21) E [ X ∗ ( s )] (cid:27) ds + ( − √ s + 2 X ∗ ( s ) + 2 √ (cid:20) s + 1 − Π( s ) (cid:21) E [ X ∗ ( s )] ) dW ( s ) , s ∈ [ t, ,X ∗ ( t ) = ξ. Now we present an example in which the mean-field LQ problem is open-loop solvable, but the costfunctional is not uniformly convex. Hence, the optimal control cannot be constructed directly in terms ofthe Riccati equations. However, an optimal could still be found by making use of Theorem 3.2 and 5.2.
Example 6.2.
Consider the following Problem (MF-LQ) with one-dimensional state equation(6.3) ( dX ( s ) = (cid:8) X ( s ) − E [ X ( s )] + E [ u ( s )] (cid:9) ds + u ( s ) dW ( s ) , s ∈ [ t, T ] ,X ( t ) = ξ, and cost functional(6.4) J ( t, ξ ; u ( · )) = E ( | X ( T ) | + | E [ X ( T )] | + Z Tt (cid:16) − | X ( s ) | −| u ( s ) | +4 | E [ X ( s )] | −| E [ u ( s )] | (cid:17) ds ) . In this example, ( A = 1 , ¯ A = − , B = 0 , ¯ B = 1 , C = ¯ C = 0 , D = 1 , ¯ D = 0 ,G = 2 , ¯ G = 1 , Q = − , ¯ Q = 4 , S = ¯ S = 0 , R = ¯ R = − . Clearly, the condition (1.4) does not hold. The Riccati equations for the problem are(6.5) ( ˙ P ( s ) + 2 P ( s ) − , s ∈ [ t, T ] ,P ( T ) = 2 , and(6.6) ˙Π( s ) − Π( s ) P ( s ) − , s ∈ [ t, T ] , Π( T ) = 3 . It is easy to see that P ( · ) ≡ R ( s ) + ¯ R ( s ) + [ D ( s ) + ¯ D ( s )] ⊤ P ( s )[ D ( s ) + ¯ D ( s )] = P ( s ) − , s ∈ [ t, T ] ,
25e cannot use (6.6) to solve the problem directly. To investigate the open-loop solvability of the aboveproblem, let us now consider the following cost functionals for ε > J ε ( t, ξ ; u ( · )) = J ( t, ξ ; u ( · )) + ε E Z Tt | u ( s ) | ds = E ( | X ( T ) | + | E [ X ( T )] | + Z Tt (cid:16) − | X ( s ) | + ( ε − | u ( s ) | + 4 | E [ X ( s )] | − | E [ u ( s )] | (cid:17) ds ) . We denote the corresponding mean-field LQ problem and value function by Problem (MF-LQ) ε and V ε ( · , · ),respectively. The Riccati equations for Problem (MF-LQ) ε are ( ˙ P ε ( s ) + 2 P ε ( s ) − , s ∈ [ t, T ] ,P ε ( T ) = 2 , and ˙Π ε ( s ) − Π ε ( s ) ε − P ε ( s ) = 0 , s ∈ [ t, T ] , Π ε ( T ) = 3 . A straightforward calculation leads to P ε ( s ) = 2 , Π ε ( s ) = 3 εε + 3( T − s ) ; s ∈ [ t, T ] . Since R + ε + D ⊤ P ε D = 1 + ε, R + ε + ¯ R + ( D + ¯ D ) ⊤ P ε ( D + ¯ D ) = ε, by Theorem 5.2, the map u ( · ) J ε ( t, u ( · )) is uniformly convex for all ε > u ( · ) J ( t, u ( · ))is convex. Moreover,(6.8) V ε ( t, ξ ) = E h P ε ( t )( ξ − E [ ξ ]) , ξ − E [ ξ ] i + h Π ε ( t ) E [ ξ ] , E [ ξ ] i , and the unique optimal control of Problem (MF-LQ) ε at ( t, ξ ) is given by(6.9) u ∗ ε ( s ) = − Π ε ( s ) ε E [ X ∗ ε ( s )] , s ∈ [ t, T ] , with X ∗ ε ( · ) being the solution to the following closed-loop system: dX ∗ ε ( s ) = (cid:26) X ∗ ε ( s ) − (cid:18) ε ( s ) ε (cid:19) E [ X ∗ ε ( s )] (cid:27) ds − Π ε ( s ) ε E [ X ∗ ε ( s )] dW ( s ) , s ∈ [ t, T ] ,X ∗ ε ( t ) = ξ. Letting ε → V ( t, ξ ) = lim ε → V ε ( t, ξ ) = ( ξ ] , t < T, E [ ξ ] + ( E [ ξ ]) , t = T. Note that d E [ X ∗ ε ( s )] = − Π ε ( s ) ε E [ X ∗ ε ( s )] ds, s ∈ [ t, T ] , E [ X ∗ ε ( t )] = E [ ξ ] . Hence, E [ X ∗ ε ( s )] = E [ ξ ] exp (cid:26) − Z st Π ε ( r ) ε dr (cid:27) = ε + 3( T − s ) ε + 3( T − t ) E [ ξ ] , s ∈ [ t, T ] , u ∗ ε ( s ) = − Π ε ( s ) ε E [ X ∗ ε ( s )] = − E [ ξ ] ε + 3( T − t ) s ∈ [ t, T ] . It is clear that for t ∈ [0 , T ), u ∗ ε ( s ) converges uniformly to(6.11) u ∗ ( s ) ≡ − E [ ξ ] T − t s ∈ [ t, T ] , which, by Theorem 3.2, is an optimal control of the original problem at ( t, ξ ). Acknowledgements.
The author wishes to thank Prof. Jiongmin Yong for his valuable comments,which have helped to improve the quality of the manuscript. The author also would like to thank Dr. XunLi for his useful suggestions and financial support.
References [1] N. U. Ahmed,
Nonlinear diffusion governed by McKean-Vlasov equation on Hilbert space and optimalcontrol,
SIAM J. Control Optim.,
46 (2007), 356–378.[2] N. U. Ahmed and X. Ding,
A semilinear McKean-Vlasov stochastic evolution equation in Hilbert space,
Stoch. Proc. Appl.,
60 (1995), 65–85.[3] N. U. Ahmed and X. Ding,
Controlled McKean-Vlasov equations,
Comm. Appl. Anal.,
A maximum principle for SDEs of mean-field type,
Appl. Math. Optim.,
63 (2011), 341–356.[5] V. S. Borkar and K. S. Kumar,
McKean-Vlasov limit in portfolio optimization,
Stoch. Anal. Appl.,
A general maximum principle for SDEs of mean-field type,
AppliedMathematics & Optimization,
64 (2011), 197–216.[7] R. Buckdahn, B. Djehiche, J. Li and S. Peng,
Mean-field backward stochastic differential equations: Alimit approach,
Ann. Probab.,
37 (2009), 1524–1565.[8] R. Buckdahn, J. Li and S. Peng,
Mean-field backward stochastic differential equations and related partialdifferential equations,
Stoch. Process. Appl.,
119 (2009), 3133–3154.[9] T. Chan,
Dynamics of the McKean-Vlasov equation,
Ann. Probab.,
22 (1994), 431–441.[10] T. Chiang,
McKean-Vlasov equations with discontinuous coefficients,
Soochow J. Math.,
20 (1994),507–526.[11] D. Crisan and J. Xiong,
Approximate McKean-Vlasov representations for a class of SPDEs,
Stochastics,
82 (2010), 53–68.[12] D. A. Dawson,
Critical dynamics and fluctuations for a mean-field model of cooperative behavior,
J.Statist. Phys.,
31 (1983), 29–85.[13] D. A. Dawson and J. G¨artner,
Large deviations from the McKean-Vlasov limit for weakly interactingdiffusions,
Stochastics,
20 (1987), 247–308.[14] J. G¨artner,
On the Mckean-Vlasov limit for interacting diffusions,
Math. Nachr.,
137 (1988), 197–248.[15] C. Graham,
McKean-Vlasov Ito-Skorohod equations, and nonlinear diffusions with discrete jump sets,
Stoch. Proc. Appl.,
40 (1992), 69–82. 2716] J. Huang, X. Li, and J. Yong,
A linear-quadratic optimal control problem for mean-field stochasticdifferential equations in infinite horizon,
Appl. Math. Optim.,
70 (2014), 29–59.[17] M. Huang, R. P. Malham´e, and P. E. Caines,
Large population stochastic dynamic games: Closed-loopMcKean-Vlasov systems and the Nash certainty equivalence principle,
Comm. Inform. Systems,
Foundations of kinetic theory,
Proc. 3rd Berkeley Sympos. Math. Statist. Prob.,
Macroscopic limit for stochastic partial differential equations ofMcKean-Vlasov type,
Prob. Theory Rel. Fields,
146 (2010), 189–222.[20] N. I. Mahmudov and M. A. McKibben,
On a class of backward McKean-Vlasov stochastic equations inHilbert space: Existence and convergence properties,
Dynam. Systems Appl.,
16 (2007), 643–664.[21] H. P. McKean,
A class of Markov processes associated with nonlinear parabolic equations,
Proc. Natl.Acad. Sci. USA,
56 (1966), 1907–1911.[22] T. Meyer-Brandis, B. Øksendal and X. Y. Zhou,
A mean-field stochastic maximum principle via Malli-avin calculus,
Stochastics,
84 (2012), 643–666.[23] J. Y. Park, P. Balasubramaniam and Y. H. Kang,
Controllability of McKean-Vlasov stochastic integrod-ifferential evolution equation in Hilbert spaces,
Numer. Funct. Anal. Optim.,
29 (2008), 1328–1346.[24] M. Scheutzow,
Uniqueness and non-uniqueness of solutions of Vlasov-McKean equations,
J. Austral.Math. Soc., Ser. A,
43 (1987), 246–256.[25] J. Sun, X. Li, and J. Yong,
Open-Loop and Closed-Loop Solvabilities for Stochastic Linear QuadraticOptimal Control Problems, arXiv: 1508.02163. [26] J. Sun and J. Yong,
Linear Quadratic Stochastic Differential Games: Open-Loop and Closed-Loop SaddlePoints,
SIAM J. Control Optim.,
52 (2014), 4082–4121.[27] A. Yu. Veretennikov,
On ergodic measures for McKean-Vlasov stochastic equations, in Monte Carlo andquasi-Monte Carlo methods 2004 ,
Springer, Berlin, 2006, 471–486.[28] J. Yong,
Linear-Quadratic Optimal Control Problems for Mean-Field Stochastic Differential Equations,
SIAM J. Control Optim.,
51 (2013), 2809–2838.[29] J. Yong and X. Y. Zhou,