Necessary conditions for optimality for stochastic evolution equations
aa r X i v : . [ m a t h . O C ] A ug Necessary conditions for optimality for stochasticevolution equations ∗ AbdulRahman Al-Hussein
Department of Mathematics, College of Science, Qassim University,P.O.Box 6644, Buraydah 51452, Saudi Arabia E -mail: [email protected] Abstract
This paper is concerned with providing the maximum principle for acontrol problem governed by a stochastic evolution system on a separableHilbert space. In particular, necessary conditions for optimality for thisstochastic optimal control problem are derived by using the adjoint back-ward stochastic evolution equation. Moreover, all coefficients appearing inthis system are allowed to depend on the control variable. We achieve ourresults through the semigroup approach.
MSC 2010:
Keywords:
Stochastic evolution equation, optimal control, maximum princi-ple, necessary conditions for optimality, backward stochastic evolution equation.
Consider a stochastic controlled problem governed by the following stochasticevolution equation (SEE): (cid:26) dX ( t ) = ( AX ( t ) + b ( X ( t ) , ν ( t ))) dt + σ ( X ( t ) , ν ( t )) dW ( t ) , t ∈ (0 , T ] ,X (0) = x . (1.1)We shall be interested in trying to minimize the cost functional, which is givenby equation (2.2) below, over a set of admissible controls.This system is driven mainly by a possibly unbounded linear operator A on aseparable Hilbert space H and a cylindrical Wiener process W on H. Here ν ( · )denotes a control process.We shall derive the maximum principle for this control problem. More pre-cisely, we shall concentrate on providing necessary conditions for optimality for ∗ This work is supported by the Science College Research Center at Qassim University,project no. SR-D-012-1610. AbdulRahman Al-Hussein this optimal control problem, which gives this minimization. For this purpose weshall apply the theory of backward stochastic evolution equations (BSEEs shortly)as in equation (3.2) in Section 3. These equations together with backward stochas-tic differential equations (BSDEs) have become of great importance in a numberof fields. For example in [4], [6], [13], [15], [16], [17] and [19] one can find applica-tions of BSDEs to stochastic optimal control problems. Some of these referenceshave also studied the maximum principle to find either necessary or sufficientconditions for optimality for stochastic differential equations (SDEs) or stochas-tic partial differential equations (SPDEs). Necessary conditions for optimality ofthe control process ν ( · ) and its corresponding solution X ν ( · ) but for the case whenthe noise term σ does not depend on ν ( t ) can be found in [13].In our work here we allow σ to depend on the control variable and study astochastic control problem associated with the former SEE. This control problemis explained in details in Section 2, and the main theorem is stated in Section 3 andis proved together with all necessary estimates in Section 4. Sufficient conditionsfor optimality for this optimal control problem can be found in [6]. We refer thereader also to [4].On the other hand, we recall that control problems governed by SPDEs thatare driven by martingales are studied in [5]. In fact in [5] we derived the maximumprinciple (necessary conditions) for optimality of stochastic systems governed bySPDEs. The technique used there relies heavily on the variational approach. Thereason beyond that is that the only known way until now to find solutions to theresulting adjoint BSPDEs is achieved through the same variational approach, andis established in details in [3]. Thus the semigroup approach to get mild solutions(as done here in Theorem 3.1 below and in Section 3) cannot be used to studysuch adjoint BSPDEs considered in [5]. Moreover, it is not obvious how one canallow the control variable ν ( t ) to enter in the noise term and in particular in themapping G in equation (1.1) of [5] and obtain a result like Theorem 3.2 below.This problem is still open and is also pointed out in [5, Remark 6.4].In the present work, we shall show how to handle this open problem in greatsuccess, and as we stated earlier, we can and will allow all coefficients in (1.1)and especially in the diffusion term to depend on the control variable ν ( t ) . Weemphasize that our work here does not need go through the technique of Hamilton-Jacobi-Bellman equations nor the technique of viscosity solutions. We refer thereader to [10] for this business and to [8] and some of the related references thereinfor the semi-group technique. Thus our results here are new. In this respect wethank the anonymous referee for pointing out the recent and relevant work ofFuhrman et al. in [11]. ecessary conditions for optimality for SEEs Let (Ω , F , P ) be a complete probability space and denote by N the collection of P -null sets of F . Let { W ( t ) , ≤ t ≤ T } be a cylindrical Wiener process on H withits completed natural filtration F t = σ { ℓ ◦ W ( s ) , ≤ s ≤ t , ℓ ∈ H ∗ } ∨ N , t ≥ E denote by L F (0 , T ; E ) to the space of all {F t , ≤ t ≤ T } - progressively measurable processes f with values in E such that E [ Z T | f ( t ) | E dt ] < ∞ . This space is Hilbert with respect to the norm || f || = (cid:16) E [ Z T | f ( t ) | E dt ] (cid:17) / . Moreover, if f ∈ L F (0 , T ; L ( H )) , where L ( H ) is the space of all Hilbert-Schmidtoperators on H, the stochastic integral R f ( t ) dW ( t ) can be defined and is a con-tinuous stochastic martingale in H. The norm and inner product on L ( H ) willbe denoted respectively by || · || and (cid:10) · , · (cid:11) . Let us assume that O is a separable Hilbert space equipped with an innerproduct (cid:10) · , · (cid:11) O , and U is a convex subset of O . We say that ν ( · ) : [0 , T ] × Ω → O is admissible if ν ( · ) ∈ L F (0 , T ; O ) and ν ( t ) ∈ U a.e., a.s.
The set of admissiblecontrols will be denoted by U ad . Suppose that b : H × O → H and σ : H × O → L ( H ) are two continuousmappings, and consider the following controlled SEE: (cid:26) dX ( t ) = ( AX ( t ) + b ( X ( t ) , ν ( t ))) dt + σ ( X ( t ) , ν ( t )) dW ( t ) ,X (0) = x , (2.1)where ν ( · ) ∈ U ad . A solution (in the sense of the following theorem) of (2.1) willbe denoted by X ν ( · ) to indicate the presence of the control process ν ( · ) . Let ℓ : H × O → R and φ : H → R be two measurable mappings such thatthe following cost functional is defined: J ( ν ( · )) := E [ Z T ℓ ( X ν ( · ) ( t ) , ν ( t )) dt + φ ( X ν ( · ) ( T )) ] , ν ( · ) ∈ U ad . (2.2)For example one can take ℓ and φ to satisfy the assumptions of Theorem 3.2 inSection 3. AbdulRahman Al-Hussein
The optimal control problem of the system (2.1) is to find the value function J ∗ := inf { J ( ν ( · )) : ν ( · ) ∈ U ad } and an optimal control ν ∗ ( · ) ∈ U ad such that J ∗ = J ( ν ∗ ( · )) . (2.3)If this happens, the corresponding solution X ν ∗ ( · ) is called an optimal solution ofthe stochastic control problem (2.1)–(2.3) and ( X ν ∗ ( · ) , ν ∗ ( · )) is called an optimalpair. We close this section by the following theorem.
Theorem 2.1
Assume that A is an unbounded linear operator on H that gener-ates a C -semigroup { S ( t ) , t ≥ } on H , and b, σ are continuously Fr´echet dif-ferentiable with respect to x and their derivatives b x , σ x are uniformly bounded.Then for every ν ( · ) ∈ U ad there exists a unique mild solution X ν ( · ) on [0 , T ] to(2.1). That is X ν ( · ) is a progressively measurable stochastic process such that X (0) = x and for all t ∈ [0 , T ] ,X ν ( · ) ( t ) = S ( t ) x + Z t S ( t − s ) b ( X ν ( · ) ( s ) , ν ( s )) ds + Z t S ( t − s ) σ ( X ν ( · ) ( s ) , ν ( s )) dW ( s ) . (2.4)The proof of this theorem can be derived in a similar way to those in [9, Chapter7] or [14].From here on we shall assume that A is the infinitesimal generator of a C -semigroup { S ( t ) , t ≥ } on H. Its adjoint operator A ∗ : D ( A ∗ ) ⊂ H → H is then the infinitesimal generator of the adjoint semigroup { S ∗ ( t ) , t ≥ } of { S ( t ) , t ≥ } . It is known from the literature that BSDEs play a fundamental role in derivingthe maximum principle for SDEs. In this section we shall search for such a rolefor SEEs like (2.1). To prepare for this business let us first define the
Hamiltonian by the following formula: H : H × O × H × L ( H ) → R , ecessary conditions for optimality for SEEs H ( x, ν, y, z ) := ℓ ( x, ν ) + (cid:10) b ( x, ν ) , y (cid:11) H + (cid:10) σ ( x, ν ) , z (cid:11) . (3.1)Then we consider the following BSEE on H : − dY ν ( · ) ( t ) = (cid:0) A ∗ Y ν ( · ) ( t ) + ∇ x H ( X ν ( · ) ( t ) , ν ( t ) , Y ν ( · ) ( t ) , Z ν ( · ) ( t )) (cid:1) dt − Z ν ( · ) ( t ) dW ( t ) , ≤ t < T,Y ν ( · ) ( T ) = ∇ φ ( X ν ( · ) ( T )) , (3.2)where ∇ φ denotes the gradient of φ, which is defined, by using the directionalderivative Dφ ( x )( h ) of φ at a point x ∈ H in the direction of h ∈ H, as (cid:10) ∇ φ ( x ) , h (cid:11) H = Dφ ( x )( h ) ( = φ x ( h ) ) . This equation is the adjoint equation of(2.1).As in the previous section a mild solution (or a solution) of (3.2) is a pair(
Y, Z ) ∈ L F (0 , T ; H ) × L F (0 , T ; L ( H )) such that we have P - a.s. for all t ∈ [0 , T ] Y ν ( · ) ( t ) = S ∗ ( T − t ) ∇ φ ( X ν ( · ) ( T ))+ Z Tt S ∗ ( s − t ) ∇ x H ( X ν ( · ) ( s ) , ν ( s ) , Y ν ( · ) ( s ) , Z ν ( · ) ( s )) ds − Z Tt S ∗ ( s − t ) Z ν ( · ) ( s ) dW ( s ) . (3.3) Theorem 3.1
Assume that b, σ, ℓ, φ are continuously Fr´echet differentiable withrespect to x, the derivatives b x , σ x , σ ν , ℓ x are uniformly bounded, and | φ x | L ( H,H ) ≤ k (1 + | x | H ) for some constant k > . Then there exists a unique (mild) solution ( Y ν ( · ) , Z ν ( · ) ) of BSEE (3.2). The proof of this theorem can be found in [2] or [12]. An alternative proof byusing finite dimensional framework through the Yosida approximation of A canbe found in [18].Our main result is the following. Theorem 3.2
Suppose that the following two conditions hold.(i) b, σ, ℓ are continuously Fr´echet differentiable with respect to x, ν, φ is contin-uously Fr´echet differentiable with respect to x, the derivatives b x , b ν , σ x , σ ν , ℓ x , ℓ ν are uniformly bounded, and | φ x | L ( H,H ) ≤ k (1 + | x | H ) AbdulRahman Al-Hussein for some constant k > . (ii) ℓ x is Lipschitz with respect to u uniformly in x. If ( X ν ∗ ( · ) , ν ∗ ( · )) is an optimal pair for the control problem (2.1)–(2.3), thenthere exists a unique solution ( Y ν ∗ ( · ) , Z ν ∗ ( · ) ) to the corresponding BSEE (3.2) s.t.the following inequality holds: (cid:10) ∇ ν H ( X ν ∗ ( · ) ( t ) , ν ∗ ( t ) , Y ν ∗ ( · ) ( t ) , Z ν ∗ ( · ) ( t )) , ν ∗ ( t ) − ν (cid:11) O ≤ a.e. t ∈ [0 , T ] , a.s. ∀ ν ∈ U. The proof of this theorem will be given in Section 4 below. Now to illustratethis theorem let us present an example.
Example 3.3
Let H and O be two separable Hilbert spaces as considered earlier,and let U = O . We shall study in this example a special case of the control problem(2.1)–(2.3). In particular, given φ as in Theorem 3.2, we would like to minimizethe cost functional: J ( ν ( · )) = E [ Z T | ν ( t ) | O dt ] + E [ φ ( X ν ( · ) ( T )) ] (3.4) subject to: (cid:26) dX ν ( · ) ( t ) = ( A X ν ( · ) ( t ) + B ν ( t ) ) dt + D ν ( t ) dW ( t ) , t ∈ (0 , T ] ,X ν ( · ) (0) = x ∈ H, (3.5) where B is a bounded linear operator from O into H and D is another boundedlinear operator from O into L ( H ) . The Hamiltonian is then given by the formula: H ( x, ν, y, z ) = | ν | O + (cid:10) B ν , y (cid:11) H + (cid:10) Dν , z (cid:11) L ( H ) , where ( x, ν, y, z ) ∈ H × O × H × L ( H ) , and the adjoint BSEE is (cid:26) − dY ν ( · ) ( t ) = A ∗ Y ν ( · ) ( t ) dt − Z ν ( · ) ( t ) dW ( t ) , t ∈ [0 , T ) ,Y ν ( · ) ( T ) = ∇ φ ( X ν ( · ) ( T )) . (3.6) From the construction of the solution of (3.6), as e.g. in [2, Lemma 3.1], thisBSEE attains an explicit solution: Y ν ( · ) ( t ) = E [ S ∗ ( T − t ) ∇ φ ( X ν ( · ) ( T )) | F t ] , ecessary conditions for optimality for SEEs Z ν ( · ) ( t ) = S ∗ ( T − t ) R ν ( · ) ( t ) , where R ν ( · ) is the unique element of L F (0 , T ; L ( H )) satisfying ∇ φ ( X ν ( · ) ( T )) = E [ ∇ φ ( X ν ( · ) ( T )) ] + Z T R ν ( · ) ( t ) dW ( t ) . On the other hand, for fixed ( x, y, z ) , we note that the function ν
7→ H ( x, ν, y, z ) attains its minimum at ν = (cid:0) B ∗ y + D ∗ z (cid:1) ( ∈ U ) , where B ∗ : H → O and D ∗ : L ( H ) → O are the adjoint operators of B and D respec-tively. So we elect ν ∗ ( t, ω ) = 12 (cid:0) B ∗ Y ν ∗ ( · ) ( t, ω ) + D ∗ Z ν ∗ ( · ) ( t, ω ) (cid:1) (3.7) as a candidate optimal control.It is easy to see that with these choices all the requirements of Theorem 3.2are verified. Hence this candidate ν ∗ ( · ) given in (3.7) is an optimal control for theproblem (3.4)–(3.5), and its corresponding optimal solution X ν ∗ ( · ) is the solutionof the following SEE: dX ν ∗ ( · ) ( t ) = (cid:16) A X ν ∗ ( · ) ( t ) + B (cid:0) B ∗ Y ν ∗ ( · ) ( t ) + D ∗ Z ν ∗ ( · ) ( t ) (cid:1)(cid:17) dt + D (cid:0) B ∗ Y ν ∗ ( · ) ( t ) + D ∗ Z ν ∗ ( · ) ( t ) (cid:1)(cid:17) dW ( t ) , t ∈ (0 , T ] ,X ν ∗ ( · ) (0) = x . Finally, the value function attains the formula J ∗ = 14 E (cid:2) Z T | B ∗ Y ν ∗ ( · ) ( t ) + D ∗ Z ν ∗ ( · ) ( t ) | O dt (cid:3) + E [ φ ( X ν ∗ ( · ) ( T )) ] . Remark 3.4
A concrete example in the setting of Example 3.3 can be constructedby taking H = O = L ( R d ) , d ≥ , A = ∆ (half-Laplacian), B = id H , Dν := (cid:10) v , h (cid:11) H Q / , φ ( x ) = (cid:10) ρ , x (cid:11) H , for some fixed elements h, ρ of H and a positivedefinite nuclear operator Q on H. The computations in this case of H , Y ∗ , Z ∗ , ν ∗ , X ∗ become direct from the cor-responding equations in Example 3.3. AbdulRahman Al-Hussein
Let ν ∗ ( · ) be an optimal control and X ∗ ≡ X ν ∗ ( · ) be the corresponding solution of(2.1). Let ν ( · ) be an element of L F (0 , T ; O ) such that ν ∗ ( · ) + ν ( · ) ∈ U ad . For agiven 0 ≤ ε ≤ ν ε ( t ) = ν ∗ ( t ) + ε ν ( t ) , t ∈ [0 , T ] . We note that the convexity of U implies that ν ε ( · ) ∈ U ad . Considering this control ν ε ( · ) we shall let X ν ε ( · ) be the solution of the SEE (2.1) corresponding to ν ε ( · ) , and denote it briefly by X ε . Let p be the solution of the following linear equation: dp ( t ) = (cid:0) A p ( t ) + b x ( X ∗ ( t ) , ν ∗ ( t )) p ( t ) + b ν ( X ∗ ( t ) , ν ∗ ( t )) ν ( t ) (cid:1) dt + (cid:0) σ x ( X ∗ ( t ) , ν ∗ ( t )) p ( t ) + σ ν ( X ∗ ( t ) , ν ∗ ( t )) ν ( t ) (cid:1) dW ( t ) ,p (0) = 0 . (4.1)The following three lemmas contain estimates that will play a vital role in de-riving the desired variational equation and the maximum principle for our controlproblem. Lemma 4.1
Assume condition (i) of Theorem 3.2. Then sup t ∈ [0 ,T ] E [ | p ( t ) | ] < ∞ . Proof.
The solution of (4.1) is given by the formula p ( t ) = Z t S ( t − s ) (cid:0) b x ( X ∗ ( s ) , ν ∗ ( s )) p ( s ) + b ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) (cid:1) ds + Z t S ( t − s ) (cid:0) σ x ( X ∗ ( s ) , ν ∗ ( s )) p ( s ) + σ ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) (cid:1) dW ( s ) . (4.2)By using Minkowski’s inequality (triangle inequality), Holder’s inequality,Burkholder’s inequality for stochastic convolution together with assumption (i)and Gronwall’s inequality we obtain easilysup t ∈ [0 ,T ] E [ | p ( t ) | ] ≤ C (4.3)for some constant C > . ecessary conditions for optimality for SEEs Lemma 4.2
Assuming condition (i) of Theorem 3.2, we have sup t ∈ [0 ,T ] E [ | X ε ( t ) − X ∗ ( t ) | ] = O ( ε ) . Proof.
Observe first from (2.4) that X ε ( t ) − X ∗ ( t ) = Z t S ( t − s ) (cid:0) b ( X ε , ν ε ( s )) − b ( X ∗ ( s ) , ν ∗ ( s )) (cid:1) ds + Z t S ( t − s ) (cid:0) σ ( X ε , ν ε ( s )) − σ ( X ∗ ( s ) , ν ∗ ( s )) (cid:1) dW ( s ) . (4.4)Hence E [ | X ε ( t ) − X ∗ ( t ) | ] ≤ M T E [ Z t | b ( X ε ( s ) , ν ε ( s )) − b ( X ∗ ( s ) , ν ∗ ( s )) | ds ]+ 2 M E [ Z t || σ ( X ε , ν ε ( s )) − σ ( X ∗ ( s ) , ν ∗ ( s )) || ds ] , (4.5)where M := sup t ∈ [0 ,T ] || S ( t ) || L ( H,H ) . Secondly, from condition (i) we get E [ Z t | b ( X ε ( s ) , ν ε ( s )) − b ( X ∗ ( s ) , ν ∗ ( s )) | ds ] ≤ E [ Z t | b ( X ε ( s ) , ν ε ( s )) − b ( X ∗ ( s ) , ν ε ( s )) | ds ]+ 2 E [ Z t | b ( X ∗ ( s ) , ν ε ( s )) − b ( X ∗ ( s ) , ν ∗ ( s )) | ds ]= 2 E [ Z t | ˜ b x ( s, ε )( X ε ( s ) − X ∗ ( s )) | ds ] + 2 E [ Z t | δ ε b ( s ) | ds ] ≤ C E [ Z t | X ε ( s ) − X ∗ ( s ) | ds ] + 2 C ε , (4.6)where, for y ∈ H, ˜ b x ( s, ε )( y ) = Z b x ( X ∗ ( s ) + θ ( X ε ( s ) − X ∗ ( s )) , ν ε ( s ))( y ) dθ,δ ε b ( s ) = b ( X ∗ ( s ) , ν ε ( s )) − b ( X ∗ ( s ) , ν ∗ ( s )) , AbdulRahman Al-Hussein C is a positive constant, and C is another positive constant coming thanks to(i) from the following inequality: E [ Z T | δ ε b ( s ) | ds ] = E [ Z T | b ( X ∗ ( s ) , ν ε ( s )) − b ( X ∗ ( s ) , ν ∗ ( s )) | ds ]= E [ Z T | Z b ν ( X ∗ ( s ) , ν ∗ ( s ) + θ ( ν ε ( s ) − ν ∗ ( s ))) ( ν ε ( s ) − ν ∗ ( s )) dθ | ds ] ≤ C ε . (4.7)Similarly, E [ Z t | σ ( X ε ( s ) , ν ε ( s )) − σ ( X ∗ ( s ) , ν ∗ ( s )) | ds ] ≤ C E [ Z t | X ε ( s ) − X ∗ ( s ) | ds ] + 2 C ε , (4.8)for some positive constants C , C . Finally, by applying (4.6), (4.8) in (4.4) and then using Gronwall’s inequalitywe find that E [ | X ε ( t ) − X ∗ ( t ) | ] ≤ C ε (4.9)for some constant C > C i , i = 1 , . . . , , and M. Hence the proof is complete.Keeping the notations ˜ b x and δ ε b used in the preceding proof let us state thefollowing lemma. Lemma 4.3
Let η ε ( t ) = X ε ( t ) − X ∗ ( t ) ε − p ( t ) . Then, under condition (i) of Theo-rem 3.2, lim ε → + sup t ∈ [0 ,T ] E [ | η ε ( t ) | ] = 0 . ecessary conditions for optimality for SEEs Proof.
From the corresponding equations (2.1) and (4.1) we deduce that η ε ( t ) = Z t S ( t − s ) (cid:2) ε (cid:0) b ( X ε ( s ) , ν ε ( s )) − b ( X ∗ ( s ) , ν ε ( s )) (cid:1) − b x ( X ∗ ( s ) , ν ∗ ( s )) p ( s ) (cid:3) ds + Z t S ( t − s ) (cid:2) ε δ ε b ( s ) − b ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) (cid:3) ds + Z t S ( t − s ) (cid:2) ε (cid:0) σ ( X ε ( s ) , ν ε ( s )) − σ ( X ∗ ( s ) , ν ε ( s )) (cid:1) − σ x ( X ∗ ( s ) , ν ∗ ( s ))) p ( s ) (cid:3) dW ( s )+ Z t S ( t − s ) (cid:2) ε δ ε σ ( s ) − σ ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) (cid:3) ds = Z t S ( t − s ) (cid:2) ˜ b x ( s, ε ) η ε ( s ) + ( ˜ b x ( s, ε ) − b x ( X ∗ ( s ) , ν ∗ ( s )) ) p ( s ) (cid:3) ds + Z t S ( t − s ) (cid:2) ε δ ε b ( s ) − b ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) (cid:3) ds + Z t S ( t − s ) (cid:2) ˜ σ x ( s, ε ) η ε ( s ) + ( ˜ σ x ( s, ε ) − σ x ( X ∗ ( s ) , ν ∗ ( s )) p ( s ) (cid:3) dW ( s )+ Z t S ( t − s ) (cid:2) ε δ ε σ ( s ) − σ ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) (cid:3) dW ( s ) , (4.10)where δ ε σ ( s ) = σ ( X ∗ ( s ) , ν ε ( s )) − σ ( X ∗ ( s ) , ν ∗ ( s ))and ˜ σ x ( s, ε )( y ) = Z σ x ( X ∗ ( s ) + θ ( X ε ( s ) − X ∗ ( s )) , ν ε ( s ))( y ) dθ, y ∈ H. Consequently, from (i) and as in the proof of Lemma 4.2, it follows that E [ | η ε ( t ) | ] ≤ C Z t E [ | η ε ( s ) | ] ds + ρ ( ε ) , (4.11)2 AbdulRahman Al-Hussein for all t ∈ [0 , T ] , where ρ ( ε ) = 8 M T E [ Z T | ( ˜ b x ( s, ε ) − b x ( X ∗ ( s ) , ν ∗ ( s )) ) p ( s ) | ds ]+ 8 M E [ Z T || ( ˜ σ x ( s, ε ) − σ x ( X ∗ ( s ) , ν ∗ ( s )) ) p ( s ) || ds ]+ 4 M T E [ Z T | ε δ ε b ( s ) − b ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) | ds ]+ 4 M E [ Z T || ε δ ε σ ( s ) − σ ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) || ds ] . (4.12)But (i), (4.3) and the dominated convergence theorem give E [ Z T | ( ˜ b x ( s, ε ) − b x ( X ∗ ( s ) , ν ∗ ( s )) ) p ( s ) | ds ]= E [ Z T | Z (cid:0) b x ( X ∗ ( s ) + θ ( X ε ( s ) − X ∗ ( s )) , ν ε ( s )) − b x ( X ∗ ( s ) , ν ∗ ( s )) (cid:1) p ( s ) dθ | ds ] ≤ Z T Z E [ | (cid:0) b x ( X ∗ ( s ) + θ ( X ε ( s ) − X ∗ ( s )) , ν ε ( s )) − b x ( X ∗ ( s ) , ν ∗ ( s )) (cid:1) p ( s ) | ] dθ ds → , as ε → + . Similarly we have E [ Z T || ( ˜ σ x ( s, ε ) − σ x ( X ∗ ( s )) ) p ( s ) || ds ] → , (4.13)as ε → + . On the other hand, as done for (4.7), E [ Z T | ε δ ε b ( s ) − b ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) | ds ] ≤ Z T Z E h (cid:12)(cid:12)(cid:12) (cid:16) b ν ( X ∗ ( s ) , ν ∗ ( s ) + θ ( ν ε ( s ) − ν ∗ ( s ))) − b ν ( X ∗ ( s ) , ν ∗ ( s )) (cid:17) ν ( s ) (cid:12)(cid:12)(cid:12) i dθ ds → , (4.14)if ε → + , by using (i) and the dominated convergence theorem. Similarly, E [ Z T || ε δ ε σ ( s ) − σ ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) || ds ] → , (4.15) ecessary conditions for optimality for SEEs ε → + . Finally applying (4.13)–(4.15) in (4.12) shows that ρ ( ε ) → , as ε → + . Hence from (4.11) and Gronwall’s inequality we obtainsup t ∈ [0 ,T ] E [ | η ε ( t ) | ] → , as ε → + . The following theorem contains our main variational equation, which is one ofthe main tools needed for deriving the maximum principle stated in Theorem 3.2.
Theorem 4.4
We suppose that (i) and (ii) in Theorem 3.2 hold. For each ε > , we have J ( ν ε ( · )) − J ( ν ∗ ( · )) = ε E [ φ x ( X ∗ ( T )) p ( T ) ]+ ε E [ Z T ℓ x ( X ∗ ( s ) , ν ∗ ( s )) p ( s ) ds ]+ E [ Z T (cid:0) ℓ ( X ∗ ( s ) , ν ε ( s )) − ℓ ( X ∗ ( s ) , ν ∗ ( s )) (cid:1) ds ] + o ( ε ) . (4.16) Proof.
We can write J ( ν ε ( · )) − J ( ν ∗ ( · )) as J ( ν ε ( · )) − J ( ν ∗ ( · )) = I ( ε ) + I ( ε ) , (4.17)with I ( ε ) = E [ φ ( X ε ( T )) − φ ( X ∗ ( T )) ]and I ( ε ) = E [ Z T (cid:0) ℓ ( X ε ( s ) , ν ε ( s )) − ℓ ( X ∗ ( s ) , ν ∗ ( s )) (cid:1) ds ] . Note that with the help of our assumptions and by making use of Lemma 4.3,Lemma 4.2, Lemma 4.1 and the dominated convergence theorem we deduce that1 ε I ( ε ) = 1 ε E [ Z φ x ( X ∗ ( T ) + θ ( X ε ( T ) − X ∗ ( T ) ) ( X ε ( T ) − X ∗ ( T )) dθ ]= E [ Z φ x ( X ∗ ( T ) + θ ( X ε ( T ) − X ∗ ( T ) ) ( p ( T ) + η ε ( T )) dθ ] → E [ φ x ( X ∗ ( T )) p ( T ) ] , as ε → + . AbdulRahman Al-Hussein
Hence I ( ε ) = ε E [ φ x ( X ∗ ( T )) p ( T ) ] + o ( ε ) . (4.18)Similarly1 ε I ( ε ) = 1 ε E [ Z T (cid:0) ℓ ( X ε ( s ) , ν ε ( s )) − ℓ ( X ∗ ( s ) , ν ε ( s )) (cid:1) ds ]+ 1 ε E [ Z T (cid:0) ℓ ( X ∗ ( s ) , ν ε ( s )) − ℓ ( X ∗ ( s ) , ν ∗ ( s )) (cid:1) ds ]= E [ Z T Z ℓ x ( X ∗ ( s ) + θ ( X ε ( s ) − X ∗ ( s )) , ν ε ( s )) ( p ( s ) + η ε ( s )) dθ ds ]+ 1 ε E [ Z T (cid:0) ℓ ( X ∗ ( s ) , ν ε ( s )) − ℓ ( X ∗ ( s ) , ν ∗ ( s )) (cid:1) ds ] . On the other hand, applying Lemma 4.3, Lemma 4.2, Lemma 4.1, using thecontinuity and boundedness of ℓ x in (i), (ii) and the dominated convergence the-orem imply that E [ Z T Z ℓ x ( X ∗ ( s ) + θ ( X ε ( s ) − X ∗ ( s )) , ν ∗ ( s ) + ε ν ( s )) ( p ( s ) + η ε ( s )) dθ ds ] → E [ Z T ℓ x ( X ∗ ( s ) , ν ∗ ( s )) p ( s ) ds ] . In particular we obtain I ( ε ) = ε E [ Z T ℓ x ( X ∗ ( s ) , ν ∗ ( s )) p ( s ) ds ]+ E [ Z T (cid:0) ℓ ( X ∗ ( s ) , ν ε ( s )) − ℓ ( X ∗ ( s ) , ν ∗ ( s )) (cid:1) ds ] + o ( ε ) . (4.19)As a result the theorem follows from (4.17)–(4.19).Let us next introduce an important variational inequality. Lemma 4.5
Let hypotheses (i), (ii) in Theorem 3.2 hold. Let ( Y ∗ , Z ∗ ) ≡ ( Y ν ∗ ( · ) , Z ν ∗ ( · ) ) be the solution of BSEE (3.2) corresponding to the optimal pair ( X ∗ , ν ∗ ( · )) . Then ε E (cid:10) Y ∗ ( T ) , p ( T ) (cid:11) + ε E [ Z T ℓ x ( X ∗ ( s ) , ν ∗ ( s )) p ( s ) ds ]+ E (cid:2) Z T (cid:0) δ ε H ( s ) − (cid:10) δ ε b ( s ) , Y ∗ ( s ) (cid:11) − (cid:10) δ ε σ ( s ) , Z ∗ ( s ) (cid:11) (cid:1) ds (cid:3) ≥ o ( ε ) , (4.20) ecessary conditions for optimality for SEEs where δ ε H ( s ) = H ( X ∗ ( s ) , ν ε ( s ) , Y ∗ ( s ) , Z ∗ ( s )) − H ( X ∗ ( s ) , ν ∗ ( s ) , Y ∗ ( s ) , Z ∗ ( s )) . Proof.
Since ν ∗ ( · ) is an optimal control, then J ( ν ε ( · )) − J ( ν ∗ ( · )) ≥ . Hence theresult follows from (4.16) and (3.1).The following duality relation between (4.1) and (3.2) is also needed in orderto establish of proof of Theorem 3.2.
Lemma 4.6
Under hypothesis (i) in Theorem 3.2, we have E (cid:10) Y ∗ ( T ) , p ( T ) (cid:11) = − E [ Z T ℓ x ( X ∗ ( s ) , ν ∗ ( s )) p ( s ) ds ]+ E [ Z T (cid:10) b ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) , Y ∗ ( s ) (cid:11) ds ]+ E [ Z T (cid:10) σ ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) , Z ∗ ( s ) (cid:11) ds ] . (4.21) Proof.
The proof is done by using Yosida approximation of the operator A andItˆo’s formula for the resulting SDEs, and can be gleaned directly from the proofof Theorem 2.1 in [18].We are now ready to establish (or complete in particular) the proof of Theo-rem 3.2. Proof of Theorem 3.2.
Recall the BSEE (3.2): − dY ν ( · ) ( t ) = (cid:0) A ∗ Y ν ( · ) ( t ) + ∇ x H ( X ν ( · ) ( t ) , ν ( t ) , Y ν ( · ) ( t ) , Z ν ( · ) ( t )) (cid:1) dt − Z ν ( · ) ( t ) dW ( t ) , ≤ t < T,Y ν ( · ) ( T ) = ∇ φ ( X ν ( · ) ( T )) . From Theorem 3.1 there exists a unique solution ( Y ∗ , Z ∗ ) to it. Thereby it remainsto prove (3.4).Applying (4.20) and (4.21) gives E h Z T (cid:16) δ ε H ( s ) + (cid:10) ε b ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) − δ ε b ( s ) , Y ∗ ( s ) (cid:11) + (cid:10) ε σ ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) − δ ε σ ( s ) , Z ∗ ( s ) (cid:11) (cid:17) ds i ≥ o ( ε ) . (4.22)6 AbdulRahman Al-Hussein
But, as done for (4.14), by using the continuity and boundedness of b ν in assump-tion (i) and the dominated convergence theorem, one can find that1 ε E [ Z T (cid:10) ε b ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) − δ ε b ( s ) , Y ∗ ( s ) (cid:11) ds = − E (cid:2) Z T (cid:10) Y ∗ ( s ) , Z (cid:16) b ν ( X ∗ ( s ) , ν ∗ ( s ) + θ ( ν ε ( s ) − ν ∗ ( s ))) − b ν ( X ∗ ( s ) , ν ∗ ( s )) (cid:17) ν ( s ) dθ (cid:11) ds (cid:3) → , as ε → + . This means that E [ Z T (cid:10) ε b ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) − δ ε b ( s ) , Y ∗ ( s ) (cid:11) ds ] = o ( ε ) . Similarly, E [ Z T (cid:10) ε σ ν ( X ∗ ( s ) , ν ∗ ( s )) ν ( s ) − δ ε σ ( s ) , Z ∗ ( s ) (cid:11) ds ] = o ( ε ) . Now by applying these two former identities in (4.22) we deduce that E [ Z T δ ε H ( s ) ds ] ≥ o ( ε ) . (4.23)Therefore, by dividing (4.23) by ε and letting ε → + , the following inequalityholds: E [ Z T (cid:10) ∇ ν H ( t, X ∗ ( t ) , ν ∗ ( t ) , Y ∗ ( t ) , Z ∗ ( t )) , ν ( t ) (cid:11) O dt ] ≥ . Finally, (3.4) follows by arguing, if necessary, as in [7, P. 280] for instance.
Acknowledgement.
The author would like to thank the associate editor andanonymous referee(s) for their remarks and also for pointing out the recent workof Fuhrman et al., [11].
References [1] Al-Hussein, A., Martingale representation theorem in infinite dimensions,
Arab J. Math. Sc. , 10, 1 (2004), 1–18. ecessary conditions for optimality for SEEs
Bull. Malays. Math. Sci. Soc.,
30, 2 (2007), 159–183.[3] Al-Hussein, A., Backward stochastic partial differential equations driven byinfinite dimensional martingales and applications,
Stochastics,
81, 6 (2009),601–626.[4] Al-Hussein, A., Sufficient conditions of optimality for backward stochasticevolution equations,
Commun. Stoch. Anal.,
4, 3 (2010), 433–442.[5] Al-Hussein, A., Necessary conditions for optimal control of stochastic evolu-tion equations in Hilbert spaces,
Appl. Math. Optim.,
63, 3 (2011), 385–400.[6] Al-Hussein, A., Sufficient conditions for optimality for stochastic evolutionequations,
Statist. Probab. Lett.,
83, 9 (2013), 2103–2107.[7] Bensoussan, A.,
Stochastic control of partially observable systems,
CambridgeUniversity Press, Cambridge, 1992.[8] Cerrai, S.,
Second order PDE’s in finite and infinite dimension. A probabilis-tic approach.
Lecture Notes in Mathematics, 1762. Springer-Verlag, Berlin,2001.[9] Da Prato, G., Zabczyk, J.,
Second order partial differential equations inHilbert spaces,
London Mathematical Society Lecture Note Series, 293, Cam-bridge University Press, Cambridge, 2002.[10] Debussche, A., Hu, Y., Tessitore, G., Ergodic BSDEs under weak dissipativeassumptions,
Stoch. Proc. Appl.,
C. R. Math. Acad. Sci. Paris , 350 (2012), 683–688.[12] Hu, Y., Peng, S. G., Adapted solution of a backward semilinear stochasticevolution equation,
Stochastic Anal. Appl.,
9, 4 (1991), 445–459.[13] Hu, Y., Peng, S. G., Maximum principle for optimal control of stochasticsystem of functional type,
Stochastic Anal. Appl.,
14, 3 (1996), 283–301.[14] Ichikawa, A., Stability of semilinear stochastic evolution equations,
J. Math.Anal. Appl.
90, 1 (1982), 12–44.8
AbdulRahman Al-Hussein [15] Øksendal, B., Optimal Control of Stochastic Partial Differential Equations,
Stochastic Anal. Appl.,
23 (2005), 165–179.[16] Øksendal, B., Proske, F., Zhang, T., Backward stochastic partial differentialequations with jumps and application to optimal control of random jumpfields,
Stochastics,
77, 5 (2005), 381–399.[17] Peng, S. G., Backward stochastic differential equations and applications tooptimal control,
Appl. Math. Optim.,
27, 2 (1993), 125–144.[18] Tessitore, G., Existence, uniqueness and space regularity of the adapted so-lutions of a backward spde,
Stochastic Anal. Appl.,
14, 4 (1996), 461–486.[19] Yong, J., Zhou, X. Y.,