Mean-Field Pontryagin Maximum Principle
Mattia Bongini, Massimo Fornasier, Francesco Rossi, Francesco Solombrino
aa r X i v : . [ m a t h . O C ] A p r Mean-Field Pontryagin Maximum Principle
Mattia Bongini ∗ , Massimo Fornasier † , Francesco Rossi ‡ , Francesco Solombrino § September 12, 2018
Abstract
We derive a Maximum Principle for optimal control problems with constraints given by the cou-pling of a system of ODEs and a PDE of Vlasov-type. Such problems arise naturally as Γ-limits ofoptimal control problems subject to ODE constraints, modeling, for instance, external interventionson crowd dynamics. We obtain these first-order optimality conditions in the form of Hamiltonianflows in the Wasserstein space of probability measures with forward-backward boundary conditionswith respect to the first and second marginals, respectively. In particular, we recover the equationsand their solutions by means of a constructive procedure, which can be seen as the mean-field limitof the Pontryagin Maximum Principle applied to the discrete optimal control problems, under asuitable scaling of the adjoint variables.
Keywords:
Sparse optimal control, mean-field limit, Γ-limit, optimal control with ODE-PDEconstraints, subdifferential calculus, Hamiltonian flows.
The study of large crowds of interacting agents has received a significantly growing attention in themathematical literature of the last decade with applications in biology, ecology, social sciences, and eco-nomics. Starting from the seminal papers [25, 27, 37, 39], emphasis has been put on self-organization,i.e., the formation of macroscopic patterns from the superimposition of simple, reiterated, binary in-teractions. A quintessential situation is the convergence of a crowd to a common state, which may becalled consensus, agreement, or rendezvous. Several examples show that spontaneous convergence topattern formation is not always guaranteed, e.g., for highly dispersed initial configurations in consensusproblems [17, 18, 21, 30], hence, the issue of controlling and stabilizing these systems arises naturally.Two major interpretations of control of multiagent systems have received much attention: on the onehand, with the decentralized approach , the problem is recast into a game-theoretic framework, whereagents optimize their individual cost and solutions correspond to Nash equilibria. On the other hand,following the concept of centralized intervention , an external policy-maker controlling the dynamics isintroduced, with the task of minimizing its intervention cost. ∗ Technische Universit¨at M¨unchen, Fakult¨at Mathematik, Boltzmannstrasse 3 D-85748, Garching bei M¨unchen, Ger-many. [email protected] † Technische Universit¨at M¨unchen, Fakult¨at Mathematik, Boltzmannstrasse 3 D-85748, Garching bei M¨unchen, Ger-many. [email protected] ‡ Aix Marseille Universit´e, CNRS, ENSAM, Universit´e de Toulon, LSIS UMR 7296, 13397, Marseille, France. [email protected] § Technische Universit¨at M¨unchen, Fakult¨at Mathematik, Boltzmannstrasse 3 D-85748, Garching bei M¨unchen, Ger-many. [email protected] curseof dimensionality , term first coined by Bellman precisely in the context of dynamic optimization: thecomplexity of numerical computations of the solutions of the above problems blows up as the size ofthe population increases. A possible way out is the so-called mean-field approach , where the individualinfluence of the entire population on the dynamics of a single agent is replaced by an averaged one.This substitution principle results in a unique mean-field equation and allows the computation ofsolutions, cutting loose from the dimensionality.In the game-theoretic setting, the mean-field approach has led to the development of mean-fieldgames [26, 29], which model populations whose agents are competing freely with the others towards themaximization of their individual payoff, as for instance in the financial market. The landmark featureof such systems is their capability to autonomously stabilize without external intervention. However,in reality, societies exhibit either convergence to undesired patterns or tendencies toward instabilitythat only an external government can successfully dominate. The need of such interventions, togetherwith the limited amount of resources that governments have at their disposal, makes the design ofparsimonious stabilization strategies a key issue, which has been extensively studied in the context ofdynamics given by systems of ODEs, see [7, 8, 9, 10, 13].Nevertheless, the concept of sparse control has to be handled with care when trying to generalizeit at the level of a mean-field dynamics. Indeed, the indistinguishability of agents is a fundamentalproperty of the mean-field setting, and it is in sharp contrast with controls acting sparsely on specificagents. Figuratively, trying to stabilize a huge crowds with these controls is like steering a river bymeans of toothpicks! A first solution to this ambiguity was given in [6, 23], where the control is definedas a locally Lipschitz feedback control with respect to the state variables, and sparsity refers to itsproperty of having a small support. Such concept was successfully used in [33] to implement sparsestabilizers for a consensus problem. This interpretation of sparsity appears also in the framework ofthe control of more classical PDEs, see [16, 34, 35, 38]. An alternative solution for a proper definitionof sparse mean-field control was proposed in [22], where the control is sparsely applied on a finitenumber of individuals immersed in the mean-field dynamics of the rest of the population, resulting ina system where the controlled ODEs are coupled with a control-free mean-field PDE (but indirectlycontrolled via the coupling). The same kind of control was considered in [1] to model the efficientevacuation of a large crowd of pedestrians with the help of very few informed agents.While in the context of mean-field games and optimal control problems with PDE constraints,first-order optimality conditions have received enormous attention, see for instance [5, 11, 14, 36], upto now no corresponding results have appeared in the literature for coupled ODE-PDE systems of thekind considered in [22], to the best of our knowledge. This paper is devoted to the development ofa Pontryagin Maximum Principle to characterize optima of such control problems. We first remarkthat we are not interested in all possible optima, but mainly on those which arise as limits of optimalstrategies of the original discrete problems. We call this subclass of the set of optima mean-fieldoptimal controls (see Definition 1.4). The interest in this class complies with the wish of using thecontinuous models as approximations of the finite-dimensional ones. Furthermore in the model casesconsidered in [22, 23], it is exactly the existence of mean-field optimal controls that is proved.We summarize our result, borrowing a leaf from the diagram in [14], as follows:2 iscrete OC Problem m ODEs + N ODEs Continuous OC Problem m ODEs + PDEPMP2 m ODEs + 2 N ODEs Extended PMP2 m ODEs + PDE N → + ∞ optimization N → + ∞ optimization We shall provide a set of hypotheses for which the dashed line from the upper-right to the bottom-right box is valid. Our strategy shall be the following: we apply the Pontryagin Maximum Principle(see e.g. [15, Theorem 23.11]) to the finite-dimensional optimal control problems (the solid line fromthe upper-left to the bottom-left box), and we pass to the mean-field limit the system of equationsobtained with this procedure (the solid line from the bottom-left to the bottom-right box). Thederived limit equation for the state and the (rescaled) adjoint variables are obtained in the form ofHamiltonian flows in the Wasserstein space of probability measures, in the sense of [3]. The resultwill be a first-order condition valid for all mean-field optimal controls . The existence of such controlsis also proved (see Corollary 2.15).More formally, we are interested in deriving optimality conditions for the solutions of the followingoptimal control problem subject to coupled ODE-PDE constraints.
Problem 1.
For
T > u ∗ ∈ L ([0 , T ]; U ) minimizing the cost functional F ( u ) = Z T [ L ( y ( t ) , µ ( t )) + γ ( u ( t ))] dt, (1.1)where ( y, µ ) solve ( ˙ y k ( t ) = ( K ⋆ µ ( t ))( y k ( t )) + f k ( y ( t )) + B k u ( t ) , k = 1 , . . . , m,∂ t µ ( t ) = −∇ x · [( K ⋆ µ ( t ) + g ( y ( t ))) µ ( t )] , (1.2)for the given initial datum ( y (0) , µ (0)) = ( y , µ ) ∈ R dm × P c ( R d ).Here, γ is a strictly convex cost functional, the finite dimensional set of controls U is convex andcompact, B k are constant matrices, and P c ( R d ) is the set of probability measures on R d with compactsupport.We shall prove the following main result. Theorem 1.1.
Fix an initial datum ( y , µ ) ∈ R dm × P c ( R d ) and assume that Hypotheses (H) inSection 1.1 hold. Then there exists a mean-field optimal control for Problem 1. Furthermore, if u ∗ is a mean-field optimal control for Problem 1 and ( y ∗ , µ ∗ ) is the corresponding trajectory, then ( u ∗ , y ∗ , µ ∗ ) satisfies the following extended Pontryagin Maximum Principle : here exists ( q ∗ ( · ) , ν ∗ ( · )) ∈ Lip([0 , T ]; R dm × P ( R d )) such that • there exists R T > , depending only on y , supp( µ ) , d, K, g, f k , B k , U , and T , such that supp( ν ∗ ( · )) ⊆ B (0 , R T ) and it satisfies ν ∗ ( t )( E × R d ) = µ ∗ ( t )( E ) for all t ∈ [0 , T ] and forevery Borel set E ⊆ R d ; • it holds ˙ y ∗ k = ∇ q k H c ( y ∗ , q ∗ , ν ∗ , u ∗ ) , ˙ q ∗ k = −∇ y k H c ( y ∗ , q ∗ , ν ∗ , u ∗ ) ,∂ t ν ∗ = −∇ ( x,r ) · (( J ∇ ν H c ( y ∗ , q ∗ , ν ∗ , u ∗ )) ν ∗ ) ,u ∗ = arg max u ∈U H c ( y ∗ , q ∗ , ν ∗ , u ) (1.3) where J ∈ R d × d is the symplectic matrix J = (cid:18) − Id 0 (cid:19) , the Hamiltonian H c : R dm × P c ( R d ) × R D → R is defined as H c ( y, q, ν, u ) = ( H ( y, q, ν, u ) if supp( ν ) ⊆ B (0 , R T ) , + ∞ elsewhere;and H : R dm × P c ( R d ) × R D → R is defined as H ( y, q, ν, u ) = 12 Z R d ( r − r ′ ) · K ( x − x ′ ) dν ( x, r ) dν ( x ′ , r ′ ) + Z R d r · g ( y )( x ) dν ( x, r )+ m X k =1 Z R d q k · K ( y k − x ) dν ( x, r ) + m X k =1 q k · ( f k ( y ) + B k u ) − L ( y, π ν ) − γ ( u ) . (1.4) • the following conditions for system (1.3) hold at time : y ∗ (0) = y and ν ∗ (0)( E × R d ) = µ ( E ) for every Borel set E ⊆ R d , • the following conditions for system (1.3) hold at time T : q ∗ ( T ) = 0 and ν ∗ ( T )( R d × E ) = δ ( E ) for every Borel set E ⊆ R d , where δ is the Dirac measure centered in . As already mentioned, the formulation given above shows that the dynamics of ( y ∗ , q ∗ , ν ∗ ) isessentially an Hamiltonian flow in the Wasserstein space of probability measures with respect to stateand adjoint variables with Hamiltonian H , in the sense of [3]. The definition of H c is introduced tosimplify some technical details and does not alter the result. This fact is remarkably consistent withthe dynamics (1.2), since both are flows in a Wasserstein space. We believe that this formulation ofthe optimality conditions making use of the formalism of subdifferential calculus in Wasserstein spacesof probability measures constitutes one of the novelties of the work.4 emark 1.2. For every ( y, q, ν ) with supp( ν ) ⊆ B (0 , R T ), (1.4) immediately implies that u ∈ arg max u ∈U H c ( y, q, ν, u ) ⇐⇒ u ∈ arg max u ∈U m X k =1 q k · B k u − γ ( u ) ! . Then, the strict convexity of γ and the convexity and the compactness of U imply that u isuniquely determined by ( y, q, ν ). This is the reason why we write the equality symbol in u ∗ =arg max u ∈U H c ( y ∗ , q ∗ , ν ∗ , u ) in place of an inclusion.We point out the difference between the usual gradient in R d with respect to the state variables x and the adjoint variables r , denoted by ∇ ( x,r ) , and the Wasserstein gradient ∇ ν of H c , which, asshown in Section 4, whenever ν has supported contained in B (0 , R T ) can be computed explicitly asfollows: • For l = 1 , . . . , d , it holds ∇ ν H c ( y, q, ν, u )( x, r ) · e l = Z R d ( r − r ′ ) · ( D K ( x − x ′ ) e l ) dν ( x ′ , r ′ ) + r · ( D x g ( y )( x ) e l ) − m X k =1 q k · ( D K ( y k − x ) e l ) − ∇ ξ ℓ ( y, x, R ωµ ) · e l − (cid:0) ∇ ς ℓ ( y, x, R ωµ ) D ω ( x ) (cid:1) · e l . (1.5)These are the components of ∇ ν H c ( y, q, ν, u )( x, r ) in the x l coordinates. • For l = d + 1 , . . . , d it holds ∇ ν H c ( y, q, ν, u )( x, r ) · e l = Z R d K ( x − x ′ ) · e l − d dν ( x ′ , r ′ ) + g ( y )( x ) · e l − d . (1.6)These are the components of ∇ ν H c ( y, q, ν, u )( x, r ) in the r l − d coordinates.In (1.5) and (1.6), the functions ℓ ∈ C ( R dm × R d × R d ; R ) and ω ∈ C ( R d ; R d ) are related to thefunctional L in (1.1) via L ( y, µ ) = Z R d ℓ (cid:0) y, x, R ωµ (cid:1) dµ ( x ) , where R ωµ := ωµ ( R d ), while ∇ ξ ℓ and ∇ ς ℓ denote the partial derivatives of the function ℓ ( η, ξ, ς ),and D ω ( x ) is the Jacobian of the function ω evaluated at x . Notice that ∇ ν H ( y, q, ν, u ) actuallydoes not depend on u , as a consequence of the fact that the control does not act directly on the PDEcomponent of (1.2).The main tool we use to prove Theorem 1.1 is the Pontryagin Maximum Principle (henceforth,simply addressed as PMP) for optimal control problems with ODE constraint. We shall apply it to thefollowing finite-dimensional problems, whose constraints converge to the coupled ODE-PDE systemof Problem 1, as we will show in Section 2. For this reason, we call Theorem 1.1 the extended PMP . Problem 2.
For
T > u ∗ ∈ L ([0 , T ]; U ) minimizing the cost functional F N ( u ) = Z T [ L ( y ( t ) , µ N ( t )) + γ ( u ( t ))] dt, (1.7)5here ( y, µ N ) solve ( ˙ y k = N P Nj =1 K ( y k − x j ) + f k ( y ) + B k u, k = 1 , . . . , m ˙ x i = N P Nj =1 K ( x i − x j ) + g ( y )( x i ) , i = 1 , . . . , N, (1.8)for the given initial datum ( y (0) , x (0)) = ( y , x ) ∈ R dm × R dN , where µ N ( t )( x ) = 1 N N X i =1 δ ( x − x i ( t )) , is the empirical measure centered on the trajectory x ( · ) = ( x ( · ) , . . . , x N ( · )).The extended PMP will be derived after reformulating the finite-dimensional PMP applied toProblem 2 in terms of the empirical measure in the product space of state variables x i and adjointvariables p i , defined as ν N ( x, r ) = 1 N N X i =1 δ ( x − x i , r − N p i ) . Notice that rescaling the adjoint variables p i by the number N of agents is needed in order to observea nontrivial dynamics in the limit (see also Remark 3.7); indeed, within this scaling, the right-handside of the finite-dimensional PMP is brought back to the form considered, for instance, in [19], witha different Hamiltonian.The following diagram recollects the strategy of the proof, making use of the notation alreadyintroduced and reporting in which part of the paper each result is proved: Find u ∗ N subject to (1.8)( y, µ N ) variables Find u ∗ subject to (1.2)( y, µ ) variables u ∗ N satisfies finite-dimensional PMPwith H N ( y, q, ν N , u N ) u ∗ satisfies extended PMPwith H ( y, q, ν, u )Section 2Theorem 3.2 Section 5 Theorem 1.1 The structure of the paper is the following. In Section 1.1 we recall notations and the mainHypotheses (H). In Section 2, we study the controlled dynamics subject to a coupled ODE-PDEconstraint of the form (1.2), establishing existence and uniqueness results for solutions. In Section 3we study the finite-dimensional Problem 2, and apply the PMP to it. In Section 4, we recall basicfacts about subdifferential calculus in Wasserstein spaces, and we explicitly compute ∇ ν H c . In Section5, we prove the extended PMP, i.e., Theorem 1.1. Finally, Section 6 is devoted to the study of aninteresting example of Problem 1, the Cucker-Smale system. We start this section by recalling the notation used throughout the paper.6he constants d, D are two positive integers (the dimension of the space of the agents and of thecontrol, respectively),
T > U is a convex compact subset of R D (set in which controls take values).Functionals have the following expressions: K : R d → R d , each f k satisfies f k : R dm → R d , andfor every y ∈ R dm and µ ∈ P ( R d ), g ( y ) : R d → R d and L ( y, µ ) : R d → R . The matrices B k areconstant d × D matrices.The space P ( R n ) is the set of probability measures which take values on R n , while the space P p ( R n ) is the subset of P ( R n ) whose elements have finite p -th moment, i.e., Z R n k x k p dµ ( x ) < + ∞ . We denote by P c ( R n ) the subset of P ( R n ) which consists of all probability measures with compactsupport. Notice that, if ( µ n ) n ∈ N is a sequence in P c ( R n ) and it exists R > µ n ) ⊆ B (0 , R ) for all n ∈ N , then ( µ n ) n ∈ N is compact in P p ( R n ) for all p ≥ µ ∈ P ( R n ) and any Borel function r : R n → R n , we denote by r µ ∈ P ( R n ) the push-forward of µ through r , defined by r µ ( B ) := µ ( r − ( B )) for every Borel set B of R n . In particular, if one considers the projection operators π and π defined on the product space R n × R n , for every ρ ∈ P ( R n × R n ) we call first (resp., second ) marginal of ρ the probabilitymeasure π ρ (resp., π ρ ). Given µ ∈ P ( R n ) and ν ∈ P ( R n ), we denote with Γ( µ, ν ) the subsetof all probability measures in P ( R n × R n ) with first marginal µ and second marginal ν .On the set P p ( R n ) we shall consider the following distance, called the Wasserstein or Monge-Kantorovich-Rubinstein distance , W pp ( µ, ν ) = inf (cid:26)Z R n k x − y k p dρ ( x, y ) : ρ ∈ Γ( µ, ν ) (cid:27) . (1.9)If p = 1 we have the following equivalent expression for the Wasserstein distance: W ( µ, ν ) = sup (cid:26)Z R n ϕ ( x ) d ( µ − ν )( x ) : ϕ ∈ Lip( R n ) , Lip( ϕ ) ≤ (cid:27) . We denote by Γ o ( µ, ν ) the set of optimal plans for which the minimum is attained, i.e., ρ ∈ Γ o ( µ, ν ) ⇐⇒ ρ ∈ Γ( µ, ν ) and Z R n k x − y k p dρ ( x, y ) = W pp ( µ, ν ) . It is well-known that Γ o ( µ, ν ) is non-empty for every ( µ, ν ) ∈ P p ( R n ) × P p ( R n ), hence the infimumin (1.9) is actually a minimum. For more details, see e.g. [40, 4].For any µ ∈ P ( R d ) and K : R d → R d , the notation K ⋆ µ stands for the convolution of K and µ , i.e., ( K ⋆ µ )( x ) = Z R d K ( x − x ′ ) dµ ( x ′ );this quantity is well-defined whenever K is continuous and sublinear , i.e., there exists C suchthat k K ( ξ ) k ≤ C (1 + k ξ k ) for all ξ ∈ R d . Furthermore we shall deal also with the convolution( ∇ ( x ′ ,r ′ ) h r ′ , K ( x ′ ) i ) ⋆ ν in R d , whose explicit expression is (cid:0) ( ∇ ( x ′ ,r ′ ) h r ′ , K ( x ′ ) i ) ⋆ ν (cid:1) ( x, r ) = Z R d (cid:0) ∇ ( x ′ ,r ′ ) h r − r ′ , K ( x − x ′ ) i (cid:1) dν ( x ′ , r ′ ) . We follow the notation of [4]. ν ∈ P ( R d ). It is nonetheless well-defined for measures ν ∈ P c ( R d ), that is to say for all thecases that will appear in the sequel.We shall denote with M b ( R n ; R n ) the space of bounded Radon vector measures from R n to R n , and with k · k M b ( R n ; R n ) the total variation norm on it. If ω ∈ C ( R d ; R d ) is sublinear and µ ∈ P ( R d ), the Radon measure ωµ ∈ M b ( R d ; R d ) is defined as ωµ ( E ) := Z E ω ( x ) dµ ( x ) , for every E ⊂ R d bounded.We shall denote with R ωµ := ωµ ( R d ).In what follows, we shall consider the space X := R dm × P ( R d ), together with the followingdistance k ( y, µ ) − ( y ′ , µ ′ ) k X := k y − y ′ k + W ( µ, µ ′ ) , (1.10)where k y − y ′ k := P mk =1 k y k − y ′ k k ℓ ( R d ) .Finally, for every N ∈ N , the mapping Π N : R dN → P ( R d ) is defined as followsΠ N : ( x , p , . . . , x N , p N ) N N X i =1 δ ( · − x i , · − N p i ) . (1.11)Henceforth, we assume that the following regularity properties hold. Hypotheses (H)(K)
The function K ∈ C ( R d ; R d ) is odd and sublinear, i.e., there exists C K > x ∈ R d it holds k K ( x ) k < C K (1 + k x k ) . (L) The function L : R dm × P ( R d ) → R is L ( y, µ ) = Z R d ℓ (cid:0) y, x, R ωµ (cid:1) dµ ( x ) , with ℓ ∈ C ( R dm × R d × R d ; R ) and ω ∈ C ( R d ; R d ). (G) The function g ∈ C ( R dm ; C ( R d ; R d )) satisfies for all x ∈ R d and all y ∈ R dm g ( y )( x ) · x ≤ G k x k + G max l =1 ,...,m k y l k + G , where the constants G , G and G are independent on x and y . (F) For each k = 1 , . . . , m , the function f k ∈ C ( R dm ; R d ) satisfies for all y ∈ R dm f k ( y ) · y k ≤ F max l =1 ,...,m k y l k + F , where the constants F and F are independent on y and k . (U) The set
U ⊆ R D is compact and convex. ( γ ) The function γ : U → R is strictly convex. 8 emark 1.3. We briefly compare Hypotheses (H) with those of [5, 11]. In [5], which deals with anSDE-constrained optimal control problem, C , functionals with respect to state variables and thecontrol are considered. Therefore our hypotheses are just slightly more restrictive. On the otherhand, we do not require differentiability of the running cost. The authors of [11] deal, instead, with amean-field game type optimality conditions to model evacuation scenarios. They derive a first-ordercondition under the hypotheses of continuous differentiability of the functionals with respect to thestate variables together with convexity and positivity assumptions. Furthermore, they deal specificallywith an L control cost, while we allow ours to be strictly convex.We now give the rigorous definition of mean-field optimal control . Definition 1.4.
Let ( y , µ ) ∈ R dm × P c ( R d ) be given. An optimal control u ∗ for Problem 1 withinitial datum ( y , µ ) is a mean-field optimal control if there exists a sequence ( u N ) N ∈ N ⊂ L ([0 , T ]; U )and a sequence ( µ N ) N ∈ N ∈ P c ( R d ) such that( i ) for every N ∈ N , µ N ( · ) := N P Ni =1 ( · − x i,N ) is a sequence of empirical measures for some x i,N ∈ supp( µ ) + B (0 ,
1) such that µ N ⇀ µ weakly ∗ in the sense of measures;( ii ) for every N ∈ N , u ∗ N is a solution of Problem 2 with initial datum ( y , µ N );( iii ) there exists a subsequence of ( u N ) N ∈ N converging weakly in L ([0 , T ]; U ) to u ∗ . Remark 1.5.
As mentioned before, the above definition is motivated by our interest in optimizersthat are close to optimal controls for the original finite-dimensional problems. Notice also, that sincethe measures µ N have all compact support contained in supp( µ ) + B (0 ,
1) , the build a compactsequence in P p ( R n ) for all p ≥
1, and therefore, due to weak ∗ convergence to µ , we also have thatlim N →∞ W p ( µ N , µ ) = 0. In this section, we first recall results for PDE equations of transport type with nonlocal interactionvelocities, like the one appearing in the second equation of (1.2). We then study the coupled ODE-PDEdynamics (1.2) and we state existence and uniqueness results of solutions, together with continuousdependence on the initial data ( y , µ ) and on the control u . The proofs follow closely in the footstepsof similar results in [3, 22, 31, 32]. We also show that finite-dimensional ODE dynamics (1.8) areembedded in (1.2), in the sense that the solution of (1.2) with an initial data that is an empiricalmeasure coincides with the empirical measure with support on the solution of (1.8). In this section, we study equations for the dynamics of measures, recalling results of existence anduniqueness. We first define the meaning of solution for the equation ∂ t µ ( t ) = −∇ x · ( v ( t, x, µ ( t )) µ ( t )) , (2.1)where v : [0 , T ] × R n × P ( R n ) → R n is a given vector field and n ∈ N is the dimension of theunderlying Euclidean space. Definition 2.1.
We say that a map µ : [0 , T ] → P ( R n ) is a solution of (2.1) if the following holds:( i ) µ has uniformly compact support, i.e., there exists R > µ ( · )) ∈ B (0 , R );9 ii ) µ is continuous with respect to the Wasserstein distance W ;( iii ) µ satisfies (2.1) in the weak sense, i.e. (see [4, Equation (8.1.4)]), ddt Z R n φ ( x ) dµ ( t )( x ) = Z R n ∇ φ ( x ) · v ( t, x, µ ( t )) dµ ( t )( x ) , for every φ ∈ C ∞ c ( R n ; R ).Now, we can formally define the concept of solution of the controlled ODE-PDE system (1.2),which applies, mutatis mutandis , to system (1.3) as well. Definition 2.2.
Let u ∈ L ([0 , T ]; U ) and ( y , µ ) ∈ X , with µ of bounded support, be given. Wesay that a map ( y, µ ) : [0 , T ] → X is a solution of the system (1.2) with control u if( i ) ( y (0) , µ (0)) = ( y , µ );( ii ) the solution is continuous in time with respect to the metric (1.10) in X ;( iii ) the y coordinates define a Carath´eodory solution of the following controlled ODE problem˙ y k ( t ) = ( K ⋆ µ ( t ))( y k ( t )) + f k ( y ( t )) + B k u ( t ) , k = 1 , . . . , m, for all t ∈ [0 , T ];( iv ) µ is a solution of (2.1), where v : [0 , T ] × R d × P ( R d ) → R d is the time-varying vector fielddefined as follows v ( t, x, µ ( t ))( x ) := ( K ⋆ µ ( t ) + g ( y ( t )))( x ) . We now derive the existence of solutions of (1.2) as limits for N → ∞ of the system of ODE (1.8).We first prove that solutions of (1.8) coincide with specific solutions of (1.2). We then prove the limitresult with the help of Lemmata 2.4 and 2.5. Proposition 2.3.
Let N be fixed, and the control u ∈ L ([0 , T ]; U ) be given. Let ( y, x N ) : [0 , T ] → X be the corresponding solution of (1.8) , with x N ( t ) = ( x ,N ( t ) , . . . , x N,N ( t )) . Then, the couple ( y, µ N ) :[0 , T ] → R dm + dN , with µ N ( t ) being the empirical measure µ N ( t )( x ) := 1 N N X i =1 ( x − x i,N ( t )) , is a solution of (1.2) with control u .Proof. It can be easily proved by rewriting (1.2) with µ N and arguing exactly as in [23, Lemma4.3]. Lemma 2.4.
Let K : R d → R d satisfy (K) and µ ∈ P ( R d ) . Then for all y ∈ R d it holds k ( K ⋆ µ )( y ) k ≤ C K (cid:18) k y k + Z R d k x k dµ ( x ) (cid:19) . Proof.
See, for instance, [23, Lemma 6.4]. 10 emma 2.5.
Let K : R d → R d satisfy (K) and let µ : [0 , T ] → P c ( R d ) and µ : [0 , T ] → P ( R d ) betwo continuous maps with respect to W satisfying supp( µ ( t )) ∪ supp( µ ( t )) ⊆ B (0 , R ) , for every t ∈ [0 , T ] , for some R > . Then for every ρ > there exists constant L ρ,R such that k K ⋆ µ ( t ) − K ⋆ µ ( t ) k L ∞ ( B (0 ,ρ )) ≤ L ρ,R W ( µ ( t ) , µ ( t )) for every t ∈ [0 , T ] .Proof. A proof of this result may be found, for instance, in [23, Lemma 6.7].
Proposition 2.6.
Let y ∈ R dm , µ ∈ P c ( R d ) , and µ N ( · ) be as in Definition 1.4– ( i ) . Let ( u N ) N ∈ N ⊆ L ([0 , T ]; U ) be a sequence of controls such that u N ⇀ u , for some u ∈ L ([0 , T ]; U ) .Then, the sequence of solutions ( y N , µ N ) ∈ Lip([0 , T ]; X ) of (1.8) with initial data ( y , µ N ) andcontrol u N converges to a solution ( y, µ ) ∈ Lip([0 , T ]; X ) of (1.2) with initial data ( y , µ ) and control u . Moreover, there exists ρ T > , depending only on y , supp( µ ) , K, g, f k , B k , U , and T , such thatfor every N ∈ N , for every k = 1 , . . . , m and for every t ∈ [0 , T ] it holds k y k,N ( t ) k , k y k ( t ) k ≤ ρ T and supp( µ N ( t )) , supp( µ ( t )) ⊆ B (0 , ρ T ) . Proof.
We start by fixing
N > k y k,N ( t ) k + k x i,N ( t ) k for k = 1 , . . . , m and i = 1 , . . . N . Let Σ = { ( l, j ) : l = 1 , . . . , m and j = 1 , . . . N } . From Hypotheses (H), Lemma 2.4and the compactness of U , it holds12 ddt (cid:0) k y k,N k + k x i,N k (cid:1) = ˙ y k,N · y k,N + ˙ x i,N · x i,N = (( K ⋆ µ N )( y k,N ) + f k ( y ) + B k u ) · y k,N + (( K ⋆ µ N )( x i ) + g ( y )( x i,N )) · x i,N ≤ k ( K ⋆ µ N )( y k,N ) k k y k,N k + f k ( y N ) · y k,N + k B k u kk y k,N k + k ( K ⋆ µ N )( x i,N ) kk x i,N k + g ( y N )( x i,N ) · x i,N ≤ C K k y k,N k + 1 N N X j =1 k x j,N k k y k,N k + F max l =1 ,...m k y l,N k + F + M k y k,N k + C K k x i,N k + 1 N N X j =1 k x j,N k k x i,N k + G k x i,N k + G max l =1 ,...m k y l,N k + G ≤ C max ( ℓ,j ) ∈ Σ (cid:8) k y ℓ,N k + k x j,N k (cid:9) + C , with C = 4 C K + F + G + M and C = C K + F + G + M . If we denote with b ( k,i ) ( t ) = k y k,N ( t ) k + k x i,N ( t ) k and with a ( t ) = max ( l,j ) ∈ Σ { b ( l,j ) ( t ) } , then the Lipschitz continuity of a impliesthat a is a.e. differentiable, while by Stampacchia’s Lemma (see for instance [28, Chapter 2, LemmaA.4]) for a.e. t ∈ [0 , T ] there exists a ( l, j ) ∈ Σ such that˙ a ( t ) = ddt (cid:0) k y l,N ( t ) k + k x j,N ( t ) k (cid:1) ≤ C a ( t ) + 2 C . Hence, Gronwall’s Lemma and Definition 1.4–( i ) imply that a ( t ) ≤ ( a (0) + 2 C t ) e C t ≤ ( C + 2 C t ) e C t , (2.2)11or some uniform constant C only depending on y and supp( µ ). It then follows that the trajectories( y N ( · ) , µ N ( · )) are bounded uniformly in N in a ball B (0 , ρ T ) ⊂ R d , for ρ T := p C + 2 C T e C T , that is positive and does not depend on t or on N . This in turn implies that the trajectories( y N ( · ) , µ N ( · )) are uniformly Lipschitz continuous in N , as can be easily verified by computing k ˙ y k,N k and k ˙ x i,N k and noticing that all the functions involved are bounded by Hypotheses (H) and the factthat we are inside B (0 , ρ T ). Therefore k ˙ y k,N ( t ) k ≤ ρ ′ T , k ˙ x i,N ( t ) k ≤ ρ ′ T , (2.3)where the constant ρ ′ T does not depend on t or on N .By an application of the Ascoli-Arzel`a theorem for functions on [0 , T ] and values in the completemetric space X , there exists a subsequence, again denoted by ( y N ( · ) , µ N ( · )) converging uniformlyto a limit ( y ( · ) , µ ( · )), whose trajectories are also contained in B (0 , ρ T ). Due to the equi-Lipschitzcontinuity of ( y N ( · ) , µ N ( · )) and the continuity of the Wasserstein distance, we thus obtain for some L T > k ( y ( t ) , µ ( t )) − ( y ( t ) , µ ( t )) k X = lim N → + ∞ k ( y N ( t ) , µ N ( t )) − ( y N ( t ) , µ N ( t )) k X ≤ L T | t − t | , for all t , t ∈ [0 , T ]. Hence, the limit trajectory ( y ∗ ( · ) , µ ∗ ( · )) belongs as well to Lip([0 , T ]; X ).It is now necessary to show that the limit ( y ( · ) , µ ( · )) is a solution of (1.2). We first verify that y is asolution of the ODEs part for µ = µ . To this end, we observe that the limit ( y N ( · ) , µ N ( · )) → ( y ( · ) , µ ( · ))in X specifies into (cid:26) y N ⇒ y, in [0 , T ] , ˙ y N ⇀ ˙ y, in L ([0 , T ] , R d ) . (2.4)and lim N → + ∞ W ( µ N ( t ) , µ ( t )) = 0 , (2.5)uniformly with respect to t ∈ [0 , T ]. As a consequence of (2.4), (2.5), hypothesis (K), and Lemma2.5, for all k = 1 , . . . , m we have in [0 , T ] for N → + ∞ ( K ⋆ µ N )( y k,N ) ⇒ ( K ⋆ µ )( y k ) ,f k ( y N ) ⇒ f k ( y ) . (2.6)To prove that y ( t ) is actually the Carath´eodory solution of (1.8), we have only to show that forall k = 1 , . . . , m one has ˙ y k = ( K ⋆ µ )( y k ) + f k ( y ) + B k u. This is clearly equivalent to the following: for every η ∈ R d and every ˆ t ∈ [0 , T ] it holds η · Z ˆ t ˙ y k ( t ) dt = η · Z ˆ t [( K ⋆ µ ( t ))( y k ( t )) + f k ( y ( t )) + B k u ( t ))] dt, (2.7)which follows from (2.6) and from the weak L -convergence of ˙ y k,N to ˙ y k and of u N to u for N → + ∞ .We are now left with verifying that µ is a solution of (1.2) for y = y . For all ˆ t ∈ [0 , T ] and for all φ ∈ C c ( R d ; R ) we infer that h φ, µ N (ˆ t ) − µ N (0) i = Z ˆ t (cid:20)Z R d ∇ φ ( x ) · [( K ⋆ µ N )( x ) + g ( y N )( x )] dµ N ( t )( x ) (cid:21) dt, ddt h φ, µ N ( t ) i = 1 N ddt N X i =1 φ ( x i ( t )) = 1 N " N X i =1 ∇ φ ( x i ( t )) · ˙ x i ( t ) , and directly applying the substitution ˙ x i = ( K ⋆ µ N )( x i ) + g ( y N )( x i ). By Lemma 2.5 and (2.5), wealso have that for every ρ > N → + ∞ k K ⋆ µ N ( t ) − K ⋆ µ ( t ) k L ∞ ( B (0 ,ρ )) = 0 in [0 , T ] , and, as φ ∈ C c ( R d ) has compact support, it follows thatlim N → + ∞ k∇ φ · ( K ⋆ µ N ( t ) − K ⋆ µ ( t )) k ∞ = 0 in [0 , T ] . Similarly, we have lim N → + ∞ k∇ φ · ( g ( y N ( t )) − g ( y ( t ))) k ∞ = 0 in [0 , T ] , by the compact support of φ , the C -continuity of g and the uniform convergence of y N to y .Denote with L x [0 , ˆ t ] the Lebesgue measure on the time interval [0 , ˆ t ]. Since the product measures L x [0 , ˆ t ] × t µ N ( t ) converge in P ([0 , ˆ t ] × R d ) to L x [0 , ˆ t ] × t µ ( t ), we finally getlim N → + ∞ Z ˆ t Z R d ∇ φ ( x ) · [ K ⋆ µ N ( t )+ g ( y N ( t )]( x ) dµ N ( t )( x ) dt = Z ˆ t Z R d ∇ φ ( x ) · [ K ⋆ µ ( t ) + g ( y ( t ))]( x ) dµ ( t )( x ) dt, that, together with (2.7), proves that ( y, µ ) is a solution of (1.8) with initial data ( y , µ ) and control u . Corollary 2.7.
Let y ∈ R dm , µ ∈ P c ( R d ) , and u ∈ L ([0 , T ]; U ) . Then, there exists a solution of (1.2) with control u and initial datum ( y , µ ) .Proof. Follows from Proposition 2.6 by taking any sequence of empirical measures µ N as in Definition1.4–( i ), and the constant sequence u N ≡ u for all N ∈ N .The following intermediate results shall be helpful in proving the continuous dependance on theinitial data. Proposition 2.8.
Let K : R d → R d and g : R dm → C ( R d ; R d ) satisfy hypotheses (K) and (G).Then, for every R > , there exists L ′ R > satisfying L ′ R ≤ C ′ (1 + R ) for some C ′ > , and k ( K ⋆ µ )( x ) − ( K ⋆ µ )( x ) k ≤ L ′ R ( W ( µ , µ ) + k x − x k ) , (2.8) for all x , x ∈ B (0 , R ) ⊂ R d and µ , µ ∈ P ( R d ) with supp( µ ) , supp( µ ) ⊆ B (0 , R ) .Moreover, for every R > , there exists L R > satisfying L R ≤ C (1 + R ) for some C > , and k ( K ⋆ µ )( x ) + g ( y )( x ) − ( K ⋆ µ )( x ) − g ( y )( x ) k ≤ L R ( W ( µ , µ ) + k x − x k ) , (2.9) for all x , x ∈ B (0 , R ) ⊂ R d , y ∈ B (0 , R ) ⊂ R dm and µ , µ ∈ P ( R d ) with supp( µ ) , supp( µ ) ⊆ B (0 , R ) . roof. By hypothesis, we have k ( K ⋆ µ )( x ) − ( K ⋆ µ )( x ) k = (cid:13)(cid:13)(cid:13)(cid:13)Z R n K ( x − x ′ ) d ( µ − µ )( x ′ ) (cid:13)(cid:13)(cid:13)(cid:13) ≤ Lip R ( K ) W ( µ , µ ) , and k ( K ⋆ µ )( x ) − ( K ⋆ µ )( x ) k ≤ Z R n k K ( x − x ) − K ( x − x ) k dµ ( x ) ≤ Lip R ( K ) k x − x k , where Lip R ( K ) stands for the Lipschitz constant of K on B (0 , R ). Since from (K) it followsLip R ( K ) ≤ C K (1 + R ) , this proves (2.8) for L ′ R := Lip R ( K ) and C ′ := C K . Moreover, there exists ξ ∈ { tx + (1 − t ) x : t ∈ [0 , } such that k g ( y )( x ) − g ( y )( x ) k ≤ sup ξ ∈ B (0 ,R ) ⊂ R d ,ς ∈ B (0 ,R ) ⊂ R dm k D y g ( ς )( ξ ) kk x − x k ≤ M k x − x k , for some M >
0, from the regularity of g . It then suffices to observe that, for some C >
0, it holdsLip R ( K ) + M ≤ C K (1 + R ) + M ≤ C (1 + R ) . This proves (2.9) for L R = Lip R ( K ) + M .The estimate in Proposition 2.8 shows that the following general result holds for vector fields ofthe form v ( t, x, µ ( t )) := ( K ⋆ µ ( t ) + g ( y ( t )))( x ), since from Proposition 2.6 follows that x, y and µ liein domains with a priori known bounds. Proposition 2.9.
Let v, w : [0 , T ] × R d × P ( R d ) → R d be vector fields that satisfy the followinghypotheses:1. v and w are measurable with respect to t ;2. for every R > there exists L R satisfying L R ≤ C (1 + R ) such that for all µ , µ ∈ P ( R d ) with support in B (0 , R ) and all x , x ∈ R d it holds k v ( t, x , µ ) − v ( t, x , µ ) k ≤ L R ( W ( µ , µ ) + k x − x k ) , k w ( t, x , µ ) − w ( t, x , µ ) k ≤ L R ( W ( µ , µ ) + k x − x k ) . (2.10) Moreover, given µ , , µ , ∈ P c ( R d ) , assume that there exist two corresponding solutions µ , µ of (2.1) with vector fields v, w , respectively, and final time T . Then there exist constants C and C such that W ( µ ( t ) , µ ( t )) ≤ e C t W ( µ , , µ , ) + Z t C e C s sup x ∈ B (0 ,R ) k v ( s, x, µ ( s )) − w ( s, x, µ ( s )) k ds, (2.11) where C and C depend on the final time T , on the radius R and L R the Lipschitz constant in (2.10) .Proof. See proofs in [22, Lemma 6.5, Lemma 6.6, Theorem 6.8].We now prove the continuous dependence on the initial data, that also gives uniqueness of thesolution for (1.2). 14 roposition 2.10.
Let the Hypotheses (H) hold. Let u ∈ L ([0 , T ] , U ) be given, and take two solu-tions ( y , µ ) and ( y , µ ) of (1.2) with control u and with initial data ( y , , µ , ) , ( y , , µ , ) ∈ X ,respectively, where µ , and µ , have both compact support. Then there exists a constant C T > such that k ( y ( t ) , µ ( t )) − ( y ( t ) , µ ( t )) k X ≤ C T k ( y , , µ , ) − ( y , , µ , ) k X , for all t ∈ [0 , T ] Proof.
We start by noticing that, by the definition of a solution, we infer the existence of a ρ T > y ( · ) , y ( · ) ∈ B (0 , ρ T ) ⊂ R dm and supp( µ ( · )) , supp( µ ( · )) ⊆ B (0 , ρ T ) ⊂ R d .We shall show the continuous dependence estimate by chaining the stability of the ODE˙ y k ( t ) = ( K ⋆ µ ( t ))( y k ( t )) + f k ( y ( t )) + B k u ( t ) , k = 1 , . . . , m, (2.12)with the one of the PDE ∂ t µ ( t ) = −∇ x · [( K ⋆ µ ( t ) + g ( y ( t ))) µ ( t )] , (2.13)first addressing the dependence of (2.12). By integration we have k y k ( t ) − y k ( t ) k ≤k y , k − y , k k + Z t (cid:0) k ( K ⋆ µ ( s ))( y k ( s )) − ( K ⋆ µ ( s ))( y k ( s )) k + k f k ( y ( s )) − f k ( y ( s )) k (cid:1) ds. (2.14)For the sake of notation, we shall denote with F = max k =1 ,...,m Lip ρ T ( f k ) ,G = sup ξ ∈ B (0 ,ρ T ) ⊂ R d ,ς ∈ B (0 ,ρ T ) ⊂ R dm k D y g ( ς )( ξ ) k . For the left-hand side of (2.14), (2.8), the C -regularity of f k for k = 1 , . . . , m , and the uniform boundon y ( · ) and y ( · ) yield k y k ( t ) − y k ( t ) k ≤ k y , k − y , k k + (2.15)+ Z t (cid:0) L ′ ρ T W ( µ ( s ) , µ ( s )) + L ′ ρ T k y k ( s ) − y k ( s )) k + F k y ( s ) − y ( s ) k (cid:1) ds We now consider (2.13). Define the vector fields v ( t, x, µ ) := ( K ⋆ µ + g ( y ( t )))( x ) , v ( t, x, µ ) := ( K ⋆ µ + g ( y ( t )))( x ) , and let Φ : R d → R be a C ∞ cutoff function on B (0 , ρ T ) with k∇ Φ k ≤ R d . Observe that, since k y ( · ) k , k y ( · ) k ≤ ρ T and supp( µ ( · )) , supp( µ ( · )) ⊆ B (0 , ρ T ), then µ and µ also solve (2.1) with Φ v and Φ v in place of v and v , respectively. It then follows easily fromProposition 2.8 that Proposition 2.9 holds for v = Φ v and w = Φ v . Hence, from (2.11) and takinginto account that v = Φ v and v = Φ v in B (0 , ρ T ), we have W ( µ ( t ) , µ ( t )) ≤ e C t W ( µ , , µ , ) + Z t C e C s sup x ∈ B (0 ,ρ T ) k v ( s, x, µ ( s )) − v ( s, x, µ ( s )) k ds, for some constants C and C . By (2.9) and the regularity of g , for every s ∈ [0 , T ] we have k v ( s, x, µ ( s )) − v ( s, x, µ ( s )) k ≤ L ρ T W ( µ ( s ) , µ ( s )) + G k y ( s ) − y ( s ) k . W ( µ ( t ) , µ ( t )) ≤ e C t W ( µ , , µ , )+ Z t C e C s (cid:0) L ρ T W ( µ ( s ) , µ ( s )) + G k y ( s ) − y ( s ) k (cid:1) ds. (2.16)We now consider the function ε ( t ) := k ( y ( t ) , µ ( t )) − ( y ( t ) , µ ( t )) k X and, combining (2.15) for each k = 1 , . . . , m and (2.16), we obtain ε ( t ) ≤k y , − y , k + Z t (cid:0) L ′ ρ T W ( µ ( s ) , µ ( s )) + L ′ ρ T k y ( s ) − y ( s )) k + mF k y ( s ) − y ( s ) k (cid:1) ds + e C t W ( µ , , µ , ) + Z t C e C s (cid:0) L ρ T W ( µ ( s ) , µ ( s )) + G k y ( s ) − y ( s ) k (cid:1) ds ≤ ε (0) e C t + Z t ( L ′ ρ T + mF + ( L ρ T + G ) C e C s ) ε ( s ) ds. Gronwall’s lemma then implies ε ( t ) ≤ ε (0) e C t (cid:18) ( L ′ ρ T + mF ) t + ( L ρ T + G ) C C ( e C t − (cid:19) . Since t ∈ [0 , T ], the result is proved. Remark 2.11.
Going back to the application of the Ascoli-Arzel´a Theorem in Proposition 2.6, con-sider another converging subsequence of ( y N , µ N ). We can prove that its limit is another solution of(1.8). Since the solution is unique for Proposition 2.10, we have that all converging subsequences of( y N , µ N ) have the same limit, hence the sequence ( y N , µ N ) has itself limit ( y ∗ , µ ∗ ). Remark 2.12.
Since equicompactly supported solutions are unique, given the initial datum, byProposition 2.10, combined with Proposition 2.6 we infer that the support of the unique solution canbe estimated as a function of the data, namely it is contained in a ball B (0 , ρ T ), where the constantis depending only on y , supp( µ ) , K, g, f k , B k , U , and T . In this section, we prove that Problem 1 admits a solution which is a mean-field optimal control. Theproof generalizes similar results in [22].We first recall the main definition of Γ-convergence. We then define the sequence of functionals( F N ) N ∈ N related to Problem 2 and F related to Problem 1 and prove that ( F N ) N ∈ N Γ-converge to F . Definition 2.13 (Γ-convergence) . [20, Definition 4.1, Proposition 8.1] Let X be a metrizable sep-arable space and F N : X → ( −∞ , ∞ ], N ∈ N be a sequence of functionals. We say that ( F N ) N ∈ N Γ -converges to F , written as F N Γ −→ F , for a given F : X → ( −∞ , ∞ ], if1. lim inf -condition: For every u ∈ X and every sequence u N → u , F ( u ) ≤ lim inf N → + ∞ F N ( u N );16. lim sup -condition: For every u ∈ X , there exists a sequence u N → u , called recovery sequence ,such that F ( u ) ≥ lim sup N → + ∞ F N ( u N ) . Furthermore, we call the sequence ( F N ) N ∈ N equi-coercive if for every c ∈ R there is a compactset K ⊆ X such that { u : F N ( u ) ≤ c } ⊆ K for all N ∈ N . As a direct consequence of equi-coercivity,assuming u ∗ N ∈ arg min F N = Ø for all N ∈ N , there is a subsequence ( u ∗ N k ) k ∈ N and u ∗ ∈ X suchthat u ∗ N k → u ∗ ∈ arg min F. In view of the definition of Γ-convergence, let us fix as our domain X = L ([0 , T ]; U ) which,endowed with the weak L -topology, is actually a metrizable space.Fix now an initial datum ( y , µ ) ∈ X , with µ compactly supported, and choose a sequence µ N as in Definition 1.4–( i ).Consider the functional F ( u ) on X defined in (1.1), where the pair ( y, µ ) defines the uniquesolution of (1.2) with initial datum ( y , µ ) and control u . Similarly, consider the functional F N ( u )on X defined in (1.7), where the pair ( y N , µ N ) defines the unique solution of (1.2) with initial datum( y , µ N ) and control u . As recalled in Proposition 2.6, such solution coincides with the solution ofthe ODE system (1.8).The rest of this section is devoted to the proof of the Γ-convergence of the sequence of functionals( F N ) N ∈ N on X to the target functional F . Let us mention that Γ-convergence in optimal controlproblems has been already considered, see for instance [12], but, to our knowledge, it has been onlyrecently specified in connection to mean-field limits in [22, 23]. Theorem 2.14.
Let the functionals (1.1) - (1.7) and dynamics (1.2) satisfy Hypotheses (H). Consideran initial datum ( y , µ ) ∈ R dm × P ( R d ) , and a sequence ( µ N ) N ∈ N , where µ N is as in Definition1.4– ( i ) . Then the sequence of functionals ( F N ) N ∈ N on X = L ([0 , T ]; U ) defined in (1.7) Γ -convergesto the functional F defined in (1.1) .Proof. Let us start by showing the Γ − lim inf condition. Let us fix a weakly convergent sequence ofcontrols u N ⇀ u ∗ in X . We associate to each of these controls a sequence of solutions ( y N , µ N ) of(1.2) uniformly convergent to a solution ( y ∗ , µ ∗ ) with control u ∗ and initial datum ( y , µ ). In viewof the fact that solutions ( y N , µ N ) and ( y ∗ , µ ∗ ) will have uniformly bounded supports with respect to N and t ∈ [0 , T ] and by the uniform convergence of trajectories y N ( t ) ⇒ y ∗ ( t ) as well as the uniformconvergence W ( µ N ( t ) , µ ∗ ( t )) → t ∈ [0 , T ], it follows from the continuity of L under Hypotheses(H) that lim N → + ∞ Z T L ( y N ( t ) , µ N ( t )) = Z T L ( y ∗ ( t ) , µ ∗ ( t )) dt. (2.17)By the assumed weak convergence of ( u N ) N ∈ N to u ∗ ∈ X and Ioffe’s Theorem (see, for instance, [2,Theorem 5.8]) we obtain the lower-semicontinuity of γ lim inf N → + ∞ Z T γ ( u N ( t )) dt ≥ Z T γ ( u ∗ ( t )) dt. (2.18)By combining (2.17) and (2.18), we immediately obtain the Γ − lim inf conditionlim inf N → + ∞ F N ( u N ) ≥ F ( u ∗ ) .
17e now prove the Γ − lim sup condition. We now fix u ∗ and consider the trivial recovery sequence u N ≡ u ∗ for all N ∈ N . Similarly as above for the argument of the Γ − lim inf condition, we canassociate to each of these controls a sequence of solutions ( y N ( t ) , µ N ( t )) of (1.2) uniformly convergentto a solution ( y ∗ ( t ) , µ ∗ ( t )) with control u ∗ and initial datum ( y , µ ) and we can similarly concludethe limit (2.17). Additionally, since ( u N ) N ∈ N is a constant sequence, we havelim inf N → + ∞ Z T γ ( u N ( t )) dt = Z T γ ( u ∗ ( t )) dt. (2.19)Hence, combining (2.17) and (2.19) we can easily inferlim sup N →∞ F N ( u N ) = lim N →∞ F N ( u ∗ ) = F ( u ∗ ) . Corollary 2.15.
Let the Hypotheses (H) in Section 1.1 hold. For every initial datum ( y , µ ) ∈ R dm × P c ( R d ) , there exists a mean-field optimal control u ∗ for Problem 1.Proof. Consider empirical measures µ N as in Definition 1.4–( i ). Notice that the optimal controls u ∗ N of Problem 2 belong to X = L ([0 , T ]; U ), which is a compact set with respect to the weak topologyof L . Hence, the sequence ( F N ) N ∈ N is equicoercive, and ( u ∗ N ) N ∈ N admits a subsequence, which wedo not relabel, weakly convergent to some u ∗ ∈ X .We can associate to each of these controls u ∗ N and initial data ( y , µ N ) a solution ( y N , µ N ) of(1.2). The sequence of solutions ( y N , µ N ) is then uniformly convergent to a solution ( y ∗ , µ ∗ ) of (1.2)with control u ∗ , by Proposition 2.6. In order to conclude that u ∗ is an optimal control for Problem1 (and hence, by construction, that u ∗ is a mean-field optimal control ) we need to show that it isactually a minimizer of F . For that we use the fact that F is the Γ-limit of the sequence ( F N ) N ∈ N asproved in Theorem 2.14. Let u ∈ X be an arbitrary control and let ( u N ) N ∈ N be a recovery sequencegiven by the Γ − lim sup condition, so that F ( u ) ≥ lim sup N → + ∞ F N ( u N ) . (2.20)By using now the optimality of ( u ∗ N ) N ∈ N , we havelim sup N → + ∞ F N ( u N ) ≥ lim sup N → + ∞ F N ( u ∗ N ) ≥ lim inf N → + ∞ F N ( u ∗ N ) . (2.21)Applying the Γ − lim inf condition yieldslim inf N → + ∞ F N ( u ∗ N ) ≥ F ( u ∗ ) . (2.22)By chaining the inequalities (2.20)-(2.21)-(2.22) we have F ( u ) ≥ F ( u ∗ ) , for all u ∈ X, i.e., that u ∗ is an optimal control. Remark 2.16.
Observe that the previous result does not state uniqueness of the optimal control forthe infinite dimensional problem. Indeed, in general, we cannot ensure that all solutions of Problem1 are mean-field optimal controls. 18
The finite-dimensional problem
In this section we study the discrete Problem 2 and state the PMP for it. We first recall the followingexistence result for the optimal control problem.
Proposition 3.1 (Theorem 23.11, [15]) . Under Hypotheses (H), Problem 2 admits solutions.
We now introduce the adjoint variables of x i and y k , denoted by p i and q k , respectively, andstate the PMP in the following box. Theorem 3.2 (Theorem 22.2, [15]) . Let u ∗ N be a solution of Problem 2 with initial datum ( y (0) , x (0)) = ( y , x ) , and denote with ( y ∗ ( · ) , x ∗ ( · )) : [0 , T ] → R dm + dN the corresponding trajec-tory. Then there exists a Lipschitz curve ( y ∗ ( · ) , q ∗ ( · ) , x ∗ ( · ) , p ∗ ( · )) ∈ Lip([0 , T ] , R dm +2 dN ) solvingthe system ˙ y ∗ k = ∇ q k H N ( y ∗ , q ∗ , x ∗ , p ∗ , u ∗ )˙ q ∗ k = −∇ y k H N ( y ∗ , q ∗ , x ∗ , p ∗ , u ∗ ) k = 1 , . . . , m, ˙ x ∗ i = ∇ p i H N ( y ∗ , q ∗ , x ∗ , p ∗ , u ∗ )˙ p ∗ i = −∇ x i H N ( y ∗ , q ∗ , x ∗ , p ∗ , u ∗ ) i = 1 , . . . , N,u ∗ N = arg max u ∈U H N ( y ∗ , q ∗ , x ∗ , p ∗ , u ) , (3.1) with initial datum ( y (0) , x (0)) = ( y , x ) and terminal datum ( q ( T ) , p ( T )) = 0 , where the Hamil-tonian H N : R dm +2 dN → R is given by H N ( y, q, x, p, u ) = N X i =1 p i · N N X j =1 K ( x i − x j ) + g ( y )( x i ) ++ m X k =1 q k · N N X j =1 K ( y k − x j ) + f k ( y ) + B k u − L ( y, µ N ) − γ ( u ) , (3.2) with µ N = N P Ni =1 δ ( x − x i ) . Remark 3.3.
The general statement of the PMP contains both normal and abnormal minimizers.In our case, the simpler formulation of the PMP is given by the fact that we have normal minimizersonly. This is a consequence of the fact that the final configuration is free, see e.g. [15, Corollary 22.3].
Remark 3.4.
The uniqueness of the maximizer of H N follows from the same motivations reportedin Remark 1.2. Indeed, the form of the Hamiltonian implies that for each u ∗ ∈ U it holds u ∗ = arg max u ∈U H N ( y ∗ , q ∗ , x ∗ , p ∗ , u ) when u ∗ = arg max u ∈U m X k =1 q ∗ k · B k u − γ ( u ) ! . In other terms, since the control acts on the y variables only, then we have a simpler formulation forthe maximization of the Hamiltonian H N .We now want to embed solutions of the PMP for Problem 2 as solutions of the extended PMP forProblem 1. As a first step, we prove that pairs control-trajectories ( u ∗ N , ( y ∗ N , q ∗ N , x ∗ N , p ∗ N )) satisfyingsystem (3.1) have support uniformly bounded in time and in N ∈ N .19 roposition 3.5. Let y ∈ R dm , µ ∈ P c ( R d ) , and µ N be as in Definition 1.4– ( i ) . Let u ∗ N be asolution of Problem 2 with initial datum ( y , µ N ) , and let ( u ∗ N , ( y ∗ N , q ∗ N , x ∗ N , p ∗ N )) be a pair control-trajectory satisfying the PMP for Problem 2 with initial datum ( y , µ N ) and control u ∗ N given byTheorem 3.2.Then the trajectories ( y ∗ N ( · ) , q ∗ N ( · ) , ν ∗ N ( · )) , where ν ∗ N := Π N ( x ∗ N , p ∗ N ) , are equibounded and equi-Lipschitz continuous from [0 , T ] to Y , where the space Y := R dm × P ( R d ) is endowed with thedistance k ( y, q, ν ) − ( y ′ , q ′ , ν ′ ) k Y = k y − y ′ k + k q − q ′ k + W ( ν, ν ′ ) . (3.3) Furthermore, there exists R T > , depending only on y , supp( µ ) , d, K, g, f k , B k , U , and T , such that supp( ν ∗ N ( · )) ⊆ B (0 , R T ) for all N ∈ N . In particular, it holds H ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N ) = H c ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N ) .Proof. As a first step, notice that the pair ( y ∗ N , x ∗ N ) solves the system (1.8). It then follows from(2.2) and (2.3) that there exist two constants ρ T and ρ ′ T , not depending on N such that, for all i = 1 , . . . , N , for all k = 1 , . . . , m , and a.e. t ∈ [0 , T ] we have k y ∗ k,N ( t ) k ≤ ρ T , k x ∗ i,N ( t ) k ≤ ρ T (3.4) k ˙ y ∗ k,N ( t ) k ≤ ρ ′ T , k ˙ x ∗ i,N ( t ) k ≤ ρ ′ T . (3.5)It follows in particular that there exists a uniform constant W T such that (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) N N X i =1 ω ( x ∗ i,N ( t )) (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) ≤ W T for all t ∈ [0 , T ].We now observe that by an explicit computation N ∇ x i H N ( y ∗ N , q ∗ N , x ∗ N , p ∗ N , u ∗ N ) · e l = N r ∗ i,N N · N N X j =1 D K ( x ∗ i,N − x ∗ j,N ) e l − N N X j =1 r ∗ j,N N · N D K ( x ∗ j,N − x ∗ i,N ) e l + N r ∗ i,N N · ∇ g ( y ∗ N )( x ∗ i,N ) − N m X k =1 q ∗ k,N · N D K ( y ∗ k,N − x ∗ i,N ) e l − N N ∇ ξ ℓ (cid:16) y ∗ N , x ∗ i,N , N P Nj =1 ω ( x ∗ j,N ) (cid:17) · e l + ∇ ς ℓ (cid:16) y ∗ N , x ∗ i,N , N P Nj =1 ω ( x ∗ j,N ) (cid:17) D ω ( x ∗ i,N ) · e l ! = 1 N N X j =1 ( r ∗ i,N − r ∗ j,N ) · (cid:0) D K ( x ∗ i,N − x ∗ j,N ) e l (cid:1) + r ∗ i,N · D x g ( y ∗ N )( x ∗ i,N ) − m X k =1 q ∗ k,N · ( D K ( y ∗ k,N − x ∗ i,N ) e l ) − ∇ ξ ℓ (cid:16) y ∗ N , x ∗ i,N , N P Nj =1 ω ( x ∗ j,N ) (cid:17) · e l − (cid:16) ∇ ς ℓ (cid:16) y ∗ N , x ∗ i,N , N P Nj =1 ω ( x ∗ j,N ) (cid:17) D ω ( x ∗ i,N ) (cid:17) · e l , (3.6)20or each i = 1 , . . . , N and each l = 1 , . . . , d (where we have used that D K is even and we merged thefirst two terms). Therefore, since ˙ p ∗ i,N solves (3.1) and ˙ r ∗ i,N = N ˙ p ∗ i,N , we get − ˙ r ∗ i,N ( t ) · e l = 1 N N X j =1 ( r ∗ i,N ( t ) − r ∗ j,N ( t )) · ( D K ( x ∗ i,N ( t ) − x ∗ j,N ( t )) e l )+ r ∗ i,N ( t ) · ( D y g ( y ∗ N ( t ))( x ∗ i,N ( t )) e l ) − m X k =1 q ∗ k,N ( t ) · ( D K ( y ∗ k,N ( t ) − x ∗ i,N ( t )) e l ) − ∇ ξ ℓ (cid:16) y ∗ N ( t ) , x ∗ i,N ( t ) , N P Nj =1 ω ( x ∗ j,N ( t )) (cid:17) · e l − (cid:16) ∇ ς ℓ (cid:16) y ∗ N ( t ) , x ∗ i,N ( t ) , N P Nj =1 ω ( x ∗ j,N ( t )) (cid:17) D ω ( x ∗ i,N ( t )) (cid:17) · e l , for each i = 1 , . . . , N , each l = 1 , . . . , d , and a.e. t ∈ [0 , T ], where we have used the fact that D K iseven. We now denote with L T a uniform constant such that k D K k L ∞ ( B (0 ,ρ T ) , R d × d ) ≤ L T , sup | y |≤√ mR T k D y g ( y )( · ) k L ∞ ( B (0 ,ρ T ) , R d × d ) ≤ L T k∇ ξ ℓ k L ∞ ( B (0 , √ mρ T ) × B (0 ,ρ T ) × B (0 ,W T ) , R d ) ≤ L T , k∇ ς ℓ k L ∞ ( B (0 , √ mρ T ) × B (0 ,ρ T ) × B (0 ,W T ) , R d ) ≤ L T , k D ω k L ∞ ( B (0 ,ρ T ) , R d × d ) ≤ L T , and we easily get the estimate k ˙ r ∗ i,N ( t ) k ≤ √ dL T k r ∗ i,N ( t ) k + 1 N N X j =1 k r ∗ j,N ( t ) k + m X k =1 k q ∗ k,N ( t ) k + 1 + L T (3.7)for each i = 1 , . . . , N and a.e. t ∈ [0 , T ]. An explicit computation of ∇ y k H N and a similar argument,possibly with another constant L T , show the estimate k ˙ q ∗ k,N ( t ) k ≤ √ dL T N N X i =1 k r ∗ i,N ( t ) k + 2 k q ∗ k,N ( t ) k + L T ! (3.8)for each k = 1 , . . . , m and a.e. t ∈ [0 , T ]. We now set ε N ( t ) := m X k =1 k q ∗ k,N ( t ) k + 1 N N X i =1 k r ∗ i,N ( t ) k , and observe that it holds | ˙ ε N ( t ) | ≤ m X k =1 k ˙ q ∗ k,N ( t ) k + 1 N N X i =1 k ˙ r ∗ i,N ( t ) k , therefore (3.7) and (3.8) yield | ˙ ε N ( t ) | ≤ √ dL T (4 ε N ( t ) + 1 + 2 L T ) . (3.9)Defining then the increasing functions η N ( t ) through η N ( t ) := sup τ ∈ [0 ,t ] ε N ( T − τ ), and observing thatit holds η N (0) = 0 for the boundary conditions in Theorem 3.2, from (3.9) and Gronwall’s Lemma weobtain η N ( τ ) ≤ √ dL T τ (1 + 2 L T ) e (4 √ dL T ) τ ε N ( t ) ≤ η N ( T ) ≤ √ dL T T (1 + 2 L T ) e (4 √ dL T ) T := C T (3.10)for all t ∈ [0 , T ]. Plugging into (3.9), we get the existence of a constant C ′ T such that | ˙ ε N ( t ) | ≤ C ′ T (3.11)for a.e. t ∈ [0 , T ]. Since by definition of ν ∗ N ( t ) and standard properties of the Wasserstein distance W it holds W ( ν ∗ N ( t + τ ) , ν ∗ N ( t )) ≤ √ N N X i =1 k x ∗ i,N ( t + τ ) − x ∗ i,N ( t ) k + 1 N N X i =1 k r ∗ i,N ( t + τ ) − r ∗ i,N ( t )) k ! , from the previous inequality, (3.4), (3.5), (3.9), (3.10), and (3.11) we obtain that y ∗ N ( t ) and q ∗ N ( t ) areequibounded, that there exist a constant, denoted by R T , such that supp( ν ∗ N ( t )) ⊂ B (0 , R T ) for all t ∈ [0 , T ] and that ( y ∗ N , q ∗ N , ν ∗ N ) are equi-Lipschitz continuous from [0 , T ] with values in Y . Proposition 3.6.
Let N ∈ N and u ∗ N ∈ L p ([0 , T ]; U ) be an optimal control for Problem 2 withgiven initial datum ( y N , x N ) ∈ R dm + dN , and ( y ∗ N ( · ) , q ∗ N ( · ) , x ∗ N ( · ) , p ∗ N ( · )) ∈ Lip([0 , T ] , R dm +2 dN ) acorresponding trajectory of the PMP with maximized Hamiltonian H N .Define ν ∗ N := Π N ( x ∗ ,N , p ∗ ,N , . . . , x ∗ N,N , p ∗ N,N ) with Π N as in (1.11) , and assume that supp( ν ∗ N ( · )) ⊆ B (0 , R T ) . Then, the control u ∗ N is optimal for Problem 1 and ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N ) satisfies the extendedPontryagin Maximum Principle.Proof. First observe that, by Proposition 3.5, H c ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N ) = H ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N ) and that forevery t ∈ [0 , T ] u ∗ N ( t ) = arg max u ∈U H N ( y ∗ N ( t ) , q ∗ N ( t ) , x ∗ N ( t ) , p ∗ N ( t ) , u ) ⇐⇒ u ∗ N ( t ) = arg max u ∈U H ( y ∗ N ( t ) , q ∗ N ( t ) , ν ∗ N ( t ) , u ) , due to the specific form of the Hamiltonian H N and H , see Remark 3.4.We now prove that ( ˙ y ∗ k,N = ∇ q k H N ( y ∗ N , q ∗ N , x ∗ N , p ∗ N , u ∗ N ) , ˙ q ∗ k,N = −∇ y k H N ( y ∗ N , q ∗ N , x ∗ N , p ∗ N , u ∗ N ) , = ⇒ ( ˙ y ∗ k,N = ∇ q k H ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N ) , ˙ q ∗ k,N = −∇ y k H ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N ) , i.e., that if the ( y, q ) variables satisfy the PMP for Problem 2 then they satisfy the extended PMPfor Problem 1. It is sufficient to observe that H N can be rewritten in terms of ν ∗ N ( · ) as follows H N ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N ) = Z R d r · ( K ⋆ π ν ∗ N )( x ) dν ∗ N ( x, r ) + Z R d r · g ( y ∗ N )( x ) dν ∗ N ( x, r )+ m X k =1 Z R d q ∗ k,N · K ( y ∗ k,N − x ) dν ∗ N ( x, r ) + m X k =1 q ∗ k,N · ( f k ( y ∗ k,N ) + B k u ∗ N ) − L ( y ∗ N , π ν ∗ N ) − γ ( u ∗ N ) , where we used the variable r = N p . Comparing it with H ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N ), one has that theirexpressions coincide up to the first term. Since such first term is independent on y k and q k , then ∇ y k H N = ∇ y k H and ∇ q k H N = ∇ q k H , hence equations for ˙ y ∗ k,N , ˙ q ∗ k,N in the PMP for Problem 2 andin the extended PMP for Problem 1 coincide. 22e now prove a similar result for the ( x ∗ i,N , r ∗ i,N ) variables, with r ∗ i,N = N p ∗ i,N . After this changeof variable, the third and the fourth equation in (3.1) become ( ˙ x ∗ i,N = N ∇ r i H N ( y ∗ N , q ∗ N , x ∗ N , p ∗ N , u ∗ N )˙ r ∗ i,N = − N ∇ x i H N ( y ∗ N , q ∗ N , x ∗ N , p ∗ N , u ∗ N ) . We want to prove that the following identity holds J ( ∇ ν H c ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N ))( x ∗ i,N , r ∗ i,N ) = (cid:18) N ∇ r i H N ( y ∗ N , q ∗ N , x ∗ N , p ∗ N , u ∗ N ) − N ∇ x i H N ( y ∗ N , q ∗ N , x ∗ N , p ∗ N , u ∗ N ) (cid:19) , (3.12)i.e., that the Hamiltonian vector fields generated by H and H N coincide in each point ( x ∗ i,N , r ∗ i,N ).The presence of the constant N in the right-hand side is due to the change of variables r ∗ i,N = N p ∗ i,N .By applying J − on both sides of (3.12), we need to prove ∇ ν H c ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N )( x ∗ i,N , r ∗ i,N ) · e l = N ∇ x i H N ( y ∗ N , q ∗ N , x ∗ N , p ∗ N , u ∗ N ) · e l for l = 1 , . . . , d, (3.13) ∇ ν H c ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N )( x ∗ i,N , r ∗ i,N ) · e l = N ∇ r i H N ( y ∗ N , q ∗ N , x ∗ N , p ∗ N , u ∗ N ) · e l − d for l = d + 1 , . . . , d. (3.14)By writing explicitly the left hand sides of (3.13) and (3.14) by using the expressions (1.5)-(1.6) andevaluating them in ( x ∗ i,N , r ∗ i,N ), we have ∇ ν H c ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N )( x ∗ i,N , r ∗ i,N ) · e l = 1 N N X j =1 ( r ∗ i,N − r ∗ j,N ) · (cid:0) D K ( x ∗ i,N − x ∗ j,N ) e l (cid:1) + r ∗ i,N · (cid:0) D x g ( y ∗ N )( x ∗ i,N ) e l (cid:1) − m X k =1 q ∗ k,N · (cid:0) D K ( y ∗ k,N − x ∗ i,N ) e l (cid:1) − ∇ ξ ℓ (cid:16) y ∗ N , x ∗ i,N , N P Nj =1 ω ( x ∗ j,N ) (cid:17) · e l − (cid:16) ∇ ς ℓ (cid:16) y ∗ N , x ∗ i,N , N P Nj =1 ω ( x ∗ j,N ) (cid:17) D ω ( x ∗ i,N ) (cid:17) · e l , for l = 1 , . . . , d , so that (3.13) follows immediately from (3.6). Similarly, we have ∇ ν H c ( y ∗ N , q ∗ N , ν ∗ N , u ∗ N )( x ∗ i,N , r ∗ i,N ) · e l = 1 N N X j =1 K ( x ∗ i,N − x ∗ j,N ) · e l − d + g ( y ∗ N )( x ∗ i,N ) · e l − d for l = d + 1 , . . . , d , which coincides with the right hand side of (3.14) by an explicit computation.Since the boundary conditions of Problem 2 and Problem 1 coincide too, after the identification ν ∗ N := Π N ( x ∗ ,N , p ∗ ,N , . . . , x ∗ N,N , p ∗ N,N ), the result follows now by (3.13)-(3.14) arguing, for instance, asin [23, Lemma 4.3].
Remark 3.7.
It is interesting to observe that the embedding of a trajectory of the PMP for Problem2 to the empirical measure formulation depends on the number N of agents, see the definition ofΠ N in (1.11). This is a consequence of the fact that the Hamiltonian of the PMP for Problem 2actually depends on the number of agents. Indeed, consider a population composed of a unique agent( x , p ), for which the first term of the Hamiltonian reads as p · g ( y )( x ). Consider now a populationcomposed of two agents ( x , p , x , p ) satisfying x = x and p = p , for which the first term of theHamiltonian reads as 2 p · g ( y )( x ).Clearly, in both cases the empirical measure in the state variables is µ = µ = δ ( x − x ),while the definition of Π N gives two different empirical measures for the cotangent bundle: ν = δ ( x − x , r − p ) and ν = δ ( x − x , r − p ). This difference is needed to compensate the dependenceof the Hamiltonian of the PMP for Problem 2 on the number N of agents.23 The Wasserstein gradient
We anticipated in Section 1 that the dynamics of ν ∗ in (1.3) is an Hamiltonian flow in the Wassersteinspace of probability measures, in the sense of [3]. This means that the vector field ∇ ν H c ( ν ∗ ) is anelement with minimal norm in the Fr´echet subdifferential at the point ν ∗ of the maximized Hamiltonian H c introduced in Theorem 1.1 (we drop for simplicity the y , q and u dependency). The proof of thisfact shall follow the strategy adopted to obtain analogous results in [4, Chapter 10], which howevercannot be applied verbatim to our case due to the peculiar nature of our operators. In order to usethose techniques, we consider our functionals defined on P ( R d ) instead than on P ( R d ). Since wehave already proven in Proposition 3.5 that, whenever we start from a compactly supported initialdatum, the dynamics remains compactly supported uniformly in time, this assumption does not alterour conclusions.We start with some basic definitions and general results on functionals defined on P ( R d ): thefollowing one is motivated by Definition 10.3.1 and Remark 10.3.3 in [4]. Definition 4.1.
Let ψ : P ( R d ) → ( −∞ , + ∞ ] be a proper and lower semicontinuous functional, andlet ν ∈ D ( ψ ). We say that w ∈ L ν ( R d ) belongs to the (Fr´echet) subdifferential of ψ at ν , insymbols w ∈ ∂ψ ( ν ) if and only if for any ν ∈ P ( R d ) it holds ψ ( ν ) − ψ ( ν ) ≥ inf ρ ∈ Γ o ( ν ,ν ) Z R d w ( z ) · ( z − z ) dρ ( z , z ) + o ( W ( ν , ν )) . Proposition 4.2 ([4], Theorem 10.3.10) . Fix the functional ψ : P ( R d ) → ( −∞ , + ∞ ] . Then, forevery ν ∈ D ( ψ ) , the metric slope | ∂ψ | ( ν ) = lim sup ν → ν ( ψ ( ν ) − ψ ( ν )) + W ( ν , ν ) satisfies | ∂ψ | ( ν ) ≤ k w k L ν (4.1) for every w ∈ ∂ψ ( ν ) . The following property shall guarantee that the subdifferential of H c is nonempty. Definition 4.3.
A proper, lower semicontinuous functional ψ : P ( R n ) → ( −∞ , + ∞ ] is semiconvexalong geodesics whenever, for every ν , ν ∈ P ( R n ) and ρ ∈ Γ o ( ν , ν ) there exists C ∈ R for whichit holds ψ (((1 − s ) π + sπ ) ρ ) ≤ (1 − s ) ψ ( ν ) + sψ ( ν ) + Cs (1 − s ) W ( ν , ν )) for every s ∈ [0 , . In what follows, we shall fix y, q ∈ R dm and u ∈ L ([0 , T ]; U ) and we write, for the sake ofcompactness, H c ( ν ) in place of H c ( y, q, ν, u ). Moreover, K shall denote a convex, compact subset of R d and z = ( x, r ) a variable in R d .Whenever supp( ν ) ⊆ B (0 , R T ) , H c ( ν ) can be rewritten as H c ( ν ) = 12 Z R d F ( z − z ′ ) dν ( z ) ν ( z ′ ) + Z R d G ( z ) dν ( z ) − Z R d ℓ ( π ( z ) , R ωπ ν ) dν ( z ) + Q, where F ( x, r ) = r · K ( x ) G ( x, r ) = r · g ( y )( x ) + m X k =1 q k · K ( y k − x ) , Q collects all the remaining terms not depending on ν . Notice that F is an even function.In order to prove the semiconvexity of H c , we shall establish the semiconvexity of the followingfunctionals: ˆ H c ( ν ) = 12 Z R d ˆ F ( z − z ′ ) dν ( z ) ν ( z ′ ) + Z R d ˆ G ( z ) dν ( z ) , ˆ H c ( ν ) = Z R d ˆ ℓ ( z, R ˆ ων ) dν ( z ) , where ˆ F , ˆ G , ˆ ℓ , and ˆ ω are C functions. The desired result will then follow by noticing that H c ( ν ) =ˆ H c ( ν ) + ˆ H c ( ν ) for ˆ F = F , ˆ G = G , ˆ ℓ = − ℓ ◦ ( π , Id), ˆ ω = ω ◦ π and K = B (0 , R T ) .The following simple property will be needed to prove semiconvexity of the above functionals. Lemma 4.4.
Let ν , ν ∈ P c ( R d ) with support contained in K . Let ρ ∈ Γ( ν , ν ) and set ν s = ((1 − s ) π + sπ ) ρ, (4.2) for every s ∈ [0 , . Then, it holds supp( ν s ) ⊆ K for all s ∈ [0 , . Proof.
We first notice, that for every ρ ∈ Γ( ν , ν ) it holdssupp( ρ ) ⊆ K × K . (4.3)This follows from the equality R d \ ( K × K ) = ( R d × ( R d \K )) ∪ (( R d \K ) × R d )and from the fact that both R d × ( R d \K )) and ( R d \K ) × R d are ρ -null sets by hypothesis.To prove the Lemma, it suffices to show that for all f ∈ C ( R d ) satisfying f ≡ K it holds Z R d f dν s = 0 . (4.4)Indeed, Z R d f dν s = Z R d f d ((1 − s ) π + sπ ) ρ ( z , z )= Z R d f ((1 − s ) z + sz ) dρ ( z , z )= Z K×K f ((1 − s ) z + sz ) dρ ( z , z ) , since, by (4.3), supp( ρ ) ⊆ K × K . From the convexity of K follows that (1 − s ) z + sz ∈ K for every s ∈ [0 , f ≡ K , yield (4.4), as desired.In what follows, we shall make use of the following, well-known result. Remark 4.5.
Let K be a convex, compact subset of R d and let f ∈ C ( R d ; R ). Then there exists C K ,f ∈ R depending only on K and f such that f ((1 − s ) x + sx ) ≤ (1 − s ) f ( x ) + sf ( x ) + C K ,f s (1 − s ) k x − x k , (4.5)for every x , x ∈ R d and s ∈ [0 , H c . Lemma 4.6.
Let ν , ν ∈ P c ( R d ) and let ρ ∈ Γ( ν , ν ) . Then, there exists C ∈ R independent of ν and ν for which ˆ H c (((1 − s ) π + sπ ) ρ ) ≤ (1 − s ) ˆ H c ( ν ) + s ˆ H c ( ν ) + Cs (1 − s ) W ( ν , ν ) holds for every s ∈ [0 , .Proof. We may assume supp( ν ) , supp( ν ) ⊆ K for some convex and compact set K ⊂ R d , otherwisethe inequality is trivial. Hence, from Lemma 4.4, it follows supp( ν s ) ⊆ K for every s ∈ [0 , F and ˆ G are both C , the result follows as in [4, Proposition 9.3.2, Proposition 9.3.5]. Corollary 4.7.
Let ˆ ω ∈ C ( R d ; R d ) , ν , ν ∈ P c ( R d ) , ρ ∈ Γ( ν , ν ) and define ν s as in (4.2) for s ∈ [0 , . If we set ξ s = Z R d ˆ ωdν s , (4.6) then k ξ s − (1 − s ) ξ − sξ k ≤ Cs (1 − s ) W ( ν , ν ) , for all s ∈ [0 , , where C is independent of ν and ν .Proof. Follows from Lemma 4.6 applied first to the functions ˆ
F ≡
G ≡ ˆ ω , and then to ˆ F ≡
G ≡ − ˆ ω .The semiconvexity of ˆ H c will be deduced as a corollary of the following estimate. Lemma 4.8.
Suppose that ˆ ℓ ∈ C ( R d × R d ; R ) , let z , z ∈ K and set z s = (1 − s ) z + sz for all s ∈ [0 , . Furthermore, let ν , ν ∈ P c ( R d ) , ρ ∈ Γ( ν , ν ) and define ν s and ξ s as in (4.2) and (4.6) for s ∈ [0 , . Then, for all s ∈ [0 , , it holds ˆ ℓ ( z s , ξ s ) ≤ (1 − s )ˆ ℓ ( z , ξ ) + s ˆ ℓ ( z , ξ ) + C K , ˆ ℓ, ˆ ω s (1 − s ) W ( ν , ν ) + C K , ˆ ℓ, ˆ ω s (1 − s ) k z − z k , for some constant C K , ˆ ℓ, ˆ ω depending only on K , ˆ ℓ and ˆ ω .Proof. Since K is compact, z s ∈ K for all s ∈ [0 , − s ) ξ + sξ ∈ K ′ for all s ∈ [0 , K ′ ⊂ R d . Notice that from (4.5) followsˆ ℓ ( z s , (1 − s ) ξ + sξ ) ≤ (1 − s )ˆ ℓ ( z , ξ ) + s ˆ ℓ ( z , ξ ) + C K , K ′ s (1 − s ) (cid:0) k z − z k + k ξ − ξ k (cid:1) , (4.7)and from the definition of ξ s and Jensen’s inequality, we get k ξ − ξ k ≤ Lip K ( ω ) W ( ν , ν ) ≤ Lip K ( ω ) W ( ν , ν ) . (4.8)Moreover, for every s ∈ [0 ,
1] it holds k ˆ ℓ ( z s , ξ s ) − ˆ ℓ ( z s , (1 − s ) ξ + sξ ) k ≤ Lip
K×K ′ k ξ s − (1 − s ) ξ − sξ k≤ Lip
K×K ′ s (1 − s ) C W ( ν , ν ) . (4.9)Hence, for every s ∈ [0 , ℓ ( z s , ξ s ) = ˆ ℓ ( z s , ξ s ) − ˆ ℓ ( z s , (1 − s ) ξ + sξ ) + ˆ ℓ ( z s , (1 − s ) ξ + sξ ) ≤ (1 − s )ˆ ℓ ( z , ξ ) + s ˆ ℓ ( z , ξ ) + C K , ˆ ℓ, ˆ ω s (1 − s ) W ( ν , ν ) + C K , ˆ ℓ, ˆ ω s (1 − s ) k z − z k . orollary 4.9. Let ν , ν ∈ P c ( R d ) and ρ ∈ Γ o ( ν , ν ) . Then, there exists C ∈ R independent of ν and ν for which ˆ H c (((1 − s ) π + sπ ) ρ ) ≤ (1 − s ) ˆ H c ( ν ) + s ˆ H c ( ν ) + Cs (1 − s ) W ( ν , ν ) holds for every s ∈ [0 , .Proof. Notice that, by Lemma 4.4, ˆ H c ( ν s ) can be rewritten asˆ H c ( ν s ) = Z K×K ˆ ℓ ( z s , ξ s ) dρ ( z , z ) , Furthermore, since ρ ∈ Γ o ( ν , ν ) it holds Z K×K k z − z k dρ ( z , z ) = Z R d k z − z k dρ ( z , z ) = W ( ν , ν ) , the thesis follows from Lemma 4.8. Proposition 4.10.
The functional H c is semiconvex along geodesics.Proof. Follows directly from Lemma 4.6 and Corollary 4.9, by noticing that H c ( ν ) = ˆ H c ( ν ) + ˆ H c ( ν )for ˆ F = F , ˆ G = G , ˆ ℓ = − ℓ ◦ ( π , Id), ˆ ω = ω ◦ π and K = B (0 , R T ).We define the vector field ∇ ν L : R d → R d as ∇ ν L ( z ) = (cid:20) ∇ ξ ℓ ( y, π ( z ) , R ωπ ν ) + ∇ ς ℓ ( y, π ( z ) , R ωπ ν ) D ω ( π ( z ))0 (cid:21) , for every z ∈ R d . This notation is reminiscent of the fact that this vector field will eventually turnout to be the 2-Wasserstein gradient of the functional L , as it will follow from Theorem 4.12 in thecase F ≡ G ≡
0. We can thus define our candidate vector field for the Wasserstein gradient ∇ ν H c ( ν )in the case that supp( ν ) ⊆ B (0 , R T ): w := ( ∇F ) ⋆ ν + ∇G − ∇ ν L . (4.10)Notice that, by Hypotheses (H), w is a continuous function in z , and hence it is well-defined ν -a.e.. Lemma 4.11.
Let ν ∈ P c ( R d ) . Then w defined by (4.10) belongs to L pν ( R d ) for every p ∈ [1 , + ∞ ] ,and it satisfies Z R d w ( z ) · ( z − z ) dρ ( z , z ) = Z R d ( ∇F ( z − z ) + ∇G ( z ) − ∇ ν L ( z )) · ( z − z ) dρ ( z , z ) dν ( z )(4.11) for every plan ρ ∈ Γ( ν, ν ′ ) such that ν ′ ∈ P c ( R d ) .Proof. Since w is continuous, the fact that w is L pν -integrable follows the fact that ν has compact sup-port. Equation (4.11) then follows by Fubini-Tonelli and from the fact that ρ is compactly supportedtoo by Remark 4.4. Theorem 4.12.
Let ν ∈ P ( R d ) be such that supp( ν ) ⊆ B (0 , R T ) . Then ν ∈ D ( | ∂ H c | ) if and onlyif w as in (4.10) belongs to L ν ( R d ) . In this case, k w k L ν = | ∂ H c | ( ν ) , i.e., w is an element withminimal norm in ∂ H c ( ν ) . roof. We start by assuming that ν ∈ P ( R d ) satisfies | ∂ H c | ( ν ) < + ∞ and proving that this impliesthat w belongs to L ν ( R d ) and that k w k L ν ≤ | ∂ H c | ( ν ). We compute the directional derivative of H c along a direction induced by the transport map Id + ξ , where ξ is a smooth function with compactsupport such that supp(( Id + sξ ) ν ) ⊆ B (0 , R T ) for any sufficiently small s >
0. If we denote with L ( s ) = ℓ ( y, π ( z ) + s ( π ◦ ξ )( z ) , R ωd ( π ◦ ( Id + sξ )) ν ) , L ( s ) = ℓ ( y, π ( z ) , R ωd ( π ◦ ( Id + sξ )) ν ) , then the map s
7→ F (( z − z ) + s ( ξ ( z ) − ξ ( z ))) − F ( z − z ) s + G ( z + sξ ( z )) − G ( z ) s − L ( s ) − L ( s ) s − L ( s ) − L (0) s , as s → ∇F ( z − z ) · ( ξ ( z ) − ξ ( z )) + ( ∇G ( z ) − ∇ ν L ( z )) · ξ ( z ) . Since ν has compact support, the dominated convergence theorem, the identity (4.11) and since ∇F is odd, it holds+ ∞ > lim s → H c (( Id + sξ ) ν ) − H c ( ν ) s = 12 Z R d ∇F ( z − z ) · ( ξ ( z ) − ξ ( z )) dν ( z ) dν ( z ) + Z R d ( ∇G ( z ) − ∇ ν L ( z )) · ξ ( z ) dν ( z )= Z R d w ( z ) · ξ ( z ) dν ( z ) . From the last inequality, the assumption that | ∂ H c | ( ν ) < + ∞ and using the trivial estimate W (( Id + sξ ) ν, ν ) ≤ s k ξ k L ν , we get Z R d w ( z ) · ξ ( z ) dν ( z ) ≤ | ∂ H c | ( ν ) k ξ k L ν , and hence, changing the sign of ξ , (cid:12)(cid:12)(cid:12)(cid:12)Z R d w ( z ) · ξ ( z ) dν ( z ) (cid:12)(cid:12)(cid:12)(cid:12) ≤ | ∂ H c | ( ν ) k ξ k L ν . This proves that w ∈ L ν ( R d ) and that k w k L ν ≤ | ∂ H c | ( ν ).We now prove that if the vector w belongs to L ν ( R d ), then it is in the subdifferential of H c ; thisshall imply, by (4.1), that w ∈ D ( | ∂ H c | ) and that it is a minimal selection ∂ H c ( ν ), by the previousestimate and Proposition 4.2.We thus consider a test measure ν , a plan ρ ∈ Γ o ( ν, ν ), and we compute the directional derivativeof H c along the direction induced by ρ . Denoting with L ( s ) = ℓ ( y, (1 − s ) z + sz , R ωd ((1 − s ) π + sπ ) ρ ) , L ( s ) = ℓ ( y, z , R ωd ((1 − s ) π + sπ ) ρ ) , s ∈ [0 , s
7→ F ((1 − s )( z − z ) + s ( z − z )) − F ( z − z ) s + G ((1 − s ) z + sz ) − G ( z ) s − L ( s ) − L ( s ) s − L ( s ) − L (0) s , as s → ∇F ( z − z ) · (( z − z ) − ( z − z )) + ( ∇G ( z ) − ∇ ν L ( z )) · ( z − z ) . Hence, from Proposition 4.10, the dominated convergence theorem, the identity (4.11) and since ∇F is odd, we get H c ( ν ) − H c ( ν ) ≥ lim s → H c (((1 − s ) π + sπ ) ρ ) − H c ( ν ) s + o ( W ( ν, ν ))= 12 Z R d ∇F ( z − z ) · (( z − z ) − ( z − z )) dρ ( z , z ) dρ ( z , z )+ Z R d ( ∇G ( z ) − ∇ ν L ( z )) · ( z − z ) dρ ( z , z ) + o ( W ( ν, ν ))= Z R d w ( z ) · ( z − z ) dρ ( z , z ) + o ( W ( ν, ν )) . We have thus proven that w ∈ ∂ H c ( ν ). In this section, we prove Theorem 1.1. We first recall that we already proved in Corollary 2.15 thatthere exists a mean-field optimal control for Problem 1. We now want to prove that all mean-fieldoptimal controls are solutions of the extended PMP.Let u ∗ be a mean-field optimal control for Problem 1 with initial datum ( y , µ ). Fix µ N as inDefinition 1.4–( i ), and consider a sequence ( u ∗ N ) N ∈ N of optimal controls of Problem 2 with initialdatum ( y , µ N ), having a subsequence (which, for simplicity, we do not relabel) weakly convergingto u ∗ in L ([0 , T ]; U ). Denote with ( y ∗ N , x ∗ N ) the trajectory of (1.8) corresponding to the control u ∗ N and the initial datum ( y , µ N ) of Problem 2. Compute the corresponding pair control-trajectory( u ∗ N , ( y ∗ N , q ∗ N , x ∗ N , p ∗ N )) satisfying the PMP for Problem 2, that exists due to Theorem 3.2. Set ν ∗ N :=Π N ( x ∗ N , p ∗ N ) and r ∗ N := N p ∗ N . By Proposition 3.5, the trajectories ( y ∗ N , q ∗ N , ν ∗ N ) are equibounded andequi-Lipschitz from [0 , T ] to the product space Y = R dm ×P ( R d ) endowed with the distance eqrefe-Y, and the empirical measures ν ∗ N have equibounded support. Moreover, the pair ( u ∗ N , ( y ∗ N , q ∗ N , ν ∗ N ))satisfies the extended PMP by Proposition 3.6.By the Ascoli-Arzel`a theorem, we have that there exists a subsequence, which we denote againwith ( y ∗ N , q ∗ N , ν ∗ N ), that converges to ( y ∗ , q ∗ , ν ∗ ) : [0 , T ] → R dm × P ( R d ) uniformly with respect to t ∈ [0 , T ]. Since by definition π ν ∗ N = µ ∗ N , by the convergence of µ ∗ N to µ ∗ proved in Proposition2.6, we get π ν ∗ = µ ∗ . Observe that ( y ∗ , q ∗ , ν ∗ ) is a Lipschitz function with respect to time and ν ∗ has support contained in B (0 , R T ) for all t ∈ [0 , T ]. Moreover, by the boundary conditions for each N , we have that y ∗ (0) = y , π ( ν ∗ (0)) = µ and q ∗ ( T ) = 0, π ( ν ∗ ( T ))( r ) = δ ( r ).Fix now t ∈ [0 , T ]. To shorten notation, let E : R dm × R D → R be the functional, strictly concavewith respect to u , defined as E ( q, u ) = m X k =1 q k · B k u − γ ( u ) . u ∗ N ( t ) satisfies u ∗ N ( t ) = arg max u ∈U E ( q ∗ N ( t ) , u ) , since the maximum is uniquely determined by strict concavity. Since U is bounded, by definition E ( · , u ) is continuous uniformly with respect to u ∈ U . The convergence of q ∗ N ( t ) to q ∗ ( t ) then impliesthat every accumulation point v t ∈ U of u ∗ N ( t ) must satisfy v t = arg max u ∈U E ( q ∗ ( t ) , u ) (5.1)and is therefore uniquely determined. This shows that the sequence u ∗ N is pointwise converging in[0 , T ] to the function v ( t ) := v t . Due to the boundedness of U , we further have that u ∗ N → v in L ((0 , T ); U ). Since u ∗ N was already converging to u ∗ weakly in L ((0 , T ); U ) it must be u ∗ ( t ) = v ( t )for a.e. t ∈ (0 , T ), which together with (5.1) implies that u ∗ N → u ∗ strongly in L ((0 , T ); U ) (5.2)and that u ∗ ( t ) = arg max u ∈U E ( q ∗ ( t ) , u )for a.e. t ∈ [0 , T ]. Due to the explicit expression of H ( y, q, ν, u ) in (1.4), this is equivalent to say that H ( y ∗ ( t ) , q ∗ ( t ) , ν ∗ ( t ) , u ∗ ( t )) = arg max u ∈U H ( y ∗ ( t ) , q ∗ ( t ) , ν ∗ ( t ) , u )for a.e. t ∈ [0 , T ].We finally prove that ( y ∗ , q ∗ , ν ∗ ) satisfies the Hamiltonian system (1.3) with control u ∗ . Due toequi-Lipschitz continuity, we have that the derivatives ( ˙ y ∗ N , ˙ q ∗ N ), and ∂ t ν ∗ N converge to ( ˙ y ∗ , ˙ q ∗ ), and ∂ t ν ∗ , respectively, weakly in L ([0 , T ]; R md ) and in the sense of distributions. Observe now that by(1.5) and (1.6) the vector field ∇ ν H c ( y, q, ν )( · , · ), which is independent of u , is continuously dependingon ( y, q, ν ). By the uniform convergence of ( y ∗ N , q ∗ N , ν ∗ N ) and since supp( ν ∗ N ( t )) ⊂ B (0 , R T ) for all t ∈ [0 , T ] we get that ∇ ν H c ( y ∗ N ( t ) , q ∗ N ( t ) , ν ∗ N ( t ))( x, r ) ⇒ ∇ ν H c ( y ∗ ( t ) , q ∗ ( t ) , ν ∗ ( t ))( x, r )uniformly with respect to t ∈ [0 , T ] and ( x, r ) ∈ B (0 , R T ). From this, using again the narrowconvergence of ν ∗ N ( t ) to ν ∗ ( t ) and since supp( ν ∗ N ( t )) ⊂ B (0 , R T ), we then get the uniform bound k ( J ∇ ν H c ( y ∗ N ( t ) , q ∗ N ( t ) , ν ∗ N ( t ))) ν ∗ N ( t ) k M b ( R D , R D ) ≤ C T , for some constant C T independent of t ∈ [0 , T ], as well as the narrow convergence( J ∇ ν H c ( y ∗ N ( t ) , q ∗ N ( t ) , ν ∗ N ( t ))) ν ∗ N ( t ) ⇀ ( J ∇ ν H c ( y ∗ ( t ) , q ∗ ( t ) , ν ∗ ( t ))) ν ∗ ( t )for all t ∈ [0 , T ]. Testing with functions φ ∈ C ∞ c ([0 , T ] × R d ; R ), the two above properties are enoughto show that ∇ ( x,r ) · (( J ∇ ν H c ( y ∗ N ( t ) , q ∗ N ( t ) , ν ∗ N ( t ))) ν ∗ N ( t )) ⇀ ∇ ( x,r ) · (( J ∇ ν H c ( y ∗ ( t ) , q ∗ ( t ) , ν ∗ ( t ))) ν ∗ ( t ))in the sense of distributions, so that ν ∗ solves the third equation in (1.3).For all k = 1 , . . . , m , taking derivatives in the explicit expression in (1.4) and using the definitionof H c , we have that ∇ y k H c ( y, q, ν, u ) is actually independent of u and is continuous with respect to30he Euclidean convergence on ( y, q ) and the narrow convergence on measures ν with compact supportin a fixed ball B (0 , R T ). Therefore, since ( y ∗ N , q ∗ N , ν ∗ N ) converges to ( y ∗ , q ∗ , ν ∗ ) uniformly with respectto t ∈ [0 , T ], and there is no dependence on u , for all k = 1 , . . . , m we have that ∇ y k H c ( y ∗ N ( t ) , q ∗ N ( t ) , ν ∗ N ( t ) , u ∗ N ( t )) → ∇ y k H c ( y ∗ ( t ) , q ∗ ( t ) , ν ∗ ( t ) , u ∗ ( t ))in R d uniformly with respect to t ∈ [0 , T ]. It then follows that q ∗ solves the second equation in (1.3).A similar argument, also using the L convergence of u ∗ N to u ∗ proved in (5.2), shows that ∇ q k H c ( y ∗ N ( t ) , q ∗ N ( t ) , ν ∗ N ( t ) , u ∗ N ( t )) → ∇ q k H c ( y ∗ ( t ) , q ∗ ( t ) , ν ∗ ( t ) , u ∗ ( t ))in L ([0 , T ]; R d ) for all k = 1 , . . . , m , so that y ∗ solves the first equation in (1.3). This concludes theproof of Theorem 1.1. In this section, we show the application of the extended Pontryagin Maximum Principle to a toymodel for crowd interactions. The Cucker-Smale model, introduced in [18], was first studied in itsmean-field limit form in [24]. It models the phenomenon of alignment of velocities in crowds, that canbe observed, e.g., in flocks of birds.In this model, each agent is identified by its position x i and velocity v i , and it adjusts its velocityby relaxing it towards a weighted mean of the velocities of the group. The weight is a nonincreasingfunction φ of the distance between individuals. In the original paper [18], the authors propose φ ( λ ) = K ( σ + λ ) β , for some fixed parameters K, σ > β ≥
0. For our computations, we consider φ ∈C ( R d , R + ) being a radial function.The finite-dimensional dynamics is given by the ODE system ˙ x i = v i , ˙ v i = 1 N N X j =1 φ ( x i − x j )( v j − v i ) , i = 1 , . . . , N. We add to it m leaders with positions and velocities given by ( y k , w k ) for every k = 1 , . . . , m , onwhich a control variable u k is active. Since the control acts as an external force, u k will directly affectthe evolution of the velocities w k only. The mean-field limit for N → + ∞ of the resulting system isgiven by (see, e.g., [22]) ˙ y k = w k , ˙ w k = (Φ ⋆ µ )( y k , w k ) + 1 m m X j =1 φ ( y k − y j )( w j − w k ) + u k , k = 1 , . . . , m∂ t µ = − v · ∇ x µ − ∇ v · Φ ⋆ µ + 1 m m X j =1 φ ( x − y j )( w j − v ) µ , (6.1)where µ = µ ( x, v ) is the density of followers and Φ( x, v ) := φ ( x )( − v ). Notice, this is a particularcase of (1.2), where the state variables for the leaders are y k := (cid:18) y k w k (cid:19) and y = ( y , . . . , y m ), the31nes for the followers are x = (cid:18) xv (cid:19) , and one chooses K ( x ) := (cid:18) x ) (cid:19) , f k ( y ) = (cid:18) w k m P mj =1 φ ( y k − y j )( w j − w k ) (cid:19) ,g ( y )( x ) = (cid:18) v m P mj =1 φ ( x − y j )( w j − v ) (cid:19) , and, for every k = 1 , . . . , m , B k is the 2 d × ( dm ) matrix that maps u = u . . .u m ∈ R dm into theelement (cid:18) u k (cid:19) ∈ R d . Notice that since φ is a radial function, the function Φ , and thus K , is odd.A standard problem in the study of the Cucker-Smale model is to find conditions to ensure flocking,i.e., alignment of the whole crowd to the same velocity. For this reason, it is interesting in our case tostudy the minimization of the variance of the crowd, by choosing L ( y , µ ) := Z R d m m X k =1 k w k k + 2 k v k ! dµ ( x, v ) − (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) m m X k =1 w k + Z v dµ ( x, v ) (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) = Z R d m m X k =1 k w k k + 2 k v k − m m X k =1 w k + Z v ′ dµ ( x ′ , v ′ ) ! · m m X k =1 w k + v !! dµ ( x, v ) , (6.2)that is of the form L = R R d ℓ ( y , x , R ωµ ) dµ ( x ) by choosing ω ( x ) = v and ℓ ( y , x , ς ) = 2 m m X k =1 k w k k + 2 k v k − m m X k =1 w k + ς ! · m m X k =1 w k + v ! . For the control constraints, we assume U := [ − , dm and we choose to penalize the L -norm of thecontrol, hence γ ( u ) := k u k . Remark 6.1.
Other forms for the cost L can be of interest. For example, on may want to drive thecrowd to a given fixed velocity ¯ v . In this case, one can minimize L ( y , µ ) := Z R d m m X k =1 k w k − ¯ v k + 12 k v − ¯ v k ! dµ ( x, v ) , that is again of the form R R d ℓ ( y , x , R ωµ ) dµ ( x ), with ℓ not depending on its third variable, this time.Since Hypotheses (H) are clearly satisfied, we now apply the extended Pontryagin MaximumPrinciple to the optimal control problem with cost functional (6.2) constrained by the system (6.1).For simplicity of notation, we study the 1-dimensional problem, i.e., d = 1. We introduce the dualvariables of y k and x denoted by q k = ( q k , z k ) and r = ( r, s ), respectively. The Hamiltonian H in For simplicity of computation, we consider minimization of 4 times the variance. H ( y , q , ν, u ) = 12 Z R ( s − s ′ ) φ ( x − x ′ )( v ′ − v ) dν ( x ′ , v ′ , r ′ , s ′ ) dν ( x, v, r, s )+ Z R rv + s m m X j =1 φ ( x − y j )( w j − v ) dν ( x, v, r, s )+ m X k =1 (cid:18) q k w k + z k Z R φ ( y k − x )( v − w k ) dν ( x, v, r, s ) (cid:19) + m X k =1 z k m m X j =1 φ ( y k − y j )( w j − w k ) + z k u k − Z R ℓ ( y , x , R ωµ ) dµ ( x ) − | u | . The optimal control can be explicitly computed by (3.2) as follows u ∗ k ( y , q ) = ( z k if z k ∈ [ − , , sign( z k ) otherwise . Denoting with µ the first marginal of ν , the PMP dynamics of the state and adjoint variables is givenby ˙ y k = w k , ˙ w k = (Φ ⋆ µ )( y k , w k ) + m P mk =1 φ ( y k − y j )( v j − v k ) + u ∗ k ( y , q ) , ˙ q k = m R R sφ ′ ( x − y k )( w k − v ) dν ( x, v, r, s ) − z k R R φ ′ ( y k − x )( v − w k ) dν ( x, v, r, s ) − m P j = k z j φ ′ ( y k − y j )( w j − w k ) , ˙ z k = − R R (cid:0) m s − z k (cid:1) φ ( x − y k ) dν ( x, v, r, s ) − q k + P j = k z j φ ( y k − y j ) − m w k + m P mj =1 w j + m R R v dµ ( x, v ) ,∂ t ν = −∇ ( x,v,r,s ) · (( J ∇ ν H c ( y , q , ν, u ∗ )) ν ) , where the components of the vector field ∇ ν H c ( y , q , ν, u ∗ ) are given at every point ( x, v, r, s ) ∈ R by ∇ ν H c · e = Z R ( s − s ′ ) φ ′ ( x − x ′ )( v ′ − v ) dν ( x ′ , v ′ , r ′ , s ′ ) + s m m X j =1 φ ′ ( x − y j )( w j − v ) − m X k =1 z k φ ′ ( y k − x )( v − w k ) , ∇ ν H c · e = − Z R ( s − s ′ ) φ ( x − x ′ ) dν ( x ′ , v ′ , r ′ , s ′ ) + r − s m m X j =1 φ ( x − y j ) + m X k =1 z k φ ( y k − x ) − v + 2 m m X k =1 w k + Z R vdµ ( x, v ) , ∇ ν H c · e = v, ∇ ν H c · e = (Φ ⋆ µ )( x, v ) + 1 m m X j =1 φ ( x − y j )( w j − v ) .
33e remark that, as it happens for the standard PMP, the explicit computation of the third andthe fourth components gives exactly the vector field determining the dynamics of µ , the first marginalof ν , in accordance with (6.1). Acknowledgement
The authors acknowledge the support of the PHC-PROCOPE Project “Sparse Control of MultiscaleModels of Collective Motion”. Mattia Bongini and Massimo Fornasier additionally acknowledge thesupport of the ERC-Starting Grant Project “High-Dimensional Sparse Optimal Control”.
References [1] G. Albi, M. Bongini, E. Cristiani, and D. Kalise. Invisible sparse control of self-organizing agentsleaving unknown environments.
Submitted , 2015.[2] L. Ambrosio, N. Fusco, and D. Pallara.
Functions of Bounded Variation and Free DiscontinuityProblems. , volume 254. Clarendon Press Oxford, 2000.[3] L. Ambrosio and W. Gangbo. Hamiltonian ODEs in the Wasserstein space of probability mea-sures.
Comm. Pure Appl. Math. , 61(1):18–53, 2008.[4] L. Ambrosio, N. Gigli, and G. Savar´e.
Gradient Flows in Metric Spaces and in the Space ofProbability Measures . Lectures in Mathematics ETH Z¨urich. Birkh¨auser Verlag, Basel, secondedition, 2008.[5] D. Andersson and B. Djehiche. A maximum principle for SDEs of mean-field type.
Appl. Math.Comput. , 63(3):341–356, 2011.[6] A. Bensoussan, J. Frehse, and P. Yam.
Mean Field Games and Mean Field Type Control Theory .Springer, 2013.[7] M. Bongini and M. Fornasier. Sparse stabilization of dynamical systems driven by attraction andavoidance forces.
Netw. Heterog. Media , 9(1):1–31, 2014.[8] M. Bongini, M. Fornasier, F. Fr¨olich, and L. Hagverdi. Sparse control of force field dynamics. In
International Conference on NETwork Games, COntrol and OPtimization , October 2014.[9] M. Bongini, M. Fornasier, O. Junge, and B. Scharf. Sparse control of alignment models in highdimension.
To appear in Netw. Heterog. Media , 2014. Preprint at arXiv:1408.6362.[10] M. Bongini, M. Fornasier, and D. Kalise. (Un)conditional consensus emergence under feedbackcontrols.
To appear in Discrete Contin. Dyn. Syst. , 35(9), 2015. Preprint at arXiv:1502.06100.[11] M. Burger, M. Di Francesco, P. Markowich, and M.-T. Wolfram. Mean field games with nonlinearmobilities in pedestrian dynamics.
Discrete Contin. Dyn. Syst. Ser. B , 19:1311–1333, 2014.[12] G. Buttazzo and G. D. Maso. Γ-convergence and optimal control problems.
J. Optim. TheoryAppl. , 38(3):385–407, 1982.[13] M. Caponigro, M. Fornasier, B. Piccoli, and E. Tr´elat. Sparse stabilization and control of align-ment models.
Math. Models Methods Appl. Sci. , 25(03):521–564, 2015.3414] R. Carmona, F. Delarue, and A. Lachapelle. Control of McKean–Vlasov dynamics versus meanfield games.
Math. Financ. Econ. , 7(2):131–166, 2013.[15] F. Clarke.
Functional Analysis, Calculus of Variations and Optimal Control , volume 264.Springer, 2013.[16] C. Clason and K. Kunisch. A duality-based approach to elliptic control problems in non-reflexiveBanach spaces.
ESAIM Control Optim. Calc. Var. , 17(01):243–266, 2011.[17] F. Cucker and J.-G. Dong. A general collision-avoiding flocking framework.
IEEE Trans. Automat.Control , 56(5):1124–1129, 2011.[18] F. Cucker and S. Smale. Emergent behavior in flocks.
IEEE Trans. Automat. Control , 52(5):852–862, 2007.[19] M. Cullen, W. Gangbo, and G. Pisante. The semigeostrophic equations discretized in referenceand dual variables.
Archive for Rational Mechanics and Analysis , 185(2):341–363, 2007.[20] G. Dal Maso.
An Introduction to Γ -Convergence . Progr. Nonlinear Differential Equations Appl.Birkh¨auser Boston Inc., Boston, MA, 1993.[21] M. R. D’Orsogna, Y.-L. Chuang, A. L. Bertozzi, and L. S. Chayes. Self-propelled particles withsoft-core interactions: patterns, stability, and collapse. Phys. Rev. Lett. , 96(10):104302, 2006.[22] M. Fornasier, B. Piccoli, and F. Rossi. Mean-field sparse optimal control.
Philos. Trans. R. Soc.Lond. Ser. A Math. Phys. Eng. Sci. , 372(2028):20130400, 2014.[23] M. Fornasier and F. Solombrino. Mean-field optimal control.
ESAIM Control Optim. Calc. Var. ,20(4):1123–1152, 2014.[24] S.-Y. Ha and E. Tadmor. From particle to kinetic and hydrodynamic description of flocking.
Kinet. Relat. Models , 1(3):415–435, 2008.[25] D. Helbing, I. Farkas, and T. Vicsek. Simulating dynamical features of escape panic.
Nature ,407(6803):487–490, 2000.[26] M. Huang, P. E. Caines, and R. P. Malham´e. Individual and mass behaviour in large populationstochastic wireless power control problems: centralized and nash equilibrium solutions. In , volume 1, pages 98–103. IEEE, 2003.[27] R. L. Hughes. A continuum theory for the flow of pedestrians.
Transport. Res. B Meth. , 36(6):507–535, 2002.[28] D. Kinderlehrer and G. Stampacchia.
An Introduction to Variational Inequalities and their Ap-plications . Academic Press, New York, NY, 1980.[29] J.-M. Lasry and P.-L. Lions. Mean field games.
Jpn. J. Math. , 2(1):229–260, 2007.[30] S. Motsch and E. Tadmor. Heterophilious dynamics enhances consensus.
SIAM Rev. , 56(4):577–621, 2014.[31] B. Piccoli and F. Rossi. Transport equation with nonlocal velocity in Wasserstein spaces: Con-vergence of numerical schemes.
Acta Appl. Math. , 124(1):73–105, 2013.3532] B. Piccoli and F. Rossi. Generalized Wasserstein distance and its application to transport equa-tions with source.
Arch. Ration. Mech. Anal. , 211(1):335–358, 2014.[33] B. Piccoli, F. Rossi, and E. Tr´elat. Control to flocking of the kinetic cucker-smale model.
Sub-mitted , 2014. Preprint at arXiv:1411.4687.[34] K. Pieper and B. Vexler. A priori error analysis for discretization of sparse elliptic optimal controlproblems in measure space.
SIAM J. Control Optim. , 51(4):2788–2808, 2013.[35] Y. Privat, E. Tr´elat, and E. Zuazua. Optimal location of controllers for the one-dimensional waveequation.
Ann. Inst. H. Poincar´e Anal. Non Lin´eaire , 30(6):1097–1126, 2013.[36] J. P. Raymond and H. Zidani. Hamiltonian Pontryagin’s principles for control problems governedby semilinear parabolic equations.
Appl. Math. Optim. , 39(2):143–177, 1999.[37] C. W. Reynolds. Flocks, herds and schools: A distributed behavioral model.
ACM SIGGRAPHComputer Graphics , 21(4):25–34, 1987.[38] G. Stadler. Elliptic optimal control problems with L -control cost and applications for theplacement of control devices. Comput. Optim. Appl. , 44(2):159–181, 2009.[39] T. Vicsek, A. Czir´ok, E. Ben-Jacob, I. Cohen, and O. Shochet. Novel type of phase transition ina system of self-driven particles.
Phys. Rev. Lett. , 75(6):1226, 1995.[40] C. Villani.
Topics in Optimal Transportation , volume 58 of