Monotone solutions for mean field games master equations : finite state space and optimal stopping
aa r X i v : . [ m a t h . A P ] J u l MONOTONE SOLUTIONS FOR MEAN FIELD GAMES MASTEREQUATIONS : FINITE STATE SPACE AND OPTIMALSTOPPING
CHARLES BERTUCCI Abstract.
We present a new notion of solution for mean field games masterequations. This notion allows us to work with solutions which are merely con-tinuous. We prove first results of uniqueness and stability for such solutions. Itturns out that this notion is helpful to characterize the value function of meanfield games of optimal stopping or impulse control and this is the topic of thesecond half of this paper. The notion of solution we introduce is only useful inthe monotone case. We focus in this paper in the finite state space case.
Contents
Introduction 2General introduction 2Regularity of the solution of the master equation 3Mean field games with optimal stopping or singular control 3A comment on modeling 4Structure of the paper 4Notations 41. Preliminary results 51.1. Uniqueness results for the master equation in finite state space 71.2. Existence results for the master equation in finite state space 81.3. Stegall’s variational principle 102. Monotone solutions of the master equation 112.1. The stationary case 112.2. The time dependent case 152.3. A generalization of this method 173. The master equation for mean field games with optimal stopping 183.1. The penalized master equation 183.2. Results on the stationary penalized equation 193.3. The limit master equation 223.4. Comments on this notion of solution in the optimal stopping case 233.5. The time dependent case 24 : CMAP, Ecole Polytechnique, UMR 7641, 91120 Palaiseau, France . . The master equation for mean field games of impulse control 254.1. Description of the model 254.2. The penalized master equation and monotone solutions of the limitgame 264.3. Results for particular mean field games of impulse control 285. A mean field game of entry and exit 296. Conclusion and future perspectives 31Acknowledgments 31References 31Appendix A. Two maximum principle results 33Appendix B. Finite state MFG on the orthant 34 Introduction
In some sense, this paper is the fourth of a series devoted to the systematic studyof mean field games (MFG for short) of optimal stopping or impulse control. Herewe study the master equation associated with optimal stopping or impulse controlin finite state space. In order to do so, we introduce a new notion of solutionfor the master equation which is of interest outside the cases of optimal stoppingor impulse controls. In [5], by considering Nash equilibria of such games withoutcommon noise, we showed that those equilibria are in general in mixed strategiesand translated this statement in terms of the system of partial differential equations(PDE) characterizing them. In [6], we extended this notion to the case of impulsecontrol. In [8] we presented numerical methods for such problems.
General introduction.
The MFG theory is concerned with the study of gamesinvolving an infinite number of non-atomic players interacting through mean fieldterms. If such games have been studied in Economics for quite some time, a generalmathematical framework has only been developed around fifteen years ago by J.-M.Lasry and P.-L. Lions. It is presented in [26, 27]. This theory has known too manydevelopments for us to present them all but we are going to indicate some of them.For the moment, uniqueness of Nash equilibria has been proven almost exclusivelyin two cases, either under a smallness condition (on the coupling between theplayers or on the duration of the game) or in the so-called monotone regime, see[26, 27, 32]. In this monotone regime, the study of the Nash equilibria of thegame reduces to the study of the master equation, a PDE which is satisfied by thevalue function of a player seen as depending on its own state and on the measuredescribing the states of the other players. As soon as the state space is infinite,the master equation is an infinite dimensional PDE [16], whereas in the finite statespace case, this PDE reduces to a system of finite dimensional PDE [27]. In thecase in which the game is deterministic or when the randomness is distributed inan i.i.d. fashion among the players, Nash equilibria can be characterized with a ystem of finite dimensional forward and backward equations [26, 16]. Several otheraspects of MFG have been studied, such as their long time average [13, 14] and theconvergence of the N player game toward the MFG [16, 24]. Let us also recall thata probabilistic approach of MFG have been developed, we refer to [18, 23] for moredetails on this approach. Finally let us mention that numerical methods have beendeveloped to compute equilibria of MFG, mostly in the forward-backward setting,more details can be found in [1, 2, 12]. In recent years, the study of MFG ofoptimal stopping or other ”singular” controls have been the subject of a growingnumber of researches, namely because such games have natural applications inEconomy. Concerning the case of optimal stopping, let us mention [5, 17, 29, 30],for impulse control we refer to [6] and to [22] for optimal switching. Let us alsomention that the approach of [5, 6] has been used by P.-L. Lions in [28] to studya case of MFG of singular controls. Regularity of the solution of the master equation.
In the monotone regime,it is known since the work of J.-M. Lasry and P.-L. Lions that we can define avalue function for a MFG. This is a consequence of a uniqueness property ofNash equilibria of the game. Let us recall that, formally, the monotone regimeis a situation in which the players have a tendency to repel each other. Preciseassumptions on the monotonicity shall be made later on.If it is smooth, this value function satisfies the master equation. However thesmoothness of this value function may be difficult to prove in general [16] and forthe moment no general weak notion of solution for the master equation has beenestablished. We provide in this paper a notion of solution which demands onlyfor the value function to be continuous. We refer to these solutions as monotonesolutions since our approach relies heavily on the monotonicity of the MFG. Letus mention that in this paper we present this solutions in the finite state spacecase, but that this approach can be extended. In fact the case of a continuousstate space shall be treated in [7].
Mean field games with optimal stopping or singular control.
An objectiveof this paper is to study the master equation, in finite state space, associatedto optimal stopping or impulse controls. In those situations, the players can,respectively, decide to exit the game or to change instantly their state to anyother state. The main difficulty arising from the study of those games is that theevolution of a population of players using such controls is not smooth in general,independently of any regularity assumptions. This fact, already presented in [5, 6],is a crucial obstacle to overcome. Indeed as the evolution of the population ofplayer is not smooth, the formulation of the associated master equation has totake into account the fact that this measure can instantly jump from one state toanother. As it is often the case in the optimal control literature, we shall see thatthis singular behavior for the evolution of the density of players does not translateinto a loss of regularity for the value function. We believe that, in some sense, his is reminiscent of the problem of Hamilton-Jacobi equations associated withsingular controls as in [25]. A comment on modeling.
Let us comment on a modeling choice we make inthis paper. We intend to look at the master equation set on the whole ( R + ) d .In the continuous state space case, in the setting of [16] for instance, this wouldcorrespond to look at the master equation posed on the set of positive measuresinstead that on the set of probability measure. In the appendix we explain howwe can pass from one setting to another in finite state space. This choice to workon ( R + ) d is clearly motivated in the optimal stopping case as the mass of playersis not constant (it can decrease). However, we argue that this kind of approachis also meaningful in general. Indeed, in a MFG the number of player is infiniteand the choice to normalize their mass to one is only an arbitrary choice. If ina lot of cases studied in the literature the mass of player is preserved, it is alsonatural to consider, outside of stopping or entry game, MFG in which playersleave or enter the game. For instance let us refer to [19]. Moreover, in the casein which the mass of player is preserved, solving the master equation for anyinitial mass of player is a suitable strategy. To conclude this word on modeling,let us mention that to have the same point of view in the probabilistic approachof MFG, in which the measure characterizing the distribution of players is oftenthought as the probability measure of the state of one player, one need to considereither unnormailzed measures for the state of a player or either an additional realparameter which stands for the mass of players. Structure of the paper.
The rest of this paper is organized as follows. In section1, we present some preliminary results concerning the study of master equations inthe finite state space case, in particular for situations in which there is a boundary.In section 2, we present our notion of monotone solutions. In section 3, we derive acharacterization for the value function in the optimal stopping case, as well as weprove existence and uniqueness for such value functions. We present in section 4the master equation associated with an impulse control problem. Finally section 5is concerned with a MFG model of entry and exit in a simplified economic marketwhich applies the concepts developed in the previous sections.
Notations.
We shall use the following notations. • We denote by d an integer greater than 1. • O d = ( R + ) d . • The euclidean norm of R d is denoted | · | and its associated operator norm k · k . The euclidean scalar product between x, y ∈ R d is denoted either x · y or h x, y i . • An application A : O → R d , defined on a subset O ⊂ R d is said to bemonotone if h A ( x ) − A ( y ) , x − y i ≥ , for x, y ∈ O . We define for R ≥ B R := { x ∈ O d , x + ... + x d ≤ R } . • For A ∈ L ( R d ), we define k A k , − := inf {h ξ, Aξ i|| ξ | ≤ } . • We denote the term by term product between matrices or vector by ∗ . • For x ∈ R d , we note x ≤ x are negative. • If f is a real function and x ∈ R d , then f ( x ) := ( f ( x ) , ..., f ( x d )).1. Preliminary results
This section introduces extensions of the results of [27] to the case in which themaster equation is posed on a domain of R d , in particular when it is ( R + ) d := O d .We also recall a weak form of Stegall’s variational lemma at the end of this section.Let us mention the work [3], which is also concerned with the study of the masterequation in finite state space. In this article the authors derived and studied aparticular form of the master equation arising from the presence of a so-calledWright-Fischer common noise in the MFG. This work is radically different thanours because they use the structure of the noise to establish regularity in a non-monotone setting while we make an extensive use of monotonicity to avoid havingto use regularity, without using the structure of the randomness.The main results of this section are concerned with existence and uniqueness ofsolutions of the master equation(1) ∂ t U + ( F ( x, U ) · ∇ x ) U + λ ( U − T ∗ U ( t, T x )) = G ( x, U ) in (0 , ∞ ) × O d ,U (0 , x ) = U ( x ) in O d , or its discounted stationary counterpart(2) rU + ( F ( x, U ) · ∇ x ) U + λ ( U − T ∗ U ( t, T x )) = G ( x, U ) in O d . where U : (0 , ∞ ) × O d → R d is the value function of the MFG which is characterizedby F, G : O d × R d → R d , λ > T ∈ L ( R d ). Let us note that these PDE arenot written on an open set and that no boundary conditions are imposed.The interpretation of (1) is that it is the PDE satisfied by the value functionof the MFG. The value of the game in the state i ∈ { , ..., d } , when the quantityof player in each state j ∈ { , ..., d } is x j and the time remaining in the game is t ≥ U i ( t, x ). The term λ ( U − T ∗ U ◦ T ) stands for the modeling of commonnoise and we refer to [9] for more details on this question. The function F standsfor the evolution of the ”density” of players and G for the evolution of the valueof the MFG. That is, in the case λ = 0, the characteristics of (1) are given by( V ( t ) , y ( t )) ≤ t ≤ t f solution of(3) ( ˙ V ( t ) = G ( y ( t ) , V ( t )) , < t < t f , V (0) = U ( y (0)) , ˙ y ( t ) = F ( y ( t ) , V ( t )) , < t < t f , y ( t f ) = y , or any y ∈ O d , t f ≥
0. The same kind of interpretation holds for (2) and we donot detail it here. Let us only insist on the obvious fact that the characteristicsassociated to (2) are set on an infinite time scale and that despite the fact theequation (2) is stationary, the evolution of the density of players is not trivial.Because no boundary conditions are imposed on the equation, we have to restrictthe set of functions F which leave invariant O d as well as transformation T whichleave invariant O d . Namely we shall assume the following. Hypothesis 1.
The function F is such that for any p ∈ R d , i ∈ { , ..., d } (4) x i = 0 ⇒ F i ( x, p ) ≤ . Moreover T ( O d ) ⊂ O d . Remark 1.1.
In certain situations, we may be force to consider a coupling term F such that (4) does not hold. In such a situation, we may be able to obtain, fromother assumptions, that (5) x i = 0 ⇒ F i ( x, U ( x )) ≤ , for a solution U of the master equation, which is sufficient. We refer to [10] foran example of such a situation. This assumption clearly enforces the fact that O d is invariant for the trajectoriesgenerated by (3). We shall also made the following assumption in the stationarycase. Hypothesis 2.
There exists
R > such that for all x ∈ O d with | x | ≥ R , for all p ∈ R d , (6) d X i =1 F i ( x, p ) ≥ . Moreover, T ( B R ) ⊂ B R . This assumption states that when the mass of player is sufficiently large ( | x | ≥ R ), it cannot grow anymore. Thus it has the effect to bound the region of interestwhen starting from an initial distribution of mass. Although it seems possible totreat situations in which the mass of players can always increase, it is not theobjective of this paper.As already mentioned several times above, monotonicity plays a crucial role inthe well-posedness of (1) and (2). We say that we are in the monotone regimewhen the following assumption is satisfied. Hypothesis 3.
The functions U and ( G, F ) are monotone. In order to obtain existence of solutions of either (1) or (2), we shall use thefollowing stronger assumption. ypothesis 4. The functions U , F and G are lipschitz continuous. Moreover,there exists α > such that (7) h ξ, D x U ( x ) ξ i ≥ α | D x U ( x ) ξ | , ∀ x ∈ O d , ξ ∈ R d , (8) (cid:18) D x G ( x, p ) D x F ( x, p ) D p G ( x, p ) D p F ( x, p ) (cid:19) ≥ α (cid:18) Id
00 0 (cid:19) ∀ x ∈ O d , p ∈ R d , in the order of positive matrices. When addressing the existence of solutions of equations of the type of (2), weshall also make the following assumption.
Hypothesis 5.
The discount rate r satisfies (9) r > k D x F ( x, p ) k , − ∀ x ∈ O d , p ∈ R d . Uniqueness results for the master equation in finite state space.
Wenow present uniqueness results concerning (1) and (2). Those results are directextensions of the results established in [27] and do need any new ideas, however, asthey play a crucial role in the notion of solution we introduce in the next section,we detail their proofs.
Proposition 1.1.
Under hypotheses 1 and 3, there exists at most one smoothsolution of (1). If this solution exists, it is monotone.Proof.
Let us take U and V two smooth solutions of (1). We define W by(10) W ( t, x, y ) = h U ( t, x ) − V ( t, y ) , x − y i . This function satisfies(11) W (0 , x, y ) = h U ( x ) − U ( y ) , x − y i in O d , (12) ∂ t W + F ( x, U ) · ∇ x W + F ( y, V ) · ∇ y W + λ ( W − W ( t, T x, T y ))= h G ( x, U ) − G ( y, V ) , x − y i + h F ( x, U ) − F ( y, V ) , U − V i in (0 , ∞ ) × O d . From hypothesis 3 the right hand side of the previous two equations are positive.From lemma A.1 (in appendix), we deduce that W is positive for all time. We canconclude that i) U = V (or otherwise W should change sign around some point),ii) U is monotone for all time. (cid:3) Proposition 1.2.
Under hypotheses 1 - 3, there exists at most one smooth solutionof (2), and if it exists it is monotone.Proof.
Let us take U and V two smooth solutions of (1). We define W by(13) W ( x, y ) = h U ( x ) − V ( y ) , x − y i . his function satisfies(14) rW + F ( x, U ) ·∇ x W + F ( y, V ) · ∇ y W + λ ( W − W ( T x, T y ))= h G ( x, U ) − G ( y, V ) , x − y i + h F ( x, U ) − F ( y, V ) , U − V i in O d . We then conclude as in the previous proof by using this time lemma A.2 in ap-pendix. (cid:3)
Existence results for the master equation in finite state space.
Wenow turn to the questions of existence of solutions of (1) and (2). As the nextsection gives a precise definition of a solution of (1) and (2), we do not focus onthe sense in which solutions satisfy the PDE but rather on how we can obtain apriori estimates.In the monotone setting (i.e. under hypothesis 3), an a priori estimate on thespatial gradient of the solution can be proved, exactly as it is already the case in[27].
Proposition 1.3.
Under hypotheses 1, 3 and 4, there exists a lipschitz function U , solution of (1) almost everywhere, such that for any t f > , there exists C suchthat (15) k D x U ( t, x ) k ≤ C, ∀ x ∈ O d , t ≤ t f . Proof.
This proposition can be obtained as a consequence of an a priori estimateon the solution U of (1). This idea is mostly borrowed from the lecture [27] inwhich this technique is presented. The only difference between this lecture andour situation is that we consider a master equation on O d , that is why we are notgoing to enter in a lot of details here but only indicate the main differences withthe case in R d . For U a smooth solution of (1), let us define W and Z with(16) W ( t, x, ξ ) = h U ( t, x ) , ξ i , (17) Z β,γ ( t, x, ξ ) = h∇ x W ( t, x, ξ ) , ξ i − β ( t ) k∇ x W ( t, x, ξ ) k + γ ( t ) | ξ | , or some functions β and γ to be defined later on. Arguing as in [27], we deducethat Z β,γ satisfies(18) ∂ t Z β,γ + h F ( x, ∇ ξ W ) , ∇ x Z β,γ i + h D p F ( x, ∇ ξ W ) ∇ ξ Z β,γ , ∇ x W i − h D p G ( x, ∇ ξ W ) ∇ ξ Z β,γ , ξ i + λ ( Z β,γ − Z β,γ ( t, T x, T ξ ))= h D x G ( x, ∇ x W ) ξ, ξ i − h D p G ( x, ∇ ξ W ) ∇ x W, ξ i − h D x F ( x, ∇ ξ W ) ∇ x W, ξ i + h D p F ( x, ∇ ξ W ) ∇ x W, ∇ x W i− β h D x G ( x, ∇ x W ) ξ, ∇ x W i + 2 β h D x F ( x, ∇ ξ W ) ∇ x W, ∇ x W i + βλ (cid:18) |∇ x W | − h∇ x W ( t, T x, T ξ ) , S ∇ x W i + |∇ x W ( t, T x, T ξ ) | (cid:19) − ddt β |∇ x W | + 2 γ ( h D p F ( x, ∇ ξ W ) ξ, ∇ x W i − h D p G ( x, ∇ ξ W ) ξ, ξ i )+ ddt γ | ξ | + λγ ( | ξ | − | T ξ − e | ) . Using the monotonicity of U , G and F and the strong monotonicity of U and G ,we deduce that by choosing γ = 0 and(19) β ( t ) = αe − t (2 k∇ x F k +2 k∇ x G k + λ ( k T k− + ) , the right hand side of (18) is positive and thus that by lemma A.1, Z β,γ is positivefor all time which yields the required a priori estimate on U . Concerning thequestion of existence of a lipschitz function satisfying (1), the main argument (asin [27]) is to remark that the previous technique to obtain a priori estimates stillworks when one add particular degenerate elliptic second order terms in (1). Thisis a consequence of the so-called Bernstein method. We now indicate a particularchoice of such terms.Let us take ǫ > σ : R → R a smooth real bounded function suchthat σ ( x ) = x in a neighborhood of 0. Assume that U is a smooth solution of(20) ∂ t U i + F ( x, U ) · ∇ x U i − ǫ d X j =1 σ ( x j ) ∂ jj U i ! − ǫσ ′ ( x i ) ∂ i U i + λ ( U i − ( T ∗ ) i U ( t, T x )) = G i ( x, U ) in O d , ∀ i ∈ { , ..., d } ,U (0 , x ) = U ( x ) in O d . Let us remark that as the terms in ǫ in (20) preserves the monotonicity, we areable to adapt the previous technique to (20) to establish a, uniform in ǫ ∈ (0 , ǫ → (cid:3) e now provide an existence result for (2). As the proof of the following state-ment is very similar to the one of the previous result, we do not detail it here. Proposition 1.4.
Under hypotheses 1-5, any smooth function U solution of (2)satisfies (21) ∀ x ∈ O d , k D x U ( x ) k ≤ C, for C > depending only on r, G, F, λ and T . Remark 1.2.
Under the assumptions of the previous proposition, existence of alipschitz solution of (2) is then easy to obtain once some local boundness can beestablished for a solution of (2). Such a property can usually be obtained easilyon a case by case basis. We give such an example in the next section on optimalstopping.
Remark 1.3.
The main difference for the proof of this statement compare withthe time dependent case is that the functions β and γ in the previous proof mustbe chosen constant. Remark 1.4.
We firmly believe the restriction on r from hypothesis 5 to be mainlytechnical and due to the rather abstract framework in which we are working. Letus for instance mention [13] in which a lipschitz estimate is proved for the solutionof a first order stationary master equation for any discount rate. Stegall’s variational principle.
We end this section on preliminary resultsby a recalling a quite weak version of Stegall’s lemma [33, 34, 21] that we shall usemany times in the rest of the paper. Moreover, we present a proof of this resultthat we believe to be new.
Lemma 1.1.
Let φ : Ω → R be a weakly sequentially lower semi continuousfunction from a compact set Ω ⊂ X of a separable Hilbert space X . Then there isa dense number of points c in X such that x → φ ( x ) + h c, x i has a strict globalminimum on Ω .Proof. This proof relies on convex analysis. Let us consider the operator A : X →P ( X ) defined by A ( c ) is the set of the points at which x → φ ( x ) + h c, x i reaches itsminimum over Ω. Clearly for all c , this set is non empty and well defined. Let uscheck that the operator − A is cyclically monotone. We consider a finite sequence c , c , ..., c n = c and for all i , y i ∈ A ( c i ). For all i , let us remark that(22) φ ( y i ) + h c i , y i i ≤ φ ( y i +1 ) + h c i , y i +1 i . Rearranging we get(23) h c i , y i − y i +1 i ≤ φ ( y i +1 ) − φ ( y i ) . et us now compute n X i =1 h c i − c i − , y i i = n X i =1 h c i , y i − y i +1 i , ≤ n X i =1 φ ( y i +1 ) − φ ( y i ) , ≤ . Therefore the operator − A is cyclically monotone. Thus we deduce that A ⊂ ∂ψ where ψ is a concave function on X and ∂ψ is its super-differential. From ageneralization of Alexandrov theorem for separable Hilbert spaces [11], we finallydeduce the required result. (cid:3) Monotone solutions of the master equation
In this section we provide a notion of solution for the master equation whichdoes not require the solution to be differentiable with respect to the space variable.We define first our solutions in the easier case of the stationary master equationand then present the time dependent case.2.1.
The stationary case.
The main idea we exploit in this section is somehowcontained in the proof of propositions 1.2. Namely let us remark that for the proofof proposition 1.2 to hold, we only need to have information on the solution U of(2) at points of minima of φ V,y : x → h U ( x ) − V, x − y i for V ∈ R d , y ∈ O d . Let usnow remark that if U is smooth, so is φ V,y and(24) ∇ x φ V,y ( x ) = U ( x ) − V + D x U ( x ) · ( x − y ) . In particular, if x is a point of minimum of φ V,y in the interior of O d , then(25) D x U ( x ) · ( x − y ) = V − U ( x ) . The right hand side of (25) does not depend on derivatives of U . This leads us tounderstand how we can generalize the notion of solution of (2) for a function U which is not differentiable. Accordingly to this heuristic, we introduce the followingdefinition. Definition 2.1.
A function U ∈ C ( O d , R d ) is said to be a monotone solution of(2) if for any V ∈ R d , y ∈ O d , R > sufficiently large and x a point of strictminimum of φ V,y : x → h U ( x ) − V, x − y i in B R , the following holds (26) r h U ( x ) , x − y i + λ h U ( x ) − T ∗ U ( T x ) , x − y i ≥h G ( x , U ( x )) , x − y i + h F ( x , U ( x )) , U ( x ) − V i . Remark 2.1.
Let us remark that this notion of solution is reminiscent of thedefinition of viscosity solution introduced by Crandall and Lions [20]. We feel thatit is useful to remark that, for the master equation (2), we could have defined a eak solution by the fact that the function φ V,y was a viscosity super-solution ofa certain PDE for all V ∈ R d , y ∈ O d . However, such a formalism would nothave allow us to define solution of the master equation for the cases of optimalstopping or impulse control, that is why we prefer the definition of solutions wejust presented. Remark 2.2.
Let us remark that this notion of solutions could be stated in moregeneral domains than O d , as long as we have a boundary condition of the type ofhypothesis 1. Let us insist that in the previous definition x may be on the boundary of O d .We introduce the ball B R because we shall place ourselves under hypothesis 2,which, as we already mentioned, has the effect to bound the trajectories. Beforecommenting our definition of solution, let us state the two following results whichjustify this choice of definition. Proposition 2.1.
Under hypotheses 1 and 2, a smooth solution of (2) is also amonotone solution of (2) in the sense of definition 2.1.Proof.
This result is fairly simple so we only sketch its proof here. Let us considera classical solution U of (2). If the point of strict minimum of x → h U ( x ) − V, x − y i is in the interior of B R , then thanks to (25), there is equality in (55). If this pointis on the boundary, thanks to the assumptions 1 and 2, the inequality holds. (cid:3) Theorem 2.1.
Under hypotheses 1-3, there exists at most one continuous mono-tone solution of (2) in the sense of definition 2.1. If it exists it is a monotoneapplication.Proof.
Let us consider U and V two such solutions. Let us define W : O d → R by(27) W ( x, y ) = h U ( x ) − U ( y ) , x − y i . Thanks to lemma 1.1, for any ǫ >
0, there exists ( a, b ) ∈ R d , | a | ≤ ǫ , such that( x, y ) → h U ( x ) − V ( y ) , x − y i + h a, x i + h b, y i has a strict minimum on ( B R ) (for R > ǫ ), attained at ( x , y ).Thus because U is a monotone solution the following holds.(28) r h U ( x ) , x − y i + λ h U ( x ) − T ∗ U ( T x ) , x − y i ≥h G ( x , U ( x )) , x − y i + h F ( x , U ( x )) , U ( x ) − V ( y ) + a i . On the other hand, because V is a monotone solution, we deduce that(29) r h V ( y ) , y − x i + λ h V ( y ) − T ∗ V ( T y ) , y − x i ≥h G ( y , V ( y )) , y − x i + h F ( y , V ( y )) , V ( y ) − U ( x ) + b i . umming the two previous equations, we obtain(30) rW ( x , y ) + λ ( W ( x , y ) − W ( T x , T y )) ≥h G ( x , U ( x )) − G ( y , V ( y )) , x − y i + h F ( x , U ( x )) , a i + h F ( y , V ( y )) , b i ++ h F ( x , U ( x )) − F ( y , V ( y )) , U ( x ) − V ( y ) i . From this we deduce the following.(31) rW ( x , y ) ≥ λ ( h a, x − T x i + h b, y − T y i ) + h F ( x , U ( x )) , a i + h F ( y , V ( y )) , b i . Because U and V are continuous, we deduce from the fact that ǫ can be chosenarbitrary small, that for any R >
0, for all η > W ≥ − η on B R . Thus weconclude as in the proof of proposition 1.2 that U = V and that U is monotone. (cid:3) This previous result is a strong justification for our notion of solutions. By con-sidering the proof of this result and the equivalent result in the smooth regime,one can realize that we have simply use all the ingredients useful to prove unique-ness of solutions of the master equation in the monotone regime and use it hasa definition of solutions. One can wonder if this notion is not too weak. In thenext section we show how it can be sufficient to describe solutions in the optimalstopping case or in the impulse control one. Moreover we now present results ofstability and consistency concerning monotone solutions.
Proposition 2.2.
Consider a sequence ( F n , G n ) n ∈ N of applications from O d × R d into R d which converges uniformly over all compact toward ( F, G ) . If for all n , U n is a continuous monotone solution of the master equation (2) associated to F n and G n and if ( U n ) n ∈ N converges uniformly toward U , then U is a monotone solutionof the master equation associated to F and G .Proof. Let us consider V ∈ R d , y ∈ O d , R > x a point of strict minimum of φ : x → h U ( x ) − V, x − y i in B R . From lemma 1.1, we can consider a sequence( a n ) n ∈ N ∈ ( R d ) N such that for all n ,(32) ( | a n | ≤ n − ; φ n : x → h U n ( x ) − V, x − y i + h a n , x i has a strict minimum x n in B R . Let us now remark that for all n ≥ h U n ( x n ) − V, x n − y i + h a n , x n i ≤ h U n ( x ) − V, x − y i + h a n , x i . From this last inequality, we deduce that ( x n ) n ∈ N converges toward x . Finally letus remark that because for all n ≥ U n is a monotone solution, we can write(34) r h U n ( x n ) , x n − y i + λ h U n ( x n ) − T ∗ U n ( T x n ) , x n − y i ≥h G n ( x n , U n ( x n )) , x n − y i + h F n ( x n , U n ( x n )) , U n ( x n ) − V + a n i . Passing to the limit in this last expression yields the required result. (cid:3) emark 2.3. The same type of results can be obtained in the case in which oneseeks stability for the terms λ and T in (2). This can be done by changing mildlythe previous proof. We now show consistency of this notion of solution under an additional mono-tonicity assumption. That is we show that if a smooth function U is a monotonesolution of (2) in the sense of definition 2.1 and that it satisfies an additionalmonotonicity assumption, then it is a classical solution of (2). Proposition 2.3.
Assume that U ∈ W , ∞ is a monotone solution of (2) in thesense of definition 2.1. Assume furthermore that for all x ∈ O d , D x U ( x ) > inthe order of positive definite matrix. Then U satisfies (35) rU i + ( F ( x, U ) · ∇ x ) U i + λ (cid:0) U i − ( T ∗ U ( t, T x )) i (cid:1) = G i ( x, U ) in { x i > } ;(36) rU i + ( F ( x, U ) · ∇ x ) U i + λ (cid:0) U i − ( T ∗ U ( t, T x )) i (cid:1) ≤ G i ( x, U ) in { x i = 0 } . Proof.
Let us fix x in the interior of O d . Let us define φ V,y as in definition 2.1.Let us remark that(37) ∇ x φ V,y ( x ) = D x U ( x )( x − y ) + U ( x ) − V ;(38) D x φ V,y ( x ) = 2 D x U ( x ) + D x U ( x )( x − y ) . Let us note that from the assumption on the monotonicity of U , there is a neigh-borhood O of x such that for any point y ∈ O , the right hand side of (38) isstrictly positive. Then, taking such a y and choosing V such that (37) vanishes,we have that x is a point of strict minimum of φ V,y . Because U is a monotonesolution of (2), we deduce that(39) r h U ( x ) , x − y + h F ( x , U ( x )) , D x U ( x )( x − y ) ii ≥ h G ( x , U ( x )) , x − y i . This last inequality holds for any y ∈ O . From this we easily deduce that U satisfies (2) at x . We argue in the same way when x is on the boundary of O d . (cid:3) Remark 2.4.
Let us remark that the assumption D x U ( x ) > in the order ofpositive definite matrix on O d is usually verified when some strict monotonicityassumption is made on ( G, F ) . Remark 2.5.
As it is usually the case in the MFG theory, the value function doesnot necessary satisfy the master equation on states where there is no player, butonly an inequality. This is reminiscent of the weak solutions studied in [15] forinstance. .2. The time dependent case.
In this section we present the analogue of def-inition 2.1 for the case of (1). Let us note that in general in the MFG theory, thetime regularity is not necessary the main challenge and that we could easily definea notion of monotone solutions which are smooth in time. However we prefer,for completeness, to present this concept for functions which are not necessarysmooth in time, even though it makes this section more technical. Following theprevious part, we define a monotone solution in the time dependent setting withthe following :
Definition 2.2.
A function U : (0 , ∞ ) × O d → R d is a monotone solution of (1) if • for any V ∈ R d , y ∈ O d and R > sufficiently large, for any ( t , x ) ∈ (0 , ∞ ) × B R such that x is a point of strict minimum of x → h U ( t , x ) − V, x − y i on B R , for any smooth real function φ such that φ ( t ) < h U ( t, x ) , x − y i for t ∈ ( t − ǫ, t ) for some ǫ > with φ ( t ) = h U ( t , x ) , x − y i , thefollowing holds : (40) ddt φ ( t ) + λ h U ( t , x ) − T ∗ U ( t , T x ) , x − y i ≥h F ( x, U ( t , x )) , U ( t , x ) − V i + h G ( x, U ( t , x )) , x − y i . • The initial condition holds. (41) U (0 , x ) = U ( x ) on O d . In some sense, we are treating the time derivative using techniques from vis-cosity solutions. As in the stationary case, we now present results of existence,uniqueness, stability and consistency for this notion of solutions.
Proposition 2.4.
Let U : (0 , ∞ ) × O d → R d be a smooth function, classicalsolution of (1). Then it is a monotone solution of (1) in the sense of definition2.2. Moreover, under the assumptions of proposition 1.3, there exists a monotonesolution of (1).Proof. We only state that for any x, y ∈ O d and t >
0, for any φ such that φ ( t ) < h U ( t, x ) , x − y i for t ∈ ( t − ǫ, t ) for some ǫ > φ ( t ) = h U ( t , x ) , x − y i ,then one has ddt φ ( t ) ≥ h ∂ t U ( t , x ) , x − y i . The rest of the proof follows easily. (cid:3) Theorem 2.2.
Under hypotheses 1 and 3, there exists at most one uniformlycontinuous monotone solution of (1) in the sense of definition 2.2.Proof.
Let us denote by U and V two such solutions. We define W : [0 , ∞ ) × O d → R by(42) W ( t, x, y ) = h U ( t, x ) − V ( t, y ) , x − y i . Our aim is to proceed as usual by proving that W ≥
0. Let us assume thatthere exists ( t , x , y ) such that W ( t , x , y ) <
0. Let us define the function : [0 , t ] × O d → R by(43) Z ( t, s, x, y ) = e − γ ( t + s ) h U ( t, x ) − V ( s, y ) , x − y i + 1 α ( t − s ) + h a, x i + h b, y i + δ t + δ s. This function is well defined and depends on the parameters γ, α, a, b, δ and δ .For any γ , there exists ǫ > | a | + | b | + | δ | + | δ | < ǫ then(45) min Z < − c for some c > γ and ǫ . This easily comes from the evaluationof Z at ( t , t , x , y ). From lemma 1.1, there exists a, b, δ , δ satisfying (44) suchthat Z has a strict minimum on [0 , t ] × O d at ( t ∗ , s ∗ , x ∗ , y ∗ ). If t ∗ >
0, then(46) φ : t → h V ( s ∗ , y ∗ ) , x ∗ − y ∗ i + e γ ( t − t ∗ ) h U ( t ∗ , x ∗ ) − V ( s ∗ , y ∗ ) , x ∗ − y ∗ i + e γ ( t + s ∗ ) δ ( t ∗ − t )can be chosen as a test function in (75) for U . If s ∗ > φ for V . Thus if both t ∗ > s ∗ >
0, then one obtain(47) γ h U ( t ∗ , x ∗ ) − V ( s ∗ , y ∗ ) , x ∗ − y ∗ i + λ h U ( t ∗ , x ∗ ) − V ( s ∗ , y ∗ ) , x ∗ − y ∗ i ≥ e γ ( t ∗ + s ∗ ) ( δ + δ + h F ( x ∗ , U ( t ∗ , x ∗ )) , a i + h F ( y ∗ , V ( s ∗ , y ∗ )) , b i )+ h U ( t ∗ , T x ∗ ) − V ( s ∗ , T y ∗ ) , T ( x ∗ − y ∗ ) i . Let us note that a posteriori, choosing ǫ in (44) as small as we want (what we cando and which does not alter (47)), we can contradict (45).We now treat the case in which t ∗ = 0 (the case s ∗ = 0 is similar). Let us take η, η ′ , η ′′ >
0. We want to show that Z ( t ∗ , s ∗ , x ∗ , y ∗ ) ≥ − η ′′ for a certain choice ofthe parameters. Because we can choose α as small as we want (without altering theproof of the previous case), we can assume that if t ∗ = 0, then s ∗ ≤ η . Thus usingthe continuity of V (if η is small enough compared to η ′ ) that | V ( s ∗ , y ∗ ) − U ( y ∗ ) | ≤ η ′ . From which we deduce using the continuity of U (if η ′ is small enough comparedto η ′′ ) that Z ( t ∗ , s ∗ , x ∗ , y ∗ ) ≥ − η ′′ which contradicts (45) since η ′′ is as small as wewant. Thus the function W is positive and we conclude as usual. (cid:3) Proposition 2.5.
Consider a sequence ( F n , G n ) n ∈ N of applications from O d × R d into R d which converges uniformly over all compact toward ( F, G ) . If for all n , U n is a continuous monotone solution of the master equation associated to F n and G n and if ( U n ) n ∈ N converges uniformly toward U , then U is a monotone solutionof the master equation associated to F and G .Proof. Let us take V ∈ R d , y ∈ O d and R > t ∗ , x ) ∈ (0 , ∞ ) × B R such that x is a point of strict minimum of x → h U ( t ∗ , x ) − V, x − y i on B R , and asmooth real function φ such that φ ( t ) < h U ( t, x ) , x − y i for t ∈ ( t ∗ − ǫ, t ∗ ) for some ǫ > φ ( t ∗ ) = h U ( t ∗ , x ) , x − y i . Reasoning as in the proof of proposition 2.2, here exists a sequence ( a n ) n ∈ N ∈ ( R d ) N which converges toward 0 and such thatfor all n ≥ x → h U n ( t ∗ , x ) − V + a n , x − y i has a strict minimum on B R attainedat x n . As in the proof of proposition 2.2, we obtain that ( x n ) n ∈ N converges toward x . We claim that there exists ( t n ) n ∈ N ∈ ( t ∗ − ǫ, t ∗ ) N and ( φ n ) n ∈ N a sequence offunctions in C ( R ) such that : • t n → t ∗ as n → ∞ . • k φ n − φ k C → n → ∞ . • φ n ( t ) < h U n ( t, x n ) , x n − y i for t ∈ ( t ∗ − ǫ, t n ). • φ n ( t n ) = h U n ( t n , x n ) , x n − y i .Let us detail why such sequences exists. Let us define u n ( t ) = h U n ( t, x n ) , x n − y i .For any n ≥
0, extracting a subsequence if necessary, we can assume that(48) α n := inf (cid:26) u n ( t ) − φ ( t ) | t ∈ (cid:18) t ∗ − ǫ, t ∗ − n + 1 (cid:19)(cid:27) > , (49) | u n ( t ∗ ) − φ ( t ∗ ) | ≤ α n . Thus from lemma 1.1, there exists | η n | < min { n +1 ; α n t ∗ +1) } such that t → u n ( t ) − φ ( t ) − η n t has a strict minimum on [ t ∗ − ǫ, t ∗ ]. By construction this minimum is in[ t ∗ − n +1 , t ∗ ]. Defining t n this point of strict minimum and φ n ( t ) := φ ( t ) + η n ( t − t n ) + u n ( t n ) − φ ( t n ), we obtain the aforementioned sequences.Because for all n ≥ U n is a monotone solution, we obtain the following.(50) ddt φ ( t n ) + η n + λ h U n ( t n , x n ) − T ∗ U n ( t n , T x n ) , x n − y i ≥h F n ( x n , U n ( t n , x n )) , U n ( t n , x n ) − V + a n i + h G n ( x, U n ( t n , x n )) , x n − y i . Thus passing to the limit n → ∞ we obtain the required result. (cid:3) A generalization of this method.
As already mentioned above, the aimof this paper is to present a new notion of solution for MFG master equationsand not necessary to enter into too much details on these solutions. However webelieve the next remark to be worth mentioning. It has been pointed out to us byPierre-Louis Lions (Coll`ege de France).Let us consider the case λ = 0. The main argument to establish uniqueness ofmonotone solutions is to consider two such solutions U and V and to prove that W defined by(51) W ( x, y ) = h U ( x ) − V ( y ) , x − y i is positive. Instead, for instance, we could have defined the function W with(52) W ( x, y ) = h U ( x ) − V ( y ) , φ ( x ) − φ ( y ) i or some φ : O d → R d . Now let us remark that if D x φ ( x ) is an invertible matrixfor any x in the interior of O d , the property(53) W ≥ ⇒ U = V still holds. This remark immediately generalizes the result of this section to awider class systems. Indeed by replacing the condition ( G, F ) is monotone by forany x, y ∈ O d , U, V ∈ R d (54) h G ( x, U ) − G ( y, V ) , φ ( x ) − φ ( y ) i + h F ( x, U ) − F ( y, V ) , U.D x φ ( x ) − V.D x φ ( y ) i ≥ , we obtain the uniqueness of the associated monotone solutions. This concept ofsolution depending on φ can be defined (in the stationary case) by Definition 2.3.
A function U ∈ C ( O d , R d ) is said to be a φ -monotone solution of(2) if for any V ∈ R d , y ∈ φ ( O d ) , R > sufficiently large and x a point of strictminimum of ψ V,y : x → h U ( x ) − V, φ ( x ) − y i in B R , the following holds (55) r h U ( x ) , φ ( x ) − y i ≥ h G ( x , U ( x )) , x − y i + h F ( x , U ( x )) , ( U ( x ) − V ) D x φ ( x ) i . The master equation for mean field games with optimal stopping
This section introduces the formulation of the master equation modeling a MFGin which the players have the possibility to leave the game. As already mentionedin the introduction, let us recall that MFG of optimal stopping have been thesubject of several works but that the case of the master equation for such MFGhas not been treated up to now.In this section we are interested with a MFG, similar to the ones representedby (1) and (2), except for the fact that the players are allowed to leave the gamewhenever they decide by paying a certain exit cost. Moreover, once they have leftthe game, they do not interact anymore with the other players. We refer to [5] formore details on this kind of MFG.Following [5] we study first a penalized version of the game, and then show howwe can pass to the limit.3.1.
The penalized master equation.
In the penalized version of the MFG ofoptimal stopping, the players cannot decide to leave instantly the game, they canonly control the intensity of a Poisson process which give their exit time, and theintensity of this process is bounded by ǫ − for ǫ >
0. Even though we do not wantto enter into the precise formulation of this penalized game, let us insist on the factthat those aforementioned Poisson processes are supposed to be independent fromone player to the other. The penalized master equation is then of the following orm in the time dependent setting.(56) ∂ t U + 1 ǫ β ( U ) + (cid:18) ǫ β ′ ( U ) ∗ x + F ( x, U ) (cid:19) · ∇ x U ++ λ ( U − T ∗ U ( t, T x )) = G ( x, U ) in (0 , ∞ ) × O d ,U (0 , x ) = U ( x ) in O d , where β stands for the positive part. We recall that ∗ stands for the term byterm product and that β ( U ) is understood component wise. Clearly β ′ is not welldefined but we shall come back on this technicality later. In the stationary setting,the form of the penalized master equation is then(57) rU + 1 ǫ β ( U )+ (cid:18) ǫ β ′ ( U ) ∗ x + F ( x, U ) (cid:19) ·∇ x U + λ ( U − T ∗ U ( t, T x )) = G ( x, U ) in O d . In the case λ = 0, the form of the characteristics of the previous equations is givenby(58) ( ˙ V ( t ) = G ( y ( t ) , V ( t )) − ǫ β ( V ) , ˙ y ( t ) = F ( y ( t ) , V ( t )) + ǫ β ′ ( V ) ∗ y. This type of characteristics is clearly what we expect from the study in [5].In (56)-(57), the exit cost paid by the players to leave the game is 0. The casein which the exit cost depends on the state of the player which is leaving the gamebut not on the density of all the players can easily be treated in a similar fashion.The case in which the exit cost depends on the density of players is much moreinvolved as structural assumptions have to be made on the form of the exit cost toensure the propagation of monotonicity. We refer to [5] for more detailed on thistopic and we leave this case for future research.For the rest of this study we focus on the stationary case and we mention thetime dependent setting at the end of this part on optimal stoping.3.2.
Results on the stationary penalized equation.
This section presentsthree, quite simple, results on the penalized master equation which enable us topass to the limit ǫ → ǫ , a prioriestimate for the solution of (57). The third result is concerned with monotonesolutions of (57). Proposition 3.1.
Under hypotheses 1-2, there exists at most one smooth solution U of (57) such that β ′ ( U ) is understand as being a function which satisfies (59) β ′ ( U i ( x )) ∈ ∂β ( U i ( x )) , where ∂β is the subdifferential of β . If it exists, this solution is monotone. roof. The proof is very similar to the one of proposition 1.2. Let us take U and V two smooth solutions. We introduce W : O d → R defined by :(60) W ( x, y ) = h U ( x ) − V ( y ) , x − y i . Let us remark that W satisfies(61) rW + (cid:18) ǫ β ′ ( U ( x )) ∗ x + F ( x, U ( x )) (cid:19) · ∇ x W + (cid:18) ǫ β ′ ( V ( y )) ∗ y + F ( y, V ( y )) (cid:19) · ∇ y W = h G ( x, U ( x )) − G ( y, V ( y )) , x − y i + h F ( x, U ( x )) − F ( y, V ( y )) , x − y i ++ 1 ǫ ( h β ′ U ( x ) ∗ x − β ′ ( V ( y )) ∗ y, U ( x ) − V ( y ) i − h β ( U ( x )) − β ( V ( y )) , x − y i ) , = h G ( x, U ( x )) − G ( y, V ( y )) , x − y i + h F ( x, U ( x )) − F ( y, V ( y )) , x − y i ++ 1 ǫ ( h x, β ( V ( y )) − β ′ ( U ( x )) V ( y ) i + h y, β ( U ( x )) − β ′ ( V ( y )) U ( x ) i ) , ≥h G ( x, U ( x )) − G ( y, V ( y )) , x − y i + h F ( x, U ( x )) − F ( y, V ( y )) , x − y i . The rest of the proof follows as the one of proposition 1.2. (cid:3)
Remark 3.1.
Two conclusions can be drawn from this result. The first one isthat the terms arising from the modeling of optimal stopping only reinforce themonotonicity of the equation. The second one is that we can indeed use the notation β ′ ( · ) quite freely as it is justified by this uniqueness property. Remark 3.2.
Let us briefly insist on the fact that the addition of the term β ′ ( U ) ∗ x in the dynamics does not alter the assumptions 1 and 2. The following result is the main argument why we are able to pass to the limit ǫ → Proposition 3.2.
Under the assumptions of proposition 1.4, there exists
C > independent of ǫ such that if U is the classical solution of (57), then (62) k∇ x U k ≤ C. Proof.
This proof is similar to the one of proposition 1.4. Let us define W and Z by(63) W ( x, ξ ) = h U ( x ) , ξ i , (64) Z ( x, ξ ) = h∇ x W ( x, ξ ) , ξ i − δ |∇ x W | + γ | ξ | , or some constants β and γ . Let us remark that, arguing as if β was a smoothfunction, Z satisfies(65) rZ + h ǫ − β ′ ( U ) ∗ x + F ( x, ∇ ξ W ) , ∇ x Z i + h ( ǫ − β ′′ ( U ) ∗ x + D p F ( x, ∇ ξ W )) ∇ ξ Z, ∇ x W i−− h D p G ( x, ∇ ξ W ) ∇ ξ Z, ξ i + λ ( Z − Z ( t, T x, T ξ ))= h D x G ( x, ∇ x W ) ξ, ξ i − h D p G ( x, ∇ ξ W ) ∇ x W, ξ i − h D x F ( x, ∇ ξ W ) ∇ x W, ξ i + h D p F ( x, ∇ ξ W ) ∇ x W, ∇ x W i + h ǫ − β ′′ ( U ) ∗ x ∗ ∇ x W, ∇ x W i− δ h D x G ( x, ∇ x W ) ξ, ∇ x W i + 2 δ h D x F ( x, ∇ ξ W ) ∇ x W, ∇ x W i + 2 δ h β ′ ( U ) ∗ ∇ x W, ∇ x W i + δλ (cid:18) |∇ x W | − h∇ x W ( T x, T ξ ) , T ∇ x W i + |∇ x W ( T x, T ξ ) | (cid:19) + rδ |∇ x W | + 2 γ ( h D p F ( x, ∇ ξ W ) ξ, ∇ x W i − h D p G ( x, ∇ ξ W ) ξ, ξ i ) − rγ | ξ | + λγ ( | ξ | − | T ξ | ) . Let us remark that, since β is an increasing and convex function, all the terms inthe right hand side involving ǫ are positive. Thus we conclude that there exists ana priori estimate independent of ǫ . To remark that this fact immediately extendto the case β ( · ) = ( · ) + , it suffices to realize that β can approximate uniformly bysmooth convex and increasing functions. (cid:3) As already mentioned in the previous section, existence of solution of a station-ary master equation can be established from estimates such as (62) if some localboundness holds. We here give an assumption for which such a property can beproven.
Hypothesis 6.
For any p ∈ R d , F (0 , p ) = 0 . Moreover there is a unique solution V ∈ O d of (66) rV + λ ( V − T ∗ V ) = − G (0 , − V ) . We believe this assumption to be quite mild as it only assumes that i) once themass of players reaches 0, it stays at 0, ii) the MFG with a 0 mass of player (whichis thus an optimal control problem) is well defined and players remaining do notexit it. Let us remark that there can indeed still be player in the MFG, even ifthe mass of players is 0. In such a case, the remaining players do not ”see” eachother.
Proposition 3.3.
For ǫ > , under the assumptions of proposition 1.4 and hy-pothesis 6, there exists a unique monotone solution U ǫ of (57). It is a solutionof (57) almost everywhere. The sequence ( U ǫ ) ǫ> is uniformly lipschitz continuousand ( U ǫ (0)) ǫ> is a bounded sequence. The proof of this result follows exactly the argument of the previous part. .3. The limit master equation.
We show in this section how we can character-ize the value function of a MFG of optimal stopping using the notion of monotonesolutions. The main idea consists in characterizing the limit of the sequence ( U ǫ ) ǫ> of solutions of (57) when ǫ →
0. Let us recall that, because U ǫ is a monotone so-lution of (57), for any V ∈ R d , y ∈ O d and x point of strict local minimum of φ V,y : x → h U ǫ ( x ) − V, x − y i , the following holds.(67) r h U ǫ ( x ) , x − y i + λ h U ǫ ( x ) − T ∗ U ǫ ( T x ) , x − y i≥h G ( x, U ǫ ( x )) , x − y i + h F ( x, U ǫ ( x )) , U ǫ ( x ) − V i ++ 1 ǫ ( h β ′ ( U ǫ ) ∗ x, U ǫ ( x ) − V i − h β ( U ǫ ( x )) , x − y i ) , = h G ( x, U ǫ ( x )) , x − y i + h F ( x, U ǫ ( x )) , U ǫ ( x ) − V i + − ǫ ( h β ′ ( U ǫ ) ∗ x, V i + h β ( U ǫ ( x )) , y i ) . From this computation, we deduce that if V ∈ R d is such that V ≤
0, then for any y ∈ O d and x point of strict local minimum of φ V,y : x → h U ǫ ( x ) − V, x − y i , weobtain that(68) r h U ǫ ( x ) , x − y i + λ h U ǫ ( x ) − T ∗ U ǫ ( T x ) , x − y i ≥h G ( x, U ǫ ( x )) , x − y i + h F ( x, U ǫ ( x )) , U ǫ ( x ) − V i . As we clearly expect that the limit of ( U ǫ ) ǫ> (if it exists) is negative, this leadsus to the following definition. Definition 3.1.
A function U ∈ C ( O d , R d ) is said to be a monotone solution ofthe master equation for the MFG of optimal stopping if • U ≤ , • for any V ∈ R d such that V ≤ , for any y ∈ O d , R > sufficiently largeand x a point of strict minimum of φ V,y : x → h U ( x ) − V, x − y i in B R ,the following holds (69) r h U ( x ) , x − y i + λ h U ( x ) − T ∗ U ( T x ) , x − y i ≥h G ( x , U ( x )) , x − y i + h F ( x , U ( x )) , U ( x ) − V i . Let us insist that the only thing which differs from the non-optimal stopping caseis that U has to be negative component-wise and that we only have informationfor V which are also negative component wise. The existence of such a solution isstated in the next result. Theorem 3.1.
Under the assumption of proposition 1.4 and hypothesis 6, thereexists a monotone solution U of the MFG of optimal stopping in the sense ofdefinition 3.1. roof. For ǫ > U ǫ the unique monotone solution of (57). Fromproposition 3.3, we know that, extracting a subsequence if necessary, ( U ǫ ) ǫ> con-verges uniformly toward a function U . By considering maxima of U ǫ on B R for R > ǫ >
0, we immediately obtain from maximum likeresult (thanks to hypothesis 2) that U ǫ is bounded from above by Cǫ on B R , where C > R but not on ǫ . Hence we deduce that U ≤
0. The rest of the proof follows as in the proof of proposition 2.5 thanksto (68). Hence we do not detail the rest of the proof. Let us only mention thatthanks to lemma 1.1, using the notations of the proof of proposition 2.5, we canchoose ( a n ) n ∈ N such that V − a n ≤ n ≥ (cid:3) We now present a result of uniqueness for this type of solution.
Theorem 3.2.
Under the hypotheses 1-3, there exists at most one monotone so-lution of the MFG of optimal stopping in the sense of definition 3.1.Proof.
The proof of this statement follows exactly the one of theorem 2.1 by re-marking that thanks to lemma 1.1, using the notation of the aforementioned proof, a and b can be chosen such that V − a ≤ U − b ≤ (cid:3) Comments on this notion of solution in the optimal stopping case.
In this section we want to discuss how the knowledge of the value function of theMFG is helpful to understand the behavior of the population of agents.First, we expect that when no player is leaving the game, U solves a certain PDE,which is the master equation characterizing the MFG without optimal stopping.The set which corresponds to the fact that no player is leaving is the set ∩ di =1 { U i < } . The fact that U satisfies this property can be obtained by two arguments. First,from the PDE satisfied by U ǫ for ǫ > ǫ → U i ( x ) < i and D x U ( x ) > U indeed satisfies(70) rU i + ( F ( x, U ) · ∇ x ) U i + λ (cid:0) U i − ( T ∗ U ( t, T x )) i (cid:1) = G i ( x, U ) in { x i > } ∩ { U < } , (71) rU i + ( F ( x, U ) · ∇ x ) U i + λ (cid:0) U i − ( T ∗ U ( t, T x )) i (cid:1) ≤ G i ( x, U ) in { x i = 0 } ∩ { U < } . Thus in the set { U < } , we can infer the evolution of the population of agents asin the case without optimal stopping.On the other hand, when players are actually leaving the game, we would liketo gain information on how they are leaving the game. Although the question ofdescribing the precise evolution of the population is not the central question ofthis paper, we indicate formally what happens for the density of players. We referto [5] for more details on this question in the deterministic case. Let us considera distribution of players x ∈ O d such that it is optimal for some players to leave he game. For this to happen, one must have I ( x ) = ∅ where I is defined by(72) I ( x ) = { i ∈ { , ..., d }| U i ( x ) = 0 } . A natural requirement could be to expect that starting from x , the density ofplayers should instantly become ˜ x defined by(73) ( ˜ x i = x i for i / ∈ I ( x ) , ˜ x i = 0 for i ∈ I ( x ) . This type of behavior correspond to considering only symmetric Nash equilibriain pure strategies. From [5], we know that we have to consider Nash equilibriain mixed strategies and that players can play a certain leaving rate. Hence, eventhough some players are leaving the game in the state i ∈ I ( x ), this does notnecessary mean that ˜ x i = 0. However, because some players are leaving and someare staying in this situation, there is a way to determine ˜ x i . Following [5], weexpect that ˜ x is characterized by(74) ˜ x i = x i for i / ∈ I ( x ) ,U (˜ x ) = U ( x ) ,G i (˜ x, U (˜ x ))˜ x i = 0 for i ∈ I ( x ) . The last line of (74) stands for the fact that either all the players in state i haveleft the game, or either it is also optimal to stay in the game and thus one musthave G i (˜ x, U (˜ x )) = 0 (recall that the exit cost is 0). Finally, let us remark thatwhen there is a strict monotonicity assumption on G , for any x ∈ O d , there is atmost one ˜ x solution of (74).3.5. The time dependent case.
As we already mentioned above, we are notgoing to treat in full details the time dependent case. However we indicate thenatural generalizations of the results above for this problem. The definition ofsolution of the problem is straightforward from definitions 2.2 and 3.1. It is givenby
Definition 3.2.
A function U : (0 , ∞ ) × O d → R d is a monotone solution of thetime dependent MFG of optimal stopping if • U ≤ , • for any V ∈ R d , V ≤ , y ∈ O d and R > sufficiently large, for any ( t , x ) ∈ (0 , ∞ ) × B R such that x is a point of strict minimum of x →h U ( t , x ) − V, x − y i on B R , for any smooth real function φ such that φ ( t ) < h U ( t, x ) , x − y i for t ∈ ( t − ǫ, t ) for some ǫ > with φ ( t ) = h U ( t , x ) , x − y i , the following holds : (75) ddt φ ( t ) + λ h U ( t , x ) − T ∗ U ( t , T x ) , x − y i ≥h F ( x, U ( t , x )) , U ( t , x ) − V i + h G ( x, U ( t , x )) , x − y i . (76) U (0 , x ) = min( U ( x ) , on O d . The fact that the initial condition is only satisfied when U is smaller than 0 isclassical feature of optimal stopping problem. The uniqueness of such solutions isa direct adaptation of the proof of theorem 2.2. It can be summarizes as follows. Theorem 3.3.
Under hypotheses 1 and 3, for any continuous function U , thereexists at most one continuous function solution of the time dependent MFG ofoptimal stopping in the sense of definition 3.2. The question of existence of such a solution is more involved that in the sta-tionary case. We expect that it can also be proven by penalization by consideringthe sequence ( U ǫ ) ǫ> of solutions of (56) and taking the limit ǫ →
0. The mainargument to consider this method is that, as in the stationary case, a, uniformin ǫ , estimates on k D x U ǫ k can be established. However, unlike in the case of (1),because of the terms in ǫ − in (56), this does not automatically translates intouniform estimates on the time regularity of U ǫ .4. The master equation for mean field games of impulse control
This section generalizes the results of the previous section to a case in whichthe players have the possibility to use impulse controls. Let us briefly insist on thefact that in finite state space, the evolution of the state of a player is necessarydiscontinuous. Hence it could be thought that all MFG in finite state space aregames of impulse controls. This is not the case. Indeed if one were to detail thegame modeled by a master equation of the type of (1), then the transition ratesbetween the states, which are controlled by the players, would be either boundedor get a dissuasive cost as they become higher. In this section, we consider thepossibility for a player to change instantly its state by paying a certain (finite)cost, this is what we call a MFG of impulse control. We refer to [4] for a completepresentation of impulse control problems and to [6] for the study of MFG of impulsecontrol without common noise.As the formulation of impulse control is very general, and thus difficult to workwith, we shall focus at some point in our study on a particular instance of MFGof impulse control. We hope that the forthcoming results convince the reader ofthe generality of this method. As we did in the optimal case, we shall also focuson the stationary case in this section.4.1.
Description of the model.
We consider a MFG in finite state space whichis described by
F, G, λ and T as in (2). We add the possibility for a player in thestate i to instantly jump to the state j by paying a cost k ij ≥
0. The so-calledjump operator is then defined by(77) (
M p ) i = min { k ij + p j | j ∈ { , ...d }} , for p ∈ R d , i ∈ { , ..., d } . e shall use the notation M i p = ( M p ) i for i ∈ { , ..., d } . Given the value function U of the MFG of impulse control, the function M U plays formally the same roleas an exit cost. Indeed when a player is in a state i and the distribution of otherplayers is x ∈ O d , such that(78) U i ( x ) < M i U ( x ) , then it is strictly sub-optimal to use an impulse control. On the other hand, whenthere is equality in (78), then it is optimal for players in state i to jump to a state j which reaches the minimum of M i U ( x ) in (77).As in the optimal stopping case, we introduce first a penalized version of themaster equation and then explain how we can pass to the limit.4.2. The penalized master equation and monotone solutions of the limitgame.
Following the previous section, we are interested in a penalized version ofthe impulse control MFG in which the players cannot exactly jump instantly fromone state to another but only control an intensity of jump which is bounded by ǫ − for ǫ > rU i + 1 ǫ β ( U i − M i U ) + 1 ǫ (( α · ) ∗ x − x · α ) · ∇ x U i + ( F ( x, U )) · ∇ x U i ++ λ ( U i − ( T ∗ ) i U ( T x )) = G i ( x, U ) in O d , i ∈ { , ..., d } . where = (1 , ...,
1) and α = ( α ij ) ≤ i,j ≤ d is a matrix made of d real functions on O d which satisfy(80) α ij ( x ) = 0 if U i ( x ) < U j ( x ) + k ij on O d , P dj =1 α ij ( x ) = 1 if U i ( x ) ≥ U j ( x ) + k ij on O d , ≤ α ij ≤ , P dj =1 α ij ≤ . Formally, for i, j ∈ { , ..., d } , the function α ij indicates the proportion of playersin state i which chooses to jump (using the impulse control) to state j . Let us notethat in particular that different jumps may be optimal to use in the same state andthus that we are forced to use this family of functions ( α ij ) ≤ i,j ≤ d . Let us insiston the fact established in [5] for the optimal stopping case, that if we restrict toourselves to situation in which only a single behavior is optimal in every state, thena Nash equilibrium may not exist. From [6], we expect that some monotonicityproperty holds for (79)-(80). This following result gives a precise statement of thisidea. Proposition 4.1.
Under hypothese (1)-(3), there exists at most one smooth func-tion U such that ( U, α ) is a solution of (79)-(80). Moreover if such a solutionexists, U is monotone. roof. As usual we take (
U, α ) and (
V, γ ) two such solutions and we define W : O d → R by(81) W ( x, y ) = h U ( x ) − V ( y ) , x − y i . Let us remark that W is a solution of(82) rW + (cid:18) F ( x, U ) + 1 ǫ (( α · ) ∗ x − x · α ) (cid:19) ∇ x W + (cid:18) F ( y, V ) + 1 ǫ (( γ · ) ∗ y − y · γ ) (cid:19) ∇ y W = h G ( x, U ) − G ( y, V ) , x − y i + h F ( x, U ) − F ( y, V ) , U − V i − λ ( W ( x, ξ ) − W ( T x, T ξ ))++ 1 ǫ h ( V − M V ) + − ( U − M U ) + , x − y i ++ 1 ǫ h ( α · ) ∗ x − x · α − ( γ · ) ∗ y + y · γ, U − V i . Let us remark that(83) h ( V − M V ) + − ( U − M U ) + , x − y i + h ( α · ) ∗ x − x · α − ( γ · ) ∗ y + y · γ, U − V i = h x, ( V − M V ) + − ( U − M U ) + + ( U − V ) ∗ ( α · ) − α · ( U − V ) i + h y, ( U − M U ) + − ( V − M V ) + + ( V − U ) ∗ ( γ · ) − γ · ( V − U ) i , ≥h x, ( V − M V ) + − V ∗ ( α · ) + α · V + ( α ∗ k ) · i + h y, ( U − M U ) + − U ∗ ( γ · ) + γ · U + ( α ∗ k ) · i , ≥ , where k = ( k ij ) ≤ i,j ≤ d . Thus we deduce that W satisfies(84) rW + (cid:18) F ( x, U ) + 1 ǫ (( α · ) ∗ x − x · α ) (cid:19) ∇ x W + (cid:18) F ( y, V ) + 1 ǫ (( γ · ) ∗ y − y · γ ) (cid:19) ∇ y W + λ ( W ( x, ξ ) − W ( T x, T ξ )) ≥ . Thus we conclude as we did in proposition 1.2, first that W ≥
0, and then therequired results. (cid:3)
This result clearly suggests that the master equation for the impulse controlMFG is well posed. Indeed, as we shall see, as the solution of the problem aremonotone, the notion of monotone solution introduced in section 2 shall be helpful.As in the case of optimal stopping, let us note consider a monotone solution U of the penalized equation (79) for ǫ > α ij ) ≤ i,j ≤ d . Thus for any V ∈ R d , y ∈ O d and x a point of strict local minimum f x → h U ( x ) − V, x − y i , the following holds :(85) r h U ( x ) , x − y i + λ h U ( x ) − T ∗ U ( T x ) , x − y i≥ h G ( x , U ( x )) − β ( U ( x ) − M U ( x )) , x − y i ++ h F ( x , U ( x )) + 1 ǫ (( α ( x ) · ) ∗ x − x · α ( x )) , U ( x ) − V i . From the same calculations of the previous proof, because (80) holds, the previousequation can be rewritten to obtain the following.(86) r h U ( x ) ,x − y i + λ h U ( x ) − T ∗ U ( T x ) , x − y i ≥ h G ( x , U ( x )) , x − y i + h F ( x , U ( x )) , U ( x ) − V i − ǫ h x , (( α ( x ) · ) ∗ V + α ( x )) · V − ( α ∗ k ) · i . Hence, if V satisfies V ≤ M V , then we obtain(87) r h U ( x ) , x − y i + λ h U ( x ) − T ∗ U ( T x ) , x − y i ≥h G ( x , U ( x )) , x − y i + h F ( x , U ( x )) , U ( x ) − V i . This remark leads us to the following definition.
Definition 4.1.
A function U ∈ C ( O d , R d ) is said to be a monotone solution ofthe master equation for the MFG of impulse control if • U ≤ M U , • for any V ∈ R d such that V ≤ M V , for any y ∈ O d , R > sufficientlylarge and x a point of strict minimum of φ V,y : x → h U ( x ) − V, x − y i in B R , the following holds (88) r h U ( x ) , x − y i + λ h U ( x ) − T ∗ U ( T x ) , x − y i ≥h G ( x , U ( x )) , x − y i + h F ( x , U ( x )) , U ( x ) − V i . Remark 4.1.
As we can see, this definition is similar to the one of the optimalstopping case, as they only differ in the fact that the constraint of being negativecomponent wise has been replaced by satisfying an inequality associated with thejump operator M . Results for particular mean field games of impulse control.
In thepresent section, we assume a particular form for the jump operator M and weprove a result of uniqueness for monotone solutions in the impulse control case.Let us assume that not all impulse jumps are feasible, that is we assume that k ij = + ∞ for some i, j ∈ { , ..., d } . More precisely we assume the following. Hypothesis 7.
For all i ∈ { , ..., d } , there exists no non constant sequence i = i , i , ...i n +1 = i such that for all k ∈ { , ..., n } , (89) k i k i k +1 < + ∞ . Moreover, for all i, j ∈ { , ..., d } , k ij > . e comment on this assumption after the following result. Theorem 4.1.
Under the hypotheses 1-3 and 7, there exists at most one solutionof the master equation of the MFG of impulse control in the sense of definition4.1. If it exists, this solution is monotone.Proof.
From assumption 7, there exists an open set
O ⊂ R d such that 0 ∈ ¯ O andfor any V ∈ R d , a ∈ O :(90) V ≤ M V ⇒ V + a ≤ M ( V + a ) . The rest of the proof follows exactly the one of theorem 2.1 by remarking thatthanks to lemma 1.1, using the notations of the aforementioned proof, a and b canbe chosen in O . (cid:3) Remark 4.2.
Hypothesis 7 restricts the jumps such that it is not possible to do asequence of successive jumps to exit one state and enter it again. We firmly believethis assumption is purely technical, even though we have not been able to prove it.Let us also mention that no particular assumption of this form is made on F , inparticular, using ”usual” controls, the players can come back to state from whichthey have jump. A mean field game of entry and exit
In this section we consider an application of the tools developed in this paper.We consider a market which agents can enter or exit by paying a certain cost,and in which the revenue of the agents in the market is a function of the totalnumber of agents in the market. The simple model we are about to present isclosely related to a model for cryptocurrency mining introduced in [10], namelythe version of the model in the section 4.4.The number of agents in the market is denoted by the variable K . The cost toexit the market is denoted by s ∈ R and the cost to enter the market is denotedby b ∈ R . We assume(91) b < s. Knowing the evolution of the number of agents in the market ( K t ) t ≥ , the revenueof an agent in the market is given by(92) Z τ e − rt g ( K t ) dt, where g is a real function, r > ≤ τ ≤ + ∞ is the time at whichthe agent exits the market. We want to characterize the value function U of thisMFG which gives for a number of agents K in the market, the value of the game U ( K ) for agents in the market. Following section 3, we introduce first a penalizedversion of the game in which the player cannot exit or enter the market freely. For >
0, a plausible penalization of the MFG leads to the following penalized masterequation.(93) rU + 1 ǫ ( β ′ ( U − s ) K − β ( b − U )) ∂ K U + 1 ǫ β ( U − s ) = g ( K ) in R + , where β ( · ) = ( · ) + . The following result holds true. Proposition 5.1.
Let U be a smooth solution of (93). Assume that g is an in-creasing and lipschitz function. There exists C > depending only on g and r such that (94) k ∂ K U k ∞ ≤ C. The proof of this statement follows exactly the one of proposition 3.2 so we donot detail it here. From this result, we easily deduce that, if g is lipschitz andincreasing, there exists a monotone solution of (93). Such a monotone solution U satisfies that for any V ∈ R , y ∈ R + , K a point of local strict minimum of K → ( U ( K ) − V )( K − y )(95) rU ( K )( K − y ) ≥ g ( K )( K − y ) + 1 ǫ ( β ′ ( U ( K ) − s ) K − β ( b − U ( K ))) ( U ( K ) − V ) −− ǫ β ( U ( K ) − s )( K − y ) . Let us remark that if b ≤ V ≤ s , we obtain :(96) rU ( K )( K − y ) ≥ g ( K )( K − y ) . This remark leads us to the following definition.
Definition 5.1.
A continuous real function U is said to be a monotone solutionof the MFG of entry and exit if • b ≤ U ≤ s • for any V ∈ [ b, s ] , for any y ∈ R + , for any K point of strict local minimumof K → ( U ( K ) − V )( K − y ) , (97) rU ( K )( K − y ) ≥ g ( K )( K − y ) . The following result is easily obtained following the previous parts of this article.
Theorem 5.1.
Assume that g is a lipschitz and increasing function. There existsa unique monotone solution of the MFG of entry and exit in the sense of definition5.1. Remark 5.1.
Obviously, we only intend to apply the concepts developed in theprevious sections here, and extensions of this model could easily be considered,following modeling as in [10] or simply by adding terms which plays the same roleas F and G in the previous sections. . Conclusion and future perspectives
In this paper we have provided a notion of solution to study a class of first ordersystems of PDE arising from the MFG theory and referred to as master equations.This notion of monotone solutions relies heavily on the so-called monotone struc-ture of the master equation (hypothesis 3). Although it was already well known inthe MFG community that regularity and uniqueness for the solution of the mas-ter equation can be obtained in a monotone regime, we hope that our notion ofsolution can be helpful to reduce the assumptions made to obtain well-posednessof master equations. Moreover we have shown in this paper that we are able tocharacterize the value function of MFG involving a variety of novel actions for theplayers (stopping, jumping, entering) using monotone solutions.The natural extensions for this notion of solutions are the case in which themaster equation takes the form of an infinite dimensional PDE and the case ofsecond order equations. The generalization of this work to infinite dimensionalcases is standard and shall be the subject of another work. Concerning the case ofsecond order master equations, defining a notion of monotone solutions for func-tions which are one time differentiable is straightforward following the techniqueswe here developed, however to treat functions which are merely continuous is muchmore involved and shall also be the subject of a future work.Finally, we do not claim that the notion of monotone solutions is appropriateto address the question of the characterization of a value function in a MFG ina non-monotone regime, in particular because in such a situation, if such a valuefunction exists, it may be discontinuous. Personally, we do not believe this previousproblem to be solvable outside under additional structural assumptions on theMFG (potential case, particular couplings, particular information structure...).
Acknowledgments
We would like to thank Pierre-Louis Lions (Coll`ege de France) for pointing out tous the question of optimal stopping in MFG a few years ago and for the numerousdiscussions we had on this topic.
References [1] Yves Achdou and Italo Capuzzo-Dolcetta. Mean field games: Numerical meth-ods.
SIAM Journal on Numerical Analysis , 48(3):1136–1162, 2010.[2] Yves Achdou and Mathieu Lauri`ere. Mean field games and applications: Nu-merical aspects. arXiv preprint arXiv:2003.04444 , 2020.[3] Erhan Bayraktar, Alekos Cecchin, Asaf Cohen, and Francois Delarue. Fi-nite state mean field games with wright-fisher common noise. arXiv preprintarXiv:1912.06701 , 2019.[4] Alain Bensoussan and Jacques Louis Lions.
Impulse control and quasi-variational inequalities . Gaunthier-Villars, 1984.
5] Charles Bertucci. Optimal stopping in mean field games, an obstacle problemapproach.
Journal de Math´ematiques Pures et Appliqu´ees , 120:165–194, 2018.[6] Charles Bertucci. Fokker-planck equations of jumping particles and mean fieldgames of impulse control. In
Annales de l’Institut Henri Poincar´e C, Analysenon lin´eaire . Elsevier, 2020.[7] Charles Bertucci. Work in progress. 2020.[8] Charles Bertucci. A remark on uzawa’s algorithm and an application to meanfield games systems.
ESAIM: Mathematical Modelling and Numerical Analy-sis , 54(3):1053–1071, 2020.[9] Charles Bertucci, Jean-Michel Lasry, and Pierre-Louis Lions. Some remarkson mean field games.
Communications in Partial Differential Equations , 44(3):205–227, 2019.[10] Charles Bertucci, Louis Bertucci, Jean-Michel Lasry, and Pierre-Louis Lions.Mean field game approach to bitcoin mining. arXiv preprint arXiv:2004.08167 ,2020.[11] Jonathan M Borwein and Dominikus Noll. Second order differentiability ofconvex functions in banach spaces.
Transactions of the American Mathemat-ical Society , pages 43–81, 1994.[12] Luis Brice˜no-Arias, Dante Kalise, Ziad Kobeissi, Mathieu Lauri`ere, A MateosGonz´alez, and Francisco Jos´e Silva. On the implementation of a primal-dual algorithm for second order time-dependent mean field games with localcouplings.
ESAIM: Proceedings and Surveys , 65:330–348, 2019.[13] Pierre Cardaliaguet and Alessio Porretta. Long time behavior of the masterequation in mean field game theory.
Analysis & PDE , 12(6):1397–1453, 2019.[14] Pierre Cardaliaguet, Jean-Michael Lasry, Pierre-Louis Lions, and Alessio Por-retta. Long time average of mean field games.
Networks & HeterogeneousMedia , 7(2), 2012.[15] Pierre Cardaliaguet, P Jameson Graber, Alessio Porretta, and Daniela Tonon.Second order mean field games with degenerate diffusion and local coupling.
Nonlinear Differential Equations and Applications NoDEA , 22(5):1287–1317,2015.[16] Pierre Cardaliaguet, Fran¸cois Delarue, Jean-Michel Lasry, and Pierre-LouisLions.
The Master Equation and the Convergence Problem in Mean FieldGames:(AMS-201) , volume 201. Princeton University Press, 2019.[17] Rene Carmona, Fran¸cois Delarue, and Daniel Lacker. Mean field games oftiming and models for bank runs.
Applied Mathematics & Optimization , 76(1):217–260, 2017.[18] Ren´e Carmona, Fran¸cois Delarue, et al.
Probabilistic Theory of Mean FieldGames with Applications I-II . Springer, 2018.[19] Julien Claisse, Zhenjie Ren, and Xiaolu Tan. Mean field games with branching. arXiv preprint arXiv:1912.11893 , 2019.
20] Michael G Crandall and Pierre-Louis Lions. Viscosity solutions of hamilton-jacobi equations.
Transactions of the American mathematical society , 277(1):1–42, 1983.[21] Mari´an Fabian and Catherine Finet. On stegall’s smooth variational principle.
Nonlinear Analysis: Theory, Methods & Applications , 66(3):565–570, 2007.[22] Diogo A Gomes and Stefania Patrizi. Weakly coupled mean-field game sys-tems.
Nonlinear Analysis , 144:110–138, 2016.[23] Daniel Lacker. A general characterization of the mean field limit for stochasticdifferential games.
Probability Theory and Related Fields , 165(3-4):581–648,2016.[24] Daniel Lacker. On the convergence of closed-loop nash equilibria to the meanfield game limit. arXiv preprint arXiv:1808.02745 , 2018.[25] Jean-Michel Lasry and Pierre-Louis Lions. Une classe nouvelle de probl`emessinguliers de contrˆole stochastique.
Comptes Rendus de l’Acad´emie desSciences-Series I-Mathematics , 331(11):879–885, 2000.[26] Jean-Michel Lasry and Pierre-Louis Lions. Mean field games.
Japanese Jour-nal of Mathematics , 2(1):229–260, 2007.[27] Pierre-Louis Lions. Cours au college de france. , 2011.[28] Pierre-Louis Lions. Cours au college de france. , 2019,2018.[29] Marcel Nutz. A mean field game of optimal stopping.
SIAM Journal onControl and Optimization , 56(2):1206–1221, 2018.[30] Marcel Nutz, Jaime San Martin, Xiaowei Tan, et al. Convergence to themean field game limit: a case study.
The Annals of Applied Probability , 30(1):259–286, 2020.[31] Olga Oleinik.
Second-order equations with nonnegative characteristic form .Springer Science & Business Media, 2012.[32] Alessio Porretta. Weak solutions to fokker–planck equations and mean fieldgames.
Archive for Rational Mechanics and Analysis , 216(1):1–62, 2015.[33] Charles Stegall. Optimization of functions on certain subsets of banach spaces.
Mathematische Annalen , 236(2):171–176, 1978.[34] Charles Stegall. Optimization and differentiation in banach spaces.
LinearAlgebra and Its Applications , 84:191–211, 1986.
Appendix A. Two maximum principle results
Lemma A.1.
Let φ : (0 , ∞ ) × O d → R be a smooth solution of (98) ∂ t φ + F ( x, φ ) ·∇ x φ − d X i =1 σ ( x i ) ∂ ii φ + λ ( φ − φ ( T x )) ≥ in (0 , ∞ ) × O d , φ (0 , x ) ≥ in O d , where σ is a positive function with σ (0) = 0 and T : O d → O d . Under hypothesis1 φ is a positive function. roof. We reason by contradiction. Arguing as in the proof of lemma 3 in [9], wecan assume without loss of generality that there exists ( t , x ) ∈ (0 , ∞ ) × O d suchthat(99) ∂ t φ ( t , x ) < φ ( t , x ) = 0 ≤ φ ( t , y ) , ∀ y ∈ O d ; P di =1 σ ( x i ) ∂ ii φ ( t , x ) ≥ F ( x , φ ) · ∇ x φ ( t , x ) ≥ . Thus we conclude to a contradiction by evaluating the PDE satisfied by φ at( t , x ). Hence φ is positive. (cid:3) Lemma A.2.
Let φ : O d → R be a smooth solution of (100) rφ + F ( x, φ ) ·∇ x φ − d X i =1 σ ( x i ) ∂ ii φ + λ ( φ − φ ( T x )) ≥ in (0 , ∞ ) × O d , φ (0 , x ) ≥ in O d , where σ is a positive function with σ (0) = 0 and T : O d → O d . Under hypotheses1 and 2, φ is a positive function.Proof. Consider R sufficiently large and x := argmin { φ ( x ) | x ∈ B R } . Assumptions1 and 2 precisely state that − F ( x, p ) points inward B R for any x ∈ ∂B R , p ∈ R d .Thus evaluating the PDE satisfied by φ at x we deduce that φ ( x ) ≥
0, hencethat φ is a positive function. (cid:3) Appendix B. Finite state MFG on the orthant
In this section we consider a problem which arises from the discretization of acontinuous problem, mainly to highlight the links between a master equation on O d and on the simplex. For other examples of master equations in finite statespace, we refer to [3, 10]. Namely we are interested in the case in which G is givenby(101) G i ( x, p ) = f i ( x ) − X j = i H (( p j − p i ) − ); ( x, p ) ∈ ( R + ) d × R d ;where H : R → R is a function with quadratic growth whose second derivativeis bounded from above and from below by a non negative constant. We thendefine F ( x, p ) = − D p G ( x, p ) x . The master equation in such a cases is the PDE ofunknown U given by(102) ∂ t U + ( F ( x, U ) · ∇ x ) U = G ( x, U ) in (0 , T ) × R d ;where for the sake of clarity we omit terms modeling common noise, such termscould be treated following the same technique as in the previous sections. Obvi-ously such a master equation is associated to games in which the number of players s conserved, i.e.(103) X i F i = 0 . Let us consider a reduction of (102). We define V : (0 , T ) × R d − → R d − by :(104) V i ( t, z ) = U i ( t, φ ( z )) − U d ( t, φ ( z ));for 1 ≤ i ≤ d , where φ : R d − → R is defined by φ ( z ) = ( z , z , ..., z d − , − P i z i ).The natural equation to consider for V is(105) ∂ t V + ( ˜ F ( z, V ) · ∇ z ) V = ˜ G ( z, V ) in (0 , ∞ ) × Σ d ;where Σ d := { x ∈ O d − | x + ... + x d − = 1 } and ˜ G and ˜ F are defined for i ∈{ , ..., d − } by(106) ( ˜ G i ( z, V ) = G i ( φ ( z ) , ψ ( V )) − G d ( φ ( z ) , ψ ( V )) , ˜ F i ( z, V ) = F i ( φ ( z ) , ψ ( V )) , where ψ ( p ) = ( p , p , ..., p d − ,
0) for p ∈ R d − . The following result then easilyfollows. Proposition B.1.
Let us consider two functions U : (0 , ∞ ) × R d → R d and V : (0 , ∞ ) × R d − → R d − which satisfy (104). Then if U solves (102), V solves(105).Moreover, if x → ( f i ( x )) ≤ i ≤ d is monotone and H is convex, then both ( G, F ) and ( ˜ G, ˜ F ) are monotone.are monotone.