Socio-economic applications of finite state mean field games
aa r X i v : . [ m a t h . A P ] M a r Socio-economic applications of finite state meanfield games
Diogo Gomes, Roberto M. Velho, Marie-Therese Wolfram
Abstract
In this paper we present different applications of finite state mean fieldgames to socio-economic sciences. Examples include paradigm shifts in thescientific community or the consumer choice behaviour in the free market.The corresponding finite state mean field game models are hyperbolicsystems of partial differential equations, for which we present and validatedifferent numerical methods. We illustrate the behaviour of solutions withvarious numerical experiments, which show interesting phenomena likeshock formation. Hence we conclude with an investigation of the shockstructure in the case of two-state problems.
Mean field games have become a powerful mathematical tool to model the dy-namics of agents in economics, finance and the social sciences. Different set-tings have been considered in the literature such as discrete and continuousin time or finite and continuous state space. Originally finite state mean fieldgames, see [Lio11, GMS10, Gu´e11a, Gu´e11b, GMS13, FG13], were studied asan attempt to understand the more general continuous state problems intro-duced by Lasry & Lions in [LL06a, LL06b, LL07] as well as by Huang et al. in[HMC06, HCM07]. For additional information see also the recent surveys suchas [LLG10, Car11, Ach13, GS13, BFY13].In this paper we apply finite state mean field games to two classical problemsin socio-economic sciences: consumer choice behaviour and paradigm shifts ina scientific community, see [BD14]. These mean field game models give riseto systems of hyperbolic partial differential equations, for which we develop anew numerical method. The mathematical modelling is based on the followingsituation. Let us consider a system of N + 1 identical players or agents, whichcan switch between d ∈ N different states. Each player is in a state i ∈ I = { , . . . , d } and can choose a switching strategy to any other state j ∈ I . Theonly information available to each player, in addition to its own state, is thenumber n j of players he/she sees in the different states j ∈ I . The fraction ofplayers in each state i ∈ I is denoted by θ i = n i N and we define the probabilityvector θ = ( θ , . . . , θ d ), θ ∈ P ( I ) = { θ ∈ R d : P i θ i = 1 , θ i ≥ } , whichencodes the statistical information about the ensemble of players. Each playerfaces an optimisation problem over all possible switching strategies. Here thekey question is the existence of a Nash equilibrium and to determine the limitas the number of players tends to infinity. Gomes et al. showed in [GMS13]1hat the N + 1 player Nash equilibrium always exists.Next we consider the limit N → ∞ . Gomes et al. proved in [GMS13] thatat least for short time the value U i = U i ( θ, t ) for a player in state i , whenthe distribution of players among the different states is given by θ , satisfies thehyperbolic system − U it ( θ, t ) = X j ∈I g j ( U, θ ) ∂ θ j U i ( θ, t ) + h ( U, θ, i ) , U ( θ, T ) = ψ ( θ ) . (1)Here U i : P ( I ) × [0 , T ] → R , g : R d × P ( I ) → R d , h : R d × P ( I ) × I → R ,and ψ : P ( I ) → R d , and ∂ θ j denotes the partial derivative with respect to thevariable θ j .In general first order hyperbolic equations do not admit smooth solutions andone needs to consider an appropriate notion of solution. Adequate definitionsof solutions are well known for conservation laws and equations which admita maximum principle, e.g. Hamilton-Jacobi equations. Up to now the appro-priate notion of solutions for (1), which encodes the mean field limit, is notclear. In this paper we present a numerical method, which is based on the Nashequilibrium equations for N + 1 agents. Therefore these equations will, in caseof convergence, automatically yield the appropriate limit. In particular, thanksto the results in [GMS13], convergence always holds for short time.For certain classes of finite state mean field games, called potential mean fieldgames, system (1) can be regarded as the gradient of a Hamilton-Jacobi equa-tion, see [Lio11, GMS13]. We use this property to validate the presented nu-merical method. Our computational experiments show the expected formationof shocks. We analyse these shock structures, by introducing an auxiliary con-servation law. This allows us to derive a Rankine-Hugoniot condition thatcharacterises the qualitative behaviour of such systems.This paper is organised as follows: we start section 2 by recalling the N + 1player model, which gives rise both to (1) and to the numerical method pre-sented here. Section 3 focuses on two-state mean field games, the numericalimplementation of (1) and our applications in socio-economic sciences. We il-lustrate the behaviour of the various model in section 4. In section 5 we brieflyanalyse the shock structure for two-state mean field games. We start with a more detailed presentation of finite state mean field games andthe formal derivation of (1). N + 1 player problem We consider a system of N + 1 identical players or agents. We fix one of them,called the reference player , and denote by i t its state at time t . All other playerscan be in any state j ∈ I at each time t . We denote by n tj the number of players(distinct from the reference player) at state j at time t , and n t = ( n t , . . . , n td ).Players change their states according to two mechanisms: either by a Markovianswitching rate chosen by the player, or by interactions with the other players.2e assume for the moment that all players (except the reference player) havechosen an identical Markovian strategy β ( n, i, t ) ∈ ( R +0 ) d . The reference playeris allowed to choose a possibly different strategy α ( n, i, t ) ∈ ( R +0 ) d . Giventhese strategies, the joint process ( i t , n t ) is a time inhomogeneous Markov chain(neither n t nor i t are Markovian when considered separately). The generator ofthis process can be written as Aϕ in = A α ϕ in + A β ϕ in + A ω ϕ in , where the definition of each term will be given in what follows. Let e k be the k − th vector of the canonical basis of R d and e jk = e j − e k . Then A α ϕ in = X j ∈I α j (cid:0) ϕ jn − ϕ in (cid:1) ,A β ϕ in = X j,k ∈I γ n,iβ,jk (cid:16) ϕ in + e jk − ϕ in (cid:17) , and A ω ϕ in = X j,k ∈I ω jk n j n k N (cid:16) ϕ in + e jk + ϕ in + e kj − ϕ in (cid:17) + X j ∈I ω ij n j N (cid:16) ϕ jn + ϕ in + e ij − ϕ in (cid:17) . The terms A α and A β correspond to the transitions due to the switching strate-gies α and β . Select one of the players distinct from the reference player. Denoteby k t its position at time t , and call m t the vector m t = n t + e i t k t , the processthat records the number of other players in any state from the point of view ofthis player. Suppose further that there are no interactions ( A ω = 0). Then for j = k , we have P (cid:16) k t + δ = j k m t = m, k t = k (cid:17) = β j ( m, k, t ) δ + o ( δ ) . Assuming symmetry and independence of transitions from any state k to a state j , k = j , we have P (cid:16) n t + δ = n + e jk k n t = n, i t = i (cid:17) = γ n,iβ,jk ( t ) δ + o ( δ ) , where the transition rates of the process n t are given by γ n,iβ,jk ( t ) = n k β j ( n + e ik , k, t ) . (2)Similarly, the reference player switching probabilities are P (cid:16) i t + δ = j | i t = i, n t = n (cid:17) = α j ( n, i, t ) δ + o ( δ ) . The transitions between different states due to interactions give rise to the term A ω . Its particular structure comes from the assumption that any two playerswith distinct states j and k can meet with rate ω jk N (with ω kj = ω jk ≥ j or state k (with probability respectively).We assume that all players have the same running cost determined by a function c : I × P ( I ) × ( R +0 ) d → R as well as an identical terminal cost ψ ( θ ), which is3ipschitz continuous in θ . The running cost c ( i, θ, α ) depends on the state i of the player, the mean field θ , that is the distribution of players amongstates, and on the switching rate α . As in [GMS13], we suppose that c isLipschitz continuous in θ with a Lipschitz constant (with respect to θ ) boundedindependently of α . Let the running cost c be differentiable with respect to α ,and ∂c∂α ( i, θ, α ) be Lipschitz with respect to θ , uniformly in α . We assume thatfor each i ∈ I , the running cost c ( i, θ, α ) does not depend on the i -th coordinate α i of α . Furthermore we make the additional assumptions on c :(A1) For any i ∈ I , θ ∈ P ( I ), α, α ′ ∈ ( R +0 ) d , with α j = α ′ j , for some j = i , c ( i, θ, α ′ ) − c ( i, θ, α ) ≥ ∇ α c ( i, θ, α ) · ( α ′ − α ) + γ k α ′ − α k . (3)(A2) The function c is superlinear on α j , j = i , that is,lim α j →∞ c ( i, θ, α ) k α k → ∞ . Let us fix a reference player and set the Markovian strategy β for the remaining N players. The objective of the reference player is to minimise its total cost,whose minimum over all Markovian strategies α is given by u i,βn ( t ) = inf α E β,α ( i t , n t )=( i,n ) "Z Tt c (cid:16) i s , n s N , α ( s ) (cid:17) ds + ψ i T (cid:16) n T N (cid:17) . (4)We define ∆ i ϕ n ( t ) = (cid:0) ϕ n ( t ) − ϕ in ( t ) , . . . , ϕ dn ( t ) − ϕ in ( t ) (cid:1) . The generalised Leg-endre transform of c is given by h ( z, θ, i ) = min µ ∈ ( R +0 ) d c ( i, θ, µ ) + µ · ∆ i z. (5)Note that h only depends on the differences between coordinates of the variable z , that is, if ∆ i z = ∆ i ˜ z then h ( z, θ, i ) = h (˜ z, θ, i ).The function u i,βn is the solution to the ODE − ∂u i,βn ∂t = h (cid:16) u i,βn , nN , i (cid:17) + A β u i,βn + A ω u i,βn . Next we define, for j = i , α ∗ j ( z, θ, i ) = arg min µ ∈ ( R +0 ) d c ( i, θ, µ ) + µ · ∆ i z. (6)If h is differentiable, for j = i , α ∗ j (∆ i z, θ, i ) = ∂h (∆ i z, θ, i ) ∂z j . (7)For convenience and consistency with (7), we require X j ∈I α ∗ j ( z, θ, i ) = 0 . (8)4hen the optimal strategy for the reference player is given by¯ α ( n, i, t ) = α ∗ (cid:16) ∆ u in , nN , i (cid:17) . We say that a strategy β is a Nash equilibrium if the optimal response of thereference player is β itself, i.e., β = ¯ α . Thus setting u in = u i, ¯ αn we have the Nashequilibrium equation for the value function − ∂u in ∂t = h (cid:16) u in , nN , i (cid:17) + X j,k ∈I γ n,ijk (cid:16) u in + e jk − u in (cid:17) (9)+ X j,k ∈I ω jk n j n k N (cid:16) u in + e jk + u in + e kj − u in (cid:17) + X j ∈I ω ij n j N (cid:16) u jn + u in + e ij − u in (cid:17) , where we define γ n,ijk ( t ) = n k ¯ α j ( n + e ik , k, t ) . (10) Now we investigate the asymptotic behaviour as N → ∞ of the N + 1 playerdynamics (9). For that we suppose there is a smooth function U : P ( I ) × [0 , T ] → R d such that u in ( t ) = U i (cid:16) nN , t (cid:17) . Then we have the following expansions: A ¯ α u in = P j,k ∈I θ k h(cid:16) ∂ θi − ∂ θk N (cid:17) α ∗ j (∆ k U, θ, k ) i (cid:20)(cid:18) ( ∂ θ j − ∂ θ k ) + ( ∂ θj − ∂ θk ) N (cid:19) U i (cid:21) + O (cid:0) N (cid:1) , A ω u in = X j,k ∈I ω jk N θ j θ k (cid:16) ∂ θ j θ j U i + ∂ θ k θ k U i − ∂ θ j θ k U i (cid:17) + X j ∈I ω ij ( U j − U i )+ X j ∈I ω ij θ j (cid:0) ∂ θ i U i − ∂ θ j U i (cid:1) + O (cid:18) N (cid:19) . We observe that for fixed j and k the operators θ k α ∗ j (cid:0) ∂ θ j − ∂ θ k (cid:1) , and ω jk θ j θ k (cid:0) ∂ θ j − ∂ θ k (cid:1) are degenerate elliptic operators. The first one is degen-erate because θ k , α ∗ j ≥
0; the second because ω jk ≥
0. Hence their sum is alsoa degenerate operator. Therefore the combination of the second order terms inthe expansion of A ¯ α and A ω can be written as X l,m ∈I b lm ∂ θ l θ m = X j,k ∈I θ k α ∗ j + 2 ω jk θ j θ k (cid:0) ∂ θ j − ∂ θ k (cid:1) , for a suitable non-negative matrix b . We conclude that (9) can be formallyapproximated by the parabolic system − U it ( θ, t ) = X j ∈I g Nj ( U, ∂ θ U, θ, i ) ∂ θ j U i + h ( U, θ, i ) + 1 N X l,m ∈I b lm ( U, θ ) ∂ θ l θ m U i , (11)5ith suitable g N : R d × R d × P ( I ) × I → R d . Furthermore g N converges locallyuniformly in compacts to g j ( U, θ ) = X i ∈I θ i α ∗ j ( U, θ, i ) . (12)This implies that the limit of (11) is (1), which does not depend on the inter-action between players ( ω jk ). Note that X j ∈I g j ( U, θ ) = X j ∈I X i ∈I θ i α ∗ j ( U, θ, i ) = X i ∈I θ i X j ∈I α ∗ j ( U, θ, i ) = 0 , (13)since P j ∈I α ∗ j ( U, θ, i ) = 0, from (8). Additionally, g j ( U, θ, i ) = g j (∆ i U, θ, i ) , (14)using in (12) the equation (6). Next we consider a special class of mean field games, in which system (1) canbe written as the gradient of a Hamilton-Jacobi equation. Suppose that h ( u, θ, i ) = ˜ h ( u, i ) + f ( i, θ ) , i ∈ I , (15)and f ( i, θ ) = ∂ θ i F ( θ ), for some potential F : R d → R . We set H ( u, θ ) = X k ∈I θ k ˜ h (∆ k u, k ) + F ( θ ) . (16)Let Ψ : R d → R be a continuous function and consider a smooth enoughsolution Ψ : R d × [0 , T ] → R to the Hamilton-Jacobi equation − ∂ Ψ( θ, t ) ∂t = H ( ∂ θ Ψ , θ ) , Ψ( θ, T ) = Ψ ( θ ) . (17)Note that θ ∈ R d . In some cases it is possible to reduce the dimensionality atthe price of introducing suitable boundary conditions (see section 4-(4.2)). Thisreduction will be used in the applications presented later.Set U j ( θ, t ) = ∂ θ j Ψ( θ, t ). If we differentiate (17) with respect to θ i we obtain − U it = X j ∈I ∂ u j H ( U, θ ) ∂ θ i U j + ˜ h (∆ i U, i ) + ∂ θ i F. The first term on the right hand side can be written as X j ∈I ∂ u j H ( U, θ ) ∂ θ i U j = X k,j ∈I θ k ∂ u j ˜ h (∆ k U, k ) ∂ θ i U j = X j ∈I g j ( U, θ ) ∂ θ j U i , taking into account the identity ∂ θ i U j = ∂ θ j U i . From this we get that − U it = X j ∈I g j ( U, θ ) ∂ θ j U i + ˜ h (∆ i U, i ) + ∂ θ i F, U i is indeed a solution of (1).Potential mean field games have remarkable properties and connections to cal-culus of variations. For instance, long time convergence properties of theseproblems can be addressed through Γ-convergence techniques, see for instance[FG13]. In this section we present several applications of finite state mean field games tosocio-economic sciences. In order to keep the presentation simple we consideronly two-state problems. We start by stating the explicit equations, whereagents can choose between two options. The classical examples discussed hereare the consumer choice behaviour and a paradigm shift model in the scientificcommunity. Note that the number of choices can be increased, but we focus ontwo-state applications for reasons of clarity and readability.
Consider a two-state mean field game, where the fraction of players in eitherstate, 1 or 2, is given by θ i , i = 1 , θ + θ = 1, and θ i ≥
0. Since the limitequation (1) does not depend on the interactions (although the N + 1 playermodel does), we set ω = 0. Note that ω = 0 would result in different numericalmethods (and potentially different solutions) for (1). We suppose further thatthe running cost c = c ( i, θ, µ ) in (4) depends quadratically on the switching rate µ , i.e. c ( i, θ, µ ) = f ( i, θ ) + c ( i, µ ) , with c ( i, µ ) = 12 X j = i µ j . (18)Then h ( z, θ,
1) = f (1 , θ ) − (cid:0) ( z − z ) + (cid:1) and h ( z, θ,
2) = f (2 , θ ) − (cid:0) ( z − z ) + (cid:1) . (19)The optimal switching rate α ∗ is given by: α ∗ ( z, θ,
1) = arg min µ ∈ R , µ ≥ (cid:2) f (1 , θ ) + 12 µ + ( µ µ ) · (cid:0) z − z (cid:1)(cid:3) ⇒ α ∗ ( z, θ,
1) = ( z − z ) + ,α ∗ ( z, θ,
2) = arg min µ ∈ R , µ ≥ (cid:20) f (2 , θ ) + 12 µ + ( µ µ ) · (cid:0) z − z (cid:1)(cid:21) ⇒ α ∗ ( z, θ,
2) = ( z − z ) + . Since α ∗ ( U, θ,
1) = − (cid:0) U − U (cid:1) + ; α ∗ ( U, θ,
1) = (cid:0) U − U (cid:1) + ; α ∗ ( U, θ,
2) = (cid:0) U − U (cid:1) + ; α ∗ ( U, θ,
2) = − (cid:0) U − U (cid:1) + ,
7e conclude from (12) that g ( U, θ ) = − θ (cid:0) U − U (cid:1) + + θ (cid:0) U − U (cid:1) + , (20) g ( U, θ ) = θ (cid:0) U − U (cid:1) + − θ (cid:0) U − U (cid:1) + = − g ( U, θ ) . Note that if the function f is a gradient field, i.e. f = ∇ F , the two-stateproblem is a potential mean field game, cf. section 2-(2.3). In this case, for p = ( p , p ) ∈ R , (16) is given by H ( p, θ ) = F ( θ ) − θ (( p − p ) + ) + θ (( p − p ) + ) . (21)The above calculations allow us to introduce a numerical method based on thetwo-state mean field model for N + 1 players. Let n i , i = 1 , i (as seen by the reference player excluding itself)and N = n + n . The vector n gives the number of players in each state, i.e. n = ( n , n ) = ( n , N − n ). As in (10) we have γ n, jl = n l α ∗ j (cid:18) ∆ l u n + e l , n + e l N , l (cid:19) and γ n, jl = n l α ∗ j (cid:18) ∆ l u n + e l , n + e l N , l (cid:19) . Then equation (9) for the value function u in reads as − du n dt = X j,l =1 γ n, jl (cid:16) u n + e jl − u n (cid:17) + h (cid:16) ∆ u n , nN , (cid:17) , − du n dt = X j,l =1 γ n, jl (cid:16) u n + e jl − u n (cid:17) + h (cid:16) ∆ u n , nN , (cid:17) , (22)which can be rewritten as − du n dt = ( N − n ) α ∗ (cid:0) ∆ u n + e , n + e N , (cid:1) (cid:0) u n + e − u n (cid:1) + n α ∗ (cid:0) ∆ u n , nN , (cid:1) (cid:0) u n + e − u n (cid:1) + h (cid:0) ∆ u n , nN , (cid:1) , − du n dt = ( N − n ) α ∗ (cid:0) ∆ u n , nN , (cid:1) (cid:0) u n + e − u n (cid:1) + n α ∗ (cid:0) ∆ u n + e , n + e N , (cid:1) (cid:0) u n + e − u n (cid:1) + h (cid:0) ∆ u n , nN , (cid:1) . (23)Since θ + θ = 1 we use θ = ( ζ, − ζ ), for ζ ∈ [0 , , N equidistant subintervals and define ζ k = kN , 0 ≤ k ≤ N, k ∈ N . Thevariable ζ k corresponds to the fraction of players in state 1. Then the fractionof players in state 2 is given by 1 − ζ k = N − kN . Consequently (23) takes the form − du k dt = N (1 − ζ k ) (cid:0) u k +1 − u k +1 (cid:1) + (cid:0) u k +1 − u k (cid:1) + N ζ k (cid:0) u k − u k (cid:1) + (cid:0) u k − − u k (cid:1) + f (1 , ζ k ) − (cid:16)(cid:0) u k − u k (cid:1) + (cid:17) , − du k dt = N (1 − ζ k ) (cid:0) u k − u k (cid:1) + (cid:0) u k +1 − u k (cid:1) + N ζ k (cid:0) u k − − u k − (cid:1) + (cid:0) u k − − u k (cid:1) + f (2 , − ζ k ) − (cid:16)(cid:0) u k − u k (cid:1) + (cid:17) . (24)8ote that in system (24) the terms ζ k and (1 − ζ k ) vanish for k = 0 and k = N ,thus no particular care has to be taken concerning the ghost points at ζ N +1 and ζ − . This is the discrete analogue to not imposing boundary conditions on (1).A similar situation occurs in state constrained problems for Hamilton-Jacobiequations. According to Kuhn, see [Kuh70], a paradigm shift corresponds to a changein a basic assumption within the ruling theory of science. Classical cases ofparadigm shifts are the transition from Ptolemaic cosmology to Copernican one,the development of quantum mechanics which replaced classical mechanics onthe microscopic scale or the acceptance of Mendelian inheritance as opposed toPangenesis. Bensancenot and Dogguy modelled a paradigm shift in a scientificcommunity by a two-state mean field game approach and analysed the compe-tition between two different scientific hypothesis, see [BD14]. In our examplewe consider a simpler model, but follow their general ideas and assumptions.Let us consider a scientific community with N researchers working on two dif-ferent hypothesis. Each researcher working on paradigm i , i = 1 ,
2, wants tomaximise his/her productivity measured by a cost function of the form (18).Here the function f = f ( i, θ ) corresponds to the productivity of a researcherworking on paradigm i , and c ( i, µ ) = − P j = i µ j to the cost of switching tothe other objective. Note the negative sign of the switching costs, since agentswant to maximise their productivity. We assume that the productivity is di-rectly related to the number of researchers working on the paradigm, since forexample more scientific activities like conference and collaborations. In the caseof two different fields, θ gives the fraction of researchers working on paradigm1 and θ = 1 − θ on paradigm 2. We choose the functions f ( i, θ ), i = 1 ,
2, ofthe form f (1 , θ ) = [ a θ r + (1 − a ) (1 − θ ) r ] r , (25) f (2 , θ ) = [ a (1 − θ ) r + (1 − a ) θ r ] r . These functions are called productivity functions with constant elasticity of sub-stitution and are commonly used in economics to combine two or more produc-tive inputs (in our case scientific activities in the different fields) to an outputquantity. The constant r ∈ R , r = 0, denotes the elasticity of substitution , andit measures how easy one can substitute one input for the other. The constants a i ∈ [0 ,
1] measure the dependence of paradigm i with respect to the other. If a i is close to one, the field is more autonomous and little influenced by the activityin the other field. Consumer choice models relate preferences to consumption expenditure. Weconsider two choices of consumption goods and denote by θ the fraction ofagents consuming good 1 and by θ = 1 − θ the fraction consuming good 2.We assume that the price of a good is strongly determined by the consumption9ate, in particular we choose, for i = 1 , f ( i, θ ) = θ − ηi − − η + s i , η > , η = 1 , ln( θ i ) + s i , η = 1 , (26)where s i ∈ R + corresponds to the minimum price of the good. In economicliterature the function f is called the isoelastic utility function . We illustrate the behaviour of the discrete system (24) with several examples.Let N = 100, i.e. the interval [0 ,
1] is discretized into 100 equidistant intervals.Each grid point corresponds to the the percentage of players being in state 1.System (24) is solved using an explicit in time discretization with time steps ofsize ∆ t = 10 − . In all examples in this section the terminal time T is set to T = 10 if not stated otherwise. Example I (Shock formation):
In this first example we would like to il-lustrate the formation of shocks, a phenomena well known for Hamilton-Jacobiequations. We choose a terminal cost of the form u ( θ,
10) = θ −
12 and u ( θ,
10) = θ − , a running cost as in (18) with f (1 , θ ) = 1 − θ and f (2 , θ ) = 1 − θ = θ .Figure 1 clearly illustrates the formation of a shock for smooth terminal data.This shock is also evident when we consider the difference u − u of the utilities.This difference is a relevant variable in this problem, since both g and h , givenby (14) and (5) respectively, depend only on the difference between the utilities.This structure will be explored in more detail in section 5. θ u ( θ ) at t=0u ( θ ) at T=10u ( θ ) at t=0u ( θ ) at T=10 (a) Utility functions u and u . θ t=10t=9t=8t=7t=6t=0 (b) Difference u − u at different times. Figure 1: Example I - Shock formation.10 xample II (Paradigm shift):
In this example we illustrate the outcome ofa two-state mean field game modelling a paradigm shift (section 3-(3.2)) withina scientific community. Note that we use the negative cost functional in thisexample, since we always consider minimisation problems. The terminal utilitiesare given by u ( θ, T = 10) = 1 − θ and u ( θ, T = 10) = θ , (27)and the parameters in (25) are set to a = , a = , r = . In figure 2 θ u ( θ ) at t=0u ( θ ) at T=10u ( θ ) at t=0u ( θ ) at T=10 Figure 2: Example II - paradigm shiftwe observe the paradigm shift within the scientific community. At T = 10 theoptimal states are θ = 1 and θ = 1 since the functions u and u take theirminimum value at these points respectively. In figure 2 we observe that this isnot the case at t = 0. Here u takes its minimum value at θ = 0, i.e. paradigm1 is not popular any more. Example III (Consumer choice):
In our final example we consider theconsumer choice behaviour, see section 3-(3.3). We set the final utility functionto u ( θ,
10) = 1 − θ and u ( θ,
10) = θ . At time T = 10 the utility functions take their minimum value at θ = 1 and θ = 0, i.e. their minimum corresponds to the case that either all of them chooseproduct 1 or product 2, respectively. Figure 3 illustrates the utility functionsfor two sets of parameters, namely η = 0 . , s = 0 . , s = 0 . η = 1 , s = 0 . , s = 0 . . We observe for the second set of parameters that u takes its minimum valueon the interval θ ∈ [0 . , θ ≈ .
65 in both utilities indicatesthe existence of a critical acceptance rate. If less than 65% of the consumersbuy product 1, the price is increasing and the product will not be competitive.11 θ u ( θ ) at t=0u ( θ ) at T=10u ( θ ) at t=0u ( θ ) at T=10 (a) η = 0 . s = 0 .
075 and s = 0 . θ u ( θ ) at t=0u ( θ ) at T=10u ( θ ) at t=0u ( θ ) at T=10 (b) η = 1, s = 0 . s = 0 . Figure 3: Example III - consumer choice
In order to validate our methods, we consider Example I, that can be writtenas a potential mean field game where F and H in (21) are given by F ( θ , θ ) = θ θ ,H ( p , p , θ , θ ) = − θ (( p − p ) + ) − θ (( p − p ) + ) + F ( θ , θ ) . Then we compare the numerical simulations of (24) with the ones for the cor-responding Hamilton-Jacobi equation as we explain in what follows.For i = 1 ,
2, set f ( i, θ ) = ∂∂θ i F ( θ , θ ) and Ψ ( θ , θ ) = 12 (cid:18) θ − (cid:19) + 12 (cid:18) θ − (cid:19) . In order to simplify the numerical implementation, we perform a dimensionalityreduction. Define ( θ , θ ) = ( ζ, − ζ ), with ζ ∈ [0 , ζ, t ) = Ψ( ζ, − ζ, t ) . We observe that − ∂ Υ ∂t = ˜ H ( ∂ ζ Υ , ζ ) , (28)where ˜ H ( ∂ ζ Υ , ζ ) = − ζ (cid:2) ( ∂ ζ Υ) + (cid:3) −
12 (1 − ζ ) (cid:2) ( − ∂ ζ Υ) + (cid:3) + F ( ζ, − ζ ) . We use Godunov’s method and an explicit Runge-Kutta method to discretize(28). Particular care has to be taken at the boundary. Since ζ ∈ [0 ,
1] representsthe first component of a probability vector, the natural boundary conditionsfor this problem are state constraints. A possibility to implement this is bysupplementing (28) with large Dirichlet boundary values, i.e.Υ(0 , t ) = Υ(1 , t ) = c D with c D ∈ R + . (29)12 θ dU/d θ at time T=0Difference u −u from Example 1 Figure 4: Derivative of U with respect to θ versus the difference u − u cal-culated in Example I.To implement the Dirichlet boundary conditions we follow the works of Abgralland Waagan, see [Abg03, Waa08]. Again we consider an equidistant discretiza-tion of the interval [0 ,
1] into N subintervals of size ∆ ζ , and we approximate thesolution Υ( ζ, τ ) to (28) by Υ τ ( ζ ) for τ = T − l ∆ t , and ζ = k ∆ ζ , 0 ≤ k ≤ N ; l, k ∈ N . We set Υ T ( ζ ) = Ψ ( ζ, − ζ ). Then, the Godunov scheme can bewritten as Υ τ − ∆ t = Υ τ − ∆ t ˆ H ( δ − ζ Υ τ , δ + ζ Υ τ , ζ ) , (30)where δ + ζ and δ − ζ are the difference operators δ − ζ Φ( ζ ) = Φ( ζ ) − Φ( ζ − ∆ ζ )∆ ζ and δ + ζ Φ( ζ ) = Φ( ζ + ∆ ζ ) − Φ( ζ )∆ ζ , and ˆ H in (30) is given byˆ H ( α, β, ζ ) = min α ≤ q ≤ β ˜ H ( q, ζ ) , if α ≤ β ;max β ≤ q ≤ α ˜ H ( q, ζ ) , if β ≤ α. At the boundary ζ = 0 ,
1, we setΥ τ − ∆ t (0) = min h Υ τ (0) − ∆ tH − ( δ + ζ Υ τ (0) , , c D i , Υ τ − ∆ t (1) = min h Υ τ (1) − ∆ tH + ( δ − ζ Υ τ (1) , , c D i , where H − ( p, ζ ) = ˆ H (0 , p, ζ ) and H + ( p, ζ ) = ˆ H ( p, , ζ ). Figure 4 shows thederivative of U with respect to θ as well as the difference u − u calculatedfrom (24) at time t = 0. The same spatial and temporal discretization (i.e. N = 100 and ∆ t = 10 − ) was used in both simulations.13 Shock structure for two-state problems
Finally we consider two-state problems and investigate in detail the shock struc-ture. For this purpose we perform a reduction of the dimension (since the keyissues are related to the difference of the values, more than to its proper value)and obtain a hyperbolic scalar equation. Then we introduce a related conserva-tion law, which yields a Rankine-Hugoniot condition for possible shocks. Thisnew formulation allows us to study finite state mean field games from anothernumerical perspective and gives new insights into the shock structure and thequalitative behaviour of solutions to (1).
Let U ( θ, t ) be a C solution to (1) with d = 2. We define w ( ζ, t ) = U ( θ, t ) − U ( θ, t ) , where θ = ( ζ, − ζ ). From (1) we have that (cid:0) U − U (cid:1) t = − g ( U , U , θ , θ ) ∂ θ (cid:0) U − U (cid:1) − g ( U , U , θ , θ ) ∂ θ (cid:0) U − U (cid:1) − h ( U , U , θ , θ ,
1) + h ( U , U , θ , θ , . (31)Then h and g , given by (5) and (14), can be written as h ( U , U , θ , θ , i ) = h ( w ( ζ, t ) , , ζ, − ζ, i ) and g ( U , U , θ , θ ) = g ( w ( ζ, t ) , , ζ, − ζ ). For two-state problems equation (13) gives g = − g . Hence we obtain w t = − g ( w ( ζ, t ) , , ζ, − ζ ) (cid:2) ∂ θ (cid:0) U − U (cid:1) − ∂ θ (cid:0) U − U (cid:1)(cid:3) − h ( w ( ζ, t ) , , ζ, − ζ,
1) + h ( w ( ζ, t ) , , ζ, − ζ, . Define r and q by r ( w ( ζ, t ) , ζ ) = − g ( w ( ζ, t ) , , ζ, − ζ ) ,q ( w ( ζ, t ) , ζ ) = h ( w ( ζ, t ) , , ζ, − ζ, − h ( w ( ζ, t ) , , ζ, − ζ, , and denote ∂w∂ζ by ∂w ( ζ,t ) ∂ζ = (cid:16) ∂∂θ − ∂∂θ (cid:17) (cid:0) U − U (cid:1) | ( ζ, − ζ ) . Then equation (31)for the difference of U i can be written as − w t ( ζ, t ) + r ( w ( ζ, t ) , ζ ) ∂ ζ w ( ζ, t ) = q ( w ( ζ, t ) , ζ ) . (32) Consider the following conservation law associated to (32) on the interval [0 , − P t ( ζ, t ) + ∂ ζ ( r ( w ( ζ, t ) , ζ ) P ( ζ, t )) = 0 , (33)supplemented with the boundary condition P (0 , t ) = P (1 , t ) = 0 for all times t ∈ [0 , T ].If P is a sufficiently smooth solution to (33) and P ( ζ, ≥
0, then the max-imum principle implies that P ( ζ, t ) ≥
0. Furthermore, if R P ( ζ, dζ = 1, wehave that R P ( ζ, t ) dζ = 1. Assuming that P ( ζ,
0) is a probability distribution,we can regard P ( ζ, t ) as a probability distribution on the set P ( I ), I = { , } ,14s we have a natural identification for two-state problems: P ( I ) ≃ [0 , P ( I ). Un-certainty in the initial distribution of the mean field ζ can be encoded in theinitial condition P ( ζ,
0) and propagated through (33).Since equation (33) may not have globally smooth solutions, we use the Rankine-Hugoniot condition to characterise certain possibly discontinuous solutions. Let s : [0 , T ] → [0 ,
1] be a C curve and suppose that P is a C function on both0 < ζ < s ( t ) and s ( t ) < ζ <
1, for t ∈ [0 , T ]. Assume further that (33) holds inthis set. Let B : [0 , × [0 , T ] → R . We denote by [ B ] the jump of B across thecurve s ( t ), that is, [ B ] = B ( s + ( t ) , t ) − B ( s − ( t ) , t ). Equation (33) leads to theRankine-Hugoniot condition of the form[ P ] ˙ s = − [ r P ] . (34)If we start with initial condition P ( ζ,
0) = 1, then the support of P is theclosure of the set of all mean field states which can be reached from some initialmean field state (all possible choices of ζ at time 0). Suppose B = { ( ζ, t ) =( s ( t ) , t ) , ≤ t ≤ T } . Suppose that there is a discontinuity in P at theboundary B of the set P = 0. Then we conclude from the Rankine-Hugoniotcondition that ˙ s = − r. Hence B is a characteristic for (32).Using (33) we can also derive local Lipschitz bounds for the solution to (32): Proposition 5.1.
Let w be a smooth solution to (32) . Then there exists atime t < T and a constant C , depending only the L ∞ norm and the Lipschitzconstant of w ( x, T ) , such that for t ≤ t ≤ T , k ∂ ζ w ( · , t ) k L ∞ ([0 , ≤ C ( t − t ) . Proof.
Let P be a solution to (33) with P (0 , t ) = P (1 , t ) = 0, t ≤ T , P ( ζ, t ) ≥ R P ( ζ, t ) dζ = 1. We differentiate equation (32) with respect to ζ andmultiply it by 2 ∂ ζ w and obtain − ∂ t S + r∂ ζ S = − ∂ ζ r∂ ζ w − ∂ w rS∂ ζ w + 2 ∂ w qS + 2 ∂ ζ q∂ ζ w, using that S ( ζ, t ) = ( ∂ ζ w ( ζ, t )) . Then we can deduce the following estimate: − ddt Z S ( ζ, t ) P ( ζ, t ) dζ ≤ Z ( − ∂ ζ r∂ ζ w − ∂ w rS∂ ζ w + ∂ w qS + ∂ ζ q∂ ζ w ) P ( ζ, t ) dζ ≤ C Z S / P ( ζ, t ) dζ + C , where we use the fact that w is bounded, see [GMS13], in the last step. Then Z S ( ζ, t ) P ( ζ, t ) dζ ≤ C + Z Tt k S ( · , s ) k / ∞ ds + k S ( · , T ) k ∞ . Taking P ( ζ, t ) to be an arbitrary probability measure on [0 ,
1] this yields that k S ( · , t ) k ∞ ≤ C + Z Tt k S ( · , s ) k / ∞ ds + k S ( · , T ) k ∞ . This estimate does not give global bounds, but a nonlinear version of Gronwall’sinequality yields the existence of t . 15 .3 Numerical analysis of the shock structure Finally we discuss the particular formulation of equations (32) and (33) as wellas the numerical realisation of example I presented in section 4. Equation (32)can be written as − w t ( ζ, t ) = (1 − ζ ) | w | − w ∂ ζ w ( ζ, t )+ 12 | w | w − [ f (1 , ζ, − ζ ) − f (2 , ζ, − ζ )] . (35)using that h and g are given by (19) and (20) respectively. Similarly, (33) takesthe form − P t ( ζ, t ) + ∂ ζ (cid:20) − (1 − ζ ) | w | − w P ( ζ, t ) (cid:21) = 0 . (36)Note that the difference w = u − u has to satisfy w (0 , t ) ≥ w (1 , t ) ≤ ζ = 0 in a situation where state 1 would be preferred to state2 (and analogously for ζ = 1). Therefore equation (36) does not require anyboundary conditions since for ζ = 0 and ζ = 1 the advection term vanishes, i.e., (1 − ζ ) | w |− w = 0 . We simulate (36) using a semidiscrete central upwind scheme introduced byKurganov et al., see [KNP01], and use the same parameters as in Section 4.The initial datum P ( ζ,
0) is set to P ( ζ,
0) = 1 for ζ ∈ (0 ,
1) and P ( ζ,
0) = 0 for ζ ∈ { , } . Due to the vanishing advection terms at ζ = 0 ,
1, we set homogeneous Dirichletboundary conditions for P , that is P ( ζ, t ) = 0 if ζ ∈ { , } . We choose anequidistant discretization of N = 125 points and times steps ∆ t = 10 − . Theevolution of P for example I presented in section 3 is illustrated in figure 5. We ξ Figure 5: Evolution of P for Example I discussed in Section 3.observe that the function P does not see the shock in w (see Figure 1), whichis located at ζ = 0 .
5. 16
Conclusions
In this paper we present effective numerical methods for two-state mean fieldgames and discuss a number of illustrative examples in socio-economic sciences.We compare our method with numerical results obtained from classical and wellestablished schemes for Hamilton-Jacobi equations. As presented in the exam-ples, our method captures shocks effectively. We analyse the shock structureusing an associated conservation law and prove a local Lipschitz estimate forthe solutions of (1).Several challenging and interesting mathematical questions will be addressed ina future paper, like the development of numerical schemes based on the dualvariable method and the Lions transversal variable method, see [Lio11].
References [Abg03] R´emi Abgrall. Numerical discretization of boundary con-ditions for first order Hamilton-Jacobi equations.
SIAMJ. Numer. Anal. , 41(6):2233–2261 (electronic), 2003.http://dx.doi.org/10.1137/S0036142998345980.[Ach13] Yves Achdou. Finite difference methods for mean field games.In
Hamilton-Jacobi Equations: Approximations, Numerical Analysisand Applications , pages 1–47. Springer, 2013.[BD14] D Besancenot and H. Dogguy. Paradigm shift: a mean-field game approach.
Bull. Econ. Res. , 138, 2014.http://dx.doi.org/10.1111/boer.12024.[BFY13] Alain Bensoussan, Jens Frehse, and Phillip Yam.
Meanfield games and mean field type control theory . SpringerBriefs in Mathematics. Springer, New York, 2013.http://dx.doi.org/10.1007/978-1-4614-8508-7.[Car11] P. Cardaliaguet. Notes on mean-field games. 2011.[FG13] R. Ferreira and D. Gomes. On the convergence of finite state mean-field games through Γ − convergence. Preprint - To appear in Journalof Mathematical Analysis and Applications , 2013.[GMS10] D. Gomes, J. Mohr, and R. R. Souza. Discrete time, finite state spacemean field games.
Journal de Math´ematiques Pures et Appliqu´ees ,93(2):308–328, 2010.[GMS13] D. Gomes, J. Mohr, and R. R. Souza. Continuous time finite statemean-field games.
Appl. Math. and Opt. , 68(1):99–143, 2013.[GS13] D. Gomes and J. Saude. Mean-field games: a brief survey.
To appearDyn. Games Appl. , 2013.[Gu´e11a] O. Gu´eant. An existence and uniqueness result for mean field gameswith congestion effect on graphs.
Preprint , 2011.17Gu´e11b] O. Gu´eant. From infinity to one: The reduction of some mean fieldgames to a global control problem.
Preprint , 2011.[HCM07] M. Huang, P. E. Caines, and R. P. Malham´e. Large-population cost-coupled LQG problems with nonuniform agents:individual-mass behavior and decentralized ǫ -Nash equilib-ria. IEEE Trans. Automat. Control , 52(9):1560–1571, 2007.http://dx.doi.org/10.1109/TAC.2007.904450.[HMC06] M. Huang, R. P. Malham´e, and P. E. Caines. Large pop-ulation stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalenceprinciple.
Commun. Inf. Syst. , 6(3):221–251, 2006.http://projecteuclid.org/getRecord?id=euclid.cis/1183728987.[KNP01] Alexander Kurganov, Sebastian Noelle, and Guergana Petrova.Semidiscrete central-upwind schemes for hyperbolic conservation lawsand Hamilton–Jacobi equations.
SIAM Journal on Scientific Com-puting , 23(3):707–740, 2001.[Kuh70] Thomas S. Kuhn.
The structure of scientific revolutions . Universityof Chicago Press, Chicago, 1970.[Lio11] P.-L. Lions. College de france course on mean-field games. 2007-2011.[LL06a] J.-M. Lasry and P.-L. Lions. Jeux `a champ moyen. I. Le cas station-naire.
C. R. Math. Acad. Sci. Paris , 343(9):619–625, 2006.[LL06b] J.-M. Lasry and P.-L. Lions. Jeux `a champ moyen. II. Horizon finiet contrˆole optimal.
C. R. Math. Acad. Sci. Paris , 343(10):679–684,2006.[LL07] J.-M. Lasry and P.-L. Lions. Mean field games.
Jpn. J. Math. ,2(1):229–260, 2007.[LLG10] J.-M. Lasry, P.-L. Lions, and O. Gu´eant. Mean field games and ap-plications.
Paris-Princeton lectures on Mathematical Finance , 2010.[Waa08] Knut Waagan. Convergence rate of monotone numericalschemes for Hamilton-Jacobi equations with weak boundaryconditions.
SIAM J. Numer. Anal. , 46(5):2371–2392, 2008.http://dx.doi.org/10.1137/050632932.