Mean Field Equilibrium: Uniqueness, Existence, and Comparative Statics
aa r X i v : . [ ec on . T H ] J un Mean Field Equilibrium: Uniqueness, Existence, andComparative Statics ∗ Bar Light † and Gabriel Y. Weintraub ‡ June 5, 2020
Abstract :The standard solution concept for stochastic games is Markov perfect equilibrium(MPE); however, its computation becomes intractable as the number of players in-creases. Instead, we consider mean field equilibrium (MFE) that has been popular-ized in the recent literature. MFE takes advantage of averaging effects in modelswith a large number of players. We make three main contributions. First, our mainresult provides conditions that ensure the uniqueness of an MFE. We believe thisuniqueness result is the first of its nature in the class of models we study. Second, wegeneralize previous MFE existence results. Third, we provide general comparativestatics results. We apply our results to dynamic oligopoly models and to heteroge-neous agent macroeconomic models commonly used in previous work in economicsand operations.
Keywords: Dynamic games; Mean field equilibrium; Uniqueness of equilibrium; Comparative statics;Dynamic oligopoly models; Heterogeneous agent macroeconomic models ∗ The authors wish to thank Aaron Bodoh-Creed, Ramesh Johari, Bob Wilson, three anonymous referees,and the associate and area editors, as well as seminar participants at Stanford and several conferences for theirvaluable comments. The second author thanks Joseph and Laurie Lacob for the support during the 2017-2018and 2018-2019 academic years as a Joseph and Laurie Lacob Faculty Scholar at Stanford Graduate School ofBusiness. † Graduate School of Business, Stanford University, Stanford, CA 94305, USA. e-mail: [email protected] ‡ Graduate School of Business, Stanford University, Stanford, CA 94305, USA. e-mail: [email protected]
Introduction
In this paper we consider a general class of stochastic games in which every player has anindividual state that impacts payoffs. Historically, Markov perfect equilibrium (MPE) has been astandard solution concept for this type of stochastic games (Maskin and Tirole, 2001). However,in realistically-sized applications, MPE suffers from two drawbacks. First, because in MPEplayers keep track of the state of every competitor, the state space grows very quickly as thenumber of players grows, making the analysis and computation of MPE infeasible in manyapplications of practical interest. Second, as the number of players increases, it becomes difficultto believe that players can in fact track the exact state of the other players and optimize theirstrategies accordingly.As an alternative, mean field equilibrium (MFE) has received extensive attention in the recentliterature. In an MFE, each player optimizes her expected discounted payoff, assuming that thedistribution of the other players’ states is fixed. Given the players’ strategy, the distribution ofthe players’ states is an invariant distribution of the stochastic process that governs the states’dynamics. As a solution concept for stochastic games, MFE offers several advantages over MPE.First, because players only condition their strategies on their own state (the competitors’ stateis assumed to be fixed), MFE is computationally tractable. Second, as several of the paperswe cite below prove, due to averaging effects MFE provides accurate approximations of optimalbehavior as the number of players grows. As a result, it provides an appealing behavioral modelin games with many players.MFE models have many applications in economics, operations research, and optimal control;e.g., studies of anonymous sequential games (Jovanovic and Rosenthal, 1988), continuous-timemean field models (Huang et al. (2006) and Lasry and Lions (2007)), dynamic user equilibrium(Friesz et al., 1993), auction theory (Iyer et al. (2014), Balseiro et al. (2015), and Bimpikis et al.(2018)), dynamic oligopoly models (Weintraub et al. (2008) and Adlakha et al. (2015)), hetero-geneous agent macro models (Hopenhayn (1992) and Heathcote et al. (2009)), matching markets(Kanoria and Saban (2019) and Arnosti et al. (2020)), spatial competition (Yang et al., 2018),and evolutionary game theory (Tembine et al., 2009).We provide three main contributions regarding MFE. First, we provide conditions that ensurethe uniqueness of an MFE. This novel result is important because it implies sharp counterfactualpredictions. Second, we generalize previous existence results to a general state space setting.Our existence result includes the case of a countable state space and a countable number ofplayers, as well as the case of a continuous state space and a continuum of players. In addition,we provide novel comparative statics results for stochastic games that do not exhibit strategiccomplementarities.We apply our results to well-known dynamic oligopoly models in which individual statesrepresent the firms’ ability to compete in the market (Doraszelski and Pakes, 2007). MFE and2he related concept of oblivious equilibrium have previously been used to analyze such models. In the models we study, for each firm, being in a larger state is more profitable, while if competi-tors’ states are larger it is less profitable. This structure is quite natural in dynamic models ofcompetition that have been studied in the operations research and economics literature, and weleverage it to prove our uniqueness result. We provide examples of dynamic investments modelsof quality, capacity, and advertising, as well as a dynamic reputation model of an online market.We also apply our results to commonly used heterogeneous agent macroeconomic models.We now explain our contributions in more detail and compare them to previous work onMFE.
Uniqueness . We do not know of any general uniqueness result regarding MFE in discrete-time mean field equilibrium models. Only a few papers have obtained uniqueness results inspecific applications. Hopenhayn (1992) proves the uniqueness of an MFE in a specific dynamiccompetition model. Light (2020) proves the uniqueness of an MFE in a Bewley-Aiyagari modelunder specific conditions on the model’s primitives (see a related result in Hu and Shmaya(2019)). Our main theorem in this paper is a novel result that provides conditions ensuring theuniqueness of an MFE for broader classes of models. Informally, under mild additional technicalconditions, we show that if the probability that a player reaches a higher state in the next periodis decreasing in the other players’ states, and is increasing in the player’s own state in the currentperiod, then the MFE is unique (see Theorem 1). Hence, the conditions reduce the difficulty ofshowing that a stochastic game has a unique MFE to proving properties of the players’ optimalstrategies.In many applications, one can show that these properties of the optimal strategies arisenaturally. For example, in several dynamic models of competition in operations research andeconomics, a higher firm’s state (e.g., the quality of the firm’s product or the firm’s capacity)implies higher profitability, and the firm can make investments in each period in order to improveits state. In this setting, one can show that a firm invests less when its competitors’ states arehigher; hence, competitors’ higher states induce a lower state for the firm in the next period. Incontrast, if the firm’s own current state is higher, it induces a higher state in the next period.Another example is heterogeneous agent macro models where each agent solves a consumption-savings problem. The agents’ states correspond to their current savings level and current laborproductivity. Under certain conditions it can be shown that an agent saves less when the otheragents save more. On the other hand, the agents’ next period’s savings are increasing in theircurrent savings.We apply our uniqueness result to a general class of dynamic oligopoly models and hetero- For example, Adlakha et al. (2015) use MFE, which they call stationary equilibrium . Adlakha et al. (2015)was motivated by Hopenhayn (1992) who introduced the term to study models with infinite numbers of firms.Weintraub et al. (2008) introduce oblivious equilibrium to study settings with finite numbers of firms. Lasry and Lions (2007) prove the uniqueness of an MFE in a continuous time setting under a certain mono-tonicity condition (see also Carmona and Delarue (2018)). This monotonicity condition is different and doesnot hold in the applications studied in the present paper.
Existence.
Prior literature has considered the existence of equilibria in stochastic games.Some prior work considered the existence of Markov perfect equilibria (MPE) (see Doraszelski and Satterthwaite(2010) and He and Sun (2017)). Adlakha et al. (2015) prove the existence of an MFE forthe case of a countable and unbounded state space. Acemoglu and Jensen (2015) consider aclosely related notion of equilibrium that is called stationary equilibrium and prove its exis-tence for the case of a compact state space and a specific transition dynamic that is com-monly used in economics (see Stokey and Lucas (1989)). Stationary equilibrium in the sense ofAcemoglu and Jensen (2015) is an MFE where the players’ payoff functions depend on the otherplayers’ states through an aggregator. Our existence result applies for a general compact statespace, more general dependence on the payoff function, and more general transitions. In thissense, it is more closely related to the result of Adlakha and Johari (2013). Adlakha and Johari(2013) prove the existence of an MFE for the case of a compact state space in stochastic gameswith strategic complementarities using a lattice-theoretical approach. Instead, we do not assumestrategic complementarities and our state space can be any compact separable metric space. Forour existence result, we assume the standard continuity conditions on model primitives that areassumed in the papers mentioned above. In addition, we assume that the optimal stationarystrategy of the players is single-valued. Concavity conditions on the profit function and thetransition function can be imposed in order to ensure that the optimal stationary strategy isindeed single-valued. The main technical difficulty in proving existence is to prove the weakcontinuity of the nonlinear MFE operator (see Theorem 3).
Comparative statics.
While some papers contain certain specific results on how equilibriachange with the parameters of the model (for example, see Hopenhayn (1992) and Aiyagari(1994)), only a few papers have obtained general comparative results in large dynamic economies(see Acemoglu and Jensen (2015) for a discussion of the difficulties associated with derivingsuch results). Three notable exceptions are Adlakha and Johari (2013), Acemoglu and Jensen(2015), and Acemoglu and Jensen (2018). Adlakha and Johari (2013) use the techniques forcomparing equilibria developed in Milgrom and Roberts (1994) to derive general comparative In the dynamic oligopoly models and the heterogeneous agent macro models that we study in Sections 4and 5, previous literature assumes that the players use pure strategies. Motivated by this fact, we focus on purestrategy MFE. In this case, if the optimal stationary strategy of the players is not single-valued then the MFEoperator may not be convex-valued. Similar problems arise in proving the existence of a pure-strategy Nashequilibrium.
In this section we define our general model of a stochastic game and define mean field equilibrium(MFE). The model and the definition of an MFE are similar to Adlakha and Johari (2013) andAdlakha et al. (2015).
In this section we describe our stochastic game model. Differently to standard stochastic gamesin the literature (see Shapley (1953)), in our model, every player has an individual state. Playersare coupled through their payoffs and state transition dynamics. A stochastic game has thefollowing elements:
Time . The game is played in discrete time. We index time periods by t = 1 , , . . . . Players.
There are m players in the game. We use i to denote a particular player. States.
The state of player i at time t is denoted by x i,t ∈ X where X is a separable metricspace. Typically, we assume that the state space X is in R n or that X is countable. We denotethe state of all players at time t by x t and the state of all players except player i at time t by x − i , t . Actions.
The action taken by player i at time t is denoted by a i,t ∈ A where A ⊆ R q . Weuse a t to denote the action of all players at time t . The set of feasible actions for a player instate x is given by Γ( x ) ⊆ A . States’ dynamics.
The state of a player evolves in a Markov fashion. Formally, let h t = { x , a , . . . , x t − , a t − } denote the history up to time t . Conditional on h t , players’ statesat time t are independent of each other. This assumption implies that random shocks areidiosyncratic, ruling out aggregate random shocks that are common to all players. Player i ’sstate x i,t at time t depends on the past history h t only through the state of player i at time t − x i,t − ; the states of other players at time t − x − i , t − ; and the action taken by player i at time t − a i,t − .If player i ’s state at time t − x i,t − , the player takes an action a i,t − at time t −
1, the5tates of the other players at time t − x − i , t − , and ζ i,t is player i ’s realized idiosyncraticrandom shock at time t , then player i ’s next period’s state is given by x i,t = w ( x i,t − , a i,t − , x − i , t − , ζ i,t ) . We assume that ζ is a random variable that takes values ζ j ∈ E with probability p j for j =1 , . . . , n . w : X × A × X m − × E → X is the transition function. Payoff.
In a given time period, if the state of player i is x i , the state of the other playersis x − i , and the action taken by player i is a i , then the single-period payoff to player i is π ( x i , a i , x − i ) ∈ R . In Section 2.2 we extend our model to a model in which players are alsocoupled through actions, that is, the functions w and π can also depend on the rivals’ currentactions. Discount factor.
The players discount their future payoff by a discount factor 0 < β < i ’s infinite horizon payoff is given by: P ∞ t =1 β t − π ( x i,t , a i,t , x − i , t ).In many games, coupling between players is independent of the identity of the players. Thisnotion of anonymity captures scenarios where the interaction between players is via aggregate in-formation about the state (see Jovanovic and Rosenthal (1988)). Let s ( m ) − i,t ( y ) denote the fractionof players excluding player i that have their state as y at time t . That is, s ( m ) − i,t ( y ) = 1 m − X j = i { x j,t = y } where 1 D is the indicator function of the set D . We refer to s ( m ) − i,t as the population state at time t (from player i ’s point of view). Definition 1 (Anonymous stochastic game). A stochastic game is called an anonymous stochas-tic game if the payoff function π ( x i,t , a i,t , x − i , t ) and the transition function w ( x i,t , a i,t , x − i , t , ζ i,t +1 ) depend on x − i,t only through s ( m ) − i,t . In an abuse of notation, we write π ( x i,t , a i,t , s ( m ) − i,t ) for thepayoff to player i , and w ( x i,t , a i,t , s ( m ) − i,t , ζ i,t +1 ) for the transition function for player i . For the remainder of the paper, we focus our attention on anonymous stochastic games. Forease of notation, we often drop the subscripts i and t and denote a generic transition function by w ( x, a, s, ζ ) and a generic payoff function by π ( x, a, s ) where s represents the population stateof players other than the player under consideration. Anonymity requires that a player’s single-period payoff and transition function depend on the states of other players via their empiricaldistribution over the state space, and not on their specific identify. In anonymous stochasticgames the functional form of the payoff function and transition function are the same, regardlessof the number of players m . In that sense, we often interpret the profit function π ( x, a, s ) asrepresenting a limiting regime in which the number of players is infinite. Our results also generalize for models in which the primitives depend on the number of players m like inthe study of oblivious equilibria (Weintraub et al., 2008)).
6e now provide a simple model of capacity competition that illustrates some of the notationpresented above. This is one of the dynamic competition models that we study in Section 4.1.
Example 1
Our example is based on the capacity competition models of Besanko and Doraszelski(2004) and Besanko et al. (2010). We consider an industry with homogeneous products, whereeach firm’s state variable determines its production capacity. If the firm’s state is x , then itscapacity is ¯ q ( x ) . In each period, each firm takes a costly action to improve its capacity in thenext period. Further, in each period, firms compete in a capacity-constrained quantity settinggame. The inverse demand function is given by P ( Q ) , where Q represents the total industryoutput. For simplicity, we assume that the marginal costs of all the firms are equal to zero.Given the total quantity produced by its competitors Q − i , the profit maximization problem forfirm i is given by max ≤ q i ≤ ¯ q ( x i ) P ( q i + Q − i ) q i .In general, one could solve for the equilibrium of the capacity-constrained static quantitygame played by firms, and these static equilibrium actions would determine the single-periodprofits. However, we focus on the limiting regime with a large number of firms with out marketpower, that is, firms take Q as fixed. In this case, each firm produces at full capacity and thelimiting profit function is given by: π ( x, a, s ) = P (cid:18)Z X ¯ q ( y ) s ( dy ) (cid:19) ¯ q ( x ) − da, where a is the firm’s investment and d is the unit investment cost (see also Ifrach and Weintraub(2016)). The next period’s state depends on the amount of investment, the current state, and arandom shock. For example, assuming that the state depreciates at rate δ , a possible transitionfunction is given by: w ( x, a, s, ζ ) = ((1 − δ ) x + k ( a )) ζ, where k is an increasing function that determines the impact of the firm’s investment and ζ represents uncertainty in the investment process. Now, we let P ( X ) be the set of all possible population states on X , that is P ( X ) is theset of all probability measures on X . We endow P ( X ) with the weak topology. Since P ( X )is metrizable, the weak topology on P ( X ) is determined by weak convergence (for details seeAliprantis and Border (2006)). We say that s n ∈ P ( X ) converges weakly to s ∈ P ( X ) if for allbounded and continuous functions f : X → R we havelim n →∞ Z X f ( x ) s n ( dx ) = Z X f ( x ) s ( dx ) . For the rest of the paper , we assume the following conditions on the primitives of the model:
Assumption 1 (i) π is bounded and (jointly) continuous. w is continuous. Recall that we endow P ( X ) with the weak topology. ii) X is compact.(iii) The correspondence Γ : X → A is compact-valued and continuous. We note two extensions that can be important in applications for which we can extend ourresults.First, in our basic mean field model, we assume that the players are coupled through theirstates: both the transition function and the payoff function of each player depend on the statesof all other players. We note that even in this setting, a player’s payoff function can dependon rivals’ actions as long as these actions do not affect the evolution of their own state nor theevolution of the population state. For instance, the players’ payoff functions can depend on the static pricing or quantity decisions of the other players. In Section 4.1 we study models in whichthe firms’ (static) actions affect other players’ current payoffs but do not affect the evolution offuture states.In certain models of interest such as learning-by-doing and dynamic advertising, however,players’ states are coupled through the dynamic actions, a i,t . That is, the actions of other players, a − i , t , affect a player’s transition function and payoff function. For these cases, we consider amodel where the transition function and the payoff function of each player depend on both thestates and the actions of all other players. The model is like our original model except that nowthe probability measure s describes the joint distribution of players over actions and states andnot only over states, that is, s ∈ P ( X × A ). Thus, the transition function w ( x, a, s, ζ ) and thepayoff function π ( x, a, s ) depend on the joint distribution over state-action pairs s ∈ P ( X × A ).All the results in the paper can be extended to this setting where the population state is ameasure on P ( X × A ) (see Section A.1 in the Appendix for more details). The monotonicityconditions that are needed in order to prove the uniqueness of an MFE in the case that thepopulation is a measure on P ( X × A ) are similar to the conditions that are needed in the casethat the population is a measure on P ( X ). In Section 4.2 we prove the uniqueness of an MFEfor a dynamic advertising model where the players’ payoff functions depend on the other players’actions (advertising expenditures), and thus, the population state is a measure on P ( X × A ).Our second extension relaxes the assumption on our base model that players are ex-antehomogeneous. To consider players that may be ex-ante heterogeneous with different modelprimitives, we extend our model to a setting in which each player has a fixed type throughout the time horizon that is drawn from a finite set. Then, the payoff function and transitionfunction can depend on this type. We show that all our results hold in this more general setting(see Section A.2 for more details). In particular, we show that if the conditions that we usein order to prove our results hold for every type, then the results are valid for the model withex-ante heterogeneous players. By continuous we mean both upper hemicontinuous and lower hemicontinuous. .3 Mean Field Equilibrium In Markov perfect equilibrium (MPE), players’ strategies are functions of the population state.However, MPE quickly becomes intractable as the number of players grows, because the numberof possible population states becomes too large. Instead, in a game with a large number ofplayers, we might expect that idiosyncratic fluctuations of players’ states “average out”, andhence the actual population state remains roughly constant over time. Because the effect ofother players on a single player’s payoff and transition function is only via the population state,it is intuitive that, as the number of players increases, a single player’s effect on the outcomeof the game is negligible. Based on this intuition, related schemes for approximating Markovperfect equilibrium (MPE) have been proposed in different application domains via a solutionconcept we call mean field equilibrium (MFE).Informally, an MFE is a strategy for the players and a population state such that: (1) Eachplayer optimizes her expected discounted payoff assuming that this population state is fixed;and (2) Given the players’ strategy, the fixed population state is an invariant distribution ofthe states’ dynamics. The interpretation is that a single player conjectures the population stateto be s . Therefore, in determining her future expected payoff stream, a player considers apayoff function and a transition function evaluated at the fixed population state s . In MFE, theconjectured s is the correct one given the strategies being played. MFE alleviates the complexityof MPE, because in the former the population state is fixed, while in the latter players keeptrack of the exact evolution of the population state. We refer the reader to the papers cited inSection 1 for a more detailed motivation and rigorous justifications for using MFE.Let X t := X × . . . × X | {z } t times . For a fixed population state, a nonrandomized pure strategy σ is a sequence of (Borel) measurable functions ( σ , σ , . . . , ) such that σ t : X t → A and σ t ( x , . . . , x t ) ∈ Γ( x t ) for all t ∈ N . That is, a strategy σ assigns a feasible action to everyfinite string of states. Note that a single player’s strategy depends only on her own history ofstates and does not depend on the population state. This strategy is called an oblivious strategy(see Weintraub et al. (2008) and Adlakha et al. (2015)).For each initial state x ∈ X and long run average population state s ∈ P ( X ), a strategy σ induces a probability measure over the space X N , describing the evolution of a player’s state. We denote the expectation with respect to that probability measure by E σ , and the associatedstates-actions stochastic process by { x ( t ) , a ( t ) } ∞ t =1 .When a player uses a strategy σ , the population state is fixed at s ∈ P ( X ), and the initialstate is x ∈ X , then the player’s expected present discounted value is V σ ( x, s ) = E σ ∞ X t =1 β t − π ( x ( t ) , a ( t ) , s ) ! . The probability measure on X N is uniquely defined (see for example Bertsekas and Shreve (1978)). V ( x, s ) = sup σ V σ ( x, s ) . That is, V ( x, s ) is the maximal expected payoff that the player can achieve when the initialstate is x and the population state is fixed at s ∈ P ( X ). We call V the value function and astrategy σ attaining it optimal .Standard dynamic programming arguments (see Bertsekas and Shreve (1978)) show that thevalue function satisfies the Bellman equation: V ( x, s ) = max a ∈ Γ( x ) π ( x, a, s ) + β n X j =1 p j V ( w ( x, a, s, ζ j ) , s ) . Under Assumption 1, there exists an optimal stationary Markov strategy (see Lemma 3 in theAppendix). Let G ( x, s ) be the optimal stationary strategy correspondence, i.e., G ( x, s ) = argmax a ∈ Γ( x ) π ( x, a, s ) + β n X j =1 p j V ( w ( x, a, s, ζ j ) , s ) . Let B ( X ) be the Borel σ -algebra on X . For a strategy g ∈ G and a fixed population state s ∈ P ( X ), the probability that player i ’s next period’s state will lie in a set B ∈ B ( X ), giventhat her current state is x ∈ X and she takes the action a = g ( x, s ), is: Q g ( x, s, B ) = Pr( w ( x, g ( x, s ) , s, ζ ) ∈ B ) . Now suppose that the population state is s , and all players use a stationary strategy g ∈ G .Because of averaging effects, we expect that if the number of players is large, then the long runpopulation state should in fact be an invariant distribution of the Markov kernel Q g on X thatdescribes the dynamics of an individual player.We can now define an MFE. In an MFE, every player conjectures that s is the fixed longrun population state and plays according to a stationary strategy g . On the other hand, if everyagent plays according to g when the population state is s , then the long run population state ofall players, s , should constitute an invariant distribution of Q g . Definition 2
A stationary strategy g and a population state s ∈ P ( X ) constitute an MFE ifthe following two conditions hold:1. Optimality: g is optimal given s , i.e., g ( x, s ) ∈ G ( x, s ) .2. Consistency: s is an invariant distribution of Q g . That is, s ( B ) = Z X Q g ( x, s, B ) s ( dx ) . for all B ∈ B ( X ) , where we take Lebesgue integral with respect to the measure s . G ( x, s ) is nonempty, compact-valued and upperhemicontinuous. The proof is a standard application of the maximum theorem. We provide theproof for completeness (see Lemma 3). In Theorem 3 we prove the existence of a populationstate that satisfies the consistency requirement in Definition 2. In this section we present our main results. In Section 3.1 we provide conditions that ensurethe uniqueness of an MFE. In Section 3.2 we prove the existence of an MFE. In Section 3.3 weprovide conditions that ensure unambiguous comparative statics results regarding MFE.
In this section we present our uniqueness result.We recall that a stationary strategy-population state pair ( g, s ) is an MFE if and only if g is optimal and s is a fixed point of the operator Φ : P ( X ) → P ( X ) defined byΦ s ( B ) = Z X Q g ( x, s, B ) s ( dx ) , (1)for all B ∈ B ( X ).We prove uniqueness by showing that the operator Φ has a unique fixed point. In orderto prove uniqueness we will assume that G is single-valued. For the rest of the section we willassume that g ∈ G is the unique selection from the optimal strategy correspondence G . In thenext section we provide conditions that ensure that G is indeed single-valued (see Lemma 1). G being single-valued and Theorem 3 (see Section 3.2) imply that Φ has at least one fixed point.In Theorem 1 we will show that under certain conditions the operator Φ has at most one fixedpoint.We omit the reference to g in Q g ( x, s, B ), i.e., we write Q ( x, s, B ) instead of Q g ( x, s, B ).Since the Markov kernel Q depends on s , it is complicated to work directly with the operatorΦ. Thus, to prove the uniqueness of an MFE and to prove our comparative statics results, weintroduce an auxiliary operator that is easier to work with. For each s ∈ P ( X ), define theoperator M s : P ( X ) → P ( X ) by M s θ ( B ) = Z X Q ( x, s, B ) θ ( dx ) . We introduce the following useful definition.
Definition 3
We say that Q is X -ergodic if the following two conditions hold:(i) For any s ∈ P ( X ) , the operator M s has a unique fixed point µ s .(ii) M ns θ converges weakly to µ s for any probability measure θ ∈ P ( X ) . s is an MFE if and only if µ s = s is a fixed point of the operator M s . X -ergodicitymeans that for every population state s ∈ P ( X ) the players’ long-run state is independent ofthe initial state. The X -ergodicity of Q can be established using standard results from thetheory of Markov chains in general state spaces (see Meyn and Tweedie (2012)). When Q isincreasing in x , which we assume in order to prove the uniqueness of an MFE (see Assumption 2),then the X -ergodicity of Q can be established using results from the theory of monotone Markovchains. These results usually require a splitting condition (see Bhattacharya and Lee (1988) andHopenhayn and Prescott (1992)) that typically holds in applications of interest. Specifically, inSections 4 and 5 we show that X -ergodicity holds in important classes of dynamic models.We now introduce other notation and definitions that are helpful in proving uniqueness.We assume that X is endowed with a closed partial order ≥ . In the important case X = R n , x, y ∈ X we write x ≥ y if x i ≥ y i for each i = 1 , .., n . Let S ⊆ X . We say that a function f : S → R is increasing if f ( y ) ≥ f ( x ) whenever y ≥ x and we say that f is strictly increasingif f ( y ) > f ( x ) whenever y > x .For s , s ∈ P ( X ) we say that s stochastically dominates s and we write s (cid:23) SD s if forevery increasing function f : X → R we have Z X f ( x ) s ( dx ) ≥ Z X f ( x ) s ( dx ) , when the integrals exist. We say that B ∈ B ( X ) is an upper set if x ∈ B and x ≥ x imply x ∈ B . Recall from Kamae et al. (1977) that s (cid:23) SD s if and only if for every upper set B wehave s ( B ) ≥ s ( B ).In addition, for the rest of the section we will assume that there exists a binary relation (cid:23) on P ( X ), such that s ∼ s (i.e., s (cid:23) s and s (cid:23) s )) implies π ( x, a, s ) = π ( x, a, s ) for all( x, a ) ∈ X × A and w ( x, a, s , ζ ) = w ( x, a, s , ζ ) for all ( x, a, ζ ) ∈ X × A × E .Note that such binary relation always exists, for example one can take s ∼ s ⇔ s = s . Forour uniqueness result we will further require that the binary relation (cid:23) on P ( X ) is complete, thatis, for all s , s ∈ P ( X ) we either have s (cid:23) s or s (cid:23) s . In many applications (see Section4 and Section 5) there exists a function H : P ( X ) → R such that ˜ π ( x, a, H ( s )) = π ( x, a, s )and ˜ w ( x, a, H ( s ) , ζ ) = w ( x, a, s, ζ ), where H is continuous and increasing with respect to thestochastic dominance order (cid:23) SD . In this case, a natural complete order (cid:23) on P ( X ) arises bydefining s (cid:23) s if and only if H ( s ) ≥ H ( s ). Below, we also discuss the case of a non-completeorder. We say that (cid:23) agrees with (cid:23) SD if for any s , s ∈ P ( X ), s (cid:23) SD s implies s (cid:23) s .We say that Q is increasing in x if for each s ∈ P ( X ), we have Q ( x , s, · ) (cid:23) SD Q ( x , s, · )whenever x ≥ x . In addition, we say that Q is decreasing in s if for each x ∈ X , we have Q ( x, s , · ) (cid:23) SD Q ( x, s , · ) whenever s (cid:23) s . We now state the main theorem of the paper. Weshow that if Q is X -ergodic, Q is increasing in x and decreasing in s , and (cid:23) is complete andagrees with (cid:23) SD , then if an MFE exists, it is unique.12ntuitively, Q decreasing in s implies that the probability that a player will move to a higherstate in the next period is decreasing in the current period’s population state. If there are twoMFEs, s and s , such that s (cid:23) s (i.e., s is “higher” than s ), then the probability of movingto a higher state under s is lower than under s , which is not consistent with s (cid:23) s , with thedefinition of an MFE, and the fact that (cid:23) agrees with (cid:23) SD . Assumption 2 (i) Q is X -ergodic. Q is increasing in x and decreasing in s .(ii) (cid:23) agrees with (cid:23) SD .(iii) G is single-valued. Theorem 1
Suppose that Assumption 2 holds. If the binary relation (cid:23) is complete, then if anMFE exists, it is unique.
Proof.
Let θ , θ ∈ P ( X ) and assume that θ (cid:23) SD θ . Let B be an upper set and let s , s betwo MFEs such that s (cid:23) s . We have M s θ ( B ) = Z X Q ( x, s , B ) θ ( dx ) ≤ Z X Q ( x, s , B ) θ ( dx ) ≤ Z X Q ( x, s , B ) θ ( dx )= M s θ ( B ) . Thus, for any upper set B we have M s θ ( B ) ≤ M s θ ( B ) which implies that M s θ (cid:23) SD M s θ .The first inequality follows from the fact that Q ( x, s, B ) is decreasing in s for an upper set B andall x . The second inequality follows from the fact that θ (cid:23) SD θ and Q ( x, s, B ) is increasing in x for an upper set B and any s .We conclude that M ns θ (cid:23) SD M ns θ for all n ∈ N . Q being X -ergodic implies that M ns i θ i converges weakly to µ s i = s i . Since (cid:23) SD is closed under weak convergence (see Kamae et al.(1977)), we have s (cid:23) SD s .We conclude that if s and s are two MFEs such that s (cid:23) s , then s (cid:23) SD s . Since (cid:23) agrees with (cid:23) SD , we have s (cid:23) s . That is, s ∼ s , which implies that π ( x, a, s ) = π ( x, a, s )and w ( x, a, s , ζ ) = w ( x, a, s , ζ ). Thus, under s the players play according to the same strategyas under s (i.e., g ( x, s ) = g ( x, s ) for all x ∈ X ). We conclude that Q ( x, s , B ) = Q ( x, s , B )for all x ∈ X and B ∈ B ( X ). X -ergodicity of Q implies that M s and M s have a unique fixedpoint. Thus, µ s = µ s , i.e., s = s . Similarly, we can show that s (cid:23) s implies that s = s . In some models, the condition that Q is decreasing in s follows from the fact that the policy function g isdecreasing in the population state s (see Section 4). Xu and Hajek (2013) prove the uniqueness of an equilib-rium in a supermarket mean field game under a similar monotonicity condition on the policy function. Theirsetting is different from ours because the players do not have individual states nor they dynamically optimize. Recall that µ s is the unique fixed point of M s and that s is an MFE if and only if µ s = s . (cid:23) is complete if s and s are two MFEs we have s (cid:23) s or s (cid:23) s . Thus, we provedthat if s and s are two MFEs then s = s . We conclude that if an MFE exists, it is unique.The assumptions on Q in Theorem 1 involve assumptions on the optimal strategy g . Thus,these assumptions are not over the primitives of the model. In Section 4 we introduce conditionson the primitives of dynamic oligopoly models that guarantee the uniqueness of an MFE. Inparticular, we show that the monotonicity conditions over Q arise naturally in important classesof these models. In Section 5 we apply our result to prove the uniqueness of an MFE inheterogeneous agent macro models.In some applications the assumption that the binary relation (cid:23) is complete is restrictive. Inthe case that (cid:23) is not complete and Assumption 2 holds, the following Corollary shows thatthe MFEs are not comparable by the binary relation (cid:23) . This Corollary can be used to deriveproperties on the MFE when there are multiple MFEs. For example, suppose that there exist twofunctions H i : P ( X ) → R , i = 1 , π ( x, a, H ( s )) = π ( x, a, s ) and ˜ w ( x, a, H ( s ) , ζ ) = w ( x, a, s, ζ ), where H i is continuous and increasing with respect to the stochastic dominanceorder (cid:23) SD . We can define an order (cid:23) on P ( X ) by defining s (cid:23) s if H ( s ) ≥ H ( s )and H ( s ) ≥ H ( s ). Clearly, this may not be a complete order. The following Corollaryprovides conditions that imply that if s and s are two MFEs, then it cannot be the case that H ( s ) > H ( s ) and H ( s ) > H ( s ). We write s ≻ s if s (cid:23) s and s (cid:15) s . Corollary 1
Suppose that Assumption 2 holds. If s and s are two MFEs then s ⊁ s and s ⊁ s . Proof.
Suppose, in contradiction, that s ≻ s . The argument in the proof of Theorem 1implies that s (cid:23) SD s . Since (cid:23) agrees with (cid:23) SD , we have s (cid:23) s , which is a contradiction.We conclude that s ⊁ s . Similarly, we can show that s ⊁ s .When the state space X is given by the product space X = X × X where X and X are separable metric spaces, a modification of our uniqueness result can be applied to prove theuniqueness of an MFE under slightly different conditions than the conditions of Assumption 2.Assumption 2 requires that Q be increasing in x on X . However, when X = X × X , and X i is endowed with the closed partial order ≥ i , it is enough to assume that Q is increasing in x i on X i for some i = 1 , Q is increasing in x if for all functions f : X × X → R that are increasing in x on X , for all s ∈ P ( X ), and forall x ∈ X , the function Z X f ( y , y ) Q (( x , x ) , s, d ( y , y )) (2)is increasing in x . Similarly, Q is decreasing in s with respect to x if for all functions f : X × X → R that are increasing in x on X and for all x ∈ X the function in (2) is decreasingin s . In Sections 4.3 and 5 we show the usefulness of Theorem 2. We establish the uniquenessof an MFE for dynamic reputation models and heterogeneous agent macro models by proving14hat Q is increasing in x i for some i = 1 ,
2. In these models it is not necessarily true that Q is increasing in x on X , so Theorem 1 cannot be applied directly. The Appendix contains theproofs not presented in the main text. Theorem 2
Suppose that X = X × X . Suppose that Assumption 2 holds, apart from thecondition that Q is increasing in x and decreasing in s . Suppose that Q is increasing in x i anddecreasing in s with respect to x i for some i = 1 , . If the binary relation (cid:23) is complete, then ifan MFE exists, it is unique. In this section we study the existence of an MFE. We show that if G is single-valued, then theoperator Φ defined in Equation (1) has a fixed point and thus, there exists an MFE. Theorem 3
Assume that G is single-valued. There exists a mean field equilibrium. Note that we do not impose Assumption 2 for this result. Also note that X can be anycompact separable metric space in the proof of Theorem 3, so the existence result holds for theimportant cases of finite state spaces, countable state spaces, and X ⊆ R n . In addition, theproof of existence does not depend on the number of players in the game; the number of playersin the game can be finite, countable or uncountable. Finally, we note that we do not require X -ergodicity (see Definition 3) to show existence; instead we use compactness and continuity(see Assumption 1). The main challenge to prove existence is to prove the weak continuityof the nonlinear MFE operator. To do so, we leverage a generalized version of the boundedconvergence theorem by Serfozo (1982).We now provide conditions over the model primitives that guarantee that G is single-valuedwhen X is a convex set in R n . Similar conditions have been used in dynamic oligopoly models. Assumption 3
Suppose that X ⊆ R n and is convex.(i) Assume that π ( x, a, s ) is concave in ( x, a ) , strictly concave in a and increasing in x foreach s ∈ P ( X ) .(ii) Assume that w is increasing in x and concave in ( x, a ) for each ζ ∈ E .(iii) Γ( x ) is convex-valued and increasing in the sense that x ≥ x implies Γ( x ) ⊇ Γ( x ) . The following Lemma shows that the preceding conditions on the primitives of the modelensure that G is single-valued. Lemma 1
Suppose that Assumption 3 holds. Then G is single-valued. For similar results in a countable state space setting see Adlakha et al. (2015) andDoraszelski and Satterthwaite (2010)).
Corollary 2
Suppose that Assumption 3 holds. Then, there exists an MFE.
In this section we derive comparative statics results. Let ( I, (cid:23) I ) be a partially ordered set thatinfluences the players’ optimal decisions. We denote a generic element in I by e . For example, e can be the discount factor, a parameter that influences the players’ payoff functions, or aparameter that influences the players’ transition dynamics. Throughout this section we slightlyabuse notation and when the parameter e influences the players’ optimal decisions we add it asa parameter. For instance, we write Q ( x, s, e, · ) instead of Q ( x, s, · ). We say that Q is increasingin e if Q ( x, s, e , · ) (cid:23) SD Q ( x, s, e , · ) for all x , s , and all e , e ∈ I such that e (cid:23) I e . We provethat under the assumptions of Theorem 1, if Q is increasing in e then e (cid:23) I e implies that theunique MFE under e is higher than the unique MFE under e with respect to (cid:23) .Adlakha and Johari (2013) derive comparative statics results for MFE in the case that Q isincreasing in s , x and e . They prove that e (cid:23) I e implies s ( e ) (cid:23) SD s ( e ) where s ( e ) is themaximal MFE with respect to (cid:23) SD under e . Adlakha and Johari (2013) use the techniques tocompare equilibria developed in Milgrom and Roberts (1994) (see also Topkis (2011)). We notethat under the assumptions of Theorem 1, Q is increasing in x but decreasing in s . Thus, theresults in Adlakha and Johari (2013) do not apply to our setting. However, with the help of theuniqueness of an MFE, we derive a general comparative statics result. Theorem 4
Let ( I, (cid:23) I ) be a partial order. Assume that Q is increasing in e on I . Then,under the assumptions of Theorem 1, the unique MFE s ( e ) is increasing in the following sense: e (cid:23) I e implies s ( e ) (cid:23) s ( e ) . The same result can be shown with a similar argument under the assumptions of Theorem 2.We omit the details for sake of brevity. We note that our comparative statics result is with respectto the order (cid:23) and not with respect to the usual stochastic dominance order. The machinerymentioned in the paragraph above is not directly applicable in our models, and without it webelieve that comparative statics results with respect to the usual stochastic dominance order aremuch harder to obtain. We discuss the usefulness of our comparative static result with respectto the order (cid:23) in the context of dynamic oligopoly models below.16
Dynamic Oligopoly Models
In this section we study various dynamic models of competition or dynamic oligopoly modelsthat capture a wide range of phenomena in economics and operations research. We leverageour results to provide conditions under which a broad class of dynamic oligopoly models admita unique MFE. We also provide comparative statics results.More specifically, we show that under concavity assumptions and a natural substitutabilitycondition, the MFE is unique. The substitutability condition requires that the firms’ profitfunction has decreasing differences in each firm’s own state and the states of the other firms.This condition implies that the marginal profit of a firm (with respect to its own state) isdecreasing in the other firms’ states. It arises naturally in many dynamic oligopoly models. InSection 4.1 we consider well studied capacity competition and quality ladder models. In Section4.2 we consider a dynamic advertising model. In Section 4.3 we introduce a dynamic reputationmodel of an online market. In all of these models, it holds that the firms’ actions are higherwhen their own state is higher and the firms’ actions are lower when the competitors’ states(or the competitors’ actions) are higher. These are essentially the conditions that imply theuniqueness of an MFE for dynamic oligopoly models.
In this section we consider dynamic capacity competition models and dynamic quality laddermodels which have received significant attention in the recent operations research and economicsliterature. In these models, firms’ states correspond to a variable that affects their profits. Forexample, the state can be the firm’s capacity or the quality of the firm’s product. Per-periodprofits are based on a static competition game that depends on the heterogeneous firms’ statevariables. Firms take actions in order to improve their individual state over time.We now describe the models we consider.
States.
The state of firm i at time t is denoted by x i,t ∈ X where X ⊆ R + and is convex. Actions.
At each time t , firm i invests a i,t ∈ A = [0 , ¯ a ] to improve its state. The investmentchanges the firm’s state in a stochastic fashion. States’ dynamics.
A firm’s state evolves in a Markov fashion. Let 0 < δ < i ’s state at time t − x i,t − , the firm takes an action a i,t − at time t −
1, and ζ i,t is firm i ’s realized idiosyncratic random shock at time t , then firm i ’s state in thenext period is given by: x i,t = ((1 − δ ) x i,t − + k ( a i,t − )) ζ i,t where k : A → R is typically an increasing function that determines the impact of investment Even though we study models with potentially large numbers of firms, we keep the name dynamicoligopoly to be consistent with previous literature in which MFE or its variants have been used to approximateoligopolistic behavior (for example, see Qi (2013), Adlakha et al. (2015), and Onishi (2016)). . We assume that ζ takes positive values 0 < ζ < . . . < ζ n , where ζ < ζ n > p , p n > ζ and a positive probability fora good shock ζ n . In each period, the firm’s state is naturally depreciating at rate δ , but thefirm can make investments in order to improve it. Further, the outcome of depreciation andinvestment is subject to an idiosyncratic random shock ( ζ ) that, for example, could captureuncertainty in R&D or a marketing campaign. Related dynamics have been used in previousliterature. Further, our uniqueness result for capacity competition and quality ladder modelsholds under other states’ dynamics. For example, we could also assume additive dynamics x i,t = (1 − δ ) x i,t − + k ( a i,t − ) + ζ i,t . We make the following assumption over the dynamicsthat we discuss later before Theorem 5.
Assumption 4 (i) k ( a ) is strictly concave, continuously differentiable, strictly increasing and k (0) > . (ii) (1 − δ ) ζ n < .Payoff. The cost of a unit of investment is d > We assume there is a single-period profitfunction u ( x, s ) derived from a static game. When a firm invests a ∈ A , the firm’s state is x ∈ X , and the population state is s ∈ P ( X ), then the firm’s single-period payoff function isgiven by π ( x, a, s ) = u ( x, s ) − da .We assume that there exists a complete and transitive binary relation (cid:23) on P ( X ) such that s ∼ s implies that u ( x, s ) = u ( x, s ) for all s , s ∈ P ( X ) and x ∈ X . Furthermore, weassume that (cid:23) agrees with (cid:23) SD (cf. Section 3.1).To prove the uniqueness of an MFE for capacity competition and quality ladder models, weintroduce the following conditions on the primitives of the model. It is simple to verify that bothof the dynamic oligopoly models introduced in the examples below satisfy these assumptions.We believe the conditions are quite natural, and thus other commonly used dynamic oligopolymodels may satisfy them as well.Recall that a function f ( x, s ) is said to have decreasing differences in ( x, s ) on X × S if forall x ≥ x and s (cid:23) s we have f ( x , s ) − f ( x , s ) ≤ f ( x , s ) − f ( x , s ) . f is said to haveincreasing differences if − f has decreasing differences. Assumption 5 u ( x, s ) is jointly continuous. Further, it is concave and continuously differen-tiable in x , for each s ∈ P ( X ) . In addition, u ( x, s ) has decreasing differences in ( x, s ) . For our results to hold we need to impose some constraints on these additive dynamics so that the statespace remains compact. We can also assume an exogenous bound on the state as in Section 4.3. We believethat our results also hold if we drop the assumption that X is compact, under some additional conditions overmodel primitives that ensure some form of “decreasing returns to larger states” (see Adlakha et al. (2015)). The differentiability assumptions can be relaxed. We assume differentiability of u and k in order to sim-plify the proof of Theorem 5. The investment cost could be a convex function, but linearity simplifies the comparative static results inthe parameter d .
18e now provide two classic examples of profit functions u ( x, s ) that are commonly used inthe literature. For these examples, we explicitly define the binary relation (cid:23) .The first one is the capacity competition model described in Example 1. Recall that ifthe firm’s state is x , then its capacity is ¯ q ( x ). We assume that ¯ q is an increasing, continuouslydifferentiable, concave, and bounded function. We also assume that the inverse demand function P ( · ) is decreasing and continuous. In this model, u ( x, s ) = P (cid:18)Z X ¯ q ( y ) s ( dy ) (cid:19) ¯ q ( x ) . For the capacity competition model, we define s (cid:23) s if and only if R ¯ q ( y ) s ( dy ) ≥ R ¯ q ( y ) s ( dy ).Since ¯ q is an increasing function, (cid:23) agrees with (cid:23) SD . It can be verified that u satisfies theconditions of Assumption 5.Our second example is a classic quality ladder model, where individual states represent thequality of a firm’s product (see, e.g., Pakes and McGuire (1994) and Ericson and Pakes (1995)).Consider a price competition under a logit demand system. There are N consumers in themarket. The utility of consumer j from consuming the good produced by firm i at period t isgiven by u ijt = θ ln( x it + 1) + θ ln( Y − p it ) + v ijt , where θ < , θ > p it is the price of the good produced by firm i , Y is the consumer’s income, x it is the quality of the good produced by firm i , and { v ijt } i,j,t are i.i.d Gumbel random variablesthat represent unobserved characteristics for each consumer-good pair.There are m firms in the market and the marginal production cost is constant and thesame across firms. There is a unique Nash equilibrium in pure strategies of the pricing game(see Caplin and Nalebuff (1991)). These equilibrium static prices determine the single-periodprofits. Now, the limiting profit function that we focus on can be obtained from the asymptoticregime in which the number of consumers N and the number of firms m grow to infinity at thesame rate. The limiting profit function corresponds to a logit model of monopolistic competitiongiven by: u ( x, s ) = ˜ c ( x + 1) θ R X ( y + 1) θ s ( dy )(see Besanko et al. (1990)). ˜ c is a constant that depends on the limiting equilibrium price, themarginal production cost, the consumer’s income, and θ . For the quality ladder model, wedefine s (cid:23) s if and only if R ( y + 1) θ s ( dx ) ≥ R ( y + 1) θ s ( dy ). It is easy to see that (cid:23) agreeswith (cid:23) SD . It can also be verified that u satisfies the conditions of Assumption 5.The proof of our uniqueness result for the capacity competition and quality ladder modelsconsists of showing that Assumptions 4 and 5 imply Assumptions 1 and 2, and that (cid:23) is acomplete order. These are the conditions we use to show the existence of a unique MFE inSections 3.1 and 3.2. 19pecifically, similarly to Lemma 1, one can show that the concavity assumptions in Assump-tions 4 and 5 imply that G is single-valued. The assumption that k (0) > M s which could violate X -ergodicity (see Section 3.1). In ad-dition, condition (ii) in Assumption 4 is used to control the growth of firms, so that one canshow that the state space remains compact. We believe our results hold with a milder versionof this assumption. With this, the only remaining assumption that we need to show in orderto prove the uniqueness of an MFE for our capacity competition and quality ladder models isAssumption 2(i). For this, we use the fact that the profit function has decreasing differences inthe state x and the population state s . This implies that firms invest less when the populationstate is higher (see Lemma 4). We use this fact to show the desired monotonicity of Q .Our main result for dynamic capacity competition and dynamic quality ladder models is thefollowing: Theorem 5
Suppose that Assumptions 4 and 5 hold. Then there exists a unique MFE for thecapacity competition and quality ladder models.
Under Assumptions 4 and 5 we can also derive comparative statics results for our capacitycompetition and quality ladder models. In particular, we show that an increase in the costof a unit of investment decreases the unique MFE population state. Note that an increasein the investment cost decreases firms incentives to invest. However, a lower population stateincentivizes the firms to invest more. As a consequence, our model does not have the propertiesof a supermodular game (e.g., Topkis (1979)). Despite this, relying on the uniqueness of anMFE and on Theorem 4 we are able to show that in fact the unique MFE decreases when thecost of a unit of investment increases.We also derive comparative statics results regarding a change in a parameter that influencesthe profit function and a change in the discount factor. We show that if there is a parameter c such that the marginal profit of the firms is decreasing in that parameter, then the unique MFEdecreases in the parameter c . For example, in the quality ladder model, as the marginal costof production goes up, the unique MFE decreases. In the capacity competition model, as thepotential market size increases, the MFE increases. In addition, we show that an increase in thediscount factor increases the unique MFE.We note that all of our comparative statics results are with respect to the order (cid:22) andnot with respect to the usual stochastic dominance order as one would typically obtain usingsupermodularity arguments (e.g., Adlakha and Johari (2013)). We believe that these resultsprovide helpful information because the order (cid:22) relates to the single-period profit function, andtherefore, MFE can be ordered in terms of firms’ payoffs. Further, (cid:22) typically orders a variableof economic interest, such as the average capacity level in the capacity competition model orthe average quality level in the quality ladder model.20 heorem 6 Suppose that Assumptions 4 and 5 hold. We denote by s ( e ) the unique MFE whenthe parameter that influences the firms’ decisions is e .(i) If the cost of a unit of investment increases, then the unique MFE decreases, i.e., d ≤ d implies s ( d ) (cid:23) s ( d ) .(ii) Let c ∈ I ⊆ R be a parameter that influences the firms’ profit function. If the profitfunction u ( x, s, c ) has decreasing differences in ( x, c ) , then the unique MFE decreases in c , i.e., c ≥ c implies s ( c ) (cid:23) s ( c ) .(iii) Assume that u ( x, s ) is increasing in x . If the discount factor β increases, then theunique MFE s ( β ) increases, i.e., β ≥ β implies s ( β ) (cid:23) s ( β ) . In this section we consider dynamic advertising competition models. In these models, firms’states correspond to customer goodwill or market share. In each period, the firms decide ontheir advertising expenditures a . The probability that the next period’s customer goodwill ishigher increases when the firms spend more on advertising. The firms’ payoff functions dependon their own spending on advertising, on their own state, on the other firms’ states, and onthe other firms’ spending on advertising. Thus, a firm’s payoff function depends on the otherfirms’ dynamic actions (in Sections 2.2 and A.1 we extend the model and the results presented inSections 2 and 3 to the case in which each player’s payoff function depends on the other players’actions). Variants of dynamic models with this structure have been studied in the operationsresearch literature in contexts other than advertising (for example, see Hall and Porteus (2000)).We now describe our specific model. States.
The state of firm i at time t is denoted by x i,t ∈ X where X = R + . The state of afirm x i,t ∈ X represents the customer goodwill. Actions.
At each time t , firm i chooses an amount of money to spend on advertising a i,t ∈ A = [1 , ¯ a ] where ¯ a > States’ dynamics.
When the firm spends more on advertising, the customer goodwill in-creases. The customer goodwill depreciates over time at rate 0 < δ <
1. If firm i ’s state at time t − x i,t − , the firm takes an action a i,t − at time t −
1, and ζ i,t is firm i ’s realized idiosyncraticrandom shock at time t , then firm i ’s state in the next period is given by x i,t = (1 − δ )( x i,t − + a i,t − ) ζ i,t . We assume that ζ takes positive values 0 < ζ < . . . < ζ n . To ensure compactness we alsoassume that (1 − δ ) ζ n < Our model is a mean field version of the dynamic advertising model presented in Heyman and Sobel (2004)and in Section 4.3 in Olsen and Parker (2014)). ayoff. When a firm chooses to spend a ∈ A on advertising, the firm’s state is x ∈ X , andthe population action-state profile is s ∈ P ( X × A ), then the firm’s single-period payoff functionis given by π ( x, a, s ) = r ( x + a ) γ ( R ( x ′ + a ′ ) s ( d ( x ′ , a ′ ))) γ − a where ( x + a ) γ ( R ( x ′ + a ′ ) s ( d ( x ′ ,a ′ ))) γ is the expected demand, r > < γ <
1, 0 < γ < (cid:23) on P ( X × A ), by s (cid:23) s if and only if ( R ( x ′ + a ′ ) s ( d ( x ′ , a ′ ))) γ ≥ ( R ( x ′ + a ′ ) s ( d ( x ′ , a ′ ))) γ . Clearly, (cid:23) agrees with (cid:23) SD (see Section 3.1).We can also derive comparative statics results for the dynamic advertising model. For example,using similar arguments to the arguments in Section 4.1 we can show that when the discountfactor β increases, then the unique MFE increases in the following sense: if β > β , then s ( β ) (cid:23) s ( β ) where s ( β ) is the unique MFE under discount factor β . We also show that theunique MFE increases when the market price r increases. Theorem 7 (i) The dynamic advertising competition model has a unique MFE.(ii) Let s ( β ) be the unique MFE under the discount factor β . Then β > β implies s ( β ) (cid:23) s ( β ) .(iii) Let s ( r ) be the unique MFE under the price r . Then r > r implies s ( r ) (cid:23) s ( r ) . In this section we consider a dynamic reputation model. Motivated by the proliferation of onlinemarkets, reputation models and the design of reputation systems have recently been widelystudied in the operations and management science literature. These systems can mitigatethe mistrust between buyers and sellers participating in the marketplace (see Tadelis (2016)).Further, online markets typically consist of many small sellers, and therefore, assuming an MFElimit is natural.We study a dynamic reputation model in which sellers improve their reputation level overtime. The state of each seller consists of the average review given to her in the past historyand the total number of reviews she has received. In each period, each seller receives a reviewfrom buyers. A seller’s ranking is a simple average of her past reviews. Sellers invest in theirproducts in order to improve their reviews over time. For example, Airbnb hosts can invest in For example, see Dellarocas (2003), Aperjis and Johari (2010), Bolton et al. (2013), Papanastasiou et al.(2017), and Besbes and Scarsini (2018). Typically, review systems report simple averages; the number of reviews may also be relevant as it maysignal more sales and more experience from a seller. This assumption is made only for simplicity. We can also assume that reviews arrive according to a Pois-son process. − β where 0 < β <
1. For each seller i that departs, a newseller immediately arrives. We assign the new seller the same label i , and a 0 ranking, and 0reviews. Under this assumption, it is straightforward to show that the seller’s decision problemis the same stationary, infinite horizon, expected discounted reward maximization problem thatwe introduced in Section 2, where the discount factor is the probability of remaining in themarket. We now describe the dynamic reputation model in more detail.
States.
The state of seller i at time t is denoted by x i,t = ( x i,t, , x i,t, ) ∈ X × X = X . x i,t, represents seller i ’s average numerical review rating up to time t . We call x i,t, seller i ’s rankingat period t . x i,t, represents the number of reviews seller i has received up to period t . Actions.
At each time t , seller i chooses an action a i,t ∈ A = [0 , ¯ a ] in order to improve herranking. The action changes the seller’s state in a stochastic fashion. States’ dynamics.
If seller i ’s state at time t − x i,t − , the seller takes an action a i,t − attime t −
1, and ζ i,t is seller i ’s realized idiosyncratic random shock at time t , then seller i ’s statein the next period is given by: x i,t = (cid:18) min (cid:18) x i,t − , x i,t − , x i,t − , + 11 + x i,t − , ( k ( a ) + ζ i,t ) , M (cid:19) , min ( x i,t − , + 1 , M ) (cid:19) , where k : A → R is a strictly increasing and strictly concave function that determines theimpact of the seller’s investment on the next period’s review. The next period’s numericalreview, k ( a ) + ζ , is assumed to be non-negative. M > M > For example, Iyer et al. (2014) provide a similar regenerative model of arrivals and departures. In order to simplify the analysis and preserve Assumption 1, we assume that the numerical value of a re-view k ( a ) + ζ can be any non-negative number and not a discrete number. In a model where k ( a ) + ζ is discreteour results still hold as long as the optimal strategy is single-valued.
23f reviews. Similarly to the previous models, the random shocks represent uncertainty in thereview process.
Payoff.
The cost of a unit of investment is d >
0. When the seller’s ranking is x , theseller’s number of reviews is x , the seller chooses an action a ∈ A , and the population state is s ∈ P ( X ), then the seller’s single-period payoff is given by π ( x, a, s ) = ν ( x , x ) R ν ( x , x ) s ( d ( x , x )) − da where ν is increasing in x and x , concave, continuously differentiable in x , and positive. Thefunctional form resembles the logit model studied in Section 4.1.The cost of a unit of investment can be seen as a lever that a platform may impact bydesign. In particular, a platform can reduce the cost of a unit of investment for the sellers byintroducing tools to improve the buyers’ experience of using the sellers’ products. For example,an e-commerce platform could help facilitating logistics for its sellers, and a rental sharingplatform could help its hosts connecting cleaning services.We define a complete and transitive binary relation (cid:23) on P ( X ) by s (cid:23) s if and only if R ν ( x , x ) s ( d ( x , x )) ≥ R ν ( x , x ) s ( d ( x , x )). It is easy to see that (cid:23) agrees with (cid:23) SD (seeSection 3.1).We use Theorem 2 to prove that the dynamic reputation model admits a unique MFE. We also show that when the platform reduces the cost of a unit of investment then the MFEincreases.
Theorem 8 (i) The dynamic reputation model has a unique MFE.(ii) Let s ( d ) be the unique MFE under the unit of investment cost d . Then d ≥ d implies s ( d ) (cid:22) s ( d ) . In this section we consider heterogeneous agent macro models. In these models, there is a con-tinuum of agents facing idiosyncratic risks only (and no aggregate risks). The heterogeneousagents make decisions given certain market prices (in Aiyagari (1994), for example, the mar-ket prices are the interest rate and the wage rate). The market prices are determined by theaggregate decisions of all the agents in the market. We consider a setting similar to the one pre-sented in Acemoglu and Jensen (2015). We note that this setting encompasses many importantmodels in the economics literature. Examples include Bewley-Aiyagari models (see Bewley(1986), and Aiyagari (1994)), and models of industry equilibrium (see Hopenhayn (1992)).While Acemoglu and Jensen (2015) derive important existence and comparative statics results For this model we are able to show the monotonicity of the kernel Q with respect to x but not with re-spect to x . States.
The state of player i at time t is denoted by x i,t = ( x i,t, , x i,t, ) ∈ X × X = X where X ⊆ R and X ⊆ R n − . For example, in Bewley models x i,t, typically represents agent i ’s savings at period t and x represents agent i ’s income or labor productivity at period t (inthis case n = 2). Actions.
At each time t , player i chooses an action a i,t ∈ Γ( x i,t ) ⊂ R . States’ dynamics.
The state of a player evolves in a Markovian fashion. If player i ’s stateat time t − x i,t − , player i takes an action a i,t − at time t −
1, and ζ i,t is player i ’s realizedidiosyncratic random shock at time t , then player i ’s state in the next period is given by( x i,t, , x i,t, ) = ( a i,t − , m ( x i,t − , , ζ i,t )) , where m : X × E → X is a continuous function. For example, in Bewley models, in each periodagents choose how much to save for future consumption and how much to consume in the currentperiod. The agents’ labor productivity evolves exogenously and the labor productivity function m determines the next period’s labor productivity given the current labor productivity. So if anagent chooses to save a , ζ is the realized random shock, and her current labor productivity is x , then the agent’s next period state (savings-labor productivity pair) is given by ( a, m ( x , ζ )). Payoff.
As in Acemoglu and Jensen (2015), we assume that the payoff function dependson the population state through an aggregator. That is, if the population state is s , then theaggregator is given by H ( s ) where H : P ( X ) → R is a continuous function. If the aggregatoris H ( s ), the player’s state is x ∈ X , and the player takes an action a ∈ Γ( x ), then the player’ssingle-period payoff function is given by ˜ π ( x, a, H ( s )).We define a complete and transitive binary relation (cid:23) on P ( X ) by s (cid:23) s if and only if H ( s ) ≥ H ( s ). We assume that (cid:23) agrees with (cid:23) SD . This assumption holds in most of theheterogeneous agent macro models, where H is usually assumed to be increasing with respectto first order stochastic dominance (see Acemoglu and Jensen (2015)).Note that under the states’ dynamics defined above, and assuming that g ( x, s ) = ˜ g ( x, H ( s ))is the optimal stationary strategy, the transition kernel Q is given by Q ( x , x , s, B × B ) = 1 B (˜ g ( x , x , H ( s )) X j p j B ( m ( x , ζ j )) , where B × B ∈ B ( X × X ).We show that the model has a unique MFE if the optimal strategy is decreasing in theaggregator, i.e., if H ( s ) ≥ H ( s ) implies ˜ g ( x , x , H ( s )) ≤ ˜ g ( x , x , H ( s )), Q is X -ergodic,and ˜ g is increasing in x . We note that we cannot apply Theorem 1 to this model, since in most25pplications the optimal stationary strategy ˜ g is not increasing in x , and thus Q may not beincreasing in x . However, in most applications (for example, all the applications discussed inAcemoglu and Jensen (2015)) ˜ g is increasing in x . Thus, we can use Theorem 2 to show thatthe heterogeneous agent macro model has a unique MFE under the conditions stated above. Corollary 3
Assume that G is single-valued, Q is X -ergodic, and ˜ g is increasing in x anddecreasing in the aggregator. Then the heterogeneous agent macro model has a unique MFE. In most applications, the payoff function ˜ π has increasing differences in ( x , a ) which ensuresthat ˜ g is increasing in x . The condition that Q is X -ergodic also usually holds in applica-tions. For example, Aiyagari (1994) proves that Q is X -ergodic in his model. Thus, in manyapplications, in order to ensure uniqueness, one only needs to check that ˜ g is decreasing in theaggregator. In the next section we illustrate this in a Bewley-type model introduced in Aiyagari(1994). A Bewley-Aiyagari Model . Bewley models are widely studied and used in the modernmacroeconomics literature (for a survey see Heathcote et al. (2009)). As previously mentioned,in Bewley models agents receive a state-dependent income in each period and they solve aninfinite horizon consumption-savings problem; that is, the agents must decide how much to saveand how much to consume in each period. The agents can transfer assets from one period toanother only by investing in a risk-free bond, and have some borrowing limit. Aiyagari (1994)extends the Bewley model to a general equilibrium model with production. We now describethe model of Aiyagari (1994) in the setting of a mean field game.In a Bewley-Aiyagari model, x represents the agents’ savings and x represents the agents’labor productivity. m ( x , ζ ) represents the labor productivity function. That is, if the currentlabor productivity is x then the next period’s labor productivity is given by m ( x , ζ j ) withprobability p j . If the agents’ labor productivity is x then their income is given by wx where w > R > t , the agents choose their next period’s savings level a ∈ Γ( x , x ) whereΓ( x , x ) = [ − b, min { Rx + wx , ¯ b } ], and consume c = Rx + wx − a . That is, the agents’savings are lower than their cash-on-hand Rx + wx and higher than the borrowing constraint b ≥
0. ¯ b is an upper bound on savings that ensures compactness.The wage rate and the interest rate are determined in general equilibrium. There is arepresentative firm with a production function F ( K, N ) that is homogeneous of degree one. N is the labor supply and K is the capital. We assume that F is twice continuously differentiable,strictly concave, and strictly increasing. The first order conditions of the firm’s maximizationproblem yield F k ( K, N ) = R + δ − F N ( K, N ) = w where δ > Note that an MFE is usually called a stationary equilibrium in the economics literature (e.g.,Acemoglu and Jensen (2015)). The firm’s maximization problem is given by max
K,N F ( K, N ) − ( R − δ ) K − wN . For more details see,for example, Acemoglu and Jensen (2015) and Light (2020). F i ( K, N ) denotes the partial derivative of F with respect to i = K, N . A standardargument shows that R = f ′ ( K ) − δ + 1 and w = f ( K ) − f ′ ( K ) K where F ( K,
1) = f ( K ).In equilibrium we have R X x s ( d ( x , x )) = K where s is an invariant savings-labor produc-tivities distribution. That is, the aggregate supply of savings equals the total capital. We define H ( s ) = R X x s ( d ( x , x )). It is easy to see that (cid:23) agrees with (cid:23) SD (see Section 3.1).The agents’ utility from consumption is given by a utility function u which is assumed to bestrictly concave and strictly increasing. If the agents choose to save a then their consumption inthe current period is Rx + wx − a . Thus, using the equilibrium conditions R = f ′ ( H ( s )) − δ + 1and w = f ( H ( s )) − f ′ ( H ( s )) H ( s ), in a Bewley-Aiyagari model the payoff function ˜ π is given by˜ π ( x, a, H ( s )) = u (cid:0) ( f ′ ( H ( s )) − δ + 1) x + ( f ( H ( s )) − f ′ ( H ( s )) H ( s )) x − a (cid:1) . It is easy to establish that G is single-valued and that Assumption 1 holds. Thus, the existenceof an equilibrium in a Bewley-Aiyagari model follows from Theorem 3. Under mild technical conditions on the utility function (for example, if u is bounded or if u belongs to the constant relative risk aversion class), the X -ergodicity of Q can be proven ina similar manner to Acikgoz (2018). It can be established also that the next period’s savingsare increasing in the current period’s savings, i.e., ˜ g is increasing in x . Thus, to prove theuniqueness of an MFE in a Bewley-Aiyagari model, one needs to prove that ˜ g is decreasing inthe aggregator H ( s ). In a recent paper, Light (2020) proves the uniqueness of an MFE for thespecial case that the agents’ utility function is in the CRRA (constant relative risk aversion)class with a relative risk aversion coefficient that is less than or equal to one, and the productionfunction’s elasticity of substitution is bounded below by 1. Under these assumptions, we can usethe results in Light (2020) to show that ˜ g is decreasing in the aggregator H ( s ). Then, we canuse Corollary 3 to prove the uniqueness of an MFE. As a note for future research, our resultssuggest that the result in Light (2020) could be generalized, weakening the conditions on therelative risk aversion and on the production function. With this, we believe our approach couldbe used to show uniqueness for a broader class of heterogeneous agent macro models. Finally,we note that the uniqueness result in Hopenhayn (1992) can be obtained from Corollary 3 also.For the sake of brevity we omit the details here. This paper studies the existence and uniqueness of an MFE in stochastic games with a generalstate space. We provide conditions that ensure the uniqueness of an MFE. We also prove that Since F is homogeneous of degree one we have F ( K,
1) = KF K ( K,
1) + F N ( K, f ( K ) = Kf ′ ( K ) + w . Some of the previous existence results rely on the X -ergodicity of Q (e,g., Acikgoz (2018)) or on mono-tonicity arguments (e.g., Acemoglu and Jensen (2015)). The proof presented in this paper shows that theseconditions are not needed in order to establish the existence of an equilibrium. References
Acemoglu, D. and M. K. Jensen (2015): “Robust Comparative Statics in Large DynamicEconomies,”
Journal of Political Economy , 587–640.——— (2018): “Equilibrium Analysis in the Behavioral Neoclassical Growth Model,”
WorkingPaper . Acikgoz, O. (2018): “On the Existence and Uniqueness of Stationary Equilibrium in BewleyEconomies with Production,”
Journal of Economic Theory , 18–55.
Adlakha, S. and R. Johari (2013): “Mean Field Equilibrium in Dynamic Games with Strate-gic Complementarities,”
Operations Research , 971–989.
Adlakha, S., R. Johari, and G. Y. Weintraub (2015): “Equilibria of Dynamic Gameswith Many Players: Existence, Approximation, and Market Structure,”
Journal of EconomicTheory . Aiyagari, S. R. (1994): “Uninsured Idiosyncratic Risk and Aggregate Saving,”
The QuarterlyJournal of Economics , 659–684.
Aliprantis, C. D. and K. Border (2006):
Infinite Dimensional Analysis: a hitchhiker’sguide , Springer.
Aperjis, C. and R. Johari (2010): “Optimal Windows for Aggregating Ratings in ElectronicMarketplaces,”
Management Science , 56, 864–880.
Arnosti, N., R. Johari, and Y. Kanoria (2020): “Managing Congestion in MatchingMarkets,”
Working Paper . 28 alseiro, S. R., O. Besbes, and G. Y. Weintraub (2015): “Repeated Auctions withBudgets in Ad Exchanges: Approximations and Design,”
Management Science , 864–884.
Benveniste, L. M. and J. A. Scheinkman (1979): “On the Differentiability of the ValueFunction in Dynamic Models of Economics,”
Econometrica , 727–732.
Bertsekas, D., A. Nedi, and A. Ozdaglar (2003):
Convex Analysis and Optimization ,Athena Scientific.
Bertsekas, D. P. and S. E. Shreve (1978):
Stochastic Optimal Control: The Discrete TimeCase , Academic Press New York.
Besanko, D. and U. Doraszelski (2004): “Capacity Dynamics and Endogenous Asymme-tries in Firm Size,”
RAND Journal of Economics , 23–49.
Besanko, D., U. Doraszelski, L. X. Lu, and M. Satterthwaite (2010): “Lumpy Ca-pacity Investment and Disinvestment Dynamics,”
Operations Research , 58, 1178–1193.
Besanko, D., M. K. Perry, and R. H. Spady (1990): “The Logit Model of MonopolisticCompetition: Brand diversity,”
The Journal of Industrial Economics , 397–415.
Besbes, O. and M. Scarsini (2018): “On Information Distortions in Online Ratings,”
Oper-ations Research , 66, 597–610.
Bewley, T. (1986): “Stationary Monetary Equilibrium with a Continuum of IndependentlyFluctuating Consumers,”
Contributions to mathematical economics in honor of G´erard De-breu . Bhattacharya, R. N. and O. Lee (1988): “Asymptotics of a Class of Markov ProcessesWhich are not in General Irreducible,”
The Annals of Probability , 1333–1347.
Bimpikis, K., W. J. Elmaghraby, K. Moon, and W. Zhang (2018): “Managing MarketThickness in Online B2B Markets,”
Management Science, forthcoming . Bolton, G., B. Greiner, and A. Ockenfels (2013): “Engineering Trust: Reciprocity inthe Production of Reputation Information,”
Management science , 59, 265–285.
Caplin, A. and B. Nalebuff (1991): “Aggregation and Imperfect Competition: On theExistence of Equilibrium,”
Econometrica , 25–59.
Carmona, R. and F. Delarue (2018):
Probabilistic Theory of Mean Field Games with Ap-plications I-II , Springer.
Dellarocas, C. (2003): “The Digitization of Word of Mouth: Promise and Challenges ofOnline Feedback Mechanisms,”
Management science , 49, 1407–1424.
Doraszelski, U. and A. Pakes (2007): “A Framework for Applied Dynamic Analysis in IO,”
Handbook of Industrial Organization . Doraszelski, U. and M. Satterthwaite (2010): “Computable Markov-Perfect IndustryDynamics,”
The RAND Journal of Economics , 215–243.
Ericson, R. and A. Pakes (1995): “Markov-Perfect Industry Dynamics: A Framework forEmpirical Work,”
The Review of Economic Studies , 53–82.
Friesz, T. L., D. Bernstein, T. E. Smith, R. L. Tobin, and B.-W. Wie (1993): “A29ariational Inequality Formulation of the Dynamic Network User Equilibrium Problem,”
Op-erations Research , 179–191.
Hall, J. and E. Porteus (2000): “Customer service competition in capacitated systems,”
Manufacturing & Service Operations Management , 2, 144–165.
He, W. and Y. Sun (2017): “Stationary Markov Perfect Equilibria in Discounted StochasticGames,”
Journal of Economic Theory , 35–61.
Heathcote, J., K. Storesletten, and G. L. Violante (2009): “Quantitative Macroeco-nomics with Heterogeneous Households,”
Annual Reviews in Economics , 319–352.
Heyman, D. P. and M. J. Sobel (2004):
Stochastic Models in Operations Research: StochasticOptimization , vol. 2, Courier Corporation.
Hopenhayn, H. A. (1992): “Entry, Exit, and Firm Dynamics in Long Run Equilibrium,”
Econometrica , 1127–1150.
Hopenhayn, H. A. and E. C. Prescott (1992): “Stochastic Monotonicity and StationaryDistributions for Dynamic Economies,”
Econometrica , 1387–1406.
Hu, T.-W. and E. Shmaya (2019): “Unique Monetary Equilibrium with Inflation in a Sta-tionary Bewley–Aiyagari Model,”
Journal of Economic Theory , 180, 368–382.
Huang, M., R. P. Malham´e, P. E. Caines, et al. (2006): “Large Population StochasticDynamic Games: Closed-Loop McKean-Vlasov Systems and the Nash Certainty EquivalencePrinciple,”
Communications in Information & Systems , 221–252.
Ifrach, B. and G. Y. Weintraub (2016): “A Framework for Dynamic Oligopoly in Concen-trated Industries,”
The Review of Economic Studies , 84, 1106–1150.
Iyer, K., R. Johari, and M. Sundararajan (2014): “Mean Field Equilibria of DynamicAuctions with Learning,”
Management Science , 2949–2970.
Jovanovic, B. and R. W. Rosenthal (1988): “Anonymous Sequential Games,”
Journal ofMathematical Economics . Kamae, T., U. Krengel, and G. L. O’Brien (1977): “Stochastic Inequalities on PartiallyOrdered Spaces,”
The Annals of Probability , 899–912.
Kanoria, Y. and D. Saban (2019): “Facilitating the Search for Partners on Matching Plat-forms,”
Working Paper . Lasry, J.-M. and P.-L. Lions (2007): “Mean Field Games,”
Japanese Journal of Mathemat-ics , 229–260.
Light, B. (2020): “Uniqueness of Equilibrium in a Bewley-Aiyagari Model,”
Economic Theory ,69, 435–450.
Lovejoy, W. S. (1987): “Ordered Solutions For Dynamic Programs,”
Mathematics of Opera-tions Research , 269–276.
Maskin, E. and J. Tirole (2001): “Markov Perfect Equilibrium I. Observable Actions,”
Journal of Economic Theory , 191–219.
Meyn, S. P. and R. L. Tweedie (2012):
Markov Chains and Stochastic Stability , Springer30cience & Business Media.
Milgrom, P. and J. Roberts (1994): “Comparing Equilibria,”
The American EconomicReview , 441–459.
Olsen, T. L. and R. P. Parker (2014): “On Markov Equilibria in Dynamic InventoryCompetition,”
Operations Research , 62, 332–344.
Onishi, K. (2016): “Quantity Discounts and Capital Misallocation in Vertical Relationships:The Case of Aircraft and Airline Industries,”
Working paper . Pakes, A. and P. McGuire (1994): “Computing Markov-Perfect Nash Equilibria: NumericalImplications of a Dynamic Differentiated Product Model,”
The Rand Journal of Economics ,555–589.
Papanastasiou, Y., K. Bimpikis, and N. Savva (2017): “Crowdsourcing Exploration,”
Management Science , 64, 1727–1746.
Qi, S. (2013): “The impact of advertising regulation on industry: The cigarette advertising banof 1971,”
The RAND Journal of Economics , 44, 215–248.
Roberts, G. O. and J. S. Rosenthal (2004): “General State Space Markov Chains andMCMC Algorithms,”
Probability surveys , 1, 20–71.
Serfozo, R. (1982): “Convergence of Lebesgue Integrals with Varying Measures,”
Sankhy¯a:The Indian Journal of Statistics, Series A , 380–402.
Shapley, L. S. (1953): “Stochastic Games,”
Proceeding of the National Academy of Sciences . Stokey, N. and R. Lucas (1989):
Recursive Methods in Economic Dynamics , Harvard Uni-versity Press.
Tadelis, S. (2016): “Reputation and Feedback Systems in Online Platform Markets,”
AnnualReview of Economics , 8, 321–340.
Tembine, H., J.-Y. Le Boudec, R. El-Azouzi, and E. Altman (2009): “Mean FieldAsymptotics of Markov Decision Evolutionary Games and Teams,” in
Game Theory for Net-works, 2009. GameNets’ 09. International Conference on , IEEE, 140–150.
Topkis, D. M. (1978): “Minimizing a Submodular Function on a Lattice,”
Operations Research ,305–321.——— (1979): “Equilibrium Points in Nonzero-Sum n-person Submodular Games,”
Siam Jour-nal on Control and Optimization , 773–787.——— (2011):
Supermodularity and Complementarity , Princeton university press.
Weintraub, G. Y., C. L. Benkard, and B. Van Roy (2008): “Markov Perfect IndustryDynamics with Many Firms,”
Econometrica , 1375–1411.
Xu, J. and B. Hajek (2013): “The Supermarket Game,”
Stochastic Systems , 3, 405–441.
Yang, P., K. Iyer, and P. Frazier (2018): “Mean Field Equilibria for Resource Competitionin Spatial Settings,”
Stochastic Systems , 8, 307–334.31
Appendix: Extensions
In this section we extend the model presented in Section 2. In Section A.1 we study a modelwhere the players are coupled through actions and in Section A.2 we study a model where theplayers are ex-ante heterogeneous.
A.1 Coupling Through Actions
In this section we consider a model where the transition function and the payoff function of eachplayer depend on both the states and the actions of all other players. The model is the sameas the original model in Section 2 except that now the probability measure s describes the jointdistribution of players over actions and states and not only over states, that is, s ∈ P ( X × A ).Thus, the transition function w ( x, a, s, ζ ) and the payoff function π ( x, a, s ) depend on the jointdistribution over state-action pairs s ∈ P ( X × A ). We refer to s ∈ P ( X × A ) as the populationaction-state profile and to the marginal distribution of the population action-state profile over X as the population state (i.e., the population state’s distribution is described by the probabilitymeasure s ( · , A )).An MFE is defined similarly to the definition in Section 2. In an MFE, every player con-jectures that s is the fixed long run population action-state profile, and plays according to astationary strategy g . If every player plays according to the strategy g when the populationaction-state profile is s , then s constitutes an invariant distribution.Given the stationary strategy g , s ∈ P ( X × A ) is an invariant distribution if s ( B × D ) = Z X Z B D ( g ( y, s )) Q ( x, s, dy ) s ( dx, A ) = Z X Q ( x, s, B × D ) s ( dx, A ) , (3)for all B × D ∈ B ( X × A ) where Q ( x, s, B ) = Pr( w ( x, s, g ( x, s ) , ζ ) ∈ B ) and Q ( x, s, B × D ) = Z B D ( g ( y, s )) Q ( x, s, dy ) . To see that Equation (3) holds, first assume that X and A are discrete sets. The joint probabilitymass function of a stationary distribution s ( y, a ) is given by s ( y, a ) = s ( y, A ) s ( a | y ) = s ( y, A )1 { a } ( g ( y, s ))where s ( a | y ) is the probability of playing the action a ∈ A given that the state is y ∈ X . Sincethe players use the pure strategy g we have s ( a | y ) = 1 { a } ( g ( y, s )). Thus, s ( B × D ) = X y ∈ B X a ∈ D s ( y, A )1 { a } ( g ( y, s )) = X y ∈ B s ( y, A )1 D ( g ( y, s )) . Note that Q is a Markov kernel on X × A.
32n addition, since s is invariant, the marginal distribution s ( · , A ) must satisfy s ( y, A ) = P x ∈ X s ( x, A ) Q ( x, s, y ). Thus, s ( B × D ) = X x ∈ X X y ∈ B D ( g ( y, s )) Q ( x, s, y ) s ( x, A ) . Similarly, Equation (3) holds in the general state space.If A is compact then X × A is compact, and thus, P ( X × A ) is compact in the weaktopology. Similar arguments to the arguments in the proof of Theorem 3 show that the operatorΦ : P ( X ) → P ( X ) defined byΦ s ( B × D ) = Z X Q ( x, s, B × D ) s ( dx, A ) . is continuous (see more details in the proof of Theorem 9). Thus, as in the proof of Theorem 3,we can apply Schauder-Tychonoff’s fixed point theorem to prove that Φ has a fixed point.The uniqueness result holds under the same conditions as the conditions in Theorem 1except that the assumptions on the Markov kernel Q in Assumption 2 part (i) are assumed onthe Markov kernel Q . The proof of Theorem 9 part (i) is essentially the same as the proof ofTheorem 1. Similarly, Theorem 9 part (iii) holds when the assumptions on the Markov kernel Q are assumed on the Markov kernel Q .We summarize the discussion in the following Theorem. Theorem 9
Consider the model described in this section. Suppose that the action set A iscompact.(i) Under the assumptions of Theorem 1 where Q is replaced by Q the MFE is unique.(ii) Under the assumptions of Theorem 3 there exists an MFE.(iii) Let ( I, (cid:23) I ) be a partially ordered set. Assume that Q is increasing in e on I .Then, underthe assumptions of part (i), the unique MFE s ( e ) is increasing in the following sense: e (cid:23) I e implies s ( e ) (cid:23) s ( e ) . The assumptions on Q that are needed in order to guarantee the uniqueness of an MFE canbe verified in a similar manner to the assumptions on Q . In particular, in some models it isenough to show that the policy function g ( x, s ) is increasing in the state x and decreasing in thepopulation action-state profile state s which is a natural property in many dynamic oligopolymodels (see Section 4). In Section 4.2 we prove that the policy function g ( x, s ) is increasing in x and decreasing in s in a dynamic advertising model where each player’s payoff function dependson the other players’ actions, and we use Theorem 9 to prove that the model has a unique MFE. Recall that we say that Q is increasing in e if Q ( x, s, e , · ) (cid:23) SD Q ( x, s, e , · ) for all x , s , and all e , e ∈ I such that e (cid:23) I e . Note that the orders (cid:23) SD and (cid:23) are on measures over state-action pairs. .2 Ex-ante Heterogeneity In this section we study a mean field model with ex-ante heterogeneous players. We assumethat the players are heterogeneous in their payoff functions and in their transition functions.Assume that before the time horizon, each player has a type θ ∈ Θ, where Θ is a finite partiallyordered set. Each player’s type is fixed throughout the horizon. Let Υ be the probability massfunction over the type space; Υ( θ ) is the mass of players whose type is θ ∈ Θ, which is commonknowledge. Adding the argument θ ∈ Θ to the functions defined in Section 2, we can modifythe definitions of Section 2 to include the ex-ante heterogeneity of the players. In particular,we denote by w ( x, a, s, ζ, θ ) the transition function of type θ ∈ Θ and by π ( x, a, s, θ ) the payofffunction of type θ ∈ Θ.Let X h = X × Θ be an extended state space for the mean field model with ex-ante hetero-geneous players. If a player’s extended state is x h = ( x, θ ) ∈ X h then the player’s state is x and the player’s type is θ . Let s h be the population state over the extended state space, i.e., s h ∈ P ( X × Θ).For a probability measure s h ∈ P ( X × Θ), define a probability measure S ( s h ) ∈ P ( X ) by S ( s h ) ( B ) = X θ ∈ Θ s h ( B, θ )for all B ∈ B ( X ). That is, S ( s h ) is the marginal distribution of s h that describes the populationstate.For the model with ex-ante heterogeneous players we define the payoff function π h ( x h , a, s h ) = π ( x, a, S ( s h ) , θ ). Note that we consider a model where each player’s payoff function depends onthe other players’ states (the population state) and not on the other players’ types. This seemsreasonable in most applications, as types usually represent ex-ante heterogeneity in the payofffunctions, discount factors, etc. We now define the transition function.For a fixed extended population state s h ∈ P ( X × Θ) and a strategy g ( x, S ( s h ) , θ ), theprobability that player i ’s next period’s state will lie in a set B × D ∈ B ( X ) × Θ , given thather current state is x h = ( x, θ ) ∈ X h , her type is θ , and she takes the action a = g ( x, S ( s h ) , θ ),is: Q h ( x h , s h , B × D ) = Pr( w ( x, g ( x, S ( s h ) , θ ) , S ( s h ) , ζ, θ ) ∈ B )1 D ( θ ) . These definitions map the payoff function and transition function in the model with ex-anteheterogeneous players to the model with ex-ante homogeneous players that we considered inSection 2. Thus, all the results in this paper hold also in the case of ex-ante heterogeneity wherethe assumptions that we made on π , w and Q are now assumed on π h , w h and Q h . Thus, allour results can easily be extended to the case of ex-ante heterogeneous players. Note that inthis model, players of different types may play different MFE strategies. We now provide moredetails. 34imilarly to Section 2, in an MFE every player plays according to the strategy g when theextended population state is s h and s h constitutes an invariant distribution given the strategy g . That is, s h satisfies s h ( B × D ) = Z X h Q h ( x h , s h , B × D ) s h ( dx h )for all B × D ∈ B ( X ) × Θ .The following theorem follows immediately from the results in the main text when Q isreplaced by Q h . Note that X h = X × Θ is a product space so we can use Theorem 2 instead ofTheorem 1 to prove the uniqueness of an MFE.
Theorem 10
Consider the model described in this section.(i) Under the assumptions of Theorem 2 (with the state space X × Θ ) where Q is replacedby Q h , the MFE is unique.(ii) Under the assumptions of Theorem 3 there exists an MFE.(iii) Let ( I, (cid:23) I ) be a partially ordered set. Assume that Q h is increasing in e on I . Then,under the assumptions of part (i), the unique MFE s h ( e ) is increasing in the following sense: e (cid:23) I e implies s h ( e ) (cid:23) s h ( e ) . We define the X -transition function of a type θ player by Q θ ( x, s h , B ) = Pr( w ( x, g ( x, S ( s h ) , θ ) , S ( s h ) , ζ, θ ) ∈ B )for all B ∈ B ( X ). As discussed in Section 3.1, the key assumption that implies the uniquenessof an MFE is related to the transition function’s monotonicity properties. In particular, theassumption is that the transition function is increasing in the players’ own states and decreasingin the extended population state. In the case of ex-ante heterogeneity, the next Lemma showsthat if the transition function of each player Q θ is increasing in x and decreasing in s h for everytype θ then Q h is increasing in x and decreasing in s h with respect to x . This fact is useful forapplications when we want to verify the monotonicity conditions needed in Theorem 10 part (i)that imply the uniqueness of an MFE. Lemma 2
Assume that Q θ is increasing in x and decreasing in s h for every type θ . Then Q h is increasing in x and decreasing in s h with respect to x . B Appendix: Proofs
B.1 Uniqueness: Proof of Theorem 2
Proof of Theorem 2.
Assume without loss of generality that Q is increasing in x anddecreasing in s with respect to x . 35or s , s ∈ P ( X ) we write s (cid:23) SD,X s if for all functions f : X × X → R that areincreasing in the first argument (i.e., x ′ ≥ x implies that f ( x ′ , x ) ≥ f ( x , x ) for all x ∈ X )we have Z X f ( x , x ) s ( d ( x , x )) ≥ Z X f ( x , x ) s ( d ( x , x )) . We note that if (cid:23) agrees with (cid:23) SD , then (cid:23) agrees with (cid:23) SD,X (recall that s (cid:23) SD s if thelast inequality holds for every increasing function f : X × X → R ).Let f : X × X → R be increasing in the first argument, θ , θ ∈ P ( X ) and assume that θ (cid:23) SD,X θ . Let s , s be two MFEs such that s (cid:23) s . We have Z X f ( y , y ) M s θ ( d ( y , y )) = Z X Z X f ( y , y ) Q (( x , x ) , s , d ( y , y )) θ ( d ( x , x )) ≤ Z X Z X f ( y , y ) Q (( x , x ) , s , d ( y , y )) θ ( d ( x , x )) ≤ Z X Z X f ( y , y ) Q (( x , x ) , s , d ( y , y )) θ ( d ( x , x ))= Z f ( x , x ) M s θ ( d ( x , x )) . Thus, M s θ (cid:23) SD,X M s θ . The first inequality follows from the facts that f is increasing inthe first argument, Q is increasing in x , and θ (cid:23) SD,X θ . The second inequality follows fromthe fact that Q is decreasing in s with respect to x .We conclude that M ns θ (cid:23) SD,X M ns θ for all n ∈ N . Q being X -ergodic implies that M ns i θ i converges weakly to µ s i = s i . Since (cid:23) SD,X is a closed order, we have s (cid:23) SD,X s whichimplies that s (cid:23) s . The rest of the proof is the same as the proof of Theorem 1. B.2 Existence: Proofs of Theorem 3 and Lemma 1
We first introduce preliminary notation and results.Let B ( X × P ( X )) be the space of all bounded functions on X × P ( X ). Define the operator T : B ( X × P ( X )) → B ( X × P ( X )) by T f ( x, s ) = max a ∈ Γ( x ) h ( x, a, s, f )where h ( x, a, s, f ) = π ( x, a, s ) + β n X j =1 p j f ( w ( x, a, s, ζ j ) , s ) . The operator T is called the Bellman operator. Lemma 3
The optimal strategy correspondence G ( x, s ) is non-empty, compact-valued and upperhemicontinuous. roof. Assume that f ∈ B ( X × P ( X )) is (jointly) continuous. Then for each ζ ∈ E , f ( w ( x, a, s, ζ ) , s ) is continuous as the composition of continuous functions. Thus, h ( x, a, s, f ) iscontinuous as the sum of continuous functions. Since Γ( x ) is continuous, the maximum theorem(see Theorem 17.31 in Aliprantis and Border (2006)) implies that T f ( x, s ) is jointly continuous.We conclude that for all n = 1 , , . . . , T n f is continuous. Under Assumption 1, standarddynamic programming arguments (see Bertsekas and Shreve (1978)) show that T n f convergesto V uniformly. Since the set of continuous functions is closed under uniform convergence, V iscontinuous. Thus, h ( x, a, s, V ) is continuous. From the maximum theorem, G ( x, s ) is non-empty,compact-valued and upper hemicontinuous.We say that k n : X → R converges continuously to k if k n ( x n ) → k ( x ) whenever x n → x .The following Proposition is a special case of Theorem 3.3 in Serfozo (1982). Proposition 1
Assume that k n : X → R is a uniformly bounded sequence of functions. If k n : X → R converges continuously to k and s n converges weakly to s then lim n →∞ Z X k n ( x ) s n ( dx ) = Z X k ( x ) s ( dx ) . In order to establish the existence of an MFE, we will use the following Proposition (seeCorollary 17.56 in Aliprantis and Border (2006)).
Proposition 2 (Schauder-Tychonoff ) Let K be a nonempty, compact, convex subset of a locallyconvex Hausdorff space, and let f : K → K be a continuous function. Then the set of fixed pointsof f is compact and nonempty. Proof of Theorem 3.
Let g ( x, s ) = G ( x, s ) be the unique optimal stationary strategy. FromLemma 3, g is continuous.Consider the operator Φ : P ( X ) → P ( X ) defined byΦ s ( B ) = Z X Q g ( x, s, B ) s ( dx ) . If s is a fixed point of Φ then s is an MFE. Since X is compact P ( X ) is compact (i.e., compact inthe weak topology, see Aliprantis and Border (2006)). Clearly P ( X ) is convex. P ( X ) endowedwith the weak topology is a locally convex Hausdorff space. If Φ is continuous, we can applySchauder-Tychonoff’s fixed point theorem to prove that Φ has a fixed point. We now show thatΦ is continuous.First, note that for every bounded and measurable function f : X → R and for every s ∈ P ( X ) we have Z X f ( x )Φ s ( dx ) = Z X X j p j f ( w ( x, g ( x, s ) , s, ζ j ) s ( dx ) . (4)37o see this, first assume that f = 1 B where 1 B is the indicator function of B ∈ B ( X ). Then Z X f ( x )Φ s ( dx ) = Z X B Φ s ( dx )= Z X Q g ( x, s, B ) s ( dx )= Z X X j p j B ( w ( x, g ( x, s ) , s, ζ j ) s ( dx )= Z X X j p j f ( w ( x, g ( x, s ) , s, ζ j ) s ( dx ) . A standard argument shows that (4) holds for every bounded and measurable function f .Assume that s n converges weakly to s . Let f : X → R be a continuous and bounded function.Since w is jointly continuous and g is continuous(see Lemma 3), we have f ( w ( x n , g ( x n , s n ) , s n , ζ )) → f ( w ( x, g ( x, s ) , s, ζ )whenever x n → x . Let k n ( x ) := P nj =1 p j f ( w ( x, g ( x, s n ) , s n , ζ j ) and k ( x ) := P nj =1 p j f ( w ( x, g ( x, s ) , s, ζ j ). Then k n converges continuously to k , i.e., k n ( x n ) → k ( x ) whenever x n → x . Since f is bounded, the sequence k n is uniformly bounded. UsingProposition 1 and equality (4), we havelim n →∞ Z X f ( x )Φ s n ( dx ) = lim n →∞ Z X k n ( x ) s n ( dx )= Z X k ( x ) s ( dx )= Z X f ( x )Φ s ( dx ) .Thus, Φ s n converges weakly to Φ s . We conclude that Φ is continuous. Thus, by the Schauder-Tychonoff’s fixed point theorem, Φ has a fixed point. Proof of Lemma 1.
Assume that f ∈ B ( X × P ( X )) is concave and increasing in x . Since thecomposition of a concave and increasing function with a concave function is a concave function,the function f ( w ( x, a, s, ζ ) , s ) is concave in ( x, a ) for all s and ζ . Since w and f are increasingin x then f ( w ( x, a, s, ζ ) , s ) is increasing in x for all a , s and ζ . Thus, h ( x, a, s, f ) is concave in( x, a ) and increasing in x as the sum of concave and increasing functions. A standard argumentshows that T f is increasing in x . Proposition 2.3.6 in Bertsekas et al. (2003) and the fact thatΓ( x ) is convex-valued imply that T f ( x, s ) = max a ∈ Γ( x ) h ( x, a, s, f ) is concave in x .We conclude that for all n = 1 , , . . . , T n f is concave and increasing in x . Standarddynamic programming arguments (see Bertsekas and Shreve (1978)) show that T n f converges38o V uniformly. Since the set of concave and increasing functions is closed under uniformconvergence, V is concave and increasing in x .Since π is strictly concave in a , h ( x, a, s, V ) is strictly concave in a . This implies that G ( x, s ) = argmax a ∈ Γ( x ) h ( x, a, s, V ) is single-valued which proves the Lemma. B.3 Comparative statics: Proof of Theorem 4
Proof of Theorem 4.
Under the assumptions of Theorem 1, the operator M s : P ( X ) × I →P ( X ) defined by M s ( θ, e )( · ) = Z X Q ( x, s, e, · ) θ ( dx )has a unique fixed point µ s,e for each s ∈ P ( X ) and e ∈ I .Fix s ∈ P ( X ). Let θ (cid:23) SD θ and e (cid:23) I e and let B be an upper set. We have M s ( θ , e )( B ) = Z X Q ( x, s, e , B ) θ ( dx ) ≥ Z X Q ( x, s, e , B ) θ ( dx ) ≥ Z X Q ( x, s, e , B ) θ ( dx ) = M s ( θ , e )( B ) . Thus, M s ( θ , e ) (cid:23) SD M s ( θ , e ). The first inequality holds because θ (cid:23) SD θ and Q is increas-ing in x when B is an upper set. The second inequality follows from the fact that Q is increasingin e when B is an upper set.We conclude that M s is an increasing function from P ( X ) × I into P ( X ) when P ( X ) isendowed with (cid:23) SD . Thus, M ns ( θ , e ) (cid:23) SD M ns ( θ , e ) for all n ∈ N . Q being X -ergodic impliesthat M ns ( θ i , e i ) converges weakly to µ s,e i . Since (cid:23) SD is closed under weak convergence (seeKamae et al. (1977)), we have µ s,e (cid:23) SD µ s,e .Now assume that e (cid:23) I e and let s ( e ) , s ( e ) be the corresponding MFEs. Assume incontradiction that s ( e ) ≺ s ( e ). From the same argument as in Theorem 1 we can concludethat µ s ( e ) ,e (cid:23) SD µ s ( e ) ,e for each e ∈ I . Note that s ( e ) is an MFE if and only if s ( e ) = µ s ( e ) ,e .We have s ( e ) = µ s ( e ) ,e (cid:23) SD µ s ( e ) ,e (cid:23) SD µ s ( e ) ,e = s ( e ) . Transitivity of (cid:23) SD implies s ( e ) (cid:23) SD s ( e ). But since (cid:23) SD agrees with (cid:23) , s ( e ) (cid:23) SD s ( e )implies s ( e ) (cid:23) s ( e ) which is a contradiction. We conclude that s ( e ) (cid:23) s ( e ). B.4 Dynamic Oligopoly Models: Proofs of Theorems 5, 6, 7, and 8
Proof of Theorem 5.
The idea of the proof is to show that the conditions of Theorem 1and Theorem 3 hold. In Lemma 4 we prove that the optimal stationary investment strategy issingle-valued. In Lemma 5 we prove that Q is increasing in x and decreasing in s . In Lemma39 we prove that the state space can be chosen to be compact. That is, there exists a compactset ¯ X = [0 , ¯ x ] such that Q ( x, s, ¯ X ) = 1 whenever x ∈ ¯ X and all s ∈ P ( X ). This means thatif a firm’s initial state is in ¯ X , then the firm’s state will remain in ¯ X in the next period withprobability 1. In Lemma 8 we prove that Q is ¯ X -ergodic. Thus, all conditions from Theorem 1and Theorem 3 hold and we conclude that the model has a unique MFE.We first introduce some notations.Let B ( X × P ( X )) be the space of all bounded functions on X × P ( X ). For f ∈ B ( X × P ( X ))define f x ( x, s ) := ∂f ( x, s ) ∂x . For the rest of the paper we say that f ∈ B ( X × P ( X )) is differentiable if it is differentiable inthe first argument. Similarly, we write u x ( x, s ) to denote the derivative of u with respect to x .For the proof of the theorem, it will be convenient to change the decision variable in theBellman equation. Define z = (1 − δ ) x + k ( a ) , and note that we can write a = k − ( z − (1 − δ ) x ), which is well defined because k is strictlyincreasing. The resulting Bellman operator is given by Kf ( x, s ) = max z ∈ Γ( x ) J ( x, z, s, f ) , where Γ( x ) = [(1 − δ ) x + k (0) , (1 − δ ) x + k (¯ a )] and J ( x, z, s, f ) = π ( x, z, s ) + β X j p j f ( zζ j , s ) , where π ( x, z, s ) = u ( x, s ) − dk − ( z − (1 − δ ) x ).Let µ f ( x, s ) = argmax z ∈ Γ( x ) J ( x, z, s, f ) and µ ( x, s ) = argmax z ∈ Γ( x ) J ( x, z, s, V ). Note that µ ( x, s ) = (1 − δ ) x + k ( g ( x, s )) where g is the optimal stationary investment strategy. With thischange of variables, we can use the envelope theorem (see Benveniste and Scheinkman (1979)).Since u and k are continuously differentiable, then J ( x, z, s, f ) is continuously differentiable in x . The envelope theorem implies that Kf is differentiable and Kf x ( x, s ) = ∂π ( x, µ f ( x, s ) , s ) ∂x = u x ( x, s ) + d (1 − δ )( k − ) ′ ( µ f ( x, s ) − (1 − δ ) x ) . Lemma 4 µ ( x, s ) is single-valued, increasing in x and decreasing in s . Proof.
The main step of the proof is to show that if f ∈ B ( X ×P ( X )) has decreasing differencesthen Kf ∈ B ( X × P ( X )) has decreasing differences. This implies that the value function V hasdecreasing differences. An application of a Theorem by Topkis implies that µ ( x, s ) is increasingin x and decreasing in s . Single-valuedness of µ follows from the concavity of the value function.40e provide the details below.Assume that f ∈ B ( X × P ( X )) is concave in x , differentiable, and has decreasing differences.The function f ( zζ, s ) is concave and increasing in z for all s and ζ . Since k is strictly concave andstrictly increasing, k − is strictly convex and strictly increasing. This implies that − k − ( z − (1 − δ ) x ) is concave in ( x, z ). Thus, J ( x, z, s, f ) is concave in ( x, z ) as the sum of concave functions.Proposition 2.3.6 in Bertsekas et al. (2003) and the fact that Γ( x ) is convex-valued imply that Kf ( x, s ) is concave in x .Since f has decreasing differences, then f ( zζ, s ) has decreasing differences in ( z, s ) for all ζ .Thus, J has decreasing differences in ( z, s ) as the sum of functions with decreasing differences.From Theorem 6.1 in Topkis (1978), µ f ( x, s ) is decreasing in s for every x .Let x ≥ x , z ≥ z , y ′ = z − (1 − δ ) x , y = z − (1 − δ ) x and t = z − z . Note that y ≥ y ′ . Convexity of k − implies that for y ≥ y ′ and t ≥
0, we have k − ( y + t ) − k − ( y ) ≥ k − ( y ′ + t ) − k − ( y ′ ). That is, k − ( z − (1 − δ ) x ) has decreasing differences in ( x, z ). Thus, π ( x, z, s ) = u ( x, s ) − k − ( z − (1 − δ ) x ) has increasing differences in ( x, z ).Let s (cid:23) s . For every x ∈ X we have Kf x ( x, s ) = π x ( x, µ f ( x, s ) , s ) ≥ π x ( x, µ f ( x, s ) , s ) ≥ π x ( x, µ f ( x, s ) , s ) = Kf x ( x, s ) . (5)The first and last equality follow from the envelope theorem. The first inequality follows since π has decreasing differences in ( x, s ). The second inequality follows from the facts that π hasincreasing differences in ( x, z ) and µ f ( x, s ) ≥ µ f ( x, s ). Thus, Kf has decreasing differences.Define f n = K n f := K ( K n − f ) for n = 1 , , . . . where K f := f . By iterating the previousargument we conclude that f nx ( x, s ) is decreasing in s and f n ( x, s ) is concave in x for every n ∈ N .Standard dynamic programming arguments (see Bertsekas and Shreve (1978)) show that f n converges uniformly to V . Since the set of concave functions is closed under uniform conver-gence, V is concave in x . The envelope theorem implies that f nx ( x, s ) = π x ( x, µ f n ( x, s ) , s ) forevery n ∈ N . Since J ( x, z, s, f n ) is strictly concave in z when f n is concave, µ f n is single-valued. Theorem 3.8 and Theorem 9.9 in Stokey and Lucas (1989) show that µ f n → µ . Thus, f nx ( x, s ) = π x ( x, µ f n ( x, s ) , s ) → π x ( x, µ ( x, s ) , s ) = V x ( x, s ). Using (5), we conclude that V x ( x, s )is decreasing in s ; hence, V has decreasing differences. The same argument as above showsthat J ( x, z, s, V ) has decreasing differences in ( z, s ) and increasing differences in ( x, z ). Since J ( x, z, s, V ) is strictly concave in z , then µ is single-valued. It is easy to see that Γ( x ) is as-cending in the sense of Topkis (1978) (i.e., for x ≥ x if z ∈ Γ( x ) and z ′ ∈ Γ( x ) thenmax { z, z ′ } ∈ Γ( x ) and min { z, z ′ ) ∈ Γ( x )). Theorem 6.1 in Topkis (1978) implies that µ ( x, s )is increasing in x and decreasing in s which proves the Lemma.41 emma 5 Q is increasing in x for each s ∈ S and decreasing in s for each x ∈ X . Proof.
For each s ∈ P ( X ), x ≥ x and any upper set B we have Q ( x , s, B ) = Pr(((1 − δ ) x + k ( g ( x , s )) ζ ) ∈ B )= Pr( µ ( x , s ) ζ ∈ B ) ≥ Pr( µ ( x , s ) ζ ∈ B ) = Q ( x , s, B ) , where the inequality follows since µ is increasing in x . Thus, Q ( x , s, · ) (cid:23) SD Q ( x , s, · ).Similarly since µ ( x, s ) is decreasing in s , Q is decreasing in s for each x ∈ X .We prove the following useful auxiliary lemma. Lemma 6 (i) µ ( x, s ) is strictly increasing in x .(ii) For all s ∈ P ( X ) , µ is Lipschitz-continuous in the first argument with a Lipschitzconstant . That is, | µ ( x , s ) − µ ( x , s ) | ≤ | x − x | , for all x , x and s ∈ P ( X ) . Proof. (i) Fix s ∈ P ( X ). Assume in contradiction that x > x and µ ( x , s ) = µ ( x , s ).First note that µ ( x , s ) = µ ( x , s ) ≥ (1 − δ ) x + k (0) > (1 − δ ) x + k (0) := min Γ( x ). Thus,min Γ( x ) < µ ( x , s ) ≤ max Γ( x ) < max Γ( x ). We have0 ≤ − d ( k − ) ′ ( µ ( x , s ) − (1 − δ ) x ) + β n X j =1 p j ζ j V x ( µ ( x , s ) ζ j , s ) < − d ( k − ) ′ ( µ ( x , s ) − (1 − δ ) x ) + β n X j =1 p j ζ j V x ( µ ( x , s ) ζ j , s ) , which contradicts the optimality of µ ( x , s ), since µ ( x , s ) < max Γ( x ). The first inequalityfollows from the first order condition (recall that min Γ( x ) < µ ( x , s )). The second inequalityfollows from the fact that k − is strictly convex, which implies that ( k − ) ′ is strictly increasing.Thus, µ is strictly increasing in x .(ii) Fix s ∈ P ( X ). Let x > x . If µ ( x , s ) = max Γ( x ) = (1 − δ ) x + k (¯ a ), then µ ( x , s ) − µ ( x , s ) ≤ (1 − δ )( x − x ) + k (¯ a ) − k (¯ a ) ≤ x − x . So we can assume that µ ( x , s ) < max Γ( x ). Assume in contradiction that µ ( x , s ) − µ ( x , s ) > − x . Then µ ( x , s ) − (1 − δ ) x > µ ( x , s ) − (1 − δ ) x . We have0 ≥ − d ( k − ) ′ ( µ ( x , s ) − (1 − δ ) x ) + β n X j =1 p j ζ j V x ( µ ( x , s ) ζ j , s ) > − d ( k − ) ′ ( µ ( x , s ) − (1 − δ ) x ) + β n X j =1 p j ζ j V x ( µ ( x , s ) ζ j , s ) . The first inequality follows from the first order condition. The second inequality follows fromthe facts that ( k − ) is strictly convex and V is concave (see the proof of Lemma 4). The lastinequality implies that µ ( x , s ) = min Γ( x ) = (1 − δ ) x + k (0). But µ ( x , s ) ≥ min Γ( x ) implies µ ( x , s ) − µ ( x , s ) ≤ (1 − δ )( x − x ) < x − x , which is a contradiction. We conclude that µ is Lipschitz-continuous in the first argument witha Lipschitz constant 1. Lemma 7
The state space can be chosen to be compact: There exists a compact set ¯ X = [0 , ¯ x ] such that Q ( x, s, ¯ X ) = 1 whenever x ∈ ¯ X and all s ∈ P ( X ) . Proof.
Fix s ∈ P ( X ). Since max Γ( x ) = (1 − δ ) x + k (¯ a ), for all x >
0, we have µ ( x, s ) ζ n x ≤ (1 − δ ) ζ n + k (¯ a ) ζ n x . The last inequality and the fact that (1 − δ ) ζ n < x (that does notdepend on s ) such that µ ( x, s ) ζ n < x for all x ≥ ¯ x .Let ¯ X = [0 , ¯ x ]. For all s ∈ P ( X ) and ζ ∈ E , if x ∈ ¯ X we have µ ( x, s ) ζ ≤ µ (¯ x, s ) ζ n < ¯ x. That is, µ ( x, s ) ζ ∈ ¯ X . Thus, Q ( x, s, ¯ X ) = Pr( µ ( x, s ) ζ ∈ ¯ X ) = 1 whenever x ∈ ¯ X . Lemma 8 Q is ¯ X -ergodic. Proof.
Fix s ∈ P ( X ). Define the sequences x k +1 = µ ( x k , s ) ζ n and y k +1 = µ ( y k , s ) ζ where x = 0 and y = ¯ x . Note that { x n } ∞ n =1 is strictly increasing, i.e., x k +1 > x k for all k . To seethis, first note that x = µ ( x , s ) ζ n ≥ k (0) ζ n > x . Now if x k > x k − , then µ being strictlyincreasing in x (see Lemma 6 part (i)) implies that x k +1 = µ ( x k , s ) ζ n > µ ( x k − , s ) ζ n = x k . Let C s = min { x ∈ R + : µ ( x, s ) ζ n = x } . From the facts that µ (0 , s ) ζ n ≥ k (0) ζ n > µ (¯ x, s ) ζ n < ¯ x (see Lemma 7), and µ is continuous (see Lemma 3), by Brouwer fixed point theorem C s is welldefined. Similarly, the sequence { y n } ∞ n =1 is strictly decreasing and therefore converges to a limit C ∗ s . 43e claim that C s > C ∗ s . To see this, first note that Lemma 7 implies that the function f s , defined by f s ( x, ζ ) = µ ( x, s ) ζ , is from ¯ X × E into ¯ X . Note that f s is increasing in botharguments and that ¯ X is a complete lattice. Thus, Corollary 2.5.2 in Topkis (2011) impliesthat the greatest and least fixed points of f s are increasing in ζ . Lemma 6 part (ii) and ζ < f s ( x, ζ ) = µ ( x, s ) ζ is a contraction mapping from ¯ X to itself. Thus, f s ( x, ζ ) hasa unique fixed point which equals the limit of the sequence { y n } ∞ n =1 , C ∗ s . Since the least fixedpoint of f s is increasing in ζ we conclude that C s ≥ C ∗ s . Since µ is increasing and positive wehave C s = µ ( C s , s ) ζ n > µ ( C s , s ) ζ ≥ µ ( C ∗ s , s ) ζ = C ∗ s Let x ∗ = ( C s + C ∗ s ) /
2. Since x k ↑ C s and y k ↓ C ∗ s , there exists a finite N such that x k > x ∗ for all k ≥ N , and similarly, there exists a finite N such that y k < x ∗ for all k ≥ N . Let m = max { N , N } . Thus, after m periods there exists a positive probability ( ζ m ) to movefrom the state ¯ x to the set [0 , x ∗ ], and a positive probability to move from the state 0 tothe set [ x ∗ , ¯ x ]. That is, we found x ∗ ∈ [0 , ¯ x ] and m > Q m (¯ x, s, [0 , x ∗ ]) > Q m (0 , s, [ x ∗ , ¯ x ]) >
0. Since ¯ X is compact and Q is increasing in x , then Q is ¯ X -ergodic (seeTheorem 2 in Hopenhayn and Prescott (1992) or Theorem 2.1 in Bhattacharya and Lee (1988)).Now, we prove Theorem 6. The main idea behind the proof is to show that the optimal sta-tionary strategy g is increasing or decreasing in the relevant parameter using a lattice-theoreticalapproach and then to conclude that the conditions of Theorem 4 hold.Let ( I, (cid:23) I ) be a partial order set that influences the firms’ decisions. We denote a genericelement in I by e . For instance, e can be the discount factor or the cost of a unit of investment.Throughout the proof of Theorem 6 we allow an additional argument in the functions that weconsider. For instance, the value function V is denoted by: V ( x, s, e ) = max a ∈ [0 , ¯ a ] h ( x, a, s, e, V ) . Likewise, the optimal stationary strategy is denoted by g ( x, s, e ), and u ( x, s, e ) is the one-periodprofit function. Here, we come back to the original formulation over actions a . Proof of Theorem 6. i) Assume that f ∈ B ( X × P ( X ) × I ) is concave in the first argumentand has decreasing differences in ( x, d ) where I ⊆ R + is the set of all possible unit investmentcosts endowed with the natural order, d ≥ d .Fix s ∈ P ( X ). Note that da has increasing differences in ( a, d ). Thus, u ( x, s ) − da hasdecreasing differences in ( a, d ), ( x, a ) and ( x, d ). Since f has decreasing differences and k isincreasing, the function f (((1 − δ ) x + k ( a )) ζ, s, d ) has decreasing differences in ( a, d ) and ( x, d )for every ζ ∈ E . Since f is concave in the first argument and k is increasing, it can be shownthat the function f (((1 − δ ) x + k ( a )) ζ, s, d ) has decreasing differences in ( x, a ) for every ζ ∈ E .44hus, the function h ( x, a, s, d, f ) = u ( x, s ) − da + β n X j =1 p j f (((1 − δ ) x + k ( a )) ζ j , s, d )has decreasing differences in ( x, a ), ( x, d ) and ( a, d ) as the sum of functions with decreasingdifferences.A similar argument to Lemma 1 in Hopenhayn and Prescott (1992) or Lemma 2 in Lovejoy(1987) implies that if h ( x, a, s, d, f ) has decreasing differences in ( x, a ), ( x, d ) and ( a, d ), then T f ( x, s, d ) = max a ∈ [0 , ¯ a ] h ( x, a, s, d, f ) has decreasing differences in ( x, d ). The proof of Lemma 4implies that T f is concave in x . We conclude that for all n = 1 , , ... , T n f is concave in x and hasdecreasing differences. Standard dynamic programming arguments (see Bertsekas and Shreve(1978)) show that T n f converges to V uniformly. Since the set of functions with decreasingdifferences is closed under uniform convergence, V has decreasing differences in ( x, d ). Fromthe same argument as above, h ( x, a, s, d, V ) has decreasing differences in ( a, d ). Theorem 6.1 inTopkis (1978) implies that g ( x, s, d ) is decreasing in d .Define the order (cid:23) I by d (cid:23) I d if and only if d ≥ d . Thus d (cid:23) I d implies that Q ( x, s, d , B ) = Pr(((1 − δ ) x + k ( g ( x, s, d )) ζ ∈ B ) ≥ Pr(((1 − δ ) x + k ( g ( x, s, d )) ζ ∈ B )= Q ( x, s, d , B )for all x, s and every upper set B , because g ( x, s, d ) is decreasing in d . That is, Q ( x, s, d , · ) (cid:23) SD Q ( x, s, d , · ) for all x, s and d , d ∈ I such that d (cid:23) I d . From Theorem 4 and Theorem 5 weconclude that d (cid:23) I d implies s ( d ) (cid:23) s ( d ), i.e., d ≤ d implies s ( d ) (cid:23) s ( d ).(ii) The proof of part (ii) is the same as the proof of part (i) and is therefore omitted.(iii) Assume that f ∈ B ( X × P ( X ) × I ) is increasing in the first argument and has decreasingdifferences in ( x, β ) where I = (0 ,
1) is the set of all possible discount factors endowed with thereverse order; β (cid:23) I β if and only if β ≥ β . A standard argument shows that T f is increasingin the first argument. We will only show that h ( x, a, s, β, f ) has decreasing differences in ( a, β )and ( x, β ); the rest of the proof is the same as the proof of part (i). Fix s , x and let β (cid:23) I β (i.e., β ≥ β ), and a ≥ a . Decreasing differences of f and the fact that k is increasing implythat f (((1 − δ ) x + k ( a )) ζ, s, β ) − f (((1 − δ ) + k ( a )) ζ, s, β ) is decreasing in β for all ζ ∈ E .45ince β ≥ β , f and k are increasing, we have β n X j =1 p j ( f (((1 − δ ) x + k ( a )) ζ j , s, β ) − f (((1 − δ ) x + k ( a )) ζ j , s, β )) ≤ β n X j =1 p j ( f (((1 − δ ) x + k ( a )) ζ j , s, β ) − f (((1 − δ ) x + k ( a )) ζ j , s, β )) . Thus h ( x, a, s, β, f ) has decreasing differences in ( a, β ). A similar argument shows that h ( x, a, s, β, f )has decreasing differences in ( x, β ). Proof of Theorem 7. (i) The proof of the Theorem is similar to the proof of Theorem 5.The idea of the proof is to show that the conditions of Theorem 9 hold. We now show that Q is increasing in x and decreasing in s (see Section A.1 for the definition of Q ).We use the same change of variables and notation as in the proof of Theorem 5. Define z = (1 − δ )( x + a ) (6)and note that a = (1 − δ ) − z − x . The resulting Bellman operator is given by Kf ( x, s ) = max z ∈ Γ( x ) J ( x, z, s, f ) , where Γ( x ) = [(1 − δ )( x + 1) , (1 − δ )( x + a )], J ( x, z, s, f ) = π ( x, z, s ) + β X j p j f ( zζ j , s ) , and π ( x, z, s ) = r ( x + (1 − δ ) − z − x ) γ (cid:0)R ( x ′ + (1 − δ ) − z ′ − x ′ ) s ( dx ′ , dz ′ ) (cid:1) γ − (1 − δ ) − z − x = r (cid:0) (1 − δ ) − z (cid:1) γ (cid:0)R (1 − δ ) − z ′ s ( dx ′ , dz ′ ) (cid:1) γ − x − (1 − δ ) − z. Let µ ( x, s ) = argmax z ∈ Γ( x ) J ( x, z, s, V ). Since π is concave in ( x, z ), Lemma 4 implies that thepolicy function µ ( x, s ) is single-valued.It is immediate that π has increasing differences in ( x, z ), and decreasing differences in ( z, s )and ( x, s ). Here s (cid:23) s if and only if Z (1 − δ ) − z ′ s ( dx ′ , dz ′ ) ≥ Z (1 − δ ) − z ′ s ( dx ′ , dz ′ ) . From Lemma 4, we can show that µ is increasing in x and decreasing in s .46hus, for each s ∈ P ( X × A ), x ≥ x and any upper set B × D ∈ B ( X × A ) we have Q ( x , s, B × D ) = n X j =1 p j B × D ( µ ( x , s ) ζ j , µ ( µ ( x , s ) ζ j , s )) ≥ n X j =1 p j B × D ( µ ( x , s ) ζ j , µ ( µ ( x , s ) ζ j , s ))= Q ( x , s, B × D ) . The equalities follow from the proof of Theorem 9. The inequality follows because µ is increasingin x . Thus, Q ( x , s, · ) (cid:23) SD Q ( x , s, · ), i.e., Q is increasing in x .Similarly, because µ ( x, s ) is decreasing in s , we can show that Q is decreasing in s for each x ∈ X .We conclude that Q is decreasing in s and increasing in x . Compactness of the state space X and X -ergodicity of Q can be established using similar arguments to the arguments in Theorem5. Thus, all the conditions of Theorem 9 parts (i) and (ii) hold. We conclude that the dynamicadvertising model has a unique MFE.The proofs of parts (ii) and (iii) are similar to the proof of Theorem 6 and are thereforeomitted. Proof of Theorem 8. (i) First note that the state space X = [0 , M ] × [0 , M ] is compact.We now show that Q is increasing in x and decreasing in s with respect to x .For the proof of the theorem, it will be convenient to change the decision variable in theBellman equation. Define z = x x x + 11 + x k ( a ) , and note that we can write a = k − ( z (1 + x ) − x x ), which is well defined because k is strictlyincreasing. The resulting Bellman operator is given by Kf ( x , x , s ) = max z ∈ Γ( x ,x ) J ( x , x , z, s, f ) , where Γ( x , x ) = [ x x x + x k (0) , x x x + x k (¯ a )], J ( x , x , z, s, f ) = π ( x , x , z, s ) + β X j p j f (cid:18) min (cid:18) z + ζ j x , M (cid:19) , min( x + 1 , M ) , s (cid:19) , and π ( x , x , z, s ) = ν ( x , x ) R ν ( x , x ) s ( d ( x , x )) − dk − ( z (1 + x ) − x x ) . Let µ ( x , x , s ) = argmax z ∈ Γ( x ,x ) J ( x , x , z, s, V ). From the arguments as the arguments in47emma 4, the optimal stationary strategy µ ( x , x , s ) is single-valued.Let x ′ ≤ x and s (cid:23) s . Because ν is increasing, we have ν ( x , x ) (cid:18) R ν ( x , x ) s ( d ( x , x )) − R ν ( x , x ) s ( d ( x , x )) (cid:19) ≤ ν ( x ′ , x ) (cid:18) R ν ( x , x ) s ( d ( x , x )) − R ν ( x , x ) s ( d ( x , x )) (cid:19) . Thus, π has decreasing differences in ( x , s ). In addition, π has decreasing differences in ( z, s )and increasing differences in ( x , z ) (see the proof of Lemma 4). From Lemma 4, we can showthat µ is increasing in x and decreasing in s .Recall that in every period, with probability 1 − β , each seller departs the market and a newseller with state (0 ,
0) immediately arrives to the market. With probability β , each seller staysin the market and moves to a new state according to the dynamics described in Section 4.3.Thus, we have Q ( x , x , s, B × B ) = (1 − β ) δ { (0 , } ( B × B )+ β Pr (cid:18) (min (cid:18) µ ( x , x , s ) + ζ j x , M (cid:19) , min( x + 1 , M )) ∈ B × B (cid:19) = (1 − β )1 B × B (0 , β n X j =1 p j B × B (min (cid:18) µ ( x , x , s ) + ζ j x , M (cid:19) , min( x + 1 , M ))where δ { c } is the Dirac measure on the point c ∈ R . Let f : X × X → R be increasing in thefirst argument. Assume that x ′ ≤ x . We have Z X f ( y , y ) Q (( x ′ , x ) , s, d ( y , y )) = (1 − β ) f (0 , β n X j =1 p j f (cid:18) min (cid:18) µ ( x ′ , x , s ) + ζ j x , M (cid:19) , min( x + 1 , M ) (cid:19) ≤ (1 − β ) f (0 , β n X j =1 p j f (cid:18) min (cid:18) µ ( x , x , s ) + ζ j x , M (cid:19) , min( x + 1 , M ) (cid:19) = Z X f ( y , y ) Q (( x , x ) , s, d ( y , y )) . The inequality follows from the facts that µ is increasing in x , and f is increasing in the firstargument.We conclude that Q is increasing in x . Similarly, because µ is decreasing in s , we can provethat Q is decreasing in s with respect to x . We now show that Q is X -ergodic.48he Markov chain Q is said to satisfy the Doeblin condition if there exists a positive integer n , ǫ > υ on X such that Q n ( x, s, B ) ≥ ǫυ ( B ) for all x ∈ X andall measurable B . From the definition of Q , we have Q ( x, s, B ) ≥ (1 − β ) δ { (0 , } ( B ) for everymeasurable B , so Q satisfies the Doeblin condition. Thus, Q is X -ergodic (see Theorem 8 inRoberts and Rosenthal (2004)).Thus, all the conditions of Theorem 2 and Theorem 3 are satisfied. We conclude that thedynamic reputation model has a unique MFE.(ii) The proof of part (ii) is similar to the proof of Theorem 6 and is therefore omitted. B.5 Heterogeneous Agent Macro Models: Proof of Corollary 3
Proof of Corollary 3.
From Theorem 2 we only need to show that Q is increasing in x anddecreasing in s in order to prove Corollary 3.Let f : X × X → R be increasing in the first argument. Assume that x ′ ≤ x . We have Z X f ( y , y ) Q (( x ′ , x ) , s, d ( y , y )) = X j p j f (˜ g ( x ′ , x , H ( s )) , m ( x , ζ j )) ≤ X j p j f (˜ g ( x , x , H ( s )) , m ( x , ζ j ))= Z X f ( y , y ) Q (( x , x ) , s, d ( y , y )) . The inequality follows from the facts that ˜ g is increasing in x and f is increasing in the firstargument. In a similar manner, because ˜ g is decreasing in the aggregator, we can show that Q is decreasing in s with respect to x .We conclude that Q is increasing in x and decreasing in s . B.6 Extensions: Proofs of Theorem 9 and Lemma 2
Proof of Theorem 9.
The proofs of part (i) and of part (iii) are the same as the proofs ofTheorem 1 and of Theorem 4. To prove part (ii) we need to show that the operator Φ : P ( X ) →P ( X ) defined by Φ s ( B × D ) = Z X Q ( x, s, B × D ) s ( dx, A ) . is continuous (the rest of the proof is the same as the proof of Theorem 3). The continuity ofΦ follows from a similar argument to the argument in the proof of Theorem 3. We provide theproof for completeness.Note that for every bounded and measurable function f : X × A → R and for every s ∈ ( X × A ) we have Z X × A f ( x, a ) Φ s ( d ( x, a )) = Z X n X j =1 p j f ( w ( x, g ( x, s ) , s, ζ j ) , g ( w ( x, g ( x, s ) , s, ζ j ) , s )) s ( dx, A ) . (7)To see this, first assume that f = 1 B × D for some measurable set B × D ∈ B ( X × A ). We have Z X × A f ( x, a ) Φ s ( d ( x, a )) = Φ s ( B × D )= Z X Z B D ( g ( y, s )) Q ( x, s, dy ) s ( dx, A )= Z X Z X B ( y ) 1 D ( g ( y, s )) Q ( x, s, dy ) s ( dx, A )= Z X Z X B × D ( y, g ( y, s )) Q ( x, s, dy ) s ( dx, A )= Z X n X j =1 p j B × D ( w ( x, g ( x, s ) , s, ζ j ) , g ( w ( x, g ( x, s ) , s, ζ j ) , s )) s ( dx, A )= Z X n X j =1 p j f ( w ( x, g ( x, s ) , s, ζ j ) , g ( w ( x, g ( x, s ) , s, ζ j ) , s )) s ( dx, A ) . A standard argument shows that Equation (7) holds for every bounded and measurable function f . Assume that s n converges weakly to s . Thus, the marginal distribution s n ( · , A ) convergesweakly to s ( · , A ). Let f : X × A → R be a continuous and bounded function. Because w and g are continuous, we have f ( w ( x n , g ( x n , s n ) , s n , ζ ) , g ( w ( x n , g ( x n , s n ) , s n , ζ ) , s n )) → f ( w ( x, g ( x, s ) , s, ζ ) , g ( w ( x, g ( x, s ) , s, ζ ) , s ))whenever x n → x .Let k n ( x ) := n X j =1 p j f ( w ( x, g ( x, s n ) , s n , ζ j ) , g ( w ( x, g ( x, s n ) , s n , ζ j ) , s n ))and k ( x ) := n X j =1 p j f ( w ( x, g ( x, s ) , s, ζ j ) , g ( w ( x, g ( x, s ) , s, ζ j ) , s )) . Then k n converges continuously to k , i.e., k n ( x n ) → k ( x ) whenever x n → x . Since f is50ounded, the sequence k n is uniformly bounded. Using Proposition 1 yieldslim n →∞ Z X × A f ( x, a )Φ s n ( d ( x, a )) = lim n →∞ Z X k n ( x ) s n ( dx, A )= Z X k ( x ) s ( dx, A )= Z X × A f ( x, a )Φ s ( d ( x, a ))) . Thus, Φ s n converges weakly to Φ s . We conclude that Φ is continuous. Proof of Lemma 2.
Let f : X × Θ → R be increasing in in the first. The fact that Q θ isincreasing in x implies that the function Z X × Θ f ( y, θ ′ ) Q h ( x, θ, s h , d ( y, θ ′ )) = Z X × Θ f ( y, θ ′ ) Q θ ( x, s h , dy )1 D ( dθ ′ )is increasing in x for every type θ and every extended population state s h . That is, Q h isincreasing in x . Similarly, Q h is decreasing in s h with respect to x when Q θ is decreasing in s hh