A Link between Sequential Semi-anonymous Nonatomic Games and their Large Finite Counterparts
aa r X i v : . [ q -f i n . E C ] J un A Link between Sequential Semi-anonymous NonatomicGames and their Large Finite Counterparts
Jian YangDepartment of Management Science and Information SystemsBusiness School, Rutgers UniversityNewark, NJ 07102Email: [email protected] 2015; revised, March 2016; accepted, May 2016
Abstract
We show that obtainable equilibria of a multi-period nonatomic game can be usedby players in its large finite counterparts to achieve near-equilibrium payoffs. Suchequilibria in the form of random state-to-action rules are parsimonious in form andeasy to execute, as they are both oblivious of past history and blind to other players’present states. Our transient results can be extended to a stationary case, where thefinite multi-period games are special discounted stochastic games. In both nonatomicand finite games, players’ states influence their payoffs along with actions they take;also, the random evolution of one particular player’s state is driven by all players’ statesas well as actions. The finite games can model diverse situations such as dynamic pricecompetition. But they are notoriously difficult to analyze. Our results thus suggestways to tackle these problems approximately.
Keywords:
Nonatomic Game; Convergence in Probability; ǫ -Equilibrium MSC2000 Classification Codes: Introduction
We show that an equilibrium of a random multi-period game involving a continuum ofplayers can be used to achieve asymptotically equilibrium results for its large finite coun-terparts. The latter finite games can model competitive situations involving random andaction-dependent evolution of players’ states which in turn influence period-wise payoffs.Their complex natures make equilibria difficult to locate. In contrast, those for the formercontinuum-player game are simple in form and relatively easy to obtain. Therefore, a bridgebetween the two types of games can have broad practical implications.The former continuum-player game can be termed more formally as a sequential semi-anonymous nonatomic game (SSNG). In it, a continuum of players interact with one anotherin multiple periods; also, each player’s one-time payoff and random one-period state tran-sition are both swayed by his own state and action, as well as the joint distribution ofother players’ states and actions. This is indeed the anonymous sequential game studiedby Jovanovic and Rosenthal [16]. We use the name SSNG just to be consistent with thesingle-period nonatomic-game (NG) literature, where anonymity has been reserved for amore special case. An SSNG’s finite counterpart is almost the same except that only a finitenumber of players are involved. This more realistic situation is much more difficult to handle.In a few steps, we demonstrate the usefulness of an SSNG equilibrium in finite multi-period games. First, in a precise language, Theorem 1 describes the gradual retreat ofrandomness in finite games as the number n of players tends to + ∞ . This paves way forTheorem 2, which states that an SSNG’s conditional equilibria, in terms of random state-to-action rules, can be used by its large finite counterparts to reach asymptotically equilibriumpayoffs on average. A further refinement of this result is achieved in Theorem 3. Theabove transient results can be extended to the stationary case involving discounted payoffsand infinite horizons; see Theorem 4. The conditional equilibria that facilitate our study aresimilar to well-understood distributional equilibria. Their existence is also directly verifiable.One practical situation to which our results can be applied concerns dynamic price com-petition. Here, players may be firms producing one identical product type, states may becombinations of the firms’ inventory levels and other static or dynamic characteristics such asunit costs, and actions may be unit prices the firms charge for the product. In every period,the random demand arriving to a firm is dependent on not only its own price but also pricescharged by other competitors. The actual sales is further constrained by available inventory.So the player’s one-time payoff is a function of both its own state (inventory and probably2lso cost) and action (price) and of the distribution of others’ actions (prices). Moreover,the firm’s next-period inventory level depends on its current level, the random demand, andpotentially an exogenously given production schedule. So the random single-period statetransition is potentially a function of the same factors involved in the payoff.It is a difficult task to predict or prescribe what inventory-dependent prices the firms willor should charge over a finite time horizon. This can be further complicated by diverse sce-narios where firms have different degrees of knowledge on their competitors’ inventory levelsand/or costs. Our results, on the other hand, will reveal that the nonatomic counterpartSSNG is easier to tackle. Its equilibria can be plugged back to the actual finite-player situ-ations, without regard to the particularities of the scenarios, and still make reasonably wellpredictions/prescriptions when the number of players is large enough. We are not equippedto answer how large is “large enough”. But computational study done in a related pricingsetting hinted that player numbers “in the tens” seem large enough; see, Yang and Xia [35].In the remainder of the paper, Section 2 surveys the relevant literature. We then spendSections 3 and 4 on essentials of SSNGs and finite games, respectively. In Section 5, wedemonstrate the key result that state evolutions in large finite games will not veer toofar away from their NG counterparts. Section 6 is devoted to the main transient resultand Section 7 its detailed interpretation. This result is extended to the stationary case inSection 8. Implications of these results and existence of our kind of equilibria for SSNGs areshown in Section 9. We conclude the paper in Section 10. From early on, NGs have been used as easier-to-analyze proxies of real finite-player situa-tions, such as in the study of perfect competition. Systematic research on NG started withSchmeidler [28]. He formulated a single-period semi-anonymous NG, wherein the joint dis-tribution of other players’ identities and actions may affect any given player’s payoff. Whenthe action space is finite, Schmeidler established the existence of pure equilibria when thegame becomes anonymous, so that other players’ influence on a game’s outcome is chan-neled through the marginal action distribution alone. Mas-Colell [24] showed the existenceof distributional equilibria in anonymous NGs with compact metric action spaces. The lat-ter result was extended by Khan and Sun [20] to a case where players differ on how theirpreferences over actions are influenced by external action distributions. A survey of related3orks up until the early 2000s was provided by Khan and Sun [23].Much attention has been paid to the topic of pure-equilibrium existence. Khan andSun [21] developed a purification scheme involving a countable compact metric action space.Khan and Sun [22] used non-standard measures on identity spaces and generalized Schmei-dler’s pure-equilibrium existence result for more general action spaces. Balder [4] establishedpure- and mixed-equilibrium existence results that may be regarded as generalizations ofSchmeidler’s corresponding results. Other notable works still include Yu and Zhang [36] andBalder [5]. On the other hand, Khan, Rath, and Sun [18] identified a certain limit to whichSchmeidler’s result can be extended. Recently, Khan et al. [19] took players’ diverse bio-social traits into consideration and pinpointed saturation of the player-identity distributionas the key to existence of pure equilibria.Links between NGs and their finite counterparts were covered in Green [13], Housman[15], Carmona [8], Kalai [17], Al-Najjar [3], and Yang [33]. For multi-period games withoutchanging states, Green [12], Sabourian [27], and Al-Najjar and Smorodinsky [2] showed thatequilibria for large games are nearly myopic.SSNGs are both challenging and rewarding to analyze because in them, very realistically,individual states are subject to sways of players’ own actions as well as their opponents’states and actions. Jovanovic and Rosenthal [16] established the existence of distributionalequilibria for such games. This result was generalized by Bergin and Bernhardt [6] to casesinvolving aggregate shocks. In SSNGs’ finite-player counterparts, however, randomness instate evolution will not go away. Besides, a player’s ability to observe other players’ statesand actions might also affect his decision. Presented with these difficulties, it is not surprisingthat known results on sequential finite-player games are restricted to the stationary setting,where they appear as discounted stochastic games first introduced by Shapley [29]. Accordingto Mertens and Parthasrathy [25], Duffie, Geanakoplos, Mas-Colell, and McLennan [9], andSolan [30], for instance, equilibria known to exist for these games come in quite complicatedforms that for real implementation, demand a high degree of coordination among players.It is therefore natural to ask whether sequential finite-player games can be approximatedby their NG counterparts. This question has so far been answered by two unpublished arti-cles. For a case unconcerned with the copula between marginal state and action distributions,Bodoh-Creed [7] provided an affirmative answer, and went on to show for certain cases thatlimits of large-game equilibria of a myopic form, when in existence, are NG equilibria. Also,Yang [34] verified for the approximability when both state transitions and action plans are4riven by exogenously generated idiosyncratic shocks. Our current study attempts with themost general possible setting, without unduly restricting on ways in which a player’s payoffcan be influenced by other players’ states and actions or ways in which the game can evolverandomly. To achieve results of the same spirit, we have to overcome technical challengesposed by the new phenomenon of sampling from non-product joint probabilities.Some authors went on to pursue stationary equilibria (SE), which stressed the long-run steady-state nature of individual action plans and system-wide multi-states; see, e.g.,Hopenhayn [14] and Adlakha and Johari [1]. The oblivious equilibrium (OE) concept asproposed by Weintraub, Benkard, and van Roy [31], in order to account for impacts oflarge players, took the same stationary approach by letting participants beware of only long-run average system states. Weintraub, Benkard, and van Roy [32] showed links betweenequilibria of infinite-player games and their finite-player brethren for a setting where thelong-run average system state could be defined. Though applicable to many situations, wecaution that the implicit stationarity of SE or OE is incompatible with applications that aretransient by nature; for instance, the dynamic pricing game studied by Yang [34].
The SSNG is a game in which a continuum of players interact with one another over multipleperiods. A realistic and yet complicating feature is that players possess individual stateswhich influence their payoffs along with all players’ actions. The random evolutions of thesestates, meanwhile, are affected by players’ actions. Furthermore, the semi-anonymous natureof the game means that not only what was done, but also who did what to the extent atwhich states partially reveal player identities, figure large in both payoff formation and stateevolution. We now provide a detailed account of the game.
For some natural number ¯ t ∈ N , we let periods 1 , , ...., ¯ t serve as regular periods andperiod ¯ t + 1 as the terminal period. For all periods, we let players’ individual states andactions form, respectively, separable metric spaces S and X . We further require that bothspaces be discrete . In this paper, such a space always stands for a separable metric spacewith countably many elements and the additional feature that the minimum of the distancesbetween any two points remains strictly positive. The discreteness requirement will be useful5t one occasion. But most of our derivations will work if the spaces were merely separablemetric. Given any separable metric space A , we use B ( A ) for its Borel σ -field and P ( A ) forthe set of all probability measures on the measurable space ( A, B ( A )).To each player, other players’ states and actions are immediately felt in a semi-anonymousfashion, so that what really matters is the joint distribution of other players’ states andactions. This distribution, which we dub “in-action environment”, is a member of the jointstate-action distribution space P ( S × X ). In any period t = 1 , , ..., ¯ t , a player’s state s ∈ S ,his action x ∈ X , and the in-action environment τ ∈ P ( S × X ) he faces, together determinehis payoff in that period. In particular, there is a function˜ f t : S × X × P ( S × X ) → [ − ¯ f t , ¯ f t ] , (1)where ¯ f t is some positive constant on the real line R . It is required that ˜ f t ( · , · , τ ) be ameasurable map from S × X to [ − ¯ f t , ¯ f t ] for every τ ∈ P ( S × X ). For the terminal period¯ t + 1, we let the payoff be 0 in all circumstances.Now we describe individual players’ random state transitions. Given separable metricspaces A and B , we use K ( A, B ) to represent the space of all kernels from A to B . Eachmember κ ∈ K ( A, B ) ⊆ ( P ( B )) A satisfies that(i) κ ( a ) is a member of P ( B ) for each a ∈ A , and(ii) for each B ′ ∈ B ( B ), the real-valued function κ ( ·| B ′ ) is measurable.Note that we have used κ ( a | B ′ ) rather than the more conventional κ ( B ′ | a ) to denote theconditional probability for B ′ ∈ B ( B ) when given a ∈ A . The current notation allows us toalways read a formula from left to right. Now in period 1 , , ..., ¯ t , let there be a function˜ g t : S × X × P ( S × X ) → P ( S ) , (2)so that ˜ g t ( · , · , τ ) is a member of K ( S × X, S ) for each τ ∈ P ( S × X ). For convenience, weuse G ( S, X ) to denote the space of all such functions, or what we shall call “state transitionkernels”. In period t , when a player is in individual state s ∈ S , takes action x ∈ X , andfaces in-action environment τ ∈ P ( S × X ), there will be a ˜ g t ( s, x, τ | S ′ ) chance for his statein period t + 1 to be in any S ′ ∈ B ( S ).This setup is versatile enough to embrace different player characteristics. For instance,each s ∈ S may comprise two components θ and ω , with the ˜ g t ’s defined through (2) dictatingthat θ stays static over time to serve as a player’s innate type. Certainly, the ˜ f t ’s definedthrough (1) can have all kinds of trends over θ to reflect players’ varying payoff structures.6 .2 Evolution of the Environments In any period 1 , , ..., ¯ t, ¯ t + 1, by “pre-action environment” we mean the state distribution σ ∈ P ( S ) of all players. With ¯ t , S , X , ( ˜ f t | t = 1 , , ..., ¯ t ), and (˜ g t | t = 1 , , ..., ¯ t ) all given inthe background, we use Γ( σ ) to denote an (SS)NG with σ ∈ P ( S ) as its initial period-1pre-action environment. For this NG, we can use χ [1¯ t ] = ( χ t | t = 1 , ..., ¯ t ) ∈ ( K ( S, X )) ¯ t to denote a policy profile. Here, each χ t ∈ K ( S, X ) is a map from a player’s state to theplayer’s random action choice. Together with the given initial environment σ , this policyprofile will help to generate a deterministic pre-action environment trajectory σ [1 , ¯ t +1] = ( σ t | t = 1 , , ..., ¯ t, ¯ t + 1) ∈ ( P ( S )) ¯ t +1 in an iterative fashion. This process is also intertwined withthe formation of in-action environments τ , τ , ..., τ ¯ t faced by all players in periods 1 , , ..., ¯ t .More notation is needed to precisely describe this evolution. Given distribution p ∈ P ( A )and kernel κ ∈ K ( A, B ) for separable metric spaces A and B , there is a natural product p ⊗ κ ∈ P ( A × B ), such that( p ⊗ κ )( A ′ × B ′ ) = Z A ′ p ( da ) · κ ( a | B ′ ) , ∀ A ′ ∈ B ( A ) , B ′ ∈ B ( B ) . (3)Here, p ⊗ κ is essentially the joint distribution generated by the marginal p and conditionaldistribution κ . Obviously, ( p ⊗ κ ) | A , the marginal of p ⊗ κ on A , is p . At the same time, weuse p ⊙ κ to denote the marginal ( p ⊗ κ ) | B , which satisfies( p ⊙ κ )( B ′ ) = ( p ⊗ κ ) | B ( B ′ ) = ( p ⊗ κ )( A × B ′ ) = Z A p ( da ) · κ ( a | B ′ ) , ∀ B ′ ∈ B ( B ) . (4)Suppose pre-action environment σ t ∈ P ( S ) has been given for some period t = 1 , ..., ¯ t .Then, for every player with starting state s t in the period, his random action will be sampledfrom the distribution χ t ( s t |· ) where as noted before, χ t ∈ K ( S, X ) is every player’s behavioralguide. Thus, all players will together form the commonly felt in-action environment τ t = σ t ⊗ χ t . (5)For each individual player with state s t and realized action x t , his state s t +1 in period t + 1will, by (2), be distributed according to ˜ g t ( s t , x t , τ t |· ). Thus, it will be reasonable for thepre-action environment in period t + 1 to follow σ t +1 = τ t ⊙ ˜ g t ( · , · , τ t ), with[ τ t ⊙ ˜ g t ( · , · , τ t )]( S ′ ) = Z S × X τ t ( ds × dx ) · ˜ g t ( s, x, τ t | S ′ ) , ∀ S ′ ∈ B ( S ) . (6)Although (6) has been intuitively reasoned from (2), we caution that logically it is part ofthe NG’s definition rather than something derivable from the latter.7he transition from σ t to σ t +1 through random action plan χ t is best expressed by anoperator. For any kernel χ ∈ K ( S, X ), define operator T t ( χ ) on the space P ( S ), so that T t ( χ ) ◦ σ = ( σ ⊗ χ ) ⊙ ˜ g t ( · , · , σ ⊗ χ ) = σ ⊙ χ ⊙ ˜ g t ( · , · , σ ⊗ χ ) , ∀ σ ∈ P ( S ) . (7)Basically, state distribution σ and random state-dependent action plan χ first fuse to formthe joint state-action distribution σ ⊗ χ to be felt by all players. The latter’s random statetransitions are then guided by the kernel ˜ g t ( · , · , σ t ⊗ χ ). Subsequently, after “averaging out”impacts of actions, the next-period state distribution will become σ ⊙ χ ⊙ ˜ g t ( · , · , σ ⊗ χ ). Theone-period pre-action environment transition is now representable by σ t +1 = T t ( χ t ) ◦ σ t = σ t ⊙ χ t ⊙ ˜ g t ( · , · , σ t ⊗ χ t ) . (8)For periods t and t ′ with t ≤ t ′ , as well as sequence χ [ tt ′ ] = ( χ t ′′ | t ′′ = t, ..., t ′ ) of actionplans, we can iteratively define T [ tt ′ ] ( χ [ tt ′ ] ), so that T [ tt ′ ] ( χ [ tt ′ ] ) ◦ σ t = T t ′ ( χ t ′ ) ◦ ( T [ t,t ′ − ( χ [ t,t ′ − ) ◦ σ t ) , ∀ σ t ∈ P ( S ) . (9)The left-hand side will be players’ state distribution in period t ′ + 1 when they start period t with the distribution σ t and adopt the action sequence χ [ tt ′ ] in the interim. Note that T [ tt ] ( χ [ tt ] ) is nothing but T t ( χ t ). As a default, we let T [ t,t − stand for the identity operatoron P ( S ). The environment trajectory σ [1 , ¯ t +1] satisfies σ [1 , ¯ t +1] = ( T [1 ,t − ( χ [1 ,t − ) ◦ σ | t = 1 , , ..., ¯ t, ¯ t + 1) . (10)It is deterministic by definition. n -player Game Let the same ¯ t , S , X , ( ˜ f t | t = 1 , , ..., ¯ t ), and (˜ g t | t = 1 , , ..., ¯ t ) remain in the background.For some n ∈ N \ { } and initial multi-state s = ( s , s , ..., s n ) ∈ S n , we can define an n -player game Γ n ( s ), in which each s m ∈ S is player m ’s initial state. The game’s payoffsand state evolutions are still described by the ˜ f t ’s and ˜ g t ’s, respectively. However, details aremessier as outside environments vary from player to player and their evolutions are random.For a ∈ A , where A is again a separable metric space, we use δ a to denote the singletonDirac measure with δ a ( { a } ) = 1. For a = ( a , ..., a n ) ∈ A n where n ∈ N , we use ε a for P nm =1 δ a m /n , the empirical distribution generated by the vector a . We also use P n ( A ) to8enote the space of probability measures of the type ε a for a ∈ A n , i.e., the space of empiricaldistributions generated from n samples. Now back at the game Γ n ( s ), suppose in period t = 1 , , ..., ¯ t , each player m = 1 , , ..., n is in state s tm and takes action x tm . Then, thein-action environment experienced by player 1 will be ε s t, − x t, − = ε (( s t ,x t ) ,..., ( s tn ,x tn )) . Thus,this player will receive payoff ˜ f t ( s t , x t , ε s t, − x t, − ) in the period, and his period-( t + 1) state s t +1 , will be sampled from the distribution ˜ g t ( s t , x t , ε s t, − x t, − |· ).Suppose χ [1¯ t ] = ( χ t | t = 1 , ..., ¯ t ) ∈ ( K ( S, X )) ¯ t again describes the policy adopted by all n players. Unlike in an NG, this time χ [1¯ t ] will help to generate a stochastic as opposed todeterministic environment trajectory. To describe each one-period transition in this complexprocess, we rely on the kernel χ nt ⊙ ˜ g nt ∈ K ( S n , S n ) defined by( χ nt ⊙ ˜ g nt )( s | S ′ ) = Z X n χ nt ( s | dx ) · ˜ g nt ( s, x | S ′ ) , ∀ s ∈ S n , S ′ ∈ B ( S n ) , (11)where χ nt is a member of K ( S n , X n ) that satisfies χ nt ( s | X ′ × · · · × X ′ n ) = Π nm =1 χ t ( s m | X ′ m ) , ∀ s ∈ S n , X ′ , ..., X ′ n ∈ B ( X ) , (12)and ˜ g nt is a member of K ( S n × X n , S n ) that satisfies˜ g nt ( s, x | S ′ × · · · × S ′ n ) = Π nl =1 ˜ g t ( s l , x l , ε s − l x − l | S ′ l ) , ∀ ( s, x ) ∈ S n × X n , S ′ , ..., S ′ n ∈ B ( S ) . (13)In combination, (11) can be spelled out as( χ nt ⊙ ˜ g nt )( s | S ′ × · · · × S ′ n ) = Z X n Π nm =1 χ t ( s m | dx m ) · Π nl =1 ˜ g t ( s l , x l , ε s − l x − l | S ′ l ) . (14)The above reflects that, each player m samples his action x m from the distribution χ t ( s m |· );once all players’ actions x = ( x , ..., x n ) have been determined, each player l will face hisunique in-action environment ε s − l x − l ; thus, this player’s period-( t + 1) state will be sampledfrom the distribution ˜ g t ( s l , x l , ε s − l x − l |· ).When the n players start period t with a random multi-state with distribution π nt ∈P ( S n ) and they act according to random rule χ t ∈ K ( S, X ) in the period, they will generatethe joint distribution µ nt ∈ P ( S n × X n ) of period- t multi-state and -action satisfying µ nt = π nt ⊗ χ nt . (15)According to (3) and (12), the above means that, for any S ′ ∈ B ( S n ) and X ′ , ..., X ′ n ∈ B ( X ), µ nt ( S ′ × X ′ ×· · ·× X ′ n ) = Z S ′ π nt ( ds ) · χ nt ( s | X ′ ×· · ·× X ′ n ) = Z S ′ π nt ( ds ) · Π nm =1 χ t ( s m | X ′ m ) . (16)9learly, (15) corresponds to (5) in the NG situation.By (11), the period-( t + 1) multi-state distribution µ nt ⊙ ˜ g nt ∈ P ( S n ) will follow( µ nt ⊙ ˜ g nt )( S ′ ) = Z S n × X n µ nt ( ds × dx ) · ˜ g nt ( s, x | S ′ ) , ∀ S ′ ∈ B ( S n ) . (17)Combining (15) and (17), we can see that the one-period transition between multi-states is π n,t +1 = ( π nt ⊗ χ nt ) ⊙ ˜ g nt = π nt ⊙ χ nt ⊙ ˜ g nt . (18)Note (18) is the n -player game’s answer to the NG’s (8). Similar to (9), for t ≤ t ′ , thedistribution π nt ′ of period- t ′ multi-state s t ′ is given by π nt ′ = π nt ⊙ Π t ′ − t ′′ = t ( χ nt ′′ ⊙ ˜ g nt ′′ ) . (19)When the initial multi-state s is randomly drawn from distribution π n , the entire trajectory π n, [1 , ¯ t +1] = ( π nt | t = 1 , , ..., ¯ t, ¯ t + 1) of the n -player game’s multi-state distributions can bewritten as π n, [1 , ¯ t +1] = ( π n ⊙ Π t − t ′ =1 ( χ nt ′ ⊙ ˜ g nt ′ ) | t = 1 , , .., ¯ t, ¯ t + 1) . (20)When all players’ states are sampled from some σ ∈ P ( S ), we still have (20) as the trajectoryfor multi-state distributions, but with π n = σ n . When recognizing π n = δ s , the Diracmeasure in P ( S n ) that assigns the full weight to s , (20) will help describe the evolution ofthe multi-state distribution for the n -player game Γ n ( s ), much like (10) did for Γ( σ ). Even before touching upon notions like cumulative payoffs and equilibria, we can alreadyintroduce an interesting link between finite games and NGs. It is in terms of an asymp-totic relationship between a sequence π n, [ t, ¯ t +1] = ( π nt ′ | t ′ = t, t + 1 , ..., ¯ t + 1) of multi-statedistributions in n -player games and a sequence σ [ t, ¯ t +1] = ( σ t ′ | t ′ = t, t + 1 , ..., ¯ t + 1) of statedistributions in their NG counterparts. The message is that, when starting from similarenvironments in period t and adopting the same action plan from that period on, stochasticenvironment paths experienced by large finite games will not drift too much away from theNG’s deterministic environment trajectory. We refrain from using the word convergencebecause the π nt ′ ’s reside in different spaces for different n ’s.First, we propose the concept asymptotic resemblance in order to precisely describethe way in which members in a sequence of probability measures increasingly resemble the10roducts of a given measure. For a separable metric space A , the space P ( A ) is metrized bythe Prohorov metric ρ A , which induces the weak topology on it. At fixed n ∈ N , the map ε ( · ) from A n to P n ( A ) ⊆ P ( A ) is continuous. Therefore, for any p ∈ P ( A ) and ǫ >
0, the set { a ∈ A n | ρ A ( ε a , p ) < ǫ } is an open subset of A n and thus a member of B ( A n ). Definition 1
For a separable metric space A , suppose p ∈ P ( A ) and for each n ∈ N , q n ∈ P ( A n ) . We say that sequence q n asymptotically resembles the sequence p n made up of p ’s n -th order products p × · · · × p , if for any ǫ > and n that is large enough, q n ( { a ∈ A n | ρ A ( ε a , p ) < ǫ } ) > − ǫ. Definition 1 says that sequence q n will asymptotically resemble the sequence p n of productmeasures when the empirical distribution ε a of a random vector a = ( a , ..., a n ), sampledfrom q n , is highly likely to be close to p as n approaches + ∞ . This resemblance notion isconsistent with Prohorov’s theorem (Parthasarathy [26], Theorem II.7.1), whose weak versionis presented as Lemma 2 in Appendix A. Due to it, any sequence ( p ′ ) n will asymptoticallyresemble the sequence p n if and only if p ′ = p .Some results related to the resemblance concept have been placed in Appendix A. Lemma 3stems from Dvoretzky, Kiefer, and Wolfolwitz’s [10] inequality and makes the convergencein Lemma 2 uniform in the chosen probability p . According to Lemma 4, the tampering ofone component within any n -long vector a ∈ A n would not much alter ε a . It is thereforenatural for Lemma 5 to state that the resemblance of q n to p n would lead to that of the A n − -marginal q n | A n − to p n − . Lemma 6 says that the above would also lead to the asymp-totic resemblance of p ′ × q n − to p n for any p ′ . So in general there can be nothing substantialregarding the relationship between the A -marginals q n | A and p . Finally, Lemma 7 showsthat asymptotic resemblance is preserved under the projection of A × B into A .The following one-step result states that asymptotic resemblance concerning pre-actionenvironments is translatable into that concerning in-action environments; also, the sameresemblance is preserved after undergoing one single step in a game. Proposition 1
Let state distribution σ ∈ P ( S ) , random state-dependent action plan χ ∈K ( S, X ) , and state-transition kernel g ∈ G ( S, X ) , with the latter enjoying the continuity of g ( s, x, τ ) in the joint state-action distribution τ at an ( s, x ) -independent rate. Also, multi-state distribution π n ∈ P ( S n ) for each n ∈ N . Suppose further that the sequence π n asymp-totically resembles the sequence σ n . Then, i) the sequence π n ⊗ χ n will asymptotically resemble the sequence ( σ ⊗ χ ) n , and(ii) the sequence π n ⊙ χ n ⊙ g n will asymptotically resemble the sequence ( σ ⊙ χ ⊙ g ( · , · , σ ⊗ χ )) n .Indeed, (ii) remains valid under mild contamination. That is, for any ( s, x ) ∈ S × X ,(iii) the sequence ( δ sx × ( π n − ⊗ χ n − )) ⊙ g n will asymptotically resemble the sequence ( σ ⊙ χ ⊙ g ( · , · , σ ⊗ χ )) n at a rate independent of the chosen ( s, x ) . Proposition 1 is one of our two most technical results. Its proof invokes both Prohorov’stheorem (Parthasarathy [26], Theorem II.7.1) on the convergence of empirical distributionsand for parts (ii) and (iii), Dvoretzky, Kiefer, and Wolfolwitz’s [10] inequality which providesthe uniformity of such convergence. In the proposition, part (i) stresses the passibility fromconvergence of pre-action environments to that of same-period in-action environments, see (5)and (15); part (ii) further points out that convergence in next-period pre-action environmentswill follow suit, see (8) and (18); also, part (iii) will be useful when we take the view pointfrom one single player..To take advantage of Proposition 1, we now assume the equi-continuity of the statetransitions with respect to in-action environments.
Assumption 1
Each transition kernel ˜ g t ( s, x, τ ) is continuous in τ at an ( s, x ) -independentrate. That is, for any in-action environment τ ∈ P ( S × X ) and ǫ > , there is δ > , suchthat for any τ ′ ∈ P ( S × X ) satisfying ρ S × X ( τ, τ ′ ) < δ and any ( s, x ) ∈ S × X , ρ S (˜ g t ( s, x, τ ) , ˜ g t ( s, x, τ ′ )) < ǫ. We are in a position to derive this section’s main result. It states that, when an NG and itsfinite counterparts evolve under the same action plan, environment pathways of large finitegames, though stochastic, will resemble the deterministic pathway of the NG.
Theorem 1
Let a policy profile χ [ t ¯ t ] ∈ ( K ( S, X )) ¯ t − t +1 for periods t, t + 1 , ..., ¯ t be given.When s t = ( s t , ..., s tn ) has a distribution π nt that asymptotically resembles σ nt , the series ( π nt ⊙ Π t ′ − t ′′ = t ( χ nt ′′ ⊙ ˜ g nt ′′ ) | t ′ = t, t +1 , ..., ¯ t, ¯ t +1) will asymptotically resemble (( T [ t,t ′ − ( χ [ t,t ′ − ) ◦ σ t ) n | t ′ = t, t + 1 , ..., ¯ t, ¯ t + 1) as well. That is, for any ǫ > and any n that is large enough, [ π nt ⊙ Π t ′ − t ′′ = t ( χ nt ′′ ⊙ ˜ g nt ′′ )]( ˜ A nt ′ ( ǫ )) > − ǫ, ∀ t ′ = t, t + 1 , ..., ¯ t + 1 , where for each t ′ , the set of multi-states ˜ A nt ′ ( ǫ ) ∈ B ( S n ) is such that, ρ S ( ε s t ′ , T [ t,t ′ − ( χ [ t,t ′ − ) ◦ σ t ) < ǫ, ∀ s t ′ ∈ ˜ A nt ′ ( ǫ ) . t with pre-action environment σ t and a slew of finite gamesstart the period with pre-action environments that are ever nearly sampled from σ t . Let theevolution of both types of games be guided by players acting according to the same policyprofile χ [ t ¯ t ] . Then, as the numbers of players n involved in finite games grow indefinitely,Theorem 1 predicts for ever less chances for the finite games’ period- t ′ environments ε s t ′ to beeven slightly away from the NG’s deterministic period- t ′ environment T [ t,t ′ − ( χ [ t,t ′ − ) ◦ σ t . Forsome fixed σ ∈ P ( S ), we can plug t = 1 and π n = σ n into Theorem 1. Then, we will obtainthe proximity between σ n [1 , ¯ t +1] = ( σ nt | t = 1 , , ..., ¯ t, ¯ t + 1) and π n, [1 , ¯ t +1] = ( π nt | t = 1 , , ..., ¯ t, ¯ t +1) for large n ’s, where every σ t = T [1 ,t − ( χ [1 ,t − ) ◦ σ and every π nt = σ n ⊙ Π t − t ′ =1 ( χ nt ′ ⊙ ˜ g nt ′ ).In view of (10) and (20), this means that when large games sample their initial states from anNG’s starting distribution σ , the former games’ state-distribution trajectories will remainclose to that of the latter game.Our confinement so far to discrete spaces S and X arises mainly from the need to dealwith non-product joint probabilities of the form p ⊗ κ ; see (3). In Yang [34], where randomstate transitions and random action plans were modeled through independently generatedshocks, only results pertaining to product-form probabilities p × q , where q is an ordinaryrather than conditional probability, were needed. Because of this, known properties likePropositions III.4.4 and III.4.6 of Ethier and Kurtz [11] could be put to good use. Resultsthere could thus be based on complete state and shock spaces. In contrast, if we were toconsider more general spaces here, we would face the presently unsurmountable challenge ofpassing the closeness between measures p and p i for i = 1 , , ..., n onto that between p n and Q ni =1 p i when n itself tends to infinity. We present this paper’s main result that an NG equilibrium, though oblivious of past historyand blind to other players’ states, will generate minimal regrets when adopted by players inlarge finite games. First, we introduce equilibrium concepts used in both types of games.
In defining the NG Γ( σ )’s equilibria, we subject a candidate policy profile to one-timedeviation of a single player, who is by default infinitesimal in influence. Note the deviationwill not alter the environment trajectory corresponding to the candidate profile. With this13nderstanding, we define v t ( s t , ξ [ t ¯ t ] , σ t , χ [ t ¯ t ] ) as the total expected payoff a player can receivefrom period t to ¯ t , when he starts with state s t ∈ S and adopts action plan ξ [ t ¯ t ] = ( ξ t , ..., ξ ¯ t ) ∈ ( K ( S, X )) ¯ t − t +1 throughout, while other players form initial pre-action environment σ t ∈ P ( S )and adopt policy profile χ [ t ¯ t ] = ( χ t , ..., χ ¯ t ) ∈ ( K ( S, X )) ¯ t − t +1 throughout. As a terminalcondition, we certainly have v ¯ t +1 ( s ¯ t +1 , σ ¯ t +1 ) = 0 . (21)For t = ¯ t, ¯ t − , ...,
1, we have the recursive relationship v t ( s t , ξ [ t ¯ t ] , σ t , χ [ t ¯ t ] ) = R X ξ t ( s t | dx t ) · [ ˜ f t ( s t , x t , σ t ⊗ χ t )+ R S ˜ g t ( s t , x t , σ t ⊗ χ t | ds t +1 ) · v t +1 ( s t +1 , ξ [ t +1 , ¯ t ] , T t ( χ t ) ◦ σ t , χ [ t +1 , ¯ t ] )] . (22)This is because the player’s action is guided in a random fashion by ξ t , its payoff is determinedby ˜ f t , its state evolution is governed by ˜ g t , and its future payoff is supplied by v t +1 ; also, afterundergoing the commonly adopted action plan χ t , the period-( t + 1) pre-action environment σ t +1 will be T t ( χ t ) ◦ σ t as shown in (8). The choice of ξ t affects the current player’s period- t action x t , his period-( t + 1) state s t +1 , and his future state-action trajectory. However, thechange at this negligible player does not alter the period- t in-action environment σ t ⊗ χ t aslisted in (5) or any environment in the future. This is the main reason why NGs are easierto handle than their finite-player counterparts.Now, we deem policy χ [1¯ t ] ∈ ( K ( S, X )) ¯ t a Markov equilibrium for the game Γ( σ ) when,for every t = 1 , , ..., ¯ t and ξ t ∈ K ( S, X ), v t ( s t , χ [ t ¯ t ] , σ t , χ [ t ¯ t ] ) ≥ v t ( s t , ( ξ t , χ [ t +1 , ¯ t ] ) , σ t , χ [ t ¯ t ] ) , ∀ s t ∈ S, (23)where σ t = T [1 ,t − ( χ [1 ,t − ) ◦ σ . (24)That is, policy χ [1¯ t ] will be regarded an equilibrium when no player can be better off byunilaterally deviating to any alternative plan ξ t ∈ K ( S, X ) in any single period t . Thedefinition of σ t in (24) underscores the evolution of the deterministic environment trajectoryfollowing the adoption of action plan χ [1¯ t ] by almost all players. ǫ -Equilibria in n -player Games For an n -player game, let v nt ( s t , ξ [ t ¯ t ] , ε s t, − , χ [ t ¯ t ] ) be the total expected payoff player 1 canreceive from period t to ¯ t , when he starts with state s t ∈ S and adopts action plan14 [ t ¯ t ] ∈ ( K ( S, X )) ¯ t − t +1 throughout, while other players form initial empirical state distribution ε s t, − = ε ( s t ,...,s tn ) ∈ P n − ( S ) and adopt action plan χ [ t ¯ t ] ∈ ( K ( S, X )) ¯ t − t +1 throughout. As aterminal condition, we have v n, ¯ t +1 ( s ¯ t +1 , , ε s ¯ t +1 , − ) = 0 . (25)For t = ¯ t, ¯ t − , ...,
1, we have the recursive relationship v nt ( s t , ξ [ t ¯ t ] , ε s t, − , χ [ t ¯ t ] ) = R X ξ t ( s t | dx t ) · R X n − χ n − t ( s t, − | dx t, − ) ×× [ ˜ f t ( s t , x t , ε s t, − x t, − ) + R S n ˜ g nt ( s t , x t | ds t +1 ) · v n,t +1 ( s t +1 , , ξ [ t +1 , ¯ t ] , ε s t +1 , − , χ [ t +1 , ¯ t ] )] , (26)where the meaning of χ n − t ( s t, − | dx t, − ) follows from (12) and that of ˜ g nt ( s t , x t | ds t +1 ) followsfrom (13). Note (26) differs substantially from its NG counterpart (22). With only a finitenumber of players, player 1’s one-time choice ξ t not only affects his own future actions andstates as before, but differently, starting from the altered in-action environment ε s t x t , it alsoimpacts the entire future trajectory of all other players. Note ε s t x t impacts the generationof s t +1 = ( s t +1 , , ..., s t +1 ,n ) in its projections to n different ( n − R S n ˜ g nt ( s t , x t | ds t +1 ) amounts to Π nm =1 R S ˜ g t ( s tm , x tm , ε s t, − m x t, − m | ds t +1 ,m ).For each n ∈ N \ { } , let ˆ π n − , [1¯ t ] = (ˆ π n − ,t | t = 1 , ..., ¯ t ) ∈ ( P ( S n − )) ¯ t be a series of other-player multi-state distributions. For ǫ ≥
0, we deem χ [1¯ t ] = ( χ t | t = 1 , ..., ¯ t ) ∈ ( K ( S, X )) ¯ t an ǫ -Markov equilibrium for the game family (Γ n ( s ) | s ∈ S n ) in the sense of ˆ π n − , [1¯ t ] when,for every t = 1 , ..., ¯ t , ξ [ t ¯ t ] ∈ ( K ( S, X )) ¯ t − t +1 , and s t ∈ S , R S n − ˆ π n − ,t ( ds t, − ) · v nt ( s t , χ [ t ¯ t ] , ε s t, − , χ [ t ¯ t ] ) ≥ R S n − ˆ π n − ,t ( ds t, − ) · v nt ( s t , ξ [ t ¯ t ] , ε s t, − , χ [ t ¯ t ] ) − ǫ. (27)That is, action plan χ [1¯ t ] will be an ǫ -Markov equilibrium in the sense of ˆ π n − , [1¯ t ] when underthe plan’s guidance, the average payoff from any period t and player-1 state s t on cannot beimproved by more than ǫ through any unilateral deviation, where the “average” is based onother players’ multi-state s t, − being sampled from the distribution ˆ π n − ,t . Note (27) differsfrom (23) also in that its unilateral deviation need not be one-time. Before moving on, we need the single-period payoff functions ˜ f t to be continuous. Assumption 2
Each payoff function ˜ f t ( s, x, τ ) is continuous in the in-action environment τ at an ( s, x ) -independent rate. That is, for any τ ∈ P ( S × X ) and ǫ > , there is δ > , uch that for any τ ′ ∈ P ( S × X ) satisfying ρ S × X ( τ, τ ′ ) < δ and any ( s, x ) ∈ S × X , | ˜ f t ( s, x, τ ) − ˜ f t ( s, x, τ ′ ) | < ǫ. Now we show the convergence of finite-game value functions to their NG counterpart, theproof of which is quite technical as well, and calls upon parts (i) and (iii) of Proposition 1.
Proposition 2
For any t = 1 , , ..., ¯ t + 1 , let σ t ∈ P ( S ) and ˆ π n − ,t ∈ P ( S n − ) for each n ∈ N . Suppose the sequence ˆ π n − ,t asymptotically resembles the sequence σ n − t . Then for any χ [ t ¯ t ] ∈ ( K ( S, X )) ¯ t − t +1 , the sequence R S n − ˆ π n − ,t ( ds t, − ) · v nt ( s t , ξ [ t ¯ t ] , ε s t, − , χ [ t ¯ t ] ) will convergeto v t ( s t , ξ [ t ¯ t ] , σ t , χ [ t ¯ t ] ) at a rate that is independent of both s t ∈ S and ξ [ t ¯ t ] ∈ ( K ( S, X )) ¯ t − t +1 . Combining (23) and (27), as well as Proposition 2, we can come to the main result.
Theorem 2
For some σ ∈ P ( S ) , suppose χ [1¯ t ] = ( χ t | t = 1 , , ..., ¯ t ) ∈ ( K ( S, X )) ¯ t is aMarkov equilibrium of NG Γ( σ ) . Also, suppose ˆ π n − , [1¯ t ] = (ˆ π n − ,t | t = 1 , , ..., ¯ t ) ∈ ( P ( S n − )) ¯ t is such that the sequence ˆ π n − ,t asymptotically resembles the sequence σ n − t for each t , where σ t = T [1 ,t − ( χ [1 ,t − ) ◦ σ . Then, for ǫ > and large enough n ∈ N , the given χ [1¯ t ] is also an ǫ -Markov equilibrium for the game family (Γ n ( s ) | s ∈ S n ) in the sense of ˆ π n − , [1¯ t ] . The theorem says that players in a large finite game can agree on an NG equilibrium andexpect to lose little on average, as long as the other-player multi-state distribution ˆ π n − ,t onwhich “average” is based is similar to the product form σ n − t , where σ t = T [1 ,t − ( χ [1 ,t − ) ◦ σ is the corresponding NG’s predictable equilibrium state distribution for the same period. Asto whether reasonable ˆ π n − , [1¯ t ] = (ˆ π n − ,t | t = 1 , , ..., ¯ t ) exists to satisfy this condition, theanswer is affirmative. The next section is dedicated to this point. We now present examples where the key condition in Theorem 2 can be true. In all of them,we let the initial other-player multi-state distribution ˆ π n − , = σ n − = σ n | S n − . That is, welet players’ initial states in n -player games be randomly drawn from the NG’s initial statedistribution σ . Now we discuss what can happen in periods t = 2 , , ..., ¯ t .16 .1 Two Possibilities First, we can let each ˆ π n − ,t = σ n − t . It has been discussed right after Definition 1 thatthe sequence σ n − t asymptotically resembles itself. So this choice satisfies the condition inTheorem 2. This would correspond to the case where players in large finite games take the“lazy” approach of using independent draws on the NG state distribution to assess theiropponents’ states. Note this is reasonable due to the common initial condition for bothtypes of games and Theorem 1.Second, we can let each ˆ π n − ,t = π nt | S n − , where π nt = σ n ⊙ Π t − t ′ =1 ( χ nt ′ ⊙ ˜ g nt ′ ) . (28)According to (19), π nt stands for players’ multi-state distribution in period t in an n -playergame when their initial states are randomly drawn from the distribution σ and then fromperiod 1 onward players all follow through with the NG equilibrium χ [1¯ t ] . Since the sequence σ n asymptotically resembles itself, Theorem 1 will ascertain the asymptotic resemblance of π nt to σ nt . Then, Lemma 5 in Appendix A will lead to the asymptotic resemblance of ˆ π n − ,t to σ n − t . So this choice would satisfy Theorem 2’s condition as well. Also, its meaning isclear—here players in large finite games use precise assessments on what other players’ statesmight be had they followed the NG equilibrium all along. Note that ˆ π n − ,t has not countenanced the possibility in which a player involves his own state s t in the estimation of the other-player multi-state s t, − . We now show that this is possibleat least when the state space S is finite. In that case, we can upgrade the ˆ π n − ,t ∈ P ( S n − ) inProposition 2 to ˆ π n − ,t ( · ) = (ˆ π n − ,t ( s t |· ) | s t ∈ S ) ∈ ( P ( S n − )) S and obtain the convergenceof R S n − ˆ π n − ,t ( s t | ds t, − ) · v nt ( s t , ξ [ t ¯ t ] , ε s t, − , χ [ t ¯ t ] ) to v t ( s t , ξ [ t ¯ t ] , σ t , χ [ t ¯ t ] ) at an s t -independentrate. This will lead us to the following extended version of Theorem 2. Theorem 3
Suppose σ [1 , ¯ t +1] and χ [1¯ t ] are all the same as in Theorem 2. Also, suppose ˆ π n − , [1¯ t ] ( · ) = (ˆ π n − ,t ( s t |· ) | t = 1 , , ..., ¯ t, s t ∈ S ) ∈ (( P ( S n − )) S ) ¯ t is such that the sequence ˆ π n − ,t ( s t |· ) asymptotically resembles the sequence σ n − t for each t and s t . Then, for ǫ > and large enough n ∈ N , for every t = 1 , ..., ¯ t , ξ [ t ¯ t ] ∈ ( K ( S, X )) ¯ t − t +1 , and s t ∈ S , R S n − ˆ π n − ,t ( s t | ds t, − ) · v nt ( s t , χ [ t ¯ t ] , ε s t, − , χ [ t ¯ t ] ) ≥ R S n − ˆ π n − ,t ( s t | ds t, − ) · v nt ( s t , ξ [ t ¯ t ] , ε s t, − , χ [ t ¯ t ] ) − ǫ. π n − , [1¯ t ] ( · ) be the same as inthe aforementioned two examples, in which the newly added s t -dependence is mute. But athird choice would allow each player a full-fledged Bayesian update on other players’ states.In this third choice, we still use (28) to define π nt . Then, as long as σ t ( s t ) >
0, we letˆ π n − ,t ( s t |· ) = π nt,S | S n − ( s t |· ) , (29)the other-player multi-state distribution derivable from π nt when conditioned on the currentplayer’s state s t ; otherwise, we simply let ˆ π n − ,t = π nt | S n − just as in the second example.Note the marginal π nt | S is defined by π nt | S ( { s t } ) = π nt ( { s t } × S n − ) , ∀ s t ∈ S, (30)and each conditional distribution π nt,S | S n − ( s t |· ) is defined by π nt,S | S n − ( s t | S ′ ) = π nt ( { s t } × S ′ ) π nt | S ( { s t } ) = π nt ( { s t } × S ′ ) π nt ( { s t } × S n − ) , ∀ S ′ ∈ B ( S n − ) , (31)when the denominator is strictly positive and an arbitrary value otherwise. The lone fact that π nt asymptotically resembles σ nt is actually quite far from being able todictate the asymptotic resemblance of the thus defined ˆ π n − ,t ( s t |· ) to σ n − t . Note that for ageneral q n resembling some p n , Lemma 6 in Appendix A has all but ruled out the convergenceof π n | A to p , let alone the asymptotic resemblance of q n,A | A n − to p n − . Fortunately, π nt stillenjoys the additional feature of being symmetric.For any n ∈ N , let Ψ n be the set of all n -dimensional permutations. That is, each ψ ∈ Ψ n makes ( ψ (1) , ..., ψ ( n )) a permutation of (1 , ..., n ). For a given ψ ∈ Ψ n , let ussuppose ψa = ( a ψ (1) , ..., a ψ ( n ) ) for any a = ( a , ..., a n ) ∈ A n , and then ψA ′ = { ψa | a ∈ A ′ } forany A ′ ⊆ A n . Note that, due to its innately symmetric definition, B ( A n ) is automaticallysymmetric in the sense that B ( A n ) = { ψA ′ | A ′ ∈ B ( A n ) } for any ψ ∈ Ψ n . Definition 2
For n ∈ N and separable metric space A , we say q n ∈ P ( A n ) symmetric if q n ( A ′ ) = q n ( ψA ′ ) , ∀ ψ ∈ Ψ n , A ′ ∈ B ( A n ) . We have the much needed result that asymptotic resemblance of q n to p n does lead tothe convergence of q n | A to p when q n is symmetric. This is in stark contrast with Lemma 6.18 roposition 3 Let A be a discrete metric space and q n ∈ P ( A n ) for every n ∈ N be symmet-ric. Suppose the sequence q n asymptotically resembles the sequence p n . Then, the sequence q n | A will converge to p , namely, lim n → + ∞ q n | A ( { a } ) = p ( { a } ) for every a ∈ A . This then results in the resemblance of q n,A | A n − to p n − . Proposition 4
Let A be a discrete metric space and q n ∈ P ( A n ) for every n ∈ N be symmet-ric. Suppose the sequence q n asymptotically resembles the sequence p n . Then, the sequence q n,A | A n − ( a |· ) will asymptotically resemble the sequence p n − for any a ∈ A with p ( { a } ) > . Note that π n , being equal to σ n , is symmetric. As suggested by (28), the operation ithas to go through to arrive to π nt is also symmetric. Hence, π nt is symmetric. Therefore,by Proposition 3, the marginal probability π nt | S as defined in (30) would converge to theNG state distribution σ t ; thus, the conditional distribution π nt,S | S n − ( s t |· ) as defined in (31)would be well defined when σ t ( s t ) >
0. Then, Proposition 4 can guarantee that ˆ π n − ,t ( s t |· )as defined in (29) would asymptotically resemble σ n − t and hence help to facilitate thecondition needed for Theorem 3. The above suggests that, even when players exercise themost accurate Bayesian updates on other players’ states using their own state information,they will not discern much regret on average by adhering to the NG equilibrium. Now we study an infinite-horizon model with stationary features. To this end, we keep S and X , but let there be a discount factor ¯ α ∈ [0 , f which meetsthe basic measurability and boundedness requirements, so that ˜ f t = ¯ α t − · ˜ f for t = 1 , , .... .Let us use ¯ f for the bound ¯ f that appeared in (1). In addition, there is a state transitionkernel ˜ g ∈ G ( S, X ), so that ˜ g t = ˜ g for t = 1 , , ... . For χ ∈ K ( S, X ), denote by T ( χ ) theoperator on P ( S ), so that for any σ ∈ P ( S ), T ( χ ) ◦ σ = σ ⊙ χ ⊙ ˜ g ( · , · , σ ⊗ χ ) . (32)Thus, state transition has been made stationary by the stationarity of ˜ g .Denote the stationary nonatomic game formed from the above S , X , ¯ α , ˜ f , and ˜ g byΓ ∞ . It helps to first study the corresponding games Γ t that terminate in periods t + 1, for t = 0 , , ... . Now let v t ( s, ξ [1 t ] , σ, χ [1 t ] ) be the total expected payoff a player can receive in19ame Γ t , when he starts at state s ∈ S in period 1 and adopts action plan ξ [1 t ] ∈ ( K ( S, X )) t from period 1 to t , while all other players form state distribution σ ∈ P ( S ) in the beginningand act according to χ [1 t ] ∈ ( K ( S, X )) t from period 1 to t . As a terminal condition, we have v ( s, σ ) = 0. Also, for t = 1 , , ... , v t ( s, ξ [1 t ] , σ, χ [1 t ] ) = R X ξ ( s | dx ) · [ ˜ f ( s, x, σ ⊗ χ )+ ¯ α · R S ˜ g ( s, x, σ ⊗ χ | ds ′ ) · v t − ( s ′ , ξ [2 t ] , T ( χ ) ◦ σ, χ [2 t ] )] . (33)Using the terminal condition and (33), we can inductively show that | v t +1 ( s, ξ [1 ,t +1] , σ, χ [1 ,t +1] ) − v t ( s, ξ [1 t ] , σ, χ [1 t ] ) |≤ ¯ α t · ¯ f . (34)Given s ∈ S , ξ [1 ∞ ] = ( ξ , ξ , ... ) ∈ ( K ( S, X )) ∞ , σ ∈ P ( S ), and χ [1 ∞ ] = ( χ , χ , ... ) ∈ ( K ( S, X )) ∞ , the sequence { v t ( s, ξ [1 t ] , σ, χ [1 t ] ) | t = 0 , , ... } is thus Cauchy and has a limitpoint v ∞ ( s, ξ [1 ∞ ] , σ, χ [1 ∞ ] ). The latter is the total discounted expected payoff a player canobtain in the game Γ ∞ , when he starts at state s and adopts action plan ξ [1 ∞ ] , while allother players form initial pre-action environment σ and act according to χ [1 ∞ ] .A pre-action environment σ ∈ P ( S ) is said to be associated with χ ∈ K ( S, X ) when σ = T ( χ ) ◦ σ. (35)That is, we let environment σ be associated with action plan χ when the former is invariantunder the one-period transition when all players adhere to the latter. For χ ∈ K ( S, X ), weuse χ ∞ to represent the stationary policy profile ( χ, χ, ... ) ∈ ( K ( S, X )) ∞ that players are toadopt in all periods t = 1 , , ... .We deem one-time action plan χ ∈ K ( S, X ) a stationary Markov equilibrium for thenonatomic game Γ ∞ , when there exists a σ ∈ P ( S ) that is associated with the given χ , sothat for every one-time unilateral deviation ξ ∈ K ( S, X ), v ∞ ( s, χ ∞ , σ, χ ∞ ) ≥ v ∞ ( s, ( ξ, χ ∞ ) , σ, χ ∞ ) , ∀ s ∈ S. (36)Therefore, a policy will be considered an equilibrium when it induces an invariant environ-ment under whose sway the policy turns out to be a best response in the long run.Now we move on to the n -player game Γ ∞ n made out of the same S , X , ¯ α , ˜ f , and ˜ g .Similarly to the above, we let Γ tn be its n -player counterpart that terminates in period t + 1.Now let v tn ( s , ξ [1 t ] , ε s − , χ [1 t ] ) be the total expected payoff player 1 can receive in game Γ nt ,when he starts with state s ∈ S and adopts action plan ξ [1 t ] ∈ ( K ( S, X )) t from period 1 to t ,20hile other players form initial empirical distribution ε s − = ε ( s ,...,s n ) ∈ P n − ( S ) and adoptpolicy χ [1 t ] ∈ ( K ( S, X )) t from 1 to t . As a terminal condition, we have v n ( s , ε s − ) = 0. For t = 1 , , ... , it follows that v tn ( s , ξ [1 t ] , ε s − , χ [1 t ] ) = R X ξ ( s | dx ) · R X n − χ n − ( s − | dx − ) · [ ˜ f ( s , x , ε s − x − )+ ¯ α · R S n ˜ g n ( s, x | ds ′ ) · v t − n ( s ′ , ξ [2 t ] , ε s ′− , χ [2 t ] )] . (37)Using the terminal condition and (37), we can inductively show that | v t +1 n ( s , ξ [1 ,t +1] , ε s − , χ [1 ,t +1] ) − v tn ( s , ξ [1 t ] , ε s − , χ [1 t ] ) |≤ ¯ α t · ¯ f . (38)Given s ∈ S , ξ [1 ∞ ] ∈ ( K ( S, X )) ∞ , ε s − ∈ P n − ( S ), and χ [1 ∞ ] ∈ ( K ( S, X )) ∞ , the sequence { v nt ( s , ξ [1 t ] , ε s − , χ [1 t ] ) | t = 0 , , ... } is Cauchy and has a limit point v ∞ n ( s , ξ [1 ∞ ] , ε s − , χ [1 ∞ ] ).The latter is the total discounted expected payoff a player can obtain in Γ ∞ n , when he startsat state s and adopts action plan ξ [1 ∞ ] , while all other players form the initial pre-actionenvironment ε s − and act according to χ [1 ∞ ] .For the current setting, it should be noted that Assumptions 1 and 2 translate into thecontinuity in τ at an ( s, x )-independent rate of, respectively, the transition kernel ˜ g ( s, x, τ )and payoff function ˜ f ( s, x, τ ). We now present the main result for the stationary case. Theorem 4
Suppose χ ∈ K ( S, X ) is a stationary Markov equilibrium for the stationarynonatomic game Γ ∞ . Let ˆ π n − ∈ P ( S n − ) for each n ∈ N \ { } . Also suppose the sequence ˆ π n − asymptotically resembles the sequence σ n − , where σ is associated with χ in the equi-librium definitions (35) and (36). Then, χ ∞ would be asymptotically equilibrium for games Γ ∞ n in an average sense. More specifically, for any ǫ > and large enough n ∈ N , Z S n − ˆ π n − ( ds − ) · v ∞ n ( s , χ ∞ , ε s − , χ ∞ ) ≥ Z S n − ˆ π n − ( ds − ) · v ∞ n ( s , ξ [1 ∞ ] , ε s − , χ ∞ ) − ǫ, for any s ∈ S and ξ [1 ∞ ] ∈ ( K ( S, X )) ∞ . Theorem 4 says that, players in a large finite stationary game will not regret much byadopting a stationary equilibrium for a correspondent stationary nonatomic game. Theregret can be measured in an average sense, so long as the underlying other-player multi-state distribution ˆ π n − is close to an invariant σ associated with the NG equilibrium. Justas in Section 7, we can let ˆ π n − = σ n − , indicating that players take a “lazy” approach inassessing other players’ states. We leave discussion of other possibilities to Appendx E.21 Implications of Main Results
Regarding Theorems 2 and 3, we note the following for ¯ t -period games. A prominent featureof an NG equilibrium χ [1¯ t ] ∈ ( K ( S, X )) ¯ t is its insensitivity, at any period t , to a player’spersonal history ( s t ′ , x t ′ | t ′ = 1 , , ..., t − f t nor ˜ g t depends on past history.But the more interesting independence of the latter two factors stems from players’ commonknowledge about the evolution of their environments. The ( σ t ′ ⊗ χ t ′ | t ′ = 1 , , ..., t −
1) portionof the history and the present information σ t , both about other players, are determinableby (10) before the game is even played out.For finite semi-anonymous games, however, information is gradually revealed and itsperfection is not guaranteed. We can define space O S and map ˜ o S : P ( S ) → O S to representa player’s observatory power over his present pre-action environment immediately beforeactual play. Similarly, we can define space O SX and map ˜ o SX : P ( S × X ) → O SX torepresent his observatory power over the in-action environment just experienced. So thatnew information does not contradict old information and no information gets lost, we supposefunction ˜ o SXS : O SX → O S exists, with ˜ o SXS (˜ o SX ( τ )) = ˜ o S ( τ | S ) for any τ ∈ P ( S × X ).With these definitions, a player’s decision in period t can be denoted by a map ˆ χ t :( S × X × O SX ) t − × O S × S → P ( X ). In the period, player 1’s random decision rule can bewritten as ˆ χ t (˜ h t , ˜ o S ( ε s t, − ) , s t |· ), where the history ˜ h t is expressible as˜ h t = ( s t ′ , x t ′ , ˜ o SX ( ε s t ′ , − x t ′ , − ) | t ′ = 1 , , ..., t − , (39)˜ o S ( ε s t, − ) is his observation of other players’ status, and s t represents the player’s own state.There is a whole spectrum in which O S and ˜ o S can reside. When O S = { } and ˜ o S ( · ) = 0,players are ignorant of others’ states; when O S = P ( S ) and ˜ o S is the identity map, everyplayer is fully aware of his surrounding. Similarly, there are varieties of O SX , ˜ o SX , and ˜ o SXS .Theorems 2 and 3, however, nullify the need to delve into the ( O S , ˜ o S , O SX , ˜ o SX , ˜ o SXS )-related details about finite games. They state that an equilibrium of the NG counterpart,which is necessarily both oblivious of the past history ˜ h t and blind to the present obser-vation ˜ o S ( ε s t, − ), serves as a good approximate equilibrium for games with enough players.The absence of ˜ h t again has a Markovian explanation. On the other hand, the ability to22hake off ˜ o S ( ε s t, − )’s influence is very important, since this saves players the efforts to gatherinformation about their surroundings.Regarding Theorem 4, we note the following. Each of our finite stationary games is adiscounted stochastic game. For an n -player version of the latter game in which players havefull knowledge of others’ states, equilibria are hard to compute and for their implementation,require high degrees of coordination among players; see Solan [30]. These equilibria comefrom the space (2 R n ) S n × (( R n ) X n × S n ) S n × R n ; whereas, our NG equilibria come from R S × X .Meanwhile, the discounted stochastic game one faces in real life is often semi-anonymous;see, e.g., examples listed in Jovanoic and Rosenthal [16]. For such a game, Theorem 4 hasshown that a much easier path can be taken in order to coordinate player behavior underan ǫ -sized compromise. If players all agree to exercise a corresponding NG equilibrium, thetypical player 1 has only to respond to his own state s t without giving up too much. To further buttress the claim that studying the idealistic NGs can help with the under-standing and execution of messier finite games faced in real life, we demonstrate that NGequilibria, meeting criteria (23) and (24) for the transient case and (35) and (36) for thestationary case, can be obtained relatively easily.First, we concentrate on the transient case studied in Sections 3 to 6. From (22), v t ( s t , ( ξ t , χ [ t +1 , ¯ t ] ) , σ t , χ [ t ¯ t ] ) = Z X ξ t ( dy ) · v t ( s t , ( δ y , χ [ t +1 , ¯ t ] ) , σ t , χ [ t ¯ t ] ) . (40)Hence, sup ξ t ∈K ( S,X ) v t ( s t , ( ξ t , χ [ t +1 , ¯ t ] ) , σ t , χ [ t ¯ t ] ) = sup y ∈ X v t ( s t , ( δ y , χ [ t +1 , ¯ t ] ) , σ t , χ [ t ¯ t ] ) . (41)So the equilibrium criterion (23) conveniently used by us for the ¯ t -period case is equivalentto, for every t = 1 , , ..., ¯ t , χ t ( s t | ˜ X t ( s t , σ t , χ [ t ¯ t ] )) = 1 , ∀ s t ∈ S, (42)where˜ X t ( s t , σ t , χ [ t ¯ t ] ) = { x ∈ X | v t ( s t , ( δ x , χ [ t +1 , ¯ t ] ) , σ t , χ [ t ¯ t ] ) = sup y ∈ X v t ( s t , ( δ y , χ [ t +1 , ¯ t ] ) , σ t , χ [ t ¯ t ] ) } , (43)and σ t is defined through (24). 23he form consisting of (42) and (43) is fairly close to the distributional-equilibrium con-cept used in NG literature, such as Mas-Colell [24] and Jovanovic and Rosenthal [16]. Adistributional equilibrium is an in-action environment sequence τ [1¯ t ] = ( τ t | t = 1 , , ..., ¯ t ) ∈ ( P ( S × X )) ¯ t which satisfies τ t ( ˜ U t ( τ [ t ¯ t ] )) = 1 for each t = 1 , , ..., ¯ t . Here, ˜ U t ( τ [ t ¯ t ] ) = { ( s, x ) ∈ S × X | v ′ t ( s, x, τ [ t ¯ t ] ) = sup y ∈ X v ′ t ( s, y, τ [ t ¯ t ] ) } , and v ′ t ( s, y, τ [ t ¯ t ] ) is a player’s payoff when he startsperiod t with state s and action y , but other players in all periods and he himself in laterperiods act according to τ [ t ¯ t ] ; corresponding to (24), the distributional equilibrium also sat-isfies τ | S = σ and τ t | S = τ t − ⊙ ˜ g t − ( · , · , τ t − ) for t = 2 , , ..., ¯ t . According to Jovanovicand Rosenthal [16] (Theorem 1), such an equilibrium τ [1¯ t ] would exist when S and X arecompact, each payoff ˜ f t is bounded and continuous in all arguments, and each transitionkernel ˜ g t is continuous in all arguments.When an equilibrium χ [1¯ t ] in our conditional sense exists, we can construct a distributionalequilibrium τ [1¯ t ] by resorting iteratively to τ t = σ t ⊗ χ t and σ t +1 = T t ( χ t ) ◦ σ t for t = 1 , , ..., ¯ t .Conversely, when the latter distributional equilibrium τ [1¯ t ] is available, we can nearly get aconditional equilibrium χ [1¯ t ] back. For each t = 1 , , ..., ¯ t , according to Duffie, Geanakoplos,Mas-Colell, and McLennan [9] (p. 751), we can identify a χ t ∈ K ( S, X ), which also passesas a measurable map from S to P ( X ), that satisfies τ t = τ t | S ⊗ χ t . Thus, we will be able toconstruct χ [1¯ t ] consecutively from χ up to χ ¯ t . But even then, χ [ t ¯ t ] along with σ t = τ t | S wouldsatisfy (42) only for τ t | S -almost every s t , but not necessarily every s t ∈ S . For instance, wecan suppose S = { ¯ s , ¯ s , ... } . At each t , the constructed χ [1¯ t ] could guarantee (42) for those¯ s i ’s with ( τ t | S )(¯ s i ) > τ t | S )(¯ s i ) = 0. On the other hand, a conditionalequilibrium χ [1¯ t ] can be obtained directly; see Appendix F.1 for details.When it comes to the stationary case examined in Section 8, we make parallel develop-ments. Here the property corresponding to (36) is χ ( s | ˜ X ∞ ( s, σ, χ )) = 1 , ∀ s ∈ S, (44)where ˜ X ∞ ( s, σ, χ ) = { x ∈ X | v ∞ ( s, ( δ x , χ ∞ ) , σ, χ ∞ ) = sup y ∈ X v ∞ ( s, ( δ y , χ ∞ ) , σ, χ ∞ ) } , (45)and σ satisfies (35). Again, the existence of a related distributional equilibrium τ ∈ P ( S × X )is known under quite general conditions; see, e.g., Jovanovic and Rosenthal [16] (Theorem2). However, an equilibrium τ does not exactly lead to a conditional equilibrium χ . So oncemore we focus on a direct approach for the stationary case; see Appendix F.2.24 Under a common action plan, we have shown that environments faced by players in multi-period large finite games would stay close to those of their NG counterparts. For transientand stationary settings, our results reveal that an NG equilibrium, necessarily both obliviousof past history and blind to present status of other players, could serve as a good approximateequilibrium in large finite games. We reckon that the discreteness requirement on both thestate and action spaces can be frustrating in some circumstances. Besides the relaxation ofthe aforementioned restriction, future research can also look into the issue of converge rate.
Acknowledgments
This research was supported in part by National Science Foundation Grant CMMI-0854803, as well as National Natural Science Foundation of China Grants 11371273 and71502015.
AppendicesA Concepts and Rudimentary Lemmas
Recall that ρ A stands for the Prohorov metric for the space of distributions P ( A ). Lemma 1
Let A be a separable metric space. Then, for any n ∈ N and a, a ′ ∈ A n , ρ A ( ε a , ε a ′ ) ≤ n max m =1 d A ( a m , a ′ m ) . Proof:
Let ǫ = max nm =1 d A ( a m , a ′ m ). For any A ′ ∈ B ( A ), the key observation is that δ a ′ m (( A ′ ) ǫ ) ≥ δ a m ( A ′ ) . (A.1)Then, ε a ′ (( A ′ ) ǫ ) = P nm =1 δ a ′ m (( A ′ ) ǫ ) n ≥ P nm =1 δ a m ( A ′ ) n = ε a ( A ′ ) . (A.2)Thus, ρ A ( ε a , ε a ′ ) ≤ ǫ . 25ccording to Parthasarathy [26] (Theorem II.7.1), the strong law of large numbers appliesto the empirical distribution under the weak topology, and hence under the Prohorov metric.In the following, we state its weak version. Lemma 2
Let separable metric space A and distribution p ∈ P ( A ) be given. Then, for any ǫ > , as long as n is large enough, p n ( { a ∈ A n | ρ A ( ε a , p ) < ǫ } ) > − ǫ. Due to the inequality of Dvoretzky, Kiefer, and Wolfolwitz [10], the above convergenceis uniform for certain A ’s. The inequality infers that, when A is R or countable, p n ( { a ∈ A n | ρ A ( ε a , p ) ≤ ǫ } ) > − e − nǫ , ∀ ǫ > . When n is greater than ln(3 /ǫ ) / (2 ǫ ), a number independent of p ∈ P ( A ), the above wouldentail the inequality in Lemma 2. Thus, we have the following. Lemma 3
When A is the real line R or countable, the convergence expressed in Lemma 2is uniform. Namely, a lower bound could be identified so that every n above it would realizethe inequality in the lemma for every p ∈ P ( A ) . For separable metric space A , point a ∈ A , and the ( n − p ∈ P n − ( A ), we use ( a, p ) n to represent the member of P n ( A ) that has an additional 1 /n weight on the point a , but with probability masses in p being reduced to ( n − /n times oftheir original values. For a ∈ A n and m = 1 , ..., n , we have ( a m , ε a − m ) n = ε a . Concerningthe Prohorov metric, we have also a simple but useful observation. Lemma 4
Let A be a separable metric space. Then, for any n ∈ N \ { } , a ∈ A , and p ∈ P n − ( A ) , ρ A (( a, p ) n , p ) ≤ n . Proof:
Let A ′ ∈ B ( A ) be chosen. Then p ( A ′ ) = ( m − / ( n −
1) for some m = 1 , , ..., n . If a / ∈ A ′ , then ( a, p ) n ( A ′ ) = ( m − /n and hence( a, p ) n ( A ′ ) ≤ p ( A ′ ) ≤ ( a, p ) n ( A ′ ) + 1 n . (A.3)If a ∈ A ′ , then ( a, p ) n ( A ′ ) = m/n and hence( a, p ) n ( A ′ ) − n ≤ p ( A ′ ) ≤ ( a, p ) n ( A ′ ) . (A.4)26herefore, it is always true that | ( a, p ) n ( A ′ ) − p ( A ′ ) |≤ n . (A.5)Due to the nature of the Prohorov metric, we have ρ A (( a, p ) n , p ) ≤ n . (A.6)We have thus completed the proof.For the notion of asymptotic resemblance introduced in Definition 1, we have that it ispreserved under certain projections and expansions. Lemma 5
Let A be a separable metric space. Also, q n ∈ P ( A n ) for every n ∈ N and p ∈ P ( A ) . Suppose the sequence q n asymptotically resembles the sequence p n . Then, thesequence q n | A n − will asymptotically resemble the sequence p n − . Proof:
For any ǫ >
0, due to the asymptotic resemblance of the sequence q n to the sequence p n , we have, for n large enough, q n ( A ′ n ) > − ǫ, (A.7)where A ′ n = { a ∈ A n | ρ A ( ε a , p ) < ǫ } . (A.8)By Lemma 4, we have ρ A ( ε a , ε a − ) ≤ n , ∀ a ∈ A n . (A.9)Hence, for large enough n , A ′ n ⊆ A × A ′′ n − , (A.10)where A ′′ n − = { a − ∈ A n − | ρ A ( ε a − , p ) < ǫ } . (A.11)But by (A.7), this means that( q n | A n − )( A ′′ n − ) = q n ( A × A ′′ n − ) ≥ q n ( A ′ n ) > − ǫ. (A.12)That is, q n | A n − asymptotically resembles p n − .27 emma 6 Let A be a separable metric space. Also, q n ∈ P ( A n ) for every n ∈ N and p, p ′ ∈ P ( A ) . Suppose the sequence q n asymptotically resembles the sequence p n . Then, thesequence p ′ × q n − will asymptotically resemble the sequence p n as well. Proof:
For any ǫ >
0, due to the asymptotic resemblance of the sequence q n to the sequence p n , we have, for n large enough, q n − ( A ′ n − ) > − ǫ, (A.13)where A ′ n − = { a ∈ A n − | ρ A ( ε a , p ) < ǫ } . (A.14)By Lemma 4, we have ρ A ( ε ( a ,a ) , ε a ) ≤ n , ∀ a ∈ A, a ∈ A n − . (A.15)Hence, for large enough n , A × A ′ n − ⊆ A ′′ n , (A.16)where A ′′ n = { a ∈ A n | ρ A ( ε a , p ) < ǫ } . (A.17)But by (A.13), this means that( p ′ × q n − )( A ′′ n ) ≥ ( p ′ × q n − )( A × A ′ n − ) = p ′ ( A ) × q n − ( A ′ n − ) > − ǫ. (A.18)That is, p ′ × q n − asymptotically resembles p n . Lemma 7
Let A and B be separable metric spaces. Also, q n ∈ P ( A n × B n ) for every n ∈ N and p ∈ P ( A × B ) . Suppose the sequence q n asymptotically resembles the sequence p n . Then,the sequence q n | A n will asymptotically resemble the sequence ( p | A ) n . Proof:
For any ǫ >
0, due to the asymptotic resemblance of the sequence q n to the sequence p n , we have, for n large enough, q n ( C ′ n ) > − ǫ, (A.19)where C ′ n = { c = ( a, b ) ∈ A n × B n | ρ A × B ( ε c , p ) < ǫ } . (A.20)28ut by (87) of Yang [33], ρ A ( ε a , p | A ) = ρ A ( ε c | A , p | A ) ≤ ρ A × B ( ε c , p ) , ∀ c = ( a, b ) ∈ C ′ n . (A.21)Hence, C ′ n ⊆ A ′ n × B n , (A.22)where A ′ n = { a ∈ A n | ρ A ( ε a , p | A ) < ǫ } . (A.23)Combining (A.19) and (A.22), we can obtain( q n | A n )( A ′ n ) = q n ( A ′ n × B n ) ≥ q n ( C ′ n ) > − ǫ. (A.24)This indicates that q n | A n asymptotically resembles ( p | A ) n . B Proofs of Section 5
Proof of Proposition 1:
We first prove (i). Fix some ǫ ∈ (0 , S , we can identify some I of its points ¯ s , ¯ s , ..., ¯ s I , so that each σ ( { ¯ s i } ) > I X i =1 σ ( { ¯ s i } ) > − ǫ. (B.1)For convenience, let ¯ S ′ = { ¯ s , ¯ s , ..., ¯ s I } and ¯ S ′′ = S \ ¯ S ′ .Since S is discrete, the distance d S ( ¯ S ′ , ¯ S ′′ ) = inf s ′ ∈ ¯ S ′ ,s ′′ ∈ ¯ S ′′ d S ( s ′ , s ′′ ) >
0. For i, j =1 , , ..., I , let us use d ij for d S (¯ s i , ¯ s j ) and σ i for σ ( { ¯ s i } ). Now define δ = ǫI ∧ d S ( ¯ S ′ , ¯ S ′′ ) ∧ (min i = j d ij ) ∧ (min i σ i , (B.2)which is still strictly positive. In this paper, we use a ∧ b to stand for min { a, b } and a ∨ b tostand for max { a, b } .For any n ∈ N , define S ′ n ∈ B ( S n ) so that S ′ n = { s ∈ S n | ρ S ( ε s , σ ) < δ } . (B.3)By the hypothesis that π n asymptotically resembles σ n , we can ensure π n ( S ′ n ) > − ǫ , (B.4)29y making n large enough.Consider any such n , as well as any s = ( s , s , ..., s n ) ∈ S ′ n and i = 1 , , ..., I . It followsfrom δ ≤ d S ( ¯ S ′ , ¯ S ′′ ) ∧ (min i = j d ij ) that ( { ¯ s i } ) δ , whose meaning comes from ( ?? ), is still { ¯ s i } itself. Now by (B.3), ε s ( { ¯ s i } ) < σ (( { ¯ s i } ) δ ) + δ = σ i + δ, (B.5)and ε s ( { ¯ s i } ) = 1 − ε s ( { ¯ s j | j = i } ∪ ¯ S ′′ ) > − σ (( { ¯ s j | j = i } ∪ ¯ S ′′ ) δ ) − δ = 1 − σ ( { ¯ s j | j = i } ∪ ¯ S ′′ ) − δ = σ i − δ, (B.6)which is still above δ > δ ≤ min i σ i /
2. For convenience, let n i ( s ) = n · ε s ( { ¯ s i } ), the number of components s m of s that happen to be ¯ s i . Now we know that n i ( s ) is above nδ for every s ∈ S ′ n and i = 1 , , ..., I .On the other hand, by Lemma 2, there exists some n i for each i = 1 , , ..., I , so that when n i > n i , ( χ (¯ s i )) n i ( X ′ in i ) > − ǫ I , (B.7)where X ′ in i = { x ∈ X n i | ρ X ( ε x , χ (¯ s i )) < δ } . (B.8)Since δ >
0, we can ensure that nδ and hence n i ( s ) is above n i for every i = 1 , , ..., I byletting n be large enough.Fix a big n that facilitates both (B.4) and (B.7). For any ( s, x ) ∈ S n × X n , let ˜ x i ( s, x )be the n i ( s )-long vector of x m ’s whose corresponding s m ’s happen to be ¯ s i :˜ x i ( s, x ) = ( x m | m = 1 , , ..., n but with s m = ¯ s i ) ∈ X n i ( s ) . (B.9)Define U ′ n ∈ B ( S n × X n ), so that U ′ n = { ( s, x ) ∈ S n × X n | s ∈ S ′ n and ˜ x i ( s, x ) ∈ X ′ in i ( s ) for each i = 1 , , ..., I } . (B.10)By (16), (B.4), and (B.7), we have( π n ⊗ χ n )( U ′ n ) = Z S ′ n π n ( ds ) · I Y i =1 ( χ (¯ s i )) n i ( s ) ( X ′ in i ( s ) ) > (1 − ǫ · (1 − ǫ I ) I > − ǫ. (B.11)For any ( s, x ) in U ′ n , let us examine how close ε sx = ε (( s ,x ) ,..., ( s n ,x n )) is to σ ⊗ χ . Recallthat S = { ¯ s , ¯ s , ..., ¯ s I } ∪ ¯ S ′′ . So for any U ′ ∈ B ( S × X ), U ′ = ( I [ i =1 { ¯ s i } × X ′ i ) [ U ′′ , (B.12)30here X ′ i ∈ B ( X ) for i = 1 , , ..., I , while U ′′ is such that s ′′ ∈ ¯ S ′′ for any ( s ′′ , x ′′ ) ∈ U ′′ . Noteagain that δ ≤ d S ( ¯ S ′ , ¯ S ′′ ) ∧ min i = j d ij . When we take d S × X to mean d S × X (( s ′ , x ′ ) , ( s ′′ , x ′′ )) = d S ( s ′ , s ′′ ) ∨ d X ( x ′ , x ′′ ), (B.12) would lead to I [ i =1 { ¯ s i } × ( X ′ i ) δ ⊆ ( U ′ ) δ . (B.13)Now from (B.5) and (B.8), ε sx ( { ¯ s i } × X ′ i ) = ε s ( { ¯ s i } ) · ε ˜ x i ( s,x ) ( X ′ i ) < ( σ i + δ ) · [ χ (¯ s i | ( X ′ i ) δ ) + δ ] ≤ ( σ ⊗ χ )( { ¯ s i } × ( X ′ i ) δ ) + 2 δ + δ < ( σ ⊗ χ )( { ¯ s i } × ( X ′ i ) δ ) + 3 δ, (B.14)where the last inequality is due to our choice that δ ≤ ǫ/I <
1. Meanwhile, ε sx ( U ′′ ) ≤ ε sx ( ¯ S ′′ × X ) = ε s ( ¯ S ′′ ) = 1 − I X i =1 ε s ( { ¯ s i } ) < − I X i =1 σ i + Iδ < ǫ + Iδ, (B.15)where the second-to-last inequality is due to (B.6) and the last one is due to (B.1). Com-bine (B.12) to (B.15), and we can obtain ε sx ( U ′ ) < ( σ ⊗ χ )(( U ′ ) δ ) + ǫ + 4 Iδ. (B.16)Thus, ρ S × X ( ε sx , σ ⊗ χ ) < ǫ + 4 Iδ ≤ ǫ, (B.17)where the last inequality comes from our choice that δ ≤ ǫ/I . Since (B.11) and (B.17) areto occur at any n that is large enough, we see that (i) is true.We then prove (ii). For convenience, we denote S × X by U , σ ⊗ χ by τ , and for each n ∈ N , π n ⊗ χ n by ν n . From (i), we have the sequence ν n asymptotically resembling thesequence τ n .Fix some ǫ ∈ (0 , S and X , and hence that of U , we canidentify some J points ¯ u , ¯ u , ..., ¯ u J , so that each τ ( { ¯ u j } ) > J X j =1 τ ( { ¯ u j } ) > − ǫ. (B.18)For convenience, let ¯ U ′ = { ¯ u , ¯ u , ..., ¯ u J } and ¯ U ′′ = U \ ¯ U ′ .As S and X are both discrete, so U is discrete as well. Thence, the distance d U ( ¯ U ′ , ¯ U ′′ ) =inf u ′ ∈ ¯ U ′ ,u ′′ ∈ ¯ U ′′ d U ( u ′ , u ′′ ) >
0. For j, k = 1 , , ..., J , let us use d ′ jk for d U (¯ u j , ¯ u k ) and τ j for τ ( { ¯ u j } ). Now define δ = ǫJ ∧ d U ( ¯ U ′ , ¯ U ′′ ) ∧ (min j = k d ′ jk ) ∧ (min j τ j , (B.19)31hich is still strictly positive.For any n ∈ N , define U ′ n ∈ B ( U n ) so that U ′ n = { u ∈ U n | ρ U ( ε u , τ ) _ [2 · sup u ′ ∈ U n max m =1 ρ S ( g ( u ′ , ε u − m ) , g ( u ′ , τ ))] < δ } . (B.20)By (i) that ν n asymptotically resembles τ n , the hypothesis that g ( u, · ) is continuous at a u -independent rate, and Lemma 4, we can ensure ν n ( U ′ n ) > − ǫ , (B.21)by making n large enough,Consider any such n , as well as any u = ( u , u , ..., u n ) ∈ U ′ n and j = 1 , , ..., J . It followsfrom δ ≤ d U ( ¯ U ′ , ¯ U ′′ ) ∧ (min j = k d ′ jk ) that ( { ¯ u j } ) δ is still { ¯ u j } itself. Now by (B.20), ε u ( { ¯ u j } ) < τ (( { ¯ u j } ) δ ) + δ = τ j + δ, (B.22)and ε u ( { ¯ u j } ) > − τ (( { ¯ u k | k = j } ∪ ¯ U ′′ ) δ ) − δ = τ j − δ, (B.23)which is still above δ > δ ≤ min j τ j /
2. For convenience, let n ′ j ( u ) = n · ε u ( { ¯ u j } ). Now we know that n ′ j ( u ) is above ⌊ nδ ⌋ for every j = 1 , , ..., J .Due to the countability of U and Lemma 3 on the uniform Glivenko-Cantelli property,there exists some n ′ , independent of both j and u , such that when every n ′ j ( u ) > n ′ ,( g (¯ u j , ε u \ ¯ u j )) n ′ j ( u ) ( S ′′ jn ′ j ( u ) ( u )) > − ǫ J , ∀ j = 1 , , ..., J, (B.24)where every u \ ¯ u j is the ( n − u but with only n ′ j ( u ) − u j , and S ′′ jn ′ ( u ′ ) = { s ∈ S n ′ | ρ S ( ε s , g (¯ u j , ε u ′ \ ¯ u j )) < δ } . (B.25)But in light of (B.20), we can really guarantee that( g (¯ u j , ε u \ ¯ u j )) n ′ j ( u ) ( S ′ jn ′ j ( u ) ) > − ǫ J , ∀ j = 1 , , ..., J, (B.26)where S ′ jn ′ = { s ∈ S n ′ | ρ S ( ε s , g (¯ u j , τ )) < δ } . (B.27)Since δ >
0, we can ensure that ⌊ nδ ⌋ and hence n ′ j ( u ) is above n ′ for every j = 1 , , ..., J byletting n be large enough. 32ix a big n that facilitates both (B.21) and (B.26). For any ( u, s ) ∈ U n × S n , let ˜ s j ( u, s )be the n ′ j ( u )-long vector of s m ’s whose corresponding u m ’s happen to be ¯ u j :˜ s j ( u, s ) = ( s m | m = 1 , , ..., n but with u m = ¯ u j ) ∈ S n ′ j ( u ) . (B.28)Define V ′ n ∈ B ( U n × S n ), so that V ′ n = { ( u, s ) ∈ U n × S n | u ∈ U ′ n and ˜ s j ( u, s ) ∈ S ′ jn ′ j ( u ) for each j = 1 , , ..., J } . (B.29)Let us follow the same logic as used from (B.11) to (B.17) in the proof of (i), withappropriate substitutions, such as J for I , U for S , S for X , ν n for π n , τ for σ , g ( · , · , τ ) for χ , g n for χ n , V ′ n for U ′ n , (B.21) for (B.4), and (B.26) for (B.7). We can then derive that( ν n ⊗ g n )( V ′ n ) > − ǫ, (B.30)whereas, for any ( u, s ) in V ′ n , ρ U × S ( ε us , τ ⊗ g ( · , · , τ )) < ǫ. (B.31)Since (B.30) and (B.31) are to occur at any n that is large enough, we see that ν n ⊗ g n would asymptotically resemble ( τ ⊗ g ( · , · , τ )) n . Lemma 7 will then lead to the asymptoticresemblance of the sequence ν n ⊙ g n = ( ν n ⊗ g n ) | S n to the sequence ( τ ⊙ g ( · , · , τ )) n =(( τ ⊗ g ( · , · , τ )) | S ) n . Thus (ii) is true.For (iii), denote the given ( s, x ) by u . By Lemma 4, we can make ε u = ( u , ε u − ) n arbitrarily close to ε u − for any u − = ( u , u , ..., u n ) ∈ U n − by letting n be large enough.Hence, we can follow the proof of (ii) almost verbatim, with its (B.20) replaced by U ′ n − = { u − ∈ U n − | ρ U ( ε u , τ ) _ [2 · sup u ′ ∈ U n max m =1 ρ S ( g ( u ′ , ε u − m ) , g ( u ′ , τ ))] < δ } , (B.32)its (B.21) replaced by( δ u × ν n − )( { u } × U ′ n − ) = ν n − ( U ′ n − ) > − ǫ , (B.33)any choice of u ∈ U n replaced by u − ∈ U n − , and any choice of u ∈ U ′ n replaced by u − ∈ U ′ n − . Proof of Theorem 1:
We prove by induction on t ′ .First, note that T [ t,t − ◦ σ t is merely σ t itself. Hence, the claim is true for t ′ = t becauseby the hypothesis, we do have π nt asymptotically resembling ( T [ t,t − ◦ σ t ) n = σ nt . Then,33or some t ′ = t, t + 1 , ..., ¯ t , suppose the claim is true, that π nt ′ = π nt ⊙ Π t ′ − t ′′ = t ( χ nt ′′ ⊙ ˜ g nt ′′ )asymptotically resembles σ nt ′ = ( T [ t,t ′ − ( χ [ t,t ′ − ) ◦ σ t ) n .Assumption 1 on ˜ g t ′ ( s, x, τ )’s equi-continuity in τ allows us to use part (ii) of Propo-sition 1. By it, we would have π nt ′ ⊙ χ nt ′ ⊙ ˜ g nt ′ asymptotically resembling ( σ t ′ ⊙ χ t ′ ⊙ ˜ g t ( · , · , σ t ′ ⊗ χ t ′ )) n . Since the former is merely π n,t ′ +1 = π nt ⊙ Π t ′ t ′′ = t ( χ nt ′′ ⊙ ˜ g nt ′′ ) and the latteris σ nt ′ +1 = ( T [ tt ′ ] ( χ [ tt ′ ] ) ◦ σ t ) n , we have thus proved the claim for t ′ + 1.The induction process is now complete. C Proofs of Section 6
Proof of Proposition 2:
Let us prove by induction on t . By (21) and (25), the desiredresult is true for t = ¯ t + 1.At some t = ¯ t, ¯ t − , ...,
1, suppose for any σ t +1 and any sequence ˆ π n − ,t +1 that asymptoti-cally resembles σ n − t +1 , the sequence R S n − ˆ π n − ,t +1 ( ds t +1 , − ) · v n,t +1 ( s t +1 , , ξ [ t +1 , ¯ t ] , , ε s t +1 , − , χ [ t +1 , ¯ t ] )converges to v t +1 ( s t +1 , , ξ [ t +1 , ¯ t ] , σ t +1 , χ [ t +1 , ¯ t ] ) at a rate independent of both s t +1 , and ξ [ t +1 , ¯ t ] .Now, given the sequence ˆ π n − ,t that is known to asymptotically resemble σ n − t , we areto show that R S n − ˆ π n − ,t ( ds t, − ) · v nt ( s t , ξ [ t ¯ t ] , ε s t, − , χ [ t ¯ t ] ) will converge to v t ( s t , ξ [ t ¯ t ] , σ t , χ [ t ¯ t ] )at a rate independent of both s t and ξ [ t ¯ t ] . For convenience, let σ t +1 = T t ( χ t ) ◦ σ t .Note that, by (22) and (26),sup s t ∈ S,ξ [ t ¯ t ] ∈ ( K ( S,X )) ¯ t − t +1 | v t ( s t , ξ [ t ¯ t ] , σ t , χ [ t ¯ t ] ) − R S n − ˆ π n − ,t ( ds t, − ) · v nt ( s t , ξ [ t ¯ t ] , ε s t, − , χ [ t ¯ t ] ) |≤ M n + M n + M n , (C.1)where M n = sup ( s t ,x t ) ∈ S × X R S n − × X n − (ˆ π n − ,t ⊗ χ n − t )( ds t, − × dx t, − ) ×× | ˜ f t ( s t , x t , σ t ⊗ χ t ) − ˜ f t ( s t , x t , ε s t, − x t, − ) | , (C.2) M n = sup ( s t ,x t ) ∈ S × X,ξ [ t +1 , ¯ t ] ∈ ( K ( S,X )) ¯ t − t R S ˜ g t ( s t , x t , σ t ⊗ χ t | ds t +1 , ) ×× | v t +1 ( s t +1 , , ξ [ t +1 , ¯ t ] , σ t +1 , χ [ t +1 , ¯ t ] ) − R S n − × X n − (ˆ π n − ,t ⊗ χ n − t )( ds t, − × dx t, − ) ×× Π nm =2 R S ˜ g t ( s tm , x tm , ε s t, − m x t, − m | ds t +1 ,m ) · v n,t +1 ( s t +1 , , ξ [ t +1 , ¯ t ] , ε s t +1 , − , χ [ t +1 , ¯ t ] ) | , (C.3)and M n = sup ( s t ,x t ) ∈ S × X,ξ [ t +1 , ¯ t ] ∈ ( K ( S,X )) ¯ t − t R S n − × X n − (ˆ π n − ,t ⊗ χ n − t )( ds t, − × dx t, − ) ×× | [ R S ˜ g t ( s t , x t , σ t ⊗ χ t | ds t +1 , ) − R S ˜ g t ( s t , x t , ε s t, − x t, − | ds t +1 , )] ×× Π nm =2 R S ˜ g t ( s tm , x tm , ε s t, − m x t, − m | ds t +1 ,m ) · v n,t +1 ( s t +1 , , ξ [ t +1 , ¯ t ] , ε s t +1 , − , χ [ t +1 , ¯ t ] ) | . (C.4)34e now show that each of the above three terms can be made arbitrarily small by letting n be large enough.For M n , define ˜ U n − ( δ ) ∈ B ( S n − × X n − ) for every δ >
0, so that˜ U n − ( δ ) = { ( s t, − , x t, − ) ∈ S n − × X n − | ρ S × X ( ε s t, − x t, − , σ t ⊗ χ t ) < δ } . (C.5)From (C.2), we know M n ≤ M n ( δ ) + M n ( δ ) for any δ >
0, where M n ( δ ) = sup ( s t ,x t ) ∈ S × X, ( s t, − ,x t, − ) ∈ ˜ U n − ( δ ) | ˜ f t ( s t , x t , σ t ⊗ χ t ) − ˜ f t ( s t , x t , ε s t, − x t, − ) | , (C.6)and M n ( δ ) = sup ( s t ,x t ) ∈ S × X R ( S n − × X n − ) \ ˜ U n − ( δ ) (ˆ π n − ,t ⊗ χ n − t )( ds t, − × dx t, − ) ×× [ | ˜ f t ( s t , x t , σ t ⊗ χ t ) | + | ˜ f t ( s t , x t , ε s t, − x t, − ) | ] . (C.7)Because Assumption 2 says that ˜ f t ( s, x, τ ) is continuous in τ at an ( s, x )-independent rate,we can make M n ( δ ) arbitrarily small by letting δ be small enough. Meanwhile, by theasymptotic resemblance of the sequence ˆ π n − ,t to the sequence σ n − t and part (i) of Propo-sition 1, we know that the sequence ˆ π n − ,t ⊗ χ n − t asymptotically resembles the sequence( σ t ⊗ χ t ) n − . So the measure (ˆ π n − ,t ⊗ χ n − t )(( S n − × X n − ) \ ˜ U n − ( δ )) can be made arbitrarilysmall at any δ by letting n be large enough. Since ˜ f t is bounded, this means that M n ( δ )can be made arbitrarily small as well.For M n , note the second integral in (C.3) can be understood as ˆ π n − ,t +1 ( s t , x t | ds t +1 , − ) = R S n − { [( δ s t x t × ( σ n − t ⊗ χ n − t )) ⊙ ˜ g nt ] | S n − } ( ds t +1 , − ). So we have M n ≤ sup ( s t ,x t ) ∈ S × X,s t +1 , ∈ S,ξ [ t +1 , ¯ t ] ∈ ( K ( S,X )) ¯ t − t | v t +1 ( s t +1 , , ξ [ t +1 , ¯ t ] , σ t +1 , χ [ t +1 , ¯ t ] ) − R S n − ˆ π n − ,t +1 ( s t , x t | ds t +1 , − ) · v n,t +1 ( s t +1 , , ξ [ t +1 , ¯ t ] , ε s t +1 , − , χ [ t +1 , ¯ t ] ) | . (C.8)Meanwhile, Assumption 1 allows us to use part (iii) of Proposition 1. By the asymptoticresemblance of the sequence ˆ π n − ,t to the sequence σ n − t , part (iii) of Proposition 1, andLemma 5, we know that the sequence ˆ π n − ,t +1 ( s t , x t ) asymptotically resembles the se-quence σ n − t +1 at an ( s t , x t )-independent rate. Then by the induction hypothesis where theconvergence rate is also ( s t +1 , , ξ [ t +1 , ¯ t ] )-independent, we can conclude that M n can be madearbitrarily small by letting n be large enough.For M n , define V n ( s t , x t , ε s t, − x t, − , ξ [ t +1 , ¯ t ] ) so that V n ( s t , x t , ε s t, − x t, − , ξ [ t +1 , ¯ t ] ) = | [ R S ˜ g t ( s t , x t , σ t ⊗ χ t | ds t +1 , ) − R S ˜ g t ( s t , x t , ε s t, − x t, − | ds t +1 , )] · Π nm =2 ˜ g t ( s tm , x tm , ε s t, − m x t, − m | ds t +1 ,m ) ×× v n,t +1 ( s t +1 , , ξ [ t +1 , ¯ t ] , ε s t +1 , − , χ [ t +1 , ¯ t ] ) | . (C.9)35hen, (C.4) can be written as M n = sup ( s t ,x t ) ∈ S × X,ξ [ t +1 , ¯ t ] ∈ ( K ( S,X )) ¯ t − t R S n − × X n − ×× (ˆ π n − ,t ⊗ χ n − t )( ds t, − × dx t, − ) · V n ( s t , x t , ε s t, − x t, − , ξ [ t +1 , ¯ t ] ) . (C.10)Noting the definition of ˜ U n − ( δ ) in (C.5) for any δ >
0, we see that M n ≤ M n ( δ )+ M n ( δ ),where M n ( δ ) = sup ( s t ,x t ) ∈ S × X, ( s t, − ,x t, − ) ∈ ˜ U n − ( δ ) ,ξ [ t +1 , ¯ t ] ∈ ( K ( S,X )) ¯ t − t V n ( s t , x t , ε s t, − x t, − , ξ [ t +1 , ¯ t ] ) , (C.11)and M n ( δ ) = sup ( s t ,x t ) ∈ S × X,ξ [ t +1 , ¯ t ] ∈ ( K ( S,X )) ¯ t − t R ( s t, − ,x t, − ) ∈ ( S n − × X n − ) \ ˜ U n − ( δ ) (ˆ π n − ,t ⊗ χ n − t )( ds t, − × dx t, − ) · V n ( s t , x t , ε s t, − x t, − , ξ [ t +1 , ¯ t ] ) . (C.12)We argue that M n ( δ ) can be made arbitrarily small as δ approaches 0 + . Due to As-sumption 1 that ˜ g t ( s, x, τ ) is continuous in τ at an ( s, x )-independent rate, we can make˜ g t ( s t , x t , ε s t, − x t, − ) for any ( s t, − , x t, − ) ∈ ˜ U n − ( δ ) arbitrarily close to ˜ g t ( s t , x t , σ t ⊗ χ t )by rendering δ small enough, without respect to ( s t , x t ). Due to its countability, we canwrite S = { ¯ s , ¯ s , ... } . Under known s t , x t , ε s t, − x t, − , and ξ [ t +1 , ¯ t ] , let us use the simplifiednotation γ i = ˜ g t ( s t , x t , σ t ⊗ χ t |{ ¯ s i } ) , (C.13) γ ′ i = ˜ g t ( s t , x t , ε s t, − x t, − |{ ¯ s i } ) , (C.14)and v i = Π nm =2 Z S ˜ g t ( s tm , x tm , ε s t, − m x t, − m | ds t +1 ,m ) · v n,t +1 ( s t +1 , , ξ [ t +1 , ¯ t ] , ε s t +1 , − , χ [ t +1 , ¯ t ] ) . (C.15)Then, (C.9) can be expressed as V n ( s t , x t , ε s t, − x t, − , ξ [ t +1 , ¯ t ] ) = | X i γ i · v i − X i γ ′ i · v i | . (C.16)Note the | v i | ’s are uniformly bounded, say by v , due to the boundedness of the ˜ f t ′ ’s andthe finiteness of ¯ t . Let I be the set of i ’s such that γ i ≥ γ ′ i . Then, from (C.16), we have V n ( s t , x t , ε s t, − x t, − , ξ [ t +1 , ¯ t ] ) ≤ v · X i ∈ I ( γ i − γ ′ i ) . (C.17)Let δ be below inf s = s ′ d S ( s, s ′ ) >
0. But then, ( s t, − , x t, − ) ∈ ˜ U n − ( δ ) would entail P i ∈ I ( γ i − γ ′ i ) = ˜ g t ( s t , x t , σ t ⊗ χ t |{ s i | i ∈ I } ) − ˜ g t ( s t , x t , ε s t, − x t, − |{ s i | i ∈ I } )= ˜ g t ( s t , x t , σ t ⊗ χ t |{ s i | i ∈ I } ) − ˜ g t ( s t , x t , ε s t, − x t, − | ( { s i | i ∈ I } ) δ ) < δ. (C.18)36n view of (C.17), V n ( s t , x t , ε s t, − x t, − , ξ [ t +1 , ¯ t ] ) with ( s t, − , x t, − ) ∈ ˜ U n − ( δ ) can be madearbitrarily small by decreasing δ at a rate independent of ( s t , x t , ξ [ t +1 , ¯ t ] ). In view of (C.11),we see that M n ( δ ) can be made arbitrarily small by rendering δ small enough.As noted earlier, the probability (ˆ π n − ,t ⊗ χ n − t )(( S n − × X n − ) \ ˜ U n − ( δ )) can be madearbitrarily small at any δ when n is made large enough. But since V n ( s t , x t , ε s t, − x t, − , ξ [ t +1 , ¯ t ] )is uniformly bounded, this means that M n ( δ ) can be made arbitrarily small as well.Hence, all three terms can be made arbitrarily small by letting n be large enough. Wehave thus completed the induction process. Proof of Theorem 2:
Given (23) for every t = 1 , , ..., ¯ t and ξ t ∈ K ( S, X ), we are toverify (27) for every t = 1 , , ..., ¯ t , ǫ >
0, large enough n , s t ∈ S , and ξ [ t ¯ t ] ∈ ( K ( S, X )) ¯ t − t +1 .First, we show that the one-time formulation of (23) would already imply the futility ofany multi-period unilateral deviation. Another way to write the condition is, at t ′ = 0, forany t = 1 , , ..., ¯ t − t ′ and ξ [ t,t + t ′ ] ∈ ( K ( S, X )) t ′ +1 , v t ( s t , ( ξ [ t,t + t ′ − , χ [ t + t ′ , ¯ t ] ) , σ t , χ [ t ¯ t ] ) ≥ v t ( s t , ( ξ [ t,t + t ′ ] , χ [ t + t ′ +1 , ¯ t ] ) , σ t , χ [ t ¯ t ] ) . (C.19)Now suppose (C.19) is true for some t ′ = 0 , , ..., ¯ t −
1. We are to show its validity at t ′ + 1.But by (22), for any t = 1 , , ..., ¯ t − t ′ , v t ( s t , ( ξ [ t,t + t ′ ] , χ [ t + t ′ +1 , ¯ t ] ) , σ t , χ [ t ¯ t ] ) − v t ( s t , ( ξ [ t,t + t ′ +1] , χ [ t + t ′ +2 , ¯ t ] ) , σ t , χ [ t ¯ t ] )= R X ξ t ( s t | dx t ) · R S ˜ g t ( s t , x t , σ t ⊗ χ t | ds t +1 ) ×× [ v t +1 ( s t +1 , ( ξ [ t +1 ,t + t ′ ] , χ [ t + t ′ +1 , ¯ t ] ) , T t ( χ t ) ◦ σ t , χ [ t +1 , ¯ t ] ) − v t +1 ( s t +1 , ( ξ [ t +1 ,t + t ′ +1] , χ [ t + t ′ +2 , ¯ t ] ) , T t ( χ t ) ◦ σ t , χ [ t +1 , ¯ t ] )] , (C.20)which, by the induction hypothesis (C.19), is positive. Therefore, (C.19) is true for any t = 1 , , ..., ¯ t , t ′ = 0 , , ..., ¯ t − t , and ξ [ t,t + t ′ ] ∈ ( K ( S, X )) t ′ +1 .Using (C.19) multiple times, we can derive, for any t = 1 , , ..., ¯ t and ξ [ t ¯ t ] ∈ ( K ( S, X )) ¯ t − t +1 , v t ( s t , χ [ t ¯ t ] , σ t , χ [ t ¯ t ] ) ≥ v t ( s t , ( ξ t , χ [ t +1 , ¯ t ] ) , σ t , χ [ t ¯ t ] ) ≥ v t ( s t , ( ξ [ t,t +1] , χ [ t +2 , ¯ t ] ) , σ t , χ [ t ¯ t ] ) ≥ · · · ≥ v t ( s t , ( ξ [ t, ¯ t − , χ ¯ t ) , σ t , χ [ t ¯ t ] ) ≥ v t ( s t , ξ [ t ¯ t ] , σ t , χ [ t ¯ t ] ) . (C.21)In view of (C.21), we would have (27) if for any ǫ and large enough n , Z S n − ˆ π n − ,t ( ds t, − ) · v nt ( s t , χ [ t ¯ t ] , ε s t, − , χ [ t ¯ t ] ) > v t ( s t , χ [ t ¯ t ] , σ t , χ [ t ¯ t ] ) − ǫ , (C.22)and for any ξ [ t ¯ t ] ∈ ( K ( S, X )) ¯ t − t +1 , Z S n − ˆ π n − ,t ( ds t, − ) · v nt ( s t , ξ [ t ¯ t ] , ε s t, − , χ [ t ¯ t ] ) < v t ( s t , ξ [ t ¯ t ] , σ t , χ [ t ¯ t ] ) + ǫ . (C.23)37oth (C.22) and (C.23) would be true if Z S n − ˆ π n − ,t ( ds t, − ) · v nt ( s t , ξ [ t ¯ t ] , ε s t, − , χ [ t ¯ t ] ) −→ n → + ∞ v t ( s t , ξ [ t ¯ t ] , σ t , χ [ t ¯ t ] ) , (C.24)at an ( s t , ξ [ t ¯ t ] )-independent convergence rate. But this was provided by Proposition 2. D Proofs of Section 7
Proof of Proposition 3:
Since A is discrete, we can denote it by either { ¯ a , ¯ a , ... } or { ¯ a , ..., ¯ a I } for some finite I . We work with the former only, as the latter is similarly treatable.For any n ∈ N , define N n = { ( n , n , ... ) | n i = 0 , , ..., n for each i = 1 , , ..., and + ∞ X i =1 n i = n } . (D.1)For each ( n , n , ... ) ∈ N n , define A nn n ··· so that A nn n ··· = { a ∈ A n | ε a ( { ¯ a i } ) = n i n for any i = 1 , , ... } . (D.2)Note that every A nn n ··· is symmetric, different A nn n ··· ’s are non-overlapping, and A n = [ ( n n ··· ) ∈ N n A nn n ··· . (D.3)Due to the above decomposition, each a ∈ A n belongs to its own A nn · ε a ( { ¯ a } ) ,n · ε a ( { ¯ a } ) , ··· .For any ( n , n , · · · ) ∈ N n , the set A nn n ··· contains n ! / ( Q + ∞ i =1 n i !) distinct members of A n ,say a , ..., a n ! / ( Q + ∞ i =1 n i !) . In addition, every a k is of the form ψa for some ψ ∈ Ψ n . Thus, dueto q n ’s symmetry, for k = 1 , , ..., n ! / ( Q + ∞ i =1 n i !), q n ( { a k } ) = Q + ∞ i =1 n i ! n ! · q n ( A nn n ··· ) . (D.4)Suppose n i ≥ i = 1 , , ... . Then, exactly ( n − / (( n i − · Q j = i n j !) of the a k ’swill have a k = ¯ a i . Therefore, for any such a k , q n (( { ¯ a i } × A n − ) ∩ A nn n ··· ) = ( n i − · Q j = i n j !( n − · q n ( { a k } ) = n i n · q n ( A nn n ··· ) , (D.5)where the second equality stems from (D.4). The above left- and right-hand sides are cer-tainly equated as well when n i = 0. Combine (D.3) and (D.5), and we can obtain q n ( { ¯ a i } × A n − ) = X ( n ,n ,... ) ∈ N n n i n · q n ( A nn n ··· ) . (D.6)38n the other hand, we have min i = j d A (¯ a i , ¯ a j ) > A ’s discreteness. Suppose ǫ > a ∈ A n would satisfy ρ A ( ε a , p ) < ǫ if and only if + ∞ X i =1 | ε a ( { ¯ a i } ) − p ( { ¯ a i } ) | < ǫ, (D.7)and hence only if + ∞ max i =1 | ε a ( { ¯ a i } ) − p ( { ¯ a i } ) | < ǫ. (D.8)Since the sequence q n asymptotically resembles p , for any ǫ > i = j d A (¯ a i , ¯ a j ) >
0, we can pick n large enough so that (A.7) and (A.8) in the proofof Lemma 5 are true. Define N ′ n ⊆ N n so that for any ( n , n , ... ) ∈ N ′ n , + ∞ X i =1 | n i n − p ( { ¯ a i } ) | < ǫ, and hence + ∞ max i =1 | n i n − p ( { ¯ a i } ) | < ǫ. (D.9)Due to (D.2), (D.3), and (D.7), we have the following for the A ′ n defined in (A.8): A ′ n = [ ( n ,n ,... ) ∈ N ′ n A nn n ··· . (D.10)Now for any i = 1 , , ... , we have | q n | A ( { ¯ a i } ) − p ( { ¯ a i } ) | = | q n ( { ¯ a i } × A n − ) − p ( { ¯ a i } ) | = | ( P ( n ,n ,... ) ∈ N ′ n + P ( n ,n ,... ) ∈ N n \ N ′ n ) n i · q n ( A nn n ··· ) /n − p ( { ¯ a i } ) |≤ q n ( A ′ n ) · | n i /n − p ( { ¯ a i } ) | + q n ( A n \ A ′ n ) < ǫ. (D.11)Here, the first equality comes from the definition of marginal probability, the second equalitycomes from (D.6), the first inequality can be attributed to (D.10), and the last inequality isdue to (A.7) and (D.8). Thus, for every a ∈ A , we have lim n → + ∞ q n | A ( { a } ) = p ( { a } ). Proof Proposition 4:
For the time being, it does not matter whether A = { ¯ a , ¯ a , ... } or { ¯ a , ..., ¯ a I } for some finite I . The first few steps are the same as those in the proof ofLemma 5. For any ǫ > n large enough, we can have (A.7) to (A.11) as in that proof.Fix i = 1 , , ... with p ( { ¯ a i } ) >
0. Due to (A.10), A ′ n ∩ ( { ¯ a i } × A n − ) ⊆ ( A × A ′′ n − ) ∩ ( { ¯ a i } × A n − ) = { ¯ a i } × A ′′ n − , (D.12)where A ′ n is defined in (A.8) and A ′′ n − is defined in (A.11). Thus, q n ( { ¯ a i } × A ′′ n − ) ≥ q n ( A ′ n ∩ ( { ¯ a i } × A n − )) = q n (( { ¯ a i } × A n − ) \ ( A n \ A ′ n )) ≥ q n ( { ¯ a i } × A n − ) − q n ( A n \ A ′ n ) > q n ( { ¯ a i } × A n − ) − ǫ, (D.13)39here the last inequality is due to (A.7).Since q n is symmetric, we know from Proposition 3 that, when n is large enough, q n ( { ¯ a i } × A n − ) = q n | A ( { ¯ a i } ) > p ( { ¯ a i } )2 > . (D.14)Combining (D.13) and (D.14), we can obtain q n,A | A n − (¯ a i | A ′′ n − ) = q n ( { ¯ a i } × A ′′ n − ) q n ( { ¯ a i } × A n − ) > − ǫq n ( { ¯ a i } × A n − ) > − ǫp ( { ¯ a i } ) . (D.15)With A ′′ n − ’s definition in (A.11), we get q n,A | A n − (¯ a i |· )’s asymptotic resemblance to p n − . E Developments in Section 8
Proof of Theorem 4:
Let ǫ > t = 1 , , ... and χ ∈ K ( S, X ), we use χ t todenote ( χ, χ, ..., χ ) ∈ ( K ( S, X )) t . From (38), we know | v ∞ n ( s , ξ [1 ∞ ] , ε s − , χ ∞ ) − v tn ( s , ξ [1 t ] , ε s − , χ t ) |≤ ¯ α t · ¯ f − ¯ α . (E.1)Hence, when t ≥ ln(6 ¯ f / ( ǫ · (1 − ¯ α ))) / ln(1 / ¯ α ) + 1, v ∞ n ( s , χ ∞ , ε s − , χ ∞ ) > v tn ( s , χ t , ε s − , χ t ) − ǫ , (E.2)and v ∞ n ( s , ξ [1 ∞ ] , ε s − , χ ∞ ) < v tn ( s , ξ [1 t ] , ε s − , χ t ) + ǫ , (E.3)for every s ∈ S , s − ∈ S n − , and ξ [1 ∞ ] ∈ ( K ( S, X )) ∞ . Therefore, we need merely to selectsuch a large t and show that, when n is large enough, Z S n − ˆ π n − ( ds − ) · v tn ( s , χ t , ε s − , χ t ) ≥ Z S n − ˆ π n − ( ds − ) · v tn ( s , ξ [1 t ] , ε s − , χ t ) − ǫ , (E.4)for every s ∈ S and ξ [1 t ] ∈ ( K ( S, X )) t .Since ( χ, σ ) poses as an equilibrium for Γ, we know (36) is true. Another way to writethe condition is, at t ′ = 0, for any ξ [1 ,t ′ +1] ∈ ( K ( S, X )) t ′ +1 , v ∞ ( s, ( ξ [1 t ′ ] , χ ∞ ) , σ, χ ∞ ) ≥ v ∞ ( s, ( ξ [1 ,t ′ +1] , χ ∞ ) , σ, χ ∞ ) . (E.5)Now suppose (E.5) is true for some t ′ = 0 , , ... . We are to show its validity at t ′ + 1.By (33), (35), and the uniform convergence of v t ( s, ξ [1 t ] , σ, χ t ) to v ∞ ( s, ξ [1 ∞ ] , σ, χ ∞ ), we have v ∞ ( s, ξ [1 ∞ ] , σ, χ ∞ ) = R X ξ ( s | dx ) · [ ˜ f ( s, x, σ ⊗ χ )+ ¯ α · R S ˜ g ( s, x, σ ⊗ χ | ds ′ ) · v ∞ ( s ′ , ξ [2 ∞ ] , σ, χ ∞ )] . (E.6)40herefore, v ∞ ( s, ( ξ [1 ,t ′ +1] , χ ∞ ) , σ, χ ∞ ) − v ∞ ( s, ( ξ [1 ,t ′ +2] , χ ∞ ) , σ, χ ∞ )= R X ξ ( s | dx ) · R S ˜ g ( s, x, σ ⊗ χ | ds ′ ) ×× [ v ∞ ( s ′ , ( ξ [2 ,t ′ +1] , χ ∞ ) , σ, χ ∞ ) − v ∞ ( s ′ , ( ξ [2 ,t ′ +2] , χ ∞ ) , σ, χ ∞ )] , (E.7)which, by the induction hypothesis (E.5), is positive. Therefore, (E.5) is true for t ′ = 0 , , ... .By using (E.5) multiple times, we can derive that, for any ξ [1 t ] ∈ ( K ( S, X )) t , v ∞ ( s, χ ∞ , σ, χ ∞ ) ≥ v ∞ ( s, ( ξ , χ ∞ ) , σ, χ ∞ ) ≥ v ∞ ( s, ( ξ [12] , χ ∞ ) , σ, χ ∞ ) ≥ · · · ≥ v ∞ ( s, ( ξ [1 ,t − , χ ∞ ) , σ t , χ ∞ ) ≥ v ∞ ( s, ( ξ [1 t ] , χ ∞ ) , σ, χ ∞ ) . (E.8)Also, we know from (34) that | v ∞ ( s, ζ [1 ∞ ] , σ, χ ∞ ) − v t ( s, ζ [1 t ] , σ, χ t ) |≤ ¯ α t · ¯ f − ¯ α , (E.9)regardless of the ζ [1 ∞ ] ∈ ( K ( S, X )) ∞ chosen. However, (E.8) and (E.9) would together leadto v t ( s, χ t , σ, χ t ) − v t ( s, ξ [1 t ] , σ, χ t ) ≥ − α t − · ¯ f − ¯ α ≥ − ǫ , (E.10)for any s ∈ S and ξ [1 t ] ∈ ( K ( S, X )) t .In the presence of Assumptions 1 and 2 for the corresponding t -period games, Propo-sition 2 applies. Plus, it has been hypothesized that the sequence ˆ π n − asymptoticallyresembles the sequence σ n − . Therefore, for n large enough, Z S n − ˆ π n − ( ds − ) · v tn ( s , χ t , ε s − , χ t ) > v t ( s , χ t , σ, χ t ) − ǫ , (E.11)regardless of the choice on s ∈ S , and Z S n − ˆ π n − ( ds − ) · v tn ( s , ξ [1 t ] , ε s − , χ t ) < v t ( s , ξ [1 t ] , σ, χ t ) + ǫ , (E.12)regardless of the choices on s ∈ S and ξ [1 t ] ∈ ( K ( S, X )) t . Put (E.10) to (E.12) together,and we would obtain (E.4).For something akin to the second example in Section 7, we need to consider the followinginvariant equation involving π n ∈ P ( S n ), which is inspirable from its finite- t version (28): π n = π n ⊙ χ n ⊙ ˜ g n . (E.13)Suppose (E.13) has a solution that asymptotically resembles σ n , then we can let ˆ π n − = π n | S n − . By Lemma 5, this choice would satisfy the condition in Theorem 4. Its meaning is41lso clear—let players update their estimates on other players’ states most precisely withoutusing their own state information.When the state space S is finite, we again have an extended version much like Theorem 3.If we succeed in finding a satisfactory π n , we would be able to make the third choice ofletting each ˆ π n − ( s |· ) in the extended version be the conditional probability π n,S | S n − ( s |· ).Propositions 3 and 4 would then lead to the satisfaction of the corresponding condition inthe extended version. The third choice here again means that players update other players’states in the most accurate Bayesian fashion.The above second and third choices are premised on the following conjecture. Conjecture 1
Suppose χ ∈ K ( S, X ) , ˜ g ∈ G ( S, X ) enjoys the continuity of ˜ g ( s, x, τ ) in τ at an ( s, x ) -independent rate, and σ ∈ P ( S ) is an solution to the invariant equation σ = σ ⊙ χ ⊙ ˜ g ( · , · , σ ⊗ χ ) as defined by (32) and (35). Then, there would exist a sequence π n so thatfor each n ∈ N , π n as a member of P ( S n ) satisfies the invariant equation π n = π n ⊙ χ n ⊙ ˜ g n as indicated by (E.13), and yet the sequence asymptotically resembles the sequence σ n . To tackle this conjecture, one may be tempted to show that (i) iteratively applying σ t +1 = σ t ⊙ χ ⊙ ˜ g ( · , · , σ t ⊗ χ ) leads to the convergence of σ t to an invariant σ , (ii) iterativelyapplying π n,t +1 = π nt ⊙ χ n ⊙ ˜ g n leads to the convergence of π nt to an invariant π n for each n , and (iii) these convergence results along with the asymptotic resemblance of each π nt to σ nt would lead to that of π n to σ n . So far, (i) and (ii) still elude us. On the other hand,something slightly weaker than (iii) can be achieved. Proposition 5
Let A be a separable metric space, and p i for i ∈ N and p be members of P ( A ) . Also, for each n ∈ N , let q ni for i ∈ N and q n be members of P ( A n ) . Suppose p i converges to p , q ni converges to q n for each n ∈ N , and q ni asymptotically resembles p ni .Then, in either situation (a) where the convergence of q ni to q n is at an n -independent rateor situation (b) where the asymptotic resemblance of q ni to p ni is at an i -independent rate,the sequence q n would asymptotically resemble the sequence p n . Proof of Proposition 5:
Let ǫ > p i converges to p , we have ρ A ( p, p i ) < ǫ , (E.14)as long as i is large enough. 42uppose situation (a) is true. By the equi- n convergence of q ni to q n , we can pick i largeenough to ensure both (E.14) and for any n ∈ N , q n (( A ′ n ) ǫ/ ) > q ni ( A ′ ) − ǫ , ∀ A ′ n ∈ B ( A n ) . (E.15)At such a fixed i ∈ N , due to the asymptotic resemblance of q ni to p ni , we can let n be largeenough so that q ni ( { a ∈ A n | ρ A ( ε a , p i ) < ǫ } ) > − ǫ . (E.16)Suppose situation (b) is true. Due to the equi- i asymptotic resemblance of q ni to p ni , we canpick n large enough to ensure (E.16) for any i ∈ N . By the convergence of q ni to q n , we canthen pick i large enough to ensure (E.14), as well as (E.15) for the current n ∈ N .Either way, without loss of generality, we can suppose d A n ( a, a ′ ) ≥ max nm =1 d A ( a m , a ′ m ).Then, due to Lemma 1,( { a ∈ A n | ρ A ( ε a , p i ) < ǫ } ) ǫ/ ⊆ { a ∈ A n | ρ A ( ε a , p i ) < ǫ } . (E.17)Now we can deduce that q n ( { a ∈ A n | ρ A ( ε a , p ) < ǫ } ) > q n ( { a ∈ A n | ρ A ( ε a , p i ) < ǫ/ } ) > q n (( { a ∈ A n | ρ A ( ε a , p i ) < ǫ/ } ) ǫ/ ) > q ni ( { a ∈ A n | ρ A ( ε a , p i ) < ǫ/ } ) − ǫ/ > − ǫ, (E.18)where the first inequality is due to (E.14), the second inequality is due to (E.17), the thirdinequality is due to (E.15), and the last inequality is due to (E.16). Therefore, the sequence q n asymptotically resembles the sequence p n .Like Propositions 3 and 4, Proposition 5 also helps to bolster the legitimacy of theasymptotic resemblance concept. F Developments in Section 9
F.1 The Transient Case
By the discreteness of S , every χ t ( s | X ′ ) is automatically continuous and hence measurablein s , and hence K ( S, X ) is not only a member of ( P ( X )) S , but also the latter itself. Denote43he space ( P ( S )) ¯ t − by S and the space (( P ( X )) S ) ¯ t = ( K ( S, X )) ¯ t by X . Let U = S × X .Define a correspondence H : U ⇒ U , so that for any σ [2¯ t ] ∈ S and χ [1¯ t ] ∈ X , H ( σ [2¯ t ] , χ [1¯ t ] ) = H S ( σ [2¯ t ] , χ [1¯ t ] ) × H X ( σ [2¯ t ] , χ [1¯ t ] ) , (F.1)where H S ( σ [2¯ t ] , χ [1¯ t ] ) = { σ ′ [2¯ t ] ∈ S| σ ′ t = T t − ( χ t − ) ◦ σ t − , ∀ t = 2 , , ..., ¯ t } , (F.2)and H X ( σ [2¯ t ] , χ [1¯ t ] ) = { χ ′ [1¯ t ] ∈ X | χ ′ t ( s t | ˜ X t ( s t , σ t , χ [ t ¯ t ] )) = 1 , ∀ t = 1 , , ..., ¯ t, s t ∈ S } . (F.3)A fixed point ( σ [2¯ t ] , χ [1¯ t ] ) for H would provide a Markov equilibrium χ [1¯ t ] for Γ( σ ) in thesense of (42), with σ [2¯ t ] supplying the deterministic pre-action environment pathway fromperiod 2 to ¯ t that is generated from all players adopting policy χ [1¯ t ] . We are to use Kakutani-Fan-Glicksberg fixed point theorem to prove the existence of a fixed point for H . But firstlet us work out a couple of useful continuity results. Proposition 6 (i) σ ⊗ χ is continuous in both σ ∈ P ( S ) and χ ∈ ( P ( X )) S .When g ∈ G ( S, X ) satisfies that g ( s, x, τ ) is continuous in τ at an ( s, x ) -independent rate,(ii) σ ⊙ χ ⊙ g ( · , · , σ ⊗ χ ) is continuous in both σ ∈ P ( S ) and χ ∈ ( P ( X )) S . Proof of Proposition 6:
We first prove (i) by showing that, for any two sequences σ m and χ m that converge to σ and χ , respectively, the sequence σ m ⊗ χ m would converge to σ ⊗ χ . Inthe following, we omit detailed reasonings behind some of the steps, as they have appearedin the proof of Proposition 1.Fix some ǫ ∈ (0 , I of its points ¯ s , ¯ s , ..., ¯ s I , so that (B.1) istrue. For convenience, let ¯ S ′ = { ¯ s , ¯ s , ..., ¯ s I } and ¯ S ′′ = S \ ¯ S ′ . It is known that the distance d S ( ¯ S ′ , ¯ S ′′ ) = inf s ′ ∈ ¯ S ′ ,s ′′ ∈ ¯ S ′′ d S ( s ′ , s ′′ ) >
0. For i, j = 1 , , ..., I , use d ij for d S (¯ s i , ¯ s j ) and σ i for σ ( { ¯ s i } ). Again, define δ through (B.2), whose strict positivity is guaranteed.As σ m and χ m converge to σ and χ , respectively, for large enough m , we have ρ S ( σ, σ m ) < δ, (F.4)and ρ X ( χ (¯ s i ) , χ m (¯ s i )) < δ, ∀ i = 1 , , ..., I. (F.5)Together with the fact that δ ≤ d S ( ¯ S ′ , ¯ S ′′ ) ∧ (min i = j d ij ), (F.4) would result with σ i − δ < σ m ( { ¯ s i } ) < σ i + δ. (F.6)44eanwhile, (F.5) would lead to χ m (¯ s i | X ′ ) < χ (¯ s i | ( X ′ ) δ ) + δ, ∀ i = 1 , , ..., I. (F.7)Any U ′ ∈ B ( S × X ) still enjoys the decomposition provided in (B.12), that U ′ =( S Ii =1 { ¯ s i } × X ′ i ) S U ′′ , where X ′ i ∈ B ( X ) for i = 1 , , ..., I , while U ′′ is such that s ′′ ∈ ¯ S ′′ forany ( s ′′ , x ′′ ) ∈ U ′′ . This would result in the same (B.13). On the other hand, from the righthalf of (F.6) and (F.7),( σ m ⊗ χ m )( { ¯ s i } × X ′ i ) = σ m ( { ¯ s i } ) · χ m (¯ s i | X ′ i ) < ( σ i + δ ) · [ χ (¯ s i | ( X ′ i ) δ ) + δ ] ≤ ( σ ⊗ χ )( { ¯ s i } × ( X ′ i ) δ ) + 2 δ + δ < ( σ ⊗ χ )( { ¯ s i } × ( X ′ i ) δ ) + 3 δ, (F.8)where the last inequality is due to our choice that δ ≤ ǫ/I <
1. Meanwhile,( σ m ⊗ χ m )( U ′′ ) ≤ ( σ m ⊗ χ m )( ¯ S ′′ × X ) = σ m ( ¯ S ′′ ) = 1 − P Ii =1 σ m ( { ¯ s i } ) < − P Ii =1 σ i + Iδ < ǫ + Iδ, (F.9)where the second-to-last inequality is due to the left half of (F.6) and the last one is dueto (B.1). By combining (B.12), (B.13), (F.8), and (F.9), we can obtain( σ m ⊗ χ m )( U ′ ) < ( σ ⊗ χ )(( U ′ ) δ ) + ǫ + 4 Iδ. (F.10)Thus, ρ S × X ( σ m ⊗ χ m , σ ⊗ χ ) < ǫ + 4 Iδ ≤ ǫ. (F.11)Since (F.11) is to occur at any m that is large enough, we see that (i) is true.We then prove (ii). Again, suppose two sequences σ m and χ m converge to σ and χ ,respectively. From (i), we know σ m ⊗ χ m converges to σ ⊗ χ too. According to (87) of Yang[33], for any m , ρ X ( σ m ⊙ χ m , σ ⊙ χ ) = ρ X (( σ m ⊗ χ m ) | X , ( σ ⊗ χ ) | X ) ≤ ρ S × X ( σ m ⊗ χ m , σ ⊗ χ ) . (F.12)Hence, there is also the convergence of σ m ⊙ χ m to σ ⊙ χ .On the other hand, the discrete property of S × X means g ( · , · , τ ) is a member of( P ( S )) S × X for any fixed τ ∈ P ( S × X ). Now (i) and the fact that g ( s, x, τ ) is continuous in τ at an ( s, x )-independent rate would together mean that, the sequence g ( · , · , σ m ⊗ χ m ) in( P ( S )) S × X converges to g ( · , · , σ ⊗ χ ).Let us use the convergence of σ m ⊙ χ m to σ ⊙ χ under proper substitutions. As S × X has been noted to be discrete, we can treat it as S in the convergence result. Also, let ustreat σ m ⊗ χ m as σ m , σ ⊗ χ as σ , S as X , g ( · , · , σ m ⊗ χ m ) as χ m , and g ( · , · , σ ⊗ χ ) as χ .45rom (i) on the convergence of σ m ⊗ χ m to σ ⊗ χ , now viewed as that of σ m to σ , as wellas the convergence of g ( · , · , σ m ⊗ χ m ) to g ( · , · , σ ⊗ χ ), now viewed as that of χ m to χ , we canconclude that ( σ m ⊗ χ m ) ⊙ g ( · , · , σ m ⊗ χ m ) = σ m ⊙ χ m ⊙ g ( · , · , σ m ⊗ χ m ) would converge to( σ ⊗ χ ) ⊙ g ( · , · , σ ⊗ χ ) = σ ⊙ χ ⊙ g ( · , · , σ ⊗ χ ). Thus, (ii) is true as well. Proposition 7
For each t = 1 , , ..., ¯ t + 1 , the value v t ( s t , ξ [ t ¯ t ] , σ t , χ [ t ¯ t ] ) defined in (22) iscontinuous in σ t ∈ S and χ [ t ¯ t ] ∈ X at an ( s t , ξ [ t ¯ t ] ) -independent rate. Proof of Proposition 7:
We use induction on t . By (21), our claim is certainly true for t = ¯ t + 1. Suppose for some t = ¯ t, ¯ t − , ...,
1, the function v t +1 ( s t +1 , ξ [ t +1 , ¯ t ] , σ t +1 , χ [ t +1 , ¯ t ] ) iscontinuous in σ t +1 and χ [ t +1 , ¯ t ] at a rate independent of s t +1 and ξ [ t +1 , ¯ t ] .Now we prove the continuity in σ t and χ [ t ¯ t ] at time t . From (22), we havesup s t ∈ S,ξ [ t ¯ t ] ∈ (( P ( X )) S ) ¯ t − t +1 | v t ( s t , ξ [ t ¯ t ] , σ t , χ [ t ¯ t ] ) − v t ( s t , ξ [ t ¯ t ] , σ ′ t , χ ′ [ t ¯ t ] ) |≤ M + M + M , (F.13)where M = sup ( s t ,x t ) ∈ S × X | ˜ f t ( s t , x t , σ t ⊗ χ t ) − ˜ f t ( s t , x t , σ ′ t ⊗ χ ′ t ) | , (F.14) M = sup ( s t ,x t ) ∈ S × X, ξ [ t +1 , ¯ t ] ∈ (( P ( X )) S ) ¯ t − t | [ R S ˜ g t ( s t , x t , σ t ⊗ χ t | ds t +1 ) − R S ˜ g t ( s t , x t , σ ′ t ⊗ χ ′ t | ds t +1 )] · v t +1 ( s t +1 , ξ [ t +1 , ¯ t ] , T t ( χ t ) ◦ σ t , χ [ t +1 , ¯ t ] ) | , (F.15)and M = sup ( s t ,x t ) ∈ S × X, ξ [ t +1 , ¯ t ] ∈ (( P ( X )) S ) ¯ t − t R S ˜ g t ( s t , x t , σ ′ t ⊗ χ ′ t | ds t +1 ) ×× | v t +1 ( s t +1 , ξ [ t +1 , ¯ t ] , T t ( χ t ) ◦ σ t , χ [ t +1 , ¯ t ] ) − v t +1 ( s t +1 , ξ [ t +1 , ¯ t ] , T t ( χ ′ t ) ◦ σ ′ t , χ ′ [ t +1 , ¯ t ] ) | . (F.16)By part (i) of Proposition 6, σ ′ t ⊗ χ ′ t can be made arbitrarily close to σ t ⊗ χ t by letting( σ ′ t , χ ′ t ) be close enough to ( σ t , χ t ). Then due to Assumption 2, M can be made arbitrarilysmall by doing the same.Again, suppose S = { ¯ s , ¯ s , ... } . We use the simplified notation that γ ( ′ ) i ( s t , x t ) = g t ( s t , x t , σ ( ′ ) t ⊗ χ ( ′ ) t |{ ¯ s i } ) , (F.17)and v i ( ξ [ t +1 , ¯ t ] ) = v t +1 (¯ s i , ξ [ t +1 , ¯ t ] , T t ( χ t ) ◦ σ t , χ [ t +1 , ¯ t ] ) . (F.18)Then, (F.15) can be expressed as M equalingsup ( s t ,x t ) ∈ S × X, ξ [ t +1 , ¯ t ] ∈ (( P ( X )) S ) ¯ t − t | X i γ i ( s t , x t ) · v i ( ξ [ t +1 , ¯ t ] ) − X i γ ′ i ( s t , x t ) · v i ( ξ [ t +1 , ¯ t ] ) | . (F.19)46et I ( s t , x t ) be the set of i ’s that induce γ i ( s t , x t ) ≥ γ ′ i ( s t , x t ). Note the | v i ( ξ [ t +1 , ¯ t ] ) | ’s arebounded, say by v , due to the boundedness of the ˜ f t ′ ’s and the finiteness of ¯ t . Then, (F.19)would lead to M ≤ v · sup ( s t ,x t ) ∈ S × X X i ∈ I ( s t ,x t ) ( γ i ( s t , x t ) − γ ′ i ( s t , x t )) . (F.20)For δ below inf s = s ′ d S ( s, s ′ ), the event ρ S (˜ g t ( s t , x t , σ t ⊗ χ t ) , ˜ g t ( s t , x t , σ ′ t ⊗ χ ′ t )) < δ would trigger X i ∈ I ( s t ,x t ) ( γ i ( s t , x t ) − γ ′ i ( s t , x t )) < δ, (F.21)for every ( s t , x t ) ∈ S × X ; consult (C.18) in the proof of Proposition 2. But due to As-sumption 1, the convergence of σ ′ t ⊗ χ ′ t to σ t ⊗ χ t means that we can make ˜ g t ( s t , x t , σ ′ t ⊗ χ ′ t )arbitrarily close to ˜ g t ( s t , x t , σ t ⊗ χ t ), at a rate that is independent of ( s t , x t ). Hence, by (F.20), M can be made arbitrarily small by letting ( σ ′ t , χ ′ t ) get close enough to ( σ t , χ t ).From (F.16), we can get M ≤ sup s t +1 ∈ S, ξ [ t +1 , ¯ t ] ∈ (( P ( X )) S ) ¯ t − t | v t +1 ( s t +1 , ξ [ t +1 , ¯ t ] , T t ( χ t ) ◦ σ t , χ [ t +1 , ¯ t ] ) − v t +1 ( s t +1 , ξ [ t +1 , ¯ t ] , T t ( χ ′ t ) ◦ σ ′ t , χ ′ [ t +1 , ¯ t ] ) | . (F.22)By part (ii) of Proposition 6, T t ( χ ′ t ) ◦ σ ′ t = σ ′ t ⊙ χ ′ t ⊙ ˜ g t ( · , · , σ ′ t ⊗ χ ′ t ) can be made arbitrarilyclose to T t ( χ t ) ◦ σ t = σ t ⊙ χ t ⊙ ˜ g t ( · , · , σ t ⊗ χ t ) by letting ( σ ′ t , χ ′ t ) be close enough to ( σ t , χ t ).By the induction hypothesis, M can be made arbitrarily small by doing the same.We have thus completed the induction process.Here comes the conditional-equilibrium existence result for the transient case. Theorem 5
The correspondence H allows for a fixed point ( σ [2¯ t ] , χ [1¯ t ] ) , which supplies thegame Γ( σ ) with an conditional equilibrium χ [1¯ t ] . Proof of Theorem 5:
Due to S ’s discreteness, P ( S ) is the simplex in R | S | , whether | S | be finite or infinite, and hence is compact; the same applies to P ( X ). Thus, U is a compactsubset of the vector space R | S | ¯ t − + | X | | S |· ¯ t , understood as R ∞ if either S or X is infinite.For any finite-dimensional R k , we can take the norm || · || so that || r || = P kl =1 | r l | /k for each r = ( r l | l = 1 , ..., k ) ∈ R k , whereas for the infinite-dimensional R ∞ , we can let || r || = P + ∞ l =1 | r l | / l for each r = ( r l | l = 1 , , ... ) ∈ R ∞ . A norm thus defined wouldprovide the same convergence as does the weak convergence under Prohorov metric. Sincethe convex combination of two probabilities is still a probability, U is also convex.47or any ( σ [2¯ t ] , χ [1¯ t ] ) ∈ U , the set H ( σ [2¯ t ] , χ [1¯ t ] ) is certainly non-empty, for we can constructsome ( σ ′ [2¯ t ] , χ ′ [1¯ t ] ) belonging to it. First, for t = 2 , , ..., ¯ t , we simply let σ ′ t = T t − ( χ t − ) ◦ σ t − .Then, for t = 1 , , ..., ¯ t and s ∈ S , let χ ′ t ( s ) be any measure that assigns its full weight to theset of x ’s that attain the maximum value sup y ∈ X v t ( s, ( δ y , χ [ t +1 , ¯ t ] ) , σ t , χ [ t ¯ t ] ).Now we show that H S : U ⇒ S and H X : U ⇒ X are closed- and convex-valued, aswell as upper hemi-continuous. These would lead to the same properties for H . Accordingto (F.2), each H S ( σ [2¯ t ] , χ [1¯ t ] ) contains exactly one point, and hence is automatically closedand convex. For the upper hemi-continuity property, we need only to show that the valuecontained in H S ( σ [2¯ t ] , χ [1¯ t ] ) moves continuously with both σ [2¯ t ] and χ [1¯ t ] . But this has beenguaranteed by part (ii) of Proposition 6.According to (F.3), each H X ( σ [2¯ t ] , χ [1¯ t ] ) is a set of probability vectors, with each compo-nent probability assigning the full measure to a particular measurable set. This set of prob-ability vectors is certainly convex. To show that it is closed, suppose χ ′ m, [1¯ t ] for m = 1 , , ... form a sequence in H X ( σ [2¯ t ] , χ [1¯ t ] ) that converges to a given χ ′ [1¯ t ] . We are to show that χ ′ [1¯ t ] ∈ H X ( σ [2¯ t ] , χ [1¯ t ] ) . (F.23)Now for any t = 1 , , ..., ¯ t , s ∈ S , and ǫ >
0, as long as m is large enough, χ ′ t ( s | ( ˜ X t ( s, σ [2¯ t ] , χ [1¯ t ] )) ǫ ) ≥ χ ′ mt ( s | ˜ X t ( s, σ [2¯ t ] , χ [1¯ t ] )) − ǫ = 1 − ǫ. (F.24)Due to the arbitrariness of ǫ , this means χ ′ t ( s | ˜ X t ( s, σ [2¯ t ] , χ [1¯ t ] )) = 1, and hence (F.23) is true.We now show that H X is upper hemi-continuous. Let σ m, [2¯ t ] be a sequence in S thatconverges to a given σ [2¯ t ] , χ m, [1¯ t ] a sequence in X that converges to a given χ [1¯ t ] , and χ ′ m, [1¯ t ] another sequence in X that converges to a given χ ′ [1¯ t ] . Suppose for each m = 1 , , ... , χ ′ m, [1¯ t ] ∈ H X ( σ m, [2¯ t ] , χ m, [1¯ t ] ) , (F.25)we are to show that χ ′ [1¯ t ] ∈ H X ( σ [2¯ t ] , χ [1¯ t ] ) . (F.26)By (F.3), we see that (F.25) for each m indicates that, for each t = 1 , , ..., ¯ t and s ∈ S , χ ′ mt ( s | ˜ X t ( s, σ mt , χ m, [ t ¯ t ] )) = 1; (F.27)whereas, (F.26) boils down to that, for each t = 1 , , ..., ¯ t and s ∈ S , χ ′ t ( s | ˜ X t ( s, σ t , χ [ t ¯ t ] )) = 1 . (F.28)48e fix some t and s . Let ǫ > X ′ ) ǫ and X ′ for any X ′ ⊆ X . Now since χ ′ mt converges to χ ′ t , for m large enough, χ ′ t ( s | ˜ X t ( s, σ t , χ [ t ¯ t ] )) = χ ′ t ( s | ( ˜ X t ( s, σ t , χ [ t ¯ t ] )) ǫ ) ≥ χ ′ mt ( s | ˜ X t ( s, σ t , χ [ t ¯ t ] )) − ǫ. (F.29)For the time being, suppose ˜ X t ( s, σ t , χ [ t ¯ t ] ) is known to be upper hemi-continuous in ( σ t , χ [ t ¯ t ] ).By also noting the hypothesis on the convergence of σ mt to σ t and that of χ m, [ t ¯ t ] to χ [ t ¯ t ] , wecan obtain, for m large enough,˜ X t ( s, σ mt , χ m, [ t ¯ t ] ) ⊆ ( ˜ X t ( s, σ t , χ [ t ¯ t ] )) ǫ = ˜ X t ( s, σ t , χ [ t ¯ t ] ) . (F.30)Thus, for m large enough, χ ′ t ( s | ˜ X t ( s, σ t , χ [ t ¯ t ] )) ≥ χ ′ mt ( s | ˜ X t ( s, σ t , χ [ t ¯ t ] )) − ǫ ≥ χ ′ mt ( s | ˜ X t ( s, σ mt , χ m, [ t ¯ t ] )) − ǫ, (F.31)which, according to (F.27), is above 1 − ǫ . In view of the arbitrariness of ǫ , we canachieve (F.28).We now come back to the upper hemi-continuity of ˜ X t ( s, · ) as a correspondence from P ( S ) × (( P ( X )) S ) ¯ t − t +1 to X . Suppose σ mt converges to σ t , χ m, [ t ¯ t ] converges to χ [ t ¯ t ] , and x m converges to x . For every m = 1 , , ... , suppose x m ∈ ˜ X t ( s, σ mt , χ m, [ t ¯ t ] ), which, by (43),means v t ( s, ( δ x m , χ m, [ t +1 , ¯ t ] ) , σ mt , χ m, [ t ¯ t ] ) ≥ v t ( s, ( δ y , χ m, [ t +1 , ¯ t ] ) , σ mt , χ m, [ t ¯ t ] ) , ∀ y ∈ X. (F.32)By X ’s discreteness, x m would be x for sufficiently large m . This, combined with Proposi-tion 7 and the hypothesis on the convergence of σ mt to σ t and that of χ m, [ t ¯ t ] to χ [ t ¯ t ] , wouldentail that, for any ǫ >
0, as long as m is large enough, v t ( s, ( δ x , χ [ t +1 , ¯ t ] ) , σ t , χ [ t ¯ t ] ) ≥ v t ( s, ( δ x , χ m, [ t +1 , ¯ t ] ) , σ mt , χ m, [ t ¯ t ] ) − ǫ = v t ( s, ( δ x m , χ m, [ t +1 , ¯ t ] ) , σ mt , χ m, [ t ¯ t ] ) − ǫ ≥ v t ( s, ( δ y , χ m, [ t +1 , ¯ t ] ) , σ mt , χ m, [ t ¯ t ] ) − ǫ ≥ v t ( s, ( δ y , χ [ t +1 , ¯ t ] ) , σ t , χ [ t ¯ t ] ) − ǫ, (F.33)for any y ∈ X . Since ǫ can be arbitrarily small, we see from (43) that x ∈ ˜ X t ( s, σ t , χ [ t ¯ t ] ).Thus we have the upper hemi-continuity of ˜ X t ( s, · ).In summary, H is a non-empty, closed- and convex-valued, as well as upper hemi-continuous correspondence on the compact and convex subset U that is embedded in anormed linear topological space. We can therefore apply the Kakutani-Fan-Glicksberg fixedpoint theorem to verify that H has a fixed point.49 .2 The Stationary Case Denote the space P ( S ) by S ∞ and the space ( P ( X )) S by X ∞ . Let U ∞ = S ∞ × X ∞ . Definea correspondence H ∞ : U ∞ ⇒ U ∞ , so that for any σ ∈ S ∞ and χ ∈ X ∞ , H ∞ ( σ, χ ) = H S ∞ ( σ, χ ) × H X ∞ ( σ, χ ) , (F.34)where H S ∞ ( σ, χ ) = { σ ′ ∈ S ∞ | σ ′ = T ( χ ) ◦ σ } , (F.35)and H X ∞ ( σ, χ ) = { χ ′ ∈ X ∞ | χ ′ ( s | ˜ X ∞ ( s, σ, χ )) = 1 , ∀ s ∈ S } . (F.36)A fixed point ( σ, χ ) for H ∞ would provide a stationary Markov equilibrium χ for thestationary nonatomic game Γ ∞ in the sense of (44), with σ supplying the invariant deter-ministic environment that is generated from all players adopting policy χ . To show thatsuch an equilibrium exists, we first need the following consequence of Proposition 7. Proposition 8
The value v ∞ ( s, ξ [1 ∞ ] , σ, χ ∞ ) defined in (33) is continuous in σ ∈ S ∞ and χ ∈ X ∞ at an ( s, ξ [1 ∞ ] ) -independent rate. Proof of Proposition 8:
From (34), we see that | v ∞ ( s, ξ [1 ∞ ] , σ, χ ∞ ) − v t ( s, ξ [1 t ] , σ, χ t ) |≤ ¯ α t · f − ¯ α . (F.37)Thus, for any ǫ >
0, by fixing at a large enough t , we can ensure | v ∞ ( s, ξ [1 ∞ ] , σ ′′ , ( χ ′′ ) ∞ ) − v t ( s, ξ [1 t ] , σ ′′ , ( χ ′′ ) t ) | < ǫ , (F.38)for any s , ξ [1 ∞ ] , σ ′′ , and χ ′′ . At the same time, Proposition 7 means that, for ( σ ′ , χ ′ ) closeenough to any given ( σ, χ ), we can guarantee | v t ( s, ξ [1 t ] , σ, χ t ) − v t ( s, ξ [1 t ] , σ ′ , ( χ ′ ) t ) | < ǫ , (F.39)for any s and ξ [1 t ] . Then, | v ∞ ( s, ξ [1 ∞ ] , σ, χ ∞ ) − v ∞ ( s, ξ [1 ∞ ] , σ ′ , ( χ ′ ) ∞ ) |≤| v ∞ ( s, ξ [1 ∞ ] , σ, χ ∞ ) − v t ( s, ξ [1 t ] , σ, χ t ) | + | v t ( s, ξ [1 t ] , σ, χ t ) − v t ( s, ξ [1 t ] , σ ′ , ( χ ′ ) t ) | + | v t ( s, ξ [1 t ] , σ ′ , ( χ ′ ) t ) − v ∞ ( s, ξ [1 ∞ ] , σ ′ , ( χ ′ ) ∞ ) | < ǫ. (F.40)50hus, v ∞ ( s, ξ [1 ∞ ] , σ, χ ∞ ) is continuous in ( σ, χ ) at an ( s, ξ [1 ∞ ] )-independent rate.We can then have the desired conditional-equilibrium existence result by using the Kakutani-Fan-Glicksberg fixed point theorem. Theorem 6
The correspondence H ∞ allows for a fixed point ( σ, χ ) , which supplies the game Γ ∞ with an equilibrium χ . Proof of Theorem 6:
Due to the discreteness of S and X , U ∞ is a compact subset of thevector space R | S | + | X | | S | , understood as R ∞ if either S or X is infinite. Regardless of whetherthe space is finite- or infinite-dimensional, we can take the norm adopted in the proof ofTheorem 5. Since the convex combination of two probabilities is still a probability, U ∞ isconvex.Using virtually the same corresponding arguments in the proof of Theorem 5, we canshow that H ∞ ( σ, χ ) at any ( σ, χ ) ∈ U ∞ is non-empty, closed, and convex. We separate theupper hemi-continuity of H ∞ into that for H S ∞ and that for H X ∞ .The upper hemi-continuity of H S ∞ again comes from Proposition 6. Furthermore, we canuse almost the same arguments from (F.32) to (F.33), this time relying on Proposition 8instead of Proposition 7, to show that ˜ X ∞ ( s, · ) as a correspondence from P ( S ) × ( P ( X )) S to X is upper hemi-continuous. Then, using almost the same arguments from (F.25) to (F.31),we can verify that H X ∞ is upper hemi-continuous.With all these properties, we can apply the Kakutani-Fan-Glicksberg fixed point theoremto verify that H ∞ has a fixed point. References [1] Adlakha, S. and B. Johari. 2013. Mean Field Equilibrium in Dynamic Games withComplementarities.
Operations Research , , pp. 971-989.[2] Al-Najjar, N.I. and R. Smorodinsky. 2001. Large Nonanonymous Repeated Games. Games and Economic Behavior , , pp. 26-39.[3] Al-Najjar, N.I. 2008. Large Games and the Law of Large Numbers. Games and EconomicBehavior , , pp. 1-34. 514] Balder, E.J. 2002. A Unifying Pair of Cournot-Nash Equilibrium Existence Results. Journal of Economic Theory , , pp. 437-470.[5] Balder, E.J. 2008. Comments on Purification in Continuous Games. International Jour-nal of Game Theory , , pp. 73-92.[6] Bergin, J. and D. Bernhardt. 1995. Anonymous Sequential Games: Existence and Char-acterization of Equilibria. Economic Theory , , pp. 461-489.[7] Bodoh-Creed, A.L. 2012. Approximation of Large Dynamic Games. Working Paper,Cornell University.[8] Carmona, G. 2004. Nash Equilibria of Games with a Continuum of Players. WorkingPaper, Universidade Nova de Lisboa.[9] Duffie, D., J. Geanakoplos, A. Mas-Colell, and A. McLennan. 1994. Stationary MarkovEquilibria. Econometrica , , pp. 745-781.[10] Dvoretzky, A., J. Kiefer, and J. Wolfolwitz. 1956. Asymptotic Minimax Character ofthe Sample Distribution Function and of the Classical Multinomial Estimator. Annalsof Mathematical Statistics , , pp. 642-669.[11] Ethier, S.N. and T.G. Kurtz. 1986. Markov Processes: Characterization and Conver-gence . John Wiley & Sons, New York.[12] Green, E.J. 1980. Non-cooperative Price Taking in Large Dynamic Markets.
Journal ofEconomic Theory . , pp. 155-182.[13] Green, E.J. 1984. Continuum and Finite-player Noncooperative Models of Competition. Econometrica , , pp. 975-993.[14] Hopenhayn, H.A. 1992. Entry, Exit, and Firm Dynamics in Long Run Equilibrium. Econometrica , , pp. 1127-1150.[15] Housman, D. 1988. Infinite Player Noncooperative Games and the Continuity of theNash Equilibrium Correspondence. Mathematics of Operations Research , , pp. 488-496.[16] Jovanovic, B. and R.W. Rosenthal. 1988. Anonymous Sequential Games. Journal ofMathematical Economics , , pp. 77-88.5217] Kalai, E. 2004. Large Robust Games. Econometrica , , pp. 1631-1665.[18] Khan, M.A., K.P. Rath, and Y.N. Sun. 1997. On the Existence of Pure Strategy Equi-librium in Games with a Continuum of Players. Journal of Economic Theory , , pp.13-46.[19] Khan, M.A., K.P. Rath, Y.N. Sun, and H. Yu. 2013. Large Games with a Bio-socialTypology. Journal of Economic Theory , , 1122-1149.[20] Khan, M.A. and Y.N. Sun. 1990. On a Reformulation of Cournot-Nash Equilibria. Journal of Mathematical Analysis and Applications , , pp. 442-460.[21] Khan, M.A. and Y.N. Sun. 1995. Pure Strategies in Games with Private Information. Journal of Mathematical Economics , , pp. 633-653.[22] Khan, M.A. and Y.N. Sun. 1999. Non-cooperative Games on Hyperfinite Loeb Space. Journal of Mathematical Economics , , pp. 455-492.[23] Khan, M.A. and Y.N. Sun. 2002. Non-cooperative Games with Many Players. In R.J.Aumann and S. Hart (eds.), Handbook of Game Theory with Economic Applications,Volume 3 , Elsevier Science, Amsterdam, pp. 1761-1808.[24] Mas-Colell, A. 1984. On a Theorem of Schmeidler.
Journal of Mathematical Economics , , pp. 201-206.[25] Mertens, J.F. and T. Parthasarathy. 1987. Equalibria for Discounted Stochastic Games.CORE Discussion Paper No. 8750.[26] Parthasarathy, K.R. 2005. Probability Measures on Metric Spaces . AMS Chelsea Pub-lishing, Providence, Rhode Island.[27] Sabourian, H. 1990. Anonymous Repeated Games with a Large Number of Players andRandom Outcomes.
Journal of Economic Theory , , pp. 92-110.[28] Schmeidler, D. 1973. Equilibrium Points of Nonatomic Games. Journal of StatisticalPhysics , , pp. 295-300.[29] Shapley, L.S. 1953. Stochastic Games. Proceedings of the National Academy of Sciences , , pp. 1095-1100. 5330] Solan, E. 1998. Discounted Stochastic Games. Mathematics of Operations Research , ,pp. 1010-1021.[31] Weintraub, G.Y., C. L. Benkard, and B. van Roy. 2008. Markov Perfect Industry Dy-namics with Many Firms. Econometrica , , pp. 1375-1411.[32] Weintraub, G.Y., C. L. Benkard, and B. van Roy. 2011. Industry Dynamics: Foundationsfor Models with an Infinite Number of Firms. Journal of Economic Theory , , pp.504-527.[33] Yang, J. 2011. Asymptotic Interpretations for Equilibria of Nonatomic Games. Journalof Mathematical Economics , , pp. 491-499.[34] Yang, J. 2015. Analysis of Markovian Competitive Situations using Nonatomic Games—the Shock-driven Case. Working Paper, Rutgers University.[35] Yang, J. and Y. Xia. 2013. A Nonatomic-game Approach to Dynamic Pricing underCompetition. Production and Operations Management , , pp. 88-103.[36] Yu, H. and Z. Zhang. 2007. Pure Strategy Equilibria in Games with Countable Actions. Journal of Mathematical Econonomics ,43