Equilibria in multi-player multi-outcome infinite sequential games
aa r X i v : . [ c s . L O ] M a r Equilibria in multi-player multi-outcome infinitesequential games
St´ephane Le Roux
D´epartement d’informatiqueUniversit´e libre de Bruxelles ∗ , Belgium [email protected]
Arno Pauly
D´epartement d’informatiqueUniversit´e libre de Bruxelles, Belgium&Computer LaboratoryUniversity of Cambridge, United Kingdom
We investigate the existence of certain types of equilibria (Nash, ε -Nash, subgame perfect, ε -subgame perfect, Pareto-optimal) in multi-player multi-outcome infinite sequential games.We use two fundamental approaches: one requires strong topological restrictions on thegames, but produces very strong existence results. The other merely requires some verybasic determinacy properties to still obtain some existence results. Both results are transferresults: starting with the existence of some equilibria for a small class of games, they allowus to conclude the existence of some type of equilibria for a larger class.To make the abstract results more concrete, we investigate as a special case infinitesequential games with real-valued payoff functions. Depending on the class of payoff functions(continuous, upper semi-continuous, Borel) and whether the game is zero-sum, we obtainvarious existence results for equilibria.Our results hold for games with two or up to countably many players. The present article continues the research programme to investigate sequential games in a verygeneral setting, which was initiated by the first author in [18, 19]. It extends the conferencepaper [21]. This programme reunites two mostly separate developments in the study of games.On the one hand, the first development is the investigation of variations on solution conceptsfor games, and of different formalizations of the preferences of the players, which primarilyhappened inside game theory proper. Related to that, game theory has also seen an interestin relaxing the continuity and convexity assumptions of
Nash ’s original existence theorem [28].Both for an example and references to further work see [29]. So typically, the countably infiniteis absent from game theory: sets are either finite, or in cases such as randomized strategies, havethe structure of the continuum.On the other hand, the study of infinite sequential games has a long history in logic. Manyvariations on the rules of games have been studied, albeit mostly restricted to zero-sum gameswith two players and two outcomes. Thus, here the continuum is entirely absent (disregardingits internal occurrence in the set of potential plays), and the countably infinite is only used fortime, not for e.g. the number of agents or outcomes.In our work we study infinite sequential games with perfect information (i.e. generalized Gale-Stewart games [10]) in a setting as general as possible. We can have any countable number of ∗ The project was begun while the first author was a postdoctoral researcher at the University of Darmstadt,Germany.
Multi-player multi-outcome infinite sequential games players, and we investigate various ways to represent preferences. Some results do put restrictionson the number of distinguished outcomes (as being countable), and some require a (generalized)zero-sum condition. A similar synthesis of the approaches is found in [41, 11, 8, 35].A classical result by
Martin [25] established that such a game played by two players whoonly care about whether or not the play falls into some fixed Borel set is determined, i.e. admitsa winning strategy for one of the players. While determinacy can be understood as a special caseof existence of Nash equilibrium, Nash equilibrium is often regarded as an unsatisfactory solutionconcept for sequential games in game theory. Subgame perfect equilibria are a more convincingsolution concept from a rationality perspective. This article studies these two concepts, and asbest responses are not always available, we also investigate the existence of ε -Nash equilibriaand ε -subgame perfect equilibria. Furthermore, we also study Pareto-optimal equilibria.The proofs that we provide fall into two broad categories: some of our results are obtainedby lifting Borel determinacy to more complicated settings, similar to [18, 19] or to a sketchedobservation by Mertens and
Neyman in [27]. Other results are based on topological argumentsto show strong existence results, albeit at the cost of continuity requirements in the gamecharacterizations. Both proof techniques provide general results in rather abstract settings. Inthis sense, our main results are Theorem 7 on the one hand, and Theorems 22, 23, 25 (whichshare parts of their proofs) on the other hand.A common theme of the results is that they are transfer principles: they tell us how to takea pre-existing result on a more restricted class of games and obtain from it a result for a moregeneral class of games. Theorem 7 is then used to transfer the existence of subgame perfectequilibria from finite sequential games to certain infinite ones. With Theorems 22, 23, 25, wecan in particular extend Borel determinacy to yield equilibria in multi-player multi-outcomesettings.In an attempt to make the rather abstract results somewhat more accessible, we shall con-sider in addition the corollaries obtained in the situation where the goals of the players areto maximize real-valued payoff functions. Distinguishing properties here are continuity, up-per or lower semicontinuity and Borel measurability. These settings have been studied before[9, 27, 35, 8], and usually our corollaries improve upon known results by extending them fromthe case of finitely many players to the case of countably many players. An overview of pastand new results is given in the table on Page 3.Our results showcase which requirements are actually needed for which aspects of determi-nacy, and as such may contribute to the understanding of strategic behaviour in general.Additionally, the emergence of quantitative objectives in addition to qualitative structure intraditional verification/synthesis games [2], such as mean-payoff parity games [4, 3], provides anarea of applications for abstract theorems about the existence of equilibria. Existence results insuch settings are usually not trivial, but are proven together with the introduction of the setting– thus we do not answer open questions, but are hopeful that our results may be useful in thefuture. Having results for countably many players is important for applications if multi-agentinteractions in open systems are studied. In order to employ an equilibrium existence result forfinitely many players, a bound on the number of agents involved in the interaction might needto be common knowledge from the beginning on. Our results on the other hand easily enable asetting where additional agents may join the interaction later on, and only the number of agentswho have acted in the past is finite.This article is an extended and improved version of [21]. . Le Roux & A. Pauly Overview
Payoff functions Type ε -Nash Nash ε -subgame perfect subgame perfectcontinuous finitely many yes yes yes yesplayers - - - [9, Corollary 4.2]continuous countably many yes yes yes yesplayers - - - Corollary 9upper finitely many yes yes yes nosemi-continuous players - Corollary 27 [35, Theorem 2.1] Example 35upper countably many yes yes ? nosemi-continuous players - Corollary 27 Example 35lower finitely many yes no yes nosemi-continuous players - Example 15 [8, Theorem 2.3] -lower countably many yes no no nosemi-continuous players - Example 15 [8, Subsection 4.3] -Borel zero-sum yes no yes no- Example 15 Corollary 29 -Borel finitely many yes no no noplayers Corollary 28 Example 15 Example 34 -Borel countably many yes no no noplayers Corollary 28 Example 15 Example 34 - Note: the table shows for any relevant combination of properties of the payoff functions, type of the game andtype of equilibria whether such equilibria exist always or whether there is a counterexample. The results aslisted here pertain to having finitely many choices at each stage of the game. There is no difference betweentwo-player games and games with finitely many players in any situation we investigate. As we consider perfectinformation games only, any zero-sum game is understood to be a two-player game. The combination ofsemi-continuity and zero-sum would imply continuity, and is thus left out. For continuous payoff functions, wealready see complete positive results without the zero-sum condition, and thus do not mention it explicitlyeither. Both Corollary 28 and Example 15 seem to be folklore results.
Multi-player multi-outcome infinite sequential games
In our most abstract definition, a game is a tuple h A, ( S a ) a ∈ A , ( ≺ a ) a ∈ A i consisting of a non-empty set A of agents or players , for each agent a ∈ A a non-empty set S a of strategies , andfor each agent a ∈ A a preference relation ≺ a ⊆ (cid:0)Q a ∈ A S a (cid:1) × (cid:0)Q a ∈ A S a (cid:1) . The generic settingsuffices to introduce the notion of a Nash equilibrium: a strategy profile σ ∈ (cid:0)Q a ∈ A S a (cid:1) is calleda Nash equilibrium, if for every agent a ∈ A and every strategy s a ∈ S a we find ¬ ( σ ≺ a σ a s a ),where σ a s a is defined by σ a s a ( b ) = σ ( b ) for b ∈ A \ { a } and σ a s a ( a ) = s a . In words, no agentprefers over a Nash equilibrium some other situation that only differs in her choice of strategy.We will give additional structure to games in two primary ways: in Section 3 we add topolo-gies to the strategy spaces, and then impose some topological constraints on both strategyspaces and preferences. Beyond that, we will consider games where strategy spaces and pref-erences are derived objects from more structured variants of games. One such variant is theinfinite sequential game: Definition 1 (Infinite sequential game, cf. [18, Definition 1.1]) . An infinite sequential game isan object h A, C, d, O, v, ( ≺ a ) a ∈ A i complying with the following.1. A is a non-empty set (of agents).2. C is a non-empty set (of choices).3. d : C ∗ → A (assigns a decision maker to each stage of the game).4. O is a non-empty set (of possible outcomes of the game).5. v : C ω → O (assigns outcomes to infinite sequences of choices).6. Each ≺ a is a binary relation over O (modeling the preference of agent a ).The intuition behind the definition is that agents take turns to make a choice. Whose turnit is depends on the past choices via the function d . Over time, the agents thus jointly generatesome infinite sequence, which is mapped by v to the outcome of the game. Note that usinga single set of actions C for each step just simplifies the notation, a generalization to varyingaction sets is straightforward.The infinite sequential games can be seen as abstract games: the agents remain the agentsand the strategies of agent a are the functions s a : d − ( { a } ) → C . We can then safely regarda strategy profile as a function σ : C ∗ → C whose induced play is defined below, where for aninfinite sequence p ∈ C ω we let p n be its n -th value, and p ≤ n = p
Given an infinite sequential game h A, C, d, O, v, ( ≺ a ) a ∈ A i , let the subgame perfectpreferences ≺ sgpa ⊆ C C ∗ × C C ∗ be defined by σ ≺ sgpa σ ′ iff ∃ γ ∈ C ∗ such that p γ ( σ ) ≺ a p γ ( σ ).The subgame perfect equilibria of h A, C, d, O, v, ( ≺ a ) a ∈ A i are the Nash equilibriaof h A, ( C d − ( { a } ) ) a ∈ A , ( ≺ sgpa ) a ∈ A i .We consider a further variant, namely the infinite sequential games with real-valued payoffs ,which can (but do not have to) be understood as a special case of infinite sequential games. Definition 4.
An infinite sequential game with real-valued payoffs is a tuple h A, C, d, ( f a ) a ∈ N i where A , C , d are as above, and f a : C ω → R is the payoff function of player a .Such a game can be identified with the infinite sequential game h A, C, d, R A , v, ( ≺ a ) a ∈ A i where v ( p ) = ( f a ( p )) a ∈ A and for x, y ∈ R A , we set x ≺ a y iff x a < y a .As with the introduction of subgame perfect equilibria, we can consider infinite sequentialgames with real-valued payoffs as infinite sequential games in a different way, which then givesrise to another commonly studied equilibrium concept, namely ε -Nash equilibria. Dependingon how we then translate from infinite sequential games to abstract games, we obtain also ε -subgame perfect equilibria. Given some ε >
0, we define the relation ≺ εa ⊆ R A × R A by x ≺ εa y iff y a − x a > ε . Using ≺ εa in place of ≺ a in Definition 4 then provides the above-mentionedequilibrium notions.For infinite sequential games with real-valued payoffs, every Nash equilibrium (w.r.t. thestandard preferences) is an ε -Nash equilibrium; and every subgame perfect equilibrium is an ε -subgame perfect equilibrium. For infinite sequential games, every subgame perfect equilibriumis a Nash equilibrium, in particular, any ε -subgame perfect equilibrium is an ε -Nash equilibrium.We use antagonistic game to refer to two-player games with preferences satisfying ≺ a = ≺ − b ,where x ≺ − y ⇔ y ≺ x .We proceed to recall a few more notions that are only tangentially related to the formulationof our results, but that do show up in the proofs. Definition 5.
A two-player win-loose game is a tuple h C, D, W i with D ⊆ C ∗ and W ⊆ C ω . Itcorresponds to the infinite sequential game h{ a, b } , C, d, { , } , v, { <, < − }i where d is definedvia d − ( { a } ) = D and v is defined via v − ( { } ) = W .Finally, we will extend the notion of the induced play. Given some partial function s : ⊆ C ∗ → C , we define the consistency set P ( s ) ⊆ C ω by: P ( s ) = { p ( σ ) | σ : C ∗ → C ∧ σ | dom( s ) = s } Note that the translation of preferences in the following definition does not preserve acyclicity. Preservationcould be ensured, e.g. , by giving the nodes a linear ”priority” order, in a lexicographic fashion. This, however,would complicate the definition against little benefit for the point that we want to make.
Multi-player multi-outcome infinite sequential games
Pareto-optimality
Pareto-optimality provides a notion of social desirability in game theory, and can be used bothto pick particularly nice equilibria, and to investigate whether the strategic interaction is costlyin some sense . The fundamental idea is that a Pareto-optimal outcome cannot be improved toeveryone’s satisfaction. Pareto-optimality is often only defined for linear preferences (or, slightlymore general, strict weak orders), and its extension to general preferences is not obvious. Thetwo natural choices are: Definition 6.
An outcome is realizable in some game, if it is assigned to some sequence ofchoices. We call an outcome o Pareto-optimal , if there is no other realizable outcome q suchthat for some player a we find o ≺ a q and for no player b we have q ≺ a o . We call an outcome o weakly Pareto-optimal , if there is no other realizable outcome q such that for some player a wefind o ≺ a q and for all players b we have that o (cid:22) b q .Note that for linear preferences, both notions coincide. We shall call a Nash equilibrium(weakly) Pareto-optimal iff it induces a (weakly) Pareto-optimal outcome. A strong transfer result can be obtained using topological arguments alone, with the reasoningbeing particularly well-adapted to a formulation in synthetic topology (originally [6], [32] for ashort introduction). Consider games in normal form, with potentially countably many agentswith strategy spaces S , S , . . . . Our first condition is that each S i be compact (subsequently,by Tychonoff’s Theorem, also Π i ∈ N S i ). This restriction is very common and usually combinedwith continuity of the outcome function, as it avoids pathological games such as pick-the-largest-natural-number . Our second condition is that each preference relation ≺ i is open (as a subset of(Π i ∈ N S i ) × (Π i ∈ N S i )). In the reading of synthetic topology, this means that any agent will beable to eventually confirm that he prefers a given strategy profile to another, provided he doesindeed do so. We shall call a class of games G satisfying these conditions (in a uniform way) tobe compact-strategies, open-preferences. Uniformity here means that we assume a topology on G such that the function mapping a game to the preferences is continuous itself.We will write O ( X ) for the hyperspace of open subsets of X , and K ( X ) for the hyperspaceof compact sets. By C ( X , Y ) we denote the space of continuous functions from X to Y , inparticular C ( N , X ) we denote the space of sequences in X . For precise definitions, see [32].There we also find that the following operations are continuous:1. ∃ : O ( X × Y ) → O ( X ), definedby ∃ ( U ) = { x ∈ X | ∃ y ∈ Y ( x, y ) ∈ U } S : C ( N , O ( X )) → O ( X ) Similar to the (quantitative) price of stability , see [36]. There actually is a third condition, that any strategy space is overt. A space X is overt, if {∅} ⊆ O ( X ) isa closed set, i.e. if there is a way to detect non-emptiness of open subsets. This condition could only ever fail ina constructive reading, but is always valid for topological spaces in classical logic. Synthetic topology howeverwould also allow us to read continuous map to mean computable map , in which case overtness becomes non-trivial.In this reading, though, we actually obtain an algorithmic result. In general, the continuity of this map would require Y to be overt. As explained above, in a classical reading,this condition is always satisfied. . Le Roux & A. Pauly C : O ( X ) → K ( X ), provided that X is compact.4. NonEmptyValue : C ( X , K ( Y )) → O ( X ) defined byNonEmptyValue( f ) = { x ∈ X | f ( x ) = ∅} . Theorem 7.
Let G be compact-strategies, open-preferences, and let G ′ ⊆ G be a dense subclass.If every G ∈ G ′ has a Nash equilibrium, then every G ∈ G has a Nash equilibrium. Proof.
By combining our continuous operations, we may obtain the set of all games in G with aNash equilibrium as an open set in the following way: N E := NonEmptyValue G [ i ∈ N ∃ ( ≺ Gi ) ! C ∈ O ( G )Formulating the individual steps in words: with ≺ i being open , we immediately obtain thatthe set of all strategy profiles such that player i has a better response is uniformly open in thegame. Taking the union over all players again yields an open set, which complement now isthe closed set NE( G ) of all Nash equilibria of the respective game G . As this is a subset of thecompact space (Π i ∈ N S i ), we can even treat NE( G ) as a compact set uniformly in G . By thesynthetic definition of compactness, we obtain that { G ∈ G | NE( G ) = ∅} ⊆ G is an open set.Because we have assumed G ′ to be dense in G , we see that if any game in G would fail tohave a Nash equilibrium, this would imply that some game in G ′ would fail, too, contrary to theassumption. Lemma 8.
Consider sequential games with continuous payoff-functions and finite choices sets.We find:1. The subgame-perfect preferences produce a compact-strategies, open-preference class S .2. The games with payoffs fully determined after finitely many moves are a dense subset S f ⊆ S .3. All games in S f have a Nash equilibrium. Proof.
1. First, note that the mapping ( γ, σ ) p γ ( σ ) is continuous, so for any fixed γ ∈ C ∗ , { ( σ, σ ′ ) | f a ( p γ ( σ )) < f a ( p γ ( σ ′ )) } is an open set. By taking countable union, we learn that ≺ sgpa is open. Compactness and overtness of the strategy spaces are straightforward.2. As the argument in (1) is uniform in the continuous functions f a , it suffices to argue thatthe payoff functions f : C ω → R depending only on some finite prefix of the input aredense in C ( C ω , R ). A countable base for the applicable topology is found in all { f | ∀ γ ∈ C k , p ∈ C ω . p ≤ k = γ ⇒ f ( p ) ∈ ( x γ , y γ ) } for k ∈ N , x ( · ) , y ( · ) : C k → Q A base element is non-empty, iff ∀ γ ∈ C k x γ < y γ ; and then it will contain the function f : C ω → R defined via f ( p ) = (cid:0) x p ≤ k + y p ≤ k (cid:1) , which clearly depends only on the prefixof length k of its argument. And S i being overt, see above. Multi-player multi-outcome infinite sequential games
3. As the actions of the players beyond the finite prefix determining the outputs is irrelevant,and taking into consideration the definition of the subgame-perfect preferences, the claimis that any finite game in extensive form has a subgame perfect equilibrium. This well-known result (
Kuhn [15]) is easily proven by backwards induction: let the players whomove last pick an optimal (for them) choice. Then the players who move second-but-lasthave guaranteed outcomes associated with their moves, so they can optimize, and so on.
Corollary 9 (( )) . Any sequential game with continuous payoff functions and finitely manychoices has a subgame perfect Nash equilibrium.We shall consider a number of variations/extensions. First, we investigate stochastic infinitesequential games with continuous payoff functions as a variation of Definition 4: a stochasticinfinite sequential game with real-valued payoffs is a tuple h A, C, d, ( f a ) a ∈ N , P i where A , C , f a are as above, n / ∈ A , d : C ∗ → A ∪ { n } and P assigns a probability distribution over C to each w ∈ d − ( { n } ).The notion of the play induced by a strategy profile is replaced by a probability distributionover plays induced by a strategy profile: essentially, we choose according to the strategy profilein vertices controlled by a player, and stochastically according to P ( w ) in vertices controlledby n (nature). A player prefers a strategy profile to another one if the expected value of hispayoff function regarding the former induced probability distribution exceeds the expected valueregarding the latter. We obtain a notion of a subgame perfect equilibrium as before. Lemma 10.
Consider stochastic infinite sequential games with continuous payoff-functions andfinite choices sets. We find:1. The subgame-perfect preferences produce a compact-strategies, open-preference class SS .2. The games with payoffs fully determined after finitely many moves are a dense subset SS f ⊆ SS .3. All games in SS f have a Nash equilibrium. Proof.
1. For a synthetic approach to continuity on probability measures etc, see [5]. Inparticular, the map from strategy profiles to induced probability distributions is easilyseen to be continuous; and integration is as well. The remaining argument proceeds as inLemma 8 (1).2. The stochastic vertices do not impact the argument, so it works as in Lemma 8 (2).3. As in Lemma 8 (3), we use backwards inductions: the value of a given leaf for a player isimmediately obtained from the payoff function. In any vertex controlled by a player, hepicks a choice guaranteeing him the optimal value, and the value of that vertex for anyplayer p is identical to the value of the corresponding child for p . The value of a naturevertex is the expected value of its children according to the probability distribution givenby P . This is easily seen to yield a subgame perfect equilibrium. Corollary 11.
Stochastic infinite sequential games with continuous payoff-functions and finitechoices sets have subgame-perfect equilibria. This extends [9, Corollary 4.2] from finitely many players to countably many players. In this, we are answering a question raised by
Assal´e Adj´e at CSL-LiCS. . Le Roux & A. Pauly , A v of m players act jointly and probabilistic and thus determine the next vertex in the tree. Inparticular, we are now dealing with a tree with branching C m instead of just C . The map d indicating who plays now goes into the m -element subsets of A rather than identifying just asingle player. Strategies are functions s a : ⊆ ( C m ) ∗ → P ( C ), where P ( C ) shall denote the set ofprobability distributions over C , strategy profiles are of the form s : ⊆ A × ( C m ) ∗ → P ( C m ) andpayoff functions go f a : ( C m ) ω → [0 , C m ) ω (also from a given vertex onwards), payoffs for players are then again the expectedvalues of their payoff functions given the induced probability distribution. Lemma 12.
Consider Blackwell games with continuous payoff-functions, countably many agentsand a finite choice set. We find:1. The subgame-perfect preferences produce a compact-strategies, open-preference class B .2. The games with payoffs fully determined after finitely many moves (and thus dependingonly on the choices made by finitely many players) are a dense subset B f ⊆ B .3. All games in B f have a Nash equilibrium. Proof.
1. Very much like Lemma 10 (1). The induced probability distribution depends con-tinuously on the strategy profile, and the expected value is a continuous map. It is un-problematic that C is replaced by P ( C ) now, as this space is still compact.2. As in Lemma 8 (2), we only need to argue that the continuous payoff functions determinedby their values on ( C m ) n for some n ∈ N are dense in C (( C m ) ω , [0 , Nash ’ classic result [28]. Pick a Nash equilibriumfor each such vertex, and assign it the corresponding outcomes as values. Then the layerpreceding this one can be handled in the same way, etc. – so the claim follows by backwardsinduction.
Corollary 13 (( )) . Any multi-outcome Blackwell game with countably many players andcontinuous payoff functions has a subgame-perfect Nash equilibrium.The arguments for Lemma 12 remain unaffected if the number of agents playing at a partic-ular vertex varies with the vertex (this just increases notational complexity). We can even let allplayers act at each time, as long as we then restrict their strategies to continuously depend onthe choices (note that this would produce a tree with branching factor 2 ℵ ). We can also turnone player into a designated nature player, which plays according to some prespecified strategyrather than according to preferences, and thus obtain: This extends [9, Theorem 6.1] from finitely many players to countably many players. Multi-player multi-outcome infinite sequential games
Corollary 14.
Any multi-outcome stochastic Blackwell game with countably many players andcontinuous payoff functions has a subgame-perfect Nash equilibrium.Concurrent games (in the sense of
Winskel ) may be another interesting class of games toapply this approach to. Determinacy questions of these games were discussed in [12, 13].
As soon as the continuity requirement for the payoff function (or, more generally, the opennessof the preferences) is dropped, Nash equilibria may fail to exist. We provide a generic folklorecounterexample, and will proceed to demonstrate that the underlying feature is essential forthe failure of existence of Nash equilibria. The counterexample only requires a single player,and its payoff function is in a sense the least discontinuous payoff function, and in particular is∆ -measurable. Example 15.
Let the payoff function P : { , } N → [0 ,
1] for the single player be defined by P (1 n p ) = nn +1 for all p ∈ { , } N and P (1 N ) = 0. As P does not attain its supremum, theresulting game cannot have a Nash equilibrium.a a a nn +1 p n ) n ∈ N such that a player prefers p n +1 to p n , but prefers any p n to lim i →∞ p i , is a crucial feature ofthe example above to have no Nash equilibrium. The proof will be an adaption of the mainresult of [18] by the first author. Under the additional assumption of antagonistic preferencesin a two-player game, we can even obtain subgame perfect equilibria.In this section the preferences of the players are restricted to strict weak orders, so we recalltheir definition below. Definition 16 (Strict weak order) . A relation ≺ is called a strict weak order if it satisfies: ∀ x, ¬ ( x ≺ x ) ∀ x, y, z, x ≺ y ∧ y ≺ z ⇒ x ≺ z ∀ x, y, z, ¬ ( x ≺ y ) ∧ ¬ ( y ≺ z ) ⇒ ¬ ( x ≺ z )Definition 17 below slightly rephrases Definitions 2.3 and 2.5 from [18]: the guarantee of aplayer is the smallest set of outcomes that is upper-closed w.r.t. the strict-weak-order preferenceof the player and includes every incomparability classe (of the preference) that contains anyoutcome compatible with a given strategy of the player in the subgame at a given node of agiven infinite sequential game. The best guarantee of a player consists of the intersection of allher guarantees over the set of strategies. . Le Roux & A. Pauly Definition 17 (Agent (best) guarantee) . Let h A, C, d, O, v, ( ≺ a ) a ∈ A i be a game where the ≺ a are strict weak orders. ∀ a ∈ A, ∀ γ ∈ C ∗ , ∀ s : d − ( a ) → C, g a ( γ, s ) := { o ∈ O | ∃ p ∈ P ( s | γC ω ) ∩ γC ω , ¬ ( o ≺ a v ( p )) } G a ( γ ) := T s g a ( γ, s )We write g a ( s ) and G a instead of g a ( γ, s ) and G a ( γ ) when γ is the empty word.Lemma 2.4. from [18] still holds without major changes in the proofs, so we do not displayit, but note that when speaking about ≺ a -terminal intervals (which are upper-closed sets), wenow actually refer to the terminal intervals of the lift of ≺ a from outcomes to the equivalenceclasses of outcomes induced by the strict weak order. Also, we collect some more useful facts inObservation 18 below. Observation 18.
Let h A, C, O, d, v, ( ≺ a ) a ∈ A i , let a ∈ A , assume that ≺ a is a strict weak order,and let γ ∈ C ∗ .1. d ( γ ) = a ⇒ G a ( γ ) = ∪ c ∈ C G a ( γ · c )2. d ( γ ) = a ⇒ G a ( γ ) = ∩ c ∈ C G a ( γ · c )3. d ( γ ) = a ∧ | C | < ∞ ⇒ ∃ c ∈ C, G a ( γ ) = G a ( γ · c ) Proof.
For example, for 2 . note that G a ( γ ) = ∩ s g a ( γ, s ) = ∩ c ∈ C ∩ s ( γ )= c g a ( γ, s ) = ∩ c ∈ C ∩ s g a ( γ · c, s )= ∩ c ∈ C G a ( γ · c ).This section’s proofs of existence of equilibrium rely on each player having a (minimax-style)optimal strategy if all other players team up against her. Lemma 19 below provides a sufficientcondition for such strategies to exist, i.e. for the best guarantee to be witnessed. Lemma 19.
Let h A, C, O, d, v, ( ≺ a ) a ∈ A i be a game where C is finite, let a ∈ A , and let usassume the following.1. ≺ a is a strict weak order.2. For every play p ∈ C ω , increasing ϕ : N → N , and sequence ( s n ) n ∈ N of strategies for Player a , if g a ( p <ϕ ( n +1) , s n +1 ) ( g a ( p <ϕ ( n ) , s n ) for all n ∈ N , then v ( p ) ∈ ∩ n ∈ N g a ( p <ϕ ( n ) , s n ).Then for all γ ∈ C ∗ there exists s ∈ S a such that g a ( γ, s ) = G a ( γ ). Proof.
Wlog we only prove that there exists s ∈ S a such that g a ( s ) = G a , i.e. where the γ fromthe claim is the empty word. Let s : d − ( a ) → C be a strategy for Player a and let us buildinductively a sequence ( s n ) n ∈ N of strategies for Player a , as follows, where case 3 . implicitlyinvokes Observation 18. • Let s n +1 | C 3. otherwise let s n +1 ( γ ) := c such that G a ( γ · c ) = G a ( γ ), and let s n +1 | γCC ∗ := s n | γCC ∗ .Let s be the limit strategy of the sequence ( s n ) n ∈ N and first note that, using Observation 18,one can prove by induction on γ that G a ( γ ) ⊆ G a for every γ ∈ C ∗ that is compatible with s . Next, let p ∈ P ( s ) be a path compatible with s . If p has a prefix γ that fell into Cases1 . or 2 . during the recursive construction above, then v ( p ) ∈ G a , so let us now assume thatcase 3 . applies at every node p Let h A, C, O, d, v, ( ≺ a ) a ∈ A i be a game, let a ∈ A , and let us assume the following.1. ≺ a is a strict weak order.2. For every play p ∈ C ω and increasing ϕ : N → N , if d ( p <ϕ ( n ) ) = a and G a ( p <ϕ ( n +1) ) ( G a ( p <ϕ ( n ) ) for all n ∈ N , then v ( p ) ∈ ∩ n ∈ N G a ( p <ϕ ( n ) ).3. For all γ ∈ C ∗ there exists s ∈ S a such that g a ( γ, s ) = G a ( γ ).Then there exists s such that g a ( γ, s ) = G a ( γ ) for all γ ∈ C ∗ . Proof. We proceed similarly as in the proof of Lemma 19. Let s be a strategy for Player a andlet us build inductively a sequence ( s n ) n ∈ N of strategies for Player a . The recursive definitionbelow is different from the one in the proof of Lemma 19 in three respects: the three occurrencesof G a in Cases 1 . and 2 . are replaced with G a ( γ ). Case 3 . is deleted since it never applies byassumption. Finally, two inclusions are replaced with equalities. • Let s n +1 | C Let h{ a, b } , C, O, d, v, {≺ , ≺ − }i be a two-player game. Let Γ ⊆ P ( C ω ) and assumethe following.1. ≺ is a strict weak order.2. For every play p ∈ C ω and increasing sequence ϕ : N → N , if d ( p <ϕ ( n ) ) = a and G a ( p <ϕ ( n +1) ) ( G a ( p <ϕ ( n ) ) for all n ∈ N , then v ( p ) ∈ ∩ n ∈ N G a ( p <ϕ ( n ) ).3. For all γ ∈ C ω , there exists s such that g a ( γ, s ) = G a ( γ ) (resp. g b ( γ, s ) = G b ( γ )).4. For all non-empty closed E ⊆ C ω , there are ≺ -extremal elements in v [ E ].5. For every ≺ -extremal interval I and γ ∈ C ∗ , we have (cid:0) v − [ I ] ∩ γC ω (cid:1) ∈ Γ.6. The game h C, D, W i is determined for all W ∈ Γ, D ⊆ C ∗ .Then the game h{ a, b } , C, O, d, v, {≺ , ≺ − }i has a subgame perfect equilibrium. Proof. By invoking Lemma 20 once for Player a and once for Player b , let us build a strategyprofile s : C ∗ → C , such that g X ( γ, s X ) = G X ( γ ) for all γ ∈ C ∗ and X ∈ { a, b } . Let γ ∈ C ∗ and let us prove that G a ( γ ) ∩ G b ( γ ) = { min < ( G a ( γ )) } = { max < ( G b ( γ )) } . Consider the game h C, D, W i (as in Definition 5) where the winning set is defined by W := { α ∈ γC ω | v ( α ) ∈ G a ( γ ) \{ min < ( G a ( γ )) }} and where Player a owns exactly the nodes in D := ( C ∗ \ γC ∗ ) ∪ ( d − ( { a } ) ∩ γC ∗ ). By Assumption 5 the set W is in Γ, so by Assumption 6 the game h C, D, W i is determined. By definition of the best guarantee, Player a has no winning strategy for thisgame, so Player b has a winning strategy, which means that G b ( γ ) ⊆ { min < ( G a ( γ )) } ∪ O \ G a ( γ ).Since G a ( γ ) ∩ G b ( γ ) must be non-empty, otherwise the two guarantees are contradictory, G a ( γ ) ∩ G b ( γ ) = { min < ( G a ( γ )) } . This means that the subprofile of s rooted at γ induces the outcomemin < ( G a ( γ )) (which equals max < ( G b ( γ )) by symmetry), and it is optimal for both players. Theorem 22. Let h{ a, b } , C, O, d, v, {≺ , ≺ − }i be a two-player antagonistic game, where C isfinite. Let Γ ⊆ P ( C ω ) and assume the following.1. ≺ is a strict weak order.2. For every p ∈ C ω , sequence ( s n ) n ∈ N of strategies for X ∈ { a, b } , and increasing ϕ : N → N ,if d ( p <ϕ ( n ) ) = a and g X ( p <ϕ ( n +1) , s n +1 ) ( g X ( p <ϕ ( n ) , s n ) for all n ∈ N , then v ( p ) ∈∩ n ∈ N g X ( p <ϕ ( n ) , s n ).3. For every ≺ -extremal interval I and γ ∈ C ∗ , we have (cid:0) v − [ I ] ∩ γC ω (cid:1) ∈ Γ.4. The game h C, D, W i is determined for all W ∈ Γ, D ⊆ C ∗ .4 Multi-player multi-outcome infinite sequential games Then the game h{ a, b } , C, O, d, v, {≺ , ≺ − }i has a subgame perfect equilibrium. Proof. By application of Lemma 21. (Note that Condition 2 of Theorem 22 is an ”upper bound”of Condition 2 of Lemma 19 and Condition 2 of Lemma 20.) Condition 3 is proved by Lemma 19.For Condition 4, let E be a non-empty closed subset of C ω , and let T be the tree such that[ T ] = E . Consider the game where Player a plays alone on T . Since Player a can maximise herbest guarantee by Lemma 19, and since all her guarantees are singletons, v [ E ] has a ≺ -maximum.Likewise, it has a ≺ -minimum, by considering Player b . Theorem 23. Let h{ a, b } , C, d, O, v, { <, < − }i be an infinite sequential game where O is finiteand < is a strict linear order. Let Γ ⊆ P ( C ω ) and assume the following.1. ∀ O ′ ⊆ O, ∀ γ ∈ C ∗ , { α ∈ C ω | v ( γα ) ∈ O ′ } ∈ Γ2. The game h C, D, W i is determined for all W ∈ Γ and D ⊆ C ∗ .Then the game h{ a, b } , C, d, O, v, { <, < − }i has a subgame perfect equililbrium. Proof. by Lemma 21 where Conditions 2, 3, and 4 hold by finiteness of O . Corollary 24. Let h{ a, b } , C, d, O, v, { <, < − }i be an infinite sequential game where O is finiteand < is a strict linear order. If v − ( o ) is quasi-Borel for all o ∈ O , the game has a subgameperfect equilibrium. Proof. From Theorem 23, quasi-Borel determinacy [26], and Lemma 3.1. in [18].In the remainder of this section we discuss existence of Nash equilibria in multi-player games.Theorem 25 below is our most general result. Theorem 25. Let h A, C, O, d, v, ( ≺ a ) a ∈ A i be a game, let Γ ⊆ P ( C ω ), and assume the following.1. The ≺ a are strict weak orders.2. The game h C, D, W i is determined for all W ∈ Γ, D ⊆ C ∗ .3. For every a ∈ A and ≺ a -terminal interval I and γ ∈ C ∗ , we have (cid:0) v − [ I ] ∩ γC ω (cid:1) ∈ Γ.4. For every play p ∈ C ω and increasing sequence ϕ : N → N , if d ( p <ϕ ( n ) ) = a and G a ( p <ϕ ( n +1) ) ( G a ( p <ϕ ( n ) ) for all n ∈ N , then v ( p ) ∈ ∩ n ∈ N G a ( p <ϕ ( n ) ).5. For all γ ∈ C ω , there exists s such that g a ( γ, s ) = G a ( γ ).Then the game h A, C, O, d, v, ( ≺ a ) a ∈ A i has a Nash equilibrium. Proof. Since the proof is similar to that of Theorem 2.9 in [18], we rephrase and give it a moreintuitive flavour. Let σ be a strategy profile where every player is using a witness to Lemma20. Let p be the induced play. We now turn σ into a Nash equilibrium with p as induced playby use of threats. More specifically, at each node p Let g be a h A, C, O, d, v, ( ≺ a ) a ∈ A i be a game where the ≺ a are strict weak ordersand v is Borel-measurable. The following are equivalent.1. For every X ∈ A and ( p n ) n ∈ N sequence of plays in C ω converging to some p , and increasing ϕ : N → N , if for all n ∈ N we have d ( p <ϕ ( n ) ) = X , p <ϕ ( n ) = p n<ϕ ( n ) , p ϕ ( n ) = p nϕ ( n ) , and v ( p n ) ≺ X v ( p n +1 ), then v ( p n ) ≺ X v ( p ) for all n ∈ N .2. Every finite-branching game derived from the original game by pruning has an NE. Proof. Let us first prove 1 . ⇒ . by invoking Theorem 25. More specifically, let T be a finite-branching, infinite subtree of C ω and consider the restriction of the original game to T . Con-ditions 3 and 2 follow from Borel measurability and [25]. Condition 5 comes from Lemma 19(actually a straightforward extension of Lemma 19 to trees with finite-yet-unbounded branch-ing), and Condition 4 follows directly from the assumption.For 2 . ⇒ . , let X ∈ { a, b } and let ( p n ) n ∈ N → p ∈ C ω and increasing ϕ : N → N such thatfor all n ∈ N we have d ( p <ϕ ( n ) ) = X , p <ϕ ( n ) = p n<ϕ ( n ) , p ϕ ( n ) = p nϕ ( n ) , and v ( p n ) ≺ X v ( p n +1 ). Let T be the tree made of the prefixes of p and the p n . Since the game induced by T has an NEand its tree structure is similar to Example 15, v ( p n ) ≺ X v ( p ) must hold for all n ∈ N .However, the modification of Example 15 below shows that the conditions of Theorem 26are not necessary for the mere existence of Nash equilibria.a2 a a nn +1 e.g. , [14] for a concrete example or page 3 of [17] for ageneric one). So the algorithm has to run on strict weak orders. (In [18, Theorem 2.9] it evenruns on strict linear orders.)If we wanted to consider strict partial orders and extend them linearly for the algorithm towork, we would potentially run into two problems: first, there may not exist any linear extension6 Multi-player multi-outcome infinite sequential games preserving Condition 4. Second, assumptions 2 and 3 of Theorem 25 make sure that the win-losegames associated with the ≺ a -terminal intervals are determined, which is a requirement for theproof to work. If the preferences were not strict weak orders, we might think of replacing thecondition on terminal intervals by a condition on the upper-closed sets and then extend thepreferences linearly for the algorithm to work, but in the special case where the preference ofone player were the empty relation, every subset would be an upper-closed set and its preimageby v would be in the pointclass with nice closure property, by assumption. If, in addition, eachoutcome is mapped to at most one play, it implies that each subset of C ω is in the pointclass,so Theorem 25 could be used with the axiom of determinacy only, but not with, e.g. , Boreldeterminacy. On the contrary, [18, Theorem 2.9, Assumption 3] is not an issue since there areonly countably many outcomes in that setting.Theorem 25 has a corollary pertaining to sequential games with real-valued payoffs. Ratherthan the usual Euclidean topology, we consider the lower topology generated by { ( −∞ , a ) | a ∈ Q } . This space will be denoted by R > . Note that continuous functions with codomain R > are often called upper semi-continuous. As id : R > → R is complete for the Σ -measurablefunctions [42, 40], we see that the Borel sets on R > are the same as the Borel sets on R .Moreover, if ( p n ) n ∈ N is a converging sequence of plays, and P : { , } N → R > is a continuouspayoff function, then P (lim i ∈ N p i ) ≥ lim sup i ∈ N P ( p i ). In particular, Condition 4 in Theorem 25is always satisfied for the preferences obtained from upper semi-continuous payoff functions. Corollary 27. Sequential games with countably many players, finitely many choices and uppersemi-continuous payoff functions have Nash equilibria.A rather simple argument allows us to transfer existence theorems for equilibria in gameswith Borel-measurable valuations to Borel-measurable real-valued payoff functions with upperbound, if one is willing to replace the original notions by their ε -counterparts. If v : S → ( −∞ , ω is the Borel-measurable payoff function (with a component for each of the count-ably many players), then for every positive real ǫ we define v ǫ : S → N ω by v − ǫ (( i k ) k ∈ N ) := v − (] − ( i + 1) ǫ, i ǫ ] × ] − ( i + 1) ǫ, i ǫ ] × . . . ). Then any v ǫ is again a Borel measurable valua-tion (as a product of countably many intervals is Π ). Furthermore, we define the preferences ≺ n for the n -th player by ( i k ) k ∈ N ≺ n ( j k ) k ∈ N iff i n < j n . Now every Nash equilibrium of the result-ing game is a ǫ -Nash equilibrium of the original game, and every subgame perfect equilibriumof the resulting game is a ǫ -subgame perfect equilibrium of the original game. Corollary 28. Sequential games with countably many players and Borel-measurable payofffunctions with upper-bounds admit ε -Nash equilibria. Proof. By combining the statement of Theorem 26 with the argument above. We can invokeTheorem 26 as the preferences ≺ n do not have any infinite ascending chains at all. Corollary 29. A sequential two-player zero-sum game with Borel measurable payoffs has ε -subgame perfect equilibria. Proof. By Corollary 24 and the argument above. As R > is not metric (but still countably based), the definition of the Borel hierarchy has to be modified asdemonstrated by Selivanov [37]. A move towards definitions of Borel measurability on even more general spacescan be found in [34]. In his survey [27], Mertens sketches an observation by himself and Neyman that one may use Borel deter-minacy to directly obtain the special case of this result for finitely many players and bounded payoffs. . Le Roux & A. Pauly In this section we investigate very general classes of games that guarantee existence of Nashequilibria, and such that there exists an NE that is Pareto-optimal among all the profiles of thegame (not just Pareto-optimal among all Nash equilibria). In the following, we shall assumethat any outcome is realizable to avoid unnecessary case-distinctions. Lemma 30. Let Γ be a determined pointclass. Then every infinite sequential two-player gamewith a Γ-measurable outcome function and preferences y ≺ a x ≺ a . . . ≺ a x n and y ≺ b x n ≺ b . . . ≺ b x , has a Pareto-optimal NE. Proof. By assumption that every outcome is realizable, there is some path p through the gameyielding a payoff that is not y . For each vertex along this path, by determinacy either theopponent can enforce the outcome y , or the controller can enforce some upper interval. As long asthe opponent can enforce y , he can force the controller to play along the chosen path by threatenpunishment by y for deviation. If we ever reach a vertex where the controller (w.l.o.g. a ) canenforce { x , . . . , x n } , there will be some minimal upper set { x i , . . . , x n } (from her perspective)that she can enforce. By determinacy, again, the opponent can enforce { y, x , . . . , x i } . We thenlet both players play their enforcing strategy from this node onwards.The constructed partial strategies can be extended in an arbitrary way to yield a Nashequilibrium with another outcome than y , and these are all Pareto-optimal. Theorem 31. We fix a non-empty set of players A and a non-empty set of outcomes O . Let Γbe a determined pointclass closed under rescaling and union with clopens. Then the followingare equivalent for a family ( ≺ a ) a ∈ A of linear preferences:1. The inverse of the preferences are well-founded and ∀ a, b ∈ A, ∀ x, y, z ∈ O, ¬ ( z ≺ a y ≺ a x ∧ x ≺ b z ≺ b y ).2. Every finite sequential game (built from A , O , ( ≺ a ) a ∈ A ) with three leaves has a Pareto-optimal NE.3. Every infinite sequential game (built from A , O , ( ≺ a ) a ∈ A ) with a Γ-measurable outcomefunction has a Pareto-optimal NE. Proof. . ⇒ . Clear.2 . ⇒ . By contraposition, let us assume that z ≺ a y ≺ a x and x ≺ b z ≺ b y , and note that thegame below has only one NE yielding outcome z .ba x y z . ⇒ . By [20, Lemma 4] the second assumption in 1 . implies that there exists a partition { O i } i ∈ I of O and a linear order < over I such that i < j implies x < a y for all a ∈ A and x ∈ O i and y ∈ O j , and such that < b | O i = < a | O i or < b | O i = < a | − O i for all a, b ∈ A . By thewell-foundedness assumption, I has a < -maximum m .Fix some a ∈ A , let { x , . . . , x n } := O m (again, by well-foundedness, each slice is finite)such that x n < a · · · < a x , let A := { b ∈ A | < b | O m = < a | O m } , let A := A \ A , and let8 Multi-player multi-outcome infinite sequential games y / ∈ O m . Let us derive a new game on the same tree: each vertex of the original gameowned by b ∈ A is now owned by A if b ∈ A and by A otherwise. Each play of theoriginal game that induces an outcome outside of O m induces y in the derived game. Thenew preferences are y < A x n < A · · · < A x and y < A x < A · · · < A x n . By Lemma30, the derived game has a Pareto-optimal NE (which cannot yield y , as this is the onlynon-Pareto-optimal outcome). It is also a Pareto-optimal NE for the original game.The situation for non-linear orders is less clear. Certainly, whenever some linearizationavoids the forbidden pattern from Theorem 31 (1 . ), there will be a Pareto-optimal NE (asbeing Pareto-optimal w.r.t. the linearization implies being Pareto-optimal w.r.t. the originalpreferences). However, we do not know whether partial preferences such that any linearizationhas the forbidden pattern is enough to enable absence of Pareto-optimal NE. Two examplesthat could potentially play a similar role to the generic counterexample in Theorem 31 (2 . → Example 32. We consider a finite two-player game with outcomes { x, y, z, α, β, γ } , preferences γ ≺ a y ≺ a x and z ≺ a β ≺ a α and x ≺ b z ≺ b y and α ≺ b γ ≺ b β and game tree:ba x y α βz γ The preferences avoid the forbidden pattern from Theorem 31 (1 . ); but the pattern is presentin any linear extension. In the Nash equilibria of the game, player b is choosing either z or γ ;and player a is choosing x or α . In particular, the potential equilibrium outcomes are z and γ – precisely those outcomes that are not weakly Pareto-optimal (because every player prefers y to z and β to γ ). Both y and β would even have been Pareto-optimal. Example 33. We consider a finite two-player game with outcomes { x, y, z, t } , preferences t, z ≺ a x, y and x ≺ b z ≺ b y ≺ b t and game tree:bab t y x z The preferences are strict weak orders and avoid the forbidden pattern from Theorem 31(1 . ); but the pattern is present in any linear extension. The only equilibrium outcome is z ,despite everyone preferring y ( ). In this section we will show that in the simultaneous absence of continuity and the antagonistic/zero-sum property, even a two-player game with three distinct outcomes may fail to have subgame It may be an interesting remark that in this game every player would benefit, if b could not choose t at hissecond move. . Le Roux & A. Pauly ε -subgame perfect equi-libria cannot help, either. As our (counter-) Example 15, the valuation function here is ∆ -measurable, hence, in a sense, not very discontinuous. A similar counterexample is also exhibitedin [39, Example 3]. Example 34. The following game where z < a y < a x and x < b z < b y has no subgame perfectequilibrium. ababa x y z y z y The game above is formally defined as h{ a, b } , { , } , d, O, v, { < a , < b }i , where d − ( a ) := 0 ∗ and v (0 ω ) := x and v [0 ∗ { , } ω ] := { y } and v [0 ∗ +1 { , } ω ] := { z } . Proof. Assume for a contradiction that there is a subgame perfect equilibrium for this game.Then no subprofile (starting at some node in 0 ∗ ) induces the outcome x , because Player b couldthen switch to the right and obtain z . So for infinitely many nodes in 0 ∗ , Players a or b chooses1. Also, if Player b chooses 1 at some node 0 n +1 , Player a chooses 1 at the node 0 n rightabove it. This implies that every subprofile rooted at nodes in 0 ∗ induces the outcome y , andsubsequently, Player b always chooses 0 at nodes in 0 ∗ +1 . But then Player a could alwayschoose 0 and obtain x , contradiction.Conversely, consider a game with finitely many players and outcomes such that the patternfrom Example 34 is absent from the preferences of any two players. If the outcome function ismeasurable in the Hausdorff difference hierarchy (of the open sets), [20, Corollary 2] says thatthe game has a Pareto-optimal subgame perfect equilibrium.A further example shows us that we can rule out subgame perfect equilibria even withstronger conditions on the functions by using countably many distinct payoffs. This example nolonger extends to ε -subgame perfect equilibria. Example 35. The following game where y n := (2 − n , − n ) and z n := (0 , − n − ) for all n ∈ N has no subgame perfect equilibrium, although the payoff functions are upper-semicontinuous.0 Multi-player multi-outcome infinite sequential games ababa(2 , y z y z y Proof. If the payoffs are (2 , b can improve her payoff as late as required, so there areinfinitely many ”right” choices in a subgame perfect equilibrium. If the payoffs are not (2 , a chooses ”right” at every node that she owns, so that Player b chooses ”left”, butthen Player a chooses ”left” too.In [7], an intricate counterexample is provided showing that subgame-perfect equilibria mayeven fail to exist for probabilistic strategies (in a two-player game with Borel measurable payofffunctions). There is one open question regarding infinite sequential games with real-valued payoff functions(see the question mark in the Overview table on page 3), namely: Open Question 36. Do games with countably many players and upper-semicontinuous payofffunctions have ε -subgame perfect equilibria?It seems surprising to have both Nash equilibria and ε -subgame perfect equilibria, but nosubgame perfect equilibria – but this is precisely the situation for finitely many players. Onthe other hand, given the split between finitely many players and countably many players forlower-semicontinuous payoff functions, one should be cautious about assuming that this resultwould extend. Thus, we do not present a conjecture regarding the answer to the open question.The results in this paper are generally not constructive – but neither is Nash’s theorem in[28], cf. [31]. The extent of non-constructivity is investigated in [23, 22].The condition on the payoff functions used in Section 4 seems to merit further investigation.This was that for any sequence ( p i ) i ∈ N converging to p in C ω , we find that ∀ i ∈ N v ( p i ) ≺ v ( p i +1 )implies ∀ i ∈ N v ( p i ) ≺ v ( p ). This is a weaker condition than continuity of the function wherethe upper order topology is used on the codomain, which still seems to be strong enough toformulate some results. In a sense, it is a weakening of continuity that is orthogonal to Borel-measurability. As an example, a result by Gregoriades (reported in [33]) shows that anyfunction of this type from Baire space to the countable ordinals has to be bounded.In settings inspired by verification, sequential games often take place on a finite graph ratherthan an infinite tree. In this case, rather than seeking Nash equilibria built from arbitrarystrategies, it is desirable to obtain Nash equilibria built from strategies realizable by finiteautomata. The high-level ideas of this paper can be translated (with some effort) into thissetting, as demonstrated by the authors in [24]. . Le Roux & A. Pauly References [1] D. Blackwell (1969): Infinite G δ games with imperfect information . Zastowania Matematyki Appli-cationes Mathematicae .[2] Roderick Bloem, Krishnendu Chatterjee, Thomas A. Henzinger & Barbara Jobstmann (2009): BetterQuality in Synthesis through Quantitative Objectives . In Ahmed Bouajjani & Oded Maler, editors: Computer Aided Verification , Lecture Notes in Computer Science Energy parity games . Theor. Comput. Sci. Mean-Payoff ParityGames . In: Proceedings, Twentieth Annual IEEE Symposium on Logic in Computer Science, LICS2005 , IEEE Computer Society Press, pp. 178–187.[5] Pieter Collins (2014): Computable Stochastic Processes . arXiv:1409.4667.[6] Mart´ın Escard´o (2004): Synthetic topology of datatypes and classical spaces . Electronic Notes inTheoretical Computer Science Non-existence of subgame-perfect ε -equilibrium in perfect informa-tion games with infinite horizon . International Journal of Game Theory http://dx.doi.org/10.1007/s00182-014-0412-3 .[8] J´anos Flesch, Jeroen Kuipers, Ayala Mashiah-Yaakovi, Gijs Schoenmakers, Eilon Solan & KoosVrieze (2010): Perfect-Information Games with Lower-Semicontinuous Payoffs . Mathematics ofOperations Research Subgame-perfect equilibria of finite-and infinite-horizon games . Journal of Economic Theory Infinite games with perfect information . In: Contributions to thetheory of games , Annals of Mathematical Studies 28, Princeton University Press, pp. 245–266.[11] Erich Gr¨adel & M. Ummels (2008): Solution concepts and algorithms for Infinite multiplayer game .In K. Apt & R. van Rooij, editors: New Perspectives on Games and Interaction , Texts in Logic andGames 4, Amsterdam University Press, pp. 151–178.[12] Julian Gutierrez & Glynn Winskel (2013): Borel Determinacy of Concur-rent Games . In: CONCUR 2013 - Concurrency Theory - 24th Interna-tional Conference , pp. 516–530, doi:10.1007/978-3-642-40184-8 36. Available at http://dx.doi.org/10.1007/978-3-642-40184-8_36 .[13] Julian Gutierrez & Glynn Winskel (2014): On the determinacy of concurrent games on event struc-tures with infinite winning sets . Journal of Computer and System Sciences http://dx.doi.org/10.1016/j.jcss.2014.04.005 .[14] Thomas Krieger (2003): On Pareto equilibria in vector-valued extensive form games . MathematicalMethods of Operations Research 58, pp. 449–458.[15] Harold W. Kuhn (1953): Extensive games and the problem of information . Contributions to theTheory of Games II .[16] St´ephane Le Roux (2008): Generalisation and formalisation in game theory . Ph.D. thesis, EcoleNormale Sup´erieure de Lyon.[17] St´ephane Le Roux (2009): Acyclic preferences and existence of sequential Nash equilibria: a formaland constructive equivalence . In: TPHOLs, International Conference on Theorem Proving in HigherOrder Logics , Lecture Notes in Computer Science, Springer, pp. 293–309. Multi-player multi-outcome infinite sequential games [18] St´ephane Le Roux (2013): Infinite Sequential Nash Equilibria . Logical Methods in Computer Science From winning strategy to Nash equilibrium . Mathemat-ical Logic Quarterly http://dx.doi.org/10.1002/malq.201300034 . ArXiv 1203.1866.[20] St´ephane Le Roux (2016): Infinite Subgame Perfect Equilibrium in the Hausdorff Difference Hierar-chy . In Taghi Mohammad Hajiaghayi & Reza Mohammad Mousavi, editors: Topics in TheoreticalComputer Science: The First IFIP WG 1.8 International Conference, TTCS 2015, Tehran, Iran,August 26-28, 2015, Revised Selected Papers , Springer International Publishing, pp. 147–163. Alsoavailable at http://arxiv.org/abs/1505.06320.[21] St´ephane Le Roux & Arno Pauly (2014): Infinite Sequential Games with Real-valued Pay-offs . In: CSL-LICS ’14 , ACM, pp. 62:1–62:10, doi:10.1145/2603088.2603120. Available at http://doi.acm.org/10.1145/2603088.2603120 .[22] St´ephane Le Roux & Arno Pauly (2014): Weihrauch degrees of finding equilibria in sequential games .arXiv:1407.5587.[23] St´ephane Le Roux & Arno Pauly (2015): Weihrauch Degrees of Finding Equilibria in SequentialGames . In Arnold Beckmann, Victor Mitrana & Mariya Soskova, editors: Evolving Computability , Lecture Notes in Computer Science http://dx.doi.org/10.1007/978-3-319-20028-6_25 .[24] St´ephane Le Roux & Arno Pauly (2016): Extending finite memory determinacy: General techniquesand an application to energy parity games . arXiv:1602.08912.[25] Donald A. Martin (1975): Borel Determinacy . Annals of Mathematics .[26] Donald A. Martin (1998): The Determinacy of Blackwell Games . Journal of Symbolic Logic Repeated Games . In: Proc. Internat. Congress Mathematicians ,American Mathematical Society, pp. 1528–1577.[28] John F. Nash (1950): Equilibrium Points in N-Person Games . Proceedings of the National Academyof Sciences .[29] Rabia Nessah & Kristiaan Kerstens (2008): Characterizations of the Existence of Nash Equilib-ria with Non-convex Strategy Sets . Working Papers 2008-ECO-13, IESEG School of Management.Available at http://ideas.repec.org/p/ies/wpaper/e200813.html .[30] Noam Nisan, Tim Roughgarden, ´Eva Tardos & Vijay Vazirani, editors (2007): Algorithmic GameTheory . Cambridge University Press.[31] Arno Pauly (2010): How Incomputable is Finding Nash Equilibria? Journal of Universal ComputerScience On the topological aspects of the theory of represented spaces .http://arxiv.org/abs/1204.3763.[33] Arno Pauly (2015): Computability on the countable ordinals and the Hausdorff-Kuratowski theorem .arXiv 1501.00386.[34] Arno Pauly & Matthew de Brecht (2013): Towards Synthetic Descriptive Set Theory: An instanti-ation with represented spaces . http://arxiv.org/abs/1307.1850.[35] Rogers Purves & William Sudderth (2011): Perfect Information Games with Upper SemicontinuousPayoffs . Mathematics of Operations Research Introduction to the Inefficiency of Equilibria . In: [30] ,Cambridge University Press, pp. 443–459.[37] Victor L. Selivanov (2004): Difference hierarchy in ϕ -spaces . Algebra and Logic . Le Roux & A. Pauly [38] Reinhard Selten (1965): Spieltheoretische Behandlung eines Oligopolmodells mit Nachfragetr¨agheit . Z. Gesamte Staatswissenschaft 12, pp. 301–324.[39] Eilon Solan & Nicolas Vieille (2003): Deterministic multi-player Dynkin games . Journal of Mathe-matical Economics Vergleich nicht konstruktiv l¨osbarer Probleme in der Analysis . Diplo-marbeit, Fachbereich Informatik, FernUniversit¨at Hagen.[41] M. Ummels (2005): Rational behaviour and strategy construction in infinite multiplayer games .Diploma thesis, RWTH Aachen.[42] Klaus Weihrauch (2000): Computable Analysis . Springer-Verlag. Acknowledgements We are very grateful to Vassilios Gregoriades and the anonymous referees of the conferenceversion for their helpful comments. This work benefited from the Royal Society InternationalExchange Grant IE111233. The authors participated in the Marie Curie International ResearchStaff Exchange Scheme