Weak Subgame Perfect Equilibria and their Application to Quantitative Reachability
Thomas Brihaye, Véronique Bruyère, Noémie Meunier, Jean-François Raskin
WWeak Subgame Perfect Equilibria and theirApplication to Quantitative Reachability ∗ Thomas Brihaye , V´eronique Bruy`ere , No´emie Meunier † , andJean-Fran¸cois Raskin ‡ Abstract
We study n -player turn-based games played on a finite directed graph. For each play, theplayers have to pay a cost that they want to minimize. Instead of the well-known notionof Nash equilibrium (NE), we focus on the notion of subgame perfect equilibrium (SPE), arefinement of NE well-suited in the framework of games played on graphs. We also studynatural variants of SPE, named weak (resp. very weak) SPE, where players who deviatecannot use the full class of strategies but only a subclass with a finite number of (resp. aunique) deviation step(s).Our results are threefold. Firstly, we characterize in the form of a Folk theorem the setof all plays that are the outcome of a weak SPE. We also establish a weaker version of thistheorem for SPEs. Secondly, for the class of quantitative reachability games, we prove theexistence of a finite-memory SPE and provide an algorithm for computing it (only existencewas known with no information regarding the memory). Moreover, we show that the existenceof a constrained SPE, i.e. an SPE such that each player pays a cost less than a given constant,can be decided. The proofs rely on our Folk theorem for weak SPEs (which coincide withSPEs in the case of quantitative reachability games) and on the decidability of MSO logic oninfinite words. Finally with similar techniques, we provide a second general class of games forwhich the existence of a (constrained) weak SPE is decidable. B.6.3 [design aids]: automatic synthesis; F.1.2 [Modes ofcomputation]: interactive and reactive computation
Keywords and phrases multi-player games on graphs, quantitative objectives, Nash equilib-rium, subgame perfect equilibrium, quantitative reachability
Two-player zero-sum infinite duration games played on graphs are a mathematical modelused to formalize several important problems in computer science. Reactive system synthesisis one such important problem. In this context, see e.g. [13], the vertices and the edges of thegraph represent the states and the transitions of the system; one player models the systemto synthesize, and the other player models the (uncontrollable) environment of the system. ∗ This work has been partly supported by European project Cassting (FP7-ICT-601148). † Author supported by F.R.S.-FNRS fellowship. ‡ Author supported by ERC Starting Grant (279499: inVEST). © Thomas Brihaye, V´eronique Bruy`ere, No´emie Meunier and Jean-Fran¸cois Raskin;licensed under Creative Commons License CC-BYLeibniz International Proceedings in InformaticsSchloss Dagstuhl – Leibniz-Zentrum f¨ur Informatik, Dagstuhl Publishing, Germany a r X i v : . [ c s . G T ] A p r Weak Subgame Perfect Equilibria in Quantitative Games
In the classical setting, the objectives of the two players are opposite, i.e. the environment is adversarial . Modeling the environment as fully adversarial is usually a bold abstraction ofreality and there are recent works that consider the more general setting of non zero-sumgames which allow to take into account the different objectives of each player. In this lattersetting the environment has its own objective which is most often not the negation of theobjective of the system. The concept of
Nash equilibrium (NE) [11] is central to the study ofnon zero-sum games and can be applied to the general setting of n player games. A strategyprofile is a NE if no player has an incentive to deviate unilaterally from his strategy, since hecannot strictly improve on the outcome of the strategy profile by changing his strategy only.However in the context of sequential games (such as games played on graphs), it iswell-known that NEs present a serious weakness: a NE allows for non-credible threats thatrational players should not carry out [15]. As a consequence, for sequential games, the notionof NE has been strengthened into the notion of subgame perfect equilibrium (SPE): a strategyprofile is an SPE if it is a NE in all the subgames of the original game. While the notionof SPE is rather well understood for finite state game graphs with ω -regular objectives orfor games in finite extensive form (finite game trees), less is known for game graphs with quantitative objectives in which players encounter costs that they want to minimize, likein classical quantitative objectives such as mean-payoff, discounted sum, or quantitativereachability.Several natural and important questions arise for such games: Can we decide the existenceof an SPE, and more generally the constrained existence of an SPE (i.e. an SPE in whicheach player encounters a cost less than some fixed value)? Can we compute such SPEsthat use finite-memory strategies only? Whereas several work has studied what are thehypotheses to impose on games in a way to guarantee the existence of an SPE, the previousalgorithmic questions are still widely open. In this article, we provide progresses in theunderstanding of the notion of SPE. We first establish Folk theorems that characterize thepossible outcomes of SPEs in quantitative games for SPEs and their variants. We thenderive from this characterization interesting algorithms and information on the strategiesfor two important classes of quantitative games. Our contributions are detailed in the nextparagraph. Contributions
First, we formalize a notion of deviation step from a strategy profilethat allows us to define two natural variants of NEs. While a NE must be resistant to theunilateral deviation of one player for any number of deviation steps, a weak (resp. very weak )NE must be resistant to the unilateral deviation of one player for any finite number of (resp.a unique) deviation step(s). Then we use those variants to define the corresponding notionsof weak and very weak
SPE. The latter notion is very close to the one-step deviation property[12]. Any very weak SPE is also a weak SPE, and there are games for which there exists aweak SPE but no SPE. Also, for games with upper-semicontinuous cost functions and forgames played on finite game trees, the three notions are equivalent.Second, we characterize in the form of a Folk theorem all the possible outcomes of weakSPEs. The characterization is obtained starting from all possible plays of the game andthe application of a nonincreasing operator that removes plays that cannot be outcome ofa weak SPE. We show that the limit of the nonincreasing chain of sets always exists andcontains exactly all the possible outcomes of weak SPEs. Furthermore, we show how for eachsuch outcome, we can associate a strategy profile that generates it and which is a weak SPE.Using a variant of the techniques developed for weak SPEs, we also obtain a weaker versionof this Folk theorem for SPEs.Additionally, to illustrate the potential of our Folk theorem, we show how it can be h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 3 refined and used to answer open questions about two classes of quantitative games. The firstclass of games that we consider are quantitative reachability games , such that each playeraims at reaching his own set of target states as soon as possible. As the cost functionsin those games are continuous, our Folk theorem characterizes precisely the outcomes ofSPEs and not only weak SPEs. In [1, 6], it has been shown that quantitative reachabilitygames always have SPEs. The proof provided for this theorem is non constructive since itrelies on topological arguments. Here, we strengthen this existential result by proving thatthere always exists, not only an SPE but, a finite-memory
SPE. Furthermore, we provide analgorithm to construct such a finite memory SPE. This algorithm is based on a constructiveversion of our Folk Theorem for the class of quantitative reachability games: we show thatthe nonincreasing chain of sets of potential outcomes stabilizes after a finite number of stepsand that each intermediate set is an ω -regular set that can be effectively described usingMSO sentences. The second class of games that we consider is the class of games with costfunctions that are prefix-independent , whose range of values is finite , and for which eachvalue has an ω -regular pre-image. For this general class of games, with similar techniques asfor quantitative reachability games, we show how to construct an effective representation ofall possible outcomes compatible with a weak SPE, and consequently that the existence of aweak SPE is decidable. In those two applications, we show that our construction also allowus to answer the question of existence of a constrained (weak) SPE, i.e. a (weak) SPE inwhich players pays a cost which is bounded by a given value. Related work
The concept of SPE has been first introduced and studied by the gametheory community. The notion of SPE has been first introduced by Kuhn in finite extensiveform games [9]. For such games, backward induction can be used to prove that there alwaysexist an SPE. By inspecting the backward induction proof, it is not difficult to realize thatthe notion of very weak SPE and SPE are equivalent in this context.SPEs for infinite trees defined as the unfolding of finite graphs with qualitative , i.e.win-lose, ω -regular objectives, have been studied by Ummels in [18]: it is proved that suchgames always have an SPE, and that the existence of a constrained SPE is decidable.In [8], Klimos et al. provide an effective representation of the outcomes of NEs inconcurrent priced games by constructing a B¨uchi automaton accepting precisely the languageof outcomes of all NEs satisfying a bound vector. The existence of NEs in quantitative gamesplayed on graphs is studied in [2]; it is shown that for a large class of games, there alwaysexists a finite-memory NE. This result is extended in [3] for two-player games and secureequilibria (a refinement of NEs); additionally the constrained existence problem for secureequilibria is also shown decidable for a large range of cost functions. None of these referencesconsider SPEs.In [5], the authors prove that for quantitative games with cost functions that are upper-semicontinus and with finite range, there always exits an SPE. This result also relies on anonincreasing chain of sets of possible outcomes of SPEs. The main differences with ourwork is that we obtain a Folk theorem that characterize all possible outcomes of weak SPEswith no restriction on the cost functions. Moreover we have shown that our Folk theoremcan be made effective for two classes of quantitative games of interest. Effectiveness issuesare not considered in [5]. Prior to this work, Mertens shows in [10] that if the cost functionsare bounded and Borel measurable then there always exists an (cid:15) -NE. In [6], Fudenberg et al.show that if the cost functions are all continous, then there always exists an SPE. Thoseworks were recently extended in [14] by Le Roux and Pauly. Organization of the article
In Section 2, we present the notions of quantitative game,classical NE and SPE, and their variants. In Section 3, we propose and prove our Folk
Weak Subgame Perfect Equilibria in Quantitative Games
Theorems for weak SPEs and for SPEs. In Section 4, we provide an algorithm for computinga finite-memory SPE for quantitative reachability games, and a second algorithm to decidethe constrained existence of an SPE for this class of games. In Section 5, we show that theexistence of a (constrained) weak SPE is decidable for another class of games. A conclusionand future work are given in the last section.
In this section, we recall the notions of quantitative game, Nash equilibrium, and subgameperfect equilibrium. We also introduce variants of Nash and subgame perfect equilibria, andcompare them with the classical notions.
We consider multi-player turn-based non zero-sum quantitative games in which, for eachinfinite play, players pay a cost that they want to minimize. (cid:73) Definition 1. A quantitative game is a tuple G = (Π , V, ( V i ) i ∈ Π , E, ¯ λ ) where: Π is a finite set of players, V is a finite set of vertices, ( V i ) i ∈ Π is a partition of V such that V i is the set of vertices controlled by player i ∈ Π , E ⊆ V × V is a set of edges, such that for all v ∈ V , there exists v ∈ V with ( v, v ) ∈ E , ¯ λ = ( λ i ) i ∈ Π is a cost function such that λ i : V ω → R ∪ { + ∞} is player i cost function.A play of G is an infinite sequence ρ = ρ ρ . . . ∈ V ω such that ( ρ i , ρ i +1 ) ∈ E for all i ∈ N . Histories of G are finite sequences h = h . . . h n ∈ V + defined in the same way. The length | h | of h is the number n of its edges. We denote by First( h ) (resp. Last( h ) ) the first vertex h (resp. last vertex h n ) of h . Usually histories are non-empty, but in specific situations itwill be useful to consider the empty history (cid:15) . The set of all histories (ended by a vertexin V i ) is denoted by Hist (by
Hist i ). A prefix (resp. suffix ) of a play ρ is a finite sequence ρ . . . ρ n (resp. infinite sequence ρ n ρ n +1 . . . ) denoted by ρ ≤ n or ρ Notice that the cost function ¯ λ of a quantitative game is often defined from | Π | -uples ofweights labeling the edges of the game. For instance, in inf games, λ i ( ρ ) is equal to theinfimum of player i weights seen along ρ . Some other classical examples are liminf, limsup,mean-payoff, and discounted sum games [4]. In case of quantitative reachability on graphswith weighted edges, the cost λ i ( ρ ) for player i is replaced by the sum of the weights seenalong ρ until his target set is reached. We do not consider this extension here. Notice thatwhen weights are positive integers, replacing each edge with cost c by a path of length c composed of c new edges allows to recover Definition 2.Let us recall the notions of prefix-independent, continuous, and lower- (resp. upper-)semicontinuous cost functions. Since V is endowed with the discrete topology, and thus V ω with the product topology, a sequence of plays ( ρ n ) n ∈ N converges to a play ρ = lim n →∞ ρ n if every prefix of ρ is prefix of all ρ n except, possibly, of finitely many of them. (cid:73) Definition 3. Let λ i be a player i cost function. Then λ i is prefix-independent if λ i ( hρ ) = λ i ( ρ ) for any history h and play ρ . λ i is continuous if whenever lim n →∞ ρ n = ρ , then lim n →∞ λ i ( ρ n ) = λ i ( ρ ) . λ i upper-semicontinuous (resp. lower-semicontinuous ) if whenever lim n →∞ ρ n = ρ , then lim sup n →∞ λ i ( ρ n ) ≤ λ i ( ρ ) (resp. lim inf n →∞ λ i ( ρ n ) ≥ λ i ( ρ ) ).For instance, the cost functions used in liminf and mean-payoff games are prefix-independent,contrarily to the case of inf games. Clearly, if λ i is continuous, then it is upper- and lower-semicontinuous. For instance, the cost functions of liminf and mean-payoff games are neitherupper-semicontinuous nor lower-semicontinuous, whereas cost functions of discounted sumgames are continuous. The cost functions λ i , i ∈ Π , used in quantitative reachability gamescan be transformed into continuous ones as follows [1]: λ i ( ρ ) = 1 − λ i ( ρ )+1 if λ i ( ρ ) < + ∞ ,and λ i ( ρ ) = 1 otherwise. A strategy σ for player i ∈ Π is a function σ : Hist i → V assigning to each history hv ∈ Hist i a vertex v = σ ( hv ) such that ( v, v ) ∈ E . In an initialized game ( G , v ) , σ is restricted tohistories starting with v . A player i strategy σ is positional if it only depends on the lastvertex of the history, i.e. σ ( hv ) = σ ( v ) for all hv ∈ Hist i . It is a finite-memory strategy if itneeds only finite memory of the history (recorded by a finite strategy automaton, also calleda Moore machine). A play ρ is consistent with a player i strategy σ if ρ k +1 = σ ( ρ ≤ k ) for all k such that ρ k ∈ V i . A strategy profile of G is a tuple ¯ σ = ( σ i ) i ∈ Π of strategies, where each σ i isa player i strategy. It is called positional (resp. finite-memory ) if all σ i , i ∈ Π , are positional(resp. finite-memory). Given an initial vertex v , such a strategy profile determines a uniqueplay of ( G , v ) that is consistent with all the strategies. This play is called the outcome of ¯ σ and is denoted by h ¯ σ i v .Given σ i a player i strategy, we say that player i deviates from σ i if he does not stickto σ i and prefers to use another strategy σ i . Let ¯ σ be a strategy profile. When all playersstick to their strategy σ i except player i that shifts to σ i , we denote by ( σ i , σ − i ) the derivedstrategy profile, and by h σ i , σ − i i v its outcome in ( G , v ) . In the next definition, we introducethe notion of deviation step of a strategy σ i from a given strategy profile ¯ σ . In this article we often write a history in the form hv with v ∈ V to emphasize that v is the lastvertex of this history. Weak Subgame Perfect Equilibria in Quantitative Games (cid:73) Definition 4. Let ( G , v ) be an initialized game, ¯ σ be a strategy profile, and σ i be a player i strategy. We say that σ i has a hv -deviation step from ¯ σ for some history hv ∈ Hist i ( v ) with v ∈ V i , if hv < h σ i , σ − i i v and σ i ( hv ) = σ i ( hv ) . Notice that the previous definition requires that hv is a prefix of the outcome h σ i , σ − i i v ; itsays nothing about σ i outside of this outcome. A strategy σ i can have a finite or an infinitenumber of deviation steps in the sense of Definition 4. A strategy with three deviationsteps is depicted in Figure 1 (left) such that each h k v k -deviation step from ¯ σ , ≤ k ≤ , ishighlighted with a dashed edge. We will come back to this figure later on. v v v v v v v h σ i ,σ − i i v h h h v v h ·h ¯ σ (cid:22) h i v = h ¯ σ i v v v h ·h ¯ σ (cid:22) h i v = h v ·h ¯ σ (cid:22) h v i v v v h ·h ¯ σ (cid:22) h i v = h v ·h ¯ σ (cid:22) h v i v v h σ i ,σ − i i v = h v ·h ¯ σ (cid:22) h v i v h h h Figure 1 A strategy σ i with a finite number of deviation steps. In light of Definition 4, we introduce the following classes of strategies. (cid:73) Definition 5. Let ( G , v ) be an initialized game, and ¯ σ be a strategy profile.A strategy σ i is finitely deviating from ¯ σ if it has a finite number of deviation steps from ¯ σ .It is one-shot deviating from ¯ σ if it has a v -deviation step from ¯ σ , and no other deviationstep.In other words, a strategy σ i is finitely deviating from ¯ σ if there exists a history hv < h σ i , σ − i i v such that for all h v , hv ≤ h v < h σ i , σ − i i v , we have σ i ( h v ) = σ i ( h v ) ( σ i actsas σ i from hv along h σ i , σ − i i v ). The strategy σ i is one-shot deviating from ¯ σ if it differsfrom σ i at the initial vertex v , and after v acts as σ i along h σ i , σ − i i v . As for Definition 4,the previous definition says nothing about σ i outside of h σ i , σ − i i v . Clearly any one-shotdeviating strategy is finitely deviating. The strategy of Figure 1 is finitely deviating but notone-shot deviating. h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 7 In this paper, we focus on subgame perfect equilibria and their variants. Let us first recallthe classical notion of Nash equilibrium. A strategy profile ¯ σ in an initialized game is a Nashequilibrium if no player has an incentive to deviate unilaterally from his strategy, since hecannot strictly decrease his cost when using any other strategy. (cid:73) Definition 6. Given an initialized game ( G , v ) , a strategy profile ¯ σ = ( σ i ) i ∈ Π of ( G , v ) is a Nash equilibrium (NE) if for all players i ∈ Π , for all player i strategies σ i , we have λ i ( h σ i , σ − i i v ) ≥ λ i ( h ¯ σ i v ) .We say that a player i strategy σ i is a profitable deviation for i w.r.t. ¯ σ if λ i ( h σ i , σ − i i v ) <λ i ( h ¯ σ i v ) . Therefore ¯ σ is a NE if no player has a profitable deviation w.r.t. ¯ σ .Let us propose the next variants of NE. (cid:73) Definition 7. Let ( G , v ) be an initialized game. A strategy profile ¯ σ is a weak NE (resp. very weak NE ) in ( G , v ) if, for each player i ∈ Π , for each finitely deviating (resp. one-shotdeviating) strategy σ i of player i , we have λ i ( h σ i , σ − i i v ) ≥ λ i ( h ¯ σ i v ) . (cid:73) Example 8. Consider the two-player quantitative game depicted in Figure 2. Circle (resp.square) vertices are player (resp. player ) vertices. The edges are labeled by couples ofweights such that weights (0 , are not specified. For each player i , the cost λ i ( ρ ) of a play ρ is the weight of its ending loop. In this simple game, each player i have two positionalstrategies that are respectively denoted by σ i and σ i (see Figure 2). v v v v v σ σ (3 , σ σ (3 , 4) (1 , Figure 2 A simple two-player quantitative game The strategy profile ( σ , σ ) is not a NE since σ is a profitable deviation for player w.r.t. ( σ , σ ) (player pays cost instead of cost ). This strategy profile is neither a weakNE nor a very weak NE because in this simple game, player can only deviate from σ byusing the one-shot deviating strategy σ . On the contrary, the strategy profile ( σ , σ ) is aNE with outcome v v ω of cost (3 , . It is also a weak NE and a very weak NE.By definition, any NE is a weak NE, and any weak NE is a very weak NE. The contraryis false: in the previous example, ( σ , σ ) is a very weak NE, but not a weak NE. We will seelater an example of game with a weak NE that is not an NE (see Example 12).The notion of subgame perfect equilibrium is a refinment of NE. In order to define it, weneed to introduce the following notions. Given a quantitative game G = (Π , V, ( V i ) i ∈ Π , E, ¯ λ ) and a history h of G , we denote by G (cid:22) h the game G (cid:22) h = (Π , V, ( V i ) i ∈ Π , E, ¯ λ (cid:22) h ) where ¯ λ (cid:22) h ( ρ ) = ¯ λ ( hρ ) Weak Subgame Perfect Equilibria in Quantitative Games for any play of G (cid:22) h , and we say that G (cid:22) h is a subgame of G . Given an initialized game ( G , v ) ,and a history hv ∈ Hist( v ) , the initialized game ( G (cid:22) h , v ) is called the subgame of ( G , v ) with history hv . Notice that ( G , v ) can be seen as a subgame of itself with history hv suchthat h = (cid:15) . Given a player i strategy σ in ( G , v ) , we define the strategy σ (cid:22) h in ( G (cid:22) h , v ) as σ (cid:22) h ( h ) = σ ( hh ) for all histories h ∈ Hist i ( v ) . Given a strategy profile ¯ σ = ( σ i ) i ∈ Π , we usenotation ¯ σ (cid:22) h for ( σ i (cid:22) h ) i ∈ Π , and h ¯ σ (cid:22) h i v is its outcome in the subgame ( G (cid:22) h , v ) .We can now recall the classical notion of subgame perfect equilibrium: it is a strategyprofile in an initialized game that induces a NE in each of its subgames. In particular, asubgame perfect equilibrium is a NE. (cid:73) Definition 9. Given an initialized game ( G , v ) , a strategy profile ¯ σ of ( G , v ) is a subgameperfect equilibrium (SPE) if ¯ σ (cid:22) h is a NE in ( G (cid:22) h , v ) , for every history hv ∈ Hist( v ) .As for NE, we propose the next variants of SPE. (cid:73) Definition 10. Let ( G , v ) be an initialized game. A strategy profile ¯ σ is a weak SPE (resp. very weak SPE ) if ¯ σ (cid:22) h is a weak NE (resp. very weak NE) in ( G (cid:22) h , v ) , for all histories hv ∈ Hist( v ) . (cid:73) Example 11. We come back to the game depicted in Figure 2. We have seen beforethat the strategy profile ( σ , σ ) is a NE. However it is not an SPE. Indeed consider thesubgame ( G (cid:22) v , v ) of ( G , v ) with history v v . In this subgame, σ is a profitable deviationfor player . One can easily verify that the strategy profile ( σ , σ ) is an SPE, as well as aweak SPE and a very weak SPE, due to the simple form of the game.The previous example is too simple to show the differences between classical SPEs andtheir variants. The next example presents a game with a (very) weak SPE but no SPE. (cid:73) Example 12. Consider the initialized two-player game ( G , v ) in Figure 3. The edges arelabeled by couples of weights, and for each player i the cost λ i ( ρ ) of a play ρ is the uniqueweight seen in its ending cycle. With this definition, λ i ( ρ ) can also be seen as either themean-payoff, or the liminf, or the limsup, of the weights of ρ . It is known that this game hasno SPE [16].Let us show that the positional strategy profile ¯ σ depicted with thick edges is a veryweak SPE. Due to the simple form of the game, only two cases are to be treated. Considerthe subgame ( G (cid:22) h , v ) with h ∈ ( v v ) ∗ , and the one-shot deviating strategy σ of player such that σ ( v ) = v . Then h ¯ σ (cid:22) h i v = v v v ω and h σ , σ (cid:22) h i v = v v ω , showing that σ isnot a profitable deviation for player . One also checks that in the subgame ( G (cid:22) h , v ) with h ∈ ( v v ) ∗ v , the one-shot deviating strategy σ of player such that σ ( v ) = v is notprofitable for him.Similarly, one can prove that ¯ σ is a weak SPE (see also Proposition 13 hereafter). Noticethat ¯ σ is not an SPE. Indeed the strategy σ such that σ ( hv ) = v for all h , is a profitabledeviation for player 2 in ( G , v ) . This strategy is (of course) not finitely deviating. Finallynotice that ¯ σ is a weak NE that is not an NE.From Definition 10, any SPE is a weak SPE, and any weak SPE is a very weak SPE. Thenext proposition states that weak SPE and very weak SPE are equivalent notions, but this isno longer true for SPE and weak SPE as shown previously by Example 12. In this article, we will always use notation ¯ λ ( hρ ) instead of ¯ λ (cid:22) h ( ρ ) . h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 9 v v v v (2 , , 0) (2 , 0) (0 , , 2) (0 , Figure 3 A two-player game with a (very) weak SPE and no SPE. For each player, the cost of aplay is his unique weight seen in the ending cycle. (cid:73) Proposition 13. Let ( G , v ) be an initialized game, and ¯ σ be a strategy profile. Then ¯ σ is a weak SPE iff ¯ σ is a very weak SPE.There exists an initialized game ( G , v ) with a weak SPE but no SPE. Before proving this proposition, we would like to come back to the definition of deviationstep (Definition 4), and explain it now with the concept of subgame. Given an initialized game ( G , v ) , a strategy profile ¯ σ , and a player i strategy σ i , we recall that σ i has a hv -deviationstep from ¯ σ for some hv ∈ Hist i ( v ) , if hv < h σ i , σ − i i v and σ i ( hv ) = σ i ( hv ) . Equivalently, σ i has a hv -deviation step from ¯ σ iff h ·h ¯ σ (cid:22) h i v b h σ i , σ − i i v = hv. This alternative vision of deviation step is depicted in Figure 1 (right) for a strategy withthree deviation steps. For instance, for history h v , we have h v < h σ i , σ − i i v and σ i ( h v ) = σ i ( h v ) , or equivalently h ·h ¯ σ (cid:22) h i v b h σ i , σ − i i v = h v . Notice that there is nointermediate deviation step between the h v -deviation step and the h v -deviation stepsince h ·h ¯ σ (cid:22) h i v = h v ·h ¯ σ (cid:22) h v i v as indicated in the figure. Similarly the h v -deviationstep is the first one because h ·h ¯ σ (cid:22) h i v = h ¯ σ i v , and the h v -deviation step is the thirdone because h ·h ¯ σ (cid:22) h i v = h v ·h ¯ σ (cid:22) h v i v . The latter deviation step is the last one because h σ i , σ − i i v = h v ·h ¯ σ (cid:22) h v i v . Proof of Proposition 13. This proof is based on arguments from the one-step deviationproperty used to prove Kuhn’s theorem [9]. Let ¯ σ be a very weak SPE, and let us prove thatit is a weak SPE. As a contradiction, assume that there exists a subgame ( G (cid:22) h , v ) such thatthe strategy profile ¯ σ (cid:22) h is not a weak NE. This means that there exists a player i strategy σ i in ( G (cid:22) h , v ) such that σ i is finitely deviating from ¯ σ (cid:22) h and λ i ( hρ ) > λ i ( hρ ) , (1)where ρ = h ¯ σ (cid:22) h i v and ρ = h σ i , σ − i (cid:22) h i v . Let us consider such a strategy σ i with a minimumnumber n of deviation steps from h ¯ σ (cid:22) h i v , and let g k v k , ≤ k ≤ n , be the histories in Hist i ( v ) such that σ i has a g k v k -deviation step from h ¯ σ (cid:22) h i v . Let us consider the subgame ( G (cid:22) hg n , v n ) (see Figure 4). In this subgame, σ i (cid:22) g n is not a profitable one-shot deviating strategy as ¯ σ is avery weak SPE. In other words, for % = h ¯ σ (cid:22) hg n i v n and % = h σ i (cid:22) g n , σ − i (cid:22) hg n i v n , we have λ i ( hg n % ) ≤ λ i ( hg n % ) . (2)Notice that n ≥ . Indeed, if n = 1 , then ρ = g % , ρ = g % , and λ i ( hρ ) ≤ λ i ( hρ ) by(2). Therefore σ i is not a profitable deviation in ( G (cid:22) h , v ) , in contradiction with its definition(1). We can thus construct a strategy τ i from σ i such that these two strategies are the sameexcept in the subgame ( G (cid:22) hg n , v n ) where τ i (cid:22) g n and σ i (cid:22) hg n coincide. In other words τ i has n − deviation steps from h ¯ σ (cid:22) h i v , that are exactly the g k v k -deviation steps, ≤ k ≤ n − ,of σ i . Moreover, in the subgame ( G (cid:22) h , v ) , we have h τ i , σ − i (cid:22) h i v = g n % , and λ i ( hg n % ) ≤ λ i ( hg n % ) < λ i ( hρ ) vv hρ v n ( G (cid:22) h ,v ) hρ ( G (cid:22) h n ,v n ) hg Figure 4 A strategy σ i with a minimum number n of deviation steps. h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 11 by (1), (2), and g n % = ρ . It follows that τ i is a finitely deviating strategy that is profitablefor player i in ( G (cid:22) h , v ) , with less deviation steps than σ i , a contradiction. This completesthe proof of the first statement of Proposition 13. For the second statement, it is enough toconsider Example 12. (cid:74) Under the next hypotheses on the game or the costs, the equivalence between SPE, weakSPE, and very weak SPE holds. The first case, when the cost functions are continuous, is aclassical result in game theory, see for instance [7]; the second case appears as a part of theproof of Kuhn’s theorem [9]. (cid:73) Proposition 14. Let ( G , v ) be an initialized game, and ¯ σ be a strategy profile.If all cost functions λ i are continuous, or even upper-semicontinuous , then ¯ σ is an SPEiff ¯ σ is a weak SPE iff ¯ σ is a very weak SPE.If G is a finite tree , then ¯ σ is an SPE iff ¯ σ is a weak SPE iff ¯ σ is a very weak SPE. Proof. We only prove the first statement for cost functions λ i that are upper-semicontinuous.Let ¯ σ be a strategy profile in an initialized game ( G , v ) . By Proposition 13, it remains toprove that if ¯ σ is a weak SPE, then it is an SPE, i.e., for each subgame ( G (cid:22) h , v ) , the strategyprofile ¯ σ (cid:22) h is a NE. Let σ i be a player i strategy in ( G (cid:22) h , v ) . If σ i is finitely deviating, then itis not a profitable deviation for player i w.r.t. ¯ σ (cid:22) h by hypothesis. Therefore, suppose that ρ = h σ i , σ − i (cid:22) h i v = g g . . . g n . . . such that σ i has a h n -deviation step from ¯ σ (cid:22) h for all n ≥ , where h = (cid:15) and h n = h n − g n .For each n , we define a finitely deviating strategy τ ni such that its deviation steps arethe first n deviation steps of σ i , that is, τ ni and σ i are equal except in the subgame ( G (cid:22) h n , First( g n +1 )) where τ ni (cid:22) h n = σ i (cid:22) hh n . By hypothesis, τ ni is not a profitable deviation,and thus for ρ n = h τ ni , σ − i (cid:22) h i v we have λ i ( h ·h ¯ σ (cid:22) h i v ) ≤ λ i ( hρ n ) . (3)As lim n →∞ hρ n = hρ and λ i is upper-semicontinuous, we get lim sup n →∞ λ i ( hρ n ) ≤ λ i ( hρ ) .Therefore by (3), λ i ( h ·h ¯ σ (cid:22) h i v ) ≤ λ i ( hρ ) showing that σ i is not a profitable deviation forplayer i w.r.t. ¯ σ (cid:22) h . (cid:74) Recall that discounted sum games and quantitative reachability games are continuous.Thus for these games, the three notions of SPE, weak SPE and very weak SPE, are equivalent. (cid:73) Corollary 15. Let ( G , v ) be an initialized quantitative reachability game, and ¯ σ be astrategy profile. Then ¯ σ is an SPE iff ¯ σ is a weak SPE iff ¯ σ is a very weak SPE. On the opposite, the initialized game of Figure 3 has a weak SPE but no SPE. Its cost func-tion λ is not upper-semicontinuous as lim n →∞ ( v v ) n v ω = ( v v ) ω and lim n →∞ λ (( v v ) n v ω )= 1 > λ (( v v ) ω ) . In games where the players receive a payoff that they want to maximize, the hypothesis of upper-semicontinuity has to be replaced by lower-semicontinuity. In a finite tree game, the plays are finite sequences of vertices ending in a leaf and their cost isassociated with the ending leaf. An example of such a game is depicted in Figure 2. In this section, we characterize in the form of a Folk Theorem the set of all outcomes ofweak SPEs. Our approach is inspired by work [5] where a Folk Theorem is given for the setof outcomes of SPEs in games with cost functions that are upper-semicontinuous and havefinite range. In this aim we define a nonincreasing sequence of sets of plays that initiallycontain all the plays, and then loose, step by step, some plays that for sure are not outcomesof a weak SPE, until finally reaching a fixpoint.Let ( G , v ) be a game. For an ordinal α and a history hv ∈ Hist( v ) , let us consider theset P α ( hv ) = { ρ | ρ is a potential outcome of a weak NE in ( G (cid:22) h , v ) at step α } . This set isdefined by induction on α as follows: (cid:73) Definition 16. Let ( G , v ) be a quantitative game. The set P α ( hv ) is defined as followsfor each ordinal α and history hv ∈ Hist( v ) :For α = 0 , P α ( hv ) = { ρ | ρ is a play in ( G (cid:22) h , v ) } . (4)For a successor ordinal α + 1 , P α +1 ( hv ) = P α ( hv ) \ E α ( hv ) (5)such that ρ ∈ E α ( hv ) (see Figure 5) iffthere exists a history h , hv ≤ h < hρ , and Last( h ) ∈ V i for some i ,there exists a vertex v , h v < hρ ,such that ∀ ρ ∈ P α ( h v ) : λ i ( hρ ) > λ i ( h ρ ) .For a limit ordinal α : P α ( hv ) = \ β<α P β ( hv ) . (6)Notice that an element ρ of P α ( hv ) is a play in ( G (cid:22) h , v ) (and not in ( G , v ) ). Therefore itstarts with vertex v , and hρ is a play in ( G , v ) . For α + 1 being a successor ordinal, play ρ ∈ E α ( hv ) is erased from P α ( hv ) because for all ρ ∈ P α ( h v ) , player i pays a lower cost λ i ( h ρ ) < λ i ( hρ ) , which means that ρ is no longer a potentiel outcome of a weak NE in ( G (cid:22) h , v ) .The sequence ( P α ( hv )) α is nonincreasing by definition, and reaches a fixpoint in thefollowing sense. (cid:73) Proposition 17. There exists an ordinal α ∗ such that P α ∗ ( hv ) = P α ∗ +1 ( hv ) for allhistories hv ∈ Hist( v ) . Proof. Let us fix a history hv ∈ Hist( v ) . The sequence ( P α ( hv )) α reaches a fixpoint as soonas there exists α such that P α ( hv ) = P α +1 ( hv ) . Indeed it follows that P α +1 ( hv ) = P α +2 ( hv ) and then P α ( hv ) = P β ( hv ) for all β > α . As the sequence ( P α ( hv )) α is nonincreasing,this happens at the latest with α being equal to the cardinality of P ( hv ) . Therefore with α ∗ = | V ω | being an ordinal greater than or equal to the cardinality of the set of all plays of G , we get P α ∗ ( hv ) = P α ∗ +1 ( hv ) for all hv ∈ Hist( v ) . (cid:74) In the sequel, α ∗ always refers to the ordinal mentioned in Proposition 17.Our Folk Theorem for weak SPEs is the next one. Our approach is however different. h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 13 v v v ∈ V i ρ ∈ E α ( hv ) ∀ ρ ( G (cid:22) h ,v ) P α ( h v ) h h Figure 5 ρ ∈ E α ( hv ) . (cid:73) Theorem 18. Let ( G , v ) be a quantitative game. There exists a weak SPE in ( G , v ) withoutcome ρ iff P α ∗ ( hv ) = ∅ for all hv ∈ Hist( v ) , and ρ ∈ P α ∗ ( v ) . Before proving this theorem, we illustrate it with an example. (cid:73) Example 19. Consider the example of Figure 3. Clearly, as P ( hv ) only contains theplay v ω , then P α ( hv ) = { v ω } for all α . Similarly P α ( hv ) = { v ω } for all α . Let us detailthe computation of P α ( hv ) and P α ( hv ) . α = 0 . For history hv , we have ρ = ( v v ) ω ∈ E ( hv ) , since λ ( hρ ) > λ ( hv ρ ) where ρ = v ω is the unique play of P ( hv v ) . Similarly, for all n ≥ , we have ρ = ( v v ) n v v ω ∈ E ( hv ) , since λ ( hρ ) > λ ( hv v ρ ) where ρ = v ω is the unique playof P ( hv v v ) . Thus E ( hv ) = { ( v v ) ω } ∪ ( v v ) + v v ω and P ( hv ) = { v v ω } ∪ ( v v ) + v ω . For history hv , with the same kind of computations, we get E ( hv ) = { ( v v ) ω } ∪ ( v v ) + v ω and P ( hv ) = v ( v v ) ∗ v ω .α = 1 . For history hv , we have ρ = v v ω ∈ E ( hv ) . Indeed λ ( hρ ) > λ ( hv ρ ) for all ρ ∈ P ( hv v ) = v ( v v ) ∗ v ω . Notice that at the previous step, ρ = v v ω E ( hv ) .Indeed P ( hv v ) (cid:40) P ( hv v ) , and λ ( hρ ) ≤ λ ( hv ρ ) for ρ = ( v v ) ω ∈ P ( hv v ) . Play v v ω is the only one that is removed from P ( hv ) , and no play can be removedfrom P ( hv ) . Therefore: P ( hv ) = ( v v ) + v ω , P ( hv ) = v ( v v ) ∗ v ω . This shows that as the sequence ( P α ( hv )) α is nonincreasing for each hv , a play that is not removedfrom some P α ( hv ) can be removed later from some P β ( hv ) with β > α . α = 2 . One checks that P ( hv ) = P ( hv ) , and P ( hv ) = P ( hv ) . Hence thefixpoint is reached with α ∗ = 2 , with P α ∗ ( hv ) = ( v v ) + v ω , P α ∗ ( hv ) = v ( v v ) ∗ v ω , P α ∗ ( hv ) = { v ω } , and P α ∗ ( hv ) = { v ω } . Therefore, the set of outcomes of weak SPEsin this game is equal to ( v v ) + v ω . The weak SPE depicted in Figure 3 has outcome v v v ω .The proof of Theorem 18 follows from Lemmas 20 and 21. (cid:73) Lemma 20. If ( G , v ) has a weak SPE ¯ σ , then P α ∗ ( hv ) = ∅ for all hv ∈ Hist( v ) , and h ¯ σ i v ∈ P α ∗ ( v ) . Proof. Let us show, by induction on α , that h ¯ σ (cid:22) h i v ∈ P α ( hv ) for all hv ∈ Hist( v ) .For α = 0 , we have h ¯ σ (cid:22) h i v ∈ P α ( hv ) by definition of P ( hv ) .Let α + 1 be a successor ordinal. By induction hypothesis, we have that h ¯ σ (cid:22) h i v ∈ P α ( hv ) for all hv ∈ Hist( v ) . Suppose that there exists hv such that h ¯ σ (cid:22) h i v P α +1 ( hv ) , i.e. h ¯ σ (cid:22) h i v ∈ E α ( hv ) . This means that there is a history h = hg ∈ Hist i for some i ∈ Π with hv ≤ h < hρ , and there exists a vertex v with h v < hρ , such that ∀ ρ ∈ P α ( h v ) , λ i ( h ·h ¯ σ (cid:22) h i v ) > λ i ( h ρ ) . In particular, by induction hypothesis λ i ( h ·h ¯ σ (cid:22) h i v ) > λ i ( h ·h ¯ σ (cid:22) h i v ) . (7)Let us consider the player i strategy σ i in ( G (cid:22) h , v ) such that g ·h ¯ σ (cid:22) h i v is consistent with σ i .Then σ i is a finitely deviating strategy with the (unique) g -deviation step from ¯ σ (cid:22) h . Thisstrategy is a profitable deviation for player i in ( G (cid:22) h , v ) by (7), a contradiction with ¯ σ beinga weak SPE.Let α be a limit ordinal. By induction hypothesis h ¯ σ (cid:22) h i v ∈ P β ( hv ) , ∀ β < α . Therefore h ¯ σ (cid:22) h i v ∈ P α ( hv ) = T β<α P β ( hv ) . (cid:74)(cid:73) Lemma 21. Suppose that P α ∗ ( hv ) = ∅ for all hv ∈ Hist( v ) , and let ρ ∈ P α ∗ ( v ) . Then ( G , v ) has a weak SPE with outcome ρ . Proof. We are going to show how to construct a very weak SPE ¯ σ (and thus a weak SPEby Proposition 13) with outcome ρ . The construction of ¯ σ is done step by step thanks toa progressive labeling of the histories hv ∈ Hist( v ) . Let us give an intuitive idea of theconstruction of ¯ σ . Initially, we partially construct ¯ σ such that it produces an outcome in ( G , v ) equal to ρ ∈ P α ∗ ( v ) ; we also label each non-empty prefix of ρ by ρ . Then we considera shortest non-labeled history h v , and we correctly choose some ρ ∈ P α ∗ ( h v ) (we willsee later how). We continue the construction of ¯ σ such that it produces the outcome ρ in ( G (cid:22) h , v ) , and for each non-empty prefix g of ρ , we label h g by ρ (notice that the prefixesof h have already been labeled by choice of h ). And so on. In this way, the labeling is amap γ : Hist( v ) → S hv P α ∗ ( hv ) that allows to recover from h g the outcome ρ of ¯ σ (cid:22) h in ( G (cid:22) h , v ) of which g is prefix. Let us now go into the details.Initially, none of the histories is labeled. We start with history v and the given play ρ ∈ P α ∗ ( v ) . The strategy profile ¯ σ is partially defined such that h ¯ σ i v = ρ , that is, if ρ = ρ ρ . . . , then σ i ( ρ ≤ n ) = ρ n +1 for all ρ n ∈ V i and i ∈ Π . The non-empty prefixes h of ρ are all labeled with γ ( h ) = ρ .At the following steps, we consider a history h v that is not yet labeled, but such that h has already been labeled. By induction, γ ( h ) = h ¯ σ (cid:22) h i v such that hv ≤ h . Suppose that Last( h ) ∈ V i , we then choose a play ρ ∈ P α ∗ ( h v ) such that (see Figure 6) λ i ( h ·h ¯ σ (cid:22) h i v ) ≤ λ i ( h ρ ) . (8) h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 15 v v v ∈ V i h ¯ σ (cid:22) h i v ρ = h ¯ σ (cid:22) h i v P α ∗ ( h v ) hg h Figure 6 Construction of a very weak SPE ¯ σ . Such a play ρ exists for the following reasons. By induction, we know that h ¯ σ (cid:22) h i v ∈ P α ∗ ( hv ) .Since P α ∗ ( hv ) = P α ∗ +1 ( hv ) by Proposition 17, we have h ¯ σ (cid:22) h i v E α ∗ ( hv ) , and we get theexistence of ρ by definition of E α ∗ ( hv ) . We continue to construct ¯ σ such that h ¯ σ (cid:22) h i v = ρ ,i.e. if ρ = ρ ρ . . . , then σ i ( h ρ n ) = h ρ n +1 for all ρ n ∈ V i and i ∈ Π . For all non-emptyprefixes g of ρ , we define γ ( h g ) = ρ (notice that the prefixes of h have already beenlabeled).Let us show that the constructed profile ¯ σ is a very weak SPE. Consider a history hv ∈ Hist i for some i ∈ Π , and a one-shot deviating strategy σ i from ¯ σ (cid:22) h in the subgame ( G (cid:22) h , v ) . Let v be such that σ i ( v ) = v . By definition of ¯ σ , we have γ ( hv ) = h ¯ σ (cid:22) g i u for somehistory gu ≤ hv and h ·h ¯ σ (cid:22) h i v = g ·h ¯ σ (cid:22) g i u ; and we have also γ ( hvv ) = h ¯ σ (cid:22) hv i v . Moreover λ i ( g ·h ¯ σ (cid:22) g i u ) ≤ λ i ( hv ·h ¯ σ (cid:22) hv i v ) by (8), and λ i ( hv ·h ¯ σ (cid:22) hv i v ) = λ i ( h ·h σ i , σ − i (cid:22) h i v ) because σ i isone-shot deviating. Therefore λ i ( h ·h ¯ σ (cid:22) h i v ) = λ i ( g ·h ¯ σ (cid:22) g i u ) ≤ λ i ( hv ·h ¯ σ (cid:22) hv i v ) = λ i ( h ·h σ i , σ − i (cid:22) h i v ) which shows that ¯ σ (cid:22) h is a very weak NE in ( G (cid:22) h , v ) . Hence ¯ σ is a very weak SPE, and thusalso a weak SPE. (cid:74) The next lemma will be useful in Sections 4 and 5. It states that if a play ρ belongsto P α ( hv ) , then each of its suffixes ρ also belongs to P α ( hh v ) such that h ρ = ρ and v = First( ρ ) . (cid:73) Lemma 22. Let ρ ∈ P α ( hv ) . Then for all h ρ = ρ , we have ρ ∈ P α ( hh v ) with v = First( ρ ) . Proof. The proof is by induction on α . The lemma trivially holds for α = 0 by definition of P ( hv ) . Let α + 1 be a successor ordinal. Let ρ ∈ P α +1 ( hv ) and h ρ = ρ with v = First( ρ ) .As P α +1 ( hv ) ⊆ P α ( hv ) , by induction hypothesis, we have ρ ∈ P α ( hh v ) . Suppose that ρ ∈ E α ( hh v ) (hence using a history h and a vertex v as in Definition 16). Then one caneasily check by definition of E α ( hh v ) that ρ ∈ E α ( hv ) (by using the same h and v ), which isa contradiction with ρ ∈ P α +1 ( hv ) . Therefore ρ ∈ P α ( hh v ) \ E α ( hh v ) = P α +1 ( hh v ) .Let α be a limit ordinal, and suppose that ρ ∈ P α ( hv ) . As ρ ∈ P β ( hv ) for all β < α ,we have ρ ∈ P β ( hh v ) by induction hypothesis. It follows that ρ ∈ P α ( hh v ) = T β<α P β ( hh v ) . (cid:74) In this section, as for weak SPEs, we characterize in the form of a Folk Theorem the setof all outcomes of SPEs. Nevertheless, we here need a more complex characterization withadapted sets P α ( hv ) , and this characterization only holds for cost functions that are upper-semicontinuous. The main difference appears in the definition of sets E α ( hv ) that will beused in place of E α ( hv ) . Indeed we will see that the set P α ( h v ) of Figure 5 has to bereplaced by a more complex set D H,iα ( h v ) .Let ( G , v ) be a game. For an ordinal α and a history hv ∈ Hist( v ) , as in the previoussection, we consider the set P α ( hv ) = { ρ | ρ is a potential outcome of a NE in ( G (cid:22) h , v ) at step α } . In order to define these sets P α ( hv ) , we need to introduce new definitions. InDefinition 4, we have introduced the notion of deviation step of a strategy from a givenstrategy profile. We here propose another concept of deviation step in relation with twoplays (see Figure 7). (cid:73) Definition 23. Let h v ∈ Hist( v ) and ρ ∈ ( G (cid:22) h , v ) . Let h u , h u ∈ Hist( v ) with u , u ∈ V , and ρ in P α ( h u ) . We say that ρ has a h u -deviation step from ρ if h v ≤ h u < h u < h ρ and h ρ b h ρ = h .Let us denote by H ( h v ) the set of all histories h u such that h v < h u . Given aplayer i and H ⊆ H ( hv ) , the next definition introduces the notion of ( H, i ) -decomposition ofa play ρ . Such a play has a finite or infinite number of deviation steps such that for each h n u n -deviation step, the history h n u n belongs to H . Figure 8 illustrates the second case ofDefinition 24, with the deviation steps highlighted with dashed edges. This definition alsointroduces the set D H,iα ( h v ) composed of plays with a maximal ( H, i ) -decomposition. (cid:73) Definition 24. Let h v ∈ Hist( v ) and ρ ∈ ( G (cid:22) h , v ) . Let i be a player, and H ⊆ H ( h v ) . ρ has an infinite ( H, i ) -decomposition ρ = g g . . . g n . . . iffor all n ≥ , Last( g n ) ∈ V i for all n ≥ , h n u n ∈ H , and ρ has a h n u n -deviation step from some ρ n − ∈ P α ( h n − u n − ) where h = h , h n +1 = h n g n , and u n = First( g n ) ∀ n ≥ . ρ has a finite ( H, i ) -decomposition ρ = g g . . . g m % iffor all n , ≤ n ≤ m , Last( g n ) ∈ V i for all n , ≤ n ≤ m + 1 , h n u n ∈ H , and ρ has a h n u n -deviation step from some ρ n − ∈ P α ( h n − u n − ) % ∈ P α ( h m +1 u m +1 ) For games where players receive a payoff that they want to maximize, a similar Folk Theorem alsoexists for lower-semicontinuous cost functions. instead of a weak NE h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 17 v v u u h ρ h ρ h h h Figure 7 Deviation step. where h = h , h n +1 = h n g n , u n = First( g n ) ∀ n, ≤ n ≤ m , and u m +1 = First( % ) .We denote by D H,iα ( h v ) the set of plays ρ with a maximal ( H, i ) -decomposition in thefollowing sense: ρ has an infinite ( H, i ) -decomposition, or ρ has a finite ( H, i ) -decomposition ρ = g g . . . g m % , and there exists no ρ witha ( H, i ) -decomposition ρ = g g . . . g m . . . or ρ = g g . . . g m % such that ρ b ρ = g g . . . g m +1 .In the previous definition, H can be chosen finite, infinite, or empty. If H = ∅ , then forall i ∈ Π , for each ρ ∈ D H,iα ( h v ) , ρ has no deviation step and thus ρ ∈ P α ( h v ) . Thismeans that D H,iα ( h v ) = P α ( h v ) in this case.We are ready to define the sets P α ( hv ) by induction on α . The definition is similar tothe one of P α ( hv ) , except that when we erase ρ ∈ E α ( hv ) from P α ( hv ) , we use some set D H,iα ( h v ) in place of P α ( h v ) : (cid:73) Definition 25. Let ( G , v ) be a quantitative game. The set P α ( hv ) is defined as followsfor each ordinal α and history hv ∈ Hist( v ) :For α = 0 , P α ( hv ) = { ρ | ρ is a play in ( G (cid:22) h , v ) } . (9)For a successor ordinal α + 1 , P α +1 ( hv ) = P α ( hv ) \ E α ( hv ) (10)such that ρ ∈ E α ( hv ) iffthere exists a history h , hv ≤ h < hρ , and Last( h ) ∈ V i for some i ,there exists a vertex v , h v < hρ ,there exists H ⊆ H ( h v ) ,such that ∀ ρ ∈ D H,iα ( h v ) : λ i ( hρ ) > λ i ( h ρ ) .(see Figure 5 where E α ( hv ) is replaced by E α ( hv ) , and P α ( h v ) is replaced by D H,iα ( h v ) ). v u ρ ∈ P α ( h u ) u ρ ∈ P α ( h u ) u u m ρ m ∈ P α ( h m u m ) u m +1 % ∈ P α ( h m +1 u m +1 ) ∈ V i ∈ V i ∈ V i h g g g m h h h m +1 Figure 8 ρ = g g . . . g m % with a finite ( H, i ) -decomposition. For a limit ordinal α : P α ( hv ) = \ β<α P β ( hv ) . (11)Let us comment the case α + 1 . When H = ∅ , we have D H,iα ( h v ) = P α ( h v ) . Hence werecover the previous situation of weak SPEs. Using different sets H ⊆ H ( h v ) and D H,iα ( h v ) allow to have sets E α ( hv ) bigger than E α ( hv ) , and thus more plays removed from P α ( hv ) than in P α ( hv ) . This situation will be illustrated in Example 28 hereafter.The sequence ( P α ( hv )) α is nonincreasing by definition, and reaches a fixpoint in thefollowing sense (the proof is the same as for Proposition 17). (cid:73) Proposition 26. There exists an ordinal β ∗ such that P β ∗ ( hv ) = P β ∗ +1 ( hv ) for all histories hv ∈ Hist( v ) . Our Folk Theorem for SPEs is the next one. The second statement requires to work withupper-semicontinuous cost functions λ i , i ∈ Π . (cid:73) Theorem 27. Let ( G , v ) be a quantitative game.If there exists an SPE in ( G , v ) with outcome ρ , then P β ∗ ( hv ) = ∅ for all hv ∈ Hist( v ) ,and ρ ∈ P β ∗ ( v ) . h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 19 Suppose that all cost functions λ i are upper-semicontinuous. If P β ∗ ( hv ) = ∅ for all hv ∈ Hist( v ) , then there exists an SPE in ( G , v ) with outcome ρ , for all ρ ∈ P β ∗ ( v ) . (cid:73) Example 28. Before proving this theorem, we illustrate it with the example of Figure 3by showing that this game has no SPE (as stated in [16]). Let us compute the sets P α ( hv ) and let us show that P ( hv ) = ∅ . By the first statement of Theorem 27, we will get thatthere is no SPE.We have to do computations that are more complex than the ones of Example 19, due tothe usage of sets D H,iα ( h v ) , with H ⊆ H ( h v ) , instead of P α ( h v ) . Clearly, by definition P ( hv ) = P ( hv ) for all hv , and we have P α ( hv ) = { v ω } and P α ( hv ) = { v ω } for all α asin Example 19.Let us first illustrate Definition 24 with the computation of D H, ( v ) with H = { v v v , ( v v ) v } . The play ρ = ( v v ) v v ω has a ( H, -decomposition g g % with g = g = v v and % = v ω (two deviation steps). Indeed ρ has a v v v -deviation step from v v v ω ∈ P α ( v ) and a ( v v ) v -deviation step from ( v v ) v ω ∈ P α ( v v v ) , Last( g ) , Last( g ) ∈ V ,and % ∈ P α (( v v ) v ) . This play ρ belongs to D H, ( v ) because its ( H, -decompositionuses the two possible steps of H . On the contrary ρ = ( v v ) v ω has a ( H, -decomposition g % with % = v v v ω , and does not belong to D H, ( v ) because ρ b ρ = g g . One cancheck that D H, ( v ) = { v v ω , v v v v ω } ∪ { ( v v ) % | % ∈ ( G (cid:22) ( v v ) , v ) } .We can now detail the computation of P α ( hv ) and P α ( hv ) . α = 0 . For history hv , we have E ( hv ) ⊇ E ( hv ) = { ( v v ) ω } ∪ ( v v ) + v v ω because D H,i ( h v ) = P ( h v ) = P ( h v ) when H = ∅ . In fact, one checks that E ( hv ) = E ( hv ) , and thus P ( hv ) = P ( hv ) = { v v ω } ∪ ( v v ) + v ω . For instance, if we try to remove v v ω with cost (1 , from P ( hv ) , we have to use some H such that D H, ( hv v ) ⊆ v ( v v ) ∗ v ω (with cost (0 , ). Such a H does not exist. Forhistory hv , we also have E ( hv ) = E ( hv ) and P ( hv ) = P ( hv ) = v ( v v ) ∗ v ω .α = 1 . Again we have P ( hv ) = P ( hv ) = ( v v ) + v ω , P ( hv ) = P ( hv ) = v ( v v ) ∗ v ω .α = 2 . A difference appears at this step: P ( hv ) = v ( v v ) ∗ v ω whereas P ( hv ) = ∅ . Indeed E ( hv ) = P ( hv ) . Consider for instance ρ = v v ω ∈ P ( hv ) , and H = hv v ( v v ) + ⊆ H ( hv v ) . Then D H, ( hv v ) has a unique play ρ = ( v v ) ω , and λ ( hρ ) > λ ( hv ρ ) . Proof of Theorem 27. We begin by the first statement. Let ¯ σ be an SPE. As in the proofof Lemma 20, let us show by induction on α that h ¯ σ (cid:22) h i v ∈ P α ( hv ) for all hv ∈ Hist( v ) .For α = 0 , we have h ¯ σ (cid:22) h i v ∈ P α ( hv ) by definition of P ( hv ) .Let α + 1 be a successor ordinal. By induction hypothesis, we have that h ¯ σ (cid:22) h i v ∈ P α ( hv ) for all hv . Suppose that h ¯ σ (cid:22) h i v P α +1 ( hv ) , i.e. h ¯ σ (cid:22) h i v ∈ E α ( hv ) . This means that thereis a history h = hg ∈ Hist i for some i with hv ≤ h < hρ , there exists a vertex v with h v < hρ , and there exists H ⊆ H ( h v ) , such that ∀ ρ ∈ D H,iα ( h v ) , λ i ( h ·h ¯ σ (cid:22) h i v ) > λ i ( h ρ ) . (12) Let us consider player i strategy σ i in the subgame ( G (cid:22) h , v ) such that σ i coincide with σ i (cid:22) h except that σ i ( h ) = v , for all hh v ∈ H ∪ { h v } . (13)Let gρ ∗ = h σ i , σ − i (cid:22) h i v . We get that ρ ∗ has a maximal ( H, i ) -decomposition such thateach ρ n − ∈ P α ( h n − u n − ) of Definition 24 is equal to h ¯ σ (cid:22) h n − i u n − (this play belongs to P α ( h n − u n − ) by induction hypothesis). Each deviation step of ρ ∗ in the sense of Definition 24corresponds to a deviation step of ρ ∗ in the sense of Definition 4 . Moreover the ( H, i ) -decomposition of ρ ∗ is finite (resp. infinite) iff σ i is finitely (resp. infinitely) deviating from ¯ σ (cid:22) h . Thus this play ρ ∗ belongs to D H,iα ( h v ) , and by (12) we get λ i ( h ·h ¯ σ (cid:22) h i v ) > λ i ( hgρ ∗ ) .Hence σ i is a profitable deviation for player i in ( G (cid:22) h , v ) , a contradiction with ¯ σ being anSPE.Let α be a limit ordinal. By induction hypothesis h ¯ σ (cid:22) h i v ∈ P β ( hv ) , ∀ β < α . Therefore h ¯ σ (cid:22) h i v ∈ P α ( hv ) = T β<α P β ( hv ) .Let us now turn to the second statement of Theorem 27. Let ρ ∈ P β ∗ ( v ) . By Propos-ition 14, it is enough to construct a very weak SPE ¯ σ with outcome ρ . The proof is verysimilar to the one of Lemma 21, where the construction of ¯ σ is done step by step thanks to alabeling γ of the histories. We briefly recall this proof and insist on the differences.Initially, no history is labeled. We start with the play ρ ∈ P β ∗ ( v ) , ¯ σ is partially definedsuch that h ¯ σ i v = ρ , and γ ( h ) = ρ for all non-empty prefixes h of ρ .At the following steps, let h v be a history that is not yet labeled, but such that h has already been labeled. Suppose that Last( h ) ∈ V i . By induction, γ ( h ) = h ¯ σ (cid:22) h i v suchthat hv ≤ h , and h ¯ σ (cid:22) h i v ∈ P β ∗ ( hv ) . Since P β ∗ ( hv ) = P β ∗ +1 ( hv ) by Proposition 26, wehave h ¯ σ (cid:22) h i v E β ∗ ( hv ) . Therefore, with H = ∅ and D H,iβ ∗ ( h v ) = P β ∗ ( h v ) , we know thatthere exists a play ρ ∈ P β ∗ ( h v ) such that λ i ( h ·h ¯ σ (cid:22) h i v ) ≤ λ i ( h ρ ) . Hence, we continueto construct ¯ σ such that h ¯ σ (cid:22) h i v = ρ , and all non-empty prefixes g of ρ are labeled by γ ( h g ) = ρ . And so on.The constructed ¯ σ is a very weak SPE as in the proof of Lemma 21. (cid:74) The next proposition states that for cost functions that are upper-semicontinuous, sets P β ∗ ( hv ) and P α ∗ ( hv ) are all equal. This no longer the case as soon as one cost function isnot upper-semicontinuous as shown by Examples 19 and 28. (cid:73) Proposition 29. Let ( G , v ) be a quantitative game such that all its cost functions areupper-semicontinuous. Then for all hv ∈ Hist( v ) , P β ∗ ( hv ) = P α ∗ ( hv ) . Proof. Let us prove by induction on α that P α ( hv ) ⊆ P α ( hv ) for all hv . These two setsare equal for α = 0 . Let α + 1 be a successor ordinal and suppose that P α ( hv ) ⊆ P α ( hv ) .We have E α ( hv ) ⊇ E α ( hv ) , and thus P α +1 ( hv ) ⊆ P α +1 ( hv ) , because D H,iα ( h v ) = P α ( h v ) when H = ∅ . For α being a limit ordinal, we easily have P α ( hv ) ⊆ P α ( hv ) by inductionhypothesis.Suppose now that P β ∗ ( hv ) (cid:40) P α ∗ ( hv ) for some hv . Let ρ ∈ P α ∗ ( hv ) \ P β ∗ ( hv ) , andconsider the initialized game ( G , v ) = ( G (cid:22) h , v ) . Notice that the sets P α ( h v ) and E α ( h v ) of this game ( G , v ) are exactly the sets P α ( hh v ) and E α ( hh v ) of ( G (cid:22) h , v ) . By Theorem 18,there exists a weak SPE ¯ σ in ( G , v ) with outcome ρ . Since the cost functions are upper-semicontinuous, σ is also an SPE by Proposition 14. Therefore, ρ ∈ P β ∗ ( hv ) by Theorem 27,which is a contradiction. (cid:74) History h v leads to one additional deviation step of ρ ∗ in the sense of Definition 4 (see (13)). h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 21 In this section, we focus on quantitative reachability games. Recall that in this case, thecost of a play for player i is the number of edges to reach his target set of vertices T i (seeDefinition 2). Recall also that for quantitative reachability games, SPEs, weak SPEs, andvery weak SPEs, are equivalent notions (see Corollary 15).It is known that there always exists an SPE in quantitative reachability games [1, 6]. (cid:73) Theorem 30. Each quantitative reachability game ( G , v ) has an SPE. As SPEs and weak SPEs coincide in quantitative reachability games, we get the nextresult by Theorem 18. (cid:73) Corollary 31. Let ( G , v ) be a quantitative reachability game. The sets P α ∗ ( hv ) arenon-empty, for all hv ∈ Hist( v ) , and P α ∗ ( v ) is the set of outcomes of SPEs in ( G , v ) . The proof provided for Theorem 30 is non constructive since it relies on topologicalarguments. Our main result is that one can algorithmically construct an SPE in ( G , v ) thatis moreover finite-memory, thanks to the sets P α ∗ ( hv ) . (cid:73) Theorem 32. Each quantitative reachability initialized game ( G , v ) has a finite-memorySPE. Moreover there is an algorithm to construct such an SPE. We can also decide whether there exists a (finite-memory) SPE such that the cost of itsoutcome is component-wise bounded by a given constant vector. (cid:73) Corollary 33. Let ( G , v ) be a quantitative reachability initialized game, and let ¯ c ∈ N | Π | be a given | Π | -uple of integers. Then one can decide whether there exists a (finite-memory)SPE ¯ σ such that λ i ( h ¯ σ i v ) ≤ c i for all i ∈ Π . The main ingredients of the proof of Theorem 32 are the next ones; they will be detailedin the sequel of this section. We will give afterwards the proof of Corollary 33.Given α , the infinite number of sets P α ( hv ) can be replaced by the finite number of sets P Iα ( v ) where I is the set of players that did not reach their target set along history h .The fixpoint of Proposition 17 is reached with some natural number α ∗ ∈ N .Each P Iα ( v ) is a non-empty ω -regular set, thus containing a “lasso play” of the form h · g ω .The lasso plays of each P Iα ( v ) allow to construct a finite-memory SPE.The next lemma highlights a simple useful property of the cost functions λ i used inquantitative reachability games. The proof is immediate. (cid:73) Lemma 34. Let i ∈ Π and ρ ∈ P α ( hv ) . If player i did not reach his target set alonghistory h , then λ i ( hρ ) = λ i ( ρ ) + | hv | . The next proposition is a key result that will be used several times later on. It statesthat it is impossible to have plays in P α ( hv ) with arbitrarily large costs for player i , withouthaving a play in P α ( hv ) with an infinite cost for player i . (cid:73) Proposition 35. Consider P α ( hv ) and i ∈ Π . If for all ρ ∈ P α ( hv ) , we have λ i ( ρ ) < + ∞ ,then there exists c such that for all ρ ∈ P α ( hv ) , we have λ i ( ρ ) ≤ c . The constant c onlydepends on P α ( hv ) and player i . Proof. Suppose that for all n ∈ N , there exists ρ n ∈ P α ( hv ) such that λ i ( ρ n ) > n . ByK¨onig’s lemma, there exists ρ = lim k →∞ ρ n k for some subsequence ( ρ n k ) k of ( ρ n ) n . Bydefinition of λ i in quantitative reachability games, we get λ i ( ρ ) = + ∞ . Let us prove byinduction on α that ρ ∈ P α ( hv ) ; this will establish Proposition 35.Let α = 0 . As each ρ n ∈ P ( hv ) , then ρ n is a play in ( G (cid:22) h , v ) by definition of P ( hv ) (see (4) in Definition 16). Therefore ρ is also a play in ( G (cid:22) h , v ) , and ρ ∈ P ( hv ) .Let α + 1 be a successor ordinal. As for all n , ρ n ∈ P α +1 ( hv ) ⊆ P α ( hv ) , then ρ ∈ P α ( hv ) by induction hypothesis. Let us prove that ρ ∈ P α +1 ( hv ) . Suppose on the contrary that ρ ∈ E α ( hv ) (see Definition 16 and Figure 5). Then there exists a history h ∈ Hist j with j ∈ Π and hv ≤ h < hρ , there exists a vertex v with h v < hρ , such that ∀ ρ ∈ P α ( h v ) : λ j ( hρ ) > λ j ( h ρ ) . (14)It follows that player j did not reach his target set along h . Hence by Lemma 34, we have λ j ( hρ ) = λ j ( ρ ) + | hv | and λ j ( h ρ ) = λ j ( ρ ) + | h v | . By (14), λ j ( ρ ) is bounded. Hence byinduction hypothesis with P α ( h v ) and j ∈ Π , there exists a constant c such that λ j ( ρ ) ≤ c , ∀ ρ ∈ P α ( h v ) .Suppose first that λ j ( ρ ) < + ∞ . Then, since ρ = lim k →∞ ρ n k , it follows that for a largeenough n k , the plays ρ and ρ n k share a long common prefix on which player j reaches itstarget set, i.e. λ j ( ρ ) = λ j ( ρ n k ) . It follows that with the same history h v as above, by (14)and Lemma 34, we have λ j ( hρ n k ) > λ j ( h ρ ) , ∀ ρ ∈ P α ( h v ) , showing that ρ n k ∈ E α ( hv ) , acontradiction.Suppose next that λ j ( ρ ) = + ∞ . Then, given c = | h v | − | hv | + c , we can choose a largeenough n k such that the plays ρ and ρ n k share a common prefix of length at least c .Moreover, as λ j ( ρ ) = + ∞ , player j does not reach its target set along this prefix, i.e. λ j ( ρ n k ) > c . Therefore, using the same history h v as above, by (14) and Lemma 34,we have λ j ( hρ n k ) > | hv | + c = | h v | + c ≥ λ j ( h ρ ) , ∀ ρ ∈ P α ( h v ) . This shows that ρ n k ∈ E α ( hv ) , again a contradiction.Let α be a limit ordinal. As for all n , ρ n ∈ P α ( hv ) = T β<α P β ( hv ) (see (6) in Defin-ition 16), then ρ ∈ P β ( hv ) , ∀ β < α , by induction hypothesis. Hence ρ ∈ P α ( hv ) = T β<α P β ( hv ) .We have just shown that if all plays ρ ∈ P α ( hv ) have a cost λ i ( ρ ) < + ∞ , then there exists c such that λ i ( ρ ) ≤ c for all such ρ . This constant c depends on P α ( hv ) and player i . (cid:74) As a consequence of Proposition 35, we have that sup { λ i ( ρ ) | ρ ∈ P α ( hv ) } is equal to max { λ i ( ρ ) | ρ ∈ P α ( hv ) } , and that this maximum belongs to N ∪ { + ∞} . P Iα ( v ) Let ( G , v ) be a quantitative reachability game. Given a history h = h . . . h n in ( G , v ) , wedenote by I ( h ) the set of players i such that ∀ k , ≤ k ≤ n , we have h k T i . In other words I ( h ) is the set of players that did not reach their target set along history h . If h is empty,then I ( h ) = Π . The next lemma indicates that sets P α ( hv ) only depend on v and I ( h ) ,and thus not on h (we do no longer take care of players that have reached their target setalong h ). (cid:73) Lemma 36. For h v, h v ∈ Hist( v ) , if I ( h ) = I ( h ) , then P α ( h v ) = P α ( h v ) for all α . Proof. The proof is by induction on α . By definition, we have P ( h v ) = P ( h v ) .Suppose that α + 1 is a successor ordinal. By induction hypothesis, P α ( h v ) = P α ( h v ) .Let us prove that E α ( h v ) = E α ( h v ) which will imply that P α +1 ( h v ) = P α +1 ( h v ) . If h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 23 ρ ∈ E α ( h v ) , it means that there exists a history h = h g ∈ Hist i with h < h ρ , thereexists a vertex v with h v < h ρ , such that ∀ ρ ∈ P α ( h v ) , we have λ i ( h ρ ) > λ i ( h ρ ) ,i.e. λ i ( ρ ) > λ i ( gρ ) . (15)In particular, i ∈ I ( h ) . Let us consider the history h = h g . By hypothesis, I ( h ) = I ( h ) ,and therefore I ( h ) = I ( h ) and i ∈ I ( h ) . Thus by induction hypothesis P α ( h v ) = P α ( h v ) . It follows that for ∀ ρ ∈ P α ( h v ) , we have λ i ( h ρ ) > λ i ( h gρ ) = λ i ( h ρ ) by(15), and then ρ ∈ E α ( h v ) . Symmetrically, if ρ ∈ E α ( h v ) , then ρ ∈ E α ( h v ) . We canconclude that P α +1 ( h v ) = P α +1 ( h v ) .Suppose that α is a limit ordinal. As P α ( h v ) = T β<α P β ( h v ) , and P β ( h v ) = P β ( h v ) by induction hypothesis, it follows that P α ( h v ) = P α ( h v ) . (cid:74) Thanks to this lemma, we can introduce the next definitions. (cid:73) Definition 37. Let ( G , v ) be a quantitative reachability initialized game. Let I ⊆ Π besuch that I = I ( h ) for some h ∈ Hist( v ) . We denote by P Iα ( v ) the set P α ( hv ) , and by E Iα ( v ) the set E α ( hv ) .In particular, P Π α ( v ) = P α ( v ) and E Π α ( v ) = E α ( v ) .Given α , the infinite number of sets P α ( hv ) can thus be replaced by the finite number ofsets P Iα ( v ) . Moreover, Proposition 35 can be rephrased as follows. (cid:73) Corollary 38. Consider P Iα ( v ) and i ∈ I . If for all ρ ∈ P Iα ( v ) , we have λ i ( ρ ) < + ∞ , thenthere exists c such that for all ρ ∈ P Iα ( v ) , we have λ i ( ρ ) ≤ c . The constant c only dependson α, I, v , and i . Proof. Let h be such that I = I ( h ) . Consider P α ( hv ) = P Iα ( v ) , and i ∈ I . By Proposition 35,if for all ρ ∈ P α ( hv ) , λ i ( ρ ) < + ∞ , then there exists c (depending on P α ( hv ) and i ) suchthat for all ρ ∈ P α ( hv ) , λ i ( ρ ) ≤ c . By Lemma 36, c depends on α , I , v , and i . (cid:74) As a consequence of Corollary 38, we give the next definition that indicates the maximumcosts for plays in P Iα ( v ) . (cid:73) Definition 39. Given P Iα ( v ) , we define Λ( P Iα ( v )) such that Λ i ( P Iα ( v )) = (cid:26) − if i I, max { λ i ( ρ ) | ρ ∈ P Iα ( v ) } if i ∈ I. In this definition, − indicates that player i has already visited his target set T i , and the max belongs to N ∪ { + ∞} . α ∗ ∈ N In this section, we aim at proving that the fixpoint (when computing the sets P Iα ( v ) , seeProposition 17) is reached in a finite number of steps, that is α ∗ ∈ N .We first need to introduce some notions about the sets P Iα ( v ) . Let ρ = ρ ρ . . . ∈ P Iα ( v ) .We use a map χ that decorates each ρ n by some set J ⊆ Π . The aim of the decoration χ ( ρ n ) is to indicate at vertex ρ n , which players of I did not reach their target set along ρ The first statement is immediate from definition of Λ . Let us prove the secondstatement. Consider ρ = ρ ρ . . . ∈ E Iα ( v ) . Then there exist i ∈ Π , n ∈ N and v = ρ n +1 with ρ n ∈ V i , χ ( ρ n ) = J , χ ( ρ n +1 ) = J , such that ∀ ρ ∈ P J α ( v ) we have λ i ( ρ ) > λ i ( ρ . . . ρ n ρ ) or equivalently (by Lemma 34) λ i ( ρ ) − ( n + 1) > λ i ( ρ ) . (17)(see the definition of E α ( hv ) with I ( h ) = I in Definition 16 and Figure 9). Notice that i ∈ J .Let us prove that Λ( P Jα ( u ) , u ) = Λ( P Jα +1 ( u ) , u ) with u = ρ n and u = ρ n +1 . As ρ ∈ P Iα ( v ) , vu v u ∈ V i IJJ ρ ∈ E Iα ( v ) ∀ ρ P J α ( v ) Figure 9 ρ ∈ E Iα ( v ) , with ρ n = u and ρ n +1 = u . then ρ ≥ n ∈ P Jα ( u ) by Lemma 22. As λ i ( ρ ≥ n ) = λ i ( ρ ) − n , this implies that Λ i ( P Jα ( u ) , u ) ≥ λ i ( ρ ) − n. (18)Let % ∈ P Jα ( u ) be such that % starts with edge ( u, u ) and has maximal cost Λ i ( P Jα ( u ) , u ) .One gets λ i ( % ) = Λ i ( P Jα ( u ) , u ) ≥ λ i ( ρ ) − n > λ i ( ρ n ρ ) Notice that as P Iα ( v ) is non-empty, there exists some ( v, v ) ∈ E such that this set is non-empty. h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 25 by (17) and (18). By considering the set P J α ( v ) in Figure 9, it follows that % ∈ E Jα ( u ) for allsuch plays % . Hence P Jα +1 ( u ) (cid:40) P Jα ( u ) and Λ i ( P Jα +1 ( u ) , u ) < Λ i ( P Jα ( u ) , u ) . This completesthe proof. (cid:74) We are now able to prove that the ordinal α ∗ of Proposition 17 is an integer. (cid:73) Corollary 41. There exists an integer α ∗ such that P Iα ∗ ( v ) = P Iα ∗ +1 ( v ) for all v ∈ V and I ⊆ Π . Proof. Notice that there is a finite number of sequences (Λ( P Iα ( v ) , v )) α since they dependon I ⊆ Π and ( v, v ) ∈ E . As the component-wise ordering over ( N ∪ {− , + ∞} ) Π is awell-quasi-ordering and all these sequences are nonincreasing, there exists an integer (and notonly an ordinal) α such that Λ( P Iα ( v ) , v ) = Λ( P Iα +1 ( v ) , v ) for all I ⊆ Π and ( v, v ) ∈ E .By Lemma 40, we get that α ∗ ≤ α , showing that α ∗ ∈ N . (cid:74) P Iα ( v ) are ω -regular In this section, we prove that each set P Iα ( v ) is ω -regular. Instead of providing the constructionof a B¨uchi automaton (which would lead to many technical details), we prefer to show thateach set P Iα ( v ) is MSO-definable. It is well-known that a set of ω -words is ω -regular iff itis MSO-definable, by B¨uchi theorem [17]. Moreover from the B¨uchi automaton, one canconstruct an equivalent MSO-sentence, and conversely. One can also decide whether anMSO-sentence is satisfiable [17]. We recall that MSO-logic uses:variables x, y, . . . ( X, Y, . . . resp.) to describe a position (a set of positions resp.) in an ω -word ρ , and relations X ( x ) to mention that x belongs to X ,relations Q u ( x ) , u ∈ V , to mention that such vertex u is at position x of ρ ,relations x < y and Succ ( x, y ) to mention that position y is after position x , and position y is successor of position x respectively,connectives ∨ , ∧ , ¬ and quantifiers ∃ x , ∀ x , ∃ X , ∀ X Recall that constants , , . . . are definable. We will use notation x + 1 (and more generally x + c , with c a constant) instead of Succ ( x, y ) . (cid:73) Proposition 42. Each P Iα ( v ) is an ω -regular set. We begin with a lemma that states that if P Iα ( v ) is ω -regular, then the maximum of itscosts is computable. (cid:73) Lemma 43. If P Iα ( v ) is MSO-definable, then Λ( P Iα ( v )) is computable. Proof. Before proving this lemma, we need to establish two properties. The first one statesthat one can decide whether P Iα ( v ) has a play ρ with a given cost for a given player. Thesecond one states that when Λ i ( P Iα ( v )) is finite, then this number is bounded by the numberof states of a B¨uchi automaton accepting P Iα ( v ) . (i) Let c ∈ N ∪ { + ∞} and i ∈ I . Let φ be an MSO-sentence defining P Iα ( v ) . Let usshow that one can decide whether P Iα ( v ) has a play ρ with cost λ i ( ρ ) = c . There existsan MSO-sentence ϕ expressing that λ i ( ρ ) = c . Indeed, if c = + ∞ , then ϕ is the sentence ∀ x ·¬ ( ∨ u ∈ T i Q u ( x )) , and if c < + ∞ , it is the sentence ( ∀ x < c ·¬ ( ∨ u ∈ T i Q u ( x ))) ∧ ( ∨ u ∈ T i Q u ( c )) .Therefore one can decide whether the MSO-sentence φ ∧ ϕ is satisfiable by some play ρ . (ii) Let i ∈ I and suppose that Λ i ( P Iα ( v )) < + ∞ . Let B be a B¨uchi automaton accepting P Iα ( v ) . We now show that Λ i ( P Iα ( v )) < n where n is the number of states of B . Assumethe contrary and consider an accepting run r = r r . . . of B on a play ρ = ρ ρ . . . ∈ P Iα ( v ) with λ i ( ρ ) = Λ i ( P Iα ( v )) ≥ n . The prefix r ≤ n of r has a cycle r k . . . r l with ≤ k < l ≤ n and r k = r l . This cycle can be repeated once, while keeping an accepting run labeled by ρ = ρ . . . ( ρ k . . . ρ l − ) ρ ≥ l . As λ i ( ρ ) ≥ n , it follows that λ i ( ρ ) = Λ i ( P Iα ( v )) + ( l − k ) .Therefore we get a contradiction with λ i ( ρ ) = Λ i ( P Iα ( v )) .Let us prove the lemma. By definition Λ i ( P Iα ( v )) equals − if i I , and is thuscomputable in this case. Let i ∈ I . By (i) , one can decide whether Λ i ( P Iα ( v )) = + ∞ .In case of a positive answer, Λ i ( P Iα ( v )) is thus computable. If the answer is negative, as Λ i ( P Iα ( v )) < n by (ii) , we can similarly test whether Λ i ( P Iα ( v )) = c by considering decreasingconstants c from n − to . This prove that Λ( P Iα ( v )) is computable. (cid:74) Proof of Proposition 42. Let us prove that each set P Iα ( v ) is MSO-definable by inductionon α .For α = 0 , recall that P I ( v ) is the set of plays starting with v . The required sentence isthus Q v (0) ∧ ∀ x · ∨ ( u,u ) ∈ E ( Q u ( x ) ∧ Q u ( x + 1)) .Let α ∈ N be a fixed integer. By induction hypothesis, each set P Iα ( v ) is MSO-definable,and by Lemma 43, Λ( P Iα ( v )) is computable. These sets and constants can be consideredas fixed. Let us prove that E Iα ( v ) is MSO-definable. It will follow that P Iα +1 ( v ) is alsoMSO-definable. Thanks to Λ( P Iα ( v )) , the definition of ρ ∈ E Iα ( v ) can be rephrased as follows:there exist n ∈ N , i ∈ I , and u, u , v ∈ V with u = v , ( u, v ) ∈ E , such that ρ n = u ∈ V i , ρ n +1 = u , χ ( ρ n +1 ) = J , and λ i ( ρ ) > Λ i ( P J α ( v )) + ( n + 1) (19)(see Figure 9). Notice that (19) implies that i ∈ J and Λ i ( P J α ( v )) < + ∞ . Moreover Λ i ( P J α ( v )) is a fixed integer.Let us provide an MSO-sentence ψ defining E Iα ( v ) . The next sentence φ J ,n expressesthat J ⊆ I is the subset of players of I that did not visit their target set along ρ ≤ n : φ J ,n = (cid:0) ∀ x · ( x ≤ n ) → ¬ ( ∨ j ∈ J ∨ r ∈ T j Q r ( x )) (cid:1) ∧ (cid:0) ∧ j ∈ I \ J ∃ x ≤ n · ∨ r ∈ T j Q r ( x ) (cid:1) . The next sentence ϕ J ,n,v ,i expresses that if player i visits its target set along ρ , it is after Λ i ( P J α ( v )) + n + 1 edges from ρ : ϕ J ,n,v ,i = ∀ x · (cid:16) ∨ r ∈ T i Q r ( x ) → (cid:16) x > Λ i ( P J α ( v )) + n + 1 (cid:17)(cid:17) . Notice that in the previous formula, Λ i ( P J α ( v )) is a constant since P J α ( v ) is a fixed set.The required formula ψ is then the following one: ∃ n · _ u,u = v ∈ V ( u,v ) ∈ E _ J ⊆ I _ i ∈ J ,u ∈ V i Λ i ( P J α ( v )) < + ∞ ( Q u ( n ) ∧ Q u ( n + 1) ∧ φ J ,n ∧ ϕ J ,n,v ,i ) . (cid:74) By Proposition 42, the next corollary states that one can effectively extract a lasso playfrom P Iα ( v ) that has a maximal cost for a given player i . (cid:73) Corollary 44. For all i ∈ I , each set P Iα ∗ ( v ) has a computable lasso play h · g ω with λ i ( h · g ω ) = Λ i ( P Iα ∗ ( v )) . This play depends on i , I , and v . Proof. Part (i) of the proof of Lemma 43 indicates that the set of plays ρ ∈ P Iα ∗ ( v ) withmaximal cost λ i ( ρ ) = Λ i ( P Iα ∗ ( v )) is ω -regular. Therefore, from a B¨uchi automaton acceptingthis set, we can extract an accepted lasso play of the form h · g ω with the required cost. Sucha play depends on i , I , and v ( α ∗ is fixed). (cid:74) h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 27 Thanks to the results of Sections 4.1-4.3, we have all the ingredients to prove that eachquantitative reachability game has a computable finite-memory SPE. Proof of Theorem 32. Let ( G , v ) be a quantitative reachability game. Let us summarizethe results obtained previously. By Corollary 31, each set P Iα ∗ ( v ) is non-empty with v ∈ V and I = I ( h ) for some history hv ∈ Hist( v ) , and P Π α ∗ ( v ) contains all the outcomes of SPEsin ( G , v ) . By Corollary 41 and Proposition 42, we know that α ∗ ∈ N and each P Iα ∗ ( v ) is an ω -regular set that can be constructed. Finally by Corollary 44, for all i ∈ I , one can constructa lasso play h i,I,v · ( g i,I,v ) ω ∈ P Iα ∗ ( v ) with maximal cost λ i ( h i,I,v · ( g i,I,v ) ω ) = Λ i ( P Iα ∗ ( v )) .We now show how to construct a finite-memory SPE ¯ σ from the finite set of lasso plays h i,I,v · ( g i,I,v ) ω . The procedure is similar to the one developed in the proof of Theorem 18 andmore particularly of Lemma 21. We indicate how to adapt the proof of this lemma. Againthe construction of ¯ σ is done step by step, thanks to a labeling γ of the non-empty histories.Initially, none of the histories is labeled. We start with history v and with any play h i, Π ,v · ( g i, Π ,v ) ω ∈ P Π α ∗ ( v ) , i ∈ Π . The strategy profile ¯ σ is partially defined such that h ¯ σ i v = h i, Π ,v · ( g i, Π ,v ) ω , and the non-empty prefixes h of h i, Π ,v · ( g i, Π ,v ) ω are all labeledwith γ ( h ) = ( i, Π , v ) .At the following steps, we consider a history h v that is not yet labeled, but such that h has already been labeled. By induction, γ ( h ) = ( j, I, v ) and there exists hv ≤ h such that h ¯ σ (cid:22) h i v = h j,I,v · ( g j,I,v ) ω . Suppose that Last( h ) ∈ V i and I ( h ) = J , the proof of Lemma 21requires to choose a play ρ ∈ P J α ∗ ( v ) such that λ i ( h ρ ) ≥ λ i ( h ·h ¯ σ (cid:22) h i v ) (see (8)). We simplychoose ρ = h i,J ,v · ( g i,J ,v ) ω that has maximal cost λ i ( h i,J ,v · ( g i,J ,v ) ω ) = Λ i ( P J α ∗ ( v )) .Then we continue the construction of ¯ σ such that h ¯ σ (cid:22) h i v = h i,J ,v · ( g i,J ,v ) ω , and for allnon-empty prefixes g of h i,J ,v · ( g i,J ,v ) ω , we define γ ( h g ) = ( i, J , v ) .By the proof of Lemma 21, the strategy profile ¯ σ is an SPE. It is finite-memory sincefor all h ∈ Hist i , σ i ( h ) only depends on γ ( h ) = ( j, I, v ) and h j,I,v · ( g j,I,v ) ω . There is a finitenumber of lasso plays h j,I,v · ( g j,I,v ) ω , and γ ( h ) (as well as I ( h ) ) can be computed inductivelyas follows. Initially, I ( v ) = Π , and γ ( v ) = ( i, Π , v ) for some chosen i ∈ Π . Let h ∈ Hist i and suppose that I ( h ) = J and γ ( h ) = ( j, I, v ) . Then I ( h v ) = J \ { i | v ∈ T i } . If h v respects h j,I,v · ( g j,I,v ) ω , i.e. σ i ( h ) = v , then γ ( h v ) = ( j, I, v ) . Otherwise γ ( h v ) = ( i, J , v ) with ( i, J , v ) computed as in the previous paragraph. (cid:74) It remains to prove the decidability of the constrained existence of SPE for quantitativereachability games, as announced in Corollary 33. This result is easily proved on the basis ofsome previous properties. Proof of Corollary 33. Let ( G , v ) be a game and let ¯ c ∈ N | Π | be a constant vector. In theproof of Lemma 43, we have seen that there exists an MSO-sentence expressing that play ρ has a fixed cost λ i ( ρ ) = c i . Similarly, one can express that λ i ( ρ ) ≤ c i by the next sentence ϕ i : ∃ x ≤ c i · ( ∨ u ∈ T i Q u ( x )) . By Proposition 42, we know that the set P Π α ∗ ( v ) of outcomes ofSPEs in ( G , v ) is an ω -regular set, and that one can construct an MSO-sentence φ defining it.Therefore the set of outcomes of SPEs with a cost component-wise bounded by ¯ c is definableby ∧ i ∈ Π ϕ i ∧ φ , and is then ω -regular. Moreover, one can decide whether this set is non-empty. This proof states that such a play always exists. In case of positive answer, it contains a lasso play h · g ω . Exactly as done in Section 4.4, onecan construct a finite-memory SPE ¯ σ such that h ¯ σ i v = h · g ω . This concludes the proof. (cid:74) In this section, we present a class of games for which it is decidable whether there exists aweak SPE. The hypotheses are general conditions on the cost functions λ i , i ∈ Π : eachfunction λ i must be prefix-independent (see Definition 3), λ i has to use a finite numberof values (gathered in set C i ), and the set of plays ρ with a given cost λ i ( ρ ) = c i must be ω -regular. (cid:73) Theorem 45. Let ( G , v ) be an initialized game such that:each cost function λ i is prefix-independent, and with finite range C i ⊂ Q ,for all i ∈ Π , c i ∈ C i , and v ∈ V , the set of plays ρ in ( G , v ) with λ i ( ρ ) = c i is an ω -regular set.Then one can decide whether ( G , v ) has a weak SPE ¯ σ (resp. such that λ i ( h ¯ σ i v ) ≤ c i forall i for given c i ∈ C i , i ∈ Π ). In case of positive answer, one can construct such a finite-memoryweak SPE. For example, the hypotheses of this theorem are satisfied by the liminf games and thelimsup games; they are also satisfied by the game of Example 12. We will see that the proofof this decidability result shares similar points with the proof given in the previous sectionfor quantitative reachability games. Again, we will use the Folk Theorem for weak SPEs (seeTheorem 18) to prove this result. The main steps of the proof are the following ones.Given α , the infinite number of sets P α ( hv ) can be replaced by the finite number of sets P α ( v ) .The fixpoint of Proposition 17 is reached with some natural number α ∗ ∈ N .Each P α ( v ) is an ω -regular set. Therefore there exists an algorithm to construct thesets P α ∗ ( v ) for all v ∈ V , and thus to decide whether they are all non-empty. For givenconstants c i ∈ C i , i ∈ Π , one can also decide whether P α ∗ ( v ) has a play ρ with boundedcost λ i ( ρ ) ≤ c i for all i .In case of positive answer, some lasso plays of the sets P α ∗ ( v ) allow to construct afinite-memory weak SPE (resp. with bounded cost).To establish Theorem 45, we prove a series of lemmas. The first lemma states that P α ( hv ) is independent of h . There is thus a finite number of sets P α ( v ) , v ∈ V , to study. (cid:73) Lemma 46. P α ( hv ) = P α ( v ) for all hv ∈ Hist( v ) . Proof. The proof can be easily done by induction on α . It uses the definition of E α ( hv ) andthe hypothesis of Theorem 45 that each cost function λ i is prefix-independent. (cid:74) As each cost function λ i is supposed to have finite range in Theorem 45, we can give thenext definition that indicates the maximum costs for plays in P α ( v ) (resp. starting with vv ,for some given ( v, v ) ∈ E ). Recall that a similar definition was given in case of quantitativereachability games (see Definition 39 and (16)). (cid:73) Definition 47. Given P α ( v ) and ( v, v ) ∈ E , we define for each i ∈ Π : Contrarily to quantitative reachability games, we do not know if a weak SPE always exists for gamesin this class. h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 29 Λ i ( P α ( v )) = max { λ i ( ρ ) | ρ ∈ P α ( v ) } . Λ i ( P α ( v ) , v ) = max { λ i ( ρ ) | ρ ∈ P α ( v ) and ρ ρ = vv } . In this definition, the max is equal to −∞ if it applies to an empty set.The sequence (Λ( P α ( v ) , v )) α is nonincreasing for the component-wise ordering over ( Q ∪ {−∞} ) | Π | . Therefore it reaches a fixpoint as for ( P α ( v )) α . The following lemmarelates these sequences. (cid:73) Lemma 48. If P α ( v ) = P α +1 ( v ) , then for all ( v, v ) ∈ E , Λ( P α ( v ) , v ) = Λ( P α +1 ( v ) , v ) .If P α ( v ) = P α +1 ( v ) , then there exists ( u, u ) ∈ E such that Λ( P α ( u ) , u ) = Λ( P α +1 ( u ) , u ) . Proof. The proof is similar to the proof of Lemma 40. Without mentioning it, we willrepeatedly use Lemma 46 and the hypothesis of Theorem 45 that the cost functions areprefix-independent.The first statement is immediate from definition of Λ . For the second statement, consider ρ = ρ ρ . . . ∈ E α ( v ) . Then by definition of E α ( v ) , there exist n ∈ N , i ∈ Π , and u, u , v ∈ V with u = v , ( u, v ) ∈ E , such that ρ n = u ∈ V i , ρ n +1 = u , and ∀ ρ ∈ P α ( v ) : λ i ( ρ ) > λ i ( ρ ) . (See Figure 9 adapted to the context of Lemma 48). Let us prove that Λ( P α ( u ) , u ) =Λ( P α +1 ( u ) , u ) . As ρ ∈ P α ( v ) , then ρ ≥ n ∈ P α ( u ) by Lemma 22, which implies that Λ i ( P α ( u ) , u ) ≥ λ i ( ρ ≥ n ) . Let % ∈ P α ( u ) be such that % starts with edge ( u, u ) and hasmaximal cost Λ i ( P α ( u ) , u ) . One gets λ i ( % ) = Λ i ( P α ( u ) , u ) ≥ λ i ( ρ ≥ n ) = λ i ( ρ ) > λ i ( ρ ) . Hence, using the same set P α ( v ) as for ρ , it follows that % ∈ E α ( u ) for all such plays % .Therefore P α +1 ( u ) (cid:40) P α ( u ) and Λ i ( P α ( u ) , u ) < Λ i ( P α +1 ( u ) , u ) . (cid:74) As a consequence, the ordinal α ∗ of Proposition 17 is an integer. The proof is the sameas for Corollary 41. (cid:73) Corollary 49. There exists an integer α ∗ such that P α ∗ ( v ) = P α ∗ +1 ( v ) for all v ∈ V . As done for quantitative reachability games, let us now prove that the sets P α ( v ) are ω -regular for all α and v . (cid:73) Lemma 50. Each P α ( v ) is an ω -regular set. Proof. The proof is similar to the proof of Lemma 43 and Proposition 42 (It is even simpler).Recall that as soon as P α ( v ) is empty, then P β ( v ) = ∅ for all β ≥ α .Like in Lemma 43, we first prove that if P α ( v ) is MSO-definable, then Λ i ( P α ( v )) iscomputable for each i ∈ Π . Let φ be an MSO-sentence defining P α ( v ) . One can decidewhether P α ( v ) is empty. If this is the case, then Λ i ( P α ( v )) = −∞ for all i . Suppose that P α ( v ) = ∅ , and let i ∈ Π and c ∈ C i . By hypothesis, the set of plays ρ in ( G , v ) withcost λ i ( ρ ) = c is ω -regular and thus MSO-definable by a sentence ϕ c,i . We can thus decidewhether P α ( v ) has a play ρ with cost λ i ( ρ ) = c , thanks to sentence φ ∧ ϕ c,i . Therefore, byconsidering decreasing constants c ∈ C i , we can decide whether Λ i ( P α ( v )) = c . This showsthat Λ i ( P α ( v )) is computable. More precisely each component in ( Q ∪ {−∞} ) | Π | is restricted to C i ∪ {−∞} . Let us now prove that each set P α ( v ) is MSO-definable by induction on α . For α = 0 ,we use the same defining MSO-sentence as in the proof of Proposition 42: Q v (0) ∧ ∀ x · ∨ ( u,u ) ∈ E ( Q u ( x ) ∧ Q u ( x + 1)) . Let α ∈ N be a fixed integer. By induction hypothesis, each set P α ( v ) is MSO-definable, and Λ i ( P Iα ( v )) , i ∈ Π , is computable by the first part of the proof. These sets and constants canbe considered as fixed. The only case to consider is P α ( v ) = ∅ (recall that this property isdecidable). To show that P α +1 ( v ) is also MSO-definable, it is enough to prove that E α ( v ) is MSO-definable. Recall that ρ ∈ E α ( v ) iff there exist n ∈ N , i ∈ Π , and u, u , v ∈ V with u = v , ( u, v ) ∈ E , such that ρ n = u ∈ V i , ρ n +1 = u , and ∀ ρ ∈ P α ( v ) : λ i ( ρ ) > λ i ( ρ ) .The last condition can be replaced by λ i ( ρ ) > Λ i ( P α ( v )) = −∞ . Let us provide anMSO-sentence ψ defining E α ( v ) : ∃ n · _ i ∈ Π ,u ∈ V i u = v ∈ V ( u,v ) ∈ E _ c ∈ C i c> Λ i ( P α ( v )) = −∞ ( Q u ( n ) ∧ Q u ( n + 1) ∧ ϕ c,i ) . (cid:74) We get the next corollary. The proof is the same as for Corollary 44. (cid:73) Corollary 51. If P α ( v ) = ∅ , then one can compute a lasso play h · g ω in P α ( v ) with λ i ( h · g ω ) = Λ i ( P α ( v )) . This play depends on i and v . We are now able to prove the main result of this section. Proof of Theorem 45. Let ( G , v ) be a game satisfying the hypotheses of Theorem 45.Let us summarize the results of the previous lemmas. We know that α ∗ ∈ N and thatone can construct the sets P α ∗ ( v ) , v ∈ V . As these sets are ω -regular, one can decidewhether they are all non-empty. In case of positive answer, there exists a weak SPE in ( G , v ) by Theorem 18. If in addition some constants c i ∈ C i are given, then the set P α ∗ ( v ) ∩ { ρ in ( G , v ) | λ i ( ρ ) ≤ c i , ∀ i ∈ Π } is also ω -regular. Hence one can also decidewhether this set is non-empty and thus whether there exists a weak SPE in ( G , v ) with costcomponent-wise bounded by ¯ c . This establishes the first part of Theorem 45.Suppose that such a weak SPE exists, then let us show that we can construct a weakSPE that is finite-memory with the same construction as in the proof of Theorem 32. ByCorollary 51, for all i ∈ Π , v ∈ V , one can construct a lasso play h i,v · ( g i,v ) ω ∈ P α ∗ ( v ) withmaximal cost λ i ( h i,v · ( g i,v ) ω ) = Λ i ( P α ∗ ( v )) . The construction of a finite-memory SPE ¯ σ from the finite set of lasso plays h i,v · ( g i,v ) ω is conducted as in the proof of Lemma 21. It isdone step by step thanks to a labeling γ of the non-empty histories.Initially, none of the histories is labeled. We start with history v and with any play h i,v · ( g i,v ) ω ∈ P α ∗ ( v ) , i ∈ Π . The strategy profile ¯ σ is partially defined such that h ¯ σ i v = h i,v · ( g i,v ) ω , and the non-empty prefixes h of h i,v · ( g i,v ) ω are all labeled with γ ( h ) = ( i, v ) .At the following steps, we consider a history h v that is not yet labeled, but such that h has already been labeled. By induction, γ ( h ) = ( j, v ) and there exists hv ≤ h such that Set P α ( v ) must be non-empty. When some constants c i ∈ C i are additionally given, play h i,v · ( g i,v ) ω must be replaced by anylasso play in P α ∗ ( v ) ∩ { ρ in ( G , v ) | λ i ( ρ ) ≤ c i , ∀ i ∈ Π } . h. Brihaye, V. Bruy`ere, N. Meunier and J.-F. Raskin 31 h ¯ σ (cid:22) h i v = h j,v · ( g j,v ) ω . Suppose that Last( h ) ∈ V i , the proof of Lemma 21 requires to choosea play ρ ∈ P α ∗ ( v ) such that λ i ( h ρ ) = λ i ( ρ ) ≥ λ i ( h ·h ¯ σ (cid:22) h i v ) . We choose ρ = h i,v · ( g i,v ) ω with maximal cost λ i ( ρ ) = Λ i ( P α ∗ ( v )) . Then we continue the construction of ¯ σ suchthat h ¯ σ (cid:22) h i v = h i,v · ( g i,v ) ω , and for all non-empty prefixes g of h i,v · ( g i,v ) ω , we define γ ( h g ) = ( i, v ) .We know by Lemma 21 that ¯ σ is an SPE. It is finite-memory because it only dependson the finite number of lasso plays h j,v · ( g j,v ) ω , and the labeling γ that can be computedinductively as follows. Initially, γ ( v ) = ( i, v ) for some chosen i ∈ Π . Let h ∈ Hist i andsuppose that γ ( h ) = ( j, v ) . If h v respects h j,v · ( g j,v ) ω , then γ ( h v ) = ( j, v ) , otherwise γ ( h v ) = ( i, v ) (as explained in the previous paragraph). This establishes the second partof Theorem 45. (cid:74) In this article, we have studied the existence of (weak) SPEs in quantitative games. Wehave proposed a Folk Theorem for weak SPEs, and a weaker version for SPEs. To illustratethe potential of this theorem, we have given two applications. The first one is concernedwith quantitative reachability games for which we have provided an algorithm to computea finite-memory SPE, and a second algorithm for deciding the constrained existence of a(finite-memory) SPE. The second application is concerned with another large class of gamesfor which we have proved that the (constrained) existence of a (finite-memory) weak SPE isdecidable.Future possible directions of research are the following ones. We would like to study thecomplexities of the problems studied for the two classes of games. We also want to investigatethe application of our Folk Theorem to other classes of games. The example of Figure 3 is agame with a weak SPE but no SPE (see Example 12). Recall that for this game, the cost λ i ( ρ ) can be seen as either the mean-payoff, or the liminf, or the limsup, of the weights of ρ .We do not know if games with this kind of payoff functions always have a weak SPE or not. References Thomas Brihaye, V´eronique Bruy`ere, Julie De Pril, and Hugo Gimbert. On subgameperfection in quantitative reachability games. Logical Methods in Computer Science , 9,2012. Thomas Brihaye, Julie De Pril, and Sven Schewe. Multiplayer cost games with simpleNash equilibria. In LFCS , volume 7734 of Lecture Notes in Comput. Sci. , pages 59–73.Springer, 2013. V´eronique Bruy`ere, No´emie Meunier, and Jean-Fran¸cois Raskin. Secure equilibria in weighted games. In CSL-LICS , page 26. ACM, 2014. Krishnendu Chatterjee, Laurent Doyen, and Thomas A. Henzinger. Quantitative lan-guages. ACM Trans. Comput. Log. , 11, 2010. J´anos Flesch, Jeroen Kuipers, Ayala Mashiah-Yaakovi, Gijs Schoenmakers, Eilon Solan,and Koos Vrieze. Perfect-information games with lower-semicontinuous payoffs. Math.Oper. Res. , 35:742–755, 2010. Drew Fudenberg and David Levine. Subgame-perfect equilibria of finite- and infinite-horizon games. Journal of Economic Theory , 31:251–268, 1983. Drew Fudenberg and Jean Tirole. Game Theory . MIT Press, 1991. Miroslav Klimos, Kim G. Larsen, Filip Stefanak, and Jeppe Thaarup. Nash equilibriain concurrent priced games. In LATA , volume 7183 of Lecture Notes in Comput. Sci. ,pages 363–376. Springer, 2012. H.W. Kuhn. Extensive games and the problem of information. Classics in Game Theory ,pages 46–68, 1953. Jean-Fran¸cois Mertens. Repeated games. In Internat. Congress Mathematicians , Amer-ican Mathematical Society, pages 1528–1577. Providence, RI, 1987. J. F. Nash. Equilibrium points in n -person games. In PNAS , volume 36, pages 48–49.National Academy of Sciences, 1950. Martin J. Osborne and Ariel Rubinstein. A course in Game Theory . MIT Press, Cam-bridge, MA, 1994. Amir Pnueli and Roni Rosner. On the synthesis of a reactive module. In POPL , pages179–190. ACM Press, 1989. St´ephane Le Roux and Arno Pauly. Infinite sequential games with real-valued payoffs.In CSL-LICS , page 62. ACM, 2014. Ariel Rubinstein. Comments on the interpretation of game theory. Econometrica , 59:909–924, 1991. Eilon Solan and Nicolas Vieille. Deterministic multi-player dynkin games. Journal ofMathematical Economics , 39:911–929, 2003. Wolfgang Thomas. Automata on infinite objects. In Handbook of Theoretical ComputerScience, Volume B: Formal Models and Sematics (B) , pages 133–192. 1990. Michael Ummels. Rational behaviour and strategy construction in infinite multiplayergames. In FSTTCS , volume 4337 of