Nash Equilibrium and Bisimulation Invariance
Julian Gutierrez, Paul Harrenstein, Giuseppe Perelli, Michael Wooldridge
LLogical Methods in Computer ScienceVolume 15, Issue 3, 2019, pp. 32:1–32:49https://lmcs.episciences.org/ Submitted Aug. 30, 2018Published Sep. 20, 2019
NASH EQUILIBRIUM AND BISIMULATION INVARIANCE
JULIAN GUTIERREZ a , PAUL HARRENSTEIN a ,GIUSEPPE PERELLI b , AND MICHAEL WOOLDRIDGE aa Department of Computer Science, University of Oxford e-mail address : { julian.gutierrez,paul.harrenstein,mjw } @cs.ox.ac.uk b Department of Computer Science, University of G¨oteborg e-mail address : [email protected]
Abstract.
Game theory provides a well-established framework for the analysis of concur-rent and multi-agent systems. The basic idea is that concurrent processes (agents) can beunderstood as corresponding to players in a game; plays represent the possible computationruns of the system; and strategies define the behaviour of agents. Typically, strategiesare modelled as functions from sequences of system states to player actions. Analysing asystem in such a setting involves computing the set of (Nash) equilibria in the concurrentgame. However, we show that, with respect to the above model of strategies (arguably,the “standard” model in the computer science literature), bisimilarity does not preservethe existence of Nash equilibria . Thus, two concurrent games which are behaviourallyequivalent from a semantic perspective, and which from a logical perspective satisfy thesame temporal logic formulae, may nevertheless have fundamentally different properties(solutions) from a game theoretic perspective. Our aim in this paper is to explore the issuesraised by this discovery. After illustrating the issue by way of a motivating example, wepresent three models of strategies with respect to which the existence of Nash equilibriais preserved under bisimilarity. We use some of these models of strategies to provide newsemantic foundations for logics for strategic reasoning, and investigate restricted scenarioswhere bisimilarity can be shown to preserve the existence of Nash equilibria with respectto the conventional model of strategies in the computer science literature. Introduction
The concept of bisimilarity plays a central role in both the theory of concurrency [Mil89,HM85] and logic [van76, HM85]. In the context of concurrency, bisimilar systems are regardedas behaviourally equivalent —appearing to have the same behaviour when interacting with anarbitrary environment. From a logical/verification perspective, bisimilar systems are knownto satisfy the same temporal logic properties with respect to languages such as LTL, CTL,or the µ -calculus [Pnu77, CE81, Koz83]. These features, in turn, make it possible to verifytemporal logic properties of concurrent systems using bisimulation-based approaches [SR12].For example, temporal logic model checking techniques [CGP02] may be optimised byapplying them to the smallest bisimulation equivalent model of the system being analysed; Key words and phrases:
Bisimulation, Nash Equilibrium, Logic and Games, Concurrency.
LOGICAL METHODS l IN COMPUTER SCIENCE DOI:10.23638/LMCS-15(3:32)2019 c (cid:13)
J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge CC (cid:13) Creative Commons
J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3 or, indeed, to any other model within the system’s bisimulation equivalence class. This ispossible because the properties that one is interested in checking are bisimulation invariant .Model checking [CGP02] is not the only verification technique that can benefit frombisimulation invariance: consider abstraction and refinement techniques [CGL94, CC02](where a set of states is either collapsed or broken down in order to build a somewhat simplerset of states); coinduction methods [San09] (which can be used to check the correctness ofan implementation with respect to a given specification); or reduced BDD representations ofa system [Bry92] (where isomorphic, and therefore bisimilar, subgraphs are merged, therebyeliminating part of the initial state space of the system). Bisimulation invariance is thereforea powerful and fundamental concept in the formal analysis and verification of concurrentand multi-agent systems, which plays an important role in many verification tools.Game theory provides another important framework for the analysis and verificationof concurrent and multi-agent systems. Within this framework, a concurrent/multi-agentsystem is viewed as a game, where processes/agents correspond to players, system executions(that is, computation runs) to plays, and individual process behaviours are modelled asplayer strategies, which are used to resolve the possible nondeterministic choices available toeach player. A widely-used model for strategies in concurrent games is to view a strategy fora process/agent/player i as a function f i which maps finite histories s , s , . . . , s k of systemstates to actions f i ( s , s , . . . , s k ) available to i at state s k . (In what follows, we use theterms process, agent, and player interchangeably.) We refer to this as the “conventional”model of strategies, as it is the best-known and most widely-used model in logic, AI, andcomputer science (and indeed in extensive form games [OR94]). For instance, specificationlanguages such as Alternating-time Temporal Logic (ATL [AHK02]), and formal modelssuch as concurrent game structures [AHK02] use this model of strategies. If we model aconcurrent/multi-agent system as a game in this way, then the analysis and verificationof the system reduces to computing the set of (Nash) equilibria in the associated game;in some cases, the analysis reduces to the computation of a winning strategy in the game,that is, a strategy that ensures that the players who follow such a plan will achieve theirgoal no matter how the other players in the system play, i.e. , against any other possiblecounter-strategy.Now, because bisimilar systems are regarded as behaviourally equivalent, and bisimilarsystems satisfy the same set of temporal logic properties, it is natural to ask whether theNash equilibria of bisimilar structures can be identified in a similar way; that is, we ask thefollowing question: Is Nash equilibrium invariant under bisimilarity?
We show that, for the “conventional” model of strategies, the answer to this question is,in general, no. More specifically, the answer critically depends on precisely how players’strategies are modelled. With the conventional model of strategies, we find the answeris positive only for some two-player games, but negative in general for games with morethan two players. This means, for instance, that, in the general case, bisimulation-basedtechniques cannot be used when one is also reasoning about the Nash equilibria of concurrentsystems that are formally modelled as multi-player (concurrent) games.For instance, given a concurrent and reactive system, represented as a collection ofindividual system components, say P , . . . , P n , one may want to know if a given temporallogic property, say ϕ , is satisfied by these system components whenever they choose touse strategies that form an equilibrium, that is, we want to know whether for some/every ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:3 computation run ρ ∈ NE ( P , . . . , P n ) we have ρ | = ϕ , where NE ( P , . . . , P n ) denotes the setof all computation runs that may be generated as a result of P , . . . , P n selecting strategiesthat form a Nash equilibrium. Because we are interested in concurrent systems, andbisimilarity is one of the most important behavioural equivalences in concurrency [Mil80,HM85, DV95, vGW96], it is highly desirable that properties which hold in equilibriumare sustained across all systems that are bisimilar to P , . . . , P n , meaning that for every(temporal logic) property ϕ and every process P (cid:48) i , if P (cid:48) i is bisimilar to P i ∈ { P , . . . , P n } , then ϕ is satisfied in equilibrium by P , . . . P i . . . P n if and only if is also satisfied in equilibrium by P , . . . P (cid:48) i . . . , P n , the system in which P i is replaced by P (cid:48) i , that is, across all bisimilar systemsto P , . . . , P n . This property, called invariance under bisimilarity , has been widely used fordecades for the semantic analysis ( e.g. , for modular and compositional reasoning) and formalverification ( e.g. , for temporal logic model checking) of concurrent systems. Unfortunately, asshown here, and already discussed in [GHW15a], the satisfaction of temporal logic propertiesin equilibrium is not invariant under bisimilarity, thus posing a verification challenge forthe modular and compositional reasoning of concurrent systems, since individual systemcomponents in a concurrent system cannot be replaced by (behaviourally equivalent) bisimilarones, while preserving the temporal logic properties that the overall system satisfies inequilibrium. This is also a problem from a synthesis point of view. Indeed, a strategy for asystem component P i may not be a valid strategy for a bisimilar system component P (cid:48) i . Asa consequence, the problem of building strategies for individual processes in the concurrentsystem P , . . . P i . . . P n may not, in general, be the same as building strategies for a bisimilarsystem P , . . . P (cid:48) i . . . P n , again, dashing any hope of modular reasoning on concurrent systems.Motivated by these observations—which bring together in a striking way a fundamentalconcept in game theory and a fundamental concept in logic/concurrency—the purpose of thepresent paper is to investigate these issues in detail. We first present a motivating example,to illustrate the basic point that using the conventional model of strategies, bisimulation neednot preserve Nash equilibria. We then present three alternative models of strategies in whichNash equilibria and their existence are preserved under bisimilarity. We also study the abovequestion for different classes of systems, for instance deterministic and nondeterministic ones,and explore applications to logic. Specifically, we investigate the implications of replacingthe conventional model of strategies with some of the models we propose in this paper inlogics for strategic reasoning [MMPV14, CHP10], in particular, the semantic implicationswith respect to Strategy Logic (SL [MMPV14]). We also show that, within the conventionalmodel of strategies, Nash equilibrium is preserved by bisimilarity in certain two-player gamesas well as in the class of concurrent game structures that are induced by iterated Booleangames [GHW15b], a framework that can be used to reason about the strategic behaviourof AI, autonomous, and multi-agent systems [WGH + A Motivating Example.
So far we have mentioned some cases where one needs ordesires a property to be invariant under bisimilarity. However, one may still wonder whyit is so important that the particular property of having a Nash equilibrium is preservedunder bisimilarity. One reason has its roots in automated formal verification. To illustratethis, imagine that the system of Figure 1 is given as input to a verification tool. It is likelythat such a tool will try to perform as many optimisations as possible to the system beforeany analysis is performed. Perhaps the simplest of such optimisations—as is being done byvirtually every model checking tool—is to reduce the input system by merging isomorphic
J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3 s ¯ p ¯ q s (cid:48) ¯ p ¯ q s ¯ p ¯ q s ¯ pq s p ¯ q s ¯ p ¯ q a, b , aa, b , a (cid:48) b , a, bb , a, b (cid:48) b , a, ab , a, a (cid:48) a, b , ba, b , b (cid:48) a, a, ∗ b , b , ∗ b , ∗ , aa, ∗ , b ∗ , b , a (cid:48) ∗ , a, b (cid:48) b , ∗ , aa, ∗ , b ∗ , b , a (cid:48) ∗ , a, b (cid:48) a, ∗ , ab , ∗ , b ∗ , a, a (cid:48) ∗ , b , b (cid:48) a, ∗ , ab , ∗ , b ∗ , a, a (cid:48) ∗ , b , b (cid:48) ∗ , ∗ , ∗∗ , ∗ , ∗∗ , ∗ , ∗ Figure 1: The game G on concurrent game structure M with a Nash equilibrium.subtrees. This is done, e.g. , when generating the ROBDD representation of a system. Ifsuch an optimisation is made, the tool will construct the (bisimilar) system in Figure 2.(Observe that the subgraphs rooted at s and s (cid:48) are isomorphic.) However, with respect tothe existence of Nash equilibria, such a transformation is unsound in the general case.For instance, suppose that the system in Figure 1 represents a 3-player game, whereeach transition is labelled by the choices x, y, z made by player 1, 2, and 3, respectively, andasterisk ∗ being a wildcard for any action for the player in the respective position. Thus,whereas players 1 and 2 can choose to play either a or b at each state, player 3 can choosebetween a , b , a (cid:48) , or b (cid:48) . The states are labelled by valuations xy over { p, q } , where ¯ x indicatesthat x is set to false. Assume that player 1 would like p to be true sometime, that player 2would like q to be true sometime, and that player 3 desires to prevent both player 1 andplayer 2 from achieving their goals. Accordingly, their preferences/goals can, respectively,be formally represented by the LTL formulae γ = F p, γ = F q, and γ = G ¬ ( p ∨ q ) , where, informally, F ϕ means “eventually ϕ holds” and G ϕ means “always ϕ holds”. Moreover,given these players’ goals and the conventional model of strategies, we will see later inSection 4 that the system in Figure 1 has a Nash equilibrium, whereas no Nash equilibriaexists in the (bisimilar) concurrent system presented in Figure 2.This example illustrates a major issue when analysing (the existence of) Nash equilibriain the most widely used models of strategies and multi-player games in the computer scienceliterature, namely, that even the simplest and most innocuous optimisations commonly usedin automated verification are not necessarily sound with respect to game-theoretic analyses.Because the problem is so fundamental, one may wonder whether bisimilarity is notthe right behavioural equivalence for multi-player games, or whether Nash equilibrium isnot the right solution concept for game-theoretic analyses of concurrent and multi-agentsystems modelled as multi-player games. We will discuss these questions in more detail ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:5 s ¯ p ¯ q s ¯ p ¯ q s p ¯ q s ¯ pq s ¯ p ¯ q a, b , ∗ b , a, ∗ b , ∗ , aa, ∗ , b ∗ , b , a (cid:48) ∗ , a, b (cid:48) a, a, ∗ b , b , ∗ a, ∗ , ab , ∗ , b ∗ , a, a (cid:48) ∗ , b , b (cid:48) ∗ , ∗ , ∗∗ , ∗ , ∗∗ , ∗ , ∗ Figure 2: The game G on concurrent game structure M without a Nash equilibrium.in Section 8, as we do not have a definite answer, but for now we would like to make acouple of observations. On the one hand, that our results also hold both for “alternating”(bisimilarity) relations, as defined in [AHKV98], which are intended to capture strategicbehaviour in multi-player games, as well as for trace equivalence, as defined in CSP [BHR84],an equivalence much weaker than bisimilarity. On the other hand, that our negative resultsalso hold for solution concepts stronger than Nash equilibrium, e.g. , for strong and subgame-perfect Nash equilibria, suggesting that the problem is not a particular defect of Nashequilibrium. Indeed, we think that the issue underlying the mismatch between bisimilarityand Nash equilibrium lies elsewhere. We will propose a very general solution to this problem,that is, a way to reconcile bisimilarity and Nash equilibrium, based on a new definitionof strategy in a multi-player game. To do this, some concepts and definitions have to beintroduced first. 2. Preliminaries
We begin by introducing the main notational conventions, models, and technical conceptsused in this paper.
Sets.
Given any set S = { s, q, r, . . . } , we use S ∗ , S ω , and S + for, respectively, the setsof finite, infinite, and non-empty finite sequences of elements in S . If w ∈ S ∗ and w isany other (finite or infinite) sequence, we write w w for their concatenation. The emptysequence is denoted by (cid:15) . Concurrent Game Structures.
We use the model of concurrent game structures, whichare well-established in the logic and computer science literatures (see, for instance, [AHK02]).A concurrent game structure (CGS) is a tuple M = (Ag , AP , Ac , St , s M , λ, δ ), where Ag = { , . . . , n } is a set of players or agents, AP a set of propositional variables , Ac is a set of actions , St is a set of states containing a unique initial state s M . With each player i ∈ Ag and each state s ∈ St, we associate a non-empty set Ac i ( s ) of feasible actions that, J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3 intuitively, i can perform when in state s . By a direction or decision we understand a profileof actions d = ( a , . . . , a n ) in Ac × · · · × Ac and we let Dir denote the set of directions. Adirection d = ( a , . . . , a n ) is legal at state s if a i ∈ Ac i ( s ) for every player i . Unless statedotherwise, by “direction” we will henceforth generally mean “legal direction”. Furthermore, λ : St → AP is a labelling function , associating with every state s a valuation v ∈ AP .Finally, δ is a deterministic transition function , which associates with each state s and everylegal direction d = ( a , . . . , a n ) at s a state δ ( s, a , . . . , a n ). As such δ characterises thebehaviour of the system when d = ( a , . . . , a n ) is performed at state s . Computations, Runs, and Traces.
The possible behaviours exhibited by a CGS can bedescribed at at least three different levels of abstraction. In what follows, we distinguishbetween computations , runs , and traces . Computations carry the most information, whiletraces carry the least, in the sense that every computation induces a unique run and everyrun induces a unique trace, but not necessarily the other way round. The distinctions wemake between computations, runs, and traces may appear to be insignificant, but are in factcentral in our analysis of bisimilarity and Nash equilibrium.
A state s (cid:48) is accessible from another state s whenever there is some d = ( a , . . . , a n ) suchthat d is legal at s and δ ( s, a , . . . , a n ) = s (cid:48) . For easy readability we then also write s d −→ s (cid:48) .An (infinite) computation is then an infinite sequence of directions κ = d , d , d , . . . such thatthere are states s , s , . . . with s = s M and s d −→ s d −→ s d −→ · · · . Observe that, havingassumed the transition function δ to be complete and deterministic, in every concurrentgame model the states s , s , . . . in the above definition always exist and are unique. A finitecomputation is any finite prefix of a computation κ . We also allow a finite computation to bethe empty sequence (cid:15) of directions. The sets of infinite and finite computations are denotedby comps ωM and comps M , respectively. We also use δ ∗ ( s, d , d , . . . d k ) to denote the uniquestate that is reached from the state s after applying the computation d , d , . . . d k .An (infinite) run is an infinite sequence ρ = s , s , s . . . of states of sequentially accessi-ble states, with s = s M . We say that run s , . . . , s k is induced by computation d , . . . , d k − if s d −→ s d −→ s d −→ · · · and s = s M . Thus, every computation induces a unique runand every run is induced by at least one computation. By a finite run or (finite) history wemean a finite prefix of a run. The sets of infinite and finite runs are denoted by runs ωM and runs M , respectively.An (infinite) trace is a sequence τ = v , v , v , . . . of valuations such that there isa run ρ = s , s , s , . . . in runs ωM such that v k = λ ( s k ) for every k ≥
0, that is, τ = λ ( s ) , λ ( s ) , λ ( s ) , . . . . In that case we say that trace τ is induced by run ρ , and if ρ isinduced by computation κ , also that τ is induced by κ . By a finite trace we mean a finiteprefix of a trace. We denote the sets of finite and infinite traces of a concurrent gamestructure M by traces M and traces ωM , respectively.We use ρ M ( κ ) to denote the run induced by a computation κ in CGS M , and write π M ( κ )if κ is finite on the understanding that π M ( (cid:15) ) = s M . Also, if ρ = s , s , s , . . . is a run, by τ M ( ρ ) we denote the trace λ ( s ) , λ ( s ) , λ ( s ) , . . . , and similarly for finite runs π ∈ runs M .Finally, τ M ( ρ M ( κ )) is abbreviated as τ M ( κ ). When no confusion is likely, we omit thesubscript M and the qualification ‘finite’. ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:7 Bisimilarity.
One of the most important behavioural/observational equivalences in concur-rency is bisimilarity, which is usually defined over Kripke structures or labelled transitionsystems (see, e.g. , [Mil89, HM85]). However, the equivalence can be uniformly definedfor general concurrent game structures, where decisions/directions play the role of, forinstance, actions in transition systems. Formally, let M = (AP , Ag , Ac , St , s M , λ, δ ) and M (cid:48) = (AP , Ag , Ac , St (cid:48) , s M (cid:48) , λ (cid:48) , δ (cid:48) ) be two concurrent game structures. A bisimulation , de-noted by ∼ , between states s ∗ ∈ St and t ∗ ∈ St (cid:48) is a non-empty binary relation R ⊆ St × St (cid:48) ,such that s ∗ R t ∗ and for all s, s (cid:48) ∈ St, t, t (cid:48) ∈ St (cid:48) , and d ∈ Dir: • s R t implies λ ( s ) = λ (cid:48) ( t ), • s R t and s d −→ s (cid:48) implies t d −→ t (cid:48)(cid:48) for some t (cid:48)(cid:48) ∈ St (cid:48) with s (cid:48) R t (cid:48)(cid:48) , • s R t and t d −→ t (cid:48) implies s d −→ s (cid:48)(cid:48) for some s (cid:48)(cid:48) ∈ St with s (cid:48)(cid:48) R t (cid:48) .Then, if there is a bisimulation between two states s ∗ and t ∗ , we say that they are bisimilar and write s ∗ ∼ t ∗ in such a case. We also say that concurrent game structures M and M (cid:48) are bisimilar (in symbols M ∼ M (cid:48) ) if s M ∼ s M (cid:48) . Since the transition functions of concurrentgame structures, as defined, are deterministic, we have the following simple but useful facts.We say that runs ρ = s , s , . . . and ρ (cid:48) = s (cid:48) , s (cid:48) , . . . are statewise bisimilar (in symbols ρ ˙ ∼ ρ (cid:48) )if s k ∼ s (cid:48) k for every k ≥
0. Both bisimilarity and statewise bisimilarity are equivalencerelations, which is a standard result in the literature (see, for instance, [DGL16, BK08,Mil89]).We find, moreover, that the sets of (finite) computations as well as the sets of (finite)traces of two bisimilar concurrent game structures are identical . In order to see this, thefollowing simple auxiliary result is useful.
Lemma 2.1.
Let M ∼ M (cid:48) , s, s (cid:48) ∈ St and t, t (cid:48) ∈ St (cid:48) , and d a direction. Then, s ∼ t , s d −→ s (cid:48) ,and t d −→ t (cid:48) together imply s (cid:48) ∼ t (cid:48) .Proof. Assume s ∼ t , s d −→ s (cid:48) , and t d −→ t (cid:48) . As M ∼ M (cid:48) , there is a t (cid:48)(cid:48) ∈ St such that t d −→ t (cid:48)(cid:48) and s (cid:48) ∼ t (cid:48)(cid:48) . Since the transition function is deterministic, moreover, it follows that t (cid:48)(cid:48) = t (cid:48) . Hence, s (cid:48) ∼ t (cid:48) , as desired.Using this observation we also have the following result. Lemma 2.2.
Let M and M (cid:48) be bisimilar concurrent game structures. Then, (1) comps ωM = comps ωM (cid:48) and comps M = comps M (cid:48) , (2) traces ωM = traces ωM (cid:48) and traces M = traces M (cid:48) .Proof. For part 1, let κ = d , d , . . . , d , . . . be a computation in comps ωM and s , s , s , . . . the states in St M such that s d −→ s d −→ s d −→ · · · . We show, by induction on k , that thereare states t , t , t , . . . in St M (cid:48) such that t d −→ t d −→ t d −→ · · · , where t = s M (cid:48) . It sufficesto prove by induction on k that for every k ≥ t k +1 such that t d −→ · · · d k −→ t k +1 and s k +1 ∼ t k +1 . For k = 0, we have t = s M (cid:48) . Then, observe that, by definition, s = s M and, as M ∼ M (cid:48) , it immediately follows that s ∼ t and that there is a t such that t d −→ t and s ∼ t . For the induction step, we may assume that there are t , . . . , t k with s M (cid:48) = t d −→ · · · d k − −−−→ t k and s k ∼ t k . By bisimilarity of M and M (cid:48) we then immediatelyobtain that there is a t k +1 such that t d −→ · · · d k − −−−→ t k d k −→ t k +1 . By Lemma 2.1 it thenfollows that s k +1 ∼ t k +1 . Hence, comps ωM ⊆ comps ωM (cid:48) . As the inclusion in the oppositedirection is proven by an analogous argument, we may conclude that comps ωM = comps ωM (cid:48) . J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3
It also follows that comps M = comps M (cid:48) , the latter being defined as the finite prefixesof comps ωM and comps ωM (cid:48) , respectively.Observe that from the argument in part 1 it also follows that ρ M ( κ ) ˙ ∼ ρ M (cid:48) ( κ ) forevery κ ∈ comps ωM = comps ωM (cid:48) . For part 2, consider an arbitrary trace τ ∈ traces ωM . Then,there is a computation κ ∈ comps ωM such that τ M ( κ ) = τ . By part 1, also κ ∈ comps ωM (cid:48) .Moreover, ρ M ( κ ) ˙ ∼ ρ M (cid:48) ( κ ). By the definition of (statewise) bisimilarity it then follows that τ = τ M ( κ ) = τ M (cid:48) ( κ ). Accordingly, traces ωM ⊆ traces ωM (cid:48) and the inclusion in the oppositedirection ensues by an analogous argument. We then conclude that traces M = traces M (cid:48) , thelatter being defined as the finite prefixes of traces ωM and traces ωM (cid:48) , respectively.Moreover, every (finite) computation κ gives rise to statewise bisimilar (finite) runs andidentical (finite) traces in bisimilar concurrent game structures. Lemma 2.3.
Let M and M (cid:48) be bisimilar concurrent game structures and κ ∈ comps ωM and κ (cid:48) ∈ comps M . Then, (1) ρ M ( κ ) ˙ ∼ ρ M (cid:48) ( κ ) and π M ( κ (cid:48) ) ˙ ∼ π M (cid:48) ( κ (cid:48) ) , (2) τ M ( κ ) = τ M (cid:48) ( κ ) and τ M ( κ (cid:48) ) = τ M (cid:48) ( κ (cid:48) ) .Proof. For part 1, first observe that by virtue of Lemma 2.2, we also have that κ ∈ comps ωM (cid:48) and κ ∈ comps M (cid:48) . Let κ = d , d , d , . . . , ρ M ( κ ) = s , s , s , . . . , and ρ M (cid:48) ( κ ) = t , t , t , . . . .We prove by induction on k that s k ∼ t k for every k ≥
0. If k = 0, then s = s M ∼ s M (cid:48) = t .For the induction step, we may assume that s k ∼ t k . Then, s k d k −→ s k +1 and t k d k −→ t k +1 .Lemma 2.1 now yields s k +1 ∼ t k +1 , as desired. The argument for the second part of 1proceeds by an analogous argument.Part 2 then follows almost immediately from part 1. Let ρ M ( κ ) = s , s , s , . . . and ρ M (cid:48) ( κ ) = t , t , t , . . . . Now observe that for every k ≥ λ M ( s k ) = λ M (cid:48) ( t k ).Accordingly, τ M ( κ ) = τ M (cid:48) ( κ ). For κ (cid:48) a similar argument yields the result.However, as runs are sequences of states and the states of different concurrent gamestructures M and M (cid:48) may be distinct, even if they are bisimilar, no identification of theirsets runs ωM and runs ωM (cid:48) of runs can generally be made.3. Games on Concurrent Game Structures
Concurrent game structures specify the actions the players can take at each state and whichstates are reached if they all concurrently decide on an action. In game theoretic terms,these structures loosely correspond to what are called game forms . A full understandingof the game-theoretic aspects of the system and the strategic behaviour of its constituentplayers—and therefore which computations/runs/traces will be generated in equilibrium—also essentially depends on what goals the players desire to achieve and on what strategiesthey may adopt in pursuit of these goals. We therefore augment concurrent game structureswith preferences and strategies for the players. In this way CGSs define fully fledgedstrategic games and as such they are amenable to game theoretic analysis by standardsolution concepts, among which Nash equilibrium is arguably the most prominent. ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:9
Strategies and Strategy Profiles.
Based on the distinction between computations, runs,and traces, we can also distinguish three types of strategy: computation-based, run-based,and trace-based strategies. The importance of these distinctions is additionally corroboratedby Bouyer et al. [BBMU11, BBMU15], who show how the specific model of strategiesadopted affects the computational complexity of some standard decision problems related tomulti-agent systems.A computation-based strategy for a player i in a concurrent game structure M is afunction f comp i : comps M → Ac,such that f comp i ( κ ) ∈ Ac i ( s k ) for every finite κ ∈ comps M with π M ( κ ) = s , . . . , s k . Thus,in particular, f comp i ( (cid:15) ) ∈ Ac i ( s M ), where (cid:15) is the empty sequence of directions.Similarly, a run-based strategy for player i is a function f run i : runs M → Ac,where f run i ( s , . . . , s k ) ∈ Ac i ( s k ) for every finite run ( s , . . . , s k ) ∈ runs M . Finally, a trace-based strategy for i is a function f trace i : traces M → Ac,such that f trace i ( τ ) ∈ Ac i ( s k ) for every trace τ ∈ traces M and every run π = s , . . . , s k suchthat τ = λ ( s ) , . . . , λ ( s k ).A computation-based strategy profile is then a tuple f = ( f , . . . , f n ) that associates witheach player i a computation-based strategy f i . Run-based and trace-based strategy profilesare defined analogously.Every computation-based strategy profile f = ( f , . . . , f n ) induces a unique computation κ M ( f ) = d , d , d , . . . in M that is defined inductively as follows: d = ( f ( (cid:15) ) , . . . , f n ( (cid:15) )) d k +1 = ( f ( d , . . . , d k ) , . . . , f n ( d , . . . , d k )) . A run-based strategy profile f = ( f , . . . , f n ) defines a unique computation κ M ( f ) = d , d , d , . . . in a similar manner: d = ( f ( s M ) , . . . , f n ( s M )), and d k +1 = ( f ( π ( d , . . . , d k )) , . . . , f n ( π ( d , . . . , d k ))).Finally, the computation κ M ( f ) defined by a trace-based strategy profile f is given by d = ( f ( λ ( s M )) , . . . , f n ( λ ( s M ))) d k +1 = ( f ( τ ( d , . . . , d k )) , . . . , f n ( τ ( d , . . . , d k ))).If M is clear from the context, we usually omit the subscript in κ M ( f ). For f = ( f , . . . , f n )a profile of computation-based, run-based, or trace-based strategies, we write with a slightabuse of notation ρ ( f , . . . , f n ) for ρ ( κ ( f , . . . , f n )) and τ ( f , . . . , f n ) for τ ( ρ ( f , . . . , f n )).As the computations of bisimilar concurrent games structures coincide (Lemma 2.2), wecan now establish that a player’s computation-based strategies coincide in bisimilar concurrentgame structures. Moreover, the computations induced by them will be identical. Also, fromthe coincidence of traces between bisimilar concurrent game structures (Lemma 2.2), we canestablish also trace-based strategies coincide in bisimilar concurrent game structures. J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3
Lemma 3.1.
Let M and M (cid:48) be bisimilar concurrent game structures and i a player. Then,every computation-based strategy for i in M is also a computation-based strategy for i in M (cid:48) ,and every trace-based strategy for i in M is also a trace-based strategy for i in M (cid:48) . Moreover,for every computation-based profile f for M we have that κ M ( f ) = κ M (cid:48) ( f ) , and for everytrace-based profile g that κ M ( g ) = κ M (cid:48) ( g ) .Proof. First, let f i be a computation-based strategy for i in M . We show that f i is also acomputation-based strategy for i in M (cid:48) . To this end, consider an arbitrary κ ∈ comps M (cid:48) .Let π M (cid:48) ( κ ) = t , . . . , t k . It suffices to prove that f i ( κ ) ∈ Ac i ( t k ). To see this, first observethat by Lemma 2.2 also κ ∈ comps M and let π M ( κ ) = s , . . . , s k . In virtue of Lemma 2.1,then s k ∼ t k . Moreover, because f i is a computation-based strategy for i in M , we have f i ( κ ) ∈ Ac i ( s k ). Now consider any legal direction d k = ( a , . . . , a n ) at s k with a i = f i ( κ ).Then, there is some state s k +1 ∈ St M such that s k d k −→ s k +1 . As s k ∼ t k , moreover, there isalso a state t k +1 ∈ St M (cid:48) such that t k d k −→ t k +1 . Accordingly, d k is legal at t k in M (cid:48) and inparticular a i = f i ( κ ) ∈ Ac i ( t k ) as desired.The case if g i is a trace-based strategy for i in M is similar. We then have to provethat g i is also a trace-based strategy for i in M (cid:48) as well. To this end, consider an arbitraryfinite trace τ ∈ traces M (cid:48) and run π = t , . . . , t k such that τ = λ M (cid:48) ( t ) , . . . , λ M (cid:48) ( t k ). Itthen suffices to prove that g i ( τ ) ∈ Ac i ( t k ). We may assume that π is induced by acomputation κ ∈ comps M (cid:48) , that is, π = π M (cid:48) ( κ ). By Lemma 2.2 we have κ ∈ comps M and by Lemma 2.3 both π M ( κ ) ˙ ∼ π M (cid:48) ( κ ) and τ M ( κ ) = τ M (cid:48) ( κ ). Let π M ( κ ) = s , . . . , s k .Hence, s k ∼ t k . As g i is a run-based strategy for i in M we have g i ( τ ) ∈ Ac i ( s k ). Let,furthermore, d = ( a , . . . , a k ) be a legal direction at s k with a i = g i ( τ ). Then, there is somestate s k +1 ∈ St M such that s k d k −→ s k +1 . As s k ∼ t k , there is also a state t k +1 ∈ St M (cid:48) suchthat t k d k −→ t k +1 . Accordingly, d k is legal at t k in M (cid:48) and in particular a i = g i ( τ ) ∈ Ac i ( t k ).For the second part of the lemma, let f = ( f , . . . , f n ) be a computation-based strategyprofile in M . Then, f is a computation-based strategy profile in M (cid:48) as well. Let κ M ( f ) = d , d , d , . . . and κ M (cid:48) ( f ) = d (cid:48) , d (cid:48) , d (cid:48) , . . . . We show by induction on k that for every k ≥ d k = d (cid:48) k . For k = 0, immediately, d = ( f ( (cid:15) ) , . . . , f n ( (cid:15) )) = d (cid:48) . For the inductionstep we may assume that d , . . . , d k = d (cid:48) , . . . , d (cid:48) k . Hence, d k +1 = ( f ( d , . . . , d k ) , . . . , f n ( d , . . . , d k )) = ( f ( d (cid:48) , . . . , d (cid:48) k ) , . . . , f n ( d (cid:48) , . . . , d (cid:48) k )) = d (cid:48) k +1 .Finally, let g = ( g , . . . , g n ) be a trace-based strategy profile. Again we let κ M ( f ) = d , d , d , . . . and κ M (cid:48) ( f ) = d (cid:48) , d (cid:48) , d (cid:48) , . . . and show by induction on k that for every k ≥ d k = d (cid:48) k . If k = 0, observe that having assumed M ∼ M (cid:48) also s M = s M (cid:48) .Accordingly, λ M ( s M ) = λ M (cid:48) ( s M (cid:48) ) and, hence, d = ( g ( λ M ( s M )) , . . . , g n ( λ M ( s M ))) = ( g ( λ M (cid:48) ( s M (cid:48) )) , . . . , g n ( λ M (cid:48) ( s M (cid:48) ))) = d (cid:48) k +1 .For the induction step, we may assume that d , . . . , d k = d (cid:48) , . . . , d (cid:48) k and by Lemma 2.3,moreover, τ M ( d , . . . , d k ) = τ M (cid:48) ( d (cid:48) , . . . , d (cid:48) k ). Now the following equations hold: d k +1 = ( g ( τ M ( d , . . . , d k )) , . . . , g n ( τ M ( d , . . . , d k )))= ( g ( τ M (cid:48) ( d (cid:48) , . . . , d (cid:48) k )) , . . . , g n ( τ M (cid:48) ( d (cid:48) , . . . , d (cid:48) k )))= d (cid:48) k +1 ,which concludes the proof. ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:11 With the states of bisimilar structures possibly being distinct, however, a statementanalogous to Lemma 3.1 cannot be shown to hold for run-based strategies.
Preferences and Goals.
We assume the agents of a concurrent game structure to havepreferences on basis of which they choose their strategies. Formally, we specify the preferencesof a player i of a CGS M as a subset Γ i of computations , that is, Γ i ⊆ comps ωM and referto Γ i as i ’s goal set . Player i is then understood to (strictly) prefer computations in Γ i to those not in Γ i and to be indifferent otherwise. Accordingly, each player’s preferencesare dichotomous, only distinguishing between the preferred computations in Γ i and thenot preferred ones not in Γ i . Formally, player i is said to weakly prefer computation κ tocomputation κ (cid:48) if κ ∈ Γ i whenever κ (cid:48) ∈ Γ i , and to strictly prefer κ to κ (cid:48) if i weakly prefers κ to κ (cid:48) but not the other way round. If i both weakly prefers κ to κ (cid:48) and weakly prefers κ (cid:48) to κ , player i is said to be indifferent between κ and κ (cid:48) .Our choice to assume the players’ preferences to be computation-based preferences—thatis, to model their goals as sets of computations rather than, say, sets of runs or sets oftraces—is for technical convenience and flexibility. Recall that every set of runs induces aset of computations, namely the set of computations that give rise to the same runs, andsimilarly for every set of traces. Thus, we say that a goal set Γ i ⊆ comps ωM is run-based if forany two computations κ and κ (cid:48) with ρ ( κ ) = ρ ( κ (cid:48) ) we have that κ ∈ Γ i if and only if κ (cid:48) ∈ Γ.Similarly, Γ i is said to be trace-based whenever τ ( κ ) = τ ( κ (cid:48) ) implies κ ∈ Γ i if and only if κ (cid:48) ∈ Γ i . In other words, in our setting, formally, run-based goals are computation-based goalsclosed under induced runs, and trace-based goals are computation-based goals closed underinduced traces. Sometimes—as we did in the example in the introduction—players’ goals are specifiedby temporal logic formulae [DGL16]. As the satisfaction of goals only depends on traces,they will directly correspond to trace-based goals, given our formalisation of goals andpreferences.
Games and Nash Equilibrium.
With the above definitions in place, we are now in aposition to define a game on a concurrent game structure M (also called a CGS-game ) withAg = { , . . . , n } as a tuple G = ( M, Γ , . . . , Γ n ),where, for each player i in M , the set Γ i ⊆ comps ωM is a goal set specifying i ’s dichotomouspreferences over the computations in M .In a CGS-game the players can all play either computation-based strategies, run-basedstrategies, or trace-based strategies. For each such choice of type of strategies, with the setof players and their preferences specified, every CGS-game defines a strategic game in thestandard game-theoretic sense. Observe that the set of strategies is infinite in general. Thusthe game-theoretic solution concept of Nash equilibrium becomes available for the analysisof games on concurrent game structures. If f = ( f , . . . , f n ) is a strategy profile and g i astrategy for player i , we write ( f − i , g i ) for the strategy profile ( f , . . . , g i , . . . , f n ), which is We do not directly consider sets of runs or sets of traces as possible models of players’ preferences in thispaper—formally, they are induced sets of computations. Accordingly, when talking about preferences, weneed not make the distinction between ‘run-based’ (‘trace-based’) and ‘run-invariant’ (‘trace-invariant’) as wedo for strategies. Our run-based preferences and trace-based preferences can with as much justification bereferred to as run-invariant preferences and trace-invariant preferences, respectively.
J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3 identical to f except that i ’s strategy is replaced by g i . Formally, given a CGS-game, wesay that a profile f = ( f , . . . , f n ) of computation-based strategies is a Nash equilibrium incomputation-based strategies (or computation-based equilibrium ) if, for every player i andevery computation-based strategy g i available to i , κ M ( f − i , g i ) ∈ Γ i implies κ M ( f ) ∈ Γ i .The concepts of Nash equilibrium in run-based strategies and
Nash equilibrium in trace-based strategies are defined analogously, where, importantly, the strategies in f − i and g i are required to be of the same type, that is, either they are all run-based or they are alltrace-based. If κ ( f ) / ∈ Γ i whereas κ ( f − i , g i ) ∈ Γ i , we also say that player i would like to deviate from f (and play g i instead). Thus, a run-based profile f is a run-based equilibrium whenever no player would like to deviate from it and play some run-based strategy differentfrom f i . Similarly, a trace-based profile f is a trace-based equilibrium if no player likes todeviate and play another trace-based strategy.We say that a computation κ , run ρ , or a trace τ is sustained by a Nash equilibrium f =( f , . . . , f n ) (of any type) whenever κ = κ ( f ), ρ = ρ ( f ), and τ = τ ( f ), respectively. Wealso refer to a computation, run, or trace that is sustained by a Nash equilibrium as an equilibrium computation , equilibrium run , and equilibrium trace , respectively.Computation-based equilibrium is a weaker notion than run-based equilibrium, in thesense that if f is a run-based equilibrium there is also a corresponding computation-basedequilibrium, but not necessarily the other way round. Run-based equilibrium, in turn, is ina similar way a weaker concept than trace-based equilibrium. As computation-based, run-based, and trace-based strategies are set-theoretically of different types, a comparison cannotbe made directly. To make the comparison precise, we therefore identify two subclasses ofcomputation-based strategies, run-invariant strategies and trace-invariant strategies , thatcharacterise the behaviour of, respectively, run-based and trace-based strategies.We say that a computation-based strategy f i : comps M → Ac i is run-invariant in CGS M whenever π M ( κ ) = π M ( κ (cid:48) ) implies f i ( κ ) = f i ( κ (cid:48) ), for all computations κ, κ (cid:48) ∈ comps M .Similarly, f i is trace-invariant in M whenever τ M ( κ ) = τ M ( κ (cid:48) ) implies f i ( κ ) = f i ( κ (cid:48) ), forall κ, κ (cid:48) ∈ comps M . Observe that thus a strategy f i being trace-invariant implies f i beingrun-invariant, but not necessarily the other way around.We observe that there are one-to-one correspondences between run-based strategies on theone hand and run-invariant computation-based strategies on the other, and similarly betweentrace-based strategies and trace-invariant computation-based strategies. Let f i : runs M → Acbe a run-based strategy. Then define ˇ f i : comps M → Ac as the computation-based strategysuch that for every finite computation κ ∈ comps M we have ˇ f i ( κ ) = f i ( π M ( κ )) . A similarstatment holds if g i : traces M → Ac is a trace-based strategy. Then, define ˇˇ g i : comps → Ac i as the computation-based strategy such that for every finite computation κ ∈ comps M wehave ˇˇ g i ( κ ) = g i ( τ M ( κ )) . Lemma 3.2.
For run-based strategies f i and trace-based strategies g i , the mapping thattransforms f i into ˇ f i and the mapping that transforms g i into ˇˇ g i are both one-to-one.Proof. Let f i a run-based strategy. We first show that ˇ f i is run-invariant. To this end, let κ, κ (cid:48) ∈ comps M be computations such that π M ( κ ) = π M ( κ (cid:48) ). Then,ˇ f i ( κ ) = f i ( π M ( κ )) = f i ( π M ( κ (cid:48) )) = ˇ f i ( κ (cid:48) ).To show that the mapping is onto, let g i be an arbitrary run-invariant strategy. Nowdefine run-based strategy ˆ g i such that, for every run π ∈ runs M and κ ∈ comps M with ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:13 π = π M ( κ ) we have ˆ g i ( π ) = g i ( κ ). Observe that ˆ g i is well-defined since, by run-invarianceof g i , for all κ, κ (cid:48) ∈ comps M with π M ( κ ) = π M ( κ (cid:48) ) = π we have that g i ( κ ) = g i ( κ (cid:48) ).Finally, to see that the mapping is injective, let f i and f (cid:48) i be two distinct run-basedstrategies. Then, there is a run π ∈ runs M such that f i ( π ) (cid:54) = f (cid:48) i ( π ). We may assume theexistence of a computation κ ∈ comps M such that π M ( κ ) = π . Then,ˇ f i ( κ ) = f i ( π M ( κ )) = f i ( π ) (cid:54) = f (cid:48) i ( π ) = f (cid:48) i ( π M ( κ )) = ˇ f i ( κ ),as desired. This concludes the proof.Furthermore, each profile of run-invariant strategies induces the same computation ina concurrent game structure as its run-based counterpart. A similar remark applies totrace-invariant and trace-based profiles. Lemma 3.3.
Let f = ( f , . . . , f n ) be a run-based profile and g = ( g , . . . , g n ) a trace-basedprofile. Then, κ M ( f , . . . , f n ) = κ M ( ˇ f , . . . , ˇ f n ) and κ M ( f , . . . , f n ) = κ M ( ˇˇ f , . . . , ˇˇ f n ) .Proof. Let κ M ( f , . . . , f n ) = d , d , d , . . . and κ M ( ˇ f , . . . , ˇ f n ) = d (cid:48) , d (cid:48) , d (cid:48) , . . . . We prove byinduction on k that d , . . . , d k = d (cid:48) , . . . , d (cid:48) k for every k ≥ k = 0, recall that π M ( (cid:15) ) = s M . Hence, d = ( f ( s M ) , . . . , f n ( s M )) = ( f ( π M ( (cid:15) )) , . . . , f n ( π M ( (cid:15) ))) = ( ˇ f ( (cid:15) ) , . . . , ˇ f n ( (cid:15) )) = d (cid:48) .For the induction step, we may assume that d , . . . , d k = d (cid:48) , . . . , d (cid:48) k . Now the followingequalities hold. d k +1 = ( f ( π M ( d , . . . , d k )) , . . . , f n ( π M ( d , . . . , d k )))= ( ˇ f ( d , . . . , d k ) , . . . , ˇ f n ( d , . . . , d k ))= i.h. ( ˇ f ( d (cid:48) , . . . , d (cid:48) k ) , . . . , ˇ f n ( d (cid:48) , . . . , d (cid:48) k ))= d (cid:48) k +1 .We may conclude that d , . . . , d k +1 = d (cid:48) , . . . , d (cid:48) k +1 . The argument for trace-based andtrace-invariant strategies runs along analogous lines, mutatis mutandis .We say that a computation-based profile f = ( f , . . . , f n ) is a run-invariant equilibrium in a CGS-game if f is run-invariant and no player i wishes to deviate from f and playanother run-invariant strategy f (cid:48) i . Similarly, a computation-based profile f = ( f , . . . , f n ) isa trace-invariant equilibrium in a CGS-game if f is trace-invariant and no player i wishes todeviate from f and play another trace-invariant strategy f (cid:48) i . As an immediate consequenceof Lemmas 3.2 and 3.3 we have the following corollary. Lemma 3.4.
Let f = ( f , . . . , f n ) and g = ( g , . . . , g n ) be a run-based profile, respectively,a trace-based profile in a CGS-game G = ( M, Γ , . . . , Γ n ) based on M . Then, (1) f is a run-based equilibrium if and only if ˇ f is a run-invariant equilibrium , (2) g is a trace-based equilibrium if and only if ˇˇ g is a trace-invariant equilibrium.Proof. For the run-based case, the following equivalences hold by virtue of Lemmas 3.2and 3.3: f is not a run-based equilibrium in G iff κ M ( f ) / ∈ Γ i and κ M ( f − i , f (cid:48) i ) ∈ Γ i for some run-based strategy f (cid:48) i for some player i J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3 iff κ M ( ˇ f ) / ∈ Γ i and κ M ( ˇ f − i , ˇ f (cid:48) i ) ∈ Γ i for some run-based strategy f (cid:48) i for some player i iff κ M ( ˇ f ) / ∈ Γ i and κ M ( ˇ f − i , f (cid:48)(cid:48) i ) ∈ Γ i for some run-invariant strategy f (cid:48)(cid:48) i for some player i iff ˇ f is not a run-invariant equilibrium in G The proof of the second part is by an analogous argument, mutatis mutandis .Computation-based strategies grant a player more strategic flexibility than do run-invariant strategies. A similar remark applies to run-invariant strategies and trace-invariantstrategies. Still, we find that, if a player i wishes to deviate from a computation-basedprofile f and play another computation-based strategy, i would also like to deviate by playinga run-invariant or even a trace-invariant strategy. This insight underlies the following result. Theorem 3.5.
Let f = ( f , . . . , f n ) be a run-invariant profile and g = ( g , . . . , g n ) atrace-invariant profile in CGS-game G = ( M, Γ , . . . , Γ n ) based on M . Then, (1) f is a run-invariant equilibrium if and only if f is a computation-based equilibrium, (2) g is a trace-invariant equilibrium if and only if g is a computation-based equilibrium.Proof. For part 1, first assume that f is a run-invariant equilibrium in G . For a contradictionassume moreover that f is not a computation-based equilibrium. Then, there is a player i and a computation-based strategy f (cid:48) i such that κ M ( f ) / ∈ Γ i whereas κ M ( f − i , f (cid:48) i ) ∈ Γ i . Let κ M ( f − i , f (cid:48) i ) = d (cid:48) , d (cid:48) , d (cid:48) , . . . . Observe that f (cid:48) i need not be run-invariant. We therefore definestrategy f (cid:48)(cid:48) i for player i such that f (cid:48)(cid:48) i ( (cid:15) ) = f (cid:48) i ( (cid:15) ) and, for all finite computations d , . . . , d k , f (cid:48)(cid:48) i ( d , . . . , d k ) = (cid:40) f (cid:48) i ( d (cid:48) , . . . , d (cid:48) k ) if π M ( d , . . . , d k ) = π M ( d (cid:48) , . . . , d (cid:48) k ), f i ( d , . . . , d k ) otherwise.As f i is run-invariant, this definition guarantees that f (cid:48)(cid:48) i is run-invariant as well. Let κ M ( f − i , f (cid:48)(cid:48) i ) = d (cid:48)(cid:48) , d (cid:48)(cid:48) , d (cid:48)(cid:48) , . . . . We prove by induction on k that d (cid:48) , . . . , d (cid:48) k = d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k , forevery k ≥
0, and hence that κ M ( f − i , f (cid:48) i ) = κ M ( f − i , f (cid:48)(cid:48) i ). If k = 0, we immediately obtainthat d (cid:48) = ( f ( (cid:15) ) , . . . , f (cid:48) i ( (cid:15) ) , . . . , f n ( (cid:15) )) = ( f ( (cid:15) ) , . . . , f (cid:48)(cid:48) i ( (cid:15) ) , . . . , f n ( (cid:15) )) = d (cid:48)(cid:48) .For the induction step, we may assume that d (cid:48) , . . . , d (cid:48) k = d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k . and the followingequalities hold: d (cid:48) k +1 = ( f ( d (cid:48) , . . . , d (cid:48) k ) , . . . , f (cid:48) i ( d (cid:48) , . . . , d (cid:48) k ) , . . . , f n ( d (cid:48) , . . . , d (cid:48) k ))= ( f ( d (cid:48) , . . . , d (cid:48) k ) , . . . , f (cid:48)(cid:48) i ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k ) , . . . , f n ( d (cid:48) , . . . , d (cid:48) k ))= i.h. ( f ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k ) , . . . , f (cid:48)(cid:48) i ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k ) , . . . , f n ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k ))= d (cid:48)(cid:48) k +1 .Observe that the second equality holds by virtue of the definition of f (cid:48)(cid:48) i and π M ( d (cid:48) , . . . , d (cid:48) k ) = π M ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k ). It would follow that κ M ( f − i , f (cid:48)(cid:48) i ) ∈ Γ i as well, and, as f (cid:48)(cid:48) i is run-invariant,moreover that f is not a run-invariant equilibrium, a contradiction.For the opposite direction, assume for contraposition that f is not a run-invariantequilibrium. Then, there is some player i who would like to deviate from f and play some The situation can be compared to the relation between equilibria in pure and mixed (or randomised )strategies in game theory. There every equilibrium in pure strategies is also an equilibrium in mixed strategies,because, if a player wishes to deviate from a mixed profile, she wishes to deviate by playing a pure, that is,not randomised, strategy. ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:15 run-invariant strategy f (cid:48) i . As run-invariant strategies are strategy-based by definition, itfollows that f is not a computation-based equilibrium either.Part 2 follows by an analogous argument, mutatis mutandis .Theorem 3.5 does not preclude the existence of computation-based equilibria that failto be run-invariant or trace-invariant, that is, the three equilibrium concepts—computation-based, run-invariant, and trace-invariant equilibrium—are not equivalent. However, theycan be ordered with respect to how restrictive they are, that is, with respect to the sets ofprofiles they exclude as solutions. Corollary 3.6.
Let f = ( f , . . . , f n ) be a computation-based profile in some CGS-game G =( M, Γ , . . . , Γ n ) based on M . Then, (1) f is a run-invariant equilibrium implies f is a computation-based equilibrium, (2) f is a trace-invariant equilibrium implies f is a run-invariant equilibrium.Proof. Merely observe that if f is a run-invariant equilibrium, it is also a run-invariantprofile. If f is moreover trace-invariant it is also run-invariant. The result then immediatelyfollows from Theorem 3.5.On basis of the findings in this section, we may with justification claim that everytrace-based equilibrium corresponds to a run-based equilibrium, and that every run-basedequilibrium corresponds with some computation-based equilibrium, even if the converses ofthese statements do not generally hold.4. Invariance of Nash Equilibria under Bisimilarity
From a computational point of view, one may expect games based on bisimilar concurrentgame structures and with identical players’ preferences to exhibit similar properties, inparticular with respect to their Nash equilibria. We find that that this is indeed the case forgames with computation-based strategies as well as for games with trace-based strategies.Recall that (finite) computations and (finite) traces are unaffected by (state-splitting andstate-merging) operations on CGSs that preserve bisimilarity (Lemma 2.2). As a consequencethe sets of computation-based strategies and trace-based strategies available to an again arethe same in bisimilar CGSs (Lemma 3.1), providing the intuitive basis for these observations.For games with run-based strategies the situation is considerably more complicated.Here, a key observation is that, by contrast to computation-based and trace-based strategies,there need not be a natural one-to-one mapping between the sets of run-based strategies inbisimilar concurrent game models. By restricting attention to so-called bisimulation-invariantrun-based strategies, however, we find that order can be restored.
Invariance under Bisimilarity and Preference Types.
We are primarily interestedin the Nash equilibria of games that are the same up to bisimilarity of the underlyingconcurrent game structures. The Nash equilibria of a game, however, essentially depend onthe players’ preferences. Accordingly, the Nash equilibria of two bisimilar CGS-games canonly be meaningfully compared if we also we assume that the players’ preferences in thesetwo games are identical. We formalised players’ preferences as sets of computations, and,due to Lemma 2.2, this enables a straightforward comparison of players’ goal sets acrossbisimilar concurrent game structures.
J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3 s p s p s p s ¯ pa, aa, bb, ab, b ∗ , ∗∗ , ∗∗ , ∗ t p t p t p t ¯ pa, aa, bb, ab, b ∗ , ∗∗ , ∗∗ , ∗ Figure 3: Two games G (left) and G (right) based on M and M , respectively, showingthat run-based preferences may not be preserved across bisimilar systems.In Section 3, we also distinguished run-based and trace-based preferences, that is, goalsets closed under computations that induce the same runs and traces, respectively. We arealso interested in the invariance of the existence of Nash equilibria in games on bisimilarconcurrent game structures where the players’ preferences games are what we will call congruent , that is, both the same and of the same type in both games .For computation-based and trace-based preferences the issue of congruence is moot.Observe that for bisimilar concurrent game structures M and M (cid:48) , if a goal set Γ i iscomputation-based in M , then it is also computation-based in M (cid:48) . Due to Lemma 2.3, thesame holds for trace-based preferences.This preservation of preference type under bisimilarity, however, does not extend torun-based preferences. To see this, consider Figure 3 and let the goal set Γ i of some player i be given by all computations κ = d , d , d , . . . with d = ( a, a ). Then, obviously, Γ i isrun-based in the game G based on M on the left, but not in the game G based on M tothe right. To see the latter, consider any computation κ (cid:48) = d (cid:48) , d (cid:48) , d (cid:48) , . . . with d (cid:48) = ( a, b ).Then, κ (cid:48) / ∈ Γ i , but, nevertheless, in G we have ρ M ( κ ) = ρ M ( κ (cid:48) ). By contrast, the goal setgiven by all computations κ = d , d , d , . . . such that d (cid:54) = ( b, b ) is run-based in both games. Computation-based Strategies.
If strategies are computation-based, players can havetheir actions depend on virtually all information that is available in the system. In animportant sense full transparency prevails and different actions can be chosen on bisimilarstates provided that the computations that led to them are different. Moreover, the strategiesavailable to players in bisimilar concurrent game structures are identical. Thus we obtainour first main result.
Theorem 4.1.
Let G = ( M, Γ , . . . , Γ n ) and G (cid:48) = ( M (cid:48) , Γ , . . . , Γ n ) be games on bisimilarconcurrent game structures M and M (cid:48) , respectively, and let f = ( f , . . . , f n ) be a computation-based profile. Then, f is a Nash equilibrium in computation-based strategies in G if and onlyif f is a Nash equilibrium in computation-based strategies in G (cid:48) .Proof. First assume for contraposition that f = ( f , . . . , f n ) is not a Nash equilibrium incomputation-based strategies in M (cid:48) . Then, there is a player i and a strategy g i for i in M (cid:48) such that κ M (cid:48) ( f ) / ∈ Γ i and κ M (cid:48) ( f − i , g i ) ∈ Γ i . Observe that, as the computation-based ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:17 strategies of players in bisimilar structures coincide (cf., Lemma 3.1), f is also a strategyprofile in M and g i a strategy for i in M . Moreover, recall that the computations induced bythe same strategy profile in different but bisimilar structures are identical (cf., second partof Lemma 3.1). This yields κ M ( f ) = κ M (cid:48) ( f ) and κ M ( f − i , g i ) = κ M (cid:48) ( f − i , g i ). Accordingly, κ M ( f ) / ∈ Γ i whereas κ M ( f − i , g i ) ∈ Γ i . We may conclude that f is not a computation-basedequilibrium in M either. The opposite direction follows by an analogous argument.Theorem 4.1 holds for computation-based preferences. As run-based preferences andtrace-based preferences are computation-based preferences of a special kind, the resultimmediately extends to games in which the players’ preferences are run-based in both gamesor trace-based preferences in both games. As a consequence of Theorem 4.1, moreover, wefind that sustenance of runs and traces by computation-based equilibrium is also preservedunder bisimilarity. Corollary 4.2.
Let G = ( M, Γ , . . . , Γ n ) and G (cid:48) = ( M (cid:48) , Γ , . . . , Γ n ) be games on bisimilarconcurrent game structures M and M (cid:48) , respectively, κ ∈ comps ωM , and τ ∈ traces ωM . Then, (1) κ is sustained by computation-based equilibrium in G if and only if κ is sustained by acomputation-based equilibrium in G (cid:48) . (2) τ is sustained by a computation-based equilibrium in G if and only if τ is sustained by acomputation-based equilibrium in G (cid:48) .Proof. Recall that by Lemma 2.2, we have that κ ∈ comps ωM (cid:48) and τ ∈ traces ωM (cid:48) . For part 1,let κ M ( f ) = κ , where f is a computation-based equilibrium in G (cid:48) . Then, by Theorem 4.1,profile f is a computation-based equilibrium in G (cid:48) as well. By virtue of Lemma 3.1, moreover, κ M ( f ) = κ M (cid:48) ( f ), which gives the result. The implication in the other direction follows bythe same argument mutatis mutandis .The argument for part 2 runs along analogous lines. First assume that τ is sustainedby computation-based equilibrium f , that is, τ = τ M ( f ). By Theorem 4.1, we have that f is a computation-based equilibrium in G (cid:48) as well. Now consider κ M ( f ). By Lemma 3.1then κ M (cid:48) ( f ) = κ M ( f ). Lemma 2.3 then yields τ M ( κ M ( f )) = τ M (cid:48) ( κ M (cid:48) ( f )). It thus followsthat τ is sustained by f , a computation based Nash equilibrium, in G (cid:48) . The argument inthe opposite direction is analogous, giving the result. Trace-based Strategies.
As we saw in Lemma 2.2, the sets of (finite) traces of twobisimilar concurrent game structures coincide. Lemma 3.1 shows that the same holds for thetrace-based strategies that are available to the players. As a consequence, we can directlycompare their trace-based Nash-equilibria. We find that, like computation-based equilibria,trace-based Nash equilibria are preserved in CGS-games based on bisimilar concurrent gamestructures.
Theorem 4.3.
Let G = ( M, Γ , . . . , Γ n ) and G (cid:48) = ( M (cid:48) , Γ , . . . , Γ n ) be games on bisimilarconcurrent game structures M and M (cid:48) , respectively, and f = ( f , . . . , f n ) be a trace-basedstrategy profile. Then, f is a Nash equilibrium in trace-based strategies in G if and only if f is a Nash equilibrium in trace-based strategies in G (cid:48) .Proof. The proof is analogous to the one for Theorem 4.1 for the computation-based case.First assume for contraposition that f = ( f , . . . , f n ) is not a Nash equilibrium in trace-basedstrategies in M (cid:48) . Then, there is a player i and a trace-based strategy g i for i in M (cid:48) suchthat κ ( f ) / ∈ Γ i and κ ( f − i , g i ) ∈ Γ i . Observe that, as the trace-based strategies of players in J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3 bisimilar structures coincide (cf., Lemma 3.1), we have that f is also a trace-based strategyprofile in M and g i a trace-based strategy for i in M . By the second part of Lemma 3.1,moreover, κ M ( f ) = κ M (cid:48) ( f ) and κ M ( f − i , g i ) = κ M (cid:48) ( f − i , g i ). Accordingly, κ M ( f ) / ∈ Γ i whereas κ M ( f − i , g i ) ∈ Γ i . We may conclude that f is not a trace-based equilibrium in M either. The opposite direction follows by an analogous argument.Like Theorem 4.1, this result is for computation-based preferences in general, and assuch immediately extends to the case in which the players’ preferences are stipulated tobe run-based in both games or trace-based in both games. Theorem 4.3 has further thefollowing result as an immediate consequence, which is analogous to Corollary 4.2. Corollary 4.4.
Let G = ( M, Γ , . . . , Γ n ) and G (cid:48) = ( M (cid:48) , Γ , . . . , Γ n ) be games on bisimilarconcurrent game structures M and M (cid:48) , respectively, κ ∈ comps ωM , and τ ∈ traces ωM . Then, (1) κ is sustained by trace-based equilibrium in G if and only if κ is sustained by a trace-basedequilibrium in G (cid:48) . (2) τ is sustained by a trace-based equilibrium in G if and only if τ is sustained by atrace-based equilibrium in G (cid:48) .Proof. The proof is analogous to the one for Corollary 4.2.
Run-based Strategies.
The positive results obtained using computation-based and trace-based strategies are now followed by a negative result, already mentioned in the introductionof the paper, which establishes that Nash equilibria in run-based strategies—the most widely-used strategy model in logic, computer science, and AI—are not preserved by bisimilarity.Previously we observed that the players’ run-based strategies cannot straightforwardly beidentified across two different concurrent game structures, even if they are bisimilar. Wewould therefore have to establish a correspondence between the run-based strategies in theone game and the run-based strategies in the other in an arguably ad hoc way. To cut thisGordian knot, we therefore show in this section the stronger result that the very existence ofrun-based equilibria is not preserved under bisimilarity. That is, we can have two bisimilarconcurrent game structures, say M and M (cid:48) , on which we base two games G and G (cid:48) withcongruent preferences, such that G has a Nash equilibrium and G (cid:48) does not. Theorem 4.5.
The existence of run-based Nash equilibria is not preserved under bisimilarity.That is, there are games ( M, Γ , . . . , Γ n ) and ( M (cid:48) , Γ , . . . , Γ n ) on bisimilar concurrent gamestructures M and M (cid:48) such that a Nash equilibrium in run-based strategies exists in G butnot in G (cid:48) . To see that the above statement holds, consider again the three-player game G on theconcurrent game structure M in Figure 1. Assume, as before, that player 1’s goal set Γ is given by those computations κ such that τ M ( κ ) = v , v , v , . . . , implies p ∈ v k for some k ≥
0. Similarly, Γ consists of all computations κ with τ M ( κ ) = v , v , v , . . . and q ∈ v k for some k ≥ by those computations κ with τ M ( κ ) = v , v , v , . . . and v k = ∅ forall k ≥
0. Recall that, consequently, the preferences of players 1, 2, and 3 are trace-based andcan be represented by the LTL formulas γ = F p , γ = F q , and γ = G ¬ ( p ∨ q ), respectively.Let f ∗ and f ∗ be any run-based strategies for players 1 and 2 such that f ∗ ( s ) = f ∗ ( s ) = a . Let, furthermore, player 3’s run-based strategy f ∗ be such that f ∗ ( s ) = a , f ∗ ( s , s ) = a (cid:48) , and f ∗ ( s , s (cid:48) ) = b . ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:19 Let f ∗ = ( f ∗ , f ∗ , f ∗ ) and observe that ρ M ( f ∗ ) = s , s , s , s , . . . . Accordingly, player 3has her goal achieved and does not want to deviate from f ∗ . Players 1 and 2 do nothave their goals achieved, but do not want to deviate from f ∗ either. To see this, let g be any run-based strategy for 1 such that g ( s ) = b ; observe that this is required forany meaningful deviation from f ∗ by 1. Then ρ M ( g , f ∗ , f ∗ ) = s , s , s , s , s , . . . or ρ M ( g , f ∗ , f ∗ ) = s , s , s , s , s , . . . , depending on whether f ∗ ( s , s ) = b or f ∗ ( s , s ) = a ,respectively. In either case, player 1 does not get his goal achieved and it follows that hedoes not want to deviate from f ∗ . An analogous argument—notice that the roles of player 1and 2 are symmetric—shows that player 2 does not want to deviate from f ∗ either. We maythus conclude that f ∗ is a run-based equilibrium in G .Now, consider the game G on concurrent game structure M in Figure 2 with the players’preferences as in M . It is easy to check that M and M are bisimilar. Still, there is norun-based equilibrium in G . To see this, consider an arbitrary run-based strategy profile f =( f , f , f ). First, assume that ρ M ( f ) = s , s , s , s , s , . . . . Then, player 1 gets his goalachieved and players 2 and 3 do not. If f ( s , s ) = a then f ( s , s ) = b and player 3 wouldlike to deviate and play a strategy g with g ( s , s ) = a . On the other hand, if f ( s , s ) = b ,player 3 would like to deviate and play a strategy g with g ( s , s ) = b . Player 3 wouldsimilarly like to deviate from f if we assume that ρ M ( f ) = s , s , s , s , s . . . , in whose caseit is player 2 who gets his goal achieved. Now, assume that ρ M ( f ) = s , s , s , s , s . . . . Inthis case player 3 does get her goal achieved, but players 1 and 2 do not. However, player 1would like to deviate and play a strategy g with g ( s , s ) = b or g ( s , s ) = a , dependingon whether f ( s , s ) = a or f ( s , s ) = b ; in a similar fashion, player 2 would like to deviateand play a strategy g with g ( s , s ) = b if f ( s , s ) = a (cid:48) , and to one with g ( s , s ) = a if f ( s , s ) = b (cid:48) . Finally, assume that ρ M ( f ) = s , s , s , s , . . . . Then, neither player 1nor player 2 gets his goal achieved. Now either f ( s , s ) ∈ { a, b } or f ( s , s ) ∈ { a (cid:48) , b (cid:48) } . Ifthe former, player 1 would like to deviate and play a strategy g with g ( s ) (cid:54) = f ( s ) and g ( s , s ) (cid:54) = f ( s , s ). If the latter, player 2 would like to deviate and play a strategy g with g ( s ) (cid:54) = f ( s ) and either g ( s , s ) = b if f ( s , s ) = a (cid:48) or g ( s , s ) = a if f ( s , s ) = b (cid:48) .We can then conclude that the CGS-game G does not have any run-based Nash equilibria.The main idea behind this counter-example is that in G player 3 could distinguishwhich player deviates from f ∗ if the state reached after the first round is not s : if that stateis s , it was player 1 who deviated, otherwise player 2. By choosing either a (cid:48) or b (cid:48) at s , andeither a or b at s (cid:48) , player 3 can guarantee that neither player 1 nor player 2 gets his goalachieved (“punish” them) and thus deter them from deviating from f ∗ . This possibilityto punish deviations from f ∗ by players 1 and 2 in a single strategy is not available in thegame on M : choosing from a and b will induce a deviation by player 1, choosing from a (cid:48) and b (cid:48) one by player 2.Observe that the games G and G do not constitute a counter-example against eitherthe preservation under bisimilarity of computation-based equilibria or the preservation oftrace-based equilibria. The reasons why such games fail to do so, however, are different. Forthe setting of computation-based strategies, player 3 can still distinguish and “punish” thedeviating player in G as ( a, b, a ) and ( b, a, a ) are different directions and player 3 can stillhave his action at s depend on which of these is played at s . By contrast, if we assumetrace-based strategies, player 3 has to choose the same action at both s and s (cid:48) in G . As aconsequence, and contrarily to computation-based equilibria, trace-based equilibria exist inneither G nor G . Also note that the goal sets Γ , Γ , and Γ are run-based as well as J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3 computation-based both in G and G . Accordingly, the counter-example also applies tosettings wherein the players’ preferences are assumed to be finer-grained in these two ways.Observe at this point that s and s (cid:48) are bisimilar states. Yet, players are allowed to haverun-based strategies (which depend on state histories only) that choose different actions atbisimilar states. The above counter-example shows how this relative richness of strategiesmakes a crucial difference. This raises the question as to whether we actually want playersto adopt run-based strategies in which they choose different actions at bisimilar states. Thisobservation leads us to the next section. Bisimulation-invariant Run-based Strategies.
Bisimilarity formally captures an infor-mal concept of observational indistinguishability on the part of an external observer of thesystem. Now, the players in a concurrent game structure are in essentially the same situationas an external observer if they are assumed to be only able to observe the behaviour of theother players, without knowing their internal structure or their interaction.Drawing on this idea of indistinguishability, it is natural that players cannot distinguishstatewise bisimilar runs and, as a consequence, can only adopt strategies that choose thesame action at runs that are statewise bisimilar. The situation is comparable to the one inextensive games of imperfect information, in which players are required to choose the sameaction in histories that are in the same information set, that is, histories that cannot bedistinguished (cf., e.g. , [OR94, MSZ13]).To make this idea formally precise, we say that a run-based strategy f i is bisimulation-invariant if f i ( π ) = f i ( π (cid:48) ) for all histories π and π (cid:48) that are statewise bisimilar. The conceptof Nash equilibrium is then similarly restricted to bisimulation-invariant strategies. A profile f = ( f , . . . , f n ) of bisimulation-invariant strategies is a Nash equilibrium in bisimulation-invariant strategies (or a bisimulation-invariant equilibrium ) in a game ( M, Γ , . . . , Γ n ) iffor all players i and every bisimulation-invariant strategy g i for i , τ ( f − i , g i ) ∈ Γ i implies τ ( f ) ∈ Γ i That is, f is a bisimulation-invariant equilibrium if no player wishes to deviate from f by playing a different bisimulation-invariant strategy. In contrast to the situation forgeneral run-based strategies, we find that computations and traces that are sustained bya bisimulation-invariant Nash equilibrium are preserved by bisimulation. We show thisby establishing a one-to-one correspondence between the bisimulation-invariant strategiesavailable to the players in two bisimilar structures.Based on the concept of state-wise bisimilarity, we associate with every bisimulation-invariant strategy f i for player i in concurrent game structure M , another bisimulation-invariant strategy ˜ f i for player i in any bisimilar concurrent game structure M (cid:48) such thatfor all π ∈ runs M (cid:48) and a ∈ Ac,˜ f i ( π ) = a if f i ( π (cid:48) ) = a for some π (cid:48) ∈ runs M with π ˙ ∼ π (cid:48) . Transitivity of ˙ ∼ guarantees that ˜ f i is well defined. To see this, observe that for all π (cid:48) , π (cid:48)(cid:48) ∈ runs M with π (cid:48) ˙ ∼ π and π (cid:48)(cid:48) ˙ ∼ π , we have π (cid:48) ˙ ∼ π (cid:48)(cid:48) . Having assumed that f i is bisimulation-invariant, then f i ( π (cid:48) ) = f i ( π (cid:48)(cid:48) ). By very much the same argument, ˜ f i isbisimulation-invariant, if f i is. Lemma 4.6.
Let M and M (cid:48) be bisimilar concurrent game structures and let f i be abisimulation-invariant strategy for player i in M . Then, ˜ f i is a bisimulation-invariantstrategy in M (cid:48) . ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:21 Proof.
Consider two statewise bisimilar runs π, π (cid:48) ∈ runs M (cid:48) , that is, π ˙ ∼ π (cid:48) . Then, there arecomputations κ, κ (cid:48) ∈ comps M (cid:48) such that π M (cid:48) ( κ ) = π and π M (cid:48) ( κ (cid:48) ) = π (cid:48) . By Lemma 2.3, wehave π M ( κ ) ˙ ∼ π M (cid:48) ( κ ) and π M ( κ (cid:48) ) ˙ ∼ π M (cid:48) ( κ (cid:48) ). Now, transitivity of ˙ ∼ yields π M ( κ ) ˙ ∼ π M ( κ (cid:48) ).Having assumed that f i is bisimulation-invariant, we obtain that f i ( π M ( κ )) = f i ( π M ( κ (cid:48) )).Finally, we may conclude that ˜ f i ( π ) = ˜ f i ( π (cid:48) ), as desired.Moreover, it is easily appreciated that the mapping that transforms a strategy f i intostrategy ˜ f i is one-to-one. We will find that this is an essential property for bisimulation-invariant equilibria to be preserved under bisimilarity.For a profile of bisimulation-invariant strategies f = ( f , . . . , f n ) in M we denote˜ f = ( ˜ f , . . . , ˜ f n ). We then find that profiles f and ˜ f of bisimulation-invariant strategiesinduce identical computations. Lemma 4.7.
Let M and M (cid:48) be bisimilar concurrent game structures, f = ( f , . . . , f n ) abisimulation-invariant strategy profile. Then, κ M ( f ) = κ M (cid:48) ( ˜ f ) .Proof. Let κ M ( f ) = d , d , d , . . . and κ M (cid:48) ( ˜ f ) = d (cid:48) , d (cid:48) , d (cid:48) , . . . . We prove by induction on k that d k = d (cid:48) k for every k ≥
0. If k = 0, we have d = ( f ( s M ) , . . . , f n ( s M )). Observe that, as M ∼ M (cid:48) also s M ∼ s M (cid:48) and, hence, f i ( s M ) = ˜ f i ( s M (cid:48) ). Therefore, d (cid:48) = ( ˜ f ( s M (cid:48) ) , . . . , ˜ f n ( s M (cid:48) )) = ( f ( s M ) , . . . , f n ( s M )) = d .For the induction step, we may assume that d , . . . , d k = d (cid:48) , . . . , d (cid:48) k . By Lemma 2.3,then π M ( d , . . . , d k ) ˙ ∼ π M (cid:48) ( d (cid:48) , . . . , d (cid:48) k ). Accordingly, for every player i we have that f i ( π M ( d , . . . , d k )) = ˜ f i ( π M (cid:48) ( d (cid:48) , . . . , d (cid:48) k )). It thus follows that d (cid:48) k +1 = ( ˜ f ( π M (cid:48) ( d (cid:48) , . . . , d (cid:48) k )) , . . . , ˜ f n ( π M (cid:48) ( d (cid:48) , . . . , d (cid:48) k )))= ( f ( π M ( d , . . . , d k )) , . . . , f n ( π M ( d , . . . , d k )))= d k +1 .This concludes the proof.We are now in a position to state an equilibrium preservation theorem for bisimulation-invariant strategies in a similar way as we were able to obtain Theorem 4.1, the analogousresult for computation-based and trace-based strategies. Theorem 4.8.
Let G = ( M, Γ , . . . , Γ n ) and G (cid:48) = ( M (cid:48) , Γ , . . . , Γ n ) be games on bisimilarconcurrent game structures M and M (cid:48) , respectively. Then, f is a bisimulation-invariantequilibrium in G if and only if ˜ f is a bisimulation-invariant equilibrium in G (cid:48) .Proof. First assume for contraposition that ˜ f = ( ˜ f , . . . , ˜ f n ) is not a Nash equilibrium inbisimulation-invariant strategies in G (cid:48) . Then, there is a player i and a bisimulation-invariantstrategy g i for i in M (cid:48) such that κ M (cid:48) ( ˜ f ) / ∈ Γ i and κ M (cid:48) ( ˜ f − i , g i ) ∈ Γ i . As the mapping thattransforms a strategy f i into strategy ˜ f i is one-to-one, there is a bisimulation-invariantstrategy f (cid:48) i for i in M with ˜ f (cid:48) i = g i . Accordingly, κ M (cid:48) ( ˜ f − i , ˜ f (cid:48) i ) ∈ Γ i . Lemma 4.7 then yieldsthat κ M ( f − i , f (cid:48) i ) ∈ Γ i and κ M ( f ) / ∈ Γ i . As f (cid:48) i is bisimulation-invariant, it follows that f isnot an equilibrium in bisimulation-invariant strategies in G .The proof in the opposite direction runs along analogous lines.As an immediate corollary of Theorem 4.8, it follows that the property of a computationor trace to be sustained by a bisimulation-invariant equilibria is also preserved underbisimilarity. J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3
Corollary 4.9.
Let G = ( M, Γ , . . . , Γ n ) and G (cid:48) = ( M (cid:48) , Γ , . . . , Γ n ) be games on bisimilarconcurrent game structures M and M (cid:48) , respectively. Then, for every computation κ ∈ comps ωM = comps ωM (cid:48) and every trace τ ∈ traces ωM = traces ωM (cid:48) , (1) κ is sustained by a bisimulation invariant equilibrium in G if and only if κ is sustainedby a bisimulation-invariant equilibrium in G (cid:48) . (2) τ is sustained by a bisimulation-invariant equilibrium in G if and only if τ is sustainedby a bisimulation-invariant equilibrium in G (cid:48) .Proof. For part 1, assume that κ M ( f ) = κ . Then in virtue of Lemma 4.7 also κ M (cid:48) ( ˜ f ) = κ .Moreover, by Theorem 4.8 we have that profile f is a Nash equilibrium in bisimulation-invariant strategies in G if and only if ˜ f is a Nash equilibrium in bisimulation-invariantstrategies in G (cid:48) , which gives the result.The argument for part 2 runs along analogous lines. First assume that τ is sustainedby bisimulation-invariant Nash equilibrium f . Let κ = κ M ( f ). Then, τ = τ M ( κ ). ByTheorem 4.8, moreover, ˜ f is a bisimulation-invariant Nash equilibrium in G (cid:48) . An applicationof Lemma 4.7 yields κ M ( ˜ f ) = κ M ( f ). By Lemma 2.3 then τ M ( κ ) = τ M (cid:48) ( κ ). It follows that τ is sustained by ˜ f , a bisimulation-invariant Nash equilibrium, in G (cid:48) . The argument in theopposite direction is analogous, giving the result.5. Special Cases
In the previous section we provided results about the preservation of a given Nash equilibriumunder bisimilarity, specifically, as long as we do not consider run-based strategies or goals.In this section we study two important special scenarios where this is not the case.Firstly, consider the scenario where we have two-player games with run-based strategiesand trace-based goals. This is an important special case since run-based strategies, as weemphasised in the introduction, are the “conventional” model of strategies used in logicssuch as ATL ∗ or SL, as well as in systems represented as concurrent game structures. Inparticular, we show that with respect to two-player games with run-based strategies andtrace-based goals (which include temporal logic goals), the setting coincides with the onewith bisimulation-invariant strategies and trace-based goals, for which the preservation ofNash equilibria under bisimilarity holds. A key observation in this case is that in two-playergames the existence of Nash equilibria can be characterised in terms of the existence ofcertain winning strategies, which are preserved across bisimilar systems.Secondly, we also study the scenario where concurrent game structures are restrictedto those that are induced by iterated Boolean games [GHW15b] and Reactive Modulesgames [WGH + In this case, we show that bisimulation-invariant strategies also coincide with run-based strategies, and therefore, that the positiveresults for bisimulation-invariant strategies presented in the previous section also transfer tothis special case. For instance, Reactive Modules games provide a game semantics to formal specification languages suchas Reactive Modules [AH99], which is widely used in model checking tools, such as MOCHA [AHM +
98] andPRISM [KNP11]. ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:23 s ¯ p ¯ q s (cid:48) ¯ p ¯ q s ¯ p ¯ q s ¯ pq s p ¯ q s ¯ p ¯ q c , ab , a a, a a, bb , aa, b (cid:48) b , a (cid:48) a, bb , aa, b (cid:48) b , a (cid:48) a, aa, a (cid:48) b , bb , b (cid:48) a, aa, a (cid:48) b , bb , b (cid:48) ∗ , ∗ , ∗∗ , ∗ , ∗∗ , ∗ , ∗ Figure 4: The concurrent game structure M underlying the game G . s ¯ p ¯ q s ¯ p ¯ q s p ¯ q s ¯ pq s ¯ p ¯ q b , ac , a a, bb , aa, b (cid:48) b , a (cid:48) a, a a, aa, a (cid:48) b , bb , b (cid:48) ∗ , ∗ , ∗∗ , ∗ , ∗∗ , ∗ , ∗ Figure 5: The concurrent game structure M underlying the game G . Two-Player Games.
This section concerns the preservation under bisimilarity of Nashequilibria under bisimulation in two-player games. We deal with the cases in which theplayers’ preferences are computation-based, trace-based, and run-based separately.
Computation-based Preferences.
The counter-example against the preservation of the exis-tence of Nash equilibria in Section 4 involved three players. We find that, if preferences arecomputation-based, the example can be adapted so as to involve only two players, whichgives rise to the following result.
J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3
Theorem 5.1.
There are two-player games ( M, Γ , Γ ) and ( M (cid:48) , Γ , Γ ) on bisimilar con-current game structures M and M (cid:48) with Γ and Γ computation-based such that a run-basedNash equilibrium exists in G but not in G (cid:48) .Proof. Consider the concurrent game structures M and M depicted in Figures 4 and 5,respectively. Observe that Ac ( s ) = { a, b, c } and Ac ( s ) = { a, b } at all states s distinctfrom s , and that Ac ( s ) = { a } and Ac ( s ) = { a, a (cid:48) , b, b (cid:48) } at all states s distinct from s .We define the games G = ( M , Γ , Γ ) and G = ( M , Γ , Γ ) by letting Γ contain exactlythose computations κ = d , d , d , . . . such that either d = ( b, a ) and d ∈ { ( a, b ) , ( b, a ) } or d = ( c, a ) and d ∈ { ( a, b (cid:48) ) , ( b, a (cid:48) ) } , and letting Γ consist precisely of those computations κ = d , d , d , . . . such that d = ( a, a ) or d ∈ { ( a, a ) , ( a, a (cid:48) ) , ( b, b ) , ( b, b (cid:48) ) } . By an argumentanalogous to that presented in Section 4, it can then be appreciated that G has a run-basedequilibrium, whereas G has not.To see the former, observe that any run-based profile f ∗ = ( f ∗ , f ∗ ) will be an equilibriumif f ∗ ( s ) = f ∗ ( s ) = a , f ∗ ( s ) ∈ { a (cid:48) , b (cid:48) } , and f ∗ ( s (cid:48) ) ∈ { a, b } . For every strategy g forplayer 1, we then have κ M ( g , f ∗ ) / ∈ Γ , whereas κ M ( f ∗ , f ∗ ) ∈ Γ .To see that G has no run-based equilibrium, first let f = ( f , f ) be a run-based profilesuch that κ M ( f , f ) / ∈ Γ . Then, if f ( s , s ) = a , player 1 would like to deviate and play astrategy g with g ( s ) = g ( s , s ) = b ; if f ( s , s ) = a (cid:48) , to deviate and play a strategy g (cid:48) with g (cid:48) ( s ) = c and g (cid:48) ( s , s ) = b ; if f ( s , s ) = b to deviate and play a strategy g (cid:48)(cid:48) with g (cid:48)(cid:48) ( s ) = b and g (cid:48)(cid:48) ( s , s ) = a ; and, finally, if f ( s , s ) = b (cid:48) to deviate and play a strategy g (cid:48)(cid:48)(cid:48) with g (cid:48)(cid:48)(cid:48) ( s ) = c and g (cid:48)(cid:48)(cid:48) ( s , s ) = a . On the other hand, if κ M ( f , f ) / ∈ Γ , it must be thecase that f ( s ) ∈ { b, c } . Observe, however, that player 2 would then like to deviate and playany strategy g with g ( s , s ) = a if f ( s , s ) = a and to a strategy g (cid:48) with g (cid:48) ( s , s ) = b if f ( s , s ) = b . As, furthermore, Γ and Γ are disjoint, that is, the goals players 1 and 2cannot simultaneously be satisfied, it follows that G has no run-based equilibria. Run-based Preferences.
We now address the preservation (of the existence) of Nash equilibriain two-player CGS-games where both preferences and strategies are run-based. In contrastto our findings in the previous section, we find that, under a natural closure restriction onthe players’ preferences, we are able to obtain a positive result. Our proof relies on theequivalence of run-based profiles and run-invariant profiles as expounded in Section 3.As already noted above, run-based strategies cannot generally be identified directly acrossbisimilar CGS-games. The reason for this complication is that runs are sequences of states,and the sets of states of the two CGS-games need not coincide. In Section 3, we saw, however,how run-based strategies correspond to run-invariant strategies, which are computation-basedby definition. Lemma 2.2, moreover, allows us to compare computation-based strategies,even if they may be run-invariant in the one model but not in the other.Let f = ( f , f ) be a given run-invariant equilibrium in a CGS-game G = ( M, Γ , Γ )based on M and let G (cid:48) = ( M (cid:48) , Γ , Γ ) be a CGS-game based on a concurrent game struc-ture M (cid:48) bisimilar to M . We define a (computation-based) profile f K = ( f K , f K ) that is arun-invariant equilibrium in both G and G (cid:48) . To prove that f K = ( f K , f K ) is a run-invariantequilibrium if f = ( f , f ) is, we exploit a characterisation of Nash equilibria in terms ofwinning strategies. We say that a run-invariant strategy f i for player i is winning against Winning strategies have also been used to characterise the existence of Nash equilibria in other two-playergames with binary outcomes—see, e.g. , [GHW15a, GW14]. ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:25 player j whenever κ M ( f i , f j ) / ∈ Γ j for all run-invariant strategies f j of player j . We thenhave the following lemma, which is independent of the type of preferences that players have. Lemma 5.2.
Let G = ( M, Γ , Γ ) be a game. Then, a profile f = ( f , f ) is a run-invariantequilibrium if and only if both (1) κ M ( f , f ) / ∈ Γ implies that f is a winning strategy against player , and (2) κ M ( f , f ) / ∈ Γ implies that f is a winning strategy against player .Proof. For the “if” direction assume for contraposition that f = ( f , f ) is not a run-invariantequilibrium. Then, either κ M ( f , f ) / ∈ Γ and κ M ( g , f ) ∈ Γ for some run-invariantstrategy g for player 1, or κ M ( f , f ) / ∈ Γ and κ M ( f , g ) ∈ Γ for some run-invariantstrategy g for player 2. If the former, f is not winning against player 1, refuting 1. If thelatter, f is not winning against player 2, which refutes 2.The opposite direction is also by contraposition. Assume that either 1 or 2 is notsatisfied. Without loss of generality we may assume the former. Then, κ M ( f , f ) / ∈ Γ and f is not a winning strategy against player 1. Accordingly, there is some run-invariantstrategy g for player 1 such that κ M ( g , f ) ∈ Γ and it follows that f = ( f , f ) is not arun-invariant equilibrium.In order to have a formally convenient characterisation of the goal sets Γ and Γ tobe run-based in two bisimilar CGS-games and to define the profile f K = ( f K , f K ), wefurthermore introduce the following notations and auxiliary concepts. For a concurrent gamestructure M and finite computations κ, κ (cid:48) ∈ comps M , we write κ ≡ M κ (cid:48) if π M ( κ ) = π M ( κ ).Furthermore, we say that finite computations κ and κ (cid:48) are finitely congruent in M and M (cid:48) , in symbols κ ≡ K M,M (cid:48) κ (cid:48) , whenever there are (not necessarily distinct) intermediatecomputations κ , . . . , κ m such that(1) κ = κ ,(2) κ (cid:48) = κ m , and(3) κ j ≡ M κ j +1 or κ j ≡ M (cid:48) κ j +1 , for every 0 ≤ j < m .As ≡ M and ≡ M (cid:48) are equivalence relations, we may assume that here ≡ M and ≡ M (cid:48) alternateand κ j ≡ M κ j +1 if j is even, and κ j ≡ M (cid:48) κ j +1 if j is odd. We will generally omit thesubscript in K M,M (cid:48) when M and M (cid:48) are clear from the context. For an example, see againFigure 3. Consider the (one-step) computations κ = ( a, a ), κ = ( a, b ), κ = ( b, a ), and κ = ( b, b ). Then, κ ≡ M ,M κ , because κ ≡ M κ and κ ≡ M κ . On the other hand,some reflection reveals that κ (cid:54)≡ K M ,M κ . It is worth noting that finite congruence of twocomputations implies statewise bisimilarity of the runs induced, that is, κ ≡ K M,M (cid:48) κ (cid:48) implies π M ( κ ) ˙ ∼ π M ( κ (cid:48) ). Lemma 5.3.
Let M and M (cid:48) be two bisimilar concurrent game structures and κ = d , . . . , d k and κ (cid:48) = d (cid:48) , . . . , d (cid:48) k . Then, κ ≡ K M,M (cid:48) κ (cid:48) implies π M ( κ ) ˙ ∼ π M ( κ (cid:48) ) .Proof. Assume κ ≡ K M,M (cid:48) κ (cid:48) . Then there are κ , . . . , κ m such that κ = κ , κ m = κ (cid:48) , and,for all 0 ≤ (cid:96) < m , π M ( κ (cid:96) ) = π M ( κ (cid:96) +1 ) if (cid:96) is even and π M (cid:48) ( κ (cid:96) ) = π M (cid:48) ( κ (cid:96) +1 ) if (cid:96) is odd.We assume that m is even; the case where m is odd follows by the same argument mutatismutandis . By virtue of Lemma 2.3-1, we have π M ( κ (cid:96) ) ˙ ∼ π M (cid:48) ( κ (cid:96) ) for every 0 ≤ (cid:96) < m .Hence, π M ( κ ) = π M ( κ ) ˙ ∼ π M (cid:48) ( κ ) = · · · = π M (cid:48) ( κ m − ) ˙ ∼ π M ( κ m − ) = π M ( κ m ). J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3
As obviously π M ( κ (cid:96) ) = π M ( κ (cid:96) +1 ) and π M (cid:48) ( κ (cid:96) ) = π M (cid:48) ( κ (cid:96) +1 ) imply, respectively, π M ( κ (cid:96) ) ˙ ∼ π M ( κ (cid:96) +1 ) and π M (cid:48) ( κ (cid:96) ) ˙ ∼ π M (cid:48) ( κ (cid:96) +1 ), also π M ( κ ) ˙ ∼ π M ( κ ) ˙ ∼ π M (cid:48) ( κ ) ˙ ∼ · · · ˙ ∼ π M (cid:48) ( κ m − ) ˙ ∼ π M ( κ m − ) ˙ ∼ π M ( κ m ).By transitivity of ˙ ∼ we may conclude that π M ( κ ) ˙ ∼ π M ( κ ).For bisimilar concurrent game structures M and M (cid:48) , we say that a computation-basedstrategy f is K M,M (cid:48) -invariant if κ ≡ K M,M (cid:48) κ (cid:48) implies f ( κ ) = f ( κ (cid:48) ), for all finite computations κ, κ (cid:48) ∈ comps M . We find that K M,M (cid:48) -invariance exactly captures the concept of a strategythat is run-invariant in two bisimilar concurrent game structures.
Lemma 5.4.
Let M and M (cid:48) be bisimilar concurrent game structures and f i a computation-based strategy for player i . Then, f i is K M,M (cid:48) -invariant if and only if f i is run-invariant inboth M and M (cid:48) .Proof. For the “only if”-direction, assume that f i is K M,M (cid:48) -invariant and consider arbitrary κ, κ (cid:48) ∈ comps M such that π M ( κ ) = π M ( κ (cid:48) ), that is, κ ≡ M κ (cid:48) . By K M,M (cid:48) -invariance of f i then immediately f i ( κ ) = f i ( κ (cid:48) ). Accordingly, f i is run-invariant in M . The argument for f i being run-invariant in M (cid:48) is analogous.For the “if”-direction, assume that f i is run-invariant in both M and M (cid:48) , and considerarbitrary κ, κ (cid:48) ∈ comps such that κ ≡ K κ (cid:48) . Then, we may assume that there are κ , . . . , κ m such that κ = κ ≡ M κ ≡ M (cid:48) κ ≡ M · · · ≡ M (cid:48) κ m = κ (cid:48) .Having assumed that f i is run-invariant in both M and M (cid:48) , then also f i ( κ ) = f i ( κ ) = f i ( κ ) = f i ( κ ) = · · · = f i ( κ m ) = f i ( κ (cid:48) ),from which follows that f i is K M,M (cid:48) -invariant.As we have argued in Section 1, for the question whether the Nash equilibria across twobisimilar CGS-games are preserved to make sense, the players’ preferences in the two gameshave to be congruent, that is, they have to be the same and of the same type in both games.In this section we deal with run-based preferences. We have already seen in Section 4, thatidentity of a player’s computation-based preferences in two CGSs does not guarantee theirbeing congruent as run-based preferences. By imposing an additional closedness restriction,however, we find that a computation-based goal set can be guaranteed to be run-based intwo CGS-games based on bisimilar concurrent game structures M and M (cid:48) . Accordingly, calla goal set Γ i K M,M (cid:48) -closed if for all computations κ = d , d , d , . . . and κ (cid:48) = d (cid:48) , d (cid:48) , d (cid:48) , . . . ,we have that κ ∈ Γ i implies κ (cid:48) ∈ Γ i whenever d , . . . , d k ≡ K M,M (cid:48) d (cid:48) , . . . , d (cid:48) k for all k ≥ Lemma 5.5.
Let G = ( M, Γ , . . . , Γ n ) and G (cid:48) = ( M (cid:48) , Γ , . . . , Γ n ) be CGS-games on bisimilarconcurrent game structures M and M (cid:48) , and Γ i is K M,M (cid:48) -closed for some player i . Then, Γ i is run-based in both M and M (cid:48) .Proof. Assume that Γ i is K M,M (cid:48) -invariant and consider arbitrary infinite computations κ = d , d , d , . . . and κ (cid:48) = d (cid:48) , d (cid:48) , d (cid:48) , . . . such that ρ M ( κ ) = ρ M ( κ (cid:48) ). Also assume that κ ∈ Γ i . Let ρ M ( κ ) = s , s , s , . . . and ρ M ( κ (cid:48) ) = t , t , t , . . . . Then, for every k ≥
0, wealso have that π M ( d , . . . , d k ) = s , . . . , s k +1 = t , . . . , t k +1 = π M ( d (cid:48) , . . . , d (cid:48) k ).Having assumed Γ i to be K M,M (cid:48) -invariant, it follows that κ (cid:48) ∈ Γ i , as desired. ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:27 p ppp pp ¯ pa, aa, bb, ab, b a, aa, ba, ca, aa, ba, c ∗ , ∗∗ , ∗∗ , ∗∗ , ∗ p ppp pp ¯ pa, aa, bb, ab, b a, aa, ba, ca, aa, ba, c ∗ , ∗∗ , ∗∗ , ∗∗ , ∗ Figure 6: Two games G (left) and G (right) based on M and M , respectively.For the remainder, let G = ( M, Γ , Γ ) and G (cid:48) = ( M (cid:48) , Γ , Γ ) be two two-player CGS-games based on bisimilar concurrent game structures M and M (cid:48) such that both Γ and Γ are K M,M (cid:48) -closed (and thus, in particular, run-based). We prove that if there is a run-invariantequilibrium in M , then there is also a K -invariant profile that is a run-invariant equilibriumin M . We construct for a given strategy profile f = ( f , f ) that is run-invariant in M a K M,M (cid:48) -invariant profile f K = ( f K , f K ) such that( i ) κ M ( f , f ) = κ M ( f K , f K ),( ii ) if f is a winning strategy against player 2, then so is f K ,( iii ) if f is a winning strategy against player 1, then so is f K .On basis of Lemma 5.2 we may then conclude that f K corresponds to a run-invariantequilibrium in M . Having defined f K formally as a computation-based profile, by Theorem 4.1it follows that f K is also a computation-based equilibrium in G (cid:48) . Finally, because f K is K M,M (cid:48) -invariant, we know that it furthermore corresponds to a run-invariant equilibriumin both M and M (cid:48) .For an example consider the games G and G depicted in Figure 6. The underlyingconcurrent game structures M and M only differ with respect to direction ( a, b ) at theinitial state and their bisimilarity is easily established. Assume that the goal of player 1 isto see p false at some point in the future, that is,Γ = { d , d , d , · · · ∈ comps ω : d ∈ { ( a, a ) , ( a, b ) , ( b, a ) } and d = ( a, c ) } ,and that player 2 tries to prevent this, that is, Γ = comps ω \ Γ . Observe that defined thus,the players’ preferences are run-based in both G and G . Concentrating on G first, definethe run-invariant strategy profile f = ( f , f ), such that, f ( (cid:15) ) = b f ( a, a ) = a f ( a, b ) = a f ( b, a ) = af ( (cid:15) ) = b f ( a, a ) = a f ( a, b ) = b f ( b, a ) = b We find that f = ( f , f ) is a run-invariant equilibrium in G . Observe, however, that f =( f , f ) is not run-invariant in G , as f ( a, a ) (cid:54) = f ( a, b ) even though π M ( a, a ) = π M ( a, b ).Accordingly, f = ( f , f ) fails as a K M ,M -invariant equilibrium. Let g be defined suchthat, g ( (cid:15) ) = b g ( a, a ) = b g ( a, b ) = b g ( b, a ) = b J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3
Then, ( f , g ) is readily seen to be a K M ,M -invariant equilibrium in G . We will find thatunder the conditions specified, this is no coincidence and that K -invariant equilibria can beconstructed from run-invariant equilibria in a systematic fashion.We first define the strategy profile f K = ( f K , f K ). The underlying idea is to carefullychoose for each finite computation d , . . . , d k computations κ d ,...,d k f and κ d ,...,d k f from theequivalence class of d , . . . , d k under ≡ K M,M (cid:48) , and set f K ( d , . . . , d k ) = f ( κ d ,...,d k f ) and f K ( d , . . . , d k ) = f ( κ d ,...,d k f ), respectively. This guarantees that f K = ( f K , f K ) is K M,M (cid:48) -invariant. Here we give priority to prefixes of κ ( f , f ), that is if κ ( f , f ) = d (cid:48) , d (cid:48) , d (cid:48) , . . . and d (cid:48) , . . . , d (cid:48) k ≡ K M,M (cid:48) d , . . . , d k , then κ d ,...,d k f = κ d ,...,d k f = d (cid:48) . . . , d (cid:48) k . In a similar way, wealso choose κ d ,...,d k f and κ d ,...,d k f so as to preserve the two players’ punishment strategies.This guarantees that f K = ( f K , f K ) is an equilibrium, as it inherits this property from f =( f , f ). To do so we assume a well-ordering of the action sets Ac and Ac of players 1and 2, respectively. Then, for all finite computations κ = d , . . . , d k in comps M , we defineinductively and simultaneously computations κ d ,...,d k f and κ d ,...,d k f as follows. For κ = (cid:15) wehave κ (cid:15)f = κ (cid:15)f = (cid:15), and, for κ = d , . . . , d k +1 , κ d ,...,d k ,d k +1 f = d (cid:48) , . . . , d (cid:48) k , ( x , x ) κ d ,...,d k ,d k +1 f = d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k , ( y , y ) , where κ d ,...,d k f = d (cid:48) , . . . , d (cid:48) k , κ d ,...,d k f = d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k , and( i. x = f ( d (cid:48) , . . . , d (cid:48) k ) and x = f ( d (cid:48) , . . . , d (cid:48) k ), if d (cid:48) , . . . , d (cid:48) k , ( f ( d (cid:48) , . . . , d (cid:48) k ) , f ( d (cid:48) , . . . , d (cid:48) k )) ≡ K d , . . . , d k , d k +1 ,( i. y = f ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k ) and y = f ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k ), if d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k , ( f ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k ) , f ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k )) ≡ K d , . . . , d k , d k +1 ,( ii. x = f ( d (cid:48) , . . . , d (cid:48) k ) and x is the least action available to player 2 such that d (cid:48) , . . . , d (cid:48) k , ( x , x ) ≡ K d , . . . , d k , d k +1 , if such an action x exists and ( i.
1) does not apply,( ii. y = f ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k ) and y is the least action available to player 1 such that d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k , ( y , y ) ≡ K d , . . . , d k , d k +1 , if such an action y exists and ( i.
2) does not apply,( iii. x and x are the least actions available to players 1 and 2, respectively, such that d (cid:48) , . . . , d (cid:48) k , ( x , x ) ≡ K d , . . . , d k , d k +1 , if neither ( i.
1) nor ( ii.
1) apply,( iii. y and y are the least actions available to players 1 and 2, respectively, such that d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k , ( x , x ) ≡ K d , . . . , d k , d k +1 , if neither ( i.
2) nor ( ii.
2) apply,Observe that the actions x and x in the definition above always exist. The reason for thisis that, if d , . . . , d k ≡ K M,M (cid:48) d (cid:48) , . . . , d (cid:48) k , by Lemma 5.3 also π M ( d , . . . , d k ) ˙ ∼ π M ( d (cid:48) , . . . , d (cid:48) k ). The precise definition is rather involved, and the reader may want to skip to page 33, where the maintheorem of the section is stated and proven. ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:29
Let π ( d , . . . , d k ) = s , . . . , s k , π ( d (cid:48) , . . . , d (cid:48) k ) = s (cid:48) , . . . , s (cid:48) k , and d (cid:48) k +1 = ( a , a ). Then,obviously, a and a are available to players 1 and 2 respectively at state s (cid:48) k . As s k ∼ s (cid:48) k , that should also be the case at s k . A similar argument applies to the case where d , . . . , d k ≡ K M,M (cid:48) d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k .We now define strategies f K and f K such that, for all finite computations κ = d , . . . , d k in comps M , f K ( d , . . . , d k ) = f ( κ d ,...,d k f ) and f K ( d , . . . , d k ) = f ( κ d ,...,d k f ).To illustrate how f K = ( f K , f K ) is constructed from f = ( f , f ), recall the games M and G in Figure 6, and assume that actions for both players are ordered alphabetically. For theempty computation (cid:15) , we have f K ( (cid:15) ) = f ( κ (cid:15)f ) = f ( (cid:15) ) = b and f K ( (cid:15) ) = f ( κ (cid:15)f ) = f ( (cid:15) ) = b Now consider the finite computations of length one. For computation ( b, b ), we find that( b, b ) ≡ K M ,M ( f ( κ (cid:15)f ) , f ( κ (cid:15)f )). Now case ( i.
1) applies and we obtain κ ( b,b ) f = ( b, b ).Similarly, κ ( b,b ) f = ( b, b ). For the other one-step computations ( x , x ) ∈ { ( a, a ) , ( a, b ) , ( b, a ) } ,we find that ( x , x ) (cid:54)≡ K M ,M ( f ( (cid:15) ) , f ( (cid:15) )). First consider ( a, a ), and to determine κ ( a,a ) f , firstobserve that κ (cid:15)f = d (cid:48) , . . . , d (cid:48) k = (cid:15) , d , . . . , d k = (cid:15) , and d k +1 = ( a, a ). Now, for x = f ( (cid:15) ) = b we have that ( x , a ) = ( b, a ) ≡ M ( a, b ) ≡ M ( a, a ).Accordingly, ( x , a ) ≡ M ,M ( a, a ) = d k +1 , and case ( ii.
1) applies. With a being moreoverplayer 2’s alphabetically least action, we may therefore conclude that κ ( a,a ) f = ( b, a ). In asimilar way we obtain κ ( a,b ) f = κ ( b,a ) f = ( b, a ) as well as κ ( a,a ) f = κ ( a,b ) f = κ ( b,a ) f = ( a, b ). Hence, f K ( a, a ) = f ( κ ( a,a ) f ) = f ( b, a ) = a f K ( a, a ) = f ( κ ( a,a ) f ) = f ( a, b ) = bf K ( a, b ) = f ( κ ( a,b ) f ) = f ( b, a ) = a f K ( a, b ) = f ( κ ( a,b ) f ) = f ( a, b ) = bf K ( b, a ) = f ( κ ( b,a ) f ) = f ( b, a ) = a f K ( b, a ) = f ( κ ( b,a ) f ) = f ( a, b ) = bf K ( b, b ) = f ( κ ( b,b ) f ) = f ( b, b ) = b f K ( b, b ) = f ( κ ( b,b ) f ) = f ( b, b ) = b We thus find that f K = ( f K , f K ) coincides with the K M ,M -invariant equilibrium ( f , g )that we identified above.The definition of strategies f K and f K ensures that f K = ( f K , f K ) is K M,M (cid:48) -invariant.
Lemma 5.6.
Let κ = d , . . . , d k be s finite computation in comps M . Then, for i = 1 , , d , . . . , d k ≡ K κ d ,...,d k f i .Accordingly, f K = ( f K , f K ) is K M,M (cid:48) -invariant.Proof.
Strategies f K and f K have been defined so as to be K -invariant. The claim thenfollows by induction on the length of κ . Let i = 1; the case for i = 2 is analogous. Ifthe length of κ is 0, we have κ = κ (cid:15)f = (cid:15) , and it immediately follows that κ ≡ K κ (cid:15)f . Forthe induction step, let κ = d , . . . , d k , d k +1 and assume that d , . . . , d k ≡ K κ d ,...,d k f . Let κ d ,...,d k +1 f be denoted by d (cid:48) , . . . , d (cid:48) k , d (cid:48) k +1 where d (cid:48) k +1 = ( x , x ). There are three possibilities. J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3
First, assume that d (cid:48) , . . . , d (cid:48) k , ( f ( d (cid:48) , . . . , d (cid:48) k ) , f ( d (cid:48) , . . . , d (cid:48) k )) ≡ K d , . . . , d k , d k +1 . Then,clause ( i.
1) applies and we have x = f ( d (cid:48) , . . . , d (cid:48) k ) and x = f ( d (cid:48) , . . . , d (cid:48) k ). It then followsthat d (cid:48) , . . . , d (cid:48) k , d (cid:48) k +1 ≡ K d , . . . , d k , d k +1 . Second, assume that d (cid:48) , . . . , d (cid:48) k , ( f ( d (cid:48) , . . . , d (cid:48) k ) , y ) ≡ K d , . . . , d k , d k +1 for some action y ∈ Ac , but d (cid:48) , . . . , d (cid:48) k , ( f ( d (cid:48) , . . . , d (cid:48) k ) , f ( d (cid:48) , . . . , d (cid:48) k )) (cid:54)≡ K d , . . . , d k , d k +1 . Then clause ( ii.
1) applies and we have that x = f ( d (cid:48) , . . . , d (cid:48) k ) and x is the least actionavailable to player 2 such that d (cid:48) , . . . , d (cid:48) k , ( f ( d (cid:48) , . . . , d (cid:48) k ) , y ) ≡ K d , . . . , d k , d k +1 . Again, itimmediately follows that d (cid:48) , . . . , d (cid:48) k , d (cid:48) k +1 ≡ K d , . . . , d k , d k +1 .Finally, assume that neither of the above, then clause ( iii.
1) applies and we have that x and x are the least actions available to players 1 and 2 such that d (cid:48) , . . . , d (cid:48) k , ( x , x ) ≡ K d , . . . , d k +1 . As such actions exist, we conclude that d (cid:48) , . . . , d (cid:48) k , d (cid:48) k +1 ≡ K d , . . . , d k , d k +1 .We are now in a position to prove the following crucial lemmas. Lemma 5.7.
Let f = ( f , f ) be a run-invariant strategy profile for concurrent gamestructure M and f K defined as above. Then, κ M ( f , f ) = κ M ( f K , f K ) .Proof. Let us use the following notations: κ M ( f , f ) = d , d , . . . ; κ ( f K , f K ) = d K , d K , . . . ; κ d ,...,d k f = d (cid:48) , d (cid:48) , . . . ; κ d ,...,d k f = d (cid:48)(cid:48) , d (cid:48)(cid:48) , . . . ; where, for every k ≥ d k = ( a k , a k ) , d Kk = ( b k , b k ) , d (cid:48) k = ( x k , x k ) , d (cid:48)(cid:48) k = ( y k , y k ).It then suffices to prove by induction on k that for every k ≥
0, we have d k = d Kk = d (cid:48) k = d (cid:48)(cid:48) k .For k = 0, let d = ( a , a ) and d K = ( b , b ). Then, a = f ( (cid:15) ) = f ( κ (cid:15)f ) = f K ( (cid:15) ) = b .In a similar way we find that a = b , and hence d = d K . Because both f ( (cid:15) ) = a and f ( (cid:15) ) = a , we also have d (cid:48) = d , and, a fortiori , d (cid:48) ≡ K d . Hence, clause ( i.
1) applies andtherefore d (cid:48) = ( f ( (cid:15) ) , f ( (cid:15) )) = ( a , a ) = d . In a similar way we can establish that d (cid:48)(cid:48) = d .For the induction step, we may assume that d , . . . , d k = d K , . . . , d Kk = d (cid:48) , . . . , d (cid:48) k = d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k . ( i.h. )Then, a k +11 = f ( d , . . . , d k ) = i.h. f ( d (cid:48) , . . . , d (cid:48) k ) = f K ( d , . . . , d k ) = i.h. f K ( d (cid:48) , . . . , d (cid:48) k ) = b k +11 .Observe that the third equality holds because d (cid:48) , . . . , d (cid:48) k = κ d ,...,d k f . For player 2, thefollowing equalities hold: a k +12 = f ( d , . . . , d k ) = i.h. f ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k ) = f K ( d , . . . , d k ) = i.h. f K ( d (cid:48) , . . . , d (cid:48) k ) = b k +12 .Now the third equality holds since d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k = κ d ,...,d k f . The proofs of these two lemmas extend over a couple of pages and the reader may skip to page 33, wherethe running text continues. ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:31
Finally, by the induction hypothesis, d (cid:48) , . . . , d (cid:48) k = d , . . . , d k . From the above, we alreadyhad f ( d (cid:48) , . . . , d (cid:48) k ) = a k +11 and f ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k ) = a k +12 Hence,( f ( d (cid:48) , . . . , d (cid:48) k ) , f ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k )) = d k +1 . It follows that, d (cid:48) , . . . , d (cid:48) k , ( f ( d (cid:48) , . . . , d (cid:48) k ) , f ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k )) = d , . . . , d k , d k +1 . In particular, d (cid:48) , . . . , d (cid:48) k , ( f ( d (cid:48) , . . . , d (cid:48) k ) , f ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k )) ≡ K d , . . . , d k , d k +1 . Therefore, clause ( i.
1) is applicable, and both x k +11 = f ( d (cid:48) , . . . , d (cid:48) k ) = a k +11 and x k +12 = f ( d (cid:48)(cid:48) , . . . , d (cid:48)(cid:48) k ) = a k +12 ,signifying that d k +1 = d (cid:48) k +1 . By an analogous reasoning we show that d k +1 = d (cid:48)(cid:48) k +1 .The next lemma establishes that f K and f K are winning run-invariant strategies againstplayer 2 and player 1, respectively, if the goal sets Γ and Γ are to be run-based. Noticethat this result requires Γ and Γ to be K M,M (cid:48) -closed.
Lemma 5.8.
Let f = ( f , f ) be a run-invariant strategy profile in game G = ( M, Γ , Γ ) with Γ and Γ run-based and K -closed, and f K defined as above. Then, (1) if f is a winning (run-invariant) strategy against player , then so is f K , (2) if f is a winning (run-invariant) strategy against player , then so is f K .Proof. For part 1, assume for contraposition that f K is not winning against player 2. Then,there is a strategy g for player 2 such that κ ( f K , g ) ∈ Γ . We define a K -invariantstrategy g K for player 2 such that d , . . . , d k ≡ K e , . . . , e k for every k ≥
0, where κ ( f K , g ) = d , d , d , . . . and d k = ( a , a ), κ ( f , g K ) = e , e , e , . . . . and e k = ( b , b ).By Γ being K -closed it then follows that also κ ( f , g K ) ∈ Γ , which contradicts our initialassumption that f is winning strategy against player 2. For each k ≥
0, let furthermore κ d ,...,d k = d (cid:48) , . . . , d (cid:48) k and d (cid:48) k = ( c , c ) . In order to define the strategy g K , we may assume the existence of some K -invariantstrategy h for player 2. For the empty computation (cid:15) we have g K ( (cid:15) ) = x where( i. x = f ( (cid:15) ), if ( f ( (cid:15) ) , f ( (cid:15) )) ≡ K d ,( ii. x is the least action y available to player 2 such that ( f ( (cid:15) ) , y k +12 ) ≡ K d , if suchan action y k +12 exists and case ( i ) does not apply,( iii. x k +12 = h ( (cid:15) ) in all other cases.For every finite computation d , . . . , d k , we have that g K ( d , . . . , d k ) = x k +12 , where( i. x k +12 = f ( d , . . . , d k ), if d , . . . , d k , ( f ( d , . . . , d k ) , f ( d , . . . , d k )) ≡ K d , . . . , d k , d k +1 ,( ii. x k +12 is the least action y k +12 available to player 2 such that d , . . . , d k , ( f ( d , . . . , d k ) , y k +12 ) ≡ K d , . . . , d k , d k +1 ,if such an action y k +12 exists and case ( i ) does not apply,( iii. x k +12 = h ( d , . . . , d k ) in all other cases. J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3
Observe that g K is K -invariant by construction.We now prove by induction on k that e , . . . , e k = d (cid:48) , . . . , d (cid:48) k for every k ≥
0. If k = 0,recall that e = ( b , b ) and d = ( c , c ). Observe that f K ( (cid:15) ) = f ( κ (cid:15) ) = f ( (cid:15) ). Thus wehave d = ( f ( (cid:15) ) , g ( (cid:15) )) = ( f ( (cid:15) ) , g ( (cid:15) )), and in particular d ≡ K ( f ( (cid:15) ) , g ( (cid:15) )). Hence, thereis some y such that ( f ( (cid:15) ) , y ) ≡ K d .First consider the case where ( f ( (cid:15) ) , f ( (cid:15) )) ≡ K d . Now clause ( i.
1) applies and c = f ( (cid:15) ) and c = f ( (cid:15) ). Accordingly, clause ( i.
3) is applicable as well, and we obtain both b = f ( (cid:15) ) = c and b = g k ( (cid:15) ) = f ( (cid:15) ) = c .Otherwise, there is some least x such that ( f ( (cid:15) ) , x ) ≡ K d . Thus, due to clause ( ii. c = f ( (cid:15) ) and c = x . Clause ( ii.
3) now also applies and we obtain: b = f ( (cid:15) ) = c and b = g K ( (cid:15) ) = x = c .The induction step runs along similar lines. We may assume that e , . . . , e k = d , . . . , d k . ( i.h. )Observe that f K ( d , . . . , d k ) = f ( κ d ,...,d k ) = f ( d (cid:48) , . . . , d (cid:48) k ).Thus, d k +1 = ( f K ( d , . . . , d k ) , g ( d , . . . , d k )) = ( f ( d (cid:48) , . . . , d (cid:48) k ) , g ( d , . . . , d k )).By Lemma 5.6, moreover, d , . . . , d k ≡ K d (cid:48) , . . . , d (cid:48) k , and it follows that d , . . . , d k , d k +1 ≡ K d (cid:48) , . . . , d (cid:48) k , ( f ( d (cid:48) , . . . , d (cid:48) k ) , g ( d , . . . , d k )) . Hence, there is some y k +12 such that d , . . . , d k , d k +1 ≡ K d (cid:48) , . . . , d (cid:48) k , ( f ( d (cid:48) , . . . , d (cid:48) k ) , y k +12 ) . ( ∗ )First assume that equation ( ∗ ) holds for y k +12 = f ( d (cid:48) , . . . , d (cid:48) k ). Then clause ( i.
1) appliesand for d (cid:48) k +1 = ( c k +11 , c k +12 ) we have c k +11 = f ( d (cid:48) , . . . , d (cid:48) k ) and c k +12 = f ( d (cid:48) , . . . , d (cid:48) k ) . Recall that e k +1 = ( b k +11 , b k +12 ). Now for player 1 we find that b k +11 = f ( e , . . . , e k ) = i.h. f ( d (cid:48) , . . . , d (cid:48) k ) = c k +11 .For player 2, observe that, in the case we are considering, e , . . . , e k , ( f ( d (cid:48) , . . . , d (cid:48) k ) , f ( d (cid:48) , . . . , d (cid:48) k )) = i.h. d (cid:48) , . . . , d (cid:48) k , ( f ( d (cid:48) , . . . , d (cid:48) k ) , f ( d (cid:48) , . . . , d (cid:48) k ))= d (cid:48) , . . . , d (cid:48) k , d (cid:48) k +1 .Accordingly, clause ( i.
4) applies, that is, g K ( e , . . . , e k ) = f ( d (cid:48) , . . . , d (cid:48) k ). Hence, b k +12 = g K ( e , . . . , e k ) = f ( d (cid:48) , . . . , d (cid:48) k ) = c k +12 ,and we may conclude that e k +1 = ( b k +11 , b k +12 ) = ( c k +11 , c k +12 ) = d (cid:48) k +1 .Finally, assume that equation ( ∗ ) does not hold for y k +12 = f ( d (cid:48) , . . . , d (cid:48) k ). Then, let x k +12 be the least action for player 2 for which equation ( ∗ ) does hold with y k +1 = x k +12 . As inthis case clause ( i.
2) applies and for d (cid:48) k +1 = ( c k +11 , c k +12 ), we have, c k +11 = f ( d (cid:48) , . . . , d (cid:48) k ) and c k +12 = x k +12 . ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:33 Recall that e k +1 = ( b k +11 , b k +12 ). For player 1 we again find that, b k +11 = f ( e , . . . , e k ) = i.h. f ( d (cid:48) , . . . , d (cid:48) k ) = c k +11 .For player 2, observe that, in the case we are considering, e , . . . , e k , ( f ( d (cid:48) , . . . , d (cid:48) k ) , x k +12 ) = i.h. d (cid:48) , . . . , d (cid:48) k , ( f ( d (cid:48) , . . . , d (cid:48) k ) , x k +12 )= d (cid:48) , . . . , d (cid:48) k , d (cid:48) k +1 .Accordingly, clause ( ii.
4) applies and we have g K ( e , . . . , e k ) = x k +12 . It then follows that, b k +12 = g K ( e , . . . , e k ) = f ( d (cid:48) , . . . , d (cid:48) k ) = c k +12 .Again we may conclude that e k +1 = ( b k +11 , b k +12 ) = ( c k +11 , c k +12 ) = d (cid:48) k +1 , as desired.The proof for part 2 is analogous to that of part 1.The ground has now been cleared for the main result of this paper that the existence ofrun-invariant equilibria is preserved under bisimulation in two-player games provided thatthe run-based preferences of the players are K M,M (cid:48) -closed.
Theorem 5.9.
Let G = ( M, Γ , Γ ) and G (cid:48) = ( M (cid:48) , Γ , Γ ) be two two-player games onbisimilar concurrent game structures such that Γ and Γ are run-based and K M,M (cid:48) -closed.Then, if f = ( f , f ) is a run-invariant strategy profile in M , then f K = ( f K , f K ) is arun-invariant equilibrium in M (cid:48) .Proof. Assume that f = ( f , f ) is a run-invariant strategy profile in M . By Lemma 5.2,then both(1) κ M ( f , f ) / ∈ Γ implies that f is a winning strategy against player 1 in M , and(2) κ M ( f , f ) / ∈ Γ implies that f is a winning strategy against player 2 in M .On basis of 1, Lemma 5.7 yields κ M ( f , f ) = κ M ( f K , f K ). Now assume κ M ( f K , f K ) / ∈ Γ .Then, also κ M ( f , f ) / ∈ Γ and we may assume that f is a winning strategy against player 1in M . In virtue of Lemma 5.8 we may then conclude that f K is a winning strategy againstplayer 1 in M . Assuming that κ M ( f K , f K ) / ∈ Γ , we can reason analogously and infer that f is a winning strategy against player 2 in M . Hence,(1 (cid:48) ) κ M ( f K , f K ) / ∈ Γ implies that f K is a winning strategy against player 1 in M , and(2 (cid:48) ) κ M ( f K , f K ) / ∈ Γ implies that f K is a winning strategy against player 2 in M .Accordingly, f K is a computation-based equilibrium in M . By Theorem 4.1 we may inferthat f K is also a computation-based equilibrium in M (cid:48) . Lemma 5.6 then guarantees that f K is K M,M (cid:48) -invariant, and it follows that f K is run-invariant in M (cid:48) as well. By virtue ofTheorem 3.5 we may finally conclude that f K is also a run-invariant equilibrium in M (cid:48) .As an immediate consequence of Theorem 5.9, we have the following result, which is phrasedin terms of run-based strategies instead of run-invariant strategies. Corollary 5.10.
Let G = ( M, Γ , Γ ) and G (cid:48) = ( M (cid:48) , Γ , Γ ) be two two-player games onbisimilar concurrent game structures M and M (cid:48) such that Γ and Γ are run-based and K M,M (cid:48) -closed. Let furthermore ρ ∈ runs ωM be a run in M that is sustained by a run-basedequilibrium in M . Then, there is a run ρ (cid:48) ∈ runs ωM (cid:48) in M (cid:48) that is statewise bisimilar to ρ and that is also sustained by a run-based equilibrium in M (cid:48) . J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3
Proof.
Let run ρ ∈ runs ωM be sustained by a run-based equilibrium f = ( f , f ) in G and letˇ f = ( ˇ f , ˇ f ) be the run-invariant strategy profile corresponding to f . Lemma 3.4 guaranteesthat ˇ f = ( ˇ f , ˇ f ) is a run-invariant equilibrium in G . Now construct profile ˇ f K = ( ˇ f K , ˇ f K ),which by virtue of Theorem 5.9 is then K M,M (cid:48) -invariant and is a run-invariant equilibrium inboth G and G (cid:48) . By virtue Lemma 2.2-1, it moreover follows that ρ M ( g K , g K ) ˙ ∼ ρ M (cid:48) ( g K , g K ),that is, ρ M ( g K , g K ) and ρ M (cid:48) ( g K , g K ) are statewise bisimilar, which concludes the proof.A further corollary of Theorem 5.9 is that the existence of run-based equilibria is preservedin two-player games with run-based and K M,M (cid:48) -closed preferences.
Trace-based Preferences.
We find that, with a couple of slight modifications, essentiallythe same construction as in the previous section can be leveraged to prove that run-based equilibria are also preserved under bisimulation in two-player games with trace-based preferences. It be emphasised that here we do not require the preferences to satisfy anyother condition than being trace-based.Let two CGS-games G = ( M, Γ , Γ ) and G (cid:48) = ( M (cid:48) , Γ , Γ ) on bisimilar concurrentgame structures M and M (cid:48) and with Γ and Γ trace-based be given. For a run-invariantequilibrium f = ( f , f ) in game G , we define the K M,M (cid:48) -invariant strategy-profile f K =( f K , f K ) as in the previous section. We prove that f K = ( f K , f K ) is also a run-invariantequilibrium in G (cid:48) . To this end, we adapt Lemma 5.8 so as to apply to trace-based preferencesinstead of preferences that are both run-based and K M,M (cid:48) -closed.
Lemma 5.11.
Let f = ( f , f ) be a run-invariant strategy profile in game G = ( M, Γ , Γ ) with Γ and Γ trace-based, and f K defined as above. Then, (1) if f is a winning (run-invariant) strategy against player , then so is f K , (2) if f is a winning (run-invariant) strategy against player , then so is f K .Proof. For part 1—part 2 follows by an analogous argument—assume for contrapositionthat f K is not a winning strategy against player 2. Then, there is a strategy g for player 2such that κ ( f K , g ) ∈ Γ . We define a K M,M (cid:48) -invariant strategy g K for player 2 exactly asin the proof of Lemma 5.8. Accordingly, d , . . . , d k ≡ K M,M (cid:48) e , . . . , e k for every k ≥
0, where κ ( f K , g ) = d , d , d , . . . and d k = ( a , a ), κ ( f , g K ) = e , e , e , . . . . and e k = ( b , b ).Now consider an arbitrary k ≥
0. Then, by Lemma 5.3, also π M ( d , . . . , d k ) ˙ ∼ π M ( e , . . . , e k ).Letting π M ( d , . . . , d k ) = s , . . . , s k and π M ( d (cid:48) , . . . , d (cid:48) k ) = s (cid:48) , . . . , s (cid:48) k , we then also have s k ∼ s (cid:48) k . It follows that ρ M ( f K , g ) ˙ ∼ ρ M ( f , g K ) and hence τ M ( f K , g ) = τ M ( f , g K ). As aconsequence of Γ being trace-based, we obtain κ ( f , g K ) ∈ Γ , which contradicts our initialassumption that f is winning strategy against player 2.We are now in a position to prove the counterpart of Theorem 5.9 for trace-based prefer-ences, showing that run-invariant equilibria are preserved under bisimulation if the players’preferences are trace-based. Theorem 5.12.
Let G = ( M, Γ , Γ ) and G (cid:48) = ( M (cid:48) , Γ , Γ ) be two two-player games onbisimilar concurrent game structures such that Γ and Γ are trace-based. Then, if f = ( f , f ) is run-invariant in M , then f K = ( f K , f K ) is a run-invariant equilibrium in M (cid:48) . ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:35 Proof.
The proof is fully analogous to that for Theorem 5.9, invoking Lemma 5.11 insteadof Lemma 5.8.As an immediate consequence of Theorem 5.12, we find that also the existence of run-invariantequilibria is preserved in two-player games with trace-based preferences. Furthermore, alsothe counterpart of Corollary 5.10 for trace-based preferences can easily be demonstrated.
Boolean Game Structures.
We now consider a subclass of concurrent game structures inwhich Nash equilibrium is invariant under bisimilarity. Specifically, we study games playedover the class of concurrent game structures induced by iterated Boolean games [GHW15b],a framework that can be used to reason about Nash equilibria in games and multi-agentsystems modelled using the Reactive Modules specification language [AH99].By a
Boolean game structure we understand a special type of concurrent game structure M = (Ag , AP , Ac , St , s M , λ, δ ) for which there is a partition { AP , . . . , AP n } of AP suchthat Ac i ( s ) ⊆ (2 AP i \ ∅ ) for all players i and states s and for every direction d (cid:48) = ( a , . . . , a n )in 2 AP × · · · × AP n and every state s , it holds that δ ( s, d (cid:48) ) = s (cid:48) implies λ ( s (cid:48) ) = a ∪ · · · ∪ a n .Then, informally, in a Boolean game structure, choice profiles correspond to systemstates, which is not generally the case in concurrent game structures. In other words, ina Boolean game structure M , if a strategy profile induces a run s M , s , s , . . . , then weknow that it has been induced by the computation s , s , . . . . Even more, we also knowthat the trace induced by such a computation is precisely s M , s , s , . . . . This very strongcorrespondence between computations, runs, and traces is key to the proof that in Booleangame structures all strategies for a player are in fact bisimulation-invariant. This result,in turn, can also be used to show that Nash equilibrium is invariant under bisimilarity,regardless of the model of strategies or goals that one chooses. To see this, the followingpreliminary results will be useful. Lemma 5.13.
Let M = (Ag , AP , Ac , St , s , λ, δ ) be a Boolean game structure with partition { AP , . . . , AP n } . Let π = s , . . . , s k and π (cid:48) = s (cid:48) , . . . , s (cid:48) k be statewise bisimilar finite histories,that is, π ˙ ∼ π (cid:48) . Then, π = π (cid:48) .Proof. We may assume that there are finite computations κ = d , . . . , d k − and κ (cid:48) = d (cid:48) , . . . , d (cid:48) k − such that s d −→ · · · d k − −−−→ s k and s (cid:48) d (cid:48) −→ · · · d (cid:48) k − −−−→ s (cid:48) k . We show by inductionthat s m = s (cid:48) m for all 0 ≤ m ≤ k . For the basis, we have s = s M = s (cid:48) . For the inductionstep we may assume that s m = s (cid:48) m . Moreover, as s m +1 ∼ s (cid:48) m +1 , also λ ( s m +1 ) = λ ( s (cid:48) m +1 ).Furthermore, s m d m −−→ s m +1 and s (cid:48) m d (cid:48) m −−→ s (cid:48) m +1 . As M is a Boolean game structure, it followsthat d m = ( λ ( s m +1 ) ∩ AP , . . . , λ ( s m +1 ) ∩ AP n ) and d (cid:48) = ( λ ( s (cid:48) m +1 ) ∩ AP , . . . , λ ( s (cid:48) m +1 ) ∩ AP n )and, hence, d m = d (cid:48) m . By determinism of δ , we may conclude that s m +1 = δ ( s m , d m ) = δ ( s (cid:48) m , d (cid:48) m ) = s (cid:48) m +1 .The above lemma can be used to show that in fact, for Boolean game structures, allmodels of strategies collapse to the model of bisimulation-invariant strategies. Lemma 5.14.
In Boolean game structures, all strategies for every player are bisimulation-invariant.
J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3
Preferencescomputation-based run-based trace-basedcomputation-based + (Th. 4.1) + (Th. 4.1) + (Th. 4.1)run-based (general) − (Th. 4.5) − (Th. 4.5) − (Th. 4.5)Strategies run-based (two players) − (Th. 5.1) + † (Th. 5.9) + (Th. 5.12)trace-based + (Th. 4.3) + (Th. 4.3) + (Th. 4.3)bisimulation-invariant + (Th. 4.8) + (Th. 4.8) + (Th. 4.8) † Assuming the players’ (run-based) preferences to be K M,M (cid:48) -closed.
Table 1: Summary of main bisimulation-invariance results for multi-player games in deter-ministic systems as well as the results in this paper they are based on. In thisfigure, + means that Nash equilibria are preserved in (computation/run/trace)-based strategy profiles with preferences given by sets of computations/runs/traces,while − indicates that they are not for such a pair. Proof.
Consider an arbitrary strategy f i of a player i in a Boolean game structure M alongwith arbitrary statewise bisimilar histories π, π (cid:48) ∈ runs M , that is, π ˙ ∼ π (cid:48) . By Lemma 5.13,then π = π (cid:48) . Hence, trivially, f i ( π ) = f i ( π (cid:48) ).We can now present the main result of this section. Theorem 5.15.
In Boolean game structures, (the existence of a) Nash equilibrium isinvariant under bisimilarity.Proof.
Observe that because of Lemma 5.14, in Boolean game structures, a strategy pro-file f = ( f , . . . , f n ) is a Nash equilibrium if and only if f is a Nash equilibrium in bisimulation-invariant strategies. The result then immediately follows from Corollary 4.9.6. Nondeterminism
Our results so far, summarised in Table 1, apply to profiles of deterministic strategies anddeterministic systems. In this section, we investigate the case of nondeterministic systems. Inthis more general setting, most of our notations and definitions remain the same, except fortwo that are particularly relevant: the notions of outcome of a game and Nash equilibrium .Note that in a deterministic system, a profile of deterministic strategies induces a unique system path (and therefore a unique computation, run, and trace). However, if the systemis nondeterministic, a profile of deterministic strategies might, instead, determine a set ofpaths of the system: all those complying with the profile of strategies. For instance, in thesystem in Figure 7, the deterministic strategy profile where every player i chooses to play a i at the beginning determines two different runs and traces of the system.Therefore, formally, a deterministic strategy profile f on a nondeterministic system M may determine a set of computations in comps ωM . To simplify notations, we will write κ M ( f ) ⊆ comps ωM for such a set, which will correspond to the set of computations thatcould result in M when playing strategy profile f . Likewise, we will write ρ M ( f ) ⊆ runs ωM and τ M ( f ) ⊆ traces ωM , respectively, for the sets of runs and traces determined by f on M .These three sets of computations, runs, and traces determined by f , namely κ M ( f ), ρ M ( f ),and τ M ( f ), will correspond to our more general notion of (computation, run, trace) outcome ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:37 s M x s z s ya , . . . , a k a , . . . , a k ∗ , . . . , ∗∗ , . . . , ∗ Figure 7: Nondeterministic system with sets of infinite runs given by s M . ( s ω + s ω ) andinfinite traces given by x. ( z ω + y ω ).of a multi-player game. Clearly, for deterministic systems, these sets of computations, runs,and traces will correspond to the special case where outcomes are singleton sets. Our more general definition of outcome call for a (just slightly) more general definitionof equilibrium. The generalisation is rather simple. With respect to a nondeterministicsystem M , we will define the preferences Γ i of a player i as a set of sets of computationsof M , rather than just a set of computations, as in the deterministic case. In other words,while in a deterministic system we have Γ i ⊆ comps ωM , in a nondeterministic system wehave Γ i ⊆ comps ωM . With this definition in place, we can define a Nash equilibrium inexactly the same way that it is defined for deterministic systems, that is, as a strategyprofile f = ( f , . . . , f n ) such that for every player i and every strategy g i available to i , κ M ( f − i , g i ) ∈ Γ i implies κ M ( f ) ∈ Γ i .As for deterministic systems, the concepts of Nash equilibrium in run-based strategies and
Nash equilibrium in trace-based strategies are defined analogously.We first note that all negative results for deterministic systems immediately carry overto this more general setting as those are simply the case when deterministic strategy profilesinduce a unique computation (a singleton set of computations). On the other hand, althoughpositive results for computations and traces also carry over to nondeterministic systems,this is not something that one can immediately conclude. A couple of technical results areneeded. In the reminder of this section we will study why positive results for computationsand traces do carry over to nondeterministic systems.The first observation to make is that the set of strategy profiles across bisimilar systemsis invariant, that is, that every collection of (computation-based, trace-based) strategies f =( f , . . . , f n ) is a strategy profile in a system M if and only if f is a strategy profile in M (cid:48) ,for every M (cid:48) that is bisimilar to M . Lemma 6.1.
Let M and M (cid:48) be two bisimilar systems. For all (computation-based, trace-based) strategy profiles f : f is a strategy profile in M if and only if f is a strategy profile in M (cid:48) . Later on, in this section, we will present some examples of how sets of computations/runs/traces can beinduced by (deterministic) computation-based/run-based/trace-based strategies in nondeterministic systems.
J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3
Proof.
By induction on the length of computations/traces, and noting that, for every player,the set of actions available to a player in bisimilar states is the same (as otherwise the twostates would not be bisimilar).The second observation is that, despite nondeterminism, the outcome of games acrossbisimilar systems is invariant. Formally, we have the following result.
Lemma 6.2.
Let M and M (cid:48) be two bisimilar systems. For all (computation-based, trace-based) strategy profiles f we have κ M ( f ) = κ M (cid:48) ( f ) and τ M ( f ) = τ M (cid:48) ( f ) . Proof.
There are four different cases to consider here: either f is computation-based or f is trace-based , and either the outcome of the game is taken to be the set of computations , orthe outcome of the game is taken to be the set of traces .By double inclusion, we show the first case: f being computation-based and the outcomeof the game taken to be the set of computations . To show that κ M ( f ) ⊆ κ M (cid:48) ( f ), with f computation-based, reason by contradiction. Suppose that there is a computation κ ∗ in κ M ( f ) that is not in κ M (cid:48) ( f ). Since M and M (cid:48) are bisimilar, κ ∗ is also a computationof M (cid:48) , and due to Lemma 6.1, for every prefix κ ∗ k of κ ∗ , we know that f ( κ ∗ k ) is defined.Since f is functional, f ( κ ∗ k ) in M (cid:48) is the same as f ( κ ∗ k ) in M , which must be precisely thelast direction of κ ∗ k +1 . By an inductive argument we can conclude that κ ∗ must also be acomputation of κ M (cid:48) ( f ), which is a contradiction to our hypothesis, proving the statement.We can reason in a symmetric way to prove the inclusion in the other direction. Note thatfor computation-based strategies not only κ M ( f ) = κ M (cid:48) ( f ), but also they are singleton sets.The second case we consider is when f is trace-based and the outcome of the game istaken to be the set of traces . To show this case, we can reason similarly, but, unlike forcomputation-based strategies, the sets τ M ( f ) and τ M (cid:48) ( f ) may not be singleton sets. We,again, show the result by double inclusion, and each direction by contradiction. Thus, first,suppose that there is a trace τ ∗ in τ M ( f ) that is not in τ M (cid:48) ( f ). Since M and M (cid:48) are bisimilar, τ ∗ is also a trace of M (cid:48) , and due to Lemma 6.1, for every prefix τ ∗ k of τ ∗ , we know that f ( τ ∗ k )is defined. Let τ ∗ k be the smallest prefix of τ ∗ that is not a prefix of any trace in τ M (cid:48) ( f ),and let s be any state that can be reached after following the finite trace τ ∗ k − from s M , theinitial state of M . Then, we know that s f ( τ ∗ k − ) −−−−−→ q , for some q such that λ ( q ) is the lastelement of τ ∗ k . Necessarily, the prefix τ ∗ k − is the prefix of some trace in τ M (cid:48) ( f ) that leads toa state, say s (cid:48) , that is bisimilar to s . Because s and s (cid:48) are bisimilar, s (cid:48) f ( τ ∗ k − ) −→ q (cid:48) for somestate q (cid:48) that is bisimilar to q . Lemma 6.1 ensures that f is defined at τ ∗ k − in M (cid:48) . Since q and q (cid:48) are bisimilar, it also follows that λ ( q ) = λ ( q (cid:48) ), and therefore that τ ∗ k , with λ ( q (cid:48) ) beingthe last element of τ ∗ k , is the prefix of some trace in τ M (cid:48) ( f ), which is a contradiction to ourhypothesis. Therefore, via induction on the length of traces, we can conclude, in particular,that τ ∗ ∈ τ M (cid:48) ( f ), and in general that every trace in τ M ( f ) must also be a trace in τ M (cid:48) ( f ).The inclusion in the other direction is, as before, obtained by symmetric reasoning.The third case we consider is when f is computation-based and the outcome of the gameis taken to be the set of traces . This proof is almost identical to the previous case. To showthis case we, again, show the result by double inclusion, and each direction by contradiction.First, suppose that there is a trace τ ∗ in τ M ( f ) that is not in τ M (cid:48) ( f ). Since M and M (cid:48) arebisimilar, τ ∗ is also a trace of M (cid:48) , and due to Lemma 6.1, for every prefix τ ∗ k of τ ∗ andevery computation κ k ∈ κ ( τ ∗ k ), we know that f ( κ k ) is defined. Let τ ∗ k be the smallest prefix ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:39 of τ ∗ that is not a prefix of any trace in τ M (cid:48) ( f ), and let s be any state that can be reachedafter following the finite trace τ ∗ k − from s M , the initial state of M . Then, we know that forsome computation κ k − ∈ κ ( τ ∗ k − ), we have s f ( κ k − ) −−−−−→ q , for some q such that λ ( q ) is the lastelement of τ ∗ k . Necessarily, the prefix τ ∗ k − is the prefix of some trace in τ M (cid:48) ( f ) that leads toa state, say s (cid:48) , that is bisimilar to s . Because s and s (cid:48) are bisimilar, s (cid:48) f ( κ k − ) −−−−−→ q (cid:48) for somestate q (cid:48) that is bisimilar to q . Lemma 6.1 ensures that f is defined at κ k − in M (cid:48) . Since q and q (cid:48) are bisimilar, it also follows that λ ( q ) = λ ( q (cid:48) ), and therefore that τ ∗ k , with λ ( q (cid:48) ) beingthe last element of τ ∗ k , is the prefix of some trace in τ M (cid:48) ( f ), which is a contradiction to ourhypothesis. Therefore, via induction on the length of traces, we can conclude, in particular,that τ ∗ ∈ τ M (cid:48) ( f ), and in general that every trace in τ M ( f ) must also be a trace in τ M (cid:48) ( f ).The inclusion in the other direction is also obtained by symmetric reasoning.The fourth and final case we consider is when f is trace-based and the outcome of thegame is taken to be the set of computations . This proof is also almost identical to theprevious two cases. To show this case we, again, show the result by double inclusion, andeach direction by contradiction. First, suppose that there is a computation κ ∗ in κ M ( f ) thatis not in κ M (cid:48) ( f ). Since M and M (cid:48) are bisimilar, κ ∗ is also a computation of M (cid:48) , and dueto Lemma 6.1, for every prefix κ ∗ k of κ ∗ and every trace τ k ∈ τ ( κ ∗ k ), we know that f ( τ k ) isdefined. Let κ ∗ k be the smallest prefix of κ ∗ that is not a prefix of any computation in κ M (cid:48) ( f ),and let s be any state that can be reached after following the finite computation κ ∗ k − from s M , the initial state of M , while complying with τ ∗ k − , that is, any state s such thatΘ k − = s M f ( λ ( s M )) −−−−−−→ s f ( λ ( s M ) ,λ ( s )) −−−−−−−−−→ . . . f ( τ k − ) −−−−−→ s with τ ∗ k − = λ ( s M ) , λ ( s ) , . . . , λ ( s )and κ ∗ k − = f ( λ ( s M )) , f ( λ ( s M ) , λ ( s )) , . . . , f ( τ k − ) . Then, for trace τ k − ∈ τ ( κ ∗ k − ) as above, we have s f ( τ ∗ k − ) −−−−−→ q , for some q such that λ ( q )is the last element of τ ∗ k . Necessarily, the prefix κ ∗ k − is the prefix of some computationin κ M (cid:48) ( f ) that leads to a state, say s (cid:48) , that is bisimilar to s , that is, a computation κ ∗ k − = f ( λ ( s M (cid:48) )) , f ( λ ( s M (cid:48) ) , λ ( s (cid:48) )) , . . . , f ( τ k − )with τ ∗ k − = λ ( s M (cid:48) ) , λ ( s (cid:48) ) , . . . , λ ( s (cid:48) )and Θ (cid:48) k − = s M (cid:48) f ( λ ( s M (cid:48) )) −−−−−−→ s (cid:48) f ( λ ( s M (cid:48) ) ,λ ( s (cid:48) )) −−−−−−−−−−→ . . . f ( τ k − ) −−−−−→ s (cid:48) and Θ k − ( i ) bisimilar to Θ (cid:48) k − ( i ), for every 0 ≤ i ≤ k − f is functional and λ ( s M ) , λ ( s ) , . . . , λ ( s ) = λ ( s M (cid:48) ) , λ ( s (cid:48) ) , . . . , λ ( s (cid:48) )it follows that f ( λ ( s M ) , λ ( s ) , . . . , λ ( s k )) = f ( λ ( s M (cid:48) ) , λ ( s (cid:48) ) , . . . , λ ( s (cid:48) )) and that s (cid:48) f ( τ ∗ k − ) −−−−−→ q (cid:48) for some state q (cid:48) bisimilar to s (cid:48) . Lemma 6.1 ensures that f is defined at κ k − in M (cid:48) . Since q and q (cid:48) are bisimilar, it also follows that λ ( q ) = λ ( q (cid:48) ), and therefore that τ ∗ k , with λ ( q (cid:48) )being the last element of τ ∗ k , is the prefix of some trace in τ M (cid:48) ( f ). Since τ ∗ k is indeed atrace in τ M (cid:48) ( f ) which can be obtained following some computation κ ∗ k in M (cid:48) , then we canconclude that κ ∗ k is the prefix of some computation κ ∗ in κ M (cid:48) ( f ), which is a contradiction J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3 s M x s M (cid:48) y ∗ ∗ Figure 8: Two non-bisimilar systems where every computation-based strategy profile deter-mines two different (infinite) trace outcomes, namely, x ω in the system on the left( M ) and y ω in the system on the right ( M (cid:48) ).to our hypothesis. Therefore, via induction on the length of computations, we can infer, inparticular, that κ ∗ ∈ τ M (cid:48) ( f ), and in general that every computation in κ M ( f ) must also bea computation in κ M (cid:48) ( f ). As in all previous cases, because M and M (cid:48) are bisimilar, theinclusion in the other direction is also obtained by symmetric reasoning.Using Lemma 6.1 and Lemma 6.2, we can then show that the set of (computation-based,trace-based) Nash equilibria across bisimilar systems remains invariant too. Theorem 6.3.
Let G = ( M, Γ , . . . , Γ n ) and G (cid:48) = ( M (cid:48) , Γ , . . . , Γ n ) be games on bisimilarnondeterministic concurrent game structures M and M (cid:48) , respectively. Let further f κ be acomputation-based strategy profile and f τ be a trace-based strategy profile. Then, (1) f κ is a computation-based Nash equilibrium in G if and only if f κ is a computation-basedequilibrium in G (cid:48) , and (2) f τ is a trace-based Nash equilibrium in G if and only if f τ is a trace-based equilibriumin G (cid:48) .Proof. Both proofs are by double implication, where each direction is proved by contradiction.For part (1), first assume that there is some computation-based equilibrium f κ in G thatis not a computation-based equilibrium in G (cid:48) . Because of Lemma 6.2, every player i whogets its goal achieved in M also gets its goal achieved in M (cid:48) . Then, they will not deviatein M (cid:48) . Therefore, there must be a player j who does not get its goals achieved in M andhas a beneficial deviation g j in M (cid:48) , that is, while κ M ( f κ ) (cid:54)∈ Γ j , we have κ M (cid:48) ( f κ − j , g j ) ∈ Γ j .Lemma 6.1 then ensures that g i is also a strategy in M , and Lemma 6.2 that κ M ( f κ − j , g j ) ∈ Γ j ,which is a contradiction with f κ being a computation-based Nash equilibrium in M . We canreason in a symmetric way to show the implication in the opposite direction. Notice thatbecause of Lemmas 6.1 and 6.2, the result holds for both computation-based preferencesand trace-based preferences. Finally, the proof for trace-based strategies (part (2)), witheither computation-based or trace-based preferences, follows the exact same reasoning.We note that the main idea behind the proofs in this section is that if a given strategyprofile f , whether computation-based or trace-based, does not determine the same set ofcomputations and traces in bisimilar systems, then that computation or trace could be usedto show that the two systems are in fact not bisimilar. This is the main argument behindthe four cases in Lemma 6.2, each requiring slightly different proofs that such a witness tothe non-bisimilarity of M and M (cid:48) does not exist. However, it is also important to note thatif M and M (cid:48) are not bisimilar, then a given strategy profile f , well defined in both systems,may not determine the same set of outcomes—see, for instance, the example in Figure 8.We would also like to note that even though for deterministic systems, computation-based strategies strictly generalise run-based strategies, and run-based strategies strictlygeneralise trace-base strategies, for nondeterministic systems this is no longer the case. ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:41 s M x s y s zaa cb Figure 9: A nondeterministic system in which no computation-based strategy can be defined,but where both run-based and trace-based strategies can be defined. s M x s y s zaa aa Figure 10: A system in which any computation-based strategy induces a set of runs and aset of traces containing, respectively, runs s M s ω and s M s ω and traces xy ω and xz ω .Run-based strategies still generalise trace-based strategies, but not other relation like thisholds between any other pair of models of strategies. For instance, as shown in the examplein Figure 9, there is a system in which, for instance, a trace-based strategy can be defined(as well as a run-based strategy) while a computation-based strategy cannot.In case strategies are well defined, as mentioned before, they can induce sets of computa-tions, runs, and traces in a nondeterministic system, except for one case: whenever defined, acomputation-based strategy always determines a unique computation of the system, whetherdeterministic or nondeterministic. Examples of all other cases (8 in total) are easy to build.For instance, in the nondeterministic system in Figure 9, any run-based or trace-basedstrategy will induce a set of computations containing both ab ω and ac ω . Correspondingly,they will also induce a set of runs and a set of traces, namely, those containing, respectively, s M s ω and s M s ω in case of runs, and xy ω and xz ω in case of traces. For the two remainingcases, a set of runs and a set of traces induced by a computation-based strategy, considerthe nondeterministic system in Figure 10, which is almost the same as the system in Fig-ure 9, save that a computation-based strategy can be defined. In such a system, any welldefined computation-based strategy will induce a set of runs and a set of traces containing,respectively, s M s ω and s M s ω in case of runs, and xy ω and xz ω in case of traces.Finally, the reader may have noticed that in this section we did not study the caseconsidering run-based preferences (for run-based strategies we know that the negative resultsfor deterministic systems carry over). The reason is that, as shown for deterministic systems, J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3 s M x s x s xa, ab, ba, bb, a ∗∗ s M (cid:48) x s x ∗ ∗ Figure 11: A pair of bisimilar systems, M and M (cid:48) , where the sets of run-based preferencesgiven by Γ = { s M , s , s , . . . } for player 1 and Γ = { s M , s , s , . . . } for player 2in system M , do not have a congruent counterpart in system M (cid:48) .we can ensure invariance of (the existence of) Nash equilibria with respect to bisimilarityonly if the sets of run-based preferences are congruent between bisimilar systems. As this isregarded as a major drawback, even for deterministic systems as illustrated in the examplein Figure 11, it is really not an interesting question to be investigated any further.7. Strategy Logics: New Semantic Foundations
Several logics for strategic reasoning have been proposed in the literature of computer scienceand AI, such as ATL ∗ [AHK02], Strategy Logic [MMPV14, CHP10], Coalition Logic [Pau02],Coordination Logic [FS10], Game Logic [PP03], and Equilibrium Logic [GHW17b]. Inseveral cases, the model of strategies that is used is the one that we refer to as run-based inthis paper, that is, strategies are functions from finite sequences of states (of some arena)to actions/decisions/choices of players in a given game. As can be seen from our resultsso far, of the four options we have explored, run-based strategies form the least desirablemodel of strategies from a semantic point of view since in such a case Nash equilibrium isnot preserved under bisimilarity.This does not necessarily immediately imply that a particular logic with a run-basedstrategy model is not invariant under bisimilarity. For instance, ATL ∗ is a bisimulation-invariant logic and, as shown in [GHW15a] one can reason about Nash equilibrium using ATL ∗ only up-to bisimilarity. A question then remains: whether any of these logics for strategicreasoning becomes invariant under bisimilarity—as explained before, a desirable property—ifone changes the model of strategies considered there to, for instance, computation-basedor trace-based strategies. We find that this question has a satisfactory positive answer insome cases. In particular, we will consider the above question in the context of StrategyLogic as studied in [MMPV14], and in doing so we will provide new semantic foundationsfor strategy logics.Let us start by introducing the syntax and semantics under the run-based model ofstrategies for Strategy Logic (SL [MMPV14]) as it has been given in [MMPV16]. Syntactically,SL extends LTL with two strategy quantifiers , (cid:104)(cid:104) x (cid:105)(cid:105) and [[ x ]], and an agent binding operator( i, x ), where i is an agent and x is a variable. Intuitively, these operators can be understood as ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:43 “there exists a strategy x ” , “for all strategies x ” , and “bind agent i to the strategy associatedwith the variable x ” , respectively. Formally, SL formulae are inductively built from a set ofatomic propositions AP, variables Var, and agents Ag, using the following grammar, where p ∈ AP, x ∈ Var, and i ∈ Ag: ϕ ::= p | ¬ ϕ | ϕ ∧ ϕ | X ϕ | ϕ U ϕ | (cid:104)(cid:104) x (cid:105)(cid:105) ϕ | [[ x ]] ϕ | ( i, x ) ϕ .We also use the usual abbreviations for LTL formulae, that is, those for Boolean and temporallogic formulae.We can now present the semantics of SL formulae. Given a concurrent game structure M , for all SL formulae ϕ , states s ∈ St in M , and assignments χ ∈ Asg = (Var ∪ Ag) → Str,mapping variables and agents to strategies, the relation
M, χ, s | = ϕ is defined as follows:(1) M, χ, s | = p if p ∈ λ ( s ), with p ∈ AP.(2) For all formulae ϕ , ϕ , and ϕ , we have:(a) M, χ, s | = ¬ ϕ if not M, χ, s | = ϕ ;(b) M, χ, s | = ϕ ∧ ϕ if M, χ, s | = ϕ and M, χ, s | = ϕ .(3) For all formulae ϕ and variables x ∈ Var we have:(a)
M, χ, s | = (cid:104)(cid:104) x (cid:105)(cid:105) ϕ if there is a strategy f ∈ Str such that
M, χ [ x (cid:55)→ f ] , s | = ϕ ;(b) M, χ, s | = [[ x ]] ϕ if for all strategies f ∈ Str we have that
M, χ [ x (cid:55)→ f ] , s | = ϕ .(4) For all i ∈ Ag and x ∈ Var, we have
M, χ, s | = ( i, x ) ϕ if M, χ [ i (cid:55)→ χ ( x )] , s | = ϕ .(5) Moreover, for all formulas ϕ , ϕ , and ϕ , we have:(a) M, χ, s | = X ϕ if M, ( χ, s ) , δ ( s, d ) | = ϕ , where d is the decision taken from s byfollowing χ and ( χ, s ) is the update of the assignment function as describedin [MMPV14];(b) M, χ, s | = ϕ U ϕ if there exist k ∈ N such that M, ( χ, s ) k , δ ( s, (cid:126)d ) | = ϕ and, for all h ∈ N with h ≤ k , we have M, ( χ, s ) h , δ ( s, (cid:126)d ≤ h ) | = ϕ , where (cid:126)d is the sequence ofdecisions identified by the assignment function χ starting from s , and ( χ, s ) k is theupdate of the assignment given by the execution of k steps of the strategy profilein χ starting from s .Intuitively, rules 3a and 3b, respectively, are used to interpret the existential (cid:104)(cid:104) x (cid:105)(cid:105) anduniversal [[ x ]] quantifiers over strategies, and rule 4 is used to bind an agent to the strategiesassociated with variable x . All other rules are as in LTL over concurrent game structures.As can be seen from its semantics, SL can be interpreted under different models ofstrategies and goals. As it was originally formulated, SL considers run-based strategies andtrace-based preferences/goals. More specifically, the model of goals is a proper subset of thetrace-based one, represented by LTL goals over the set AP of variables. In SL, it is possibleto represent the existence of a Nash equilibrium in a concurrent game structure [MMPV14].This implies, given Theorem 4.5, that SL under the standard interpretation is not invariantunder bisimulation, as the formula expressing the existence of a Nash equilibrium candistinguish between two bisimilar models.Given the semantics of SL formulae given above, we now consider SL under the modelof computation-based strategies, and find that in such a case SL becomes invariant underbisimilarity. Formally, we have the following result. Theorem 7.1.
Let M = (Ag , AP , Ac , St , s , λ , δ ) and M = (Ag , AP , Ac , St , s , λ , δ ) be two bisimilar CGSs. Moreover, let χ be an assignment of strategies and s ∼ s be twobisimilar states. Then, for all ϕ ∈ SL , it holds that M , χ, s , | = ϕ if and only if M , χ, s | = ϕ. J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3
Proof.
The proof proceeds by induction on the structure of ϕ . First note that we do notneed to prove all the cases, as, for example, we have that ψ ∨ ψ = ¬ ( ¬ ψ ∧ ¬ ψ ) and[[ x ]] ϕ = ¬(cid:104)(cid:104) x (cid:105)(cid:105)¬ ϕ . Moreover, recall from Lemma 3.1 that every computation-based strategyin M is also a computation-based strategy in M and vice-versa. We have the following. • ϕ = p ∈ AP. We have that M , χ, s | = p if and only if p ∈ λ ( s ) = λ ( s ) if and only if M , χ, s | = p ; • ϕ = ¬ ψ . We have that M , χ, s | = ¬ ψ if and only if M , χ, s (cid:54)| = ψ if and only if, byinduction hypothesis, M , χ, s (cid:54)| = ψ if and only if M , χ, s | = ¬ ψ . • ϕ = ψ ∧ ψ . We have that M , χ, s | = ψ ∧ ψ if and only if M , χ, s | = ψ and M , χ, s | = ψ , which holds, by induction hypothesis, if and only if M , χ, s | = ψ and M , χ, s | = ψ if and only if M , χ, s | = ψ ∧ ψ . • ϕ = X ψ . We have that M , χ, s | = X ψ if and only if M , ( χ ) , δ ( s , d ) | = ψ , where d = ( χ (1)( (cid:15) ) , . . . χ ( n )( (cid:15) )) is the decision taken by the agents on the first round of thegame, according the assignment χ . By bisimilarity, we have that δ ( s , d ) ∼ δ ( s , d ),and so, by induction hypothesis, that M , ( χ ) , δ ( s , d ) | = ψ , that holds if and only if M , χ, s | = X ψ . • ϕ = ϕ U ϕ . We have that M , χ, s | = ϕ U ϕ if and only if there exists k ∈ N such that M , χ k , δ ∗ ( s , (cid:126)d ) | = ϕ and M , χ h , δ ∗ ( s , (cid:126)d ≤ h ) | = ϕ for every h < k , where (cid:126)d is the uniquesequence of decisions identified by the k -steps application of the transition function thatfollows χ . Observe that, for each h ≤ k , we have that δ ( s , (cid:126)d ≤ h ) ∼ δ ( s , (cid:126)d ≤ h ) and so, byinduction hypothesis, we have that M , χ k , δ ∗ ( s , (cid:126)d ) | = ϕ and M , χ h , δ ∗ ( s , (cid:126)d ≤ h ) | = ϕ for every h < k , that is, if and only if M , χ, s | = ϕ U ϕ . • ϕ = ( i, x ) ψ . We have that M , χ, s | = ( i, x ) ψ if and only if M , χ [ i (cid:55)→ χ ( x )] , s | = ψ if andonly if, by induction hypothesis, M , χ [ i (cid:55)→ χ ( x )] , s | = ψ if and only if M , χ, s | = ( i, x ) ψ . • ϕ = (cid:104)(cid:104) x (cid:105)(cid:105) ψ . We have that M , χ, s | = (cid:104)(cid:104) x (cid:105)(cid:105) ψ if and only if there exists a strategy f ∈ Strsuch that M , χ [ x (cid:55)→ f ] , s | = ψ if and only if, by induction hypothesis M , χ [ x (cid:55)→ f ] , s | = ψ , if and only if M , χ, s | = (cid:104)(cid:104) x (cid:105)(cid:105) ψ .This concludes the proof.As an immediate corollary, we then obtain the following result about the semanticrelationship between the properties that can be expressed in SL and the concept of bisimilarity. Corollary 7.2.
SL with the computation-based model of strategies is invariant under bisim-ilarity.
Finally, an analogous statement to the above Corollary can also be proved if we considerthe model of trace-based strategies, leading to the next result on the semantics of SL.
Corollary 7.3.
SL with the trace-based model of strategies is invariant under bisimilarity. Concluding Remarks and Related Work
In this paper we showed that with the conventional model of strategies used in the logic,computer science, and AI literatures, the existence of Nash equilibria is not necessarilypreserved under bisimilarity—in particular this is the case for multi-player games played overdeterministic concurrent games structures. By way of some examples, we also illustratedsome of the implications of this result—for example, in the context of automated formalverification. To resolve this difficulty, we furthermore investigated alternative models of ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:45 strategies which exhibit some desirable properties, in particular, allowing for a formalisationof Nash equilibrium that is invariant under bisimilarity , even on nondeterministic systems.We studied applications of these models and found that through their use, not only Nashequilibria become invariant under bisimilarity, but also full logics such as Strategy Logic. Thisrenders it possible to combine commonly used optimisation techniques for model checkingwith decision procedures for the analysis of Nash equilibria, thus overcoming a criticalproblem of this kind of logics regarding practical applications via automated verification.Some work also in the intersection between bisimulation equivalences, concurrent gamestructures, Nash equilibria, and automated formal verification is summarised next.
Logics for Strategic Reasoning.
There is now a large literature on logics for strategicreasoning. From this literature, ATL ∗ [AHK02] and SL [MMPV14] stand out, both due totheir use within a number of practical tools for automated verification, and because of theirexpressive power. On the one hand, ATL ∗ is known to be invariant under bisimilarity usingthe conventional model of strategies. As such, Nash equilibria can be expressed within ATL ∗ only up to bisimilarity [GHW15a]. On the other hand, SL, which is strictly more expressivethan ATL ∗ , allows for a simple specification of Nash equilibria, but suffers from not beinginvariant under bisimilarity with respect to the conventional model of strategies. In thispaper, we have put forward a number of solutions to this problem. An additional advantageof replacing the model of strategies for SL (and therefore for concurrent game structures) isthat other solution concepts in game theory also become invariant under bisimilarity. Forinstance, subgame-perfect Nash equilibria and strong Nash equilibria—which are widely usedwhen considering, respectively, dynamic behaviour and cooperative behaviour in multi-agentsystems—can also be expressed in SL. Our results therefore imply that these concepts arealso invariant under bisimilarity, when considering games over concurrent game structuresand goals given by LTL formulae (which correspond to preferences over traces). Bisimulation Equivalences for Multi-Agent Systems.
Even though bisimilarity isprobably the most widely used behavioural equivalence in concurrency, in the context ofmulti-agent systems other relations may be preferred, for instance, equivalence relationsthat take a detailed account of the independent interactions and behaviour of individualcomponents in a multi-agent system. In such a setting, “alternating” relations with naturalATL ∗ characterisations have been studied [AHKV98]. Our results also apply to suchalternating equivalence relations. Alternating bisimulation is very similar to bisimilarityon labelled transition systems [Mil80, HM85], only that when defined on concurrent gamestructures, instead of action profiles taken as possible transitions, one allows individualplayer’s actions, which must be matched in the bisimulation game. Because of this, itimmediately follows that any alternating bisimulation as defined in [AHKV98] is also abisimilarity as defined here. Despite having a different formal definition, a simple observationcan be made: that the counter-example shown in Figures 1 and 2 also apply to suchalternating (bisimulation) relations. This immediately implies that Nash equilibria arenot preserved by the alternating (bisimulation) equivalence relations in [AHKV98] either.Nevertheless, as discussed in [vB02], the “right” notion of equivalence for games and theirgame theoretic solution concepts is, undoubtedly, an important and interesting topic ofdebate, which deserves to be investigated further. J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3
Computations vs. Traces.
An important remark about the difference between computa-tions and traces is that even though Nash equilibria and their existence are preserved underbisimilarity by three of the four strategy models we have studied, it is not the case that witheach strategy model we obtain the same set of Nash equilibria in a given system, or that wecan sustain the same set of computations or traces. For instance, consider again the gamesin Figures 1 and 2. As we discussed above, if we consider the model of computation-basedstrategies and LTL goals ( i.e. , trace-based goals) as shown in the example, then we obtaintwo games, each with an associated non-empty set of Nash equilibria, which are preservedby bisimilarity. However, if we consider, instead, the model of trace-based strategies andthe same LTL goals, then we obtain two concurrent games both with empty sets of Nashequilibria—thus, in this case, the non-existence of Nash equilibria is preserved by bisimilarity!To observe this, note that whereas in the case of computation-based strategies player 3 canimplement a uniform “punishment” strategy for both player 1 and player 2, in the case oftrace-based strategies player 3 cannot do so, even in the game in Figure 1.
Two-Player Games with Trace-Based Goals.
We also showed that if we consider two-player games together with the conventional model of strategies, the problems that arisewith respect to the preservation of Nash equilibria disappear. This is indeed an importantfinding since most verification games ( e.g. , model and module checking, synthesis, etc.) canbe phrased in terms of zero-sum two-player games together with temporal logic specifications( e.g. , using LTL, CTL, or ATL ∗ ). Our results, then, provide conclusive proof that, if onlytwo-player games and temporal logic goals are needed, then all equilibrium analyses can becarried out using the conventional model of strategies—along with their associated reasoningtools and formal verification techniques. Nondeterminism.
We extended our main bisimulation-invariant results to nondetermin-istic systems, making it possible to analyse more complex systems. This was possible, inturn, because our two main models of strategies, namely computation-based and trace-based,are themselves oblivious to nondeterministic choices. As a consequence, given a particularstrategy (or strategy profile, more generally), the set of outcomes of a multi-player gameacross bisimilar structures remains the same. Indeed, the definitions of strategies in thecomputation-based and trace-based models can be used to show that the set of Nash equi-libria in strategy profiles given by these two models is invariant across systems that areequivalent with respect to equivalences for concurrency that are weaker than bisimilarity;for instance, across trace equivalent systems as defined in CSP [BHR84]. Thus, with respectto this kind of systems, all our positive results also carry over, even for nondeterministicprocesses.
Tools for Model Checking and Equilibrium Analysis.
Due to the success of temporallogics and model checking in the verification of concurrent and multi-agent systems, somemodel checking tools have been extended to cope with the strategic analysis of concurrentsystems modelled as multi-player games. For instance, tools such as MCMAS [CLMM14],EAGLE [TGW15], PRALINE [Bre13], MOCHA [AHM + some strategic properties in a system. Because all of these tools relyon underlying algorithms for temporal logic model checking, hardly any optimisations arepossible when moving to the more complex game-theoretic setting where Nash equilibria ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:47 needs to be analysed. In this way, our results find a powerful, and immediate, practicalapplication. Indeed, based on the work presented in this paper, we have developed a newtool for temporal equilibrium analysis [GNPW18], which uses the computation-based modelof strategies studied here.As mentioned before, we have developed a new tool for temporal equilibrium analysis,which we call EVE [GNPW18] (Equilibrium Verification Environment). EVE uses thecomputation-based model of strategies and trace-based preferences given by LTL formulae.EVE is a formal verification tool for the automated analysis of temporal equilibrium propertiesof concurrent and multi-agent systems modelled using the Simple Reactive Module Language(SRML [AH99, vdHLW06]) as a collection of independent system components (players/agentsin a game). In particular, EVE automatically solves three key decision problems in rationalsynthesis and verification [GHW17a, WGH +
16, FKL10]:
Non-Emptiness , E-Nash , and
A-Nash . These problems ask, respectively, whether a multi-player game has at least oneNash equilibrium, whether an LTL formula holds on some
Nash equilibrium, and whetheran LTL formula holds on all
Nash equilibria. EVE uses a technique based on parity gamesto check for the existence of Nash equilibria in a concurrent and multi-player game, whichcrucially relies on the underlying model of strategies being bisimulation invariant.
Acknowledgment
This paper is a revised and extended version of [GHPW17]. All authors acknowledge withgratitude the financial support of ERC Advanced Investigator Grant 291528 (“RACE”) atthe University of Oxford. Paul Harrenstein was also supported in part by ERC StartingGrant 639945 (“ACCORD”) also at the University of Oxford. Michael Wooldridge and PaulHarrenstein furthermore acknowledge the financial support of the Alan Turing Institute inLondon. Giuseppe Perelli was also supported in part by the ERC Consolidator Grant 772459(“DSynMA”). We also thank Johan van Benthem, the reviewers of CONCUR 2017, and theparticipants of Dagstuhl seminar 17111 (“Game Theory in AI, Logic, and Algorithms”) fortheir comments and helpful discussions. Finally, we would also like to thank the reviewersof
Logical Methods in Computer Science for their detailed and thoughtful comments.
References [AH99] Rajeev Alur and Thomas A. Henzinger. Reactive modules.
Formal Methods in System Design ,15(1):7–48, 1999.[AHK02] R. Alur, T.A. Henzinger, and O. Kupferman. Alternating-Time Temporal Logic.
Journal of theACM , 49(5):672–713, 2002.[AHKV98] Rajeev Alur, Thomas A. Henzinger, Orna Kupferman, and Moshe Y. Vardi. Alternating refine-ment relations. In
CONCUR , volume 1466 of
LNCS , pages 163–178. Springer, 1998.[AHM +
98] Rajeev Alur, Thomas A. Henzinger, Freddy Y. C. Mang, Shaz Qadeer, Sriram K. Rajamani,and Serdar Tasiran. MOCHA: modularity in model checking. In
CAV , volume 1427 of
LNCS ,pages 521–525. Springer, 1998.[BBMU11] P. Bouyer, R. Brenguier, Nicolas Markey, and M. Ummels. Nash equilibria in concurrent gameswith B¨uchi objectives. In S. Chakraborty and A. Kumar, editors, , pages375—386, 2011.[BBMU15] P. Bouyer, R. Brenguier, Nicolas Markey, and M. Ummels. Pure Nash equilibria in concurrentdeterministic games.
Logical Methods in Computer Science , 11(2:9):1–72, 2015.[BHR84] Stephen D. Brookes, C. A. R. Hoare, and A. W. Roscoe. A theory of communicating sequentialprocesses.
Journal of the ACM , 31(3):560–599, 1984.
J. Gutierrez, P. Harrenstein, G. Perelli, and M. Wooldridge
Vol. 15:3 [BK08] C. Baier and J.-P. Katoen.
Principles of Model Checking . MIT Press, 2008.[Bre13] Romain Brenguier. PRALINE: A tool for computing nash equilibria in concurrent games. In
CAV , volume 8044 of
LNCS , pages 890–895. Springer, 2013.[Bry92] Randal E. Bryant. Symbolic boolean manipulation with ordered binary-decision diagrams.
ACMComput. Surv. , 24(3):293–318, 1992.[CC02] Patrick Cousot and Radhia Cousot. On abstraction in software verification. In
CAV , volume2404 of
LNCS , pages 37–56. Springer, 2002.[CE81] Edmund M. Clarke and E. Allen Emerson. Design and synthesis of synchronization skeletonsusing branching-time temporal logic. In
Logics of Programs , volume 131 of
LNCS , pages 52–71.Springer, 1981.[CGL94] Edmund M. Clarke, Orna Grumberg, and David E. Long. Model checking and abstraction.
ACMTrans. Program. Lang. Syst. , 16(5):1512–1542, 1994.[CGP02] E.M. Clarke, O. Grumberg, and D.A. Peled.
Model Checking.
MIT Press, 2002.[CHP10] Krishnendu Chatterjee, Thomas A. Henzinger, and Nir Piterman. Strategy logic.
Inf. Comput. ,208(6):677–693, 2010.[CLMM14] Petr Cerm´ak, Alessio Lomuscio, Fabio Mogavero, and Aniello Murano. MCMAS-SLK: A modelchecker for the verification of strategy logic specifications. In
CAV , volume 8559 of
LNCS , pages525–532. Springer, 2014.[DGL16] S. Demri, V. Goranko, and M. Lange.
Temporal Logics in Computer Science , volume 58 of
Cambridge Tracts in Theoretical Computer Science . Cambridge University Press, 2016.[DV95] R. De Nicola and F. W. Vaandrager. Three logics for branching bisimulation.
Journal of theACM , 42(2):458–487, 1995.[FKL10] Dana Fisman, Orna Kupferman, and Yoad Lustig. Rational synthesis. In
TACAS , volume 6015of
LNCS , pages 190–204. Springer, 2010.[FS10] Bernd Finkbeiner and Sven Schewe. Coordination logic. In Anuj Dawar and Helmut Veith,editors,
CSL , volume 6247 of
LNCS , pages 305–319. Springer, 2010.[GHPW17] Julian Gutierrez, Paul Harrenstein, Giuseppe Perelli, and Michael Wooldridge. Nash equilibriumand bisimulation invariance. In
CONCUR , volume 85 of
LIPIcs , pages 17:1–17:16. SchlossDagstuhl, 2017.[GHW15a] Julian Gutierrez, Paul Harrenstein, and Michael Wooldridge. Expresiveness and complexityresults for strategic reasoning. In
CONCUR , volume 42 of
LIPIcs , pages 268–282. SchlossDagstuhl, 2015.[GHW15b] Julian Gutierrez, Paul Harrenstein, and Michael Wooldridge. Iterated Boolean games.
Inf.Comput. , 242:53–79, 2015.[GHW17a] Julian Gutierrez, Paul Harrenstein, and Michael Wooldridge. From model checking to equilibriumchecking: Reactive modules for rational verification.
Artificial Intelligence , 248:123–157, 2017.[GHW17b] Julian Gutierrez, Paul Harrenstein, and Michael Wooldridge. Reasoning about equilibria ingame-like concurrent systems.
Ann. Pure Appl. Logic , 168(2):373–403, 2017.[GNPW18] Julian Gutierrez, Muhammad Najib, Giuseppe Perelli, and Michael Wooldridge. EVE: A tool fortemporal equilibrium analysis. In
ATVA , LNCS. Springer, 2018.[GW14] Julian Gutierrez and Michael Wooldridge. Equilibria of concurrent games on event structures. In
CSL-LICS , pages 46:1–46:10. ACM, 2014.[HM85] Matthew Hennessy and Robin Milner. Algebraic laws for nondeterminism and concurrency.
J.ACM , 32(1):137–161, 1985.[KNP11] Marta Z. Kwiatkowska, Gethin Norman, and David Parker. PRISM 4.0: Verification of proba-bilistic real-time systems. In
CAV , volume 6806 of
LNCS , pages 585–591. Springer, 2011.[Koz83] Dexter Kozen. Results on the propositional mu-calculus.
Theoretical Computer Science , 27:333–354, 1983.[KPW16] Marta Kwiatkowska, David Parker, and Clemens Wiltsche. Prism-games 2.0: A tool for multi-objective strategy synthesis for stochastic games. In
TACAS , volume 9636 of
LNCS , pages560–566. Springer, 2016.[Mil80] R. Milner.
A Calculus of Communicating Systems , volume 92 of
LNCS . Springer, 1980.[Mil89] R. Milner.
Communication and Concurrency . Prentice Hall, 1989.[MMPV14] F. Mogavero, A. Murano, G. Perelli, and M.Y. Vardi. Reasoning About Strategies: On theModel-Checking Problem.
ACM Transactions On Computational Logic , 15(4):34:1–42, 2014. ol. 15:3 NASH EQUILIBRIUM AND BISIMULATION INVARIANCE 32:49 [MMPV16] F. Mogavero, A. Murano, G. Perelli, and M. Y. Vardi. Reasoning about Strategies: on theSatisfiability Problem.
CoRR , abs/1611.08541, 2016.[MSZ13] M. Maschler, E. Solan, and S. Zamir.
Game Theory . Cambridge University Press, 2013.[OR94] M.J. Osborne and A. Rubinstein.
A Course in Game Theory.
MIT Press, 1994.[Pau02] Marc Pauly. A modal logic for coalitional power in games.
J. Log. Comput. , 12(1):149–166, 2002.[Pnu77] Amir Pnueli. The temporal logic of programs. In
FOBS , pages 46–57. IEEE Computer Society,1977.[PP03] Marc Pauly and Rohit Parikh. Game logic—an overview.
Studia Logica , 75(2):165–182, 2003.[San09] Davide Sangiorgi. On the origins of bisimulation and coinduction.
ACM Trans. Program. Lang.Syst. , 31(4), 2009.[SR12] Davide Sangiorgi and Jan Rutten, editors.
Advanced Topics in Bisimulation and Coinduction ,volume 52 of
Cambridge Tracts in Theoretical Computer Science . Cambridge University Press,2012.[TGW15] Alexis Toumi, Julian Gutierrez, and Michael Wooldridge. A tool for the automated verification ofnash equilibria in concurrent games. In
ICTAC , volume 9399 of
LNCS , pages 583–594. Springer,2015.[van76] J. van Benthem.
Modal Correspondence Theory . PhD thesis, University of Amsterdam, 1976.[vB02] Johan van Benthem. Extensive games as process models.
Journal of Logic, Language andInformation , 11(3):289–313, 2002.[vdHLW06] Wiebe van der Hoek, Alessio Lomuscio, and Michael Wooldridge. On the complexity of practicalATL model checking. In
AAMAS , pages 201–208. ACM, 2006.[vGW96] R: J. van Glabbeek and W. P. Weijland. Branching time and abstraction in bisimulation semantics.
Journal of the ACM , 43(3):555–600, 1996.[WGH +
16] Michael Wooldridge, Julian Gutierrez, Paul Harrenstein, Enrico Marchioni, Giuseppe Perelli,and Alexis Toumi. Rational verification: From model checking to equilibrium checking. In
AAAI ,pages 4184–4191. AAAI Press, 2016.
This work is licensed under the Creative Commons Attribution License. To view a copy of thislicense, visit https://creativecommons.org/licenses/by/4.0/https://creativecommons.org/licenses/by/4.0/