Distributed Computing with Adaptive Heuristics
aa r X i v : . [ c s . D C ] O c t Distributed Computing with Adaptive Heuristics
Revised version will appear in the
Proceedings of Innovations in Computer Science 2011
Aaron D. Jaggard ∗ Dept. of Computer Science, Colgate UniversityDIMACS, Rutgers University [email protected]
Michael Schapira † Dept. of Computer ScienceYale University and UC Berkeley [email protected]
Rebecca N. Wright ‡ Dept. of Computer Science and DIMACSRutgers University [email protected]
Abstract
We use ideas from distributed computing to study dynamic environments in which computa-tional nodes, or decision makers, follow adaptive heuristics [16], i.e. , simple and unsophisticatedrules of behavior, e.g. , repeatedly “best replying” to others’ actions, and minimizing “regret”,that have been extensively studied in game theory and economics. We explore when convergenceof such simple dynamics to an equilibrium is guaranteed in asynchronous computational envi-ronments, where nodes can act at any time. Our research agenda, distributed computing withadaptive heuristics , lies on the borderline of computer science (including distributed computingand learning) and game theory (including game dynamics and adaptive heuristics). We exhibita general non-termination result for a broad class of heuristics with bounded recall—that is,simple rules of behavior that depend only on recent history of interaction between nodes. Weconsider implications of our result across a wide variety of interesting and timely applications:game theory, circuit design, social networks, routing and congestion control. We also studythe computational and communication complexity of asynchronous dynamics and present somebasic observations regarding the effects of asynchrony on no-regret dynamics. We believe thatour work opens a new avenue for research in both distributed computing and game theory. ∗ Supported in part by NSF grants 0751674 and 0753492. † Supported by NSF grant 0331548. ‡ Supported in part by NSF grant 0753061.
Introduction
Dynamic environments where computational nodes, or decision makers, repeatedly interact arise ina variety of settings, such as Internet protocols, large-scale markets, social networks, multi-processorcomputer architectures, and more. In many such settings, the prescribed behavior of the nodes isoften simple, natural and myopic (that is, a heuristic or “rule of thumb”), and is also adaptive, inthe sense that nodes constantly and autonomously react to others. These “adaptive heuristics”—aterm coined in [16]—include simple behaviors, e.g. , repeatedly “best replying” to others’ actions,and minimizing “regret”, that have been extensively studied in game theory and economics.Adaptive heuristics are simple and unsophisticated, often reflecting either the desire or necessityfor computational nodes (whether humans or computers) to provide quick responses and have alimited computational burden. In many interesting contexts, these adaptive heuristics can, in thelong run, move the global system in good directions and yield highly rational and sophisticatedbehavior, such as in game theory results demonstrating the convergence of best-response or no-regret dynamics to equilibrium points (see [16] and references therein).However, these positive results for adaptive heuristics in game theory are, with but a few ex-ceptions (see Section 2), based on the sometimes implicit and often unrealistic premise that nodes’actions are somehow synchronously coordinated. In many settings, where nodes can act at anytime, this kind of synchrony is not available. It has long been known that asynchrony introducessubstantial difficulties in distributed systems, as compared to synchrony [12], due to the “limitationimposed by local knowledge” [24]. There has been much work in distributed computing on identify-ing conditions that guarantee protocol termination in asynchronous computational environments.Over the past three decades, we have seen many results regarding the possibility/impossibility bor-derline for failure-resilient computation [11, 24]. In the classical results of that setting, the risk ofnon-termination stems from the possibility of failures of nodes or other components.We seek to bring together these two areas to form a new research agenda on distributed com-puting with adaptive heuristics. Our aim is to draw ideas from distributed computing theory toinvestigate provable properties and possible worst-case system behavior of adaptive heuristics inasynchronous computational environments. We take the first steps of this research agenda. Weshow that a large and natural class of adaptive heuristics fail to provably converge to an equilib-rium in an asynchronous setting, even if the nodes and communication channels are guaranteed tobe failure-free. This has implications across a wide domain of applications: convergence of gamedynamics to pure Nash equilibria; stabilization of asynchronous circuits; convergence to a stablerouting tree of the Border Gateway Protocol, that handles Internet routing; and more. We alsoexplore the impact of scheduling on convergence guarantees. We show that non-convergence isnot inherent to adaptive heuristics, as some forms of regret minimization provably converge inasynchronous settings. In more detail, we make the following contributions:
General non-convergence result (Section 4).
It is often desirable or necessary due to prac-tical constraints that computational nodes’ ( e.g. , routers’) behavior rely on limited memory andprocessing power. In such contexts, nodes’ adaptive heuristics are often based on bounded recall — i.e. , depend solely on recent history of interaction with others—and can even be historyless — i.e. ,nodes only react to other nodes’ current actions). We exhibit a general impossibility result usinga valency argument—a now-standard technique in distributed computing theory [11, 24]—to showthat a broad class of bounded-recall adaptive heuristics cannot always converge to a stable state.More specifically, we show that, for a large family of such heuristics, simply the existence of two1equilibrium points” implies that there is some execution that does not converge to any outcomeeven if nodes and communication channels are guaranteed not to fail. We also give evidence thatour non-convergence result is essentially tight. Implications across a wide variety of interesting and timely applications (Section 5).
We apply our non-convergence result to a wide variety of interesting environments, namely conver-gence of game dynamics to pure Nash equilibria, stabilization of asynchronous circuits, diffusion oftechnologies in social networks, routing on the Internet, and congestion control protocols.
Implications for convergence of r -fairness and randomness (Section 6). We study theeffects on convergence to a stable state of natural restrictions on the order of nodes’ activations (i.e.,the order in which nodes’ have the opportunity to take steps), that have been extensively studiedin distributed computing theory: (1) r -fairness , which is the guarantee that each node selects anew action at least once within every r consecutive time steps, for some pre-specified r >
0; and(2) randomized selection of the initial state of the system and the order of nodes’ activations.
Communication and computational complexity of asynchronous dynamics (Section 7).
We study the tractability of determining whether convergence to a stable state is guaranteed. Wepresent two complementary hardness results that establish that, even for extremely restricted kindsof interactions, this feat is hard: (1) an exponential communication complexity lower bound; and(2) a computational complexity PSPACE-completeness result that, alongside its computationalimplications, implies that we cannot hope to have short witnesses of guaranteed asynchronousconvergence (unless PSPACE ⊆ NP).
Asynchronous no-regret dynamics (Section 8).
We present some basic observations aboutthe convergence properties of no-regret dynamics in our framework, that establish that, in contrastto other adaptive heuristics, regret minimization is quite robust to asynchrony.
Further discussion of a research agenda in distributed computing with adaptive heuris-tics (Section 9)
We believe that this work has but scratched the surface in the exploration ofthe behavior of adaptive heuristics in asynchronous computational environments. Many importantquestions remain wide open. We present context-specific problems in the relevant sections, andalso outline general interesting directions for future research in Section 9.Before presenting our main results, we overview related work (Section 2) and provide a detaileddescription of our model (Section 3).
Our work relates to many ideas in game theory and in distributed computing. We discuss gametheoretic work on adaptive heuristics and on asynchrony, and also distributed computing work onfault tolerance and self stabilization. We also highlight the application areas we consider.
Adaptive heuristics.
Much work in game theory and economics deals with adaptive heuristics(see Hart [16] and references therein). Generally speaking, this long line of research explores the“convergence” of simple and myopic rules of behavior ( e.g. , best-response/fictitious-play/no-regretdynamics) to an “equilibrium”. However, with few exceptions (see below), such analysis has sofar primarily concentrated on synchronous environments in which steps take place simultaneouslyor in some other predetermined prescribed order. In contrast, we explore adaptive heuristics inasynchronous environments, which are more realistic for many applications.
Game-theoretic work on asynchronous environments.
Some game-theoretic work on re-2eated games considers “asynchronous moves”. (see [23,34], among others, and references therein).Such work does not explore the behavior of dynamics, but has other research goals ( e.g. , charac-terizing equilibria, establishing Folk theorems). We are, to the best of our knowledge, the first tostudy the effects of asynchrony (in the broad distributed computing sense) on the convergence of game dynamics to equilibria. Fault-tolerant computation.
We use ideas and techniques from work in distributed computingon protocol termination in asynchronous computational environments where nodes and commu-nication channels are possibly faulty. Protocol termination in such environments, initially moti-vated by multi-processor computer architectures, has been extensively studied in the past threedecades [2,4,7,12,20,29], as nicely surveyed in [11,24]. Fischer, Lynch and Paterson [12] showed, ina landmark paper, that a broad class of failure-resilient consensus protocols cannot provably termi-nate. Intuitively, the risk of protocol nontermination in [12] stems from the possibility of failures; acomputational node cannot tell whether another node is silent due to a failure or is simply takinga long time to react. Our focus here is, in contrast, on failure-free environments.
Self stabilization.
The concept of self stabilization is fundamental to distributed computing anddates back to Dijkstra, 1973 (see [8] and references therein). Convergence of adaptive heuristics toan “equilibrium” in our model can be viewed as the self stabilization of such dynamics (where the“equilibrium points” are the legitimate configurations). Our formulation draws ideas from work indistributed computing ( e.g. , Burns’ distributed daemon model) and in networking research [14] onself stabilization.
Applications.
We discuss the implications of our non-convergence result across a wide variety ofapplications, that have previously been studied: convergence of game dynamics (see, e.g. , [18, 19]);asynchronous circuits (see, e.g. , [6]); diffusion of innovations, behaviors, etc ., in social networks (seeMorris [26] and also [21]); interdomain routing [14, 30]; and congestion control [13].
We now present our model for analyzing adaptive heuristics in asynchronous environments.
Computational nodes interacting.
There is an interaction system with n computational nodes ,1 , . . . , n . Each computational node i has an action space A i . Let A = × j ∈ [ n ] A j , where [ n ] = { , . . . , n } . Let A − i = × j ∈ [ n ] \{ i } A j . Let ∆( A i ) be the set of all probability distributions over theactions in A i . Schedules.
There is an infinite sequence of discrete time steps t = 1 , . . . . A schedule is a function σ that maps each t ∈ N + = { , , . . . } to a nonempty set of computational nodes: σ ( t ) ⊆ [ n ].Informally, σ determines (when we consider the dynamics of the system) which nodes are activated in each time-step. We say that a schedule σ is fair if each node i is activated infinitely many timesin σ , i.e. , ∀ i ∈ [ n ], there are infinitely many t ∈ N + such that i ∈ σ ( t ). For r ∈ N + , we say that aschedule σ is r -fair if each node is activated at least once in every sequence of r consecutive timesteps, i.e. , if, for every i ∈ [ n ] and t ∈ N + , there is at least one value t ∈ { t , t + 1 , . . . , t + r − } for which i ∈ σ ( t ). History and reaction functions.
Let H = ∅ , and let H t = A t for every t ≥
1. Intuitively, an Often, the term asynchrony merely indicates that players are not all activated at each time step, and thus isused to describe environments where only one player is activated at a time (“alternating moves”), or where there isa probability distribution that determines who is activated when. H t represents a possible history of interaction at time step t . For each node i , thereis an infinite sequence of functions f i = ( f ( i, , f ( i, , . . . , f ( i,t ) , . . . ) such that, for each t ∈ N + , f ( i,t ) : H t → ∆( A i ); we call f i the reaction function of node i . As discussed below, f i captures i ’sway of responding to the history of interaction in each time step. Restrictions on reaction functions.
We now present five possible restrictions on reactionfunctions: determinism, self-independence, bounded recall, stationarity and historylessness.1.
Determinism: a reaction function f i is deterministic if, for each input, f i outputs a singleaction (that is, a probability distribution where a single action in A i has probability 1).2. Self-independence: a reaction function f i is self-independent if node i ’s own (past andpresent) actions do not affect the outcome of f i . That is, a reaction function f i is self-independent if for every t ≥ g t : A t − i → ∆( A i ) such that f ( i,t ) ≡ g t .3. k -recall and stationarity: a node i has k -recall if its reaction function f i only depends onthe k most recent time steps, i.e. , for every t ≥ k , there exists a function g : H k → ∆( A i )such that f ( i,t ) ( x ) = g ( x | k ) for each input x ∈ H t ( x | k here denotes the last k coordinates, i.e. , n -tuples of actions, of x ). We say that a k -recall reaction function is stationary if the timecounter t is of no importance. That is, a k -recall reaction function is stationary if there existsa function g : H k → ∆( A i ) such that for all t ≥ k , f ( i,t ) ( x ) = g ( x | k ) for each input x ∈ H t .4. Historylessness: a reaction function f i is historyless if f i is 1-recall and stationary, that is,if f i only depends on i ’s and on i ’s neighbors’ most recent actions. Dynamics.
We now define dynamics in our model. Intuitively, there is some initial state (historyof interaction) from which the interaction system evolves, and, in each time step, some subset of thenodes reacts to the past history of interaction. This is captured as follows. Let s (0) , that shall becalled the “ initial state ”, be an element in H w , for some positive w ∈ N . Let σ be a schedule. Wenow describe the “( s (0) , σ ) -dynamics ”. The system’s evolution starts at time t = w + 1, when eachnode i ∈ σ ( w + 1) simultaneously chooses an action according to f ( i,w +1) , i.e. , node i randomizesover the actions in A i according to f ( i,w +1) ( s (0) ). We now let s (1) be the element in H w +1 for whichthe first w coordinates ( n -tuples of nodes’ actions) are as in s (0) and the last coordinate is the n -tuple of realized nodes’ actions at the end of time step t = w + 1. Similarly, in each time step t > w +1, each node in σ ( t ) updates its action according to f ( i,t ) , based on the past history s ( t − w − ,and nodes’ realized actions at time t , combined with s ( t − w − , define the history of interaction atthe end of time step t , s ( t − w ) . Convergence and convergent systems.
We say that nodes’ actions converge under the ( s (0) , σ )-dynamics if there exist some positive t ∈ N , and some action profile a = ( a , . . . , a n ), such that, forall t > t , s ( t ) = a . The dynamics is then said to converge to a , and a is called a “ stable state ” (forthe ( s (0) , σ )-dynamics), i.e. , intuitively, a stable state is a global action state that, once reached,remains unchanged. We say that the interaction system is convergent if, for all initial states s (0) and fair schedules σ , the ( s (0) , σ )-dynamics converges. We say that the system is r-convergent if,for all initial states s (0) and r-fair schedules σ , the ( s (0) , σ )-dynamics converges. Update messages.
Observe that, in our model, nodes’ actions are immediately observable toother nodes at the end of each time step (“ perfect monitoring ”). While this is clearly unrealistic insome important real-life contexts ( e.g. , some of the environments considered below), this restrictiononly strengthens our main results, that are impossibility results.
Deterministic historyless dynamics.
Of special interest to us is the case that all reaction4unctions are deterministic and historyless. We observe that, in this case, stable states have asimple characterization. Each reaction function f i is deterministic and historyless and so can bespecified by a function g i : A → A i . Let g = ( g , . . . , g n ). Observe that the set of all stable states(for all possible dynamics) is precisely the set of all fixed points of g . Below, when describing nodes’reaction functions that are deterministic and historyless we sometimes abuse notation and identifyeach f i with g i (treating f i as a function from A to A i ). In addition, when all the reaction functionsare also self-independent we occasionally treat each f i as a function from A − i to A i . We now present a general impossibility result for convergence of nodes’ actions under bounded-recalldynamics in asynchronous, distributed computational environments.
Theorem 4.1.
If each reaction function has bounded recall and is self-independent then the exis-tence of multiple stable states implies that the system is not convergent.
We note that this result holds even if nodes’ reaction functions are not stationary and arerandomized (randomized initial states and activations are discussed in Section 6). We present theproof of Theorem 4.1 in Appendix F. We now discuss some aspects of our impossibility result.
Neither bounded recall nor self-independence alone implies non-convergence
We showthat the statement of Theorem 4.1 does not hold if either the bounded-recall restriction, or theself-independence restriction, is removed.
Example . (the bounded-recall restriction cannot be removed) There are two nodes, 1and 2, each with the action space { x, y } . The deterministic and self-independent reaction functionsof the nodes are as follows: node 2 always chooses node 1’s action; node 1 will choose y if node 2’saction changed from x to y in the past, and x otherwise. Observe that node 1’s reaction function isnot bounded-recall but can depend on the entire history of interaction. We make the observationsthat the system is safe and has two stable states. Observe that if node 1 chooses y at some pointin time due to the fact that node 2’s action changed from x to y , then it shall continue to do sothereafter; if, on the other hand, 1 never does so, then, from some point in time onwards, node 1’saction is constantly x . In both cases, node 2 shall have the same action as node 1 eventually, andthus convergence to one of the two stable states, ( x, x ) and ( y, y ), is guaranteed. Hence, two stablestates exist and the system is convergent nonetheless Example . (the self-independence restriction cannot be removed) There are two nodes,1 and 2, each with action set { x, y } . Each node i ’s a deterministic and historyless reaction function f i is as follows: f i ( x, x ) = y ; in all other cases the node always (re)selects its current action ( e.g. , f ( x, y ) = x , f ( x, y ) = y ). Observe that the system has three stable states, namely all actionprofiles but ( x, x ), yet can easily be seen to be convergent. Connections to consensus protocols.
We now briefly discuss the interesting connections be-tween Theorem 4.1 and the non-termination result for failure-resilient consensus protocols in [12].We elaborate on this topic in Appendix A. Fischer et al. [12] explore when a group of processorscan reach a consensus even in the presence of failures, and exhibit a breakthrough non-terminationresult. Our proof of Theorem 4.1 uses a valency argument—an idea introduced in the proof of thenon-termination result in [12]. 5ntuitively, the risk of protocol non-termination in [12] stems from the possibility of failures; acomputational node cannot tell whether another node is silent due to a failure or is simply takinga long time to react. We consider environments in which nodes/communication channels cannotfail, and so each node is guaranteed that all other nodes react after “sufficiently long” time. Thisguarantee makes reaching a consensus in the environment of [12] easily achievable (see Appendix A).Unlike the results in [12], the possibility of nonconvergence in our framework stems from limitationson nodes’ behaviors. Hence, there is no immediate translation from the result in [12] to ours (andvice versa). To illustrate this point, we observe that in both Example 4.2 and Example 4.3, thereexist two stable states and an initial state from which both stable states are reachable (a “bivalentstate” [12]), yet the system is convergent (see Appendix A). This should be contrasted with theresult in [12] that establishes that the existence of an initial state from which two distinct outcomesare reachable implies the existence of a non-terminating execution.We investigate the link between consensus protocols and our framework further in Appendix F,where we take an axiomatic approach. We introduce a condition—“
Independence of Decisions ”(IoD)—that holds for both fault-resilient consensus protocols and for bounded-recall self-independentdynamics. We then factor the arguments in [12] through IoD to establish a non-termination re-sult that holds for both contexts, thus unifying the treatment of these dynamic computationalenvironments.
We present implications of our impossibility result in Section 4 for several well-studied environments:game theory, circuit design, social networks and Internet protocols. We now briefly summarize theseimplications, that, we believe, are themselves of independent interest. See Appendix B for a detailedexposition of the results in this section.
Game theory.
Our result, when cast into game-theoretic terminology, shows that if players’choices of strategies are not synchronized, then the existence of two (or more) pure Nash equilibriaimplies that a broad class of game dynamics ( e.g. , best-response dynamics with consistent tie-breaking) are not guaranteed to reach a pure Nash equilibrium. This result should be contrastedwith positive results for such dynamics in the traditional synchronous game-theoretic environments.
Theorem 5.1.
If there are two (or more) pure Nash equilibria in a game, then all bounded-recallself-independent dynamics can oscillate indefinitely for asynchronous player activations.
Corollary 5.2.
If there are two (or more) pure Nash equilibria in a game, then best-responsedynamics, and bounded-recall best-response dynamics (studied in [35]), with consistent tie-breaking,can fail to converge to an equilibrium in asynchronous environments.
Circuits.
Work on asynchronous circuits in computer architectures research explores the implica-tions of asynchrony for circuit design [6]. We observe that a logic gate can be regarded as executingan inherently historyless reaction function that is independent of the gate’s past and present “state”.Thus, we show that our result has implications for the stabilization of asynchronous circuits.
Theorem 5.3.
If two (or more) stable Boolean assignments exist for an asynchronous Booleancircuit, then that asynchronous circuit is not inherently stable. ocial networks. Understanding the ways in which innovations, ideas, technologies, and practices,disseminate through social networks is fundamental to the social sciences. We consider the classiceconomic setting [26] (that has lately also been approached by computer scientists [21]) where eachdecision maker has two technologies { A, B } to choose from, and each node in the social networkwishes to have the same technology as the majority of his “friends” (neighboring nodes in the socialnetwork). We exhibit a general impossibility result for this environment. Theorem 5.4.
In every social network, the diffusion of technologies can potentially never convergeto a stable global state.
Networking.
We consider two basic networking environments: (1) routing with the Border Gate-way Protocol (BGP), that is the “glue” that holds together the smaller networks that make up theInternet; and (2) the fundamental task of congestion control in communication networks, that isachieved through a combination of mechanisms on end-hosts ( e.g. , TCP), and on switches/routers ( e.g. , RED and WFQ). We exhibit non-termination results for both these environments.We abstract a recent result in [30] and prove that this result extends to several BGP-based multipath routing protocols that have been proposed in the past few years. Theorem 5.5. [30] If there are multiple stable routing trees in a network, then BGP is not safeon that network.
We consider the model for analyzing dynamics of congestion presented in [13]. We present thefollowing result.
Theorem 5.6.
If there are multiple capacity-allocation equilibria in the network then dynamics ofcongestion can oscillate indefinitely. r -Convergence and Randomness We now consider the implications for convergence of two natural restrictions on schedules: r -fairness and randomization. See Appendix C for a detailed exposition of the results in this section. Snakes in boxes and r -convergence. Theorem 4.1 deals with convergence and not r -convergence,and thus does not impose restrictions on the number of consecutive time steps in which a node canbe nonactive. What happens if there is an upper bound on this number, r ? We now show that if r < n − Example . (a system that is convergent for r < n − but nonconvergent for r = n − ) There are n ≥ , . . . , n , each with the action space { x, y } . Nodes’ deterministic, historylessand self-independent reaction functions are as follows. ∀ i ∈ [ n ], f i ( x n − ) = x and f i always outputs y otherwise. Observe that there exist two stable states: x n and y n . Observe that if r = n − y and all other nodes’ actionsare x . Then, nodes 1 and 2 are activated and, consequently, node 1’s action becomes x and node 2’saction becomes y . Next, nodes 2 and 3 are activated, and thus 2’s action becomes x and 3’s actionbecomes y . Then 3 , ,
5, and so on (traversing all nodes over and over againin cyclic order). This goes on indefinitely, never reaching one of the two stable states. Observe7hat, indeed, each node is activated at least once within every sequence of n − r < n − y , then convergence to y n is guaranteed. Clearly, if all nodes’ action is x then convergence to x n is guaranteed. Thus, anoscillation is possible only if, in each time step, exactly one node’s action is y . Observe that, givenour definition of nodes’ reaction functions, this can only be if the activation sequence is (essentially)as described above, i.e. , exactly two nodes are activated at a time. Observe also that this kind ofactivation sequence is impossible for r < n − r > n ? We use classical results in combinatorics regarding the size of a “ snake-in-the-box ” in a hypercube [1] to construct systems are r -convergent for exponentially-large r ’s, butare not convergent in general. Theorem 6.2.
Let n ∈ N be sufficiently large. There exists a system G with n nodes, in whicheach node i has two possible actions and each f i is deterministic, historyless and self-independent,such that G is r -convergent for r ∈ Ω(2 n ) , but G is not ( r + 1) -convergent. We note that the construction in the proof of Theorem 6.2 is such that there is a unique stablestate. We believe that the same ideas can be used to prove the same result for systems withmultiple stable states but the exact way of doing this eludes us at the moment, and is left as anopen question.
Problem 6.3.
Prove that for every sufficiently large n ∈ N , there exists a system G with n nodes, inwhich each node i has two possible actions, each f i is deterministic, historyless and self-independent,and G has multiple stable states, such that G is r -convergent for r ∈ Ω(2 n ) but G is not ( r + 1) -convergent. Does random choice (of initial state and schedule) help?
Theorem 4.1 tells us that, for abroad class of dynamics, a system with multiple stable states is nonconvergent if the initial stateand the node-activation schedule are chosen adversarially. Can we guarantee convergence if theinitial state and schedule are chosen at random ? Example . (random choice of initial state and schedule might not help) There are n nodes, 1 , . . . , n , and each node has action space { x, y, z } . The (deterministic, historyless and self-independent) reaction function of each node i ∈ { , . . . , n } is such that f i ( x n − ) = x ; f i ( z n − ) = z ;and f i = y for all other inputs. The (deterministic, historyless and self-independent) reactionfunction of each node i ∈ { , } is such that f i ( x n − ) = x ; f i ( z n − ) = z ; f i ( xy n − ) = y ; f i ( y n − ) = x ; and f i = y for all other inputs. Observe that there are exactly two stable states: x n and z n .Observe also that if nodes’ actions in the initial state do not contain at least n − x ’s, or at least n − z ’s, then, from that moment forth, each activated node in the set { , . . . , n } will choose theaction y . Thus, eventually the actions of all nodes in { , . . . , n } shall be y , and so none of the twostable states will be reached. Hence, there are 3 n possible initial states, such that only from 4 n + 2can a stable state be reached. When choosing the initial state uniformly at random the probabilityof landing on a “good” initial state (in terms of convergence) is thus exponentially small. We now explore the communication complexity and computational complexity of determiningwhether a system is convergent. We present hardness results in both models of computation even8or the case of deterministic and historyless adaptive heuristics. See Appendix D for a detailedexposition of the results in this section.We first present the following communication complexity result whose proof relies on combina-torial “snake-in-the-box” constructions [1].
Theorem 7.1.
Determining if a system with n nodes, each with actions, is convergent requires Ω(2 n ) bits. This holds even if all nodes have deterministic, historyless and self-independent reactionfunctions. The above communication complexity hardness result required the representation of the reactionfunctions to (potentially) be exponentially long. What if the reaction functions can be succinctlydescribed? We now present a strong computational complexity hardness result for the case thateach reaction function f i is deterministic and historyless, and is given explicitly in the form of aboolean circuit (for each a ∈ A the circuit outputs f i ( a )). We prove the following result. Theorem 7.2.
Determining if a system with n nodes, each with a deterministic and historylessreaction function, is convergent is PSPACE-complete. Our computational complexity result shows that even if nodes’ reaction functions can be suc-cinctly represented, determining whether the system is convergent is PSPACE-complete. Thisresult, alongside its computational implications, implies that we cannot hope to have short “wit-nesses” of guaranteed asynchronous convergence (unless PSPACE ⊆ NP). Proving the abovePSPACE-completeness result for the case self-independent reaction functions seems challenging.
Problem 7.3.
Prove that determining if a system with n nodes, each with a deterministic self-independent and historyless reaction function, is convergent is PSPACE-complete. Regret minimization is fundamental to learning theory, and has strong connections to game-theoretic solution concepts; if each player in a repeated game executes a no-regret algorithm whenselecting strategies, then convergence to an equilibrium is guaranteed in a variety of interesting con-texts. The meaning of convergence, and the type of equilibrium reached, vary, and are dependenton the restrictions imposed on the game and on the notion of regret. Work on no-regret dynamicstraditionally considers environments where all nodes are “activated” at each time step. We makethe simple observation that, switching our attention to r -fair schedules (for every r ∈ N + ), if analgorithm has no regret in the classic setting, then it has no regret in this new setting as well (forall notions of regret). Hence, positive results from the regret-minimization literature extend to thisasynchronous environment. See [3] for a thorough explanation about no-regret dynamics and seeAppendix E for a detailed explanation about our observations. We now mention two implicationsof our observation and highlight two open problems regarding regret minimization. Observation 8.1.
When all players in a zero-sum game use no-external-regret algorithms thenapproaching or exceeding the minimax value of the game is guaranteed.
Observation 8.2.
When all players in a (general) game use no-swap-regret algorithms the empir-ical distribution of joint players’ actions converges to a correlated equilibrium of the game. roblem 8.3. Give examples of repeated games for which there exists a schedule of player activa-tions that is not r -fair for any r ∈ N + for which regret-minimizing dynamics do not converge to anequilibrium (for different notions of regret/convergence/equilibria). Problem 8.4.
When is convergence of no-regret dynamics to an equilibrium guaranteed (for dif-ferent notions of regret/convergence/equilibria) for all r -fair schedules for non-fixed r ’s, that is, ifwhen r is a function of t ? In this paper, we have taken the first steps towards a complete understanding of distributed com-puting with adaptive heuristics. We proved a general non-convergence result and several hardnessresults within this model, and also discussed some important aspects such as the implications offairness and randomness, as well as applications to a variety of settings. We believe that we havebut scratched the surface in the exploration of the convergence properties of simple dynamics inasynchronous computational environments, and many important questions remain wide open. Wenow outline several interesting directions for future research.
Other heuristics, convergence notions, equilibria.
We have considered specific adaptiveheuristics, notions of convergence, and kinds of equilibria. Understanding the effects of asynchronyon other adaptive heuristics ( e.g. , better-response dynamics, fictitious play), for other notionsof convergence ( e.g. , of the empirical distributions of play), and for other kinds of equilibria ( e.g. ,mixed Nash equilibria, correlated equilibria) is a broad and challenging direction for future research.
Outdated and private information.
We have not explicitly considered the effects of makingdecisions based on outdated information. We have also not dealt with the case that nodes’ behaviorsare dependent on private information, that is, the case that the dynamics are “uncoupled” [18, 19].
Other notions of asynchrony.
We believe that better understanding the role of degrees offairness, randomness, and other restrictions on schedules from distributed computing literature, inachieving convergence to equilibrium points is an interesting and important research direction.
Characterizing asynchronous convergence.
We still lack characterizations of asynchronousconvergence even for simple dynamics ( e.g. , deterministic and historyless). Topological and knowledge-based approaches.
Topological [4,20,29] and knowledge-based [15]approaches have been very successful in addressing fundamental questions in distributed computing.Can these approaches shed new light on the implications of asynchrony for adaptive heuristics?
Further exploring the environments in Section 5.
We have applied our non-convergenceresult to the environments described in Section 5. These environments are of independent interestand are indeed the subject of extensive research. Hence, the further exploration of dynamics inthese settings is important.
Acknowledgements
We thank Danny Dolev, Alex Fabrikant, Idit Keidar, Jonathan Laserson, Nati Linial, YishayMansour and Yoram Moses for helpful discussions. This work was initiated partly as a result ofthe DIMACS Special Focus on Communication Security and Information Privacy. Our PSPACE-completeness result in Section 7 eliminates the possibility of short witnesses of guaranteed asyn-chronous convergence unless PSPACE ⊆ NP, but elegant characterizations are still possible. eferences [1] H. L. Abbott and M. Katchalski. On the construction of snake in the box codes. UtilitasMathematica , 40:97-116, 1991.[2] M. Ben-Or. Randomized agreement protocols. In
Fault-Tolerant Distributed Computing , pages72–83, 1986.[3] A. Blum and Y. Mansour. Learning, regret minimization, and equilibria. In
Algorithmic GameTheory, Cambridge University Press , 2007.[4] E. Borowsky and E. Gafni. Generalized FLP impossibility result for t-resilient asynchronouscomputations. In
STOC ’93: Proceedings of the twenty-fifth annual ACM symposium onTheory of computing , pages 91–100, 1993.[5] K. M. Chandy and J. Misra. How processes learn. In
PODC ’85: Proceedings of the fourthannual ACM symposium on Principles of distributed computing , pages 204–214, 1985.[6] A. Davis and S. M. Nowick. An introduction to asynchronous circuit design. Technical report,The Encyclopedia of Computer Science and Technology, 1997.[7] D. Dolev, C. Dwork, and L. Stockmeyer. On the minimal synchronism needed for distributedconsensus.
J. ACM , 34(1):77–97, 1987.[8] S. Dolev. Self stabilization.
MIT Press , 2000.[9] A. Fabrikant and C. H. Papadimitriou. The complexity of game dynamics: BGP oscillations,sink equilibria, and beyond. In
SODA ’08: Proceedings of the nineteenth annual ACM-SIAMsymposium on Discrete algorithms , pages 844–853, 2008.[10] J. Feigenbaum and S. Shenker. Distributed algorithmic mechanism design: recent results andfuture directions. In
DIALM ’02: Proceedings of the 6th international workshop on Discretealgorithms and methods for mobile computing and communications , pages 1–13, 2002.[11] F. Fich and E. Ruppert. Hundreds of impossibility results for distributed computing.
Dis-tributed Computing , 16(2–3):121–163, 2003.[12] M. J. Fischer, N. A. Lynch, and M. S. Paterson. Impossibility of distributed consensus withone faulty process.
J. ACM , 32(2):374–382, 1985.[13] P. B. Godfrey, M. Schapira, A. Zohar, and S. Shenker. Incentive compatibility and dynamicsof congestion control. In
SIGMETRICS ’10 , pages 95–106, 2010.[14] T. G. Griffin, F. B. Shepherd, and G. Wilfong. The stable paths problem and interdomainrouting.
IEEE/ACM Trans. Netw. , 10(2):232–243, 2002.[15] J. Y. Halpern and Y. Moses. Knowledge and common knowledge in a distributed environment.
JACM , 37(3):549–587, 1990.[16] S. Hart. Adaptive heuristics.
Econometrica , 73:1401–1430, 2005.1117] S. Hart and Y. Mansour. The communication complexity of uncoupled nash equilibriumprocedures. In
STOC ’07: Proceedings of the thirty-ninth annual ACM symposium on Theoryof computing , pages 345–353, 2007.[18] S. Hart and A. Mas-Colell. Uncoupled dynamics do not lead to Nash equilibrium.
AmericanEconomic Review , 93(5):1830–1836, 2003.[19] S. Hart and A. Mas-Colell. Stochastic uncoupled dynamics and Nash equilibrium.
Games andEconomic Behavior , 57(2):286–303, 2006.[20] M. Herlihy and N. Shavit. The topological structure of asynchronous computability.
J. ACM ,46(6):858–923, 1999.[21] N. Immorlica, J. Kleinberg, M. Mahdian, and T. Wexler. The role of compatibility in thediffusion of technologies through social networks. In
EC ’07: Proceedings of the 8th ACMconference on Electronic commerce , pages 75–83, 2007.[22] N. Kushman and S. Kandula and D. Katabi and B. Maggs. R-BGP: staying connected in aconnected world In
NSDI ‘07: 4th USENIX Symposium on Networked Systems Design andImplementation , 2007.[23] R. Lagunoff and A. Matsui. Asynchronous choice in repeated coordination games.
Economet-rica , 65(6):1467-1478, 1997.[24] N. Lynch. A hundred impossibility proofs for distributed computing. In
Proceedings of theEighth Annual ACM Symposium on Principles of Distributed Computing , pages 1–28, 1989.[25] D. Monderer and L. S. Shapley. Potential games.
Games and Economic Behavior , 14:124–143,1996.[26] S. Morris. Contagion.
Review of Economic Studies , 67:5778, 2000.[27] N. Nisan, M. Schapira, and A. Zohar. Asynchronous best-reply dynamics. In
WINE ’08:Proceedings of the Workshop on Internet Economics , pages 531–538, 2008.[28] R. W. Rosenthal. A class of games possessing pure-strategy Nash equilibria.
Int. J. GameTheory , 2:65-67, 1973.[29] M. E. Saks and F. Zaharoglou. Wait-free k-set agreement is impossible: The topology of publicknowledge.
SIAM J. Comput. , 29(5):1449–1483, 2000.[30] R. Sami, M. Schapira, and A. Zohar. Searching for stability in interdomain routing. In
INFOCOM , 2009.[31] G. Taubenfeld. On the nonexistence of resilient consensus protocols.
Inf. Process. Lett. ,37(5):285–289, 1991.[32] K. Varadhan, R. Govindan, and D. Estrin. Persistent route oscillations in inter-domain routing.
Computer Networks , 32(1):1–16, 2000. 1233] Y. Wang, M. Schapira, and J. Rexford. Neighbor-specific BGP: more flexible routing policieswhile improving global stability. In SIGMETRICS ’09: Proceedings of the eleventh interna-tional joint conference on Measurement and modeling of computer systems, pages 217–228,2009.[34] K. Yoon. The effective minimax value of asynchronously repeated games.
International Journalof Game Theory , 32(4):431-442, 2004.[35] A. Zapechelnyuk. Better-reply dynamics with bounded recall.
Mathematics of OperationsResearch , 33:869–879, 2008.
A Connections to Consensus Protocols
There are interesting connections between our result and that of Fischer et al. [12] for fault-resilientconsensus protocols . [12] studies the following environment: There is a group of processes , each withan initial value in { , } , that communicate with each other via messages . The objective is for all non-faulty processes to eventually agree on some value x ∈ { , } , where the “consensus” x mustmatch the initial value of some process. [12] establishes that no consensus protocol is resilient toeven a single failure. One crucial ingredient for the proof of the result in [12] is showing that thereexists some initial configuration of processes’ initial values such that, from that configuration,the resulting consensus can be both 0 and 1 (the outcome depends on the specific “schedule”realized). Our proof of Theorem 4.1 uses a valency argument—an idea introduced in the proof ofthe breakthrough non-termination result in [12] for consensus protocols.Intuitively, the risk of protocol nontermination in [12] stems from the possibility of failures; acomputational node cannot tell whether another node is silent due to a failure or is simply taking along time to react. We consider environments in which nodes/communication channels do not fail.Thus, each node is guaranteed that after “sufficiently many” time steps all other nodes will react.Observe that in such an environment reaching a consensus is easy; one pre-specified node i (the“dictator”) waits until it learns all other nodes’ inputs (this is guaranteed to happen as failures areimpossible) and then selects a value v i and informs all other nodes; then, all other nodes select v i .Unlike the results in [12], the possibility of nonconvergence in our framework stems from limitationson nodes’ behaviors. We investigate the link between consensus protocols and our framework furtherin Appendix. F, where we take an axiomatic approach. We introduce a condition—“ Independenceof Decisions ” (IoD)—that holds for both fault-resilient consensus protocols and for bounded-recallself-independent dynamics. We then factor the arguments in [12] through IoD to establish a non-termination result that holds for both contexts, thus unifying the treatment of these dynamiccomputational environments.Hence, there is no immediate translation from the result in [12] to ours (and vice versa). Toillustrate this point, let us revisit Example 4.2, in which the system is convergent, yet two stablestates exist. We observe that in the example there is indeed an initial state from which both stablestates are reachable (a “bivalent state” [12]). Consider the initial state ( y, x ). Observe that if node1 is activated first (and alone), then it shall choose action x . Once node 2 is activated it shall thenalso choose x , and the resulting stable state shall be ( x, x ). However, if node 2 is activated first(alone), then it shall choose action y . Once 1 is activated it shall also choose action y , and theresulting stable state shall be ( y, y ). Observe that in Example 4.3 too there exists an action profile( x, x ) from which multiple stable states are reachable yet the system is convergent.13 Games, Circuits, Networks, and Beyond
We present implications of our impossibility result in Section 4 for several well-studied environments:game theory, circuit design, social networks and Internet protocols.
B.1 Game Dynamics
The setting.
There are n players , 1 , . . . , n . Each player i has a strategy set S i . Let S = × j ∈ N S j ,and let S − i = × j ∈ [ n ] \{ i } S j . Each player i has a utility function u i : S → S i . For each s i ∈ S i and s − i ∈ S − i let ( s i , s − i ) denote the strategy profile in which player i ’s strategy is s i and all otherplayers’ strategies are as in s − i . Informally, a pure Nash equilibrium is a strategy profile from whichno player wishes to unilaterally deviate. Definition B.1. (pure Nash equilibria)
We say that a strategy profile s = ( s , . . . , s n ) ∈ S isa pure Nash equilibrium if, for each player i , s i ∈ argmax s i ∈ S i u i ( s i , s − i ).One natural procedure for reaching a pure Nash equilibrium of a game is best-response dynamics :the process starts at some arbitrary strategy profile, and players take turns “best replying” toother players’ strategies until no player wishes to change his strategy. Convergence of best-responsedynamics to pure Nash equilibria is the subject of extensive research in game theory and economics,and both positive [25, 28] and negative [18, 19] results are known.Traditionally, work in game theory on game dynamics ( e.g. , best-response dynamics) relies onthe explicit or implicit premise that players’ actions are somehow synchronized (in some contextsplay is sequential, while in others it is simultaneous). We consider the realistic scenario that thereis no computational center than can synchronize players’ selection of strategies. We cast the abovesetting into the terminology of Section 3 and exhibit an impossibility result for best-response, andmore general, dynamics. Computational nodes, action spaces.
The computational nodes are the n players. The actionspace of each player i is his strategy set S i . Reaction functions, dynamics.
Under best-response dynamics, each player constantly choosesa “best response” to the other players’ most recent actions. Consider the case that players haveconsistent tie-breaking rules, i.e. , the best response is always unique, and depends only on theothers’ strategies. Observe that, in this case, players’ behaviors can be formulated as deterministic,historyless, and self-independent reaction functions. The dynamic interaction between players is asin Section 3.
Existence of multiple pure Nash equilibria implies non-convergence of best-responsedynamics in asynchronous environments.
Theorem 4.1 implies the following result:
Theorem B.2.
If there are two (or more) pure Nash equilibria in a game, then asynchronousbest-response dynamics can potentially oscillate indefinitely.
In fact, Theorem 4.1 implies that the above non-convergence result holds even for the broaderclass of randomized, bounded-recall and self-independent game dynamics, and thus also to gamedynamics such as best-response with bounded recall and consistent tie-breaking rules (studiedin [35]). 14 .2 Asynchronous Circuits
The setting.
There is a Boolean circuit, represented as a directed graph G , in which verticesrepresent the circuit’s inputs and the logic gates, and edges represent connections between thecircuit’s inputs and the logic gates and between logic gates. The activation of the logic gatesis asynchronous. That is, the gates’ outputs are initialized in some arbitrary way, and then theupdate of each gate’s output, given its inputs, is uncoordinated and unsynchronized. We prove animpossibility result for this setting, which has been extensively studied (see [6]). Computational nodes, action spaces.
The computational nodes are the inputs and the logicgates. The action space of each node is { , } . Reaction functions, dynamics.
Observe that each logic gate can be regarded as a function thatonly depends on its inputs’ values. Hence, each logic gate can be modeled via a reaction function .Interaction between the different circuit components is as in Section 3.
Too much stability in circuits can lead to instability.
Stable states in this framework areassignments of Boolean values to the circuit inputs and the logic gates that are consistent witheach gate’s truth table (reaction function). We say that a Boolean circuit is inherently stable if it isguaranteed to converge to a stable state regardless of the initial boolean assignment. The followingtheorem is derived from Theorem 4.1:
Theorem B.3.
If two (or more) stable Boolean assignments exist for an asynchronous Booleancircuit, then that asynchronous circuit is not inherently stable.
B.3 Diffusion of Technologies in Social Networks
The setting.
There is a social network of users, represented by a directed graph in which users arethe vertices and edges correspond to friendship relationships. There are two competing technologies, X and Y . A user’s utility from each technology depends on the number of that user’s friends thatuse that technology; the more friends use that technology the more desirable that technology is tothe user. That is, a user would always select the technology used by the majority of his friends. Weare interested in the dynamics of the diffusion of technologies. Observe that if, initially, all usersare using X , or all users are using Y , no user has an incentive to switch to a different technology.Hence, there are always (at least) two distinct “stable states” (regardless of the topology of thesocial network). Therefore, the terminology of Section 3 can be applied to this setting. Computational nodes, actions spaces.
The users are the computational nodes . Each user i ’s action space consists of the two technologies { X, Y } . Reaction functions, dynamics.
The reaction function of each user i is defined as follows: If atleast half of i ’s friends use technology X , i selects technology X ; otherwise, i selects technology Y .In our model of diffusion of technologies, users’ choices of technology can be made simultaneously,as described in Section 3. Instability of social networks.
Theorem 4.1 implies the following:
Theorem B.4.
In every social network, the diffusion of technologies can potentially never convergeto a stable global state. .4 Interdomain Routing The setting.
The Internet is made up of smaller networks called
Autonomous Systems (ASes).
Interdomain routing is the task of establishing routes between ASes, and is handled by the
BorderGateway Protocol (BGP). In the standard model for analyzing BGP dynamics [14], there is anetwork of source
ASes that wish to send traffic to a unique destination AS d . Each AS i hasa ranking function < i that specifies i ’s strict preferences over all simple (loop-free) routes leadingfrom i to d . Under BGP, each AS constantly selects the “best” route that is available to it. See [14]for more details. Guaranteeing
BGP safety , i.e. , BGP convergence to a “stable” routing outcome isa fundamental desideratum that has been the subject of extensive work in both the networking andthe standards communities. We now cast interdomain routing into the terminology of Section 3.We then obtain non-termination results for BGP and for proposals for new interdomain routingprotocols (as corollaries of Theorem 4.1). Computational nodes, action spaces.
The ASes are the computational nodes . The action space of each node i , A i , is the set of all simple (loop-free) routes between i and the destination d thatare exportable to i , and the empty route ∅ . Reaction functions, dynamics.
The reaction function f i of node i outputs, for every vector α containing routes to d of all of i ’s neighbors, a route ( i, j ) R j such that (1) j is i ’s neighbor; (2) R j is j ’s route in α ; and (3) R j > i R for all other routes R in α . If there is no such route R j in α then f i outputs ∅ . Observe that the reaction function f i is deterministic, self-independent andhistoryless. The interaction between nodes is as described in Section 3. The multitude of stable routing trees implies global network instability.
Theorem 4.1implies a recent result of Sami et al . [30], that shows that the existence of two (or more) stablerouting trees to which BGP can (potentially) converge implies that BGP is not safe. Importantly,the asynchronous model of Section 3 is significantly more restrictive than that of [30]. Hence,Theorem 4.1 implies the non-termination result of Sami et al . Theorem B.5. [30] If there are multiple stable routing trees in a network, then BGP is not safeon that network.
Over the past few years, there have been several proposals for BGP-based multipath routing protocols, i.e. , protocols that enable each node (AS) to send traffic along multiple routes, e.g. ,R-BGP [22] and Neighbor-Specific BGP [33] (NS-BGP). Under both R-BGP and NS-BGP eachcomputational node’s actions are independent of its own past actions and are based on boundedrecall of past interaction. Thus, Theorem 4.1 implies the following:
Theorem B.6.
If there are multiple stable routing configurations in a network, then R-BGP is notsafe on that network.
Theorem B.7.
If there are multiple stable routing configurations in a network, then NS-BGP isnot safe on that network.
B.5 Congestion Control
The setting.
We now present the model of congestion control, studied in [13]. There is a networkof routers, represented by a directed graph G = ( V, E ), where | E | ≥
2, in which vertices represent ASes rankings of routes also reflect each AS’s export policy that specifies which routes that AS is willing to makeavailable to each neighboring AS. c e . There are n source-target pairs of vertices ( s i , t i ), termed “ connections ”, that represent communicating pairs of end-hosts. Each source-target pair ( s i , t i ) is connected via some fixed route, R i . Each source s i transmitsat a constant rate γ i > Routers have queue management , or queueing , policies, that dictatehow traffic traversing a router’s outgoing edge should be divided between the connections whoseroutes traverse that edge. The network is asynchronous and so routers’ queueing decisions can bemade simultaneously. See [13] for more details.
Computational nodes, action spaces
The computational nodes are the edges. The action space of each edge e intuitively consists of all possible way to divide traffic going through e betweenthe connections whose routes traverse e . More formally, for every edge e , let N ( e ) be the numberconnections whose paths go through e . e ’s action space is then A i = { x = ( x , . . . , x N ( e ) ) | x i ∈ R N ( e ) ≥ and Σ i x i ≤ c e } . Reaction functions, dynamics.
Each edge e ’s reaction function , f e , models the queueing policyaccording to which e ’s capacity is shared: for every N ( e )-tuple of nonnegative incoming flows( w , w , . . . , w N ( e ) ), f e outputs an action ( x , . . . , x N ( e ) ) ∈ A i such that ∀ i ∈ [ N ( e )] w i ≥ x i (aconnection’s flow leaving the edge cannot be bigger than that connection’s flow entering the edge).The interaction between the edges is as described in Section 3. Multiple equilibria imply potential fluctuations of connections’ throughputs. [13] showsthat, while one might expect that if sources transmit flow at a constant rate, flow will also be received at a constant rate, this is not necessarily the case. Indeed, [13] presents examples in whichconnections’ throughputs can potentially fluctuate ad infinitum. Equilibria (which correspond tostable states in Section 3), are global configurations of connections’ flows on edges such that connec-tions’ incoming and outgoing flows on each edge are consistent with the queue management policyof the router controlling that edge. Using Theorem 4.1, we can obtain the following impossibilityresult:
Theorem B.8.
If there are multiple capacity-allocation equilibria in the network then dynamics ofcongestion can potentially oscillate indefinitely. C r -Convergence and Randomness We now consider the implications for convergence of two natural restrictions on schedules: r -fairness and randomization. C.1 Snakes in Boxes and r-Convergence.
Theorem 4.1 deals with convergence and not r -convergence, and thus does not impose restrictionson the number of consecutive time steps in which a node can be nonactive. What happens if thereis an upper bound on this number, r ? We now show that if r < n − This is modeled via the addition of an edge e = ( u, s i ) to G , such that c e = γ i , and u has no incoming edges. xample C.1 . (a system that is convergent for r < n − but nonconvergent for r = n − ) There are n ≥ , . . . , n , each with the action space { x, y } . Nodes’ deterministic, historylessand self-independent reaction functions are as follows. ∀ i ∈ [ n ], f i ( x n − ) = x and f i always outputs y otherwise. Observe that there exist two stable states: x n and y n . Observe that if r = n − y and all other nodes’ actionsare x . Then, nodes 1 and 2 are activated and, consequently, node 1’s action becomes x and node 2’saction becomes y . Next, nodes 2 and 3 are activated, and thus 2’s action becomes x and 3’s actionbecomes y . Then 3 , ,
5, and so on (traversing all nodes over and over againin cyclic order). This goes on indefinitely, never reaching one of the two stable states. Observethat, indeed, each node is activated at least once within every sequence of n − r < n − y , then convergence to y n is guaranteed. Clearly, if all nodes’ action is x then convergence to x n is guaranteed. Thus, anoscillation is possible only if, in each time step, exactly one node’s action is y . Observe that, givenour definition of nodes’ reaction functions, this can only be if the activation sequence is (essentially)as described above, i.e. , exactly two nodes are activated at a time. Observe also that this kind ofactivation sequence is impossible for r < n − r > n ? We use classical results in combinatorics regarding the size of a “ snake-in-the-box ” in a hypercube [1] to show that some systems are r -convergent for exponentially-large r ’s,but are not convergent in general. Theorem 6.2 . Let n ∈ N be sufficiently large. There exists a system G with n nodes, in whicheach node i has two possible actions and each f i is deterministic, historyless and self-independent,such that1. G is r -convergent for r ∈ Ω(2 n );2. G is not ( r + 1)-convergent. Proof.
Let the action space of each of the n nodes be { x, y } . Consider the possible action profilesof nodes 3 , . . . , n , i.e. , the set { x, y } n − . Observe that this set of actions can be regarded as the( n − Q n − , and thus can be visualized as the graph whose vertices are indexed by thebinary ( n − n − Definition C.2. (chordless paths, snakes) A chordless path in a hypercube Q n is a path P =( v , . . . , v w ) such that for each v i , v j on P , if v i and v j are neighbors in Q n then v j ∈ { v i − , v i +1 } .A snake in a hypercube is a simple chordless cycle.The following result is due to Abbot and Katchalski [1]. Theorem C.3. [1] Let t ∈ N be sufficiently large. Then, the size | S | of a maximal snake in the z -hypercube Q z is at least λ × z for some λ ≥ . . Hence, the size of a maximal snake in the Q n − hypercube is Ω(2 n ). Let S be a maximal snakein { x, y } n − . W.l.o.g we can assume that x n − is on S (otherwise we can rename nodes’ actions soas to achieve this). Nodes deterministic, historyless and self-independent are as follows: • Node i ∈ { , } : f i ( x n − ) = x ; f i = y otherwise.18 Node i ∈ { , . . . , n } : if the actions of nodes 1 and 2 are both y then the action y is chosen, i.e. , f i ( yy ∗ . . . ∗ ) = y ; otherwise, f i only depends on the actions of nodes in { , . . . , n } andtherefore to describe f i it suffices to orient the edges of the hypercube Q n − (an edge fromone vertex to another vertex that differs from it in the i th coordinate determines the outcomeof f i for both). This is done as follows: orient the edges in S so as to create a cycle (in oneof two possible ways); orient edges between vertices not in S to vertices in S towards thevertices in S ; orient all other edges arbitrarily. Observation C.4. x n is the unique stable state of the system. Observation C.5.
If, at some point in time, both nodes and ’s actions are y then convergenceto the y n stable state is guaranteed. Claim C.6.
If there is an oscillation then there must be infinitely many time steps in which theactions of nodes , . . . , n are x n − .Proof. Consider the case that the statement does not hold. In that case, from some moment forth,node 1 never sees the actions x n − and so will constantly select the action y . Once that happens,node 2 shall also not see the actions x n − and will thereafter also select y . Convergence to y n isthen guaranteed.We now show that the system is convergent for r < | S | , but is nonconvergent if r = | S | . Thetheorem follows. Claim C.7. If r < | S | then convergence to the stable state y n is guaranteed.Proof. Observation C.6 establishes that in an oscillation there must be infinitely many time stepsin which the actions of nodes 2 , . . . , n are x n − . Consider one such moment in time. Observe that inthe subsequent time steps nodes’ action profiles will inevitably change as in S (given our definitionof nodes’ 3 , . . . , n reaction functions). Thus, once the action profile is no longer x n − there are atleast | S | − x n − . Observe that if 1 and 2 are activated atsome point in the intermediate time steps (which is guaranteed as r < | S | ) then the actions of bothshall be y and so convergence to y n is guaranteed. Claim C.8. If r = | S | then an oscillation is possible.Proof. The oscillation is as follows. Start at x n and activate both 1 and 2 (this will not changethe action profile). In the | S | − x n isreached again. Repeat ad infinitum.We note that the construction in the proof of Theorem 6.2 is such that there is a unique stablestate. We believe that the same ideas can be used to prove the same result for systems withmultiple stable states but the exact way of doing this eludes us at the moment, and is left as anopen question. Problem C.9.
Prove that for every sufficiently large n ∈ N , there exists a system G with n nodes, in which each node i has two possible actions and each f i is deterministic, historyless andself-independent, such that . G is r -convergent for r ∈ Ω(2 n ) ;2. G is not ( r + 1) -convergent;3. There are multiple stable states in G . C.2 Does Random Choice (of Initial State and Schedule) Help?
Theorem 4.1 tells us that a system with multiple stable states is nonconvergent if the initial stateand the node-activation schedule are chosen adversarially. Can we guarantee convergence if theinitial state and schedule are chosen at random ? Example
C.10 . (random choice of initial state and schedule might not help) There are n nodes, 1 , . . . , n , and each node has action space { x, y, z } . The (deterministic, historyless and self-independent) reaction function of each node i ∈ { , . . . , n } is such that f i ( x n − ) = x ; f i ( z n − ) = z ;and f i = y for all other inputs. The (deterministic, historyless and self-independent) reactionfunction of each node i ∈ { , } is such that f i ( x n − ) = x ; f i ( z n − ) = z ; f i ( xy n − ) = y ; f i ( y n − ) = x ; and f i = y for all other inputs. Observe that there are exactly two stable states: x n and z n .Observe also that if nodes’ actions in the initial state do not contain at least n − x ’s, or at least n − z ’s, then, from that moment forth, each activated node in the set { , . . . , n } will choose theaction y . Thus, eventually the actions of all nodes in { , . . . , n } shall be y , and so none of the twostable states will be reached. Hence, there are 3 n possible initial states, such that only from 4 n + 2can a stable state be reached.Example C.10 presents a system with multiple stable states such that from most initial states all possible choices of schedules do not result in a stable state. Hence, when choosing the initial stateuniformly at random the probability of landing on a “good” initial state (in terms of convergence)is exponentially small. D Complexity of Asynchronous Dynamics
We now explore the communication complexity and computational complexity of determiningwhether a system is convergent. We present hardness results in both models of computationeven for the case of deterministic and historyless adaptive heuristics. Our computational complex-ity result shows that even if nodes’ reaction functions can be succinctly represented, determiningwhether the system is convergent is PSPACE-complete. This intractability result, alongside itscomputational implications, implies that we cannot hope to have short “witnesses” of guaranteedasynchronous convergence (unless PSPACE ⊆ NP).
D.1 Communication Complexity
We prove the following communication complexity result, that shows that, in general, determiningwhether a system is convergent cannot be done efficiently.
Theorem D.1.
Determining if a system with n nodes, each with actions, is convergent requires Ω(2 n ) bits. This holds even if all nodes have deterministic, historyless and self-independent reactionfunctions. roof. To prove our result we present a reduction from the following well-known problem in com-munication complexity theory.2-party SET DISJOINTNESS: There are two parties, Alice and Bob. Each party holds a subset of { , . . . , q } ; Alice holds the subset E A and Bob holds the subset E B . The objective is to determinewhether E A ∩ E B = ∅ . The following is well known. Theorem D.2.
Determining whether E A ∩ E B = ∅ requires (in the worst case) the communicationof Ω( q ) bits. This lower bound applies to randomized protocols with bounded -sided error and alsoto nondeterministic protocols. We now present a reduction from 2-party SET DISJOINTNESS to the question of determiningwhether a system with deterministic, historyless and self-independent reaction functions is conver-gent. Given an instance of SET-DISJOINTNESS we construct a system with n nodes, each withtwo actions, as follows (the relation between the parameter q in SET DISJOINTNESS and thenumber of nodes n is to be specified later). Let the action space of each node be { x, y } . We nowdefine the reaction functions of the nodes. Consider the possible action profiles of nodes 3 , . . . , n , i.e. , the set { x, y } n − . Observe that this set of actions can be regarded as the ( n − Q n − , and thus can be visualized as the graph whose vertices are indexed by the binary ( n − n − Definition D.3. (chordless paths, snakes) A chordless path in a hypercube Q n is a path P =( v , . . . , v w ) such that for each v i , v j on P , if v i and v j are neighbors in Q n then v j ∈ { v i − , v i +1 } .A snake in a hypercube is a simple chordless cycle.The following result is due to Abbot and Katchalski [1]. Theorem D.4. [1] Let t ∈ N be sufficiently large. Then, the size | S | of a maximal snake in the z -hypercube Q z is at least λ × z for some λ ≥ . . Hence, the size of a maximal snake in the Q n − hypercube is Ω(2 n ). Let S be a maximal snakein { x, y } n − . We now show our reduction from SET DISJOINTNESS with q = | S | . We identifyeach element j ∈ { . . . , q } with a unique vertex v j ∈ S . W.l.o.g we can assume that x n − is on S (otherwise we can rename nodes’ actions to achieve this). For ease of exposition we also assumethat y n − is not on S (getting rid of this assumption is easy). Nodes’ reaction functions are asfollows. • Node 1: If v j = ( v j, , . . . , v j,n − ) ∈ S is a vertex that corresponds to an element j ∈ E A , then f ( y, v j, , . . . , v j,n − ) = x ; otherwise, f outputs y . • Node 2: If v j = ( v j, , . . . , v j,n − ) ∈ S is a vertex that corresponds to an element j ∈ E B , then f ( y, v j, , . . . , v j,n − ) = x ; otherwise, f outputs y . • Node i ∈ { , . . . , n } : if the actions of nodes 1 and 2 are not both x then the action y ischosen; otherwise, f i only depends on actions of nodes in { , . . . , n } and therefore to describe f i it suffices to orient the edges of the hypercube Q n − (an edge from one vertex to anothervertex that differs from it in the i th coordinate determines the outcome of f i for both). Thisis done as follows: orient the edges in S so as to create a cycle (in one of two possible ways);orient edges between vertices not in S to vertices in S towards the vertices in S ; orient allother edges arbitrarily. 21 bservation D.5. y n is the unique stable state of the system. In our reduction Alice simulates node 1 (whose reaction function is based on E A ), Bob simulatesnode 1 (whose reaction function is based on E B ), and one of the two parties simulates all othernodes (whose reaction functions are not based on neither E A nor E B ). The theorem now followsfrom the combination of the following claims: Claim D.6.
In an oscillation it must be that there are infinitely many time steps in which bothnode and ’s actions are x .Proof. By contradiction. Consider the case that from some moment forth it is never the case thatboth node 1 and 2’s actions are x . Observe that from that time onwards the nodes 3 , . . . , n willalways choose the action y . Hence, after some time has passed the actions of all nodes in { , . . . , n } will be y . Observe that whenever nodes 1 and 2 are activated thereafter they shall choose the action y and so we have convergence to the stable state y n . Claim D.7.
The system is not convergent iff E A ∩ E B = ∅ .Proof. We know (Claim D.6) that if there is an oscillation then there are infinitely many time stepsin which both node 1 and 2’s actions are x . We argue that this implies that there must be infinitelymany time steps in which both nodes select action x simultaneously . Indeed, recall that node 1only chooses action x if node 2’s action is y , and vice versa, and so if both nodes never choose x simultaneously, then it is never the case that both nodes’ actions are x at the same time step (acontradiction). Now, when is it possible for both 1 and 2 to choose x at the same time? Observethat this can only be if the actions of the nodes in { , . . . , n } constitute an element that is in both E A and E B . Hence, E A ∩ E B = ∅ . D.2 Computational Complexity
The above communication complexity hardness result required the representation of the reactionfunctions to (potentially) be exponentially long. What if the reaction functions can be succinctlydescribed? We now present a strong computational complexity hardness result for the case thateach reaction function f i is deterministic and historyless, and is given explicitly in the form of aboolean circuit (for each a ∈ A the circuit outputs f i ( a )). Theorem 7.2 . Determining if a system with n nodes, each with a deterministic and historylessreaction function, is convergent is PSPACE-complete. Proof.
Our proof is based on the proof of Fabrikant and Papadimitriou [9] that BGP safety isPSPACE-complete. Importantly, the result in [9] does not imply Theorem 7.2 since [9] only consid-ers dynamics in which nodes are activated one at a time. We present a reduction from the followingproblem.STRING NONTERMINATION: The input is a function g : Γ t → Γ ∪ { halt } , for some alphabet Γ,given in the form of a boolean circuit. The objective is to determine whether there exists an initialstring T = ( T , . . . , T t − ) ∈ Γ t such that the following procedure does not halt.1. i :=0 22. While g ( T ) = halt do • T i := g ( T ) • i := ( i + 1) modulu t STRING NONTERMINATION is closely related to STRING HALTING from [9] and is alsoPSPACE-complete. We now present a reduction from STRING NONTERMINATION to thequestion of determining whether a system with deterministic and historyless reaction functionsis convergent.We construct a system with n = t + 1 nodes. The node set is divided into t “ index nodes ”0 , . . . , t − counter node ” x . The action space of each index node is Γ ∪ { halt } andthe action space of the counter node is { , . . . , t − } × (Γ ∪ { halt } ). Let a = ( a , . . . , a t − , a x ) bean action profile of the nodes, where a x = ( j, γ ) is the action of the counter node. We now definethe deterministic and historyless reaction functions of the nodes: • The reaction function of index node i ∈ { , . . . , t − } , f i : if γ = halt , then f i ( a ) = halt ;otherwise, if j = i , and a j = γ , then f i ( a ) = γ ; otherwise, f i ( a ) = a i . • The reaction function of the counter node, f x : if γ = halt , then f x ( a ) = a x ; if a j = γ , then f i ( a ) = (( j + 1) modulu t, g ( a , . . . , a t − ); otherwise f i ( a ) = a x .The theorem now follows from the following claims that, in turn, follow from our construction: Claim D.8. ( halt, . . . , halt ) is the unique stable state of the system.Proof. Observe that ( halt, . . . , halt ) is indeed a stable state of the system. The uniqueness of thisstable state is proven via a simple case-by-case analysis.
Claim D.9.
If there exists an initial string T = ( T , . . . , T t − ) for which the procedure does notterminate then there exists an initial state from which the system does not converge to the stablestate ( halt, . . . , halt ) regardless of the schedule chosen.Proof. Consider the evolution of the system from the initial state in which the action of index node i is T i and the action of the counter node is (0 , g ( T )). Claim D.10.
If there does not exist an initial string T for which the procedure does not terminatethen the system is convergent.Proof.
Observe that if there is an initial state a = ( a , . . . , a t − , a x ) and a fair schedule for whichthe system does not converge to the unique stable state then the procedure does not halt for theinitial string T = ( a , . . . , a t − ).Proving the above PSPACE-completeness result for the case self-independent reaction functionsseems challenging. Problem D.11.
Prove that determining if a system with n nodes, each with a deterministic self-independent and historyless reaction function, is convergent is PSPACE-complete. Some Basic Observations Regarding No-Regret Dynamics
Regret minimization is fundamental to learning theory. The basic setting is as follows. There is aspace of m actions ( e.g. , possible routes to work), which we identify with the set [ m ] = { , . . . , m } .In each time step t ∈ { , . . . } , an adversary selects a profit function p t : [ m ] → [0 ,
1] ( e.g. , howfast traffic is flowing along each route), and the (randomized) algorithm chooses a distribution D t over the elements in [ m ]. When choosing D t the algorithm can only base its decision on the profitfunctions p , . . . , p t − , and not on p t (that is revealed only after the algorithm makes its decision).The algorithm’s gain at time t is g t = Σ j ∈ [ m ] D t ( j ) p t ( j ), and its accumulated gain at time t isΣ ti =1 g t . Regret analysis is useful for designing adaptive algorithms that fair well in such uncertainenvironments. The motivation behind regret analysis is ensuring that, over time, the algorithmperforms at least as well in retrospect as some alternative “simple” algorithm.We now informally present the three main notions of regret (see [3] for a thorough explanation):(1) External regret compares the algorithm’s performance to that of simple algorithms that selectthe exact same action in each time step ( e.g. , “you should have always taken Broadway, andnever chosen other routes”). (2)
Internal regret and swap regret analysis compares the gain fromthe sequence of actions actually chosen to that derived from replacing every occurrence of anaction i with another action j ( e.g. , “every time you chose Broadway you should have taken7th Avenue instead). While internal regret analysis allows only one action to be replaced byanother, swap regret analysis considers all mappings from [ m ] to [ m ]. The algorithm has no(external/internal/swap) regret if the gap between the algorithm’s gain and the gain from the bestalternative policy allowed vanishes with time.Regret minimization has strong connections to game-theoretic solution concepts. If each playerin a repeated game executes a no-regret algorithm when selecting strategies, then convergence toan equilibrium is guaranteed in a variety of interesting contexts. The notion of convergence, andthe kind of equilibrium reached, vary, and are dependent on the restrictions imposed on the gameand on the type of regret being minimized ( e.g. , in zero-sum games, no-external-regret algorithmsare guaranteed to approach or exceed the minimax value of the game; in general games, if allplayers minimize swap regret, then the empirical distribution of joint players’ actions converges toa correlated equilibrium, etc. ). (See [3] and references therein). Importantly, these results are allproven within a model of interaction in which each player selects a strategy in each and every timestep.We make the following simple observation. Consider a model in which the adversary not onlychooses the profit functions but also has the power not to allow the algorithm to select a newdistribution over actions in some time steps. That is, the adversary also selects a schedule σ suchthat ∀ t ∈ N + , σ ( t ) ∈ { , } , where 0 and 1 indicate whether the algorithm is not activated, oractivated, respectively. We restrict the schedule to be r -fair, in the sense that the schedule chosenmust be such that the algorithm is activated at least once in every r consecutive time steps. Ifthe algorithm is activated at time t and not activated again until time t + β then it holds that ∀ s ∈ { t + 1 , . . . , t + β − } , D s = D t (the algorithm cannot change its probability distribution overactions while not activated). We observe that if an algorithm has no regret in the above setting (forall three notions of regret), then it has no regret in this setting as well. To see this, simply observethat if we regard each batch of time steps in which the algorithms is not activated as one “metatime step”, then this new setting is equivalent to the traditional setting (with p t : [ m ] → [0 , r ] forall t ∈ N + ).This observation, while simple, is not uninteresting, as it implies that all regret-based results for24epeated games continue to hold even if players’ order of activation is asynchronous (see Section 3for a formal exposition of asynchronous interaction), so long as the schedule of player activationsis r -fair for some r ∈ N + . We mention two implications of this observation. Observation E.1.
When all players in a zero-sum game use no-external-regret algorithms thenapproaching or exceeding the minimax value of the game is guaranteed.
Observation E.2.
When all players in a (general) game use no-swap-regret algorithms the empir-ical distribution of joint players’ actions converges to a correlated equilibrium of the game.
Problem E.3.
Give examples of repeated games for which there exists a schedule of player acti-vations that is not r -fair for any r ∈ N + for which regret-minimizing dynamics do not converge toan equilibrium (for different notions of regret/convergence/equilibria). Problem E.4.
When is convergence of no-regret dynamics to an equilibrium guaranteed (for dif-ferent notions of regret/convergence/equilibria) for all r -fair schedules for non-fixed r ’s, that is, ifwhen r is a function of t ? F An Axiomatic Approach
We now use (a slight variation of) the framework of Taubenfeld, which he used to study resilientconsensus protocols [31], to prove Thm. 4.1. We first (Sec. F.2) define runs as sequences of events ;unlike Taubenfeld, we allow infinite runs. A protocol is then a collection of runs (which must satisfysome natural conditions like closure under taking prefixes). We then define colorings of runs (whichcorrespond to outcomes that can be reached by extending a run in various ways) and define the
IoD property.The proof of Thm. 4.1 proceeds in two steps. First, we show that any protocol that satisfies
IoD has some (fair, as formalized below), non-terminating activation sequence. We then show thatprotocols that satisfy the hypotheses of Thm. 4.1 also satisfy
IoD . F.1 Proof Sketch
Proof Sketch.
The proof follows the axiomatic approach of Taubenfeld [31] in defining asynchronousprotocols in which states are colored by sets of colors; the set of colors assigned to a state mustbe a superset of the set of colors assigned to any state that is reachable (in the protocol) fromit. We then show that any such protocol that satisfies a certain pair of properties (which we call
Independence of Decisions or IoD ) and that has a polychromatic state must have a non-terminatingfair run in which all states are polychromatic.For protocols with 1-recall, self-independence, and stationarity, we consider (in order to reacha contradiction) protocols that are guaranteed to converge. Each starting state is thus guaranteedto reach only stable states; we then color each state according to the outcomes that are reachablefrom that state. We show that, under this coloring, such protocols satisfy IoD and that, as inconsensus protocols, the existence of multiple stable states implies the existence of a polychromaticstate. The non-terminating, polychromatic, fair run that is guaranteed to exist is, in the context,exactly the non-convergent protocol run claimed by the theorem statement. We then show thatthis may be extended to non-stationary protocols with k -recall (for k > .2 Events, Runs, and Protocols Events are the atomic actions that are used to build runs of a protocol. Each event is associatedwith one or more principals; these should be thought of as the principals who might be affectedby the event ( e.g. , as sender or receiver of a message), with the other principals unable to see theevent. We start with the following definition.
Definition F.1 (Events and runs) . There is a set E whose elements are called events ; we assumea finite set of possible events (although there will be no restrictions on how often any event mayoccur). There is a set P of principals ; each event has an associated set S ⊆ P , and if S is the setassociated to e ∈ E , we will write e S .There is a set R whose elements are called runs ; each run is a (possibly infinite) sequence ofevents. We say that event e is enabled at run x if the concatenation h x ; e i ( i.e. , the sequence ofevents that is x followed by the single event e ) is also a run. (We will require that R be prefix-closedin the protocols we consider below.)The definition of a protocol will also make use of a couple types of relationship between runs;our intuition for these relationships continues to view e P as meaning that event e affects the set P of principals. From this intuitive perspective, two runs are equivalent with respect to a set S ofprincipals exactly when their respective subsequences that affect the principals in S are identical.We also say that one run includes another whenever, from the perspective of every principal ( i.e. ,restricting to the events that affect that principal), the included run is a prefix of the includingrun. Note that this does not mean that the sequence of events in the included run is a prefix ofthe sequence of events in the including run—events that affect disjoint sets of principals can bereordered without affecting the inclusion relationship. Definition F.2 (Run equivalence and inclusion) . For a run x and S ⊆ P , we let x S denote thesubsequence (preserving order and multiplicity) of events e P in x for which P ∩ S = ∅ . We say that x and y are equivalent with respect to S , and we write x [ S ] y , if x S = y S . We say that y includes x if for every node i , the restriction of x to those events e P with i ∈ P is a prefix of the restrictionof y to such events.Our definitions of x S and x [ S ] y generalize definitions given by Taubenfeld [31] for | S | = 1—allowing us to consider events that are seen by multiple principals—but other than this and theallowance of infinite runs, the definitions we use in this section are the ones he used. Importantly,however, we do not use the resilience property that Taubenfeld used.Finally, we have the formal definition of an asynchronous protocol. This is a collection of runsthat is closed under taking prefixes and only allows for finitely many (possibly 0) choices of a nextevent to extend the run. It also satisfies the property ( P below) that, if a run can be extended byan event that affects exactly the set S of principals, then any run that includes this run and that isequivalent to the first run with respect to S (so that only principals not in S see events that theydon’t see in the first run) can also be extended by the same event. Definition F.3 (Asynchronous protocol) . An asynchronous protocol (or just a protocol ) is a col-lection of runs that satisfies the following three conditions. P Every prefix of a run is a run. P Let h x ; e S i and y be runs. If y includes x , and if x [ S ] y , then h y ; e S i is also a run. P Only finitely many events are enabled at a run.26 .3 Fairness, Coloring, and Decisions
We start by recalling the definition of a fair sequence [31]; as usual, we are concerned with thebehavior of fair runs. We also introduce the notion of a fair extension, which we will use toconstruct fair infinite runs.
Definition F.4 (Fair sequence, fair extension) . We define a fair sequence to be a sequence ofevents such that: every finite prefix of the sequence is a run; and, if the sequence is finite, then noevent is enabled at the sequence, while if the sequence is infinite, then every event that is enabledat all but finitely many prefixes of the sequence appears infinitely often in the sequence. We definea fair extension of a (not necessarily fair) sequence x to be a finite sequence e , e , . . . , e k of eventssuch that e is enabled at x , e is enabled at h x ; e i , etc .We also assign a set of “colors” to each sequence of events subject to the conditions below.As usual, the colors assigned to a sequence will correspond to the possible protocol outcomes thatmight be reached by extending the sequence. Definition F.5 (Asynchronous, C -chromatic protocol) . Given a set C (called the set of colors ),we will assign sets of colors to sequences; this assignment may be a partial function. For a set C ,we will say that a protocol is C -chromatic if it satisfies the following properties. C For each c ∈ C , there is a protocol run of color { c } . C For each protocol run x of color C ′ ⊆ C , and for each c ∈ C ′ , there is an extension of x thathas color { c } . C If y includes x and x has color C ′ , then the color of y is a subset of C ′ .We say that a fair sequence is polychromatic if the set of colors assigned to it has more than oneelement.Finally, a C -chromatic protocol is called a decision protocol if it also satisfies the followingproperty: D Every fair sequence has a finite monochromatic prefix, i.e. , a prefix whose color is { c } for some c ∈ C . F.4 Independence of Decisions (
IoD ) We turn now to the key (two-part) condition that we use to prove our impossibility results.
Definition F.6 (Independence of Decisions (
IoD )) . A protocol satisfies
Independence of Decisions ( IoD ) if, whenever • a run x is polychromatic and • there is some event e is enabled at x and h x ; e i is monochromatic of color { c } ,then1. for every e ′ = e that is enabled at x , the color of h x ; e ′ i contains c , and2. for every e ′ = e that is enabled at x , if hh x ; e ′ i ; e i is monochromatic, then its color is also { c } .Figure 1 illustrates the two conditions that form IoD . Both parts of the figure include thepolychromatic run x that can be extended to h x ; e i with monochromatic color { c } ; the color of x necessarily includes c . The left part of the figure illustrates condition 1, and the right part ofthe figure illustrates condition 2. The dashed arrow indicates a sequence of possibly many events,27 e’ x {c} {c,...}{...} e’ d=c {d}{c} {...} {...}e e x Figure 1: Illustration of the two conditions of
IoD .while the solid arrows indicate single events. The labels on a node in the figure indicate what isassumed/required about the set that colors the node.Condition 1 essentially says that, if an event e decides the outcome of the protocol, then noother event can rule out the outcome that e produced. The name “Independence of Decisions”derives from condition 2, which essentially says that, if event e decides the outcome of the protocolboth before and after event e ′ , then the decision that is made is independent of whether e ′ happensimmediately before or after e .In working with IoD -satisfying protocols, the following lemma will be useful.
Lemma F.7. If IoD holds, then for any two events e and e ′ that are enabled at a run x , if both h x ; e i and h x ; e ′ i are monochromatic, then those colors are the same.Proof. By IoD , the color of h x ; e ′ i must contain the color of h x ; e i , and both of these sets aresingletons. F.5
IoD -Satisfying Protocols Don’t Always Converge
To show that
IoD -satisfying protocols don’t always converge, we proceed in two steps: first, weshow (Lemma F.8) that a polychromatic sequence can be fairly extended (in the sense of . . . ) toanother polychromatic sequence; second, we use that lemma to show (Thm. F.9) . . . .
Lemma F.8 (The Fair-Extension Lemma) . In a polychromatic decision protocol that satisfies
IoD ,if a run x is polychromatic, then x can be extended by a fair extension to another polychromaticrun.Proof. Assume that, for some C ′ , there is a run x of color C ′ that cannot be fairly extended toanother polychromatic run. Because | C ′ | >
1, there must be some event that is enabled at x ; ifnot, we would contradict D . Figure 2 illustrates this (and the arguments in the rest of the proofbelow).Consider the extensions of x that use as many distinct events as possible and that are poly-chromatic, and pick one of these y that minimizes the number of events that are enabled at everyprefix of y (after x has already been executed) but that do not appear in y . If y contains no events(illustrated in the top left of Fig. 2), then every event e that is enabled at x is such that h x ; e i is monochromatic. By Lemma F.7, these singletons must all be the same color { c } ; however, this28 ’ x {c} e {c}C’={c,d,...}C’ z {c,...} {c,d,...} {d}C’={c,d,...}C’
Any
IoD -satisfying asynchronous protocol with a polychromatic initial state has afair sequence that starts at this initial state and never reaches a decision, i.e. , it has a fair sequence hat does not have a monochromatic prefix.Proof. Start with the empty (polychromatic) run and iteratively apply the fair-extension lemma toobtain an infinite polychromatic sequence. If an event e is enabled at all but finitely many prefixesin this sequence, then in all but finitely many of the fair extensions, e is enabled at every step ofthe extension. Because these extensions are fair (in the sense of Def. F.4), e is activated in eachof these (infinitely many) extensions and so appears infinitely often in the sequence, which is thusfair. F.6 -Recall, Stationary, Self-Independent Protocols Need Not Converge We first recall the statement of Thm. 4.1. We then show that 1-recall, historyless protocols satisfy
IoD when colored as in Def. F.10. Theorem F.9 then implies that such protocols do not alwaysconverge; it immediately follows that this also applies to bounded-recall (and not just 1-recall)protocols.
Theorem 4.1.
If each node i has bounded recall, and each reaction function f i is self-independentand stationary, then the existence of two stable states implies that the computational network is notsafe. Definition F.10 (Stable coloring) . In a protocol defined as in Sec. 3, the stable coloring of protocolstates is the coloring that has a distinct color for each stable state and that colors each state in arun with the set of colors corresponding to the stable states that are reachable from that state.We model the dynamics of a 1-recall, historyless protocol as follows. There are two types ofactions: the application of nodes’ reaction functions, where e i is the action of node i acting asdictated by f i , and a “reveal” action W . The nodes scheduled to react in the first timestep do sosequentially, but these actions are not yet visible to the other nodes (so that nodes after the firstone in the sequence are still reacting to the initial state and not to the actions performed earlier inthe sequence). Once all the scheduled nodes have reacted, the W action is performed; this revealsthe newly performed actions to all the other nodes in the network. The nodes that are scheduledto react at the next timestep then act in sequence, followed by another W action, and so on. Thisconverts the simultaneous-action model of Sec. 3 to one in which actions are performed sequentially;we will use this “act-and-tell” model in the rest of the proof. We note that all actions are enabledat every step (so that, e.g. , e i can be taken multiple times between W actions; however, this isindistinguishable from a single e i action because the extra occurrences are not seen by other nodes,and they do not affect i ’s actions, which are governed by a historyless reaction function).Once we cast the dynamics of 1-recall, historyless protocols in the act-and-tell model, thefollowing lemma will be useful. Lemma F.11 (Color equalities) . In a -recall, historyless protocol (in the act-and-tell model):1. For every run pair of runs x , y and every i ∈ [ n ] , the color of hh x ; e i W e i W i ; y i is the sameas the color of hh x ; W e i W i ; y i .2. For every run pair of runs x , y and every i, j ∈ [ n ] , the color of hh x ; e i e j i ; y i is the same asthe color of hh x ; e j e i i ; y i .Informally, the first color equality says that, if all updates are announced and then i activates andthen all updates are revealed again ( i ’s new output being the only new one), it makes no difference hether or not i was activated immediately before the first reveal action. The second color equalitysays that, as long as there are no intervening reveal event, the order in which nodes compute theiroutputs does not matter (because they do not have access to their neighbors’ new outputs until thereveal event).Proof. For the first color equality, because the protocol is self-independent, the first occurrence of e i (after x ) in hh x ; e i W e i W i ; y i does not affect the second occurrence of e i . Because the protocolhas 1-recall, the later events (in y ) are also unaffected.The second color equality is immediate from the definition of the act-and-tell model. Lemma F.12.
If a protocol is -recall and historyless, then the protocol (with the stable coloring)satisfies IoD .Proof.
Color each state in the protocol’s runs according to the stable states that can be reachedfrom it. Assume x is a polychromatic run (with color C ′ ) and that some event e is such that h x ; e i is monochromatic (with color { c } ). Let e ′ be another event (recall that all events are alwaysenabled). If e and e ′ are two distinct node events e i and e j ( i = j ), respectively, then the color of hh x ; e j i ; e i i is the color of hh x ; e i i ; e j i and thus the (monochromatic) color of h x ; e i i , i.e. , { c } . If e and e ′ are both W or are the same node event e i , then the claim is trivial.If e = e i and e ′ = W (as illustrated in the left of Fig. 3), then we may extend h x ; e i i by W e i W to obtain a run whose color is again { c } . By the second color equality, this is also the color of theextension of h x ; W i by e i W , so the color of h x ; W i contains c and if the extension of h x ; W i by e i is monochromatic, its color must be { c } as well. If, on the other hand, e = W and e ′ = e i (asillustrated in the right of Fig. 3), we may extend h x ; W i by e i W and h x ; e i i by W e i W to obtainruns of color { c } ; so the color of h x ; e i i must contain c and, arguing as before, if the intermediateextension hh x ; e i i ; W i is monochromatic, its color must also be { c } . e C’C’={c,d,...}{c} x {c,...}We{c} {c} {c,...}WWW e {c} x {c} WWW{c}{c} e’e’ {c,...}{c,...}{c,...}e’ e’ W C’={c,d,...}C’ Figure 3: Illustrations of the arguments in the proof of Lem. F.12.31 emma F.13.
If a -recall, historyless computation that always converges can, for different startingstates, converge to different stable states then there is some input from which the computation canreach multiple stable states. In particular, under the stable coloring, there is a polychromatic state.Proof. Assume there are (under the stable coloring) two different monochromatic input states forthe computation, that the inputs differ only at one node v , and that the computation alwaysconverges ( i.e. , for every fair schedule) on both input states. Consider a fair schedule that activates v first and then proceeds arbitrarily. Because the inputs to v ’s reaction function are the same ineach case, after the first step in each computation, the resulting two networks have the same nodestates. This means that the computations will subsequently unfold in the same way, in particularproducing identical outputs.If a historyless computation that always converges can produce two different outputs, theniterated application of the above argument leads to a contradiction unless there is a polychromaticinitial state. Proof of -recall, stationary part of Thm. 4.1. Consider a protocol with 1-recall, self independence,and stationarity, and that has two different stable states. If there is some non-convergent run ofthe protocol, then the network is not safe (as claimed). Now assume that all runs converge; wewill show that this leads to a contradiction. Color all states in the protocol’s runs according to thestable coloring (Def. F.10). Lemma F.13 implies that there is a polychromatic state. Because, byLem. F.12, the
IoD is satisfied, we may apply Thm. F.9. In this context (with the stable coloring),this implies that there is an infinite run in which every state can reach at least two stable states;in particular, the run does not converge.
F.7 Extension to Non-stationary Protocols
We may extend our results to non-stationary protocols as well.
Theorem F.14.
If each node i has -recall, the action spaces are all finite, and each reactionfunction f i is self-independent but not necessarily stationary, then the existence of two stable statesimplies that the computational network is not safe.Proof. In this context, a stable state is a vector of actions and a time t such that, after t , theaction vector is a fixed point of the reaction functions. Let T be the largest such t over all the(finitely many) stable states (and ensure that T is at least k for generalizing to k -recall). Assumethat the protocol is in fact safe; this means that, under the stable coloring, every state gets at leastone color. If there are only monochromatic states, consider the states at time T ; we view two ofthese states as adjacent if they differ only in the action (or action history for the generalizationto k -recall) of one node. Because the protocol is self-independent, that node may be activated ( k times if necessary) to produce the same state. In particular, this means that adjacent states musthave the same monochromatic color. Because (among he states at time T ) there is a path (followingstate adjacencies) from any one state to any other, only one stable state is possible, contradictingthe hypotheses of the theorem.Considering the proof of Lem. F.12, we see that the number of timesteps required to traverseeach of the subfigures in Fig. 3 does not depend on which path (left or right) through the subfigurewe take. In particular, this means that the reaction functions are not affected by the choice ofpath. Furthermore, the non- W actions in each subfigure only involve a single node i ; the final32ction performed by i along each path occurs after one W action has been performed (after x ), sothese final actions are the same (because the timesteps at which they occur are the same, as arethe actions of all the other nodes in the network). F.8 Extension to Bounded-Recall Protocols
If we allow k -recall for k >
1, we must make a few straightforward adjustments to the proofs above.Generalizing the argument used in the proof of the color equalities (Lem. F.11), we may prove ananalogue of these for k -recall; in particular, we replace the first color equality by an equality betweenthe colors of (cid:10)(cid:10) x ; e i W ( e i W ) k (cid:11) ; y (cid:11) and (cid:10)(cid:10) x ; W ( e i W ) k (cid:11) ; y (cid:11) . This leads to the analogue of Lem. F.12for bounded-recall protocols; as in Lem. F.12, the two possible paths through each subfigure (inthe k -recall analogue of Fig. 3) require the same number of timesteps, so non-stationarity is not aproblem.Considering adjacent states as those that differ only in the actions of one node (at some pointin its depth- k history), we may construct a path from any monochromatic initial state to any othersuch state. Because the one node that differs between two adjacent states may be (fairly) activated k times to start the computation, two monochromatic adjacent states must have the same color; asin the 1-recall case, the existence of two stable states thus implies the existence of a polychromaticstate. G Implications for Resilient Decision Protocols
The consensus problem is fundamental to distributed computing research. We give a brief descrip-tion of it here, and we refer the reader to [31] for a detailed explanation of the model. We thenshow how to apply our general result to this setting. This allows us to show that the impossibilityresult in [12], which shows that no there is no protocol that solves the consensus problem, can beobtained as a corollary of Thm. F.9.
G.1 The Consensus Problem
Processes and consensus.
There are N ≥ processes , . . . , N , each process i with an initialvalue x i ∈ { , } . The processes communicate with each other via messages . The objective is forall non-faulty processes to eventually agree on some value x ∈ { , } , such that x = x i for some i ∈ [ N ] (that is, the value that has been decided must match the initial value of some process). Nocomputational limitations whatsoever are imposed on the processes. The difficulty in reaching anagreement (consensus) lies elsewhere: the network is asynchronous, and so there is no upper boundon the length of time processes may take to receive, process and respond to an incoming message.Intuitively, it is therefore impossible to tell whether a process has failed, or is simply taking a longtime. Messages and the message buffer.
Messages are pairs of the form ( p, m ), where p is theprocess the message is intended for, and m is the contents of the message. Messages are stored inan abstract data structure called the message buffer . The message buffer is a multiset of messages, i.e. , more than one of any pair ( p, m ) is allowed, and supports two operations: (1) send(p,m) : placesa message in the message buffer. (2) receive(p) : returns a message for processor p (and removes it33rom the message buffer) or the special value, that has no effects. If there are several messages for p in the message buffer then receive(p) returns one of them at random. Configurations and system evolution.
A configuration is defined by the following two factors:(1) the internal state of all of the processors (the current step in the protocol that they are executing,the contents of their memory), and (2) the contents of the message buffer. The system moves fromone configuration to the next by a step which consists of a process p performing receive(p) andmoving to another internal state. Therefore, the only way that the system state may evolve isby some processor receiving a message (or null) from the message buffer. Each step is thereforeuniquely defined by the message that is received (possibly) and the process that received it. Executions and failures.
From any initial starting state of the system, defined by the initialvalues of the processes, there are many different possible ways for the system to evolve (as the receive(p) operation is non-deterministic). We say that a protocol solves consensus if the objectiveis achieved for every possible execution. Processes are allowed to fail according to the fail-stopmodel, that is, processes that fail do so by ceasing to work correctly. Hence, in each execution,non-faulty processes participate in infinitely many steps (presumably eventually just receiving oncethe algorithm has finished its work), while processes that stop participating in an execution atsome point are considered faulty. We are concerned with the handling of (at most) a single faultyprocess. Hence, an execution is admissible if at most one process is faulty.
G.2 Impossibility of Resilient Consensus
We now show how this fits into the formal framework of Ap. F. The events are (as in [12]) messagesannotated with the intended recipient ( e.g. , m i ). In addition to the axioms of Ap. F, we also assumethat the protocol satisfies the following resiliency property, which we adapt from Taubenfeld [31];we call such a protocol a resilient consensus protocol . (Intuitively, this property ensures that ifnode i fails, the other nodes will still reach a decision.) Res
For each run x and node i , there is a monochromatic run y that extends x such that x [ i ] y .We show that resilient consensus protocols satisfy IoD . Unsurprisingly, the proof draws on ideasof Fischer, Lynch, and Paterson.
Lemma G.1.
Resilient consensus protocols satisfy
IoD .Proof.
Assume x is a polychromatic run of a resilient consensus protocol and that h x ; m i i ismonochromatic (of color { c } ). If e ′ = m ′ j for j = i , then e = m i and e ′ commute (becausethe messages are processed by different nodes) and the IoD conditions are satisfied. (In particular, hh x ; e i ; e ′ i and hh x ; e ′ i ; e i both have the same monochromatic color.)If e ′ = m ′ i , then consider a sequence σ from x that reaches a monochromatic run and that doesnot involve i (the existence of σ is guaranteed by Res ); this is illustrated in Fig. 4. Because σ doesn’t involve i , it must commute with e and e ′ ; in particular, the color of the monochromaticrun reachable by applying σ to h x ; e i is the same as the color of the run hh x ; σ i ; e i . Thus σ mustproduce the same color { c } that e does in extending x . On the other hand, we may apply this sameargument to e ′ to see that hh x ; e ′ i ; σ i must also have the same color as h x ; σ i , so the color of h x ; e ′ i contains the color of h x ; e i . The remaining question is whether hh x ; e ′ i ; e i can be monochromaticof a different color than h x ; e i . However, the color (if it is monochromatic) of hhh x ; e ′ i ; e i ; σ i mustbe the same (because σ does not involve i ) as the color of hhh x ; e ′ i ; σ i ; e i , which we have alreadyestablished is the color of h x ; e i ; thus, hh x ; e ′ i ; e i cannot be monochromatic of a different color.34 C’C’={c,d,...}{c} σσ σ σ ee x e {c} {c} {c}{c,...}{c,...}e’e’{c} Figure 4: Illustration of argument in the proof of Lem. G.1.Using Thm. F.9 and the fact that there must be a polychromatic initial configuration for theprotocol (because it can reach multiple outcomes, as shown in [12]), we obtain from this lemma thefollowing celebrated result of Fischer, Lynch, and Paterson [12].