Playing The Hypothesis Testing Minority Game In The Maximal Reduced Strategy Space
aa r X i v : . [ phy s i c s . s o c - ph ] J un Playing The Hypothesis Testing Minority Game In The Maximal Reduced StrategySpace
H. F. Chau, V. H. Chan and F. K. Chow
Department of Physics and Center of Theoretical and Computational Physics,University of Hong Kong, Pokfulam Road, Hong Kong
Hypothesis Testing Minority Game (HMG) is a variant of the standard Minority Game (MG)that models the inertial behavior of agents in the market. In the earlier study of our group, we findthat agents cooperate better in HMG than in the standard MG when strategies are picked fromthe full strategy space. Here we continue to study the behavior of HMG when strategies are chosenfrom the maximal reduced strategy space. Surprisingly, we find that, unlike the standard MG, thelevel of cooperation in HMG depends strongly on the strategy space used. In addition, a novelintermittency dynamics is also observed in the minority choice time series in a certain parameterrange in which the orderly phases are characterized by a variety of periodic dynamics. Remarkably,all these findings can be explained by the crowd-anticrowd theory.
PACS numbers: 89.65.Gh, 89.75.-k, 05.40.-aKeywords: Crowd-Anticrowd Theory, Global Cooperation, Hypothesis Testing, Minority Game, PeriodicDynamics
I. INTRODUCTION
Studying economic systems by agents-based modelshave attracted the attention among physicists in recentyears [1, 2, 3]. One of the most famous agents-basedmodel in this regard is the Minority Game (MG) [3, 4].MG does not only capture the fact that all people in themarket think inductively and selfishly [5], its complex-ity also satisfy the definition of a complex system in thestrictest sense [6]. In spite of its simple governing rules,agents in this model self-organize giving rise to an unex-pected global cooperative phenomena.Using the standard MG as blue print, various modifi-cations to the rules of the standard MG have been pro-posed [7, 8, 9, 10] to understand different aspects andphenomena in realistic economic systems. In particu-lar, Man and Chau introduced the Hypothesis TestingMinority Game (HMG) to model the inertial behaviorof agents [11]. They found that the presence of inertialagents improve global cooperation leading to a decreaseof the variance per agent over the entire parameter spaceprovided that the strategies of each agent are chosen fromthe so-called full strategy space (FSS) [11, 12].In this paper, we move on to study the agent coopera-tion and the dynamics of HMG in case the strategies arepicked from the so-called maximal reduced strategy space(MRSS) [13]. We begin by briefly reviewing the rules ofHMG and stating the parameters used in our numericalsimulations in Sec. II. Then we report our simulationresults in Sec. III. To our surprise, the behavior of HMGdepends strongly on the strategy spaces used. Specifi-cally, agents generally cooperate better when strategiesare picked from the FSS rather than the MRSS providedthat they are sufficiently reluctant to change their strate-gies. In contrast, the standard MG is so robust that itsdynamics and cooperative behavior are essentially thesame irrespective of whether the FSS or the MRSS isused. Furthermore, we find that in HMG the minority choice time series exhibits intermittency in which the or-derly phases show periodic dynamics with period up to2(2 M −
1) whenever the memory size of the strategies M is greater than 1 in a certain parameter range whenstrategies are picked from the MRSS. This novel inter-mittent phenomenon does not show up in HMG providedthat strategies are picked from the FSS as well as in thestandard MG. We explain how these differences originatefrom the choice of the strategy space by a semi-analyticalapproach known as the crowd-anticrowd theory [14, 15]in Sec. IV. In fact, the major reason responsible for thesedifferences is that it is a lot easier for an agent to keep onusing one’s currently adopted strategy when the strategypool is the MRSS than rather than FSS in certain pa-rameter regime. Finally, we summarize our findings inSec. V. Our findings show that extra care is needed tostudy variants of MG as their behavior may depend sen-sitively on the strategy space employed. Nonetheless, theability to explain the behavior of HMG using the crowd-anticrowd theory suggests that this theory may still beuseful to explain the dynamics of variants of the standardMG provided that one carries out the analysis carefully. II. HYPOTHESIS TESTING MINORITY GAME
Recall that in the standard MG, agents act accordingto the predictions of their best performing strategies. Inother words, agents in the standard MG do not hesitateto stop using their current strategies once the perfor-mance indicator, known as virtual score, shows that thestrategies are not the best. In contrast, the HMG incor-porated the inertial behavior of agents by allowing themto stick to their currently using strategies until their per-formances are too poor to be acceptable. More precisely,a fixed real number I k between 0.5 and 1.0 is assignedonce and for all to each agent k in HMG to representtheir reluctance to switch strategies. Using the value of I k as an indicator of the confidence level, agent k teststhe hypothesis that his currently using strategy is his beststrategy at hand at each turn. Furthermore, he switchesto another strategy and resets the virtual scores of all hisstrategies to 0 if the null hypothesis is rejected. Apartfrom these differences, the governing rules of HMG areidentical to those of the standard MG.We state the rules of HMG below for reader’s conve-nience. A. Rules of the game
1. HMG is a repeated game of a fixed population of N agents. A number I k ∈ [0 . ,
1) is assigned toagent k once and for all to represent his inertia.2. At each turn τ , every agent has to make a choicebetween one of the two sides (namely side 0 andside 1) based on the strategies to be described inrule 4. Those agents in the side with the least num-ber of agents (known as the minority side) win inthat turn. And in case of a tie, the winning side israndomly selected.3. The only piece of global information reveals to theagents at time τ is the winning sides in the last M turns known as the history ~µ ( τ ).4. Before the game commences, each agent k is as-signed once and for all S randomly picked strate-gies S k,i for i = 0 , , . . . , S −
1. Each strategy S k,i is a function map from the set of all possible histo-ries to the set { , } and its virtual score Ω k,i is setto 0 initially. Without loss of generality, strategy S k, is assumed to be the currently using strategyof agent k at the beginning of game.5. Agent k will switch his current strategy from S k, to S k,j if and only if the maximum virtual scoredifference ∆Ω k drops below the threshold x k √ τ k ,that is,∆Ω k ≡ max i { Ω k, − Ω k,i } = Ω k, − Ω k,j ≤ x k √ τ k (1)where x k is defined by1 √ π Z + ∞ x k e − x / dx = I k (2)and τ k is the number of turns elapsed since agent k ’slast switch of strategy. In case agent k switches hisstrategy, he exchanges the labels 0 and j so that hiscurrently using strategy is always labeled as S k, .In addition, the virtual scores Ω k,i are reset to 0for all i and τ k is reset to 1. 6. Agent k uses his current strategy to guess the mi-nority choice of the current turn. Moreover, the vir-tual score of strategy S k,i is increased (decreased)by 1 if it predicts the minority side of that turncorrectly (incorrectly). B. Parameters used in our simulation
We select the following parameters in our simulations:1. N is odd;2. S = 2;3. all values of I k are chosen to be the same indepen-dent of the agent label k (and we write this common I k as I for simplicity); and4. S k,j are picked from the so-called MRSS [13]. (Thatis, S k,j can be written in the form s τ = η + M X i =1 η i µ τ − i , (3)where s τ is the prediction of the minority side in the τ th turn, η , η , . . . , η M ∈ { , } , µ i is the minorityside in the i th turn and the arithmetic is performedin the finite field of two elements GF (2) [8, 13,16]. In other words, a strategy in the MRSS ischaracterized by ( η , η , . . . , η M ).)With the exception of point 4, the parameters used inthis study are identical to those used in our earlier studyof HMG reported in Refs. [11, 12]. In contrast, strategiesin Refs. [11, 12] are picked from the FSS, namely, theset of all possible strategies. And a strategy in the FSSmay not be expressed as a linear function of µ i ’s. Fromnow on, we use the symbols HMG FSS and HMG
MRSS todenote the HMG in which strategies are picked from theFSS and the MRSS, respectively.
III. OUR NUMERICAL SIMULATION RESULTSA. Focus of our study
We are interested in both the cooperative behaviorsand the dynamics of the game. Recall that MG andHMG are non-positive sum games in the sense that thenumber of winning agents is less than or equal to thenumber of losing agents in each turn. And we say thatthe agents (or the system) cooperate better if the av-erage number of winning agents per turn is high. Ournumerical simulations show that agents self-organize insuch a way that there is no bias in picking the minorityside when averaged over the agents and the number ofturns, so we follow the usual practice to study agent co-operation by means of the α ≡ M +1 /N S against σ /N graph where σ is the variance of the number of agentschoosing side 0 [13]. The lower the value of σ /N , thebetter the agent cooperation.As for the dynamics of HMG, our investigation focuseson the analysis of the periodicity of the minority choicetime series through the auto-correlation function. Andthe auto-correlation function can be conveniently stud-ied by means of a time lag t against auto-correlation C graph. In order to make sure that the dynamics is gen-uine and long lasting, we only consider the time seriesafter the system has equilibrated. We also perform sim-ulations using different values of N and initial quencheddisorders to make sure that the dynamics we are goingto report below are generic.Actually, the dynamics depends on the following threefactors:1. number of agents N ;2. history size M ; and3. the initial quenched disorder as reflected by thevalue I and the strategies S k,i assigned to theagents.Our choice of parameters for the HMG MRSS reported inSec. II B makes the dynamics of the game deterministicand hence enabling us to study the periodic dynamicsof the minority choice time series easily. In contrast,when played using other choices of N and S , the non-deterministic nature of this game weakens the periodicdynamics in minority choice time series, making boththe numerical and analytical studies more troublesome.Unlike the standard MG, we find that both the co-operative behavior and the dynamics of HMG dependstrongly on the strategy space chosen. We shall elabo-rate more on this point in the coming two subsections. α -3 -2 -1
10 1 / N σ -4 -3 -2 -1 I=0.53I=0.90
M=5, FSS
FIG. 1: The α against σ /N graph for HMG FSS . The valueof α is varied by fixing M and changing N . t0 50 100 150 200 250 300 C -0.200.20.40.60.81 I=0.53
M=5, N=50001, FSS (a) t0 50 100 150 200 250 300 C -0.200.20.40.60.81 I=0.90
M=5, N=50001, FSS (b)
FIG. 2: The auto-correlation for HMG
FSS with α = 6 . × − ≪ α MG c when (a) I = 0 .
53 and (b) I = 0 . B. Reviewing the simulation results of theHMG
FSS
Recall from the earlier study of our group in Refs. [11,12] that when I is chosen to be less than I c , where I c = 1 √ π Z + ∞− √ / M +1 e − x / dx , (4)the inertia of agents is not strong enough to make the dy-namics of the HMG FSS to deviate significantly from thatof the standard MG. Thus, the σ /N is about the sameas that of the standard MG. Moreover, the well-knownperiod 2 M +1 dynamics in the minority choice time seriesthat appears in the standard MG when α is less than α MG c ≈ .
3, the critical value for the standard MG, isalso present here [11, 12]. In contrast, when
I > I c , theinertia of agents becomes strong enough to significantlyreduce the herd effect amongst agents resulting in a muchlower σ /N (and hence indicating that agents cooperatebetter). Besides, the period 2 M +1 dynamics is no longerpresent when α < α MG c [11, 12]. These earlier findingsare summarized in Figs. 1 and 2. C. Simulation results of the HMG
MRSS
Contrary to our expectation, we find that the behav-ior of HMG
MRSS is significantly different from that ofHMG
FSS when
I > ∼ I c and α < ∼
1. The details of ourfindings are listed below. • I < ∼ I c : By comparing Fig. 1 with Fig. 3a andFig. 2 with Figs. 4 and 5, we know that the behaviorof HMG MRSS in this regime is similar to that ofHMG
FSS . That is, they have about the same levelof agent cooperation. In addition, the standard MG[17, 18, 19], HMG
FSS [11] and HMG
MRSS all exhibitthe same period 2 M +1 dynamics in the minoritychoice time series whenever α < ∼ α MG c and show noperiodic dynamics for α > ∼ α MG c . • I > ∼ I c and α ≥ ≫ α MG c ): By comparing Fig. 1with Fig. 3, we find that the values of σ /N areabout the same for both HMG FSS and HMG
MRSS in this regime. Moreover, the standard MG [13],HMG
FSS [11] and HMG
MRSS all show no periodicdynamics. • I > ∼ I c and α < ∼ . ≪ α MG c ): Figs. 1 and 3 showthat for the same value of α in this regime, thecooperation amongst agents for the standard MGis the worst, for HMG MRSS is in the middle andfor HMG
FSS is the best. One interesting featurefor HMG
MRSS is that, unlike HMG
FSS , the valueof σ /N increases as α decreases in this regime in-dicating that agents cooperate less and less as thenumber of agents N increases (and with M and S held fixed). As for the dynamics, Fig. 6 depicts thatthe system exhibits no obvious periodic dynamics. α -3 -2 -1
10 1 / N σ -4 -3 -2 -1 I=0.53I=0.90
M=5, MRSS (a) α -3 -2 -1
10 1 / N σ -4 -3 -2 -1 I=0.95
M=5, MRSS (b)
FIG. 3: Plots of α against σ /N for HMG MRSS with (a) I =0 . , .
90 and (b) I = 0 . Nonetheless, its minority choice time series condi-tioned on an arbitrary but fixed history exhibits avery weak period two dynamics. • I > ∼ I c and 0 . < ∼ α ≈ α MG c <
1: In this regime,we find that the value of σ /N obtained after equi-libration depends on the initial quenched disorderof the system indicating the presence of a phasetransition point. (See Fig. 3.) Actually, the val-ues of σ /N obtained in many runs are rather closeto the theoretical minimum of 1 / N (which is at-tained when there are exactly ( N − / σ /N about the same as those for I < I c . (See Fig. 7a.)In contrast, those ending up with a much smaller σ /N show intermittency. (See Fig. 7b.) That isto say, when σ /N is small, the time series exhibitsperiodic dynamics for some time and then the pe-riodicity either suddenly disappears or the periodof the dynamics changes. Also the brief episodeof aperiodicity terminates with the commencementof a new periodic dynamics (with possibly a newperiod).Interestingly, the period of the orderly phase forthis intermittency depends on the value of I . Incase I c < ∼ I < I c , where I c ≡ √ π Z + ∞−√ e − x / dx ≈ . , (5)the periods are less than or equal to 2(2 M −
1) when-ever M ≥
2. More importantly, these periods are inthe form 2 j L.C.M.( p , p , . . . , p M ) where j ≥ M arguments and p , . . . , p M are positive in-tegers dividing (2 M − FSS . We observe the trend that long pe-riod dynamics tends to be more stable in the sensethat it lasts longer. In fact, the longest periodicdynamics, namely the one with period 2(2 M − M −
1) dynamics isharder and harder to find as M increases beyondabout 7.In the case of I ≥ I c , the periodic dynamics ofthe orderly phase of the intermittency is weak com-pared with the case of I c < ∼ I < I c . As shownin Fig. 7c, the maximum period is in the form2 j (2 M −
1) where j is an integer greater than orequal to 2.Our findings of the dynamics of the minority choicetime series can be tabulated in Table I. Period of MG HMG
FSS
HMG
MRSS dynamics
I < I c I > I c I ≪ I c I > ∼ I c α ≫ α MG c nil nil nil nil nil α ≈ α MG c M +1 M +1 nil 2 M +1 intermittent a α ≪ α MG c M +1 M +1 nil 2 M +1 nil ba The maximum period of the orderly phase of this dynamics is2 j (2 M −
1) where the value of j can be found in the main text. b But the minority choice time series conditioned on an arbitrarybut fixed history shows a very weak period two dynamics
TABLE I: Summary of the dynamics in the minority choicetime series for MG and HMG for odd number of agents and M ≥ t0 50 100 150 200 250 300 C -0.200.20.40.60.81 I=0.53
M=5, N=51, MRSS (a) t0 50 100 150 200 250 300 C -0.200.20.40.60.81 I=0.90
M=5, N=51, MRSS (b)
FIG. 4: The auto-correlation of HMG
MRSS for α = 0 . ≫ α MG c when (a) I = 0 .
53 and (b) I = 0 . D. Conditions for the period j L.C.M. ( p , p , . . . , p M ) dynamics in the orderlyphase of the HMG MRSS intermittency
We find that in HMG
MRSS , some agents seldom changetheir strategies while others do so frequently. We say thatan agent is oscillating if he switches strategy within theprevious 2 M +1 turns. Otherwise, the agent is said to be frozen . It turns out that the number of frozen agentsand their performance provide important information toallow us to understand the origin of the dynamics in the t0 50 100 150 200 250 300 C -0.200.20.40.60.81 I=0.53
M=5, N=50001, MRSS
FIG. 5: The auto-correlation of HMG
MRSS for
I < I c and α = 6 . × − ≪ α MG c . t0 50 100 150 200 250 300 C -0.200.20.40.60.81 I=0.90
M=5, N=50001, MRSS (a) t0 50 100 150 200 250 300 C -0.200.20.40.60.81 I=0.95
M=5, N=50001, MRSS (b)
FIG. 6: The auto-correlation of HMG
MRSS when α = 6 . × − ≪ α MG c together with (a) I c < I < I c and (b) I ≥ I c . minority choice time series of HMG MRSS for
I > ∼ I c and α ≈ α MG c .Table II shows the average probabilities for a frozen(an oscillating) agent to correctly predict the minorityside in a typical HMG MRSS with
I > ∼ I c and α ≈ α MG c .Clearly, a frozen (an oscillating) agent will have a higher(lower) chance to correctly predict the minority side (thatis, the winning probability) in the next turn. Since an os-cillating agent must lose frequently in recent turns, ourfinding means that badly performing agents are likelyto perform badly in future. More importantly, numer-ical simulations tell us that the presence of the period2 j L.C.M.( p , p , . . . , p M ) dynamics in the orderly phaseof the intermittency where j ≥ t0 50 100 150 200 250 300 C -0.200.20.40.60.81 /N σ I=0.90, large
M=5, N=501, MRSS (a) t0 50 100 150 200 250 300 C -0.200.20.40.60.81 /N σ I=0.90, small
M=5, N=501, MRSS (b) t0 50 100 150 200 250 300 C -0.200.20.40.60.81 /N σ I=0.95, small
M=5, N=501, MRSS (c)
FIG. 7: The auto-correlation of HMG
MRSS for α = 0 . ≈ α MG c with (a) I c < I < I c and σ /N is large, (b) I c < I
90 averaged over 100 independentruns. p , . . . , p M divide (2 M −
1) is almost always accompa-nied by the presence of only one oscillating agent in theentire system. Besides, this oscillating agent uses onlytwo distinct strategies that are anti-correlated. (That is,these two strategies predict different minority side forall given histories [14, 15].) Moreover, the strategiesused by the remaining frozen agents form a collection of crowd-anticrowd pairs. Interestingly, our observed in-termittency disappears and becomes a single aperiodicphase if we replace the histories by random variables orif the number of agents N is even. Also, the sole oscillat-ing agent associated with each orderly phase of the timeseries may be different. IV. THE CROWD-ANTICROWDEXPLANATION
Let us briefly review how the crowd-anticrowd theoryexplains the behavior of the standard MG before adapt-ing it to explain the behavior of HMG
FSS and HMG
MRSS . A. The crowd-anticrowd theory for the standardMG
According to the crowd-anticrowd theory, agent coop-eration in the standard MG is determined by the numberof (effective) anti-correlated pairs of strategies in currentuse. The smaller the difference between the number ofagents currently adopting a strategy a and those cur-rently adopting its anti-correlated strategy ¯ a , the bet-ter the crowd-anticrowd cancellation leading to a bet-ter agent cooperation. Since the number of availablestrategies is much less than the strategy space size for α ≫ α MG c , crowd-anticrowd cancellation cannot be effec-tive in this regime. And because standard MG agents donot have inertia, they switch strategies immediately oncethe maximum virtual score difference is negative. Due tothe fact that the virtual score of a strategy in the stan-dard MG is independent of who owns or uses it, everystandard MG agent has the same view on the perfor-mance of a given strategy. So, when α ≪ α MG c , standardMG agents tend to adopt and drop similar strategies allthe time. This over-reaction leads to a herd effect and isthe origin of the maladaptation in the standard MG inthis regime. Thus, effective agent cooperation is possibleonly for α ≈ α MG c in the standard MG. A remarkablefeature of the standard MG is that effective agent coop-eration is indeed possible in this regime in spite of thefact that agents act independently by utilizing commonglobal coarse-grained information only [14, 15]. B. Towards the crowd-anticrowd theory for HMG
Unlike the standard MG, HMG agents use Eq. (1)to decide whether to keep their currently using strate-gies or not. In general, they are initially assigned dif-ferent strategies and begin to adopt their currently us-ing strategies at different times. So, they have differentvirtual score difference ∆Ω k and the number of turnssince the adoption of the current strategy τ k . Togetherwith the virtual score reset mechanism stated in rule 4of HMG, the same strategy may be ranked differentlyamongst HMG agents. Consequently, by picking strate-gies from the same strategy space, the effective strategydiversity for HMG is in general bigger than that for thestandard MG. Furthermore, the higher the value of I (and hence the higher the inertia), the slower the averagerate of strategy switching. All these factors reduce HMGagents’ over-reaction and the herd effect making the sys-tem to better cooperate [11, 12]. Just like the standardMG, the dynamics of HMG encourages agents to formcrowd-anticrowd pairs thereby increasing agent coopera-tion. But unlike the standard MG, this “crowd-anticrowdpair formation” driving force in HMG is more gentle andis less likely to cause over-reaction and maladaptationbecause of the presence of inertia. Hence, the higher theinertia, the longer the equilibration time. At this point,we have to emphasize that the presence of inertia neednot imply that the agents must cooperate because the“crowd-anticrowd pair formation” driving force may notbe strong enough in a certain parameter regime. Thebottom line is that the presence of inertia never worsenthe agent cooperation. In other words, for fixed values of M , N and S , the variance of attendance per agent σ /N for the standard MG has to be greater than or equal tothat for the HMG FSS or HMG
MRSS .Suppose the agents really cooperate. Still there are twoways to prevent them from cooperating forever. Con-sider a history ~µ which has non-zero probability of oc-currence. Suppose further that the minority choice timeseries conditioned on this history ~µ is biased. (That is, itis more likely to find a particular minority choice than theother in this conditioned time series.) In this case, cer-tain strategy a will outperform its anti-correlated partner¯ a in the long run. More precisely, the rate of change ofvirtual score difference between a and ¯ a averaged over asufficiently long number of turns is positive. So, after asufficiently long time, agents will begin to drop strategy¯ a and adopt strategy a , making strategy ¯ a much morepopular than strategy a . In particular, if the minoritychoice time series exhibits periodic dynamics, then thetimescale for a frozen agent to change to an oscillatingagent via this mechanism is directly proportional to theperiod of the dynamics and inversely proportional to thenumber of biased histories in the minority choice timeseries.Even if the minority choice time series conditioned onevery history is un-biased, there is still a way for agentsto stop cooperating. Provided that the value of ∆Ω k follows an un-biased random walk, after sufficiently longtime, agent k can switch his strategy once a while dueto fluctuations in ∆Ω k . These two mechanisms act likea “crowd-anticrowd pair destruction” driving force thatdecreases agent cooperation. Surely, the former mecha-nism is more efficient.In summary, it is the combined actions of the abovetwo driving forces that determine the agent cooperationand dynamics of HMG. In fact, the “crowd-anticrowdpair formation” driving force dominates the initial dy-namics of the HMG. And the “crowd-anticrowd pair de- struction” driving force becomes important after most ofthe agents have been paired up. This picture allows usto understand the simulation results reported in Sec. III. C. The crowd-anticrowd explanation forHMG
MRSS
In the case of α ≫ α MG c , there are so few strate-gies at play that most of the crowd-anticrowd pairs ismade up of only one agent. As a result, the “crowd-anticrowd pair formation” driving force is never strongenough to ensure agent cooperation irrespective of thevalue of I . Thus, crowd-anticrowd cancellation is ineffec-tive. Besides, agents in effect make random choices eachturn so that the value of σ /N approaches the coin-tosslimit as α → ∞ . Surely, the minority choice time se-ries does not show any periodic dynamics [14, 15]. Sincethe above arguments are also valid for the standard MGand HMG FSS , we understand why MG, HMG
FSS andHMG
MRSS all behave in the same way in this parameterregime.In the case of
I < ∼ I c , the inertia of agents is so lowthat agents switch strategies immediately whenever themaximum virtual score difference is negative. In otherwords, the response of standard MG and HMG agents arethe same in this parameter regime [11]. Hence, the agentcooperation and dynamics of standard MG, HMG FSS andHMG
MRSS are about the same.The remaining case to study is
I > ∼ I c and α < ∼ α MG c .Note that the behavior of HMG FSS and HMG
MRSS in thiscase differ markedly as reported in Sec. III. We begin ouranalysis by stating the following claim whose proof canbe found in Appendix A.
Claim 1
Let N be an even number. Suppose each of the N players are randomly and independently assigned S socks; and a sock has M possible colors and can eitherbe left or right. (Hence, there are M +1 kinds of socks.)Suppose further that each kind of sock is selected withequal probability. Then, provided that M +1 < N S , theprobability that there exists a way to form N/ pairs ofsocks by picking exactly one sock from each of the N play-ers is greater than or equal to − β/N for some positive β which is independent of N . Recall that two strategies are said to be anti-correlatedif they always predict different minority side. And theyare said to be uncorrelated if they have equal chance topredict the same minority side provided that each historyoccurs with equal probability. Thus, strategies in theMRSS consist of 2 M pairs of anti-correlated strategiesand that strategies picked from two distinct pairs areuncorrelated [14, 15].By identifying the 2 M +1 different kinds of socks withthe 2 M +1 different strategies in the MRSS, Claim 1 im-plies that for a sufficiently large N and for α < ⌊ N/ ⌋ pairs of anti-correlatedstrategies by picking one strategy from each agent’s strat-egy pool is high. Surely, finding this solution requirescommunications amongst the agents. If the agents couldkeep on using his particular choice of strategy throughoutthe game, maximum agent cooperation would be attainedand hence the theoretical minimum value of σ /N (thatis σ /N = 0 if N is even and 1 / N if N is odd) would beresulted. In contrast, since the strategy space size of theFSS is exponentially larger than that of the MRSS in thelarge M limit, the condition α < M < N S (and hence, α → + inthe thermodynamic limit).Claim 1 only assures the existence of an optimal wayof agent cooperation with a high probability. It remainsto show that this maximum agent cooperation can beachieved with a high chance for HMG MRSS under certainconditions. Recall from step 5 of the rules of HMG thatagent k uses the maximum virtual score difference ∆Ω k amongst all the strategies initially assigned to him todecide whether to change strategy or not. The followingconsequences of step 5 are important to understand thestrong dependence of HMG on the strategy space used:1. The higher the value of I k , the more willingly foragent k to stick to his currently using strategy.2. For fixed values of I k ’s, agents with a strategy andits anti-correlated partner in their pool of strategieshave the strongest tendency, on average, to switchtheir strategies.3. Suppose agent k has just switched to a new strat-egy and that this newly adopted strategy incor-rectly predicts the minority side in its first use, then τ k = 1 and ∆Ω k = −
2. As a result, this agent willdrop his newly adopted strategy in the next turnprovided that I k < I c .
1. The sub-case of
I > ∼ I c and α ≈ α MG c Shortly after the commencement of HMG
MRSS , the mi-nority choice time series should resemble an un-biasedrandom sequence. So provided that I is sufficiently large,Subsec. IV B tells us that the “crowd-anticrowd pair for-mation” driving force allows a large number of agentsto form crowd-anticrowd pairs. Most of these pairedagents will be frozen, and there are only a few oscillat-ing agents in the system. By simple probability consid-eration, we expect that most of the strategies hold bythese oscillating agents are un-correlated. As the dy-namics of the system, which is determined mostly by thedynamics of these oscillating agents, the minority choicetime series conditioned on an arbitrary history is likely tobe un-biased. Hence, fewer and fewer oscillating agentswill present as they gradually form crowd-anticrowd pairand become frozen. From Claim 1, we believe that for a sufficiently large I , agents in HMG MRSS have a highchance to attain maximum agent cooperation providedthat α ≈ α MG c <
1. This is consistent with the findingsin our numerical simulations reported earlier in Sec. IIIthat the highest chance of finding maximum agent co-operation is when α ≈ α MG c and I > ∼ I c . And thismaximum agent cooperation is accompanied by the ex-istence of at most one oscillating agent in the systemwho switches between a pair of anti-correlated strategies.This finding agrees with the discussion following Claim 1that the agent holding a pair of anti-correlated strategiesswitches his strategy most readily. To conclude, when N is odd, the effective number of strategies at play forHMG MRSS in this regime is reduced to one in most ofthe time. Thus, the frozen agents have an average of 1/2chance to correctly predict the minority side in the nextturn while the only oscillating agent has no chance to doso. Consequently, the frozen agents are unlikely to switchtheir currently using strategies while the only oscillatingagent is prone to strategy switching.We now discuss the dynamics of the minority choicetime series when N is odd. From the above discussions,it is clear that the system is in an orderly (chaotic) phasewhenever it has one (more than one) oscillating agent. Inaddition, this oscillating agent is most likely to be switch-ing between a pair of anti-correlated strategies. This os-cillating agent always predicts the minority side incor-rectly throughout the corresponding orderly periodic dy-namics phase. Thus, from the discussions immediatelyafter Claim 1, we conclude that this oscillating agentmust drop his currently using strategy and switch to itsanti-correlated counterpart in each turn provided that I c < ∼ I < I c . By identifying a strategy with a linearfunction of the historical minority choices µ i ’s over the fi-nite field GF (2) in the form of Eq. (3), we know that thedifference between the linear functions associated withtwo anti-correlated strategies is equal to 1. So, whenever α ≈ α MG c and I c < ∼ I < I c , the minority side in the n thturn µ n throughout this orderly periodic dynamics phaseobeys µ n = M X i =1 η i µ n − i + η + n , (6)where η i , µ i ∈ GF (2) and n denotes 1 + 1 + · · · +1 ( n terms).We show in Appendix B the following theorem. Theorem 1
The sequence { µ n } generated by the recur-sion relation in Eq. (6) is periodic. Its period is inthe form j L.C.M. ( p , p , . . . , p M ) where j ≥ , and p , . . . , p M are positive integers dividing (2 M − . More-over, the longest possible period for this sequence is M − if M ≥ and if M = 1 . Suppose thesequence is of maximum possible period and M ≥ .Denote the history ( µ n − M +1 , µ n − M +2 , . . . , µ n − , µ n ) by ~µ ( n ) . Then, the two histories of alternating ’s and ’s,namely, (0 , , , , . . . ) and (1 , , , , . . . ) appear in the se-quence of history H = { ~µ ( n ) } M − n =1 once; while all the other (2 M − possible histories all appear in the historysequence H twice. This theorem allows us to explain the period of the or-derly phase of the time series in the case of I c < ∼ I < I c and α ≈ α MG c . In particular, the proof tells us thatthe longest periodic dynamics in the orderly phase for M ≥ M − M ≥
2, the longest possibleperiod of the orderly phase is 2(2 M − M ) characteristic polynomialof the recursion relation (6), which is associated with thepair of anti-correlated strategies used by the sole oscil-lating agent, is primitive. In a single period, the twohistories that consists of alternating 0’s and 1’s appearonce while all other histories appear twice. Therefore,the number of turns between two consecutive appear-ance of each of the two histories consisting of alternating0’s and 1’s equal the even number 2(2 M − ~µ ( k ) = ~µ ( k ′ ) with ( k − k ′ ) beinga positive even number, the homogeneous parts of thesolutions of Eq. (6) for n = k − i and n = k ′ − i areequal whenever i = 0 , , . . . , M −
1. And this is possi-ble only when (2 M − | ( k − k ′ ). As ( k − k ′ ) is even,so k − k ′ ≥ M − M −
1) period dynamics can be foundin the minority choice time series although it cannot beever-lasting. Since out of the 2 M possible minority choicetime series conditional on a particular history, only twoof them show period one rather than period two dynam-ics, the period 2(2 M −
1) dynamics in the minority choicetime series is quite stable in the sense that it lasts for alonger time. This also explains why the stability of thisperiod increases with M .As M increases while keeping S and α fixed, the prob-ability of having an agent in the system that holds apair of anti-correlated strategy is about 1 − [1 − S ( S + 1) / M +2 ] N → − e − ( S +1) / α . As α ≈ α MG c ≈ . M −
1) dynamics is about 1 − [1 − S ( S + 1) ϕ (2 M − / M +1) M ] N , where ϕ denotes the Euler-Totient func-tion. By prime number theorem, this probability is equalto at least 1 − [1 − S ( S + 1)(2 M − / M +1) M c +1 ] N → − e − ( S +1) / αM c +1 as M → ∞ , where c is a positive con-stant. This explains why as M increases, there is generaltrend that the period 2(2 M −
1) dynamics orderly phaseoccurs less frequently.From Theorem 1, the second longest period in the or-derly phase is of period 4(2 M − −
1) for M ≥
2. Thisdynamics is associated with a characteristic polynomialin the form ( λ − p ( x ) where p ( x ) is a primitive poly-nomial of degree ( M − M − −
1) orderly phase is quite stable (although it isnot as stable as the period 2(2 M −
1) dynamics) as allbut four conditional minority choice time series exhibitperiod one rather than period two dynamics.For those orderly phases with shorter periods, the num-ber of distinct histories present is small so that the effec-tive diversity of the strategies at play are greatly reduced.Combined with the presence of, in general, a larger pro-portion of period one conditional minority choice timeseries in this dynamics, these shorter period dynamicsare much less stable in the sense that they can last for amuch shorter time.The same analysis can be applied to the case when I ≥ I c . In this case, the only oscillating agent in the systemswitches his strategy once every few turns. Applying thesame analysis as in the proof of Theorem 1, it can beshown that the maximum period of the orderly phaseis 2 j (2 M −
1) where j is the number of turns betweenthe adoption and termination of a strategy for the soleoscillating agent. In this situation, the strength of a lot ofperiod two dynamics in the conditional minority choicetime series are weakened making the dynamics in theorderly phase less pronounced.Note that one of the factors making the above inter-mittent behavior possible is that the number of agents N is odd. In case N is even, there are equal number ofagents choosing each side. Hence, all agents are frozenand have 1/2 chance of correctly predicting the minorityside. And the minority choice time series does not showany periodicity. This is exactly what we observe in oursimulation.Let us discuss more about the agent cooperation. Evenfor α ≈ α MG c and I sufficiently large, the average value of σ /N is still a little bit higher than the theoretical min-imum due to three reasons. First, a few initial quencheddisorders does not allow maximum agent cooperation.Second, the system may be trapped in a non-maximally0cooperative state even though the initial quenched dis-order allows maximum agent cooperation. Third, as wehave pointed out in Subsec. IV B, the aperiodic phaseof the intermittency is associated with the sub-maximalagent cooperation making the value of σ /N averagedover initial quenched disorder greater than the minimumvalue. So, it is not surprising to find that the values of σ /N for HMG MRSS and HMG
FSS are about the samefor α ≈ α MG c and I > ∼ I c .
2. The sub-case of
I > ∼ I c and α ≪ α MG c For fixed M and S , the number of agents N increasesas α decreases. From the discussions in Subsec. IV B, weknow for a sufficiently large N , the timescale for all ex-cept at most one of the agents to form crowd-anticrowdpairs is longer than the timescale for a pair of crowd-anticrowd agents to break up. Therefore, as α ≪ α MG c ,it is highly unlikely for the system to attain maximumagent cooperation. Besides, by further increasing α , over-crowding of strategies is severe as more and more agentsare using the same strategy with similar virtual scoresand number of turns since its adoption. As a result, mal-adaptation and herd effect begin to appear. These arethe reasons why for I > ∼ I c , the value of σ /N startsto increase as α falls below about 0.1. This is also thereason why the minority choice time series conditionedon an arbitrary but fixed history exhibits a very weakperiod two dynamics. However this period two dynamicsis much weaker than the one observed in the standardMG because the presence of inertia makes the agents toresponse less readily. As the effective strategy diversityof HMG FSS is greater than that of HMG
MRSS which isin turn greater than that of the standard MG, the valueof σ /N for HMG FSS is smaller than that of HMG
MRSS which is in turn smaller than that of the standard MGprovided that
I > ∼ I c .Discussions in the previous two sub-cases predict thatby decreasing α below ≈ M /N S , the value of σ /N for HMG FSS will start to increase due to overcrowding ofstrategies. Unfortunately, we are not able to check thecorrectness of our prediction because the memory andrun time requirements are too high.
V. DISCUSSIONS
In summary, we have performed extensive numericalsimulations to study the behavior of HMG
MRSS . Wefound that HMG agents cooperate better provided thattheir strategies are picked from the FSS instead of fromthe MRSS. This is because the effective diversity ofstrategies is higher in the former case. Based on thecrowd-anticrowd theory [14, 15], we understood the ori-gin of agent cooperation for HMG
MRSS in various pa-rameter ranges by studying the interplay between theso-called “crowd-anticrowd pair formation” driving force and the so-called “crowd-anticrowd pair destruction”driving force. And we found that the difference in the co-operative and dynamical behavior between HMG
FSS andHMG
MRSS is mainly due to the structure of the strategyspace used. In particular, we were able to explain thenovel intermittent behavior of the system and the orderlyphase dynamics of the minority choice time series when α ≈ α MG c , I > ∼ I c , N is odd and σ /N ≈ / N . Es-sentially, this novel orderly dynamics in the intermittentphase is caused by the fact that the game is effectivelyreduced to a similar game played by only one agent mostof the time and is accompanied by the maximum possibleagent cooperation. And this reduction is possible in casethe strategy pool is MRSS rather than FSS.On one hand, HMG MRSS appears to be special in thesense that it is the only variant of MG we have examinedwhose behavior depends sensitively on whether the FSSor the MRSS is used. On the other hand, the assump-tion that the behavior of standard MG when played inFSS or in MRSS is about the same is only a working as-sumption supported by numerical simulation results andheuristic arguments [14, 15]. Although this assumptiongreatly simplifies the space complexity of numerical sim-ulations and sometimes even enable us to obtain a fewsemi-analytical results, one should bear in mind that thisis only an assumption after all. In this regard, it is in-structive to study the conditions under which one canreplace FSS by MRSS without significantly affecting thedynamics and cooperative behavior of a system.Lastly, we remark that the standard way of using thenumber 2 M +1 to measure the diversity of strategies is nolonger appropriate for HMG. We believe that by suitablydefining the diversity of strategies (and hence also theexpression for α ), the σ /N against α curves for HMGplayed using different choices of strategy spaces can bemade to coincide, at least roughly. Acknowledgments
We thank the Computer Center of HKU for their help-ful support in providing the use of the HPCPOWER sys-tem for simulations reported in this paper.
APPENDIX A: PROOF OF CLAIM 1
Let Γ , Γ ′ denote two different partitions of the N play-ers into N/ Γ denotes the probability thatit is possible for each of the N/ Γ ’s are exchangeable. That is, Pr Γ = Pr Γ ′ . In1fact, for S ≪ M +1 ,Pr Γ ≈ ( (cid:0) M +1 (cid:1) !2 S ( M +1) (2 M +1 − S )! " − (cid:18) − S M +1 (cid:19) S N ≈ S N MNS . (A1)By de Finetti theorem [20] and its generalization byDiaconis and Freedman [21], as long as 0 < Pr Γ < N , the probability that there exists a way toform N/ ∞ ) − β/N for some β > N , where Pr( ∞ ) denotes the same prob-ability when N → ∞ .So, to prove this claim, it suffices to show that Pr( ∞ ) =1 whenever 2 M +1 < N S and S ≥
2. Under these two con-ditions, it is obvious that Pr Γ ∈ (0 ,
1) for any finite N .So, de Finetti theorem implies that Pr Γ ’s are condition-ally independent given the tail σ -field [20]. Thus,Pr( ∞ ) = lim N →∞ − Y Γ (1 − Pr Γ ) = 1 . (A2)Hence, claim 1 is proved. (cid:3) Before leaving this appendix, we point out a commonmistake people makes. The probability that all pairedplayers have matching socks for an arbitrary but fixedpartition Γ is equal to P(Γ) ≈ h − (cid:0) − S/ M +1 (cid:1) S i N/ .Thus, the expected number of partitions satisfying this“all paired players” condition is P(Γ) × N ! / [( N/ . Athreshold condition for the existence of a partition thatall paired players have matching socks can then be de-duced by saddle point approximation. The loophole inthis argument is that socks are assigned once and for allto the players so that the probabilities P(Γ) and P(Γ ′ )are not independent. APPENDIX B: PROOF OF THEOREM 1
This proof uses a number of finite field concepts andtechniques. Readers who are not familiar may consultRef. [22] before moving on. The characteristic equationof Eq. (6) is λ M − M X i =1 η i λ M − i = 0 . (B1)If the characteristic equation has no degenerate root,the homogeneous part of the solution of µ n in Eq. (6)is in the form P i g i w ni where w i are roots of Eq. (B1)over the extension field GF (2 M ) for some g i ∈ GF (2 M ).Clearly, the period of the sequence { P i g i w ni } + ∞ n =0 dividesL.C.M.( | w | , | w | , . . . , | w M | ), where | w i | denotes the or-der of w i . And the equality holds if g i ’s are all non-zero.(Remember also that | w i | divides (2 M −
1) for all i .) What if the roots of Eq. (B1) are degenerate? Sup-pose w is a double root of Eq. (B1). Then, w n and nw n are the two generators of the homogeneous part of thesolution of Eq. (6). Besides, the period of the sequence { g w n + g nw n } divides 2 | w | with the equality holds if g , g = 0. Similarly, let 2 j be the smallest integer greaterthan or equal to k . Then, it is straight-forward to checkthat 2 j | w | is divisible by the period of the homogeneouspart of the solution of the recursion relation (6) that cor-responds to a degree k root w of the characteristic equa-tion (B1). Furthermore, the period can attain the value2 j | w | provided that the coefficients g i ’s are non-zero.Combining the two cases above, we conclude that theperiod of the homogeneous solution of Eq. (6) divides2 j L.C.M.( | w | , | w | , . . . ) where w i ’s are the distinct rootsof its characteristic equation and j is a non-negative inte-ger. The maximum possible period for this homogeneoussolution is (2 M − ϕ (2 M − /M primitive polynomials ofdegree M in the GF (2)[ λ ], where ϕ is the Euler-Totientfunction. Note further that if the minority choice timeseries were generated by the homogeneous solution ofEq. (6) alone, then this time series would not contain M consecutive 0’s should its period be (2 M − M ≥ µ n = η + n (mod 2)if |{ i : η n − i = 0 }| is even or µ n = η + n + 1 (mod 2) if |{ i : η n − i = 0 }| is odd. In either case, the period of thisparticular solution is 2.The remaining case is when the number of (non-zero)terms in Eq. (B1) is even. In this case, ( λ −
1) must bea factor of the R.H.S. of Eq. (B1). We write the R.H.S.of Eq. (B1) as ( λ − k p ( λ ) for some polynomial p ( λ )with p (1) = 0. If k = 1, a particular solution is µ n = ⌊ ( n + 1 − η ) / ⌋ (mod 2) which is of period 4. Similarly,it is easy to check that there is a particular solution ofEq. (6) whose period divides 2 j where 2 j is the smallestinteger greater than ( k + 1).By combining the studies of the periods of the homo-geneous and particular solutions of Eq. (6), we concludethat the period of the general solution of Eq. (6) is in theform 2 j L.C.M.( | w | , | w | , . . . ) where w i ’s are the distinctroots of Eq. (B1) and 2 j is the smallest integer greaterthan ( k + 1). Moreover, the maximum possible periodfor the solution of the recursion relation (6) is 2(2 M − M ≥ M = 1. In the former case, sucha maximum period is attained only if the characteristicpolynomial of Eq. (6) is primitive. And in the latter case,it is attained when the characteristic polynomial is λ − { µ n } attains its maximum possible value of2(2 M − (cid:3) [1] P. W. Anderson, K. Arrow, and P. Pines, The Economyas an evolving Complex System (Addison-Wesley, NewYork, 1988).[2] P. Mirowski,