Frank Thuijsman

Cyclic Markov equilibria in stochastic games

We examine a three-person stochastic game where the only existing equilibria consist of cyclic Markov strategies. Unlike in two-person games of a similar type, stationary ε-equilibria (ε > 0) do not exist for this game. Besides we characterize the set of feasible equilibrium rewards.

Recursive repeated games with absorbing states

We show the existence of stationary limiting average e-equilibria e > 0 for two-person recursive repeated games with absorbing states. These are stochastic games where all states but one are absorbing, and in the nonabsorbing state all payoffs are equal to zero. A state is called absorbing if the probability of a transition to any other state is zero for all available pairs of actions. For the purpose of our proof, we introduce properness for stationary strategy pairs. Our result is sharp since it extends neither to the case with more nonabsorbing states, nor to the n-person case with n > 2. Moreover, it is well known that the result cannot be strengthened to the existence of 0-equilibria and that repeated games with absorbing states generally do not admit stationary e-equilibria.

The bad match; a total reward stochastic game

SummaryFor two person zero sum stochastic games we introduce a new criterion for evaluating streams of payoffs. When the players use this criterion we call such games total reward stochastic games. It is unknown whether total reward stochastic games, with the property that the average value is zero for each initial state, always have a value. We examine an example of such a total reward stochastic game in which one of the players can playε-optimal only by using history dependent strategies.ZusammenfassungFür stochastische Zwei-PersonenNull-Summen-Spiele wird ein neues Kriterium zur Bewertung der Auszahlungsströme eingeführt, das Gesamt-Gewinn-Kriterium. Es ist bisher unbekannt, ob stochastische Spiele, deren Wert bezüglich des Durchschnittsgewinn-Kriteriums gleich Null ist, bezüglich des Gesamt-Gewinn-Kriteriums einen “Wert” besitzen. Es wird ein Beispiel untersucht, in dem ein Spieler nure-optimal spielen kann, wenn er von der Vorgeschichte abhängige Strategien benutzt.

Easy initial states in stochastic games

In this paper we deal with limiting average stochastic games with finite state and action spaces. For any nonzero-sum stochastic game of this type, there exists a subset of initial states for which an almost stationary ∈-equilibrium exists. For any zero-sum stochastic game there exists for each player a subset of initial states for which this player has an optimal stationary strategy.

Total reward stochastic games and sensitive average reward strategies

In this paper, total reward stochastic games are surveyed. Total reward games are motivated as a refinement of average reward games. The total reward is defined as the limiting average of the partial sums of the stream of payoffs. It is shown that total reward games with finite state space are strategically equivalent to a class of average reward games with an infinite countable state space. The role of stationary strategies in total reward games is investigated in detail. Further, it is outlined that, for total reward games with average reward value 0 and where additionally both players possess average reward optimal stationary strategies, it holds that the total reward value exists.

Auction Analysis by Normal Form Game Approximation

Auctions are pervasive in todaypsilas society and provide a variety of real markets. This article facilitates a strategic choice between a set of available trading strategies by introducing a methodology to approximate heuristic payoff tables by normal form games. An example from the auction domain is transformed by this means and an evolutionary game theory analysis is applied subsequently. The information loss in the normal form approximation is shown to be reasonably small such that the concise normal form representation can be leveraged in order to make strategic decisions in auctions. In particular, a mix of trading strategies that guarantees a certain profit is computed and further applications are indicated.

The Power of Threats in Stochastic Games

In the theory of limiting average reward infinitely repeated games the Folk theorem tells us that any feasible and individually rational reward can be achieved as an equilibrium reward. The standard proof of this theorem involves pure strategies that yield this reward and threats to prevent the opponent from deviating from his pure strategy. In stochastic games it is not always possible to apply threats in a similar fashion, since a deviation may take play to a different state at which punishment is ineffective. Nevertheless, threats allow us to formulate sufficient, and quite general, conditions for the existence of limiting average ∈-equilibria.

Sex allocation in a polyembryonic parasitoid with female soldiers: an evolutionary simulation and an experimental test.

Parasitoid wasps are convenient subjects for testing sex allocation theory. However, their intricate life histories are often insufficiently captured in simple analytical models. In the polyembryonic wasp Copidosoma koehleri, a clone of genetically identical offspring develops from each egg. Male clones contain fewer individuals than female clones. Some female larvae develop into soldiers that kill within-host competitors, while males do not form soldiers. These features complicate the prediction of Copidosoma’s sex allocation. We developed an individual-based simulation model, where numerous random starting strategies compete and recombine until a single stable sex allocation evolves. Life-history parameter values (e.g., fecundity, clone-sizes, larval survival) are estimated from experimental data. The model predicts a male-biased sex allocation, which becomes more extreme as the probability of superparasitism (hosts parasitized more than once) increases. To test this prediction, we reared adult parasitoids at either low or high density, mated them, and presented them with unlimited hosts. As predicted, wasps produced more sons than daughters in all treatments. Males reared at high density (a potential cue for superparasitism) produced a higher male bias in their offspring than low-density males. Unexpectedly, female density did not affect offspring sex ratios. We discuss possible mechanisms for paternal control over offspring sex.

Semi-infinite Stochastic Games

We investigate two-person zero-sum stopping stochastic games with a finite number of states, for which the action sets of player I are finite and those for player II are countably infinite. Concerning the payoffs no restrictions are made. We show that for such games the value, possibly —∞ in some coordinates, exists; player I possesses optimal stationary strategies and player II possesses near-optimal stationary strategies with finite support. Furthermore we relate the existence of value and of (near-)optimal stationary strategies with a maximal solution to the Shapley-equation.

Spatial vs. non-spatial eco-evolutionary dynamics in a tumor growth model

Metastatic prostate cancer is initially treated with androgen deprivation therapy (ADT). However, resistance typically develops in about 1 year - a clinical condition termed metastatic castrate-resistant prostate cancer (mCRPC). We develop and investigate a spatial game (agent based continuous space) of mCRPC that considers three distinct cancer cell types: (1) those dependent on exogenous testosterone (T+), (2) those with increased CYP17A expression that produce testosterone and provide it to the environment as a public good (TP), and (3) those independent of testosterone (T-). The interactions within and between cancer cell types can be represented by a 3 × 3 matrix. Based on the known biology of this cancer there are 22 potential matrices that give roughly three major outcomes depending upon the absence (good prognosis), near absence or high frequency (poor prognosis) of T- cells at the evolutionarily stable strategy (ESS). When just two cell types coexist the spatial game faithfully reproduces the ESS of the corresponding matrix game. With three cell types divergences occur, in some cases just two strategies coexist in the spatial game even as a non-spatial matrix game supports all three. Discrepancies between the spatial game and non-spatial ESS happen because different cell types become more or less clumped in the spatial game - leading to non-random assortative interactions between cell types. Three key spatial scales influence the distribution and abundance of cell types in the spatial game: i. Increasing the radius at which cells interact with each other can lead to higher clumping of each type, ii. Increasing the radius at which cells experience limits to population growth can cause densely packed tumor clusters in space, iii. Increasing the dispersal radius of daughter cells promotes increased mixing of cell types. To our knowledge the effects of these spatial scales on eco-evolutionary dynamics have not been explored in cancer models. The fact that cancer interactions are spatially explicit and that our spatial game of mCRPC provides in general different outcomes than the non-spatial game might suggest that non-spatial models are insufficient for capturing key elements of tumorigenesis.