Ashok Maitra

On stochastic games

In this paper, we consider the stochastic games of Shapley, when the state and action spaces are all infinite. We prove that, under certain conditions, the stochastic game has a value and that both players have optimal strategies.

Finitely additive stochastic games with Borel measurable payoffs

Abstract. We prove that a two-person, zero-sum stochastic game with arbitrary state and action spaces, a finitely additive law of motion and a bounded Borel measurable payoff has a value.

On stochastic games, II

In this paper, we consider positive stochastic games, when the state and action spaces are all infinite. We prove that, under certain conditions, the positive stochastic game has a value and that the maximizing player has an ε-optimal stationary strategy and the minimizing player has an optimal stationary strategy.

An operator solution of stochastic games

A class of zero-sum, two-person stochastic games is shown to have a value which can be calculated by transfinite iteration of an operator. The games considered have a countable state space, finite action spaces for each player, and a payoff sufficiently general to include classical stochastic games as well as Blackwell’s infiniteGδ games of imperfect information.

Finitely additive and measurable stochastic games

We consider two-person zero-sum stochastic games with arbitrary state and action spaces, a finitely additive law of motion and limit superior payoff function. The players use finitely additive strategies and it is shown that such a game has a value, if the payoff function is evaluated in accordance with the theory of strategic measures as developed by Dubins and Savage. Moreover, when a Borel structure is imposed on the problem, together with an equicontinuity condition on the law of motion, the value of the game is the same whether calculated in terms of countably additive strategies or finitely additive ones.

Subgame-Perfect Equilibria for Stochastic Games

For an n-person stochastic game with Borel state space S and compact metric action sets A1, A2,..., An, sufficient conditions are given for the existence of subgame-perfect equilibria. One result is that such equilibria exist if the law of motion q(...∣ s, a) is, for fixed s, continuous in a = (a1,a2,...,an) for the total variation norm and the payoff functions f1, f2,...,fn are bounded, Borel measurable functions of the sequence of states (s1, s2,...) ∈ SN and, in addition, are continuous when SN is given the product of discrete topologies on S.

Leavable gambling problems with unbounded utilities

The optimal return function U of a Borel measurable gambling problem with a positive utility function is known to be universally measurable. With a negative utility function, however, U may not be so measurable. As shown here, the measurability of U for all Borel gambling problems with negative utility functions is equivalent to the measurability of all PCA sets, a property of such sets known to be independent of the usual axioms of set theory. If the utility function is further required to satisfy certain uniform integrability conditions, or if the gambling problem corresponds to an optimal stopping problem, the optimal return function is measurable

Borel stay-in-a-set games

Abstract.Consider an n-person stochastic game with Borel state space S, compact metric action sets A1,A2,…,An, and law of motion q such that the integral under q of every bounded Borel measurable function depends measurably on the initial state x and continuously on the actions (a1,a2,…,an) of the players. If the payoff to each player i is 1 or 0 according to whether or not the stochastic process of states stays forever in a given Borel set Gi, then there is an ε-equilibrium for every ε>0.

The optimal reward operator in negative dynamic programming

We consider the negative dynamic programming model of Strauch [12] and prove that the optimal reward function can be obtained by a transfinite iteration of the optimal reward operator. We show that a player loses nothing by restricting himself to measurable policies, if the returns from nonmeasurable policies are evaluated by lower integrals.

Selectors for Borel sets with large sections

We prove a result asserting the existence of a Borel selector for a Borel set in the product of two Polish spaces. This subsumes a number of results about Borel selectors for Borel sets having large sections.