Evgueni Gordienko

Adaptive control for discrete-time Markov processes with unbounded costs: Average criterion

Abstract. The paper deals with a class of discrete-time Markov control processes with Borel state and action spaces, and possibly unbounded one-stage costs. The processes are given by recurrent equations xt+1=F(xt,at,ξt), t=1,2,… with i.i.d. ℜk– valued random vectors ξt whose density ρ is unknown. Assuming observability of ξt, and taking advantage of the procedure of statistical estimation of ρ used in a previous work by authors, we construct an average cost optimal adaptive policy.

Robustness inequality for Markov control processes with unbounded costs

The paper discusses the robustness of discrete-time Markov control processes whose transition probabilities are known up to certain degree of accuracy. Upper bounds of increase of a discounted cost are derived when using an optimal control policy of the approximating process in order to control the original one. Bounds are given in terms of weighted total variation distance between transition probabilities. They hold for processes on Borel spaces with unbounded one-stage costs functions.

Average optimal switching of a Markov chain with a Borel state space

Abstract. We extend results on average per unit time optimality criterion in a switching model from a countable state space to a Borel state space. In the model we consider, a controller selects an increasing sequence of stopping times with respect to a Markov chain, and gets rewards and pays costs at them in an alternating order. The rewards and costs depend on the state of the chain. We find the optimal average gain and construct an optimal strategy. The basic tool is a variational problem with two obstacles that appears also in Dynkin games.

Stability estimates in the problem of average optimal switching of a Markov chain

Abstract. We consider a switching model for a Markov chain xt with a transition probability p(x|B). The goal of a controller is to maximize the average gain by selecting a sequence of stopping times, in which the controller gets rewards and pays costs (depending on xt) in an alternating order. We suppose that the exact transition probability function of the original “real” chain xt is not available to the controller, and he/she is forced to rely on a given approximation

Average cost Markov control processes: stability with respect to the Kantorovich metric

\widetildep

Discounted cost optimality problem: stability with respect to weak metrics

to the unknown p. The controller finds a switching policy

Estimation of robustness for controlled diffusion processes

\widetilde\pi

Estimates of stability of geometric convolutions

optimal for the Markov chain with the transition probability

New estimates of continuity in M/GI/1/\infty queues

\widetildep

A note on deterministic approximation of discounted Markov decision processes

, with a view to apply