Silvia C. Di Marco

On a system of Hamilton-Jacobi-Bellman inequalities associated to a minimax problem with additive final cost

We study a minimax optimal control problem with finite horizon and additive final cost. After introducing an auxiliary problem, we analyze the dynamical programming principle (DPP) and we present a Hamilton-Jacobi-Bellman (HJB) system. We prove the existence and uniqueness of a viscosity solution for this system. This solution is the cost function of the auxiliary problem and it is possible to get the solution of the original problem in terms of this solution.

Illustrated review of convergence conditions of the value iteration algorithm and the rolling horizon procedure for average-cost MDPs

This paper is concerned with the links between the Value Iteration algorithm and the Rolling Horizon procedure, for solving problems of stochastic optimal control under the long-run average criterion, in Markov Decision Processes with finite state and action spaces. We review conditions of the literature which imply the geometric convergence of Value Iteration to the optimal value. Aperiodicity is an essential prerequisite for convergence. We prove that the convergence of Value Iteration generally implies that of Rolling Horizon. We also present a modified Rolling Horizon procedure that can be applied to models without analyzing periodicity, and discuss the impact of this transformation on convergence. We illustrate with numerous examples the different results. Finally, we discuss rules for stopping Value Iteration or finding the length of a Rolling Horizon. We provide an example which demonstrates the difficulty of the question, disproving in particular a conjectured rule proposed by Puterman.

NUMERICAL ANALYSIS OF A MINIMAX OPTIMAL CONTROL PROBLEM WITH AN ADDITIVE FINAL COST

In this paper we deal with the numerical analysis of an optimal control problem of minimax type with finite horizon and final cost. To get numerical approximations we devise here a fully discrete scheme which enables us to compute an approximated solution. We prove that the fully discrete solution converges to the solution of the continuous problem and we also give the order of the convergence rate. Finally we present some numerical results.

A Finite State Stochastic Minimax Optimal Control Problem with Infinite Horizon

We consider here a stochastic discrete minimax control problem with infinite horizon. We prove the existence of solution, we characterize it and we present iterative methods to compute it numerically.