J Jaap Wessels

Stopping Times and Markov Programming

In this paper we consider finite state Markov decision processes with finite decision spaces for each state. The optimality criterion will be total expected discounted reward over an infinite time horizon. For these problems a great variety of optimization procedures has been developed. We divide them in two classes: policy improvement procedures and policy improvement-value determination procedures.

Conditions for Optimality in Multi-Stage Stochastic Programming Problems

In this paper it is demonstrated how necessary and sufficient conditions for optimality of a strategy in multi-stage stochastic programs may be obtained without topological assumptions. The conditions are essentially based on a dynamic programming approach. These conditions — called conserving and equalizing — show the essential difference between finite-stage and ∞-stage stochastic programs.