Athena Picarelli

State-Constrained Stochastic Optimal Control Problems via Reachability Approach

This paper deals with a class of stochastic optimal control problems (SOCPs) in the presence of state constraints. It is well known that for such problems the value function is, in general, discontinuous, and its characterization by a Hamilton--Jacobi equation requires additional assumptions involving an interplay between the boundary of the set of constraints and the dynamics of the controlled system. Here, we give a characterization of the epigraph of the value function without assuming the usual controllability assumptions. To this end, the SOCP is first translated into a state-constrained stochastic target problem. Then a level-set approach is used to describe the backward reachable sets of the new target problem. It turns out that these backward reachable sets describe the value function. The main advantage of our approach is that it allows us to easily handle the state constraints by an exact penalization. However, the target problem involves a new state variable and a new control variable that is u...

Hamilton–Jacobi–Bellman Equations

In this chapter we present recent developments in the theory of Hamilton–Jacobi–Bellman (HJB) equations as well as applications. The intention of this chapter is to exhibit novel methods and techniques introduced few years ago in order to solve long-standing questions in nonlinear optimal control theory of Ordinary Differential Equations (ODEs).

Infinite Horizon Stochastic Optimal Control Problems with Running Maximum Cost

An infinite horizon stochastic optimal control problem with running maximum cost is considered. The value function is characterized as the viscosity solution of a second-order HJB equation with mixed boundary condition. A general numerical scheme is proposed and convergence is established under the assumptions of consistency, monotonicity and stability of the scheme. A convergent semi-Lagrangian scheme is presented in detail.

Boundary Mesh Refinement for Semi-Lagrangian Schemes

We investigate the value function of Mayers problem arising in optimal control and provide three (equivalent) definitions of lower semicontinuous solutions of the associated HamiltonJacobi-Bellman equation. Under quite weak assumptions about the control system, the value function is the unique solution. It coincides with the viscosity solution whenever it is continuous.