Per Rutquist

On the infinite time solution to state-constrained stochastic optimal control problems

A method is presented for solving the infinite time Hamilton-Jacobi-Bellman (HJB) equation for certain state-constrained stochastic problems. The HJB equation is reformulated as an eigenvalue problem, such that the principal eigenvalue corresponds to the expected cost per unit time, and the corresponding eigenfunction gives the value function (up to an additive constant) for the optimal control policy. The eigenvalue problem is linear and hence there are fast numerical methods available for finding the solution.

AN EIGENVALUE APPROACH TO INFINITE-HORIZON OPTIMAL CONTROL

Abstract A method for finding optimal control policies for first order state-constrained, stochastic dynamic systems in continuous time is presented. The method relies on solution of the Hamilton-Jacobi-Bellman equation, which includes a diffusion term related to the stochastic disturbance in the model. A variable transformation is applied that turns the infinite-horizon optimal control problem into a linear eigenvalue problem in state-space. The method is demonstrated on a buffer control problem for a fuel cell-supercapacitor system. The obtained closed-form solution explains the shape of previous heuristically found control laws for this type of problem.

Finite-Time State-Constrained Optimal Control for Input-Affine systems with Actuator Noise

Abstract: We show that a linearizing transformation of the Hamilton-Jacobi-Bellman (HJB) equation can be applied to certain finite-time problem such that the time dependence can be separated and also has a simple analytical solution. The remaining state dependence is the solution to a linear eigenvalue problem that may have an analytical solution or is readily solved numerically. The efficiency of the method is illustrated by an inventory control problem.

On the infinite-time solution to state-constrained stochastic optimal control

A method is presented for solving the infinite time Hamilton-Jacobi-Bellman (HJB) equation for certain state-constrained stochastic problems. The HJB equation is reformulated as an eigenvalue problem, such that the principal eigenvalue corresponds to the expected cost per unit time, and the corresponding eigenfunction gives the value function (up to an additive constant) for the optimal control policy. The eigenvalue problem is linear and hence there are fast numerical methods available for finding the solution.

Brief paper: On the infinite time solution to state-constrained stochastic optimal control problems

A method is presented for solving the infinite time Hamilton-Jacobi-Bellman (HJB) equation for certain state-constrained stochastic problems. The HJB equation is reformulated as an eigenvalue problem, such that the principal eigenvalue corresponds to the expected cost per unit time, and the corresponding eigenfunction gives the value function (up to an additive constant) for the optimal control policy. The eigenvalue problem is linear and hence there are fast numerical methods available for finding the solution.