Alessandro Alla

An Efficient Policy Iteration Algorithm for Dynamic Programming Equations

We present an accelerated algorithm for the solution of static Hamilton--Jacobi--Bellman equations related to optimal control problems. Our scheme is based on a classic policy iteration procedure, which is known to have superlinear convergence in many relevant cases provided the initial guess is sufficiently close to the solution. This limitation often degenerates into a behavior similar to a value iteration method, with an increased computation time. The new scheme circumvents this problem by combining the advantages of both algorithms with an efficient coupling. The method starts with a coarse-mesh value iteration phase and then switches to a fine-mesh policy iteration procedure when a certain error threshold is reached. A delicate point is to determine this threshold in order to avoid cumbersome computations with the value iteration and at the same time to ensure the convergence of the policy iteration method to the optimal solution. We analyze the methods and efficient coupling in a number of examples...

Nonlinear Model Order Reduction via Dynamic Mode Decomposition

We propose a new technique for obtaining reduced order models for nonlinear dynamical systems. Specifically, we advocate the use of the recently developed Dynamic Mode Decomposition (DMD), an equation-free method, to approximate the nonlinear term. DMD is a spatio-temporal matrix decomposition of a data matrix that correlates spatial features while simultaneously associating the activity with periodic temporal behavior. With this decomposition, one can obtain a fully reduced dimensional surrogate model and avoid the evaluation of the nonlinear term in the online stage. This allows for an impressive speed up of the computational cost, and, at the same time, accurate approximations of the problem. We present a suite of numerical tests to illustrate our approach and to show the effectiveness of the method in comparison to existing approaches.

A Time-Adaptive POD Method for Optimal Control Problems

Abstract In this paper we present a time adaptive technique for the solution of optimal control problems where the dynamic is given by an evolutive semi linear PDE. The method is based on a model reduction using a POD approximation coupled with a Hamilton-Jacobi equation characterizing the value function of the corresponding control problem for the reduced system. The choice of the POD basis is updated according to the evaluation of a numerical indicator in order to guarantee a global accurate solution. This is obtained via a sub-division of the time horizon into sub-intervals where the residual is below a given threshold. Some numerical tests illustrate the main features of this approach.

Asymptotic stability of POD based model predictive control for a semilinear parabolic PDE

In this article a stabilizing feedback control is computed for a semilinear parabolic partial differential equation utilizing a nonlinear model predictive (NMPC) method. In each level of the NMPC algorithm the finite time horizon open loop problem is solved by a reduced-order strategy based on proper orthogonal decomposition (POD). A stability analysis is derived for the combined POD-NMPC algorithm so that the lengths of the finite time horizons are chosen in order to ensure the asymptotic stability of the computed feedback controls. The proposed method is successfully tested by numerical examples.

A residual based snapshot location strategy for POD in distributed optimal control of linear parabolic equations

In this paper we study the approximation of a distributed optimal control problem for linear para\-bolic PDEs with model order reduction based on Proper Orthogonal Decomposition (POD-MOR). POD-MOR is a Galerkin approach where the basis functions are obtained upon information contained in time snapshots of the parabolic PDE related to given input data. In the present work we show that for POD-MOR in optimal control of parabolic equations it is important to have knowledge about the controlled system at the right time instances. For the determination of the time instances (snapshot locations) we propose an a-posteriori error control concept which is based on a reformulation of the optimality system of the underlying optimal control problem as a second order in time and fourth order in space elliptic system which is approximated by a space-time finite element method. Finally, we present numerical tests to illustrate our approach and to show the effectiveness of the method in comparison to existing approaches.

HJB-POD Feedback Control for Navier-Stokes Equations

In this report we present the approximation of an infinite horizon optimal control problem for the evolutive Navier-Stokes system. The method is based on a model reduction technique, using a POD approximation, coupled with a Hamilton-Jacobi equation which characterizes the value function of the corresponding control problem for the reduced system. Although the approximation schemes available for the HJB are shown to be convergent for any dimension, in practice we need to restrict the dimension to rather small numbers and this limitation affects the accuracy of the POD approximation. We will present numerical tests for the control of the time-dependent Navier-Stokes system in two-dimensional spatial domains to illustrate our approach and to show the effectiveness of the method.

Model Order Reduction Approaches for Infinite Horizon Optimal Control Problems via the HJB Equation

We investigate feedback control for infinite horizon optimal control problems for partial differential equations. The method is based on the coupling between Hamilton-Jacobi-Bellman (HJB) equations and model reduction techniques. It is well-known that HJB equations suffer the so called curse of dimensionality and, therefore, a reduction of the dimension of the system is mandatory. In this report we focus on the infinite horizon optimal control problem with quadratic cost functionals. We compare several model reduction methods such as Proper Orthogonal Decomposition, Balanced Truncation and a new algebraic Riccati equation based approach. Finally, we present numerical examples and discuss several features of the different methods analyzing advantages and disadvantages of the reduction methods.

Error Analysis for POD Approximations of Infinite Horizon Problems via the Dynamic Programming Approach

In this paper infinite horizon optimal control problems for nonlinear high-dimensional dynamical systems are studied. Nonlinear feedback laws can be computed via the value function characterized as the unique viscosity solution to the corresponding Hamilton--Jacobi--Bellman (HJB) equation which stems from the dynamic programming approach. However, the bottleneck is mainly due to the curse of dimensionality, and HJB equations are solvable only in a relatively small dimension. Therefore, a reduced-order model is derived for the dynamical system, using the method of proper orthogonal decomposition (POD). The resulting errors in the HJB equations are estimated by an a priori error analysis, which is utilized in the numerical approximation to ensure a desired accuracy for the POD method. Numerical experiments illustrates the theoretical findings.

A HJB-POD Approach to the Control of the Level Set Equation

We consider an optimal control problem where the dynamics is given by the propagation of a one-dimensional graph controlled by its normal speed. A target corresponding to the final configuration of the front is given and we want to minimize the cost to reach the target. We want to solve this optimal control problem via the dynamic programming approach but it is well known that these methods suffer from the “curse of dimensionality” so that we can not apply the method to the semi-discrete version of the dynamical system. However, this is made possible by a reduced-order model for the level set equation which is based on Proper Orthogonal Decomposition. This results in a new low-dimensional dynamical system which is sufficient to track the dynamics. By the numerical solution of the Hamilton-Jacobi-Bellman equation related to the POD approximation we can compute the feedback law and the corresponding optimal trajectory for the nonlinear front propagation problem. We discuss some numerical issues of this approach and present a couple of numerical examples.

Coupling MPC and DP methods for an efficient solution of optimal control problems

We study the approximation of optimal control problems via the solution of a Hamilton-Jacobi equation in a tube around a reference trajectory which is first obtained solving a Model Predictive Control problem. The coupling between the two methods is introduced to improve the initial local solution and to reduce the computational complexity of the Dynamic Programming algorithm. We present some features of the method and show some results obtained via this technique showing that it can produce an improvement with respect to the two uncoupled methods.