Attila Kozma

A Parallel Active-Set Strategy to Solve Sparse Parametric Quadratic Programs arising in MPC

Abstract Many different approaches have been proposed for the efficient solution of quadratic programming (QP) problems arising in both linear and nonlinear model predictive control (MPC). This paper presents a novel online QP algorithm that aims at combining the respective advantages of existing methods. It allows for efficient hot-starts of the QP solution and exploits the parametric nature of the problem like other active-set methods. Moreover, like interior-point or fast gradient methods, it directly exploits the inherent sparsity of QP problems arising in MPC and is designed to be easily parallelizable. The proposed parallel active-set strategy is described in detail for MPC problems with diagonal weighting matrices that are subject to state and control bounds; also an extension to the general case is sketched. Numerical properties of the proposed algorithm are discussed and preliminary numerical results are given that are based on a prototype Matlab implementation.

Benchmarking large-scale distributed convex quadratic programming algorithms

This paper aims to collect, benchmark and implement state-of-the-art decomposable convex quadratic programming (QP) methods employing duality. In order to decouple the original problem, these methods relax some constraints by introducing dual variables and apply a hierarchical optimization scheme. In the lower level of this scheme, a sequence of parametric QPs is solved in parallel, while in the high-level problem, a gradient-based method is applied to achieve an optimal dual solution. Finding the optimal dual variables is a hard problem since the dual function is not twice continuously differentiable and not strongly convex. We investigate and compare several gradient-based methods using a set of convex QPs as benchmarks. We discuss the theoretical worst-case convergence properties of the investigated methods, but we also evaluate their practical convergence behaviour. The benchmark set as well as the suite of implemented algorithms are released as open-source software. From our experiments, it turns out that the alternating direction method of multipliers and the restarted version of the fast gradient method are the best methods for solving decomposable QPs in terms of the number of necessary, lower level QP solutions.

An improved Distributed Dual Newton-CG method for convex quadratic programming problems

This paper considers the problem of solving Programs (QP) arising in the context of distributed optimization and optimal control. A dual decomposition approach is used, where the QP subproblems are solved locally, while the constraints coupling the different subsystems in the time and space domains are enforced by performing a distributed non-smooth Newton iteration on the dual variables. The iterative linear algebra method Conjugate Gradient (CG) is used to compute the dual Newton step. In this context, it has been observed that the dual Hessian can be singular when a poor initial guess for the dual variables is used, hence leading to a failure of the linear algebra. This paper studies this effect and proposes a constraint relaxation strategy to address the problem. It is both formally and experimentally shown that the relaxation prevents the dual Hessian singularity. Moreover, numerical experiments suggest that the proposed relaxation improves significantly the convergence of the Distributed Dual Newton-CG.

A distributed method for convex quadratic programming problems arising in optimal control of distributed systems

We propose a distributed algorithm for strictly convex quadratic programming (QP) problems with a generic coupling topology. The coupling constraints are dualized via Lagrangian relaxation. This allows for a distributed evaluation of the non-smooth dual function and its derivatives. We propose to use both the gradient and the curvature information within a non-smooth variant of Newtons method to find the optimal dual variables. Our novel approach is designed such that the large Newton system never needs to be formed. Instead, we employ an iterative method to solve the Newton system in a distributed manner. The effectiveness of the method is demonstrated on an academic optimal control problem. A comparison with state-of-the-art first order dual methods is given.

Distributed Multiple Shooting for Optimal Control of Large Interconnected Systems

Abstract Large interconnected systems consist of a multitude of subsystems with their own dynamics, but coupled with each other via input-output connections. Each subsystem is typically modelled by ordinary differential equations or differential-algebraic equations. Simulation and optimal control of such systems pose a challenge both with respect to CPU time and memory requirements. We address optimal control of such systems by applying “distributed multiple shooting”, a generalization of the direct multiple shooting method, which uses the decomposable structure of the system in order to obtain a highly parallel algorithm. The interconnections are allowed to be infeasible during the iterations but are driven to feasibility by a Newtontype optimization algorithm. We evaluate the performance of the distributed multiple shooting method on a large scale estimation problem.

Distributed Multiple Shooting for Large Scale Nonlinear Systems

The distributed multiple shooting method is tailored for large scale optimal control problems with decoupled structure. It can be used as a fast and distributed solver for model predictive control subproblems. The algorithm may be regarded as a generalization of the standard multiple shooting method that decomposes the original large scale optimal control problem in both the time and spatial domain to obtain high parallelizability. In each iteration, the linearization of the original problem is calculated in a parallel manner, which is then solved by a centralized structure-exploiting optimizer. We demonstrate the approach on a simple mechanical example of two coupled pendula.

Adjoint-based distributed multiple shooting for large-scale systems

Abstract Distributed multiple shooting is a modification of the standard multiple shooting method which takes into account the structure of certain large-scale systems in order to obtain a better controller design flexibility and high parallelizability. The aim of this paper is to extend the framework where distributed multiple shooting can be deployed and to propose a new solution method based on adjoint-based sequential quadratic programming. A numerical experiment shows that this can lead to considerable savings in computational time for the sensitivity generation.

A sequential convex programming algorithm for minimizing a sum of Euclidean norms with non-convex constraints

Given and a finite set of convex polygons in , we consider the problem of finding the Euclidean shortest path starting at p then visiting the relative boundaries of the convex polygons in a given order, and ending at q. An approximate algorithm is proposed. The problem can be rewritten under a variant of minimizing a sum of Euclidean norms: , where and , subject to is on the relative boundary of , for . The objective function of the problem is convex but not everywhere differentiable and the constraints are non-convex. By using a smooth inner approximation of with parameter t, a relaxed form of the problem is constructed such that its solution, denoted by , is inside but outside the inner approximation. The relaxed problem is then solved iteratively using a sequential convex programming. The obtained solution , however, is actually not on the relative boundary of . Then a so-called refinement of is finally required to determine a solution passing through the relative boundary of , for . It is shown that the solution of the relaxed problem tends to its refined one as . The algorithm is implemented in Matlab using the CVX package. Numerical tests indicate that the solution obtained by the algorithm is close to the global one.

A compression algorithm for real-time distributed nonlinear MPC

Model Predictive Control (MPC) requires the online solution of an Optimal Control Problem (OCP) at each sampling time. Efficient online algorithms such as the Real-Time Iteration (RTI) scheme have been developed for real-time MPC implementations even for fast nonlinear dynamic systems. The RTI framework is based on direct Multiple Shooting (MS) for centralized systems. Distributed Multiple Shooting (DMS) is an MS-based OCP discretization strategy for distributed systems. Many fast dynamic systems can be described as connected subsystems and in order to exploit this structure, a DMS based RTI scheme has been developed and implemented in ACADO code generation. A novel technique called compression is proposed to reduce the dimensions of the convex subproblem, while exploiting the coupling structure. The performance of the presented scheme is illustrated on a nontrivial example from the literature, where a speedup of factor 11 in simulation time and factor 6 in the total computation time can be shown over the classical RTI scheme.

Linear convergence of distributed multiple shooting

Distributed multiple shooting is a modification of the multiple shooting approach for discretizing optimal control problems wherein the separate components of a large-scale system are discretized as well as shooting time intervals. In an SQP algorithm that solves the resulting discretized nonlinear program, the adjoint based version of the algorithm additionally discards certain derivatives appearing in the resulting quadratic programs in order to lead to computational savings in sensitivity generation and solving the QP. It was conjectured that adjoint-based distributed multiple shooting behaves like an inexact SQP method and converges linearly to the optimal solution, provided that the discarded derivatives are sufficiently uninfluential in the total dynamics of the system. This paper confirms this conjecture theoretically by providing the appropriate convergence theory, as well as numerically, by analyzing the convergence properties of the algorithm as applied to a problem involving detection of the source of smoke within a set of rooms.