Sven Leyffer

Nonlinear programming without a penalty function

Abstract.In this paper the solution of nonlinear programming problems by a Sequential Quadratic Programming (SQP) trust-region algorithm is considered. The aim of the present work is to promote global convergence without the need to use a penalty function. Instead, a new concept of a “filter” is introduced which allows a step to be accepted if it reduces either the objective function or the constraint violation function. Numerical tests on a wide range of test problems are very encouraging and the new algorithm compares favourably with LANCELOT and an implementation of Sl1QP.

Solving mixed integer nonlinear programs by outer approximation

A wide range of optimization problems arising from engineering applications can be formulated as Mixed Integer NonLinear Programming problems (MINLPs). Duran and Grossmann (1986) suggest an outer approximation scheme for solving a class of MINLPs that are linear in the integer variables by a finite sequence of relaxed MILP master programs and NLP subproblems.Their idea is generalized by treating nonlinearities in the integer variables directly, which allows a much wider class of problem to be tackled, including the case of pure INLPs. A new and more simple proof of finite termination is given and a rigorous treatment of infeasible NLP subproblems is presented which includes all the common methods for resolving infeasibility in Phase I.The worst case performance of the outer approximation algorithm is investigated and an example is given for which it visits all integer assignments. This behaviour leads us to include curvature information into the relaxed MILP master problem, giving rise to a new quadratic outer approximation algorithm.An alternative approach is considered to the difficulties caused by infeasibility in outer approximation, in which exact penalty functions are used to solve the NLP subproblems. It is possible to develop the theory in an elegant way for a large class of nonsmooth MINLPs based on the use of convex composite functions and subdifferentials, although an interpretation for thel1 norm is also given.

On the Global Convergence of a Filter--SQP Algorithm

A mechanism for proving global convergence in SQP--filter methods for nonlinear programming (NLP) is described. Such methods are characterized by their use of the dominance concept of multiobjective optimization, instead of a penalty parameter whose adjustment can be problematic. The main point of interest is to demonstrate how convergence for NLP can be induced without forcing sufficient descent in a penalty-type merit function. The proof relates to a prototypical algorithm, within which is allowed a range of specific algorithm choices associated with the Hessian matrix representation, updating the trust region radius, and feasibility restoration.

Global Convergence of a Trust-Region SQP-Filter Algorithm for General Nonlinear Programming

A trust-region SQP-filter algorithm of the type introduced by Fletcher and Leyffer [Math. Program., 91 (2002), pp. 239--269] that decomposes the step into its normal and tangential components allows for an approximate solution of the quadratic subproblem and incorporates the safeguarding tests described in Fletcher, Leyffer, and Toint [On the Global Convergence of an SLP-Filter Algorithm, Technical Report 98/13, Department of Mathematics, University of Namur, Namur, Belgium, 1998; On the Global Convergence of a Filter-SQP Algorithm, Technical Report 00/15, Department of Mathematics, University of Namur, Namur, Belgium, 2000] is considered. It is proved that, under reasonable conditions and for every possible choice of the starting point, the sequence of iterates has at least one first-order critical accumulation point.

Integrating SQP and Branch-and-Bound for Mixed Integer Nonlinear Programming

This paper considers the solution of Mixed Integer Nonlinear Programming (MINLP) problems. Classical methods for the solution of MINLP problems decompose the problem by separating the nonlinear part from the integer part. This approach is largely due to the existence of packaged software for solving Nonlinear Programming (NLP) and Mixed Integer Linear Programming problems.In contrast, an integrated approach to solving MINLP problems is considered here. This new algorithm is based on branch-and-bound, but does not require the NLP problem at each node to be solved to optimality. Instead, branching is allowed after each iteration of the NLP solver. In this way, the nonlinear part of the MINLP problem is solved whilst searching the tree. The nonlinear solver that is considered in this paper is a Sequential Quadratic Programming solver.A numerical comparison of the new method with nonlinear branch-and-bound is presented and a factor of up to 3 improvement over branch-and-bound is observed.

Mixed Integer Nonlinear Programming

Many engineering, operations, and scientific applications include a mixture of discrete and continuous decision variables and nonlinear relationships involving the decision variables that have a pronounced effect on the set of feasible and optimal solutions. Mixed-integer nonlinear programming (MINLP) problems combine the numerical difficulties of handling nonlinear functions with the challenge of optimizing in the context of nonconvex functions and discrete variables. MINLP is one of the most flexible modeling paradigms available for optimization; but because its scope is so broad, in the most general cases it is hopelessly intractable. Nonetheless, an expanding body of researchers and practitioners including chemical engineers, operations researchers, industrial engineers, mechanical engineers, economists, statisticians, computer scientists, operations managers, and mathematical programmers are interested in solving large-scale MINLP instances.

Numerical Experience with Lower Bounds for MIQP Branch-And-Bound

The solution of convex mixed-integer quadratic programming (MIQP) problems with a general branch-and-bound framework is considered. It is shown how lower bounds can be computed efficiently during the branch-and-bound process. Improved lower bounds such as the ones derived in this paper can reduce the number of quadratic programming (QP) problems that have to be solved. The branch-and-bound approach is also shown to be superior to other approaches in solving MIQP problems. Numerical experience is presented which supports these conclusions.

Filter-type Algorithms for Solving Systems of Algebraic Equations and Inequalities

The problem of solving a nonlinear system is transformed into a bi-objective nonlinear programming problem, which is then solved by a prototypical trust region filter SQP algorithm. The definition of the bi-objective problems is changed adaptively as the algorithm proceeds. The method permits the use of second order information and hence enables rapid local convergence to occur, which is particularly important for solving locally infeasible problems. A proof of global convergence is presented under mild assumptions.

Comparison of certain MINLP algorithms when applied to a model structure determination and parameter estimation problem

Abstract The maximum likelihood method is frequently used in parameter estimation. If the structure of the model is unknown, the maximization of the likelihood function can be replaced by minimizing an information criterion. One criterion that allows this to be done is Akaike’s information criterion (AIC). Minimizing the AIC is a mixed integer non-linear programming (MINLP) problem. In this paper, three different MINLP algorithms are compared in the solution of a simultaneous model structure determination and parameter estimation problem by minimizing the AIC criterion. The problem considered appears in quantitative Fourier transformed infra red (FTIR) spectroscopy where concentration estimates of certain gas components are to be obtained from measured absorbances at different wave numbers. The resulting problem is a large MINLP problem containing several hundreds, or even thousands, of variables including a huge number of possible model structures. It is, however, found that the studied algorithms solve the considered problem in quite a small number of iterations and a reasonable CPU-time.

An Active-Set Method for Second-Order Conic-Constrained Quadratic Programming

We consider the minimization of a convex quadratic objective subject to second-order cone constraints. This problem generalizes the well-studied bound-constrained quadratic programming (QP) problem. We propose a new two-phase method: in the first phase a projected-gradient method is used to quickly identify the active set of cones, and in the second-phase Newtons method is applied to rapidly converge given the subsystem of active cones. Computational experiments confirm that the conically constrained QP is solved more efficiently by our method than by a specialized conic optimization solver and more robustly than by general nonlinear programming solvers.