Richard A. Waltz

Knitro: An Integrated Package for Nonlinear Optimization

This paper describes Knitro 5.0, a C-package for nonlinear optimization that combines complementary approaches to nonlinear optimization to achieve robust performance over a wide range of application requirements. The package is designed for solving large-scale, smooth nonlinear programming problems, and it is also effective for the following special cases: unconstrained optimization, nonlinear systems of equations, least squares, and linear and quadratic programming. Various algorithmic options are available, including two interior methods and an active-set method. The package provides crossover techniques between algorithmic options as well as automatic selection of options and settings.

An interior algorithm for nonlinear optimization that combines line search and trust region steps

Abstract.An interior-point method for nonlinear programming is presented. It enjoys the flexibility of switching between a line search method that computes steps by factoring the primal-dual equations and a trust region method that uses a conjugate gradient iteration. Steps computed by direct factorization are always tried first, but if they are deemed ineffective, a trust region iteration that guarantees progress toward stationarity is invoked. To demonstrate its effectiveness, the algorithm is implemented in the Knitro [6,28] software package and is extensively tested on a wide selection of test problems.

An algorithm for nonlinear optimization using linear programming and equality constrained subproblems

Abstract.This paper describes an active-set algorithm for large-scale nonlinear programming based on the successive linear programming method proposed by Fletcher and Sainz de la Maza [10]. The step computation is performed in two stages. In the first stage a linear program is solved to estimate the active set at the solution. The linear program is obtained by making a linear approximation to the ℓ1 penalty function inside a trust region. In the second stage, an equality constrained quadratic program (EQP) is solved involving only those constraints that are active at the solution of the linear program. The EQP incorporates a trust-region constraint and is solved (inexactly) by means of a projected conjugate gradient method. Numerical experiments are presented illustrating the performance of the algorithm on the CUTEr [1, 15] test set.

Steering exact penalty methods for nonlinear programming

This paper reviews, extends and analyses a new class of penalty methods for nonlinear optimization. These methods adjust the penalty parameter dynamically; by controlling the degree of linear feasibility achieved at every iteration, they promote balanced progress toward optimality and feasibility. In contrast with classical approaches, the choice of the penalty parameter ceases to be a heuristic and is determined, instead, by a subproblem with clearly defined objectives. The new penalty update strategy is presented in the context of sequential quadratic programming and sequential linear-quadratic programming methods that use trust regions to promote convergence. The paper concludes with a discussion of penalty parameters for merit functions used in line search methods.

On the Convergence of Successive Linear-Quadratic Programming Algorithms

The global convergence properties of a class of penalty methods for nonlinear programming are analyzed. These methods include successive linear programming approaches and, more specifically, the successive linear-quadratic programming approach presented by Byrd et al. [Math. Program., 100 (2004), pp. 27--48]. Every iteration requires the solution of two trust-region subproblems involving piecewise linear and quadratic models, respectively. It is shown that, for a fixed penalty parameter, the sequence of iterates approaches stationarity of the penalty function. A procedure for dynamically adjusting the penalty parameter is described, and global convergence results for it are established.

Adaptive Barrier Update Strategies for Nonlinear Interior Methods

This paper considers strategies for selecting the barrier parameter at every iteration of an interior-point method for nonlinear programming. Numerical experiments suggest that heuristic adaptive choices, such as Mehrotras probing procedure, outperform monotone strategies that hold the barrier parameter fixed until a barrier optimality test is satisfied. A new adaptive strategy is proposed based on the minimization of a quality function. The paper also proposes a globalization framework that ensures the convergence of adaptive interior methods, and examines convergence failures of the Mehrotra predictor-corrector algorithm. The barrier update strategies proposed in this paper are applicable to a wide class of interior methods and are tested in the two distinct algorithmic frameworks provided by the ipopt and knitro software packages.

Feasible Interior Methods Using Slacks for Nonlinear Optimization

A slack-based feasible interior point method is described which can be derived as a modification of infeasible methods. The modification is minor for most line search methods, but trust region methods require special attention. It is shown how the Cauchy point, which is often computed in trust region methods, must be modified so that the feasible method is effective for problems containing both equality and inequality constraints. The relationship between slack-based methods and traditional feasible methods is discussed. Numerical results using the KNITRO package show the relative performance of feasible versus infeasible interior point methods.

Assessing the Potential of Interior Methods for Nonlinear Optimization

A series of numerical experiments with interior point (LOQO, KNITRO) and active-set sequential quadratic programming (SNOPT, filterSQP) codes are reported and analyzed. The tests were performed with small, medium-size and moderately large problems, and are examined by problem classes. Detailed observations on the performance of the codes, and several suggestions on how to improve them are presented. Overall, interior methods appear to be strong competitors of act ive-set SQP methods, but all codes show much room for improvement.

An active-set algorithm for nonlinear programming using parametric linear programming

This paper describes an active-set algorithm for nonlinear programming that solves a parametric linear programming subproblem at each iteration to generate an estimate of the active set. A step is then computed by solving an equality constrained quadratic program based on this active-set estimate. This approach represents an extension to standard sequential linear-quadratic programming (SLQP) algorithms. It can also be viewed as an attempt to implement a generalization of the gradient projection algorithm for nonlinear programming. To this effect, we explore the relation between the parametric method and the gradient projection method in the bound-constrained case. Numerical results compare the performance of this algorithm with SLQP and gradient projection, and indicate good performance and marked improvement on bound-constrained problems and quadratic programs.

An interior point method for nonlinear programming with infeasibility detection capabilities

This paper describes an interior point method for nonlinear programming endowed with infeasibility detection capabilities. The method is composed of two phases, a main phase whose goal is to seek optimality, and a feasibility phase that aims exclusively at improving feasibility. An important feature of the algorithm is the use of a step-decomposition interior-point approach in which the step is the sum of a normal component and a tangential component. The normal component of the step provides detailed information that allows the algorithm to determine whether it should transition from the main phase to the feasibility phase. We give particular attention to the reliability of the switching mechanism between the two phases. The algorithm proposed in this paper has been implemented in the knitro package as extensions of the knitro/cg method. Numerical results illustrate the performance of our method on both feasible and infeasible problems.