Nicholas I. M. Gould

Trust-region methods

Preface 1. Introduction Part I. Preliminaries: 2. Basic Concepts 3. Basic Analysis and Optimality Conditions 4. Basic Linear Algebra 5. Krylov Subspace Methods Part II. Trust-Region Methods for Unconstrained Optimization: 6. Global Convergence of the Basic Algorithm 7.The Trust-Region Subproblem 8. Further Convergence Theory Issues 9. Conditional Models 10. Algorithmic Extensions 11. Nonsmooth Problems Part III. Trust-Region Methods for Constrained Optimization with Convex Constraints: 12. Projection Methods for Convex Constraints 13. Barrier Methods for Inequality Constraints Part IV. Trust-Region Mewthods for General Constained Optimization and Systems of Nonlinear Equations: 14. Penalty-Function Methods 15. Sequential Quadratic Programming Methods 16. Nonlinear Equations and Nonlinear Fitting Part V. Final Considerations: Practicalities Afterword Appendix: A Summary of Assumptions Annotated Bibliography Subject and Notation Index Author Index.

CUTE: constrained and unconstrained testing environment

The purpose of this article is to discuss the scope and functionality of a versatile environment for testing small- and large-scale nonlinear optimization algorithms. Although many of these facilities were originally produced by the authors in conjunction with the software package LANCELOT, we believe that they will be useful in their own right and should be available to researchers for their development of optimization software. The tools can be obtained by anonymous ftp from a number of sources and may, in many cases, be installed automatically. The scope of a major collection of test problems written in the standard input format (SIF) used by the LANCELOT software package is described. Recognizing that most software was not written with the SIF in mind, we provide tools to assist in building an interface between this input format and other optimization packages. These tools provide a link between the SIF and a number of existing packages, including MINOS and OSL. Additionally, as each problem includes a specific classification that is designed to be useful in identifying particular classes of problems, facilities are provided to build and manage a database of this information. There is a Unix and C shell bias to many of the descriptions in the article, since, for the sake of simplicity, we do not illustrate everything in its fullest generality. We trust that the majority of potential users are sufficiently familiar with Unix that these examples will not lead to undue confusion.

A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds

The global and local convergence properties of a class of augmented Lagrangian methods for solving nonlinear programming problems are considered. In such methods, simple bound constraints are treated separately from more general constraints and the stopping rules for the inner minimization algorithm have this in mind. Global convergence is proved, and it is established that a potentially troublesome penalty parameter is bounded away from zero.

CUTEr and SifDec: A constrained and unconstrained testing environment, revisited

The initial release of CUTE, a widely used testing environment for optimization software, was described by Bongartz, et al. [1995]. A new version, now known as CUTEr, is presented. Features include reorganisation of the environment to allow simultaneous multi-platform installation, new tools for, and interfaces to, optimization packages, and a considerably simplified and entirely automated installation procedure for unix systems. The environment is fully backward compatible with its predecessor, and offers support for Fortran 90/95 and a general C/C++ Application Programming Interface. The SIF decoder, formerly a part of CUTE, has become a separate tool, easily callable by various packages. It features simple extensions to the SIF test problem format and the generation of files suited to automatic differentiation packages.

Constraint Preconditioning for Indefinite Linear Systems

The problem of finding good preconditioners for the numerical solution of indefinite linear systems is considered. Special emphasis is put on preconditioners that have a 2 × 2 block structure and that incorporate the (1,2) and (2,1) blocks of the original matrix. Results concerning the spectrum and form of the eigenvectors of the preconditioned matrix and its minimum polynomial are given. The consequences of these results are considered for a variety of Krylov subspace methods. Numerical experiments validate these conclusions.

Global Convergence of a Trust-Region SQP-Filter Algorithm for General Nonlinear Programming

A trust-region SQP-filter algorithm of the type introduced by Fletcher and Leyffer [Math. Program., 91 (2002), pp. 239--269] that decomposes the step into its normal and tangential components allows for an approximate solution of the quadratic subproblem and incorporates the safeguarding tests described in Fletcher, Leyffer, and Toint [On the Global Convergence of an SLP-Filter Algorithm, Technical Report 98/13, Department of Mathematics, University of Namur, Namur, Belgium, 1998; On the Global Convergence of a Filter-SQP Algorithm, Technical Report 00/15, Department of Mathematics, University of Namur, Namur, Belgium, 2000] is considered. It is proved that, under reasonable conditions and for every possible choice of the starting point, the sequence of iterates has at least one first-order critical accumulation point.

Solving the Trust-Region Subproblem using the Lanczos Method

The approximate minimization of a quadratic function within an ellipsoidal trust region is an important subproblem for many nonlinear programming methods. When the number of variables is large, the most widely used strategy is to trace the path of conjugate gradient iterates either to convergence or until it reaches the trust-region boundary. In this paper, we investigate ways of continuing the process once the boundary has been encountered. The key is to observe that the trust-region problem within the currently generated Krylov subspace has a very special structure which enables it to be solved very efficiently. We compare the new strategy with existing methods. The resulting software package is available as HSL_VF05 within the Harwell Subroutine Library.

On the Solution of Equality Constrained Quadratic Programming Problems Arising in Optimization

We consider the application of the conjugate gradient method to the solution of large equality constrained quadratic programs arising in nonlinear optimization. Our approach is based implicitly on a reduced linear system and generates iterates in the null space of the constraints. Instead of computing a basis for this null space, we choose to work directly with the matrix of constraint gradients, computing projections into the null space by either a normal equations or an augmented system approach. Unfortunately, in practice such projections can result in significant rounding errors. We propose iterative refinement techniques, as well as an adaptive reformulation of the quadratic problem, that can greatly reduce these errors without incurring high computational overheads. Numerical results illustrating the efficacy of the proposed approaches are presented.

A numerical evaluation of sparse direct solvers for the solution of large sparse symmetric linear systems of equations

In recent years a number of solvers for the direct solution of large sparse symmetric linear systems of equations have been developed. These include solvers that are designed for the solution of positive definite systems as well as those that are principally intended for solving indefinite problems. In this study, we use performance profiles as a tool for evaluating and comparing the performance of serial sparse direct solvers on an extensive set of symmetric test problems taken from a range of practical applications.

A globally convergent Lagrangian barrier algorithm for optimization with general inequality constraints and simple bounds

We consider the global and local convergence properties of a class of Lagrangian barrier methods for solving nonlinear programming problems. In such methods, simple bound constraints may be treated separately from more general constraints. The objective and general constraint functions are combined in a Lagrangian barrier function. A sequence of such functions are approximately minimized within the domain defined by the simple bounds. Global convergence of the sequence of generated iterates to a first-order stationary point for the original problem is established. Furthermore, possible numerical difficulties associated with barrier function methods are avoided as it is shown that a potentially troublesome penalty parameter is bounded away from zero. This paper is a companion to previous work of ours on augmented Lagrangian methods.