Paul T. Boggs

On the Local Convergence of Quasi-Newton Methods for Constrained Optimization

We consider the application of a general class of quasi-Newton methods to the solution of the classical equality constrained nonlinear optimization problem. Specifically, we develop necessary and sufficient conditions for the Q-superlinear convergence of such methods and present a companion linear convergence theorem. The essential conditions relate to the manner in which the Hessian of the Lagrangian function is approximated.

Sequential quadratic programming for large-scale nonlinear optimization

The sequential quadratic programming (SQP) algorithm has been one of the most successful general methods for solving nonlinear constrained optimization problems. We provide an introduction to the general method and show its relationship to recent developments in interior-point approaches, emphasizing large-scale aspects.

Solution accelerators for large scale 3D electromagnetic inverse problems

We provide a framework for preconditioning nonlinear 3D electromagnetic inverse scattering problems using nonlinear conjugate gradient (NLCG) and limited memory (LM) quasi-Newton methods. Key to our approach is the use of an approximate adjoint method that allows for an economical approximation of the Hessian that is updated at each inversion iteration. Using this approximate Hessian as a preconditoner, we show that the preconditioned NLCG iteration converges significantly faster than the non-preconditioned iteration, as well as converging to a data misfit level below that observed for the non-preconditioned method. Similar conclusions are also observed for the LM iteration; preconditioned with the approximate Hessian, the LM iteration converges faster than the non-preconditioned version. At this time, however, we see little difference between the convergence performance of the preconditioned LM scheme and the preconditioned NLCG scheme. A possible reason for this outcome is the behavior of the line search within the LM iteration. It was anticipated that, near convergence, a step size of one would be approached, but what was observed, instead, were step lengths that were nowhere near one. We provide some insights into the reasons for this behavior and suggest further research that may improve the performance of the LM methods.

A Practical Algorithm for General Large Scale Nonlinear Optimization Problems

We provide an effective and efficient implementation of a sequential quadratic programming (SQP) algorithm for the general large scale nonlinear programming problem. In this algorithm the quadratic programming subproblems are solved by an interior point method that can be prematurely halted by a trust region constraint. Numerous computational enhancements to improve the numerical performance are presented. These include a dynamic procedure for adjusting the merit function parameter and procedures for adjusting the trust region radius. Numerical results and comparisons are presented.

A Global Convergence Analysis of an Algorithm for Large-Scale Nonlinear Optimization Problems

In this paper we give a global convergence analysis of a basic version of an SQP algorithm described in [P. T. Boggs, A. J. Kearsley, and J. W. Tolle, SIAM J. Optim., 9 (1999), pp. 755--778] for the solution of large-scale nonlinear inequality-constrained optimization problems. Several procedures and options have been added to the basic algorithm to improve the practical performance; some of these are also analyzed. The important features of the algorithm include the use of a constrained merit function to assess the progress of the iterates and a sequence of approximate merit functions that are less expensive to evaluate. It also employs an interior point quadratic programming solver that can be terminated early to produce a truncated step.

Algorithm 869: ODRPACK95: A weighted orthogonal distance regression code with bound constraints

ODRPACK (TOMS Algorithm 676) has provided a complete package for weighted orthogonal distance regression for many years. The code is complete with user selectable reporting facilities, numerical and analytic derivatives, derivative checking, and many more features. The foundation for the algorithm is a stable and efficient trust region Levenberg-Marquardt minimizer that exploits the structure of the orthogonal distance regression problem. ODRPACK95 was created to extend the functionality and usability of ODRPACK. ODRPACK95 adds bound constraints, uses the newer Fortran 95 language, and simplifies the interface to the user called subroutine.

A Family of Descent Functions for Constrained Optimization

In order to achieve a robust implementation of methods for nonlinear programming problems, it is necessary to devise a procedure which can be used to test whether or not a prospective step would yield a “better” approximation to the solution than the current iterate. In this paper, we present a family of descent or merit functions which are shown to be compatible with local Q-superlinear convergence of Newton and quasi-Newton methods. A simple algorithm is used to verify that good descent and convergence properties are possible using this merit function.

Large Scale Non-Linear Programming for PDE Constrained Optimization

Three years of large-scale PDE-constrained optimization research and development are summarized in this report. We have developed an optimization framework for 3 levels of SAND optimization and developed a powerful PDE prototyping tool. The optimization algorithms have been interfaced and tested on CVD problems using a chemically reacting fluid flow simulator resulting in an order of magnitude reduction in compute time over a black box method. Sandias simulation environment is reviewed by characterizing each discipline and identifying a possible target level of optimization. Because SAND algorithms are difficult to test on actual production codes, a symbolic simulator (Sundance) was developed and interfaced with a reduced-space sequential quadratic programming framework (rSQP++) to provide a PDE prototyping environment. The power of Sundance/rSQP++ is demonstrated by applying optimization to a series of different PDE-based problems. In addition, we show the merits of SAND methods by comparing seven levels of optimization for a source-inversion problem using Sundance and rSQP++. Algorithmic results are discussed for hierarchical control methods. The design of an interior point quadratic programming solver is presented.

Efficient structure-preserving model reduction for nonlinear mechanical systems with application to structural dynamics.

This work proposes a model-reduction methodology that both preserves Lagrangian structure and leads to computationally inexpensive models, even in the presence of high-order nonlinearities. We focus on parameterized simple mechanical systems under Rayleigh damping and external forces, as structural-dynamics models often t this description. The proposed model-reduction methodology directly approximates the quantities that dene the problem’s Lagrangian structure: the Riemannian metric, the potential-energy function, the dissipation function, and the external force. These approximations preserve salient properties (e.g., positive deniteness), behave similarly to the functions they approximate, and ensure computational eciency. Results applied to a simple parameterized trussstructure problem demonstrate the importance of preserving Lagrangian structure and illustrate the method’s ability to generate speedups while maintaining observed stability, in contrast with other model-reduction techniques that do not preserve structure.

Sundance: High-level software for PDE-constrained optimization

Sundance is a package in the Trilinos suite designed to provide high-level components for the development of high-performance PDE simulators with built-in capabilities for PDE-constrained optimization. We review the implications of PDE-constrained optimization on simulator design requirements, then survey the architecture of the Sundance problem specification components. These components allow immediate extension of a forward simulator for use in an optimization context. We show examples of the use of these components to develop full-space and reduced-space codes for linear and nonlinear PDE-constrained inverse problems.