Margaret H. Wright

Convergence Properties of the Nelder--Mead Simplex Method in Low Dimensions

The Nelder--Mead simplex algorithm, first published in 1965, is an enormously popular direct search method for multidimensional unconstrained minimization. Despite its widespread use, essentially no theoretical results have been proved explicitly for the Nelder--Mead algorithm. This paper presents convergence properties of the Nelder--Mead algorithm applied to strictly convex functions in dimensions 1 and 2. We prove convergence to a minimizer for dimension 1, and various limited convergence results for dimension 2. A counterexample of McKinnon gives a family of strictly convex functions in two dimensions and a set of initial conditions for which the Nelder--Mead algorithm converges to a nonminimizer. It is not yet known whether the Nelder--Mead method can be proved to converge to a minimizer for a more specialized class of convex functions in two dimensions.

Interior Methods for Nonlinear Optimization

Interior methods are an omnipresent, conspicuous feature of the constrained optimization landscape today, but it was not always so. Primarily in the form of barrier methods, interior-point techniques were popular during the 1960s for solving nonlinearly constrained problems. However, their use for linear programming was not even contemplated because of the total dominance of the simplex method. Vague but continuing anxiety about barrier methods eventually led to their abandonment in favor of newly emerging, apparently more efficient alternatives such as augmented Lagrangian and sequential quadratic programming methods. By the early 1980s, barrier methods were almost without exception regarded as a closed chapter in the history of optimization. This picture changed dramatically with Karmarkars widely publicized announcement in 1984 of a fast polynomial-time interior method for linear programming; in 1985, a formal connection was established between his method and classical barrier methods. Since then, interior methods have advanced so far, so fast, that their influence has transformed both the theory and practice of constrained optimization. This article provides a condensed, selective look at classical material and recent research about interior methods for nonlinearly constrained optimization.

On projected Newton barrier methods for linear programming and an equivalence to Karmarkar's projective method

Interest in linear programming has been intensified recently by Karmarkar’s publication in 1984 of an algorithm that is claimed to be much faster than the simplex method for practical problems. We review classical barrier-function methods for nonlinear programming based on applying a logarithmic transformation to inequality constraints. For the special case of linear programming, the transformed problem can be solved by a “projected Newton barrier” method. This method is shown to be equivalent to Karmarkar’s projective method for a particular choice of the barrier parameter. We then present details of a specific barrier algorithm and its practical implementation. Numerical results are given for several non-trivial test problems, and the implications for future developments in linear programming are discussed.

WISE design of indoor wireless systems: practical computation and optimization

Designing a low-power system for wireless communication within a building might seem simple. Not so-walls can affect signal strength in ways that are hard to calculate. The paper considers how AT&Ts WISE software uses CAD, computational geometry, and optimization to quickly plan where to place base-station transceivers. >

Interior methods for constrained optimization

Interior methods for optimization were widely used in the 1960s, primarily in the form of barrier methods. However, they were not seriously applied to linear programming because of the dominance of the simplex method. Barrier methods fell from favour during the 1970s for a variety of reasons, including their apparent inefficiency compared with the best available alternatives. In 1984, Karmarkars announcement of a fast polynomial-time interior method for linear programming caused tremendous excitement in the field of optimization. A formal connection can be shown between his method and classical barrier methods, which have consequently undergone a renaissance in interest and popularity. Most papers published since 1984 have concentrated on issues of computational complexity in interior methods for linear programming. During the same period, implementations of interior methods have displayed great efficiency in solving many large linear programs of ever-increasing size. Interior methods have also been applied with notable success to nonlinear and combinatorial problems. This paper presents a self-contained survey of major themes in both classical material and recent developments related to the theory and practice of interior methods.

Procedures for optimization problems with a mixture of bounds and general linear constraints

Abstract : When describing active-set methods for linearly constrained optimization, it is often convenient to treat all constraints in a uniform manner. However, in many problems the linear constraints include simple bounds on the variables as well as general constraints. Special treatment of bound constraints in the implementation of an active-set method yields significant advantages in computational effort and storage requirements. In this paper, we describe how to perform the constraint-related steps of an active-set method when the constraint matrix is dense and bounds are treated separately. These steps involve updates to the TQ factorization of the working set of constraints and the Cholesky factorization of the projected Hessian (or Hessian approximation). (Author)

The interior-point revolution in optimization: History, recent developments, and lasting consequences

Interior methods are a pervasive feature of the optimization landscape today, but it was not always so. Although interior-point techniques, primarily in the form of barrier methods, were widely used during the 1960s for problems with nonlinear constraints, their use for the fundamental problem of linear programming was unthinkable because of the total dominance of the simplex method. During the 1970s, barrier methods were superseded, nearly to the point of oblivion, by newly emerging and seemingly more efficient alternatives such as augmented Lagrangian and sequential quadratic programming methods. By the early 1980s, barrier methods were almost universally regarded as a closed chapter in the history of optimization. This picture changed dramatically in 1984, when Narendra Karmarkar announced a fast polynomial-time interior method for linear programming; in 1985, a formal connection was established between his method and classical barrier methods. Since then, interior methods have continued to transform both the theory and practice of constrained optimization. We present a condensed, unavoidably incomplete look at classical material and recent research about interior methods.

A Primal-dual Interior Method for Nonconvex Nonlinear Programming

Primal-dual interior methods for nonconvex nonlinear programming have recently been the subject of significant attention from the optimization community. Several different primal-dual methods have been suggested, along with some theoretical and numerical results. Although the underlying motivation for all of these methods is relatively uniform, there axe nonetheless substantive variations in crucial details, including the formulation of the nonlinear equations, the form of the associated linear system, the choice of linear algebraic procedures for solving this system, strategies for adjusting the barrier parameter and the Lagrange multiplier estimates, the merit function, and the treatment of indefiniteness. Not surprisingly, each of these choices can affect the theoretical properties and practical performance of the method. This paper discusses the approaches to these issues that we took in implementing a specific primal-dual method.

A practical anti-cycling procedure for linearly constrained optimization

A procedure is described for preventing cycling in active-set methods for linearly constrained optimization, including the simplex method. The key ideas are a limited acceptance of infeasibilities in all variables, and maintenance of a “working” feasibility tolerance that increases over a long sequence of iterations. The additional work per iteration is nominal, and “stalling” cannot occur with exact arithmetic. The method appears to be reliable, based on computational results for the first 53 linear programming problems in theNetlib set.

A deferred correction method for nonlinear two-point boundary value problems: implementation and numerical evaluation

A deferred correction method for the numerical solution of nonlinear two-point boundary value problems has been derived and analyzed in two recent papers by the first author. The method is based on mono-implicit Runge–Kutta formulas and is specially designed to deal efficiently with problems whose solutions contain nonsmooth parts—in particular, singular perturbation problems of boundary layer or turning point type. This paper briefly describes an implementation of the method and gives the results of extensive numerical testing on a set of nonlinear problems that includes both smooth and increasingly stiff (and difficult) problems. Results on the test set are also given using the available codes COLSYS and COLNEW. Although the intent is not to make a formal comparison, the code described appears to be competitive in speed and storage requirements on these problems.