I. D. Coope

Circle fitting by linear and nonlinear least squares

The problem of determining the circle of best fit to a set of points in the plane (or the obvious generalization ton-dimensions) is easily formulated as a nonlinear total least-squares problem which may be solved using a Gauss-Newton minimization algorithm. This straight-forward approach is shown to be inefficient and extremely sensitive to the presence of outliers. An alternative formulation allows the problem to be reduced to a linear least squares problem which is trivially solved. The recommended approach is shown to have the added advantage of being much less sensitive to outliers than the nonlinear least squares approach.

On the Convergence of Grid-Based Methods for Unconstrained Optimization

The convergence of direct search methods for unconstrained minimization is examined in the case where the underlying method can be interpreted as a grid or pattern search over successively refined meshes. An important aspect of the main convergence result is that translation, rotation, scaling, and shearing of the successive grids are allowed.

A convergent variant of the Nelder-Mead algorithm

The Nelder–Mead algorithm (1965) for unconstrained optimization has been used extensively to solve parameter estimation and other problems. Despite its age, it is still the method of choice for many practitioners in the fields of statistics, engineering, and the physical and medical sciences because it is easy to code and very easy to use. It belongs to a class of methods which do not require derivatives and which are often claimed to be robust for problems with discontinuities or where the function values are noisy. Recently (1998), it has been shown that the method can fail to converge or converge to nonsolutions on certain classes of problems. Only very limited convergence results exist for a restricted class of problems in one or two dimensions. In this paper, a provably convergent variant of the Nelder–Mead simplex method is presented and analyzed. Numerical results are included to show that the modified algorithm is effective in practice.

A projected lagrangian algorithm for semi-infinite programming

A globally convergent algorithm is presented for the solution of a wide class of semi-infinite programming problems. The method is based on the solution of a sequence of equality constrained quadratic programming problems, and usually has a second order convergence rate. Numerical results illustrating the method are given.

Frame based methods for unconstrained optimization

This paper describes a wide class of direct search methods for unconstrained optimization, which make use of fragments of grids called frames. Convergence is shown under mild conditions which allow successive frames to be rotated, translated, and scaled relative to one another.

Frames and Grids in Unconstrained and Linearly Constrained Optimization: A Nonsmooth Approach

This paper describes a class of frame-based direct search methods for unconstrained and linearly constrained optimization. A template is described and analyzed using Clarkes\break nonsmooth calculus. This provides a unified and simple approach to earlier results for grid- and frame-based methods, and also provides partial convergence results when the objective function is not smooth, undefined in some places, or both. The template also covers many new methods which combine elements of previous ideas using frames and grids. These new methods include grid-based simple descent algorithms which allow moving to points off the grid at every iteration and can automatically control the grid size, provided function values are available. The concept of a grid is also generalized to that of an admissible set, which allows sets, for example, with circular symmetries. The method is applied to linearly constrained problems using a simple barrier approach.

Positive Bases in Numerical Optimization

The theory of positive bases introduced by C. Davis in 1954 does not appear in most modern texts on linear algebra but has re-emerged in publications in optimization journals. In this paper some simple properties of this highly useful theory are highlighted and applied to both theoretical and practical aspects of the design and implementation of numerical algorithms for nonlinear optimization.

Numerical experiments in semi-infinite programming

A quasi-Newton algorithm for semi-infinite programming using an L∞ exact penalty function is described, and numerical results are presented. Comparisons with three Newton algorithms and one other quasi-Newton algorithm show that the algorithm is very promising in practice.

Some numerical experience with a globally convergent algorithm for nonlinearly constrained optimization

Global convergence properties are established for a quite general form of algorithms for solving nonlinearly constrained minimization problems. A useful feature of the methods considered is that they can be implemented easily either with or without using quadratic programming techniques. A particular implementation, designed to be both efficient and robust, is described in detail. Numerical results are presented and discussed.

Frame-Based Ray Search Algorithms in Unconstrained Optimization

This paper describes a class of frame-based direct search methods for unconstrained optimization without derivatives. A template for convergent direct search methods is developed, some requiring only the relative ordering of function values. At each iteration, the template considers a number of search steps which form a positive basis and conducts a ray search along a step giving adequate decrease. Various ray search strategies are possible, including discrete equivalents of the Goldstein–Armijo and one-sided Wolfe–Powell ray searches. Convergence is shown under mild conditions which allow successive frames to be rotated, translated, and scaled relative to one another.