C. T. Kelley

A Locally-Biased form of the DIRECT Algorithm

In this paper we propose a form of the DIRECT algorithm that is strongly biased toward local search. This form should do well for small problems with a single global minimizer and only a few local minimizers. We motivate our formulation with some results on how the original formulation of the DIRECT algorithm clusters its search near a global minimizer. We report on the performance of our algorithm on a suite of test problems and observe that the algorithm performs particularly well when termination is based on a budget of function evaluations.

Convergence Analysis of Pseudo-Transient Continuation

Pseudo-transient continuation (

Detection and Remediation of Stagnation in the Nelder--Mead Algorithm Using a Sufficient Decrease Condition

\Psi

An Implicit Filtering Algorithm for Optimization of Functions with Many Local Minima

tc) is a well-known and physically motivated technique for computation of steady state solutions of time-dependent partial differential equations. Standard globalization strategies such as line search or trust region methods often stagnate at local minima. \ptc succeeds in many of these cases by taking advantage of the underlying PDE structure of the problem. Though widely employed, the convergence of \ptc is rarely discussed. In this paper we prove convergence for a generic form of \ptc and illustrate it with two practical strategies.

Optimal design for problems involving flow and transport phenomena in saturated subsurface systems

The Nelder--Mead algorithm can stagnate and converge to a nonoptimal point, even for very simple problems. In this note we propose a test for sufficient decrease which, if passed for all iterations, will guarantee convergence of the Nelder--Mead iteration to a stationary point if the objective function is smooth and the diameters of the Nelder--Mead simplices converge to zero. Failure of this condition is an indicator of potential stagnation. As a remedy we propose a new step, which we call an oriented restart, that reinitializes the simplex to a smaller one with orthogonal edges whose orientation is determined by an approximate descent direction from the current best point. We also give results that apply when the objective function is a low-amplitude perturbation of a smooth function. We illustrate our results with some numerical examples.

Convergence of iterative split-operator approaches for approximating nonlinear reactive transport problems

In this paper we describe and analyze an algorithm for certain box constrained optimization problems that may have several local minima. A paradigm for these problems is one in which the function to be minimized is the sum of a simple function, such as a convex quadratic, and high frequency, low amplitude terms that cause local minima away from the global minimum of the simple function. Our method is gradient based and therefore the performance can be improved by use of quasi-Newton methods.

Newton’s Method at Singular Points. I

Estimation problems arise routinely in subsurface hydrology for applications that range from water resources management to water quality protection to subsurface restoration. Interest in optimal design of such systems has increased over the last two decades and this area is considered an important and active area of research. In this work, we review the state of the art, assess important challenges that must be resolved to reach a mature level of understanding, and summarize some promising approaches that might help meet some of the challenges. While much has been accomplished to date, we conclude that more work remains before comprehensive, efficient, and robust solution methods exist to solve the most challenging applications in subsurface science. We suggest that future directions of research include the application of direct search solution methods, and developments in stochastic and multi-objective optimization. We present a set of comprehensive test problems for use in the research community as a means for benchmarking and comparing optimization approaches.

GMREs and the minimal polynomial

Numerical solutions to nonlinear reactive solute transport problems are often computed using split-operator (SO) approaches, which separate the transport and reaction processes. This uncoupling introduces an additional source of numerical error, known as the splitting error. The iterative split-operator (ISO) algorithm removes the splitting error through iteration. Although the ISO algorithm is often used, there has been very little analysis of its convergence behavior. This work uses theoretical analysis and numerical experiments to investigate the convergence rate of the iterative split-operator approach for solving nonlinear reactive transport problems.

Algorithms for Noisy Problems in Gas Transmission Pipeline Optimization

A theorem due to Reddien giving sufficient conditions for convergence of Newton iterates for singular problems is extended.

Additive Scaling and the DIRECT Algorithm

We present a qualitative model for the convergence behaviour of the Generalised Minimal Residual (GMRES) method for solving nonsingular systems of linear equationsAx =b in finite and infinite dimensional spaces. One application of our methods is the solution of discretised infinite dimensional problems, such as integral equations, where the constants in the asymptotic bounds are independent of the mesh size.Our model provides simple, general bounds that explain the convergence of GMRES as follows: If the eigenvalues ofA consist of a single cluster plus outliers then the convergence factor is bounded by the cluster radius, while the asymptotic error constant reflects the non-normality ofA and the distance of the outliers from the cluster. If the eigenvalues ofA consist of several close clusters, then GMRES treats the clusters as a single big cluster, and the convergence factor is the radius of this big cluster. We exhibit matrices for which these bounds are tight.Our bounds also lead to a simpler proof of existing r-superlinear convergence results in Hilbert space.