William Baritompa

Global optimization requires global information

There are many global optimization algorithms which do not use global information. We broaden previous results, showing limitations on such algorithms, even if allowed to run forever. We show that deterministic algorithms must sample a dense set to find the global optimum value and can never be guaranteed to converge only to global optimizers. Further, analogous results show that introducing a stochastic element does not overcome these limitations. An example is simulated annealing in practice. Our results show that there are functions for which the probability of success is arbitrarily small.

Implementing Pure Adaptive Search with Grover's Quantum Algorithm

Pure adaptive search (PAS) is an idealized stochastic algorithm for unconstrained global optimization. The number of PAS iterations required to solve a problem increases only linearly in the domain dimension. However, each iteration requires the generation of a random domain point uniformly distributed in the current improving region. If no regularity conditions are known to hold for the objective function, then this task requires a number of classical function evaluations varying inversely with the proportion of the domain constituted by the improving region, entirely counteracting the PAS apparent speedup. The Grover quantum computational search algorithm provides a way to generate the PAS iterates. We show that the resulting implementation, which we call the Grover adaptive search (GAS), realizes PAS for functions satisfying certain conditions, and we believe that, when quantum computers will be available, GAS will be a practical algorithm.

Pure adaptive search for finite global optimization

Pure Adaptive Search is a stochastic algorithm which has been analyzed for continuous global optimization. When a uniform distribution is used in PAS, it has been shown to have complexity which is linear in dimension. We define strong and weak variations of PAS in the setting of finite global optimization and prove analogous results. In particular, for then-dimensional lattice {1,⋯,k}n, the expected number of iterations to find the global optimum is linear inn. Many discrete combinatorial optimization problems, although having intractably large domains, have quite small ranges. The strong version of PAS for all problems, and the weak version of PAS for a limited class of problems, has complexity the order of the size of the range.

Customizing methods for global optimization-a geometric viewpoint

A new class of global optimization algorithms, extending the multidimensional bisection method of Wood, is described geometrically. New results show how the geometry of the global minimum relates to performance. Remarkably, the epigraph of the objective function, turned upside down, plays a key role. Algorithms customized to take advantage of special information about the objective function belong to the class. A number of algorithms in the literature, including those of Piyavskii-Shubert, Mladineo, Wood and Breiman & Cutler, also belong, and simple modifications of them produce customized algorithms. Comparison of various algorithms in the class is provided.

Multidimensional bisection: The performance and the context

Two aspects of the multidimensional bisection algorithms for the global optimisation of Lipschitz continuous functions are investigated. Firstly, for several test functions we examine the numerical performance of the deepest point algorithm and two acceleration procedures. Secondly, we phrase the branch and bound framework of Horst and Tuy in terms of covers, and show the algorithms to be included in this framework. A result of Basso on the convergence of localisations is extended to higher dimensions.

Grover's Quantum Algorithm Applied to Global Optimization

Grovers quantum computational search procedure can provide the basis for implementing adaptive global optimization algorithms. A brief overview of the procedure is given and a framework called Grover adaptive search is set up. A method of Durr and Hoyer and one introduced by the authors fit into this framework and are compared.

Accelerations for a variety of global optimization methods

Optimization methods for a given class are easily modified to utilize additional information and work faster on a more restricted class. In particular algorithms that use only the Lipschitz constant (e.g. Mladineo, Piyavskii, Shubert and Wood) can be modified to use second derivative bounds or gradient calculations. The algorithm of Breiman & Cutler can be modified to use Lipschitz bounds. Test cases illustrating accelerations to various algorithms are provided.

Accelerations for global optimization covering methods using second derivatives

Two improvements for the algorithm of Breiman and Cutler are presented. Better envelopes can be built up using positive quadratic forms. Better utilization of first and second derivative information is attained by combining both global aspects of curvature and local aspects near the global optimum. The basis of the results is the geometric viewpoint developed by the first author and can be applied to a number of covering type methods. Improvements in convergence rates are demonstrated empirically on standard test functions.

Towards Pure Adaptive Search

The algorithm known as Pure Adaptive Search is a global optimisation ideal with desirable complexity. In this paper we temper it to a framework we term Somewhat Adaptive Search. This retains the desirable complexity, but allows scope for a practical realisation. We introduce a new algorithm termed Pure Localisation Search which attempts to reach the practical ideal. For a certain class of one variable functions the gap is bridged.

Equivalent Methods for Global Optimization

The envelope used by the algorithm of Breiman and Cutler [4] can be smoothed to create a better algorithm. This is equivalent to an accelerated algorithm developed by the third author and Cutler in [3] which uses apparently poor envelopes. Explaining this anomaly lead to a general result concerning the equivalence of methods which use information from more than one point at each stage and those that only use the most recent evaluated point. Smoothing is appropriate for many algorithms, and we show it is an optimal strategy.