M. Montaz Ali

Population set-based global optimization algorithms: some modifications and numerical studies

This paper studies the efficiency and robustness of some recent and well known population set-based direct search global optimization methods such as Controlled Random Search, Differential Evolution and the Genetic Algorithm. Some modifications are made to Differential Evolution and to the Genetic Algorithm to improve their efficiency and robustness. All methods are tested on two sets of test problems, one composed of easy but commonly used problems and the other of a number of relatively difficult problems.

Stochastic Global Optimization: Problem Classes and Solution Techniques

There is a lack of a representative set of test problems for comparing global optimization methods. To remedy this a classification of essentially unconstrained global optimization problems into unimodal, easy, moderately difficult, and difficult problems is proposed. The problem features giving this classification are the chance to miss the region of attraction of the global minimum, embeddedness of the global minimum, and the number of minimizers. The classification of some often used test problems are given and it is recognized that most of them are easy and some even unimodal. Global optimization solution techniques treated are global, local, and adaptive search and their use for tackling different classes of problems is discussed. The problem of fair comparison of methods is then adressed. Further possible components of a general global optimization tool based on the problem classes and solution techniques is presented.

Application of stochastic global optimization algorithms to practical problems

We describe global optimization problems from three different fields representing many-body potentials in physical chemistry, optimal control of a chemical reactor, and fitting a statistical model to empirical data. Historical background for each of the problems as well as the practical significance of the first two are given. The problems are solved by using eight recently developed stochastic global optimization algorithms representing controlled random search (4 algorithms), simulated annealing (2 algorithms), and clustering (2 algorithms). The results are discussed, and the importance of global optimization in each respective field is focused.

A Numerical Comparison of Some Modified Controlled Random Search Algorithms

In this paper we propose a new version of the Controlled Random Search(CRS) algorithm of Price. The new algorithmhas been tested on thirteen global optimization test problems. Numericalexperiments indicate that the resulting algorithm performs considerablybetter than the earlier versions of the CRS algorithms. The algorithm,therefore, could offer a reasonable alternative to many currently availablestochastic algorithms, especially for problems requiring ’direct search‘type methods. Also a classification of the CRS algorithms is made based on’global technique‘ – ’local technique‘ and the relative performance ofclasses is numerically explored.

Differential evolution with preferential crossover

We study the mutation operation of the differential evolution algorithm. In particular, we study the effect of the scaling parameter of the differential vector in mutation. We derive the probability density function of points generated by mutation and thereby identify some drawbacks of the scaling parameter. We also visualize the drawbacks using simulation. We then propose a crossover rule, called the preferential crossover rule, to reduce the drawbacks. The preferential crossover rule uses points from an auxiliary population set. We also introduce a variable scaling parameter in mutation. Motivations for these changes are provided. A numerical study is carried out using 50 test problems, many of which are inspired by practical applications. Numerical results suggest that the proposed modification reduces the number of function evaluations and cpu time considerably.

Modified controlled random search algorithms

Some modifications are suggested to the controlled random search algorithm for global optimization. Numerical experiments indicate that the resulting algorithms are considerably better than the originals and offer a reasonable alternative to many currently available stochastic algorithms, especially for problems requiring ‘direct search’ type methods.

Topographical Multilevel Single Linkage

An iterative topographical Multilevel Single Linkage (TMSL) method has been introduced. The approach uses topographical information on the objective function, in particular theg-nearest-neighbour graph. The algorithm uses evenly distributed points from a Halten sequence of uniform limiting density. We discuss the implementation of the algorithm and compare its performance with other well-known algorithms. The new algorithm performs much better (in some cases several times) than the Multilevel Single Linkage method in terms of number of function evaluations but is not quite so competitive with respect to CPU time.

A direct search variant of the simulated annealing algorithm for optimization involving continuous variables

Abstract A memory-based simulated annealing algorithm is proposed which fundamentally differs from the previously developed simulated annealing algorithms for continuous variables by the fact that a set of points rather than a single working point is used. The implementation of the new method does not need differentiability properties of the function being optimized. The method is well tested on a range of problems classified as easy, moderately difficult and difficult. The new algorithm is compared with other simulated annealing methods on both test problems and practical problems. Results showing an improved performance in finding the global minimum are given. Scope and purpose The inherent difficulty of global optimization problems lies in finding the very best optimum (maximum or minimum) from a multitude of local optima. Many practical global optimization problems of continuous variables are non-differentiable and noisy and even the function evaluation may involve simulation of some process. For such optimization problems direct search approaches are the methods of choice. Simulated annealing is a stochastic global optimization algorithm, initially designed for combinatorial (discrete) optimization problems. The algorithm that we propose here is a simulated annealing algorithm for optimization problems involving continuous variables. It is a direct search method. The strengths of the new algorithm are: it does not require differentiability or any other properties of the function being optimized and it is memory-based. Therefore, the algorithm can be applied to noisy and/or not exactly known functions. Although the algorithm is stochastic in nature, it can memorise the best solution. The new simulated annealing algorithm has been shown to be reliable, fast, general purpose and efficient for solving some difficult global optimization problems.

A local exploration-based differential evolution algorithm for constrained global optimization

Abstract We propose a modified differential evolution (DE) algorithm for constrained global optimization. The modification is based on the mutation rule of DE. The new algorithm also incorporates a periodic local exploration technique. The local technique used is a ‘limited’ version of the pattern search (PS) method. The penalty functions such as the superiority of feasible points (SFP) and the parameter free penalty (PFP) are used for handling constraints. We numerically study SFP and PFP and based on a drawback observed, we adapt the selection rule of DE. The new algorithm is tested on 45 test problems. Comparisons are made with some recent algorithms.

An electromagnetism-like method for nonlinearly constrained global optimization

We propose an electromagnetism-like (EM) method for constrained global optimization. The method is a modified version of the unconstrained EM method. We introduce the charge calculation of a point based on both the function value and the total constraint violations. Hence, the calculation of the total force vector is different from the original EM method. The new method is not penalty function-based and therefore the difficulty with the choice of the penalty parameter value does not arise. We have tested our method on a set of 13 benchmark test problems. Results obtained are compared with those from some recent algorithms. The comparisons show that our proposed method is suitable for solving constrained optimization problems.