Jouni Lampinen

A fuzzy adaptive differential evolution algorithm

Abstract.The differential evolution algorithm is a floating-point encoded evolutionary algorithm for global optimization over continuous spaces. The algorithm has so far used empirically chosen values for its search parameters that are kept fixed through an optimization process. The objective of this paper is to introduce a new version of the Differential Evolution algorithm with adaptive control parameters – the fuzzy adaptive differential evolution algorithm, which uses fuzzy logic controllers to adapt the search parameters for the mutation operation and crossover operation. The control inputs incorporate the relative objective function values and individuals of the successive generations. The emphasis of this paper is analysis of the dynamics and behavior of the algorithm. Experimental results, provided by the proposed algorithm for a set of standard test functions, outperformed those of the standard differential evolution algorithm for optimization problems with higher dimensionality.

A Trigonometric Mutation Operation to Differential Evolution

Previous studies have shown that differential evolution is an efficient, effective and robust evolutionary optimization method. However, the convergence rate of differential evolution in optimizing a computationally expensive objective function still does not meet all our requirements, and attempting to speed up DE is considered necessary. In this paper, a new local search operation, trigonometric mutation, is proposed and embedded into the differential evolution algorithm. This modification enables the algorithm to get a better trade-off between the convergence rate and the robustness. Thus it can be possible to increase the convergence velocity of the differential evolution algorithm and thereby obtain an acceptable solution with a lower number of objective function evaluations. Such an improvement can be advantageous in many real-world problems where the evaluation of a candidate solution is a computationally expensive operation and consequently finding the global optimum or a good sub-optimal solution with the original differential evolution algorithm is too time-consuming, or even impossible within the time available. In this article, the mechanism of the trigonometric mutation operation is presented and analyzed. The modified differential evolution algorithm is demonstrated in cases of two well-known test functions, and is further examined with two practical training problems of neural networks. The obtained numerical simulation results are providing empirical evidences on the efficiency and effectiveness of the proposed modified differential evolution algorithm.

Differential Evolution Training Algorithm for Feed-Forward Neural Networks

An evolutionary optimization method over continuous search spaces, differential evolution, has recently been successfully applied to real world and artificial optimization problems and proposed also for neural network training. However, differential evolution has not been comprehensively studied in the context of training neural network weights, i.e., how useful is differential evolution in finding the global optimum for expense of convergence speed. In this study, differential evolution has been analyzed as a candidate global optimization method for feed-forward neural networks. In comparison to gradient based methods, differential evolution seems not to provide any distinct advantage in terms of learning rate or solution quality. Differential evolution can rather be used in validation of reached optima and in the development of regularization terms and non-conventional transfer functions that do not necessarily provide gradient information.

GDE3: the third evolution step of generalized differential evolution

A developed version of generalized differential evolution, GDE3, is proposed. GDE3 is an extension of differential evolution (DE) for global optimization with an arbitrary number of objectives and constraints. In the case of a problem with a single objective and without constraints GDE3 falls back to the original DE. GDE3 improves earlier GDE versions in the case of multi-objective problems by giving a better distributed solution. Performance of GDE3 is demonstrated with a set of test problems and the results are compared with other methods

A constraint handling approach for the differential evolution algorithm

An extension for the differential evolution algorithm is proposed for handling nonlinear constraint functions. In comparison with the original algorithm, only the replacement criterion was modified for handling the constraints. In this article the proposed method is described and demonstrated by solving a suite of ten well-known test problems.

Constrained Real-Parameter Optimization with Generalized Differential Evolution

This paper presents results for the CEC 2006 Special Session on Constrained Real-Parameter Optimization where the Generalized Differential Evolution (GDE) has been used to solve given test problems. The given problems consist of 24 problems having one objective function and one or more in-/equality constraints. Almost all the problems were solvable in a given maximum number of solution candidate evaluations. The paper also shows how GDE actually needs lower number of function evaluations than usually required.

Ranking-Dominance and Many-Objective Optimization

An alternative relation to Pareto-dominance is studied. The relation is based on ranking a set of solutions according to each separate objective and an aggregation function to calculate a scalar fitness value for each solution. The relation is called as ranking-dominance and it tries to tackle the curse of dimensionality commonly observed in multi-objective optimization. Ranking-dominance can be used to sort a set of solutions even for a large number of objectives when the Pareto-dominance relation cannot distinguish solutions from one another anymore. This permits the search to advance even with a large number of objectives. Experimental results indicate that in some cases the selection based on ranking-dominance is able to advance the search towards the Pareto-front better than the selection based on Pareto-dominance. However, in some cases it is also possible that the search does not proceed into direction of the Pareto-front because the ranking-dominance relation permits deterioration of individual objectives. The results also show that when the number of objectives increases, the selection based on just Pareto-dominance without diversity maintenance is able to advance the search better than with diversity maintenance. Therefore, diversity maintenance connives at difficulties solving problems with a high number of objectives.

An Extension of Generalized Differential Evolution for Multi-objective Optimization with Constraints

In this paper an extension of Generalized Differential Evolution for constrained multi-objective (Pareto-)optimization is proposed. The proposed extension adds a mechanism for maintaining extent and distribution of the obtained non-dominated solutions approximating a Pareto front. The proposed extension is tested with a set of five benchmark multi-objective test problems and results are numerically compared to known global Pareto fronts and to results obtained with the elitist Non-Dominated Sorting Genetic Algorithm and Generalized Differential Evolution. Results show that the extension improves extent and distribution of solutions of Generalized Differential Evolution.

Performance assessment of Generalized Differential Evolution 3 (GDE3) with a given set of problems

This paper presents results for the CEC 2007 Special Session on Performance Assessment of Multi-Objective Optimization Algorithms where Generalized Differential Evolution 3 (GDE3) has been used to solve a given set of test problems. The set consist of 19 problems having two, three, or five objectives. Problems have different properties in the sense of separability, modality, and geometry of the Pareto-front. According to the results, a near optimal set of solutions was found in the majority of the problems. Rotated problems given caused more difficulty than the other problems. Performance metrics indicate that obtained approximation sets were even better than provided reference sets for many problems.

Multi-Constrained Nonlinear Optimization by the Differential Evolution Algorithm

In this article an extension for the Differential Evolution algorithm is proposed for handling nonlinear constraint functions. From the user point of view, the proposed method allows solving multi-constrained global optimization problems virtually as easily as unconstrained problems. User is not assumed to provide a feasible solution as a starting point for searching, as required by many other methods. Furthermore, the user is not required to set any penalty parameters, any weights for individual constraints, or any other additional search parameters, as in cases for most penalty function methods. In comparison with the original Differential Evolution algorithm, only the selection operation was modified with a new selection criteria for handling the constraint functions. The proposed method is demonstrated by solving a suite of seven well-known and difficult test problems.