Steffen Finck

Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009

This paper presents results of the BBOB-2009 benchmarking of 31 search algorithms on 24 noiseless functions in a black-box optimization scenario in continuous domain. The runtime of the algorithms, measured in number of function evaluations, is investigated and a connection between a single convergence graph and the runtime distribution is uncovered. Performance is investigated for different dimensions up to 40-D, for different target precision values, and in different subgroups of functions. Searching in larger dimension and multi-modal functions appears to be more difficult. The choice of the best algorithm also depends remarkably on the available budget of function evaluations.

On the Design of Constraint Covariance Matrix Self-Adaptation Evolution Strategies Including a Cardinality Constraint

This paper describes the algorithms engineering of a covariance matrix self-adaptation evolution strategy (CMSA-ES) for solving a mixed linear/nonlinear constrained optimization problem arising in portfolio optimization. While the feasible solution space is defined by the (probabilistic) simplex, the nonlinearity comes in by a cardinality constraint bounding the number of linear inequalities violated. This gives rise to a nonconvex optimization problem. The design is based on the CMSA-ES and relies on three specific techniques to fulfill the different constraints. The resulting algorithm is then thoroughly tested on a data set derived from time series data of the Dow Jones Index.

Noisy optimization: a theoretical strategy comparison of ES, EGS, SPSA & IF on the noisy sphere

This paper presents a performance comparison of 4 direct search strategies in continuous search spaces using the noisy sphere as test function. While the results of the Evolution Strategy (ES), Evolutionary Gradient Search (EGS), Simultaneous Perturbation Stochastic Approximation (SPSA) considered are already known from literature, Implicit Filtering (IF) as the fourth strategy is firstly analyzed in this paper. After a short review of ES, EGS, and SPSA, the derivation of the quality gain formula of IF is sketched. Using the results, a comparison of the strategies is performed that worked out the similarities and differences of the strategies.

Evolution on Trees: On the Design of an Evolution Strategy for Scenario-Based Multi-Period Portfolio Optimization under Transaction Costs

Abstract Scenario-based optimization is a problem class often occurring in finance, planning and control. While the standard approach is usually based on linear stochastic programming, this paper develops an Evolution Strategy (ES) that can be used to treat nonlinear planning problems arising from Value at Risk (VaR)-constraints and not necessarily proportional transaction costs. Due to the VaR-constraints the optimization problem is generally of non-convex type and its decision version is already NP-complete. The developed ES is the first algorithm in the field of evolutionary and swarm intelligence that tackles this kind of optimization problem. The algorithm design is based on the covariance matrix self-adaptation ES (CMSA-ES). The optimization is performed on scenario trees where in each node specific constraints (balance equations) must be fulfilled. In order to evaluate the performance of the ES proposed, instances of increasing problem hardness are considered. The application to the general case with nonlinear node constraints shows not only the potential of the ES designed, but also its limitations. The latter are basically determined by the high dimensionalities of the search spaces defined by the scenario trees.

HappyCat --- a simple function class where well-known direct search algorithms do fail

A new class of simple and scalable test functions for unconstrained real-parameter optimization will be proposed. Even though these functions have only one minimizer, they yet appear difficult to be optimized using standard state-of-the-art EAs such as CMA-ES, PSO, and DE. The test functions share properties observed when evolving at the edge of feasibility of constraint problems: while the step-sizes (or mutation strength) drops down exponentially fast, the EA is still far way from the minimizer giving rise to premature convergence. The design principles for this new function class, called HappyCat, will be explained. Furthermore, an idea for a new type of evolution strategy, the Ray-ES, will be outlined that might be able to tackle such problems.

Benchmarking SPSA on BBOB-2010 noiseless function testbed

This paper presents the result for Simultaneous Perturbation Stochastic Approximation (SPSA) on the BBOB 2010 noiseless testbed. SPSA is a stochastic gradient approximation strategy which uses random directions for the gradient estimate. The paper describes the steps performed by the strategy and the experimental setup. The chosen setup represents a rather basic variant of SPSA. Overall the strategy is able to solve 2 of the 24 test functions. For each test function at least one target level was reached for D = 3.

Weighted recombination evolution strategy on a class of PDQF's

This work is concerned with a weighted recombination method for Evolution Strategies (ES) on a class of positive definite quadratic forms (PDQF). In particular, the λopt-ES and the λopt-CSA-ES will be analyzed. A characteristic of both strategies is the use of weighted recombination of all offspring within an iteration step. After obtaining equations describing the evolutionary process, the weights and the progress rate for the λopt-ES will be derived. It is shown that the optimal mutation strength (step size) for the λopt-ES yields an asymptotic limit value of 2κ, where κ is an user-chosen rescaling factor. Afterwards the cumulative step-length adaptation (CSA) is analyzed to determine the target mutation strength (the mutation strength the strategy tries to reach by means of adaptation) and the actually attained mutation strength. For both the asymptotic values are obtained at √2κ. To justify the theoretical results, comparisons with simulations are presented.

Performance of the (µ/µ, λ)-σSA-ES on a class of PDQFs

This paper investigates the behavior of (µ/µI, λ)- σSA-ES on a class of positive definite quadratic forms. After introducing the fitness environment and the strategy, the self-adaptation mechanism is analyzed with the help of the self-adaptation response function. Afterward, the steady state of the strategy is analyzed. The dynamical equations for the expectation of the mutation strength σ and the localization parameter ζ will be derived. Building on that, the progress rate ϕ is analyzed and tuned by means of the learning parameter τ. An approximate formula for τopt, yielding locally maximal progress, is presented. Finally, the performance of the σSA-rule is compared with the performance of the cumulative step size adaptation rule, and a rough approximation for the expected runtime is presented.

On the performance of evolution strategies on noisy PDQFs: Progress rate analysis

This paper analyzes the behavior of the (mu/muI,lambda) ES on a class of noisy positive definite quadratic forms (PDQFs). First the equations for the normalized progress rates are derived and then analyzed for constant normalized noise strength and constant (non-normalized) noise strength. Since in the latter case the strategy is not able to reach the optimum, formulas for the final distances to the optimizer (steady state) are derived. The theoretical predictions are then compared with empirical results. In both noise cases the influence of the strategy parameters will be investigated. Further, the equipartition conjecture is used to provide an alternative derivation of the steady state distances in the case of vanishing mutation strength.

Benchmarking CMA-EGS on the BBOB 2010 noiseless function testbed

This paper describes the implementation and the results for CMA-EGS on the BBOB 2010 function testbed. The CMA-EGS is a hybrid strategy which combines elements from gradient search and evolutionary algorithms. The paper describes the algorithm used and the experimental setup. The strategy is able to solve 11 of 24 test functions for at least 5 of the 6 search space dimensionalities. For 4 test functions the target function value is not reached for at least one search space dimensionality.