Dirk V. Arnold

Improving Evolution Strategies through Active Covariance Matrix Adaptation

This paper proposes a novel modification to the derandomised covariance matrix adaptation algorithm used in connection with evolution strategies. In existing variants of that algorithm, only information gathered from successful offspring candidate solutions contributes to the adaptation of the covariance matrix, while old information passively decays. We propose to use information about unsuccessful offspring candidate solutions in order to actively reduce variances of the mutation distribution in unpromising directions of the search space. The resulting strategy is referred to as Active-CMA-ES. In experiments on a standard suite of test functions, Active-CMA-ES consistently outperforms other strategy variants.

A Comparison of Evolution Strategies with Other Direct Search Methods in the Presence of Noise

Evolution strategies are general, nature-inspired heuristics for search and optimization. Due to their use of populations of candidate solutions and their advanced adaptation schemes, there is a common belief that evolution strategies are especially useful for optimization in the presence of noise. Empirical evidence as well as a number of theoretical findings with respect to the performance of evolution strategies on a class of spherical objective functions disturbed by Gaussian noise support that belief. However, little is known with respect to the capabilities in the presence of noise of evolution strategies relative to those of other direct optimization strategies.In the present paper, theoretical results with respect to the performance of evolution strategies in the presence of Gaussian noise are summarized and discussed. Then, the performance of evolution strategies is compared empirically with that of several other direct optimizationstrategies in the noisy, spherical environment that the theoretical results have been obtained in. Due to the simplicity of that environment, the results are easily interpretable and can serve to reveal the respective strengths and weaknesses of the algorithms. It is seen that for low levels of noise, most of the strategies exhibit similar degrees of efficiency. For higher levels of noise, their step length adaptation scheme affords evolution strategies a greater degree of robustness than the other algorithms tested.

Local performance of the (1 + 1)-ES in a noisy environment

While noise is a phenomenon present in many real world optimization problems, the understanding of its potential effects on the performance of evolutionary algorithms is still incomplete. This paper investigates the effects of fitness proportionate Gaussian noise for a (1 + 1)-ES with isotropic normal mutations on the quadratic sphere in the limit of infinite search-space dimensionality. It is demonstrated experimentally that the results provide a good approximation for finite space dimensionality. It is shown that overvaluation as a result of failure to re-evaluate parental fitness leads to both reduced success probabilities and improved performance. Implications for mutation strength adaptation rules are discussed and optimal re-sampling rates are computed.

A general noise model and its effects on evolution strategy performance

Most studies concerned with the effects of noise on the performance of optimization strategies, in general, and on evolutionary approaches, in particular, have assumed a Gaussian noise model. However, practical optimization strategies frequently face situations where the noise is not Gaussian. Noise distributions may be skew or biased, and outliers may be present. The effects of non-Gaussian noise are largely unexplored, and it is unclear whether the insights gained and the recommendations with regard to the sizing of strategy parameters that have been made under the assumption of Gaussian noise bear relevance to more general situations. In this paper, the behavior of a powerful class of recombinative evolution strategies is studied on the sphere model under the assumption of a very general noise model. A performance law is derived, its implications are studied both analytically and numerically, and comparisons with the case of Gaussian noise are drawn. It is seen that while overall, the assumption of Gaussian noise in previous studies is less severe than might have been expected, some significant differences do arise when considering noise that is of unbounded variance, skew, or biased

A (1+1)-CMA-ES for constrained optimisation

This paper introduces a novel constraint handling approach for covariance matrix adaptation evolution strategies (CMA-ES). The key idea is to approximate the directions of the local normal vectors of the constraint boundaries by accumulating steps that violate the respective constraints, and to then reduce variances of the mutation distribution in those directions. The resulting strategy is able to approach the boundary of the feasible region without being impeded in its ability to search in directions tangential to the boundaries. The approach is implemented in the (1+1)-CMA-ES and evaluated numerically on several test problems. The results compare very favourably with data for other constraint handling approaches applied to unimodal test problems that can be found in the literature.

Performance analysis of evolutionary optimization with cumulative step length adaptation

Iterative algorithms for continuous numerical optimization typically need to adapt their step lengths in the course of the search. While some strategies employ fixed schedules, others attempt to adapt dynamically in response to the outcome of trial steps or the history of the search process. Evolutionary algorithms are of the latter kind. A control strategy that is commonly used in evolution strategies is the cumulative step length adaptation approach. This paper presents a theoretical analysis of that adaptation strategy. The analysis includes the practically relevant case of noise interfering in the optimization process. Recommendations are made with respect to choosing appropriate population sizes.

Weighted multirecombination evolution strategies

Weighted recombination is a means for improving the local search performance of evolution strategies. It aims to make effective use of the information available, without significantly increasing computational costs per time step. In this paper, the potential speed-up resulting from using rank-based weighted multirecombination is investigated. Optimal weights are computed for the infinite-dimensional sphere model, and comparisons with the performance of strategies that do not make use of weighted recombination are presented. It is seen that unlike strategies that rely on unweighted recombination and truncation selection, weighted multirecombination evolution strategies are able to improve on the serial efficiency of the (1 + 1)-ES on the sphere. The implications of the use of weighted recombination for noisy optimization are studied, and parallels to the use of rescaled mutations are drawn. The significance of the findings is investigated in finite-dimensional search spaces.

Optimal weighted recombination

Weighted recombination is a means for improving the local search performance of evolution strategies. It aims to make effective use of the information available, without significantly increasing computational costs per time step. In this paper, the potential speed-up resulting from using rank-based weighted recombination is investigated. Optimal weights are computed for the sphere model, and comparisons with the performance of strategies that do not make use of weighted recombination are presented. It is seen that unlike strategies that rely on unweighted recombination and truncation selection, weighted multirecombination evolution strategies are able to improve on the serial efficiency of the (1+1)-ES on the sphere. The implications of the use of weighted recombination for noisy optimization are studied, and parallels to the use of rescaled mutations are drawn. The cumulative step length adaptation mechanism is formulated for the case of an optimally weighted evolution strategy, and its performance is analyzed.

Qualms regarding the optimality of cumulative path length control in CSA/CMA-evolution strategies

Cumulative step-size adaptation (CSA) based on path length control is regarded as a robust alternative to the standard mutative self-adaptation technique in evolution strategies (ES), guaranteeing an almost optimal control of the mutation operator. This paper shows that the underlying basic assumption in CSA the perpendicularity of expected consecutive steps does not necessarily guarantee optimal progress performance for (/I) intermediate recombinative ES

Random Dynamics Optimum Tracking with Evolution Strategies

Dynamic optimization is frequently cited as a prime application area for evolutionary algorithms. In contrast to static optimization, the objective in dynamic optimization is to continuously adapt the solution to a changing environment - a task that evolutionary algorithms are believed to be good at. At the time being, however, almost all knowledge with regard to the performance of evolutionary algorithms in dynamic environments is of an empirical nature. In this paper, tools devised originally for the analysis in static environments are applied to study the performance of a popular type of recombinative evolution strategy with cumulative mutation strength adaptation on a dynamic problem. With relatively little effort, scaling laws that quite accurately describe the behavior of the strategy and that greatly contribute to its understanding are derived and their implications are discussed.