Michael Kommenda

Architecture and Design of the HeuristicLab Optimization Environment

Many optimization problems cannot be solved by classical mathematical optimization techniques due to their complexity and the size of the solution space. In order to achieve solutions of high quality though, heuristic optimization algorithms are frequently used. These algorithms do not claim to find global optimal solutions, but offer a reasonable tradeoff between runtime and solution quality and are therefore especially suitable for practical applications. In the last decades the success of heuristic optimization techniques in many different problem domains encouraged the development of a broad variety of optimization paradigms which often use natural processes as a source of inspiration (as for example evolutionary algorithms, simulated annealing, or ant colony optimization). For the development and application of heuristic optimization algorithms in science and industry, mature, flexible and usable software systems are required. These systems have to support scientists in the development of new algorithms and should also enable users to apply different optimization methods on specific problems easily. The architecture and design of such heuristic optimization software systems impose many challenges on developers due to the diversity of algorithms and problems as well as the heterogeneous requirements of the different user groups. In this chapter the authors describe the architecture and design of their optimization environment HeuristicLab which aims to provide a comprehensive system for algorithm development, testing, analysis and generally the application of heuristic optimization methods on complex problems.

Effects of constant optimization by nonlinear least squares minimization in symbolic regression

In this publication a constant optimization approach for symbolic regression is introduced to separate the task of finding the correct model structure from the necessity to evolve the correct numerical constants. A gradient-based nonlinear least squares optimization algorithm, the Levenberg-Marquardt (LM) algorithm, is used for adjusting constant values in symbolic expression trees during their evolution. The LM algorithm depends on gradient information consisting of partial derivations of the trees, which are obtained by automatic differentiation. The presented constant optimization approach is tested on several benchmark problems and compared to a standard genetic programming algorithm to show its effectiveness. Although the constant optimization involves an overhead regarding the execution time, the achieved accuracy increases significantly as well as the ability of genetic programming to learn from provided data. As an example, the Pagie-1 problem could be solved in 37 out of 50 test runs, whereas without constant optimization it was solved in only 10 runs. Furthermore, different configurations of the constant optimization approach (number of iterations, probability of applying constant optimization) are evaluated and their impact is detailed in the results section.

Using FE Calculations and Data-Based System Identification Techniques to Model the Nonlinear Behavior of PMSMs

This paper investigates the modeling of brushless permanent-magnet synchronous machines (PMSMs). The focus is on deriving an automatable process for obtaining dynamic motor models that take nonlinear effects, such as saturation, into account. The modeling is based on finite element (FE) simulations for different current vectors in the

Visualization of genetic lineages and inheritance information in genetic programming

dq plane over a full electrical period. The parameters obtained are the stator flux in terms of the direct and quadrature components and the air-gap torque, both modeled as functions of the rotor angle and the current vector. The data are preprocessed according to theoretical results on potential harmonics in the targets as functions of the rotor angle. A variety of modeling strategies were explored: linear regression, support vector machines, symbolic regression using genetic programming, random forests, and artificial neural networks. The motor models were optimized for each training technique, and their accuracy was then compared based on the initially available FE data and further FE simulations for additional current vectors. Artificial neural networks and symbolic regression using genetic programming achieved the highest accuracy, particularly with additional test data.

Optimization Strategies for Integrated Knapsack and Traveling Salesman Problems

Many studies emphasize the importance of genetic diversity and the need for an appropriate tuning of selection pressure in genetic programming. Additional important aspects are the performance and effects of the genetic operators (crossover and mutation) on the transfer and stabilization of inherited information blocks during the run of the algorithm. In this context, different ideas about the usage of lineage and genealogical information for improving genetic programming have taken shape in the last decade. Our work builds on those ideas by introducing an evolution tracking framework for assembling genealogical and inheritance graphs of populations. The proposed approach allows detailed investigation of phenomena related to building blocks, size evolution, ancestry and diversity. We introduce the notion of genetic fragments to represent subtrees that are affected by reproductive operators (mutation and crossover) and present a methodology for tracking such fragments using flexible similarity measures. A fragment matching algorithm was designed to work on both structural and semantic levels, allowing us to gain insight into the exploratory and exploitative behavior of the evolutionary process. The visualization part which is the subject of this paper integrates with the framework and provides an easy way of exploring the population history. The paper focuses on a case study in which we investigate the evolution of a solution to a symbolic regression benchmark problem.

Gaining Deeper Insights in Symbolic Regression

In the optimization of real-world activities the effects of solutions on related activities need to be considered. The use of isolated problem models that do not adequately consider related processes does not allow addressing system-wide consequences. However, sometimes the complexity of the real-world model and its interplay with related activities can be described by a combination of simple, existing, problems. In this work we aim to discuss strategies to combine existing algorithms for simple problems in order to solve a more complex master problem. New challenges arise in such an integrated optimization approach.

On the architecture and implementation of tree-based genetic programming in HeuristicLab

A distinguishing feature of symbolic regression using genetic programming is its ability to identify complex nonlinear white-box models. This is especially relevant in practice where models are extensively scrutinized in order to gain knowledge about underlying processes. This potential is often diluted by the ambiguity and complexity of the models produced by genetic programming. In this contribution we discuss several analysis methods with the common goal to enable better insights in the symbolic regression process and to produce models that are more understandable and show better generalization. In order to gain more information about the process we monitor and analyze the progresses of population diversity, building block information, and even more general genealogy information. Regarding the analysis of results, several aspects such as model simplification, relevance of variables, node impacts, and variable network analysis are presented and discussed.

Macro-economic time series modeling and interaction networks

This article describes the architecture and implementation of the genetic programming (GP) framework of HeuristicLab. In particular we focus on the core design goals, namely extensibility, usability, and performance optimization and explain our approach to reach these goals. The overall design, the encoding, interpretation, and evaluation of programs is described and code examples are given to explain core aspects of the framework. HeuristicLab is available as open source software at http://dev.heuristiclab.com.

Analysis of selected evolutionary algorithms in feature selection and parameter optimization for data based tumor marker modeling

Macro-economic models describe the dynamics of economic quantities. The estimations and forecasts produced by such models play a substantial role for financial and political decisions. In this contribution we describe an approach based on genetic programming and symbolic regression to identify variable interactions in large datasets. In the proposed approach multiple symbolic regression runs are executed for each variable of the dataset to find potentially interesting models. The result is a variable interaction network that describes which variables are most relevant for the approximation of each variable of the dataset. This approach is applied to a macro-economic dataset with monthly observations of important economic indicators in order to identify potentially interesting dependencies of these indicators. The resulting interaction network of macro-economic indicators is briefly discussed and two of the identified models are presented in detail. The two models approximate the help wanted index and the CPI inflation in the US.

Improved homology-driven computational validation of protein-protein interactions motivated by the evolutionary gene duplication and divergence hypothesis

In this paper we report on the use of evolutionary algorithms for optimizing the identification of classification models for selected tumor markers. Our goal is to identify mathematical models that can be used for classifying tumor marker values as normal or as elevated; evolutionary algorithms are used for optimizing the parameters for learning classification models. The sets of variables used as well as the parameter settings for concrete modeling methods are optimized using evolution strategies and genetic algorithms. The performance of these algorithms is analyzed as well as the population diversity progress. In the empirical part of this paper we document modeling results achieved for tumor markers CA 125 and CYFRA using a medical data base provided by the Central Laboratory of the General Hospital Linz; empirical tests are executed using HeuristicLab.