Tomasz P. Pawlak

Semantic Backpropagation for Designing Search Operators in Genetic Programming

In genetic programming, a search algorithm is expected to produce a program that achieves the desired final computation state (desired output). To reach that state, an executing program needs to traverse certain intermediate computation states. An evolutionary search process is expected to autonomously discover such states. This can be difficult for nontrivial tasks that require long programs to be solved. The semantic backpropagation algorithm proposed in this paper heuristically inverts the execution of evolving programs to determine the desired intermediate computation states. Two search operators, random desired operator and approximately geometric semantic crossover, use the intermediate states determined by semantic backpropagation to define subtasks of the original programming task, which are then solved using an exhaustive search. The operators outperform the standard genetic search operators and other semantic-aware operators when compared on a suite of symbolic regression and Boolean benchmarks. This result and additional analysis conducted in this paper indicate that semantic backpropagation helps evolution to identify the desired intermediate computation states and makes the search process more efficient.

Locally geometric semantic crossover: a study on the roles of semantics and homology in recombination operators

This study presents an extensive account of Locally Geometric Semantic Crossover (LGX), a semantically-aware recombination operator for genetic programming (GP). LGX is designed to exploit the semantic properties of programs and subprograms, in particular the geometry of semantic space that results from distance-based fitness functions used predominantly in GP. When applied to a pair of parents, LGX picks in them at random a structurally common (homologous) locus, calculates the semantics of subprograms located at that locus, finds a procedure that is semantically medial with respect to these subprograms, and replaces them with that procedure. The library of procedures is prepared prior to the evolutionary run and indexed by a multidimensional structure (kd-tree) allowing for efficient search. The paper presents the rationale for LGX design and an extensive computational experiment concerning performance, computational cost, impact on program size, and capability of generalization. LGX is compared with six other operators, including conventional tree-swapping crossover, semantic-aware operators proposed in previous studies, and control methods designed to verify the importance of homology and geometry of the semantic space. The overall conclusion is that LGX, thanks to combination of the semantically medial operation with homology, improves the efficiency of evolutionary search, lowers the variance of performance, and tends to be more resistant to overfitting.

Approximating geometric crossover by semantic backpropagation

We propose a novel crossover operator for tree-based genetic programming, that produces approximately geometric offspring. We empirically analyze certain aspects of geometry of crossover operators and verify performance of the new operator on both, training and test fitness cases coming from set of symbolic regression benchmarks. The operator shows superior performance and higher probability of producing geometric offspring than tree-swapping crossover and other semantic-aware control methods.

Locally geometric semantic crossover

We propose Locally Geometric Crossover (LGX) for genetic programming. For a pair of homologous loci in the parent solutions, LGX finds a semantically intermediate procedure from a previously prepared library, and uses it as replacement code. The experiments involving six symbolic regression problems show significant increase in search performance when compared to standard subtree-swapping cross-over and other control methods. This suggests that semantically geometric manipulations on subprograms propagate to entire programs and improve their fitness.

Progress properties and fitness bounds for geometric semantic search operators

Metrics are essential for geometric semantic genetic programming. On one hand, they structure the semantic space and govern the behavior of geometric search operators; on the other, they determine how fitness is calculated. The interactions between these two types of metrics are an important aspect that to date was largely neglected. In this paper, we investigate these interactions and analyze their consequences. We provide a systematic theoretical analysis of the properties of abstract geometric semantic search operators under Minkowski metrics of arbitrary order. For nine combinations of popular metrics (city-block, Euclidean, and Chebyshev) used in fitness functions and of search operators, we derive pessimistic bounds on fitness change. We also define three types of progress properties (weak, potential, and strong) and verify them for operators under those metrics. The analysis allows us to determine the combinations of metrics that are most attractive in terms of progress properties and deterioration bounds.

Quantitative analysis of locally geometric semantic crossover

We investigate the properties of locally geometric semantic crossover (LGX), a genetic programming search operator that is approximately semantically geometric on the level of homologous code fragments. For a pair of corresponding loci in the parents, LGX finds a semantically intermediate procedure from a library prepared prior to evolutionary run, and creates an offspring by using such procedure as replacement code. LGX proves superior when compared to standard subtree crossover and other control methods in terms of search convergence, test-set performance, and time required to find a high-quality solution. This paper focuses in particular the impact of homology and program semantic on LGX performance.

Synthesis of Mathematical Programming Constraints with Genetic Programming

We identify a novel application of Genetic Programming to automatic synthesis of mathematical programming (MP) models for business processes. Given a set of examples of states of a business process, the proposed Genetic Constraint Synthesis (GenetiCS) method constructs well-formed constraints for an MP model. The form of synthesized constraints (e.g., linear or polynomial) can be chosen accordingly to the nature of the process and the desired type of MP problem. In experimental part, we verify syntactic and semantic fidelity of the synthesized models to the actual benchmark models of varying complexity. The obtained symbolic models of constraints can be combined with an objective function of choice, fed into an off-shelf MP solver, and optimized.

Automatic synthesis of constraints from examples using mixed integer linear programming

Constraints form an essential part of most practical search and optimization problems, and are usually assumed to be given. However, there are plausible real-world scenarios in which constraints are not known or can be only approximated, for instance when the process in question is complex and/or noisy. To address such problems, we propose a method that synthesizes constrains from examples of feasible and infeasible solutions. The method can produce linear, quadratic and trigonometric constraints that are guaranteed to separate the feasible and infeasible regions and minimize the number of terms involved. The synthesized constraints are represented symbolically and can be used to simulate, predict or optimize the original process. We assess empirically several characteristics of the method on three benchmarks, in particular the fidelity and the complexity of the synthesized constraints with respect to the actual constraints. We also demonstrate its application to a real-world process of concrete manufacturing. Experiments demonstrate that the method is capable of producing human-readable constraints that reflect well the underlying process and can be used to simulate it.

Semantic Geometric Initialization

A common approach in Geometric Semantic Genetic Programming (GSGP) is to seed initial populations using conventional, semantic-unaware methods like Ramped Half-and-Half. We formally demonstrate that this may limit GSGP’s ability to find a program with the sought semantics. To overcome this issue, we determine the desired properties of geometric-aware semantic initialization and implement them in Semantic Geometric Initialization (Sgi) algorithm, which we instantiate for symbolic regression and Boolean function synthesis problems. Properties of Sgi and its impact on GSGP search are verified experimentally on nine symbolic regression and nine Boolean function synthesis benchmarks. When assessed experimentally, Sgi leads to superior performance of GSGP search: better best-of-run fitness and higher probability of finding the optimal program.

One-class synthesis of constraints for Mixed-Integer Linear Programming with C4.5 decision trees

Abstract We propose Constraint Synthesis with C4.5 ( CSC4.5 ), a novel method for automated construction of constraints for Mixed-Integer Linear Programming (MILP) models from data. Given a sample of feasible states of a modeled entity, e.g., a business process or a system, CSC4.5 synthesizes a well-formed MILP model of that entity, suitable for simulation and optimization using an off-the-shelf solver. CSC4.5 operates by estimating the distribution of the feasible states, bounding that distribution with C4.5 decision tree and transforming that tree into a MILP model. We verify CSC4.5 experimentally using parameterized synthetic benchmarks, and conclude considerable fidelity of the synthesized constraints to the actual constraints in the benchmarks. Next, we apply CSC4.5 to synthesize from past observations two MILP models of a real-world business process of wine production, optimize the MILP models using an external solver and validate the optimal solutions with use of a competing modeling method.