Terence Soule

Effects of code growth and parsimony pressure on populations in genetic programming

Parsimony pressure, the explicit penalization of larger programs, has been increasingly used as a means of controlling code growth in genetic programming. However, in many cases parsimony pressure degrades the performance of the genetic program. In this paper we show that poor average results with parsimony pressure are a result of failed populations that overshadow the results of populations that incorporate parsimony pressure successfully. Additionally, we show that the effect of parsimony pressure can be measured by calculating the relationship between program size and performance within the population. This measure can be used as a partial indicator of success or failure for individual populations.

Breeding swarms: a GA/PSO hybrid

In this paper we propose a novel hybrid (GA/PSO) algorithm, Breeding Swarms, combining the strengths of particle swarm optimization with genetic algorithms. The hybrid algorithm combines the standard velocity and position update rules of PSOs with the ideas of selection, crossover and mutation from GAs. We propose a new crossover operator, Velocity Propelled Averaged Crossover (VPAC), incorporating the PSO velocity vector. The VPAC crossover operator actively disperses the population preventing premature convergence. We compare the hybrid algorithm to both the standard GA and PSO models in evolving solutions to five standard function minimization problems. Results show the algorithm to be highly competitive, often outperforming both the GA and PSO.

Removal bias: a new cause of code growth in tree based evolutionary programming

Presents a new cause of code growth, termed removal bias. We show that growth due to removal bias can be expected to occur whenever operations which remove and replace a variable-sized section of code, e.g. crossover or subtree mutation, are used in an evolutionary paradigm. Two forms of non-destructive crossover are used to examine the causes of code growth. The results support the protective value of inviable code and removal bias as two distinct causes of code growth. Both causes of code growth are shown to exist in at least two different problems.

An Analysis of the Causes of Code Growth in Genetic Programming

This research examines the cause of code growth (bloat) in genetic programming (GP). Currently there are three hypothesized causes of code growth in GP: protection, drift, and removal bias. We show that single node mutations increase code growth in evolving programs. This is strong evidence that the protective hypothesis is correct. We also show a negative correlation between the size of the branch removed during crossover and the resulting change in fitness, but a much weaker correlation for added branches. These results support the removal bias hypothesis, but seem to refute the drift hypothesis. Our results also suggest that there are serious disadvantages to the tree structured programs commonly evolved with GP, because the nodes near the root are effectively fixed in the very early generations.

Comparison of genetic algorithm and particle swarm optimizer when evolving a recurrent neural network

This paper compares the performance of GAs and PSOs in evolving weights of a recurrent neural network. The algorithms are tested on multiple network topologies. Both algorithms produce successful networks. The GA is more successful evolving larger networks and the PSO is more successful on smaller networks.

Genetic programming theory and practice IV

Contributing Authors.- Preface.- Foreword.- Genetic Programming: Theory and Practice.- Genome-Wide Genetic Analysis Using Genetic Programming: The Critical Need for Expert Knowledge.- Lifting the Curse of Dimensionality.- Genetic Programming for Classifying Cancer Data and Controlling Humanoid Robots.- Boosting Improves Stability and Accuracy of Genetic Programming in Biological Sequence Classification.- Othogonal Evoluton of Teams: A Class of Algorithms for Evolving Teams with Inversely Correlated Errors.- Multidimensional Tags, Cooperative Populations, and Genetic Programming.- Coevolving Fitness Models for Accelerating Evolution and Reducing Evaluations.- Multi-Domain Observations Concerning the Use of Genetic Programming to Automatically Synthesize Human-Competitive Designs for Analog Circuits, Optical Lens Systems, Controllers, Antennas, Mechanical Systems, and Quantum Computing Circuits.- Robust Pareto Front Genetic Programming Parameter Selection Based on Design of Experiments and Industrial Data.- Pursuing the Pareto Paradigm: Tournaments, Algorithm Variations and Ordinal Optimization.- Applying Genetic Programming to Reservoir History Matching Problem.- Comparison of Robustness of Three Filter Design Strategies Using Genetic Programming and Bond Graphs.- Design of Posynomial Models for Mosfets: Symbolic Regression Using Genetic Algorithms.- Phase Transitions in Genetic Programming Search.- Efficient Markov Chain Model of Machine Code Program Execution and Halting.- A Re-examination of a Real World Blood Flow Modeling Problem Using Context-aware Crossover.- Large-Scale, Time-Constrained Symbolic Regression.- Stock Selection: An Innovative Application of Genetic Programming Methodology.- Index.

Behavioral Diversity and a Probabilistically Optimal GP Ensemble

We propose N-version Genetic Programming (NVGP) as an ensemble method to enhance accuracy and reduce performance fluctuation of programs produced by genetic programming. Diversity is essential for forming successful ensembles. NVGP quantifies behavioral diversity of ensemble members and defines NVGP optimal as an ensemble that has independent fault occurrences among its members. We observed significant accuracy improvement by NVGP optimal ensembles when applied to a DNA segment classification problem.

Genetic programming: optimal population sizes for varying complexity problems

The population size in evolutionary computation is a significant parameter affecting computational effort and the ability to successfully evolve solutions. We find that population size sensitivity - how much a genetic programs efficiency varies with population size - is correlated with problem complexity. An analysis of population sizes was conducted using a unimodal, bimodal and a multi-modal problem with varying levels of difficulty. Specifically we show that a unimodal and bimodal and multimodal problems exhibit an increased sensitivity to population size with increasing levels of difficulty. We demonstrate that as problem complexity increases, determination of the optimal population size becomes more difficult. Conversely, the less complex a problem is the more sensitive the genetic programs efficiency is to population size.

Exons and Code Growth in Genetic Programming

Current theories regarding code growth (bloat) in genetic programming focus on the presence and growth of introns. In this paper we show for the first time that code growth can occur, albeit quite slowly, even in code that has a significant impact on fitness.

Novel ways of improving cooperation and performance in ensemble classifiers

There are two common methods of evolving teams of genetic programs. Research suggests Island approaches produce teams of strong individuals that cooperate poorly and Team approaches produce teams of weak individuals that cooperate strongly. Ideally, teams should be composed of strong individuals that cooperate well. In this paper we present a new class of algorithms called Orthogonal Evolution of Teams (OET) that overcomes the weaknesses of current Island and Team approaches by applying evolutionary pressure at both the level of teams and individuals during selection and replacement. We present four novel algorithms in this new class and compare their performance to Island and Team approaches as well as multi-class Adaboost on a number of classification problems.