James McDermott

Genetic programming needs better benchmarks

Genetic programming (GP) is not a field noted for the rigor of its benchmarking. Some of its benchmark problems are popular purely through historical contingency, and they can be criticized as too easy or as providing misleading information concerning real-world performance, but they persist largely because of inertia and the lack of good alternatives. Even where the problems themselves are impeccable, comparisons between studies are made more difficult by the lack of standardization. We argue that the definition of standard benchmarks is an essential step in the maturation of the field. We make several contributions towards this goal. We motivate the development of a benchmark suite and define its goals; we survey existing practice; we enumerate many candidate benchmarks; we report progress on reference implementations; and we set out a concrete plan for gathering feedback from the GP community that would, if adopted, lead to a standard set of benchmarks.

Better GP benchmarks: community survey results and proposals

We present the results of a community survey regarding genetic programming benchmark practices. Analysis shows broad consensus that improvement is needed in problem selection and experimental rigor. While views expressed in the survey dissuade us from proposing a large-scale benchmark suite, we find community support for creating a “blacklist” of problems which are in common use but have important flaws, and whose use should therefore be discouraged. We propose a set of possible replacement problems.

GEVA: grammatical evolution in Java

We are delighted to announce the release of GEVA [1], an open source software implementation of Grammatical Evolution (GE) in Java. Grammatical Evolution in Java (GEVA) was developed at UCDs Natural Computing Research & Applications group (http://ncra.ucd.ie).

Evolutionary design using grammatical evolution and shape grammars : designing a shelter

A new evolutionary design tool is presented, which uses shape grammars and a grammar-based form of evolutionary computa- tion, grammatical evolution (GE). Shape grammars allow the user to specify possible forms, and GE allows forms to be iteratively selected,

Defining locality as a problem difficulty measure in genetic programming

A mapping is local if it preserves neighbourhood. In Evolutionary Computation, locality is generally described as the property that neighbouring genotypes correspond to neighbouring phenotypes. A representation has high locality if most genotypic neighbours are mapped to phenotypic neighbours. Locality is seen as a key element in performing effective evolutionary search. It is believed that a representation that has high locality will perform better in evolutionary search and the contrary is true for a representation that has low locality. When locality was introduced, it was the genotype-phenotype mapping in bitstring-based Genetic Algorithms which was of interest; more recently, it has also been used to study the same mapping in Grammatical Evolution. To our knowledge, there are few explicit studies of locality in Genetic Programming (GP). The goal of this paper is to shed some light on locality in GP and use it as an indicator of problem difficulty. Strictly speaking, in GP the genotype and the phenotype are not distinct. We attempt to extend the standard quantitative definition of genotype-phenotype locality to the genotype-fitness mapping by considering three possible definitions. We consider the effects of these definitions in both continuous- and discrete-valued fitness functions. We compare three different GP representations (two of them induced by using different function sets and the other using a slightly different GP encoding) and six different mutation operators. Results indicate that one definition of locality is better in predicting performance.

An analysis of the behaviour of mutation in grammatical evolution

This study attempts to decompose the behaviour of mutation in Grammatical Evolution (GE). Standard GE mutation can be divided into two types of events, those that are structural in nature and those that are nodal. A structural event can alter the length of the phenotype whereas a nodal event simply alters the value at any terminal (leaf or internal node) of a derivation tree. We analyse the behaviour of standard mutation and compare it to the behaviour of its nodal and structural components. These results are then compared with standard GP operators to see how they differ. This study increases our understanding of how the search operators of an evolutionary algorithm behave.

Combining structural analysis and multi-objective criteria for evolutionary architectural design

This study evolves and categorises a population of conceptual designs by their ability to handle physical constraints. The design process involves a trade-off between form and function. The aesthetic considerations of the designer are constrained by physical considerations and material cost. In previous work, we developed a design grammar capable of evolving aesthetically pleasing designs through the use of an interactive evolutionary algorithm. This work implements a fitness function capable of applying engineering objectives to automatically evaluate designs and, in turn, reduce the search space that is presented to the user.

Flex-GP: genetic programming on the cloud

We describe Flex-GP, which we believe to be the first largescale genetic programming cloud computing system. We took advantage of existing software and selected a socket-based, client-server architecture and an island-based distribution model. We developed core components required for deployment on Amazons EC2. Scaling the system to hundreds of nodes presented several unexpected challenges and required the development of software for automatically managing deployment, reporting, and error handling. The systems performance was evaluated on two metrics, performance and speed, on a difficult symbolic regression problem. Our largest successful Flex-GP runs reached 350 nodes and taught us valuable lessons for the next phase of scaling.

Toward user-directed evolution of sound synthesis parameters

Experiments are described which use genetic algorithms operating on the parameter settings of an FM synthesizer, with the aim of mimicking known synthesized sounds. The work is considered as a precursor to the development of synthesis plug-ins using evolution directed by a user. Attention is focussed on the fitness functions used to drive the evolution: the main result is that a composite fitness function – based on a combination of perceptual measures, spectral analysis, and low-level sample-by-sample comparison – drives more successful evolution than fitness functions which use only one of these types of criterion.

Elevated Pitch: Automated Grammatical Evolution of Short Compositions

A system for automatic composition using grammatical evolution is presented. Music is created under the constraints of a generative grammar, and under the bias of an automatic fitness function and evolutionary selection. This combination of two methods is seen to be powerful and flexible. Human evaluation of automatically-evolved pieces shows that a more sophisticated grammar in combination with a naive fitness function gives better results than the reverse.