Robert I. McKay

Grammar-based Genetic Programming: a survey

Grammar formalisms are one of the key representation structures in Computer Science. So it is not surprising that they have also become important as a method for formalizing constraints in Genetic Programming (GP). Practical grammar-based GP systems first appeared in the mid 1990s, and have subsequently become an important strand in GP research and applications. We trace their subsequent rise, surveying the various grammar-based formalisms that have been used in GP and discussing the contributions they have made to the progress of GP. We illustrate these contributions with a range of applications of grammar-based GP, showing how grammar formalisms contributed to the solutions of these problems. We briefly discuss the likely future development of grammar-based GP systems, and conclude with a brief summary of the field.

Grammar model-based program evolution

In evolutionary computation, genetic operators, such as mutation and crossover, are employed to perturb individuals to generate the next population. However these fixed, problem independent genetic operators may destroy the sub-solution, usually called building blocks, instead of discovering and preserving them. One way to overcome this problem is to build a model based on the good individuals, and sample this model to obtain the next population. There is a wide range of such work in genetic algorithms; but because of the complexity of the genetic programming (GP) tree representation, little work of this kind has been done in GP. In this paper, we propose a new method, grammar model-based program evolution (GMPE) to evolved GP program. We replace common GP genetic operators with a probabilistic context-free grammar (SCFG). In each generation, an SCFG is learnt, and a new population is generated by sampling this SCFG model. On two benchmark problems we have studied, GMPE significantly outperforms conventional GP, learning faster and more reliably.

Representation and structural difficulty in genetic programming

Standard tree-based genetic programming suffers from a structural difficulty problem in that it is unable to search effectively for solutions requiring very full or very narrow trees. This deficiency has been variously explained as a consequence of restrictions imposed by the tree structure or as a result of the numerical distribution of tree shapes. We show that by using a different tree-based representation and local (insertion and deletion) structural modification operators, that this problem can be almost eliminated even with trivial (stochastic hill-climbing) search methods, thus eliminating the above explanations. We argue, instead, that structural difficulty is a consequence of the large step size of the operators in standard genetic programming, which is itself a consequence of the fixed-arity property embodied in its representation.

Software project effort estimation using genetic programming

Knowing the estimated cost of a software project early in the development cycle is a valuable asset for management. In this paper, an evolutionary computation method, grammar guided genetic programming (GGGP), is used to fit models, with the aim of improving the prediction of software development costs. Valuable results are obtained, significantly better than those obtained by simple linear regression. In this research, GGGP, because of its flexibility and the ability of incorporating background knowledge, also shows great potential in being applied in other software engineering modeling problems.

Solving the symbolic regression problem with tree-adjunct grammar guided genetic programming: the comparative results

In this paper, we show some experimental results of tree-adjunct grammar-guided genetic programming (TAG3P) on the symbolic regression problem, a benchmark problem in genetic programming. We compare the results with genetic programming (GP) and grammar-guided genetic programming (GGGP). The results show that TAG3P significantly outperforms GP and GGGP on the target functions attempted in terms of the probability of success. Moreover, TAG3P still performed well when the structural complexity of the target function was scaled up.

A Survey of Probabilistic Model Building Genetic Programming

There has been a surge of research interest in Estimation of Distribution Algorithms (EDA). Several reviews on current work in conventional EDA are available. Although most work has focused on one dimensional representations that resembles the chromosomes of Genetic Algorithms (GA), an interesting stream of EDA using more complex tree representations has recently received some attention. To date, there has been no general review of this area in the current literature. This chapter aims to provide a critical and comprehensive review of EDA with tree representation, and closely related fields.

AntTAG: a new method to compose computer programs using colonies of ants

Genetic programming (GP) plays the primary role in the discovery of programs through evolving the programs set of parse trees. We present a new technique for constructing programs through ant colony optimization (ACO) using the tree adjunct grammar (TAG) formalism. We call the method AntTAG and we show that the results are very promising.

Tree adjoining grammars, language bias, and genetic programming

In this paper, we introduce a new grammar guided genetic programming system called tree-adjoining grammar guided genetic programming (TAG3P+), where tree-adjoining grammars (TAGs) are used as means to set language bias for genetic programming. We show that the capability of TAGs in handling context-sensitive information and categories can be useful to set a language bias that cannot be specified in grammar guided genetic programming. Moreover, we bias the genetic operators to preserve the language bias during the evolutionary process. The results pace the way towards a better understanding of the importance of bias in genetic programming.

A novel mixture of experts model based on cooperative coevolution

Abstract Combining several suitable neural networks can enhance the generalization performance of the group when compared to a single network alone. However, it remains a largely open question, how best to build a suitable combination of individuals. Jacobs and his colleagues proposed the mixture of experts (ME) model, in which a set of neural networks are trained together with a gate network. This tight coupling mechanism enables the system to (i) encourage diversity between the individual neural networks by specializing them in different regions of the input space and (ii) allow for a “good” combination weights of the ensemble members to emerge by training the gate, which computes the dynamic weights together with the classifiers. In this paper, we have wrapped a cooperative coevolutionary (CC) algorithm around the basic ME model. This CC layer allows better exploration of the weight space, and hence, an ensemble with better performance. The results show that CCME is better on average than the original ME on a number of classification problems. We have also introduced a novel mechanism for visualizing the modular structures that emerged from the model.

Multiobjective evolutionary algorithms for dynamic social network clustering

The main focus of this paper is to propose integration of dynamic and multiobjective algorithms for graph clustering in dynamic environments under multiple objectives. The primary application is to multiobjective clustering in social networks which change over time. Social networks, typically represented by graphs, contain information about the relations (or interactions) among online materials (or people). A typical social network tends to expand over time, with newly added nodes and edges being incorporated into the existing graph. We reflect these characteristics of social networks based on real-world data, and propose a suitable dynamic multiobjective evolutionary algorithm. Several variants of the algorithm are proposed and compared. Since social networks change continuously, the immigrant schemes effectively used in previous dynamic optimisation give useful ideas for new algorithms. An adaptive integration of multiobjective evolutionary algorithms outperformed other algorithms in dynamic social networks.