Nguyen Xuan Hoai

Grammar-based Genetic Programming: a survey

Grammar formalisms are one of the key representation structures in Computer Science. So it is not surprising that they have also become important as a method for formalizing constraints in Genetic Programming (GP). Practical grammar-based GP systems first appeared in the mid 1990s, and have subsequently become an important strand in GP research and applications. We trace their subsequent rise, surveying the various grammar-based formalisms that have been used in GP and discussing the contributions they have made to the progress of GP. We illustrate these contributions with a range of applications of grammar-based GP, showing how grammar formalisms contributed to the solutions of these problems. We briefly discuss the likely future development of grammar-based GP systems, and conclude with a brief summary of the field.

Representation and structural difficulty in genetic programming

Standard tree-based genetic programming suffers from a structural difficulty problem in that it is unable to search effectively for solutions requiring very full or very narrow trees. This deficiency has been variously explained as a consequence of restrictions imposed by the tree structure or as a result of the numerical distribution of tree shapes. We show that by using a different tree-based representation and local (insertion and deletion) structural modification operators, that this problem can be almost eliminated even with trivial (stochastic hill-climbing) search methods, thus eliminating the above explanations. We argue, instead, that structural difficulty is a consequence of the large step size of the operators in standard genetic programming, which is itself a consequence of the fixed-arity property embodied in its representation.

Solving the symbolic regression problem with tree-adjunct grammar guided genetic programming: the comparative results

In this paper, we show some experimental results of tree-adjunct grammar-guided genetic programming (TAG3P) on the symbolic regression problem, a benchmark problem in genetic programming. We compare the results with genetic programming (GP) and grammar-guided genetic programming (GGGP). The results show that TAG3P significantly outperforms GP and GGGP on the target functions attempted in terms of the probability of success. Moreover, TAG3P still performed well when the structural complexity of the target function was scaled up.

Tree adjoining grammars, language bias, and genetic programming

In this paper, we introduce a new grammar guided genetic programming system called tree-adjoining grammar guided genetic programming (TAG3P+), where tree-adjoining grammars (TAGs) are used as means to set language bias for genetic programming. We show that the capability of TAGs in handling context-sensitive information and categories can be useful to set a language bias that cannot be specified in grammar guided genetic programming. Moreover, we bias the genetic operators to preserve the language bias during the evolutionary process. The results pace the way towards a better understanding of the importance of bias in genetic programming.

Improving the generalisation ability of genetic programming with semantic similarity based crossover

This paper examines the impact of semantic control on the ability of Genetic Programming (GP) to generalise via a semantic based crossover operator (Semantic Similarity based Crossover - SSC). The use of validation sets is also investigated for both standard crossover and SSC. All GP systems are tested on a number of real-valued symbolic regression problems. The experimental results show that while using validation sets barely improve generalisation ability of GP, by using semantics, the performance of Genetic Programming is enhanced both on training and testing data. Further recorded statistics shows that the size of the evolved solutions by using SSC are often smaller than ones obtained from GP systems that do not use semantics. This can be seen as one of the reasons for the success of SSC in improving the generalisation ability of GP.

On the roles of semantic locality of crossover in genetic programming

Locality has long been seen as a crucial property for the efficiency of Evolutionary Algorithms in general, and Genetic Programming (GP) in particular. A number of studies investigating the effects of locality in GP can be found in the literature. The majority of the previous research on locality focuses on syntactic aspects, and operator semantic locality has not been thoroughly tested. In this paper, we investigate the role of semantic locality of crossover in GP. We follow McPhee in measuring the semantics of a subtree using the fitness cases. We use this to define a semantic distance metric. This semantic distance supports the design of some new crossover operators, concentrating on improving semantic locality. We study the impact of these semantically based crossovers on the behaviour of GP. The results show substantial advantages accruing from the use of semantic locality.

Semantic similarity based crossover in GP: the case for real-valued function regression

In this paper we propose a new method for implementing the cross-over operator in Genetic Programming (GP) called Semantic Similarity based Crossover (SSC). This new operator is inspired by Semantic Aware Crossover (SAC) [20]. SSC extends SAC by adding semantics to control the change of the semantics of the individuals during the evolutionary process. The new crossover operator is then tested on a family of symbolic regression problems and compared with SAC as well as Standard Crossover (SC). The results from the experiments show that the change of the semantics (fitness) in the new SSC is smoother compared to SAC and SC. This leads to performance improvement in terms of percentage of successful runs and mean best fitness.

On Synergistic Interactions Between Evolution, Development and Layered Learning

We investigate interactions between evolution, development and lifelong layered learning in a combination we call evolutionary developmental evaluation (EDE), using a specific implementation, developmental tree-adjoining grammar guided genetic programming (GP). The approach is consistent with the process of biological evolution and development in higher animals and plants, and is justifiable from the perspective of learning theory. In experiments, the combination is synergistic, outperforming algorithms using only some of these mechanisms. It is able to solve GP problems that lie well beyond the scaling capabilities of standard GP. The solutions it finds are simple, succinct, and highly structured. We conclude this paper with a number of proposals for further extension of EDE systems.

Some Experimental Results with Tree Adjunct Grammar Guided Genetic Programming

Tree-adjunct grammar guided genetic programming (TAG3P) [5] is a grammar guided genetic programming system that uses context-free grammars along with tree-adjunct grammars as means to set language bias for the genetic programming system. In this paper, we show the experimental results of TAG3P on two problems: symbolic regression and trigonometric identity discovery. The results show that TAG3P works well on those problems.

Initialising PSO with randomised low-discrepancy sequences: the comparative results

In this paper, we investigate the use of some well-known randomised low-discrepancy sequences (Halton, Sobol, and Faure sequences) for initializing particle swarms. We experimented with the standard global-best particle swarm algorithm for function optimization on some benchmark problems, using randomised low-discrepancy sequences for initialisation, and the results were compared with the same particle swarm algorithm using uniform initialisation with a pseudo-random generator. The results show that, the former initialisation method could help the particle swarm algorithm improve its performance over the latter on the problems tried. Furthermore the comparisons also indicate that the use of different randomised low-discrepancy sequences in the initialisation phase could bring different effects on the performance of PSO.