Andrew James Turner

Cartesian genetic programming encoded artificial neural networks: a comparison using three benchmarks

Neuroevolution, the application of evolutionary algorithms to artificial neural networks (ANNs), is well-established in machine learning. Cartesian Genetic Programming (CGP) is a graph-based form of Genetic Programming which can easily represent ANNs. Cartesian Genetic Programming encoded ANNs (CGPANNs) can evolve every aspect of an ANN: weights, topology, arity and node transfer functions. This makes CGPANNs very suited to situations where appropriate configurations are not known in advance. The effectiveness of CGPANNs is compared with a large number of previous methods on three benchmark problems. The results show that CGPANNs perform as well as or better than many other approaches. We also discuss the strength and weaknesses of each of the three benchmarks.

Recurrent Cartesian Genetic Programming

This paper formally introduces Recurrent Cartesian Genetic Programming (RCGP), an extension to Cartesian Genetic Programming (CGP) which allows recurrent connections. The presence of recurrent connections enables RCGP to be successfully applied to partially observable tasks. It is found that RCGP significantly outperforms CGP on two partially observable tasks: artificial ant and sunspot prediction. The paper also introduces a new parameter, recurrent connection probability, which biases the number of recurrent connections created via mutation. Suitable choices of this parameter significantly improve the effectiveness of RCGP.

Introducing a cross platform open source Cartesian Genetic Programming library

Cartesian Genetic Programming (CGP) is a form of Genetic Programming which encodes computational structures as generic cyclic/acyclic graphs. This letter introduces a new cross platform CGP library intended for use in teaching, academic research and real world applications. This new CGP library is currently capable of evolving symbolic expressions, Boolean logic circuits and Artificial Neural Networks but can easily be extended to other domains. The CGP library, documentation and tutorials are all available at www.cgplibrary.co.uk.

Cartesian Genetic Programming: Why No Bloat?

For many years now it has been known that Cartesian Genetic Programming CGP does not exhibit program bloat. Two possible explanations have been proposed in the literature: neutral genetic drift and length bias. This paper empirically disproves both of these and thus, reopens the question as to why CGP does not suffer from bloat. It has also been shown for CGP that using a very large number of nodes considerably increases the effectiveness of the search. This paper also proposes a new explanation as to why this may be the case.

Neutral genetic drift: an investigation using Cartesian Genetic Programming

Neutral genetic drift is an evolutionary mechanism which can strongly aid the escape from local optima. This makes neutral genetic drift an increasingly important property of Evolutionary Computational methods as more challenging applications are approached. Cartesian Genetic Programming (CGP) is a Genetic Programming technique which contains explicit, as well as the more common implicit, genetic redundancy. As explicit genetic redundancy is easily identified and manipulated it represents a useful tool for investigating neutral genetic drift. The contributions of this paper are as follows. Firstly the paper presents a substantial evaluation of the role and benefits of neutral genetic drift in CGP. Here it is shown that the benefits of explicit genetic redundancy are additive to the benefits of implicit genetic redundancy. This is significant as it indicates that that levels of implicit genetic redundancy present in other Evolutionary Computational methods may be insufficient to fully utilise neutral genetic drift. It is also shown than the identification and manipulation of explicit genetic redundancy is far easier than for implicit genetic redundancy. This is significant as it makes the investigations here possible and leads to new possibilities for allowing more effective use of neutral genetic drift. This is the case not only for CGP, but many other Evolutionary Computational methods which contain explicit genetic redundancy. Finally, it is also shown that neutral genetic drift has additional benefits other than aiding the escape from local optima.

NeuroEvolution: Evolving Heterogeneous Artificial Neural Networks

NeuroEvolution is the application of Evolutionary Algorithms to the training of Artificial Neural Networks. Currently the vast majority of NeuroEvolutionary methods create homogeneous networks of user defined transfer functions. This is despite NeuroEvolution being capable of creating heterogeneous networks where each neuron’s transfer function is not chosen by the user, but selected or optimised during evolution. This paper demonstrates how NeuroEvolution can be used to select or optimise each neuron’s transfer function and empirically shows that doing so significantly aids training. This result is important as the majority of NeuroEvolutionary methods are capable of creating heterogeneous networks using the methods described.

The Importance of Topology Evolution in NeuroEvolution: A Case Study Using Cartesian Genetic Programming of Artificial Neural Networks

NeuroEvolution (NE) is the application of evolutionary algorithms to Artificial Neural Networks (ANN). This paper reports on an investigation into the relative importance of weight evolution and topology evolution when training ANN using NE. This investigation used the NE technique Cartesian Genetic Programming of Artificial Neural Networks (CGPANN). The results presented show that the choice of topology has a dramatic impact on the effectiveness of NE when only evolving weights; an issue not faced when manipulating both weights and topology. This paper also presents the surprising result that topology evolution alone is far more effective when training ANN than weight evolution alone. This is a significant result as many methods which train ANN manipulate only weights.

Classification and characterisation of movement patterns during levodopa therapy for parkinson's disease

Parkinsons disease is a chronic neurodegenerative condition that manifests clinically with various movement disorders. These are often treated with the dopamine-replacement drug levodopa. However, the dosage of levodopa must be kept as low as possible in order to avoid the drugs side effects, such as the involuntary, and often violent, muscle spasms called dyskinesia, or levodopa-induced dyskinesia. In this paper, we investigate the use of genetic programming for training classifiers that can monitor the effectiveness of levodopa therapy. In particular, we evolve classifiers that can recognise tremor and dyskinesia, movement states that are indicative of insufficient or excessive doses of levodopa, respectively. The evolved classifiers achieve clinically useful rates of discrimination, with AUC>0.9. We also find that temporal classifiers generally out-perform spectral classifiers. By using classifiers that respond to low-level features of the data, we identify the conserved patterns of movement that are used as a basis for classification, showing how this approach can be used to characterise as well as classify abnormal movement.

Recurrent Cartesian Genetic Programming Applied to Series Forecasting

Recurrent Cartesian Genetic Programming is a recently proposed extension to Cartesian Genetic Programming which allows cyclic program structures to be evolved. We apply both standard and Recurrent Cartesian Genetic Programming to the domain of series forecasting. Their performance is then compared to a number of well-known classical forecasting approaches. Our results show that not only does Recurrent Cartesian Genetic Programming outperform standard Cartesian Genetic Programming, but it also outperforms many standard forecasting techniques.

A New Evolutionary Algorithm-Based Home Monitoring Device for Parkinson’s Dyskinesia

Parkinson’s disease (PD) is a neurodegenerative movement disorder. Although there is no cure, symptomatic treatments are available and can significantly improve quality of life. The motor, or movement, features of PD are caused by reduced production of the neurotransmitter dopamine. Dopamine deficiency is most often treated using dopamine replacement therapy. However, this therapy can itself lead to further motor abnormalities referred to as dyskinesia. Dyskinesia consists of involuntary jerking movements and muscle spasms, which can often be violent. To minimise dyskinesia, it is necessary to accurately titrate the amount of medication given and monitor a patient’s movements. In this paper, we describe a new home monitoring device that allows dyskinesia to be measured as a patient goes about their daily activities, providing information that can assist clinicians when making changes to medication regimens. The device uses a predictive model of dyskinesia that was trained by an evolutionary algorithm, and achieves AUC>0.9 when discriminating clinically significant dyskinesia.