Peter Dürr

Neuroevolution: from architectures to learning

Artificial neural networks (ANNs) are applied to many real-world problems, ranging from pattern classification to robot control. In order to design a neural network for a particular task, the choice of an architecture (including the choice of a neuron model), and the choice of a learning algorithm have to be addressed. Evolutionary search methods can provide an automatic solution to these problems. New insights in both neuroscience and evolutionary biology have led to the development of increasingly powerful neuroevolution techniques over the last decade. This paper gives an overview of the most prominent methods for evolving ANNs with a special focus on recent advances in the synthesis of learning architectures.

Evolving neuromodulatory topologies for reinforcement learning-like problems

Environments with varying reward contingencies constitute a challenge to many living creatures. In such conditions, animals capable of adaptation and learning derive an advantage. Recent studies suggest that neuromodulatory dynamics are a key factor in regulating learning and adaptivity when reward conditions are subject to variability. In biological neural networks, specific circuits generate modulatory signals, particularly in situations that involve learning cues such as a reward or novel stimuli. Modulatory signals are then broadcast and applied onto target synapses to activate or regulate synaptic plasticity. Artificial neural models that include modulatory dynamics could prove their potential in uncertain environments when online learning is required. However, a topology that synthesises and delivers modulatory signals to target synapses must be devised. So far, only handcrafted architectures of such kind have been attempted. Here we show that modulatory topologies can be designed autonomously by artificial evolution and achieve superior learning capabilities than traditional fixed-weight or Hebbian networks. In our experiments, we show that simulated bees autonomously evolved a modulatory network to maximise the reward in a reinforcement learning-like environment.

Neuroevolution with analog genetic encoding

The evolution of artificial neural networks (ANNs) is often used to tackle difficult control problems. There are different approaches to the encoding of neural networks in artificial genomes. Analog Genetic Encoding (AGE) is a new implicit method derived from the observation of biological genetic regulatory networks. This paper shows how AGE can be used to simultaneously evolve the topology and the weights of ANNs for complex control systems. AGE is applied to a standard benchmark problem and we show that its performance is equivalent or superior to some of the most powerful algorithms for neuroevolution in the literature.

Genetic Representation and Evolvability of Modular Neural Controllers

The manual design of control systems for robotic devices can be challenging. Methods for the automatic synthesis of control systems, such as the evolution of artificial neural networks, are thus widely used in the robotics community. However, in many robotic tasks where multiple interdependent control problems have to be solved simultaneously, the performance of conventional neuroevolution techniques declines. In this paper, we identify interference between the adaptation of different parts of the control system as one of the key challenges in the evolutionary synthesis of artificial neural networks. As modular network architectures have been shown to reduce the effects of such interference, we propose a novel, implicit modular genetic representation that allows the evolutionary algorithm to automatically shape modular network topologies. Our experiments with plastic neural networks in a simple maze learning task indicate that adding a modular genetic representation to a state-of-the-art implicit neuroevolution method leads to better algorithm performance and increases the robustness of evolved solutions against detrimental mutations.

Evolvability of Neuromodulated Learning for Robots

Neuromodulation is thought to be one of the underlying principles of learning and memory in biological neural networks. Recent experiments have shown that neuroevolutionary methods benefit from neuromodulation in simple grid-world problems. In this paper we investigate the performance of a neuroevolutionary method applied to a more realistic robotic task. While confirming the favorable effect of neuromodulatory structures, our results indicate that the evolution of such architectures requires a mechanism which allows for selective modular targetting of the neuromodulatory connections.

The Age of Analog Networks

A large class of systems of biological and technological relevance can be described as analog networks, that is, collections of dynamical devices interconnected by links of varying strength. Some examples of analog networks are genetic regulatory networks, metabolic networks, neural networks, analog electronic circuits, and control systems. Analog networks are typically complex systems which include nonlinear feedback loops and possess temporal dynamics at different time scales. Both the synthesis and reverse engineering of analog networks are recognized as knowledge-intensive activities, for which few systematic techniques exist. In this paper we will discuss the general relevance of the analog network concept and describe an evolutionary approach to the automatic synthesis and the reverse engineering of analog networks. The proposed approach is called analog genetic encoding (AGE) and realizes an implicit genetic encoding of analog networks. AGE permits the evolution of human-competitive solutions to real-world analog network design and identification problems. This is illustrated by some examples of application to the design of electronic circuits, control systems, learning neural architectures, and the reverse engineering of biological networks.

Center of mass encoding: a self-adaptive representation with adjustable redundancy for real-valued parameters

In this paper we describe a new class of representations for real-valued parameters called Center of Mass Encoding (CoME). CoME is based on variable length strings, it is self-adaptive, and it permits the choice of the degree of redundancy of the genotype-to-phenotype map and the choice of the distribution of the redundancy over the space of phenotypes. We first describe CoME and then proceed to test its performance and compare it with other representations and with a state-of-the-art evolution strategy. We show that CoME performs well on a large set of test functions. Furthermore, we show how CoME adapts the granularity of its discretization on functions defined over nonuniformly scaled domains.

Beyond graphs: A new synthesis

Artificial neural networks, electronic circuits, and gene networks are some examples of systems that can be modeled as networks, that is, as collections of interconnected nodes. In this paper we introduce the concept of terminal graph (t-graph for short), which improves on the concept of graph as a unifying principle for the representation and computational synthesis and inference of technological and biological networks. We begin by showing how to use the t-graph concept to better understand the working of existing methods for the computational synthesis of networks. Then, we discuss the issue of the “missing methods”, that is, of new computational methods of network synthesis whose existence can be inferred using the perspective provided by the con- cept of t-graph. Finally, we comment on the application of the t-graph perspective to problems of network inference, to the field of complex networks, social networks, and to the understanding of biological networks and developmental processes.

Evolutionary selection of features for neural sleep/wake discrimination

In biomedical signal analysis, artificial neural networks are often used for pattern classification because of their capability for nonlinear class separation and the possibility to efficiently implement them on a microcontroller. Typically, the network topology is designed by hand, and a gradient-based search algorithm is used to find a set of suitable parameters for the given classification task. In many cases, however, the choice of the network architecture is a critical and difficult task. For example, hand-designed networks often require more computational resources than necessary because they rely on input features that provide no information or are redundant. In the case of mobile applications, where computational resources and energy are limited, this is especially detrimental. Neuroevolutionary methods which allow for the automatic synthesis of network topology and parameters offer a solution to these problems. In this paper, we use analog genetic encoding (AGE) for the evolutionary synthesis of a neural classifier for a mobile sleep/wake discrimination system. The comparison with a hand-designed classifier trained with back propagation shows that the evolved neural classifiers display similar performance to the hand-designed networks, but using a greatly reduced set of inputs, thus reducing computation time and improving the energy efficiency of the mobile system.