Kai Olav Ellefsen

Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills

A long-standing goal in artificial intelligence is creating agents that can learn a variety of different skills for different problems. In the artificial intelligence subfield of neural networks, a barrier to that goal is that when agents learn a new skill they typically do so by losing previously acquired skills, a problem called catastrophic forgetting. That occurs because, to learn the new task, neural learning algorithms change connections that encode previously acquired skills. How networks are organized critically affects their learning dynamics. In this paper, we test whether catastrophic forgetting can be reduced by evolving modular neural networks. Modularity intuitively should reduce learning interference between tasks by separating functionality into physically distinct modules in which learning can be selectively turned on or off. Modularity can further improve learning by having a reinforcement learning module separate from sensory processing modules, allowing learning to happen only in response to a positive or negative reward. In this paper, learning takes place via neuromodulation, which allows agents to selectively change the rate of learning for each neural connection based on environmental stimuli (e.g. to alter learning in specific locations based on the task at hand). To produce modularity, we evolve neural networks with a cost for neural connections. We show that this connection cost technique causes modularity, confirming a previous result, and that such sparsely connected, modular networks have higher overall performance because they learn new skills faster while retaining old skills more and because they have a separate reinforcement learning module. Our results suggest (1) that encouraging modularity in neural networks may help us overcome the long-standing barrier of networks that cannot learn new skills without forgetting old ones, and (2) that one benefit of the modularity ubiquitous in the brains of natural animals might be to alleviate the problem of catastrophic forgetting.

Planning Inspection Paths through Evolutionary Multi-objective Optimization

We propose a new method for planning paths allowing the inspection of complex 3D structures, given a triangular mesh model of the structure. The method differs from previous approaches in its emphasis on generating and considering also plans that result in imperfect coverage of the inspection target. In many practical tasks, one would accept imperfections in coverage if this results in a substantially more energy efficient inspection path. We therefore apply an evolutionary multi-objective optimization algorithm to the problem -- which generates a set of solutions exploring the different ways to balance energy usage and coverage degree. To our knowledge, this is the first work that exploits the power of multiobjective evolution to explore the set of inspection plans that result in the most energy-efficient coverage of structures. The performance of the optimizer is demonstrated on a model of a subsea oilfield installation -- a type of structure that has many occluded and hidden parts, and that therefore illustrates the need for a method accepting imperfectly covering solutions.

Balancing the Costs and Benefits of Learning Ability

We study the costs and benefits of plasticity by evolving agents in environments with different rates of environmental change. Evolution allows both hard-coded strategies and learned strategies, with learning rates varying throughout life. We observe a range of change rates where the balance of costs and benefits are just right for evolving learning. Inside this range, we see two separate strategies evolve: lifelong plasticity and sensitive periods of plasticity. Sensitive periods of plasticity are found to reduce the learning cost while retaining the benefits of learning. This affects the evolutionary process, by limiting genetic assimilation of learned characteristics, making agents able to remain adaptive after relatively long periods of environmental stability.

Evolved Sensitive Periods in Learning

In this paper we study age-varying plasticities across different components in an artificial neural network performing a reinforcement learning task. An evolutionary algorithm is given the task of mapping the age of agents to the plasticity levels of different network components. The results show that patterns of plasticity resembling biological sensitive periods appear, and that these periods schedule learning across the components of the network, which leads to a reduction in the total learning effort while retaining the quality of learning. The sequencing of sensitive periods forms a cascade of partiallyoverlapping learning periods, which has been proposed as a way of organizing sensory development of abilities that depend on several interrelated brain functions.

Combining MAP-Elites and Incremental Evolution to Generate Gaits for a Mammalian Quadruped Robot

Four-legged mammals are capable of showing a great variety of movement patterns, ranging from a simple walk to more complex movement such as trots and gallops. Imbuing this diversity to quadruped robots is of interest in order to improve both mobility and reach. Within the field of Evolutionary Robotics, Quality Diversity techniques have shown a remarkable ability to produce not only effective, but also highly diverse solutions. When applying this approach to four-legged robots an initial problem is to create viable movement patterns that do not fall. This difficulty stems from the challenging fitness gradient due to the mammalian morphology. In this paper we propose a solution to overcome this problem by implementing incremental evolution within the Quality Diversity framework. This allows us to evolve controllers that become more complex while at the same time utilizing the diversity produced by Quality Diversity. We show that our approach is able to generate high fitness solutions early in the search process, keep these solutions and perform a more open-ended search towards the end of evolution.

Multiobjective Coverage Path Planning: Enabling Automated Inspection of Complex, Real-World Structures

Abstract An important open problem in robotic planning is the autonomous generation of 3D inspection paths – that is, planning the best path to move a robot along in order to inspect a target structure. We recently suggested a new method for planning paths allowing the inspection of complex 3D structures, given a triangular mesh model of the structure. The method differs from previous approaches in its emphasis on generating and considering also plans that result in imperfect coverage of the inspection target. In many practical tasks, one would accept imperfections in coverage if this results in a substantially more energy efficient inspection path. The key idea is using a multiobjective evolutionary algorithm to optimize the energy usage and coverage of inspection plans simultaneously – and the result is a set of plans exploring the different ways to balance the two objectives. We here test our method on a set of inspection targets with large variation in size and complexity, and compare its performance with two state-of-the-art methods for complete coverage path planning. The results strengthen our confidence in the ability of our method to generate good inspection plans for different types of targets. The methods advantage is most clearly seen for real-world inspection targets, since traditional complete coverage methods have no good way of generating plans for structures with hidden parts. Multiobjective evolution, by optimizing energy usage and coverage together, ensures a good balance between the two – both when 100% coverage is feasible, and when large parts of the object are hidden.

Different regulation mechanisms for evolved critical periods in learning

The learning of various skills and behaviors in animals and humans goes through phases known as critical periods, where plasticity is temporarily facilitated. The experiments presented here are designed to shed further light on this phenomenon, by investigating three mechanisms that have been suggested for controlling the timing of critical periods. By evolving neural networks controlled by these mechanisms, we gain an insight into their strengths and weaknesses. We find that the mechanisms based on the developmental stage of an individual can control plasticity under predictable conditions, whereas we need to control plasticity as a function of attention level to adapt successfully to new and uncertain environments.

Dynamic robot scheduling using a genetic algorithm

This paper describes a genetic algorithm that was developed for optimizing plans in a robotic competition. The algorithm was used both as a static planner, making plans before matches, and as a dynamic replanner during matches, a task with much stricter demands of efficiency. The genetic algorithm was hybridized with a local search technique, which experiments proved essential to finding good solutions in this complex task. To enable rapid response under environmental changes, a heuristic for immediate response and a contingency planning module were also implemented. Experiments proved that the algorithm was able to generate good plans, and continuously modify them in light of a rapidly changing environment.

Evolving neural networks with multiple internal models.

Internal models allow us to simulate and predict the consequences of interacting with the objects in our environment. Applying such models in intelligent robots and machines is a key challenge in increasing their autonomy, robustness and responsiveness. One obstacle in allowing this is the need to maintain multiple internal models, corresponding to the multitude of objects in our surroundings, without interference between them. We propose evolving neural networks as a way to generate multiple internal models, and study the role of neural modularity in doing so. Intuitively, modularity should help reduce interference between internal models. In a task requiring neural networks to control multiple different objects, we demonstrate that neuroevolution can produce multiple internal inverse models. Results indicate that modularity may play a role – but the evolved neural networks reveal an unexpected modular decomposition: Rather than separating models of different objects, networks frequently divide into modu...

Deep Models for Ensemble Touch-Screen Improvisation

For many, the pursuit and enjoyment of musical performance goes hand-in-hand with collaborative creativity, whether in a choir, jazz combo, orchestra, or rock band. However, few musical interfaces use the affordances of computers to create or enhance ensemble musical experiences. One possibility for such a system would be to use an artificial neural network (ANN) to model the way other musicians respond to a single performer. Some forms of music have well-understood rules for interaction; however, this is not the case for free improvisation with new touch-screen instruments where styles of interaction may be discovered in each new performance. This paper describes an ANN model of ensemble interactions trained on a corpus of such ensemble touch-screen improvisations. The results show realistic ensemble interactions and the model has been used to implement a live performance system where a performer is accompanied by the predicted and sonified touch gestures of three virtual players.