Stefano Nolfi

Learning and evolution in neural networks

This article describes simulations on populations of neural networks that both evolve at the population level and learn at the individual level. Unlike other simulations, the evolutionary task (finding food in the environment) and the learning task (predicting the next position of food on the basis of present position and planned networks movement) are different tasks. In these conditions, learning influences evolution (without Lamarckian inheritance of learned weight changes) and evolution influences learning. Average but not peak fitness has a better evolutionary growth with learning than without learning. After the initial generations, individuals that learn to predict during life also improve their food-finding ability during life. Furthermore, individuals that inherit an innate capacity to find food also inherit an innate predisposition to learn to predict the sensory consequences of their movements. They do not predict better at birth, but they do learn to predict better than individuals of the initial generation given the same learning experience. The results are interpreted in terms of a notion of dynamic correlation between the fitness surface and the learning surface. Evolution succeeds in finding both individuals that have high fitness and individuals that, although they do not have high fitness at birth, end up with high fitness because they learn to predict.

Swarm-Bot: A New Distributed Robotic Concept

The swarm intelligence paradigm has proven to have very interesting properties such as robustness, flexibility and ability to solve complex problems exploiting parallelism and self-organization. Several robotics implementations of this paradigm confirm that these properties can be exploited for the control of a population of physically independent mobile robots.The work presented here introduces a new robotic concept called swarm-bot in which the collective interaction exploited by the swarm intelligence mechanism goes beyond the control layer and is extended to the physical level. This implies the addition of new mechanical functionalities on the single robot, together with new electronics and software to manage it. These new functionalities, even if not directly related to mobility and navigation, allow to address complex mobile robotics problems, such as extreme all-terrain exploration.The work shows also how this new concept is investigated using a simulation tool (swarmbot3d) specifically developed for quickly designing and evaluating new control algorithms. Experimental work shows how the simulated detailed representation of one s-bot has been calibrated to match the behaviour of the real robot.

Evolving Self-Organizing Behaviors for a Swarm-Bot

In this paper, we introduce a self-assembling and self-organizing artifact, called a swarm-bot, composed of a swarm of s-bots, mobile robots with the ability to connect to and to disconnect from each other. We discuss the challenges involved in controlling a swarm-bot and address the problem of synthesizing controllers for the swarm-bot using artificial evolution. Specifically, we study aggregation and coordinated motion of the swarm-bot using a physics-based simulation of the system. Experiments, using a simplified simulation model of the s-bots, show that evolution can discover simple but effective controllers for both the aggregation and the coordinated motion of the swarm-bot. Analysis of the evolved controllers shows that they have properties of scalability, that is, they continue to be effective for larger group sizes, and of generality, that is, they produce similar behaviors for configurations different from those they were originally evolved for. The portability of the evolved controllers to real s-bots is tested using a detailed simulation model which has been validated against the real s-bots in a companion paper in this same special issue.

Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems

This paper describes how agents can learn an internal model of the world structurally by focusing on the problem of behavior-based articulation. We develop an on-line learning scheme-the so-called mixture of recurrent neural net (RNN) experts-in which a set of RNN modules become self-organized as experts on multiple levels, in order to account for the different categories of sensory-motor flow which the robot experiences. Autonomous switching of activated modules in the lower level actually represents the articulation of the sensory-motor flow. In the meantime, a set of RNNs in the higher level competes to learn the sequences of module switching in the lower level, by which articulation at a further, more abstract level can be achieved. The proposed scheme was examined through simulation experiments involving the navigation learning problem. Our dynamical system analysis clarified the mechanism of the articulation. The possible correspondence between the articulation mechanism and the attention switching mechanism in thalamo-cortical loops is also discussed.

Evolving mobile robots able to display collective behaviors

We present a set of experiments in which simulated robots are evolved for the ability to aggregate and move together toward a light target. By developing and using quantitative indexes that capture the structural properties of the emerged formations, we show that evolved individuals display interesting behavioral patterns in which groups of robots act as a single unit. Moreover, evolved groups of robots with identical controllers display primitive forms of situated specialization and play different behavioral functions within the group according to the circumstances. Overall, the results presented in the article demonstrate that evolutionary techniques, by exploiting the self-organizing behavioral properties that emerge from the interactions between the robots and between the robots and the environment, are a powerful method for synthesizing collective behavior.

Coevolving Predator and Prey Robots: Do Arms Races Arise in Artificial Evolution?

Coevolution (i.e., the evolution of two or more competing populations with coupled fitness) has several features that may potentially enhance the power of adaptation of artificial evolution. In particular, as discussed by Dawkins and Krebs [3], competing populations may reciprocally drive one another to increasing levels of complexity by producing an evolutionary arms race. In this article we will investigate the role of coevolution in the context of evolutionary robotics. In particular, we will try to understand in what conditions coevolution can lead to arms races. Moreover, we will show that in some cases artificial coevolution has a higher adaptive power than simple evolution. Finally, by analyzing the dynamics of coevolved populations, we will show that in some circumstances well-adapted individuals would be better advised to adopt simple but easily modifiable strategies suited for the current competitor strategies rather than incorporate complex and general strategies that may be effective against a wide range of opposing counter-strategies.

The cooperation of swarm-bots: physical interactions in collective robotics

This paper describes the development of concepts of the SWARM-BOTS project and briefly overviews the outcomes of the project including the mechanical and electronic features of the developed robots. The paper also presents a physics-based simulator suitable to investigate time-consuming adaptive algorithms and shows examples of cooperative behaviors, both in simulation and in hardware. This novel research field addresses the design and implementation of robotic systems composed of swarms of robots that interact and cooperate to reach their goals.

Learning and Evolution

In the last few years several researchers have resorted to artificial evolution (e.g., genetic algorithms) and learning techniques (e.g., neural networks) for studying the interaction between learning and evolution. These studies have been conducted for two different purposes: (a) looking at the performance advantages obtained by combining these two adaptive techniques; (b) understanding the role of the interaction between learning and evolution in biological organisms. In this paper we describe some of the most representative experiments conducted in this area and point out their implications for both perspectives outlined above. Understanding the interaction between learning and evolution is probably one of the best examples in which computational studies have shed light on problems that are difficult to study with the research tools employed by evolutionary biology and biology in general. From an engineering point of view, the most relevant results are those showing that adaptation in dynamic environments gains a significant advantage by the combination of evolution and learning. These studies also show that the interaction between learning and evolution deeply alters the evolutionary and the learning process themselves, offering new perspectives from a biological point of view. The study of learning within an evolutionary perspective is still in its infancy and in the forthcoming years it will produce an enormous impact on our understanding of how learning and evolution operate.

Cell division and migration in a ‘genotype’ for neural networks

Much research has recently been dedicated to applying genetic algorithms to populations of neural networks. However. While in real organisms the inherited genotype maps in complex ways into the resulting phenotype, in most of this research the development process that creates the individual phenotype is ignored. In this paper we present a model of neural development which includes cell division and cell migration in addition to axonal growth and branching. This reflects, in a very simplified way, what happens in the ontogeny of real organisms. The development process of our artificial organisms shows successive phases of functional differentiation and specialization. In addition, we find that mutations that affect different phases of development have very different evolutionary consequences. A single change in the early stages of cell division and migration can have huge effects on the phenotype, while changes in later stages usually have a less dramatic impact. Sometimes, changes that affect the first de...

Econets: Neural networks that learn in an environment

Ecological networks are networks that learn in an environment. It is the environment, and not the researcher, that determines the conditions in which learning takes place such as which input patterns are seen, what the teaching input is, etc. Furthermore, input patterns at time N+1 are often a function of the output of the network at time N. Two hypotheses are explored with reference to ecological networks. One is that predicting the sensory consequences (input) for an organism of the organisms actions (output) on the environment is one of the basic tasks of this type of network—basic for constructing an environmental map or world model. The other is that learning to predict the sensory consequences of the organisms actions favourably predisposes the organism to learn to attain goals with those actions. Some data from simulations that support these two hypotheses are reported.