Pedro Mariano

Stability of multi-agent systems

This work attempts to shed light on the fundamental concepts behind the stability of multi-agent systems. We view the system as a discrete time Markov chain with a potentially unknown transitional probability distribution. The system will be considered to be stable when its state has converged to an equilibrium distribution. Faced with the non-trivial task of establishing the convergence to such a distribution, we propose a hypothesis testing approach according to which we test whether the convergence of a particular system metric has occurred. We describe some artificial multi-agent ecosystems that were developed and we present results based on these systems which confirm that this approach qualitatively agrees with our intuition.

Simulation of a trading multi-agent system

In a trading scenario agents interact with each other, selling and buying resources. In order to control the behavior of the trading scenario, the interactions must be coordinated. We present a brief discussion of communication types and coordination models applicable in multi-agent systems. We find a programmable tuple space more appropriate to manage and rule the interactions between the trading agents. We discuss the advantages of a trading agent model that deals with the trading strategy, concentrating on what to buy or sell. This relieves the agent from the task of coordinating the negotiations and their revoking or acceptances. This is the task of the programmable tuple space.

Systematic Derivation of Behaviour Characterisations in Evolutionary Robotics

Evolutionary techniques driven by behavioural diversity, such as novelty search, have shown significant potential in evolutionary robotics. These techniques rely on priorly specified behaviour characterisations to estimate the similarity between individuals. Characterisations are typically defined in an ad hoc manner based on the experimenters intuition and knowledge about the task. Alternatively, generic characterisations based on the sensor-effector values of the agents are used. In this paper, we propose a novel approach that allows for systematic derivation of behaviour characterisations for evolutionary robotics, based on a formal description of the agents and their environment. Systematically derived behaviour characterisations (SDBCs) go beyond generic characterisations in that they can contain task-specific features related to the internal state of the agents, environmental features, and relations between them. We evaluate SDBCs with novelty search in three simulated collective robotics tasks. Our results show that SDBCs yield a performance comparable to the task-specific characterisations, in terms of both solution quality and behaviour space exploration.

Novelty-driven cooperative coevolution

Cooperative coevolutionary algorithms (CCEAs) rely on multiple coevolving populations for the evolution of solutions composed of coadapted components. CCEAs enable, for instance, the evolution of cooperative multiagent systems composed of heterogeneous agents, where each agent is modelled as a component of the solution. Previous works have, however, shown that CCEAs are biased toward stability: the evolutionary process tends to converge prematurely to stable states instead of (near-)optimal solutions. In this study, we show how novelty search can be used to avoid the counterproductive attraction to stable states in coevolution. Novelty search is an evolutionary technique that drives evolution toward behavioural novelty and diversity rather than exclusively pursuing a static objective. We evaluate three novelty-based approaches that rely on, respectively (1) the novelty of the team as a whole, (2) the novelty of the agents’ individual behaviour, and (3) the combination of the two. We compare the proposed approaches with traditional fitness-driven cooperative coevolution in three simulated multirobot tasks. Our results show that team-level novelty scoring is the most effective approach, significantly outperforming fitness-driven coevolution at multiple levels. Novelty-driven cooperative coevolution can substantially increase the potential of CCEAs while maintaining a computational complexity that scales well with the number of populations.

Contributions to adaptable agent societies

The adoption of agents as utile companions faces the problem of conciliating the development of complex and intelligent functionalities with the requirements of autonomy mobility and adaptability. Our main focus will be on the agents adaptability. A hybrid agent architecture approach is proposed where a static component, which resides at the users host and includes most of the intelligence and decision support capabilities, is complemented by a mobile component that is aimed at interacting with other agents. Some adaptation strategies, based on classical and fuzzy methodologies, are also discussed using as background scenario a trading market competitive environment with buyer and seller agents interacting in it.

Adapting populations of agents

We control a population of interacting software agents. The agents have a strategy, and receive a payoff for executing that strategy. Unsuccessful agents become extinct. We investigate the repercussions of maintaining a diversity of agents. There is often no economic rationale for this. If maintaining diversity is to be successful, i.e. without lowering too much the payoff for the non-endangered strategies, it has to go on forever, because the non-endangered strategies still get a good payoff, so that they continue to thrive, and continue to endanger the endangered strategies. This is not sustainable if the number of endangered ones is of the same order as the number of non-endangered ones. We also discuss niches, islands. Finally, we combine learning as adaptation of individual agents with learning via selection in a population.

Cooperative Coevolution of Control for a Real Multirobot System

The potential of cooperative coevolutionary algorithms (CCEAs) as a tool for evolving control for heterogeneous multirobot teams has been shown in several previous works. The vast majority of these works have, however, been confined to simulation-based experiments. In this paper, we present one of the first demonstrations of a real multirobot system, operating outside laboratory conditions, with controllers synthesised by CCEAs. We evolve control for an aquatic multirobot system that has to perform a cooperative predator-prey pursuit task. The evolved controllers are transferred to real hardware, and their performance is assessed in a non-controlled outdoor environment. Two approaches are used to evolve control: a standard fitness-driven CCEA, and novelty-driven coevolution. We find that both approaches are able to evolve teams that transfer successfully to the real robots. Novelty-driven coevolution is able to evolve a broad range of successful team behaviours, which we test on the real multirobot system.

Population Dynamics of Centipede Game using an Energy Based Evolutionary Algorithm

In the context of Evolutionary Game Theory, we have developed an evolutionary algorithm without an explicit fitness function or selection function. Instead players obtain energy by playing games. Clonal reproduction subject to mutation occurs when a player’s energy exceeds some threshold. To avoid exponential growth of the population there is a death event that depends on population size. By tweaking with the relation between payoff and energy and with death event, we create another dilemma that a population must overcome: extinction. We demonstrate this phenomena in the Centipede game. Simulations show that if players can only play one of the two positions of this asymmetric game extinctions are common. If players are versatile and can play both positions there are no extinctions.

Novelty Search in Competitive Coevolution

One of the main motivations for the use of competitive coevolution systems is their ability to capitalise on arms races between competing species to evolve increasingly sophisticated solutions. Such arms races can, however, be hard to sustain, and it has been shown that the competing species often converge prematurely to certain classes of behaviours. In this paper, we investigate if and how novelty search, an evolutionary technique driven by behavioural novelty, can overcome convergence in coevolution. We propose three methods for applying novelty search to coevolutionary systems with two species: (i) score both populations according to behavioural novelty; (ii) score one population according to novelty, and the other according to fitness; and (iii) score both populations with a combination of novelty and fitness. We evaluate the methods in a predator-prey pursuit task. Our results show that novelty-based approaches can evolve a significantly more diverse set of solutions, when compared to traditional fitness-based coevolution.

Design choices for adapting bio-hybrid systems with evolutionary computation

In this paper we report ongoing work evolving bio-hybrid societies. We develop robots that are integrated into an animal society and accepted as conspecifics. We are using evolutionary algorithms to optimise robot controllers to affect the behaviour of animals. Fitness evaluation is the result of measuring the effect a robot controller has on these animals. Animal habituation and heterogeneous response are two factors that have a major role in this fitness evaluation. We discuss our choices in designing a fitness evaluation procedure and how using animals as fitness function providers impacts this.