Michalis Smyrnakis

Dynamic Opponent Modelling in Fictitious Play

Distributed optimization can be formulated as an n-player coordination game. One of the most common learning techniques in game theory is fictitious play and its variations. However, fictitious play is founded on an implicit assumption that opponents’ strategies are stationary. In this paper we present a new variation of fictitious play in which players predict opponents’ strategy using a particle filter algorithm. This allows us to use a more realistic model of opponent strategy. We used pre-specified opponents’ strategies to examine if our algorithm can efficiently track the strategies. Furthermore, we have used these experiments to examine the impact of different values of our algorithm parameters on the results of strategy tracking. We then compared the results of the proposed algorithm with those of stochastic and geometric fictitious play in three different strategic form games: a potential game and two climbing hill games, one with two players and the other with three players. We also tested our algorithm in two different distributed optimization scenarios, a vehicle-target assignment game and a disaster management problem. Our algorithm converges to the optimum faster than both the competitor algorithms in the strategic form games and the vehicle-target assignment game. Hence by placing a greater computational demand on the individual agents, less communication is required between the agents. In the disaster management scenario we compared the results of particle filter fictitious play with the ones of Matlabs centralized algorithm bintprog and the centralized pre-planning algorithm of (Gelenbe, E. and Timotheou, S. (2008) Random neural networks with synchronized interactions. Neural Comput., 20(9), 2308–2324). In this scenario our algorithm performed better than the pre-planning algorithm in two of the three performance measures we used.

Sequentially updated Probability Collectives

Multi-agent coordination problems can be cast as distributed optimization tasks. Probability Collectives (PCs) are techniques that deal with such problems in discrete and continuous spaces [15]. In this paper we are going to propose a new variation of PCs, Sequentially updated Probability Collectives. Our objective is to show how standard techniques from the statistics literature, Sequential Monte Carlo methods and kernel regression, can be used as building blocks within PCs instead of the ad hoc approaches taken previously to produce samples and estimate values in continuous action spaces. We test our algorithm in three different simulation scenarios with continuous action spaces. Two classical distributed optimization functions, the three and six dimensional Hartman functions [6] and a vehicle target assignment type game [1]. The results for the Hartman functions were close to the global optimum, and the agents managed to coordinate to the optimal solution of the target assignment game.

Classifying ischemic events using a Bayesian inference Multilayer Perceptron and input variable evaluation using automatic relevance determination

In this paper we present a Bayesian inference Multilayer Perceptron (MLP) which was used to classify the events of the Long Term ST Database (LTSTDB) as ischaemic or non-ischaemic episodes with an accuracy of 89.1%, sensitivity of 82.3% and specificity of 91.2% when the accuracy of the winning paper was 90.7%. The Automatic Relevance Determination (ARD) method was used to identify which of the extracted features that were used as input in the Bayesian inference MLP were the most important with respect to the models performance. ARD indicated that DeltaT, a combination of the ST deviation and the duration of the episode, inspired from Langley et al., was the most important feature for determining Ischaemic episodes, given the data. A simple MLP which had as input variable of only DeltaT was trained to verify the results of the ARD method. The classification accuracy was 85.8% on the test set. We can conclude from the results that the most important extracted feature was DeltaT.

Effects of communication and utility-based decision making in a simple model of evacuation

We present a simple cellular automaton based model of decision making during evacuation. Evacuees have to choose between two different exit routes, resulting in a strategic decision making problem. Agents take their decisions based on utility functions, these can be revised as the evacuation proceeds, leading to complex interaction between individuals and to jamming transitions. The model also includes the possibility to communicate and exchange information with distant agents, information received may affect the decision of agents. We show that under a wider range of evacuation scenarios performance of the model system as a whole is optimal at an intermediate fraction of evacuees with access to communication.

Learning of cooperative behaviour in robot populations

This paper addresses convergence and equilibrium properties of game theoretic learning algorithms in robot populations using simple and broadly applicable reward/cost models of cooperation between robotic agents. New models for robot cooperation are proposed by combining regret based learning methods and network evolution models. Results of mean-field game theory are employed in order to show the asymptotic second moment boundedness in the variation of cooperative behaviour. The behaviour of the proposed models are tested in simulation results, which are based on sample networks and a single lane traffic flow case study.

City evacuations: An interdisciplinary approach

Evacuating a city is a complex problem that involves issues of governance, preparedness education, warning, information sharing, population dynamics, resilience and recovery. As natural and anthropogenic threats to cities grow, it is an increasingly pressing problem for policy makers and practitioners. The book is the result of a unique interdisciplinary collaboration between researchers in the physical and social sciences to consider how an interdisciplinary approach can help plan for large scale evacuations. It draws on perspectives from physics, mathematics, organisation theory, economics, sociology and education. Importantly it goes beyond disciplinary boundaries and considers how interdisciplinary methods are necessary to approach a complex problem involving human actors and increasingly complex communications and transportation infrastructures. Using real world case studies and modelling the book considers new approaches to evacuation dynamics. It addresses questions of complexity, not only in terms of theory, but examining the latest challenges for cities and emergency responders. Factors such as social media, information quality and visualisation techniques are examined to consider the ‘new’ dynamics of warning and informing, evacuation and recovery.

Decentralized Optimisation of Resource Allocation in Disaster Management

A resource-allocation problem derived from a scenario in disaster management is studied using computer simulations of game theoretic learning algorithms. Specifically we consider a scenario in which a number of incidents occur in an emergency, and where multiple resources need to be delivered to each incident by a limited number of carriers. Assuming that communication with a central decision maker is disrupted we map the scenario to a problem in game theory, and use several learning rules, based on the celebrated fictitious play algorithm to find optimal solutions.

Robustness of Fictitious Play in a Resource Allocation Game

Nowadays it is well known that decentralised optimisation tasks can be represented as so-called “potential games”. An example of a resource allocation problem that can be cast as a game is the “vehicle-target assignment problem” originally proposed by Marden et al.

Convergence of probability collectives with adaptive choice of temperature parameters

There are numerous applications of multi-agent systems like disaster management [1], sensor networks [2], traffic control [3] and scheduling problems [4] where agents should coordinate to achieve a common goal. In most of these cases a centralized solution is inefficient because of the scale and the complexity of the problems and thus distributed solutions are required.