Pablo García-Sánchez

Effect of Noisy Fitness in Real-Time Strategy Games Player Behaviour Optimisation Using Evolutionary Algorithms

This paper investigates the performance and the results of an evolutionary algorithm (EA) specifically designed for evolving the decision engine of a program (which, in this context, is called bot) that plays Planet Wars. This game, which was chosen for the Google Artificial Intelligence Challenge in 2010, requires the bot to deal with multiple target planets, while achieving a certain degree of adaptability in order to defeat different opponents in different scenarios. The decision engine of the bot is initially based on a set of rules that have been defined after an empirical study, and a genetic algorithm (GA) is used for tuning the set of constants, weights and probabilities that those rules include, and therefore, the general behaviour of the bot. Then, the bot is supplied with the evolved decision engine and the results obtained when competing with other bots (a bot offered by Google as a sparring partner, and a scripted bot with a pre-established behaviour) are thoroughly analysed. The evaluation of the candidate solutions is based on the result of non-deterministic battles (and environmental interactions) against other bots, whose outcome depends on random draws as well as on the opponents’ actions. Therefore, the proposed GA is dealing with a noisy fitness function. After analysing the effects of the noisy fitness, we conclude that tackling randomness via repeated combats and reevaluations reduces this effect and makes the GA a highly valuable approach for solving this problem.

Optimizing player behavior in a real-time strategy game using evolutionary algorithms

This paper describes an Evolutionary Algorithm for evolving the decision engine of a bot designed to play the Planet Wars game. This game, which has been chosen for the Google Artificial Intelligence Challenge in 2010, requires that the artificial player is able to deal with multiple objectives, while achieving a certain degree of adaptability in order to defeat different opponents in different scenarios. The decision engine of the bot is based on a set of rules that have been defined after an empirical study. Then, an Evolutionary Algorithm is used for tuning the set of constants, weights and probabilities that define the rules, and, therefore, the global behavior of the bot. The paper describes the Evolutionary Algorithm and the results attained by the decision engine when competing with other bots. The proposed bot defeated a baseline bot in most of the playing environments and obtained a ranking position in top-20% of the Google Artificial Intelligence competition.

Service oriented evolutionary algorithms

This work presents a service oriented architecture for evolutionary algorithms, and an implementation of this architecture using a specific technology (called OSGiLiath). Service oriented architecture is a computational paradigm where users interact using services to increase the integration between systems. The presented abstract architecture is formed by loosely coupled, highly configurable and language-independent services. As an example of an implementation of this architecture, a complete process development using a specific service oriented technology is explained. With this implementation, less effort than classical development in integration, distribution mechanisms and execution time management has been attained. In addition, steps, ideas, advantages and disadvantages, and guidelines to create service oriented evolutionary algorithms are presented. Using existing software, or from scratch, researchers can create services to increase the interoperability in this area.

Cloud-based evolutionary algorithms: An algorithmic study

This paper presents a cloud-computing based evolutionary algorithm using a synchronous storage service as pool for exchange information among population of solutions. The multi-computer was composed of several normal PCs or laptops connected via Wifi or Ethernet. In this work the effect of how the distributed evolutionary algorithm reached the solution when new PCs was added was tested whether that effect also translates to the algorithmic performance of the algorithm. To this end different (and hard) problems was addressed using the proposed multi-computer, analyzing the effects that the automatic load-balancing and synchronization had on the speed of algorithm successful, and analyzing how the number of evaluation per second increases when the multi-computer includes new nodes. The measure used for the analysis was number of evaluation per second which was increased when the multi-computer includes new nodes. The algorithm solved the proposed problems and it was viable to run it in homogeneous or heterogeneous platforms. The experiments includes two problems and different configuration for the distributed evolutionary algorithm in order to check the results of the algorithm for several rates of information exchange with the selected storage service. Results shows that the system is viable with homogeneous or heterogeneous nodes and there is no significative differences for the synchronous storage services we have tested. But when the problem is harder, and the threads of the algorithm does not stop for each information exchange (migration of individual from one population to another one), the differences of using a specific service became significative in terms of success of the algorithm.

Deploying intelligent e-health services in a mobile gateway

This work presents the design of a mobile gateway for independent life and e-health support. Technologies such as OSGi and Web Services have been used to deploy several services as an example, such as bio-medical parameter monitorization, alerts, and communication with a coordination center. A complete functional service which uses context-awareness has been explained and the benefits of using Service Oriented Architecture are explained. Finally, the lessons learned of this kind of development, which involves several types of systems, data, languages, and development groups, are discussed.

Adaptive bots for real-time strategy games via map characterization

This paper presents a proposal for a fast on-line map analysis for the RTS game Planet Wars in order to define specialized strategies for an autonomous bot. This analysis is used to tackle two constraints of the game, as featured in the Google AI Challenge 2010: the players cannot store any information from turn to turn, and there is a limited action time of just one second. They imply that the bot must analyze the game map quickly, to adapt its strategy during the game. Based in our previous work, in this paper we have evolved bots for different types of maps. Then, all bots are combined in one, to choose the evolved strategy depending on the geographical configuration of the game in each turn. Several experiments have been conducted to test the new approach, which outperforms our previous version, based on an off-line general training.

Towards automatic StarCraft strategy generation using genetic programming

Among Real-Time Strategy games few titles have enjoyed the continued success of StarCraft. Many research lines aimed at developing Artificial Intelligences, or “bots”, capable of challenging human players, use StarCraft as a platform. Several characteristics make this game particularly appealing for researchers, such as: asymmetric balanced factions, considerable complexity of the technology trees, large number of units with unique features, and potential for optimization both at the strategical and tactical level. In literature, various works exploit evolutionary computation to optimize particular aspects of the game, from squad formation to map exploration; but so far, no evolutionary approach has been applied to the development of a complete strategy from scratch. In this paper, we present the preliminary results of StarCraftGP, a framework able to evolve a complete strategy for StarCraft, from the building plan, to the composition of squads, up to the set of rules that define the bots behavior during the game. The proposed approach generates strategies as C++ classes, that are then compiled and executed inside the OpprimoBot open-source framework. In a first set of runs, we demonstrate that StarCraftGP ultimately generates a competitive strategy for a Zerg bot, able to defeat several human-designed bots.

Pareto-based multi-colony multi-objective ant colony optimization algorithms: an island model proposal

Multi-objective algorithms are aimed to obtain a set of solutions, called Pareto set, covering the whole Pareto front, i.e. the representation of the optimal set of solutions. To this end, the algorithms should yield a wide amount of near-optimal solutions with a good diversity or spread along this front. This work presents a study on different coarse-grained distribution schemes dealing with Multi-Objective Ant Colony Optimization Algorithms (MOACOs). Two of them are a variation of independent multi-colony structures, respectively having a fixed number of ants in every subset or distributing the whole amount of ants into small sub-colonies. We introduce in this paper a third method: an island-based model where the colonies communicate by migrating ants, following a neighbourhood topology which fits to the search space. All the methods are aimed to cover the whole PF, thus each sub-colony or island tries to search for solutions in a limited area, complemented by the rest of colonies, in order to obtain a more diverse high-quality set of solutions. The models have been tested by implementing them considering three different MOACOs: two well-known and CHAC, an algorithm previously proposed by the authors. Three different instances of the bi-Criteria travelling salesman problem have been considered. The experiments have been performed in a parallel environment (inside a cluster platform), in order to get a time improvement. Moreover, the system scaling factor with respect to the number of processors will be also analysed. The results show that the proposed Pareto-island model and its novel neighbourhood topology performs better than the other models, yielding a more diverse and more optimized set of solutions. Moreover, from the algorithmic point of view, the proposed algorithm, named CHAC, yields the best results on average.

Dealing with noisy fitness in the design of a RTS game bot

This work describes an evolutionary algorithm (EA) for evolving the constants, weights and probabilities of a rule-based decision engine of a bot designed to play the Planet Wars game. The evaluation of the individuals is based on the result of some non-deterministic combats, whose outcome depends on random draws as well as the enemy action, and is thus noisy. This noisy fitness is addressed in the EA and then, its effects are deeply analysed in the experimental section. The conclusions shows that reducing randomness via repeated combats and re-evaluations reduces the effect of the noisy fitness, making then the EA an effective approach for solving the problem.

The Use of Video-Gaming Devices as a Motivation for Learning Embedded Systems Programming

As embedded systems are becoming prevalent in everyday life, many universities are incorporating embedded systems-related courses in their undergraduate curricula. However, it is not easy to motivate students in such courses since they conceive of embedded systems as bizarre computing elements, different from the personal computers with which they are familiar. This problem has been overcome at the University of Granada, Spain, by taking advantage of the connection many students have with video games.