S.C.J. Bakkes

Rapid and Reliable Adaptation of Video Game AI

Current approaches to adaptive game AI typically require numerous trials to learn effective behavior (i.e., game adaptation is not rapid). In addition, game developers are concerned that applying adaptive game AI may result in uncontrollable and unpredictable behavior (i.e., game adaptation is not reliable). These characteristics hamper the incorporation of adaptive game AI in commercially available video games. In this paper, we discuss an alternative to these current approaches. Our alternative approach to adaptive game AI has as its goal adapting rapidly and reliably to game circumstances. Our approach can be classified in the area of case-based adaptive game AI. In the approach, domain knowledge required to adapt to game circumstances is gathered automatically by the game AI, and is exploited immediately (i.e., without trials and without resource-intensive learning) to evoke effective behavior in a controlled manner in online play. We performed experiments that test case-based adaptive game AI on three different maps in a commercial real-time strategy (RTS) game. From our results, we may conclude that case-based adaptive game AI provides a strong basis for effectively adapting game AI in video games.

Opponent modelling for case-based adaptive game AI

Abstract In previous work we introduced a novel approach to adaptive game AI that was focussed on the rapid and reliable adaptation to game circumstances. We named the approach ‘case-based adaptive game AI’. In the approach, domain knowledge required to adapt to game circumstances is gathered automatically by the game AI, and is exploited immediately (i.e., without trials and without resource-intensive learning) to evoke effective behaviour in a controlled manner in online play. In the research discussed in this article we investigate to what extent incorporating opponent modelling enhances the performance of case-based adaptive game AI. In our approach, models of the opponent players are generated automatically, on the basis of observations drawn from a multitude of games. We performed experiments that test the enhanced approach in an actual, complex RTS game, and observed that the effectiveness of case-based adaptive game AI increases significantly when opponent modelling is incorporated. From these results we may conclude that opponent modelling further improves the basis for implementation of case-based adaptive game AI in commercially available video games.

Dynamic formations in real-time strategy games

Current approaches to organising units in strategic video games are typically implemented via static formations. Static formations are not capable of adapting effectively to opponent tactics. In this paper we discuss an approach to organising units by learning the effectiveness of a formation in actual play, and directly applying learned formations according to the classification of the opponent player. This approach to establish so-called dynamic formations, is tested in the ORTS game environment. From our results, we may conclude that the approach to established dynamic formations can be successfully applied in actual video-game environments.

Rapid adaptation of video game AI

Current approaches to adaptive game AI require either a high quality of utilised domain knowledge, or a large number of adaptation trials. These requirements hamper the goal of rapidly adapting game AI to changing circumstances. In an alternative, novel approach, domain knowledge is gathered automatically by the game AI, and is immediately (i.e., without trials and without resource-intensive learning) utilised to evoke effective behaviour. In this paper we discuss this approach, called dasiarapidly adaptive game AIpsila. We perform experiments that apply the approach in an actual video game. From our results we may conclude that rapidly adaptive game AI provides a strong basis for effectively adapting game AI in actual video games.

Monte Carlo Tree Search with options for general video game playing

General video game playing is a challenging research area in which the goal is to find one algorithm that can play many games successfully. “Monte Carlo Tree Search” (MCTS) is a popular algorithm that has often been used for this purpose. It incrementally builds a search tree based on observed states after applying actions. However, the MCTS algorithm always plans over actions and does not incorporate any higher level planning, as one would expect from a human player. Furthermore, although many games have similar game dynamics, often no prior knowledge is available to general video game playing algorithms. In this paper, we introduce a new algorithm called “Option Monte Carlo Tree Search” (O-MCTS). It offers general video game knowledge and high level planning in the form of “options”, which are action sequences aimed at achieving a specific subgoal. Additionally, we introduce “Option Learning MCTS” (OL-MCTS), which applies a progressive widening technique to the expected returns of options in order to focus exploration on fruitful parts of the search tree. Our new algorithms are compared to MCTS on a diverse set of twenty-eight games from the general video game AI competition. Our results indicate that by using MCTSs efficient tree searching technique on options, O-MCTS outperforms MCTS on most of the games, especially those in which a certain subgoal has to be reached before the game can be won. Lastly, we show that OL-MCTS improves its performance on specific games by learning expected values for options and moving a bias to higher valued options.

Personalized crisis management training on a tablet

In this paper, we propose a framework for personalised crisis management training through the use of an applied game. The framework particularly focuses on ubiquitously assessing and manipulating player stress levels during training, and evaluating player performance by providing personalised feedback. To achieve these goals, the framework leverages techniques for multi-modal player modeling through physiological sensors, in-game events and self-report data. Specifically, the present paper (1) discusses design decisions for the personalised crisis management training framework, and (2) presents the game prototype with which user-studies will be performed. Presently, the game prototype is being developed in close collaboration with actual crisis management experts.

Procedural zelda: a PCG environment for player experience research

To contribute to the domain of player experience research, this paper presents a new PCG environment with a relatively wide expressive range that builds upon the iconic The Legend of Zelda: A Link to the Past action-RPG game; it contributes by providing the openly-available Procedural Zelda environment for gaming research. The paper presents the design goals and design context of the research environment, and provides a detailed overview of the procedural capabilities of Procedural Zelda, together with its capabilities for data logging, to benefit, e.g., player modelling investigations.

Automated game balancing of asymmetric video games

Designing a (video) game such that it is balanced — i.e. fair for all players — is a prevailing challenge in game design. Perhaps counter-intuitively, games that are symmetric with respect to (board) design, starting conditions, and the employed action set, are not necessarily fair games. Indeed, perfect play from all players does not automatically lead to a draw, but may probabilistically favour e.g., the first player to move. Even more so, asymmetric games — in which the action set of one player is typically highly distinct from that of another player — are generally unbalanced unless meticulous care has been taken to ensure that the asymmetry in the design does not skew win probabilities. In this context, the present paper contributes a method for automatically balancing the design of asymmetric games. It employs Monte Carlo simulation to analyse the relative impact of game actions, and iteratively adjusts attributes of the game actions till the game design is balanced by approximation. To assess the effectiveness of the proposed method, experiments were performed with automatically balancing a set of tower-defence games. Preliminary experimental results revealed that the proposed method (1) is able to identify the principal component of a games imbalance, and (2) can automatically adjust the game design till it is balanced by approximation.