Pieter Spronck

Adaptive game AI with dynamic scripting

Online learning in commercial computer games allows computer-controlled opponents to adapt to the way the game is being played. As such it provides a mechanism to deal with weaknesses in the game AI, and to respond to changes in human player tactics. We argue that online learning of game AI should meet four computational and four functional requirements. The computational requirements are speed, effectiveness, robustness and efficiency. The functional requirements are clarity, variety, consistency and scalability. This paper investigates a novel online learning technique for game AI called ‘dynamic scripting’, that uses an adaptive rulebase for the generation of game AI on the fly. The performance of dynamic scripting is evaluated in experiments in which adaptive agents are pitted against a collection of manually-designed tactics in a simulated computer roleplaying game. Experimental results indicate that dynamic scripting succeeds in endowing computer-controlled opponents with adaptive performance. To further improve the dynamic-scripting technique, an enhancement is investigated that allows scaling of the difficulty level of the game AI to the human player’s skill level. With the enhancement, dynamic scripting meets all computational and functional requirements. The applicability of dynamic scripting in state-of-the-art commercial games is demonstrated by implementing the technique in the game Neverwinter Nights. We conclude that dynamic scripting can be successfully applied to the online adaptation of game AI in commercial computer games.

Rapid and Reliable Adaptation of Video Game AI

Current approaches to adaptive game AI typically require numerous trials to learn effective behavior (i.e., game adaptation is not rapid). In addition, game developers are concerned that applying adaptive game AI may result in uncontrollable and unpredictable behavior (i.e., game adaptation is not reliable). These characteristics hamper the incorporation of adaptive game AI in commercially available video games. In this paper, we discuss an alternative to these current approaches. Our alternative approach to adaptive game AI has as its goal adapting rapidly and reliably to game circumstances. Our approach can be classified in the area of case-based adaptive game AI. In the approach, domain knowledge required to adapt to game circumstances is gathered automatically by the game AI, and is exploited immediately (i.e., without trials and without resource-intensive learning) to evoke effective behavior in a controlled manner in online play. We performed experiments that test case-based adaptive game AI on three different maps in a commercial real-time strategy (RTS) game. From our results, we may conclude that case-based adaptive game AI provides a strong basis for effectively adapting game AI in video games.

An overview of genetic algorithms applied to control engineering problems

Genetic algorithms (GAs) are the most widely known evolutionary search algorithms. While they are regularly applied to control engineering problems, currently they are not a standard tool in the control engineers toolbox. This may in part be the result of the fact that few general overview of the application of GAs to control engineering problems yet exists, and the fact that they are usually reported on at conferences of computer scientists, not of control engineers. This paper attempts to alleviate that omission by presenting an overview of recent applications of GAs in the field of control engineering.

Knowledge acquisition for adaptive game AI

Game artificial intelligence (AI) controls the decision-making process of computer-controlled opponents in computer games. Adaptive game AI (i.e., game AI that can automatically adapt the behaviour of the computer players to changes in the environment) can increase the entertainment value of computer games. Successful adaptive game AI is invariably based on the games domain knowledge. We show that an offline evolutionary algorithm can learn important domain knowledge in the form of game tactics (i.e., a sequence of game actions) for dynamic scripting, an offline algorithm inspired by reinforcement learning approaches that we use to create adaptive game AI. We compare the performance of dynamic scripting under three conditions for defeating non-adaptive opponents in a real-time strategy game. In the first condition, we manually encode its tactics. In the second condition, we manually translate the tactics learned by the evolutionary algorithm, and use them for dynamic scripting. In the third condition, this translation is automated. We found that dynamic scripting performs best under the third condition, and both of the latter conditions outperform manual tactic encoding. We discuss the implications of these results, and the performance of dynamic scripting for adaptive game AI from the perspective of machine learning research and commercial game development.

Games as personality profiling tools

In this paper we investigate whether a personality profile can be determined by observing a players behavior in a game. Five personality traits are used to define a personality profile. They are adopted from the Five Factor Model of personality. The five traits are measured by the NEO-PI-R questionnaire. For our purpose, we developed a game module for the game Neverwinter Nights. The module automatically stores a players behavioral data. Experimental trials were run measuring the behavior of 44 participants. The experiment produced game behavior scores for 275 game variables per player. Correlation analysis shows relationships between all five personality traits and the video game data. From these results, we may conclude that a video game can be used to create an adequate personality profile of a player.

Player behavioural modelling for video games

Player behavioural modelling has grown from a means to improve the playing strength of computer programs that play classic games (e.g., chess), to a means for impacting the player experience and satisfaction in video games, as well as in cross-domain applications such as interactive storytelling. In this context, player behavioural modelling is concerned with two goals, namely (1) providing an interesting or effective game AI on the basis of player models and (2) creating a basis for game developers to personalise gameplay as a whole, and creating new user-driven game mechanics. In this article, we provide an overview of player behavioural modelling for video games by detailing four distinct approaches, namely (1) modelling player actions, (2) modelling player tactics, (3) modelling player strategies, and (4) player profiling. We conclude the article with an analysis on the applicability of the approaches for the domain of video games.

Opponent modelling for case-based adaptive game AI

Abstract In previous work we introduced a novel approach to adaptive game AI that was focussed on the rapid and reliable adaptation to game circumstances. We named the approach ‘case-based adaptive game AI’. In the approach, domain knowledge required to adapt to game circumstances is gathered automatically by the game AI, and is exploited immediately (i.e., without trials and without resource-intensive learning) to evoke effective behaviour in a controlled manner in online play. In the research discussed in this article we investigate to what extent incorporating opponent modelling enhances the performance of case-based adaptive game AI. In our approach, models of the opponent players are generated automatically, on the basis of observations drawn from a multitude of games. We performed experiments that test the enhanced approach in an actual, complex RTS game, and observed that the effectiveness of case-based adaptive game AI increases significantly when opponent modelling is incorporated. From these results we may conclude that opponent modelling further improves the basis for implementation of case-based adaptive game AI in commercially available video games.

Dynamic formations in real-time strategy games

Current approaches to organising units in strategic video games are typically implemented via static formations. Static formations are not capable of adapting effectively to opponent tactics. In this paper we discuss an approach to organising units by learning the effectiveness of a formation in actual play, and directly applying learned formations according to the classification of the opponent player. This approach to establish so-called dynamic formations, is tested in the ORTS game environment. From our results, we may conclude that the approach to established dynamic formations can be successfully applied in actual video-game environments.

Incongruity-based adaptive game balancing

Commercial games possess various methods of game balancing. Each of them modifies the games entertainment value for players of different skill levels. This paper deals with one of them, viz. a way of automatically adapting a games balance which is based on the theory of incongruity. We tested our approach on a group of subjects, who played a game with three difficulty settings. The idea is to maintain a specific difference in incongruity automatically. We tested our idea extensively and may report that the results coincide with the theory of incongruity as far as positive incongruity is concerned. The main conclusion is that, owing to our automatically maintained balanced difficulty setting, we can avoid that a game becomes boring or frustrating.

Monte-Carlo tree search in settlers of catan

Games are considered important benchmark opportunities for artificial intelligence research. Modern strategic board games can typically be played by three or more people, which makes them suitable test beds for investigating multi-player strategic decision making. Monte-Carlo Tree Search (MCTS) is a recently published family of algorithms that achieved successful results with classical, two-player, perfect-information games such as Go. In this paper we apply MCTS to the multi-player, non-deterministic board game Settlers of Catan. We implemented an agent that is able to play against computer-controlled and human players. We show that MCTS can be adapted successfully to multi-agent environments, and present two approaches of providing the agent with a limited amount of domain knowledge. Our results show that the agent has a considerable playing strength when compared to game implementation with existing heuristics. So, we may conclude that MCTS is a suitable tool for achieving a strong Settlers of Catan player.