Tse Guan Tan

A comparative investigation of non-linear activation functions in neural controllers for search-based game AI engineering

The creation of intelligent video game controllers has recently become one of the greatest challenges in game artificial intelligence research, and it is arguably one of the fastest-growing areas in game design and development. The learning process, a very important feature of intelligent methods, is the result of an intelligent game controller to determine and control the game objects behaviors’ or actions autonomously. Our approach is to use a more efficient learning model in the form of artificial neural networks for training the controllers. We propose a Hill-Climbing Neural Network (HillClimbNet) that controls the movement of the Ms. Pac-man agent to travel around the maze, gobble all of the pills and escape from the ghosts in the maze. HillClimbNet combines the hill-climbing strategy with a simple, feed-forward artificial neural network architecture. The aim of this study is to analyze the performance of various activation functions for the purpose of generating neural-based controllers to play a video game. Each non-linear activation function is applied identically for all the nodes in the network, namely log-sigmoid, logarithmic, hyperbolic tangent-sigmoid and Gaussian. In general, the results shows an optimum configuration is achieved by using log-sigmoid, while Gaussian is the worst activation function.

Evolving Opposition-Based Pareto Solutions: Multiobjective Optimization Using Competitive Coevolution

Recently a number of researchers have begun exploring the idea of combining Opposition-Based Learning (OBL) with evolutionary algorithms, reinforcement learning, neural networks, swarm intelligence and simulated annealing. However, an area of research that is still in infancy is the application of the OBL concept to coevolution. Hence, in this chapter, two new opposition-based competitive coevolution algorithms for multiobjective optimization called SPEA2-CE-HOF and SPEA2-CE-KR are discussed. These hybrid algorithms are the combination of Strength Pareto Evolutionary Algorithm 2 (SPEA2) with two types of the competitive fitness strategies, which are the Hall of Fame (HOF) and K-Random Opponents (KR), respectively. The selection of individuals as the opponents in the coevolutionary process strongly implements this opposition-based concept. Scalability tests have been conducted to evaluate and compare both algorithms against the original SPEA2 for seven Deb, Thiele, Laumanns, and Zitzler (DTLZ) test problems with 3 to 5 objectives. The experimental results show clearly that the performance scalability of the opposition-based SPEA2-CE-HOF and SPEA2-CE-KR were significantly better compared to the original non-opposition-based SPEA2 as the number of the objectives becomes higher in terms of the closeness to the true Pareto front, diversity maintenance and the coverage level.

Single- versus Multiobjective Optimization for Evolution of Neural Controllers in Ms. Pac-Man

The objective of this study is to focus on the automatic generation of game artificial intelligence (AI) controllers for Ms. Pac-Man agent by using artificial neural network (ANN) and multiobjective artificial evolution. The Pareto Archived Evolution Strategy (PAES) is used to generate a Pareto optimal set of ANNs that optimize the conflicting objectives of maximizing Ms. Pac-Man scores (screen-capture mode) and minimizing neural network complexity. This proposed algorithm is called Pareto Archived Evolution Strategy Neural Network or PAESNet. Three different architectures of PAESNet were investigated, namely, PAESNet with fixed number of hidden neurons (PAESNet_F), PAESNet with varied number of hidden neurons (PAESNet_V), and the PAESNet with multiobjective techniques (PAESNet_M). A comparison between the single- versus multiobjective optimization is conducted in both training and testing processes. In general, therefore, it seems that PAESNet_F yielded better results in training phase. But the PAESNet_M successfully reduces the runtime operation and complexity of ANN by minimizing the number of hidden neurons needed in hidden layer and also it provides better generalization capability for controlling the game agent in a nondeterministic and dynamic environment.

Neural network ensembles for video game AI using evolutionary multi-objective optimization

Recently, there has been an increasing interest in game artificial intelligence (AI). Game AI is a system that makes the game characters behave like human beings that is able to make smart decisions to achieve the target in a computer or video game. Thus, this study focuses on an automated method of generating artificial neural network (ANN) controller that is able to display good playing behaviors for a commercial video game. In this study, we create neural-based game controller for screen-capture of Ms. Pac-Man using a multi-objective evolutionary algorithm (MOEA) for training or evolving the architectures and connection weights (including biases) in ANN corresponding to conflicting goals of minimizing complexity in ANN and maximizing Ms. Pac-man game score. In particular, we have chosen the commonly-used Pareto Archived Evolution Strategy (PAES) algorithm for this purpose. After the entire training process is completed, the controller is tested for generalization using the optimized networks in single network (single-net) and neural network ensemble (multi-net) environments. The multi-net model is compared to single-net model, and the results reveal that neural network ensemble is able learn to play with good strategies in a complex, dynamic and difficult game environment which is not achievable by the individual neural network.

Cooperative coevolution for pareto multiobjective optimization: An empirical study using SPEA2

Cooperative coevolution has been shown to be useful in evolutionary multiobjective optimization for bi- objective problems. However, this approach has yet to be investigated for multi-objective problems with higher dimensionality. In this paper, our objective is to conduct comprehensive empirical tests for cooperative coevolution using an evolutionary multiobjective algorithm for 3-dimensional problems. A new algorithm which integrates cooperative coevolutionary (CC) and the strength Pareto evolutionary algorithm 2 (SPEA2) is proposed to achieve this objective. The resulting algorithm is referred to as the strength Pareto evolutionary algorithm 2 with cooperative coevolution (SPEA2-CC). The performance between SPEA2- CC is compared against SPEA2 in solving tri-objectives problems in DTLZ suite of test problems. The results showed that SPEA2-CC clearly outperforms SPEA2, further proving the validity and effectiveness of augmenting evolutionary multi-objective algorithms with the cooperative coevolution approach.

Automated Evaluation for AI Controllers in Tower Defense Game Using Genetic Algorithm

This paper presents the research result of implementing evolutionary algorithms towards computational intelligence in Tower Defense game (TD game). TD game is a game where player(s) need to build tower to prevent the creeps from reaching their based. Penalty will be given if player losses any creeps during gameplays. It is a suitable test bed for planning, designing, implementing and testing either new or modified AI techniques due to the complexity and dynamicity of the game. In this research, Genetic Algorithm (GA) will be implemented to the game with two different neural networks: (1) Feed- forward (FFNN) and (2) Elman Recurrent (ERNN) used as tuner of the weights. ANN will determine the placement of the towers and the fitness score will be calculated at the end of each game. As a result, it is proven that the implementation of GA towards FFNN is better compared to GA towards ERNN.

Pareto Ensembles for Evolutionary Synthesis of Neurocontrollers in a 2D Maze-Based Video Game

In this paper, we present a study of evolving artificial neural network controllers for autonomously playing maze-based video game. A system using multi-objective evolutionary algorithm is developed, which is called as Pareto Archived Evolution Strategy Neural Network (PAESNet), with the attempt to find a set of Pareto optimal solutions by simultaneously optimizing two conflicting objectives. The experiments are designed to address two research aims investigating: (1) evolving weights (including biases) of the connections between the neurons and structure of the network through multi-objective evolutionary algorithm in order to reduce its runtime operation and complexity, (2) improving the generalization ability of the networks by using neural network ensemble model. A comparative analysis between the single network model as the baseline system and the model built based on the neural ensemble are presented. The evidence from this study suggests that Pareto multi-objective paradigm and neural network ensembles can be effective for creating and controlling the behaviors of video game characters.

Performance Scalability of a Cooperative Coevolution Multiobjective Evolutionary Algorithm

Recently, numerous Multiobjective Evolutionary Algorithms (MOEAs) have been presented to solve real life problems. However, a number of issues still remain with regards to MOEAs such as convergence to the true Pareto front as well as scalability to many objective problems rather than just bi-objective problems. The performance of these algorithms may be augmented by incorporating the coevolutionary concept. Hence, in this paper, a new algorithm for multiobjective optimization called SPEA2-CC is illustrated. SPEA2-CC combines an MOEA, Strength Pareto Evolutionary Algorithm 2 (SPEA2) with Cooperative Coevolution (CC). Scalability tests have been conducted to evaluate and compare the SPEA2- CC against the original SPEA2 for seven DTLZ test problems with a set of objectives (3 to 5 objectives). The results show clearly that the performance scalability of SPEA2-CC was significantly better compared to the original SPEA2 as the number of objectives becomes higher.

Cooperative versus competitive coevolution for Pareto multiobjective optimization

In this paper, we propose the integration between Strength Pareto Evolutionary Algorithm 2 (SPEA2) with two types of coevolution concept, Competitive Coevolution (CE) and Cooperative Coevolution (CC), to solve 3 dimensional multiobjective optimization problems. The resulting algorithms are referred to as Strength Pareto Evolutionary Algorithm 2 with Competitive Coevolution (SPEA2-CE) and Strength Pareto Evolutionary Algorithm 2 with Cooperative Coevolution (SPEA2-CC). The main objective of this paper is to compare competitive against cooperative coevolution to ascertain which coevolutionary approach is preferable for multiobjective optimization. The competitive coevolution will be implemented with K-Random Opponents strategy. The performances of SPEA2-CE and SPEA2-CC for solving triobjective problems using the DTLZ suite of test problems are presented. The results show that the cooperative approach far outperforms the competitive approach when used to augment SPEA2 for tri-objective optimization in terms of all the metrics (generational distance, spacing and coverage).

Improving the Performance of Multiobjective Evolutionary Optimization Algorithms Using Coevolutionary Learning

This chapter introduces two algorithms for multiobjective optimization. These algorithms are based on a state-of-the-art Multiobjective Evolutionary Algorithm (MOEA) called Strength Pareto Evolutionary Algorithm 2 (SPEA2). The first proposed algorithm implements a competitive coevolution technique within SPEA2. In contrast, the second algorithm introduces a cooperative coevolution technique to SPEA2. Both novel coevolutionary approaches are then compared to the original SPEA2 in seven scalable DTLZ test problems with 3 to 5 objectives. Overall, the optimization results show that the two proposed approaches are superior to the original SPEA2 with regard to the average distance of the nondominated solutions to the true Pareto front, the diversity of the obtained solutions and also the coverage level. In addition, t-tests have been conducted to validate the significance of the improvements obtained by the augmented algorithms over the original SPEA2. Finally, cooperative coevolution is found to be better than competitive coevolution in terms of enhancing the performance of the original SPEA2.