Ida G. Sprinkhuizen-Kuyper

Adaptive game AI with dynamic scripting

Online learning in commercial computer games allows computer-controlled opponents to adapt to the way the game is being played. As such it provides a mechanism to deal with weaknesses in the game AI, and to respond to changes in human player tactics. We argue that online learning of game AI should meet four computational and four functional requirements. The computational requirements are speed, effectiveness, robustness and efficiency. The functional requirements are clarity, variety, consistency and scalability. This paper investigates a novel online learning technique for game AI called ‘dynamic scripting’, that uses an adaptive rulebase for the generation of game AI on the fly. The performance of dynamic scripting is evaluated in experiments in which adaptive agents are pitted against a collection of manually-designed tactics in a simulated computer roleplaying game. Experimental results indicate that dynamic scripting succeeds in endowing computer-controlled opponents with adaptive performance. To further improve the dynamic-scripting technique, an enhancement is investigated that allows scaling of the difficulty level of the game AI to the human player’s skill level. With the enhancement, dynamic scripting meets all computational and functional requirements. The applicability of dynamic scripting in state-of-the-art commercial games is demonstrated by implementing the technique in the game Neverwinter Nights. We conclude that dynamic scripting can be successfully applied to the online adaptation of game AI in commercial computer games.

Adaptive information filtering using evolutionary computation

Information Filtering is concerned with filtering data streams in such a way as to leave only pertinent data (information) to be perused. When the data streams are produced in a changing environment the filtering has to adapt too in order to remain eAective. Adaptive Information Filtering (AIF) is concerned with filtering in changing environments. The changes may occur both on the transmission side (the nature of the streams can change), and on the reception side (the interest of a user can change). Weighted trigram analysis is a quick and flexible technique for describing the contents of a document. A novel application of evolutionary computation is its use in Adaptive Information Filtering for optimizing various parameters, notably the weights associated with trigrams. The research described in this paper combines weighted trigram analysis, clustering, and a special two-pool evolutionary algorithm, to create an Adaptive Information Filtering system with such useful properties as domain independence, spelling error insensitivity, adaptability, and optimal use of user feedback while minimizing the amount of user feedback required to function properly. We designed a special evolutionary algorithm with a two-pool strategy for this changing environment. ” 2000 Elsevier Science Inc. All rights reserved.

Competing crossovers in an adaptive GA framework

Reports the results of experiments on multi-parent reproduction in an adaptive genetic algorithm (GA) framework. An adaptive mechanism based on competing subpopulations is incorporated into the algorithm in order to detect the best crossovers. Experiments on a number of test functions designed for studying crossover performance show that multi-parent reproduction is superior to traditional two-parent crossover, but the adaptive mechanism is not able to reward better crossovers according to their performance. Nevertheless, the adaptive algorithm exhibits a performance that is comparable to the non-adaptive variant using the best crossover alone. This implies that it is sound and safe to use an adaptive GA with competing subpopulations/crossovers, instead of performing time-consuming comparisons in searching for the best operators.

The error surface of the simplest xor network has only global minima

The artificial neural network with one hidden unit and the input units connected to the output unit is considered. It is proven that the error surface of this network for the patterns of the XOR problem has minimum values with zero error and that all other stationary points of the error surface are saddlepoints. Also, the volume of the regions in weight space with saddlepoints is zero, hence training this network on the four patterns of the XOR problem using, e.g., backpropagation with momentum, the correct solution with error zero will be reached in the limit with probability one.

Reactive agents and perceptual ambiguity

Reactive agents are generally believed to be incapable of coping with perceptual ambiguity (i.e., identical sensory states that require different responses). However, a recent finding suggests that reactive agents can cope with perceptual ambiguity in a simple model (Nolfi, 2002). This paper investigates to what extent reactive and nonreactive agents can cope with perceptual ambiguity, and which strategies are employed when doing so. A model of active categorical perception (called Acp) is introduced. In Acp, situated agents with different types of neurocontrollers are optimized to categorize objects by adaptively coordinating action and perception. Our experiments show that both nonreactive and reactive agents can cope with perceptual ambiguity. An analysis of the behavior reveals that nonreactive agents use their internal memory to cope with perceptual ambiguity, while reactive agents use the environment as an external memory to compensate for their lack of an internal memory. We conclude that reactive agents can cope with perceptual ambiguity in the context of active categorical perception, and that they can organize their behavior according to stimuli that are no longer present, especially when they incorporate a nonlinear sensorimotor mapping. Moreover, we may conclude that sensory state-transition diagrams provide insight into the strategies employed by reactive agents to deal with perceptual ambiguity, and their use of the environment as an external memory.

The error surface of the 2-2-1 XOR network: the finite stationary points

We investigate the error surface of the XOR problem for a 2-2-1 network with sigmoid transfer functions. It is proved that all stationary points with finite weights are saddle points with positive error or absolute minima with error zero. So, for finite weights no local minima occur. The proof results from a careful analysis of the Taylor series expansion around the stationary points. For some points coefficients of third or even fourth order in the Taylor series expansion are used to complete the proof. The proofs give a deeper insight into the complexity of the error surface in the neighbourhood of saddle points. These results can guide the research in finding learning algorithms that can handle these kinds of saddle points.

The ROC isometrics approach to construct reliable classifiers

We address the problem of applying machine-learning classifiers in domains where incorrect classifications have severe consequences. In these domains we propose to apply classifiers only when their performance can be defined by the domain expert prior to classification. The classifiers so obtained are called reliable classifiers. In the article we present three main contributions. First, we establish the effect on an ROC curve when ambiguous instances are left unclassified. Second, we propose the ROC isometrics approach to tune and transform a classifier in such a way that it becomes reliable. Third, we provide an empirical evaluation of the approach. From our analysis and experimental evaluation we may conclude that the ROC isometrics approach is an effective and efficient approach to construct reliable classifiers. In addition, a discussion about related work clearly shows the benefits of the approach when compared with existing approaches that also have the option to leave ambiguous instances unclassified.

Robust and Scalable Coordination of Potential-Field Driven Agents

In this paper, we introduce a nature-inspired multi- agent system for the task domain of resource distribution in large storage facilities. The system is based on potential fields and swarm intelligence, in which straightforward path planning is integrated. We show both experimentally and theoretically that the system is adaptive, robust and scalable. Moreover, we show that the planning component helps to overcome common pitfalls for nature-inspired systems in the task assignment domain. We end this paper with a discussion of an additional requirement for multi-agent systems interacting with humans: functionality. More precisely, we argue that such systems must behave in a fair way to be functional. We illustrate how fairness can be measured and illustrate that our system behaves in a moderately fair manner.

A local minimum for the 2-3-1 XOR network

It was assumed proven that two-layer feedforward neural networks with t-1 hidden nodes, when presented with t input patterns, can not have any suboptimal local minima on the error surface. In this paper, however, we shall give a counterexample to this assumption. This counterexample consists of a region of local minima with nonzero error on the error surface of a neural network with three hidden nodes when presented with four patterns (the XOR problem). We will also show that the original proof is valid only when an unusual definition of local minimum is used.

The local minima of the error surface of the 2-2-1 XOR network

All local minima of the error surface of the 2-2-1 XOR network are described. A local minimum is defined as a point such that all points in a neighbourhood have an error value greater than or equal to the error value in that point. It is proved that the error surface of the two-layer XOR network with two hidden units has a number of regions with local minima. These regions of local minima occur for combinations of the weights from the inputs to the hidden nodes such that one or both hidden nodes are saturated for at least two patterns. However, boundary points of these regions of local minima are saddle points. It will be concluded that from each finite point in weight space a strictly decreasing path exists to a point with error zero. This also explains why experiments using higher numerical precision find less “local minima”.