H. Jaap van den Herik

PROGRESSIVE STRATEGIES FOR MONTE-CARLO TREE SEARCH

Monte-Carlo Tree Search (MCTS) is a new best-first search guided by the results of Monte-Carlo simulations. In this article, we introduce two progressive strategies for MCTS, called progressive bias and progressive unpruning. They enable the use of relatively time-expensive heuristic knowledge without speed reduction. Progressive bias directs the search according to heuristic knowledge. Progressive unpruning first reduces the branching factor, and then increases it gradually again. Experiments assess that the two progressive strategies significantly improve the level of our Go program Mango. Moreover, we see that the combination of both strategies performs even better on larger board sizes.

Games solved: now and in the future

In this article we present an overview on the state of the art in games solved in the domain of two-person zero-sum games with perfect information. The results are summarized and some predictions for the near future are given. The aim of the article is to determine which game characteristics are predominant when the solution of a game is the main target. First, it is concluded that decision complexity is more important than state-space complexity as a determining factor. Second, we conclude that there is a trade-off between knowledge-based methods and brute-force methods. It is shown that knowledge-based methods are more appropriate for solving games with a low decision complexity, while brute-force methods are more appropriate for solving games with a low state-space complexity. Third, we found that there is a clear correlation between the first-players initiative and the necessary effort to solve a game. In particular, threat-space-based search methods are sometimes able to exploit the initiative to prove a win. Finally, the most important results of the research involved, the development of new intelligent search methods, are described.

Proof-number search

Abstract Proof-number search (pn-search) is designed for finding the game-theoretical value in game trees. It is based on ideas derived from conspiracy-number search and its variants, such as applied cn-search and αβ-cn search. While in cn-search the purpose is to continue searching until it is unlikely that the minimax value of the root will change, pn-search aims at proving the true value of the root. Therefore, pn-search does not consider interim minimax values. Pn-search selects the next node to be expanded using two criteria: the potential range of subtree values and the number of nodes which must conspire to prove or disprove that range of potential values. These two criteria enable pn-search to treat efficiently game trees with a non-uniform branching factor. It is shown that in non-uniform trees pn-search outperforms other types of search, such as α-β iterative-deepening search, even when enhanced with transposition tables, move ordering for the full principal variation, etc. Pn-search has been used to establish the game-theoretical values of Connect-Four, Qubic, and Go-Moku. There pn-search was able to find a forced win for the player to move first. The experiments described here are in the domain of Awari, a game which has not yet been solved. The experiments are repeatable for other games with a non-uniform branching factor. This article describes the underlying principles of pn-search, presents an appropriate implementation, and provides an analysis of its strengths and weaknesses.

Parallel Monte-Carlo Tree Search

Monte-Carlo Tree Search (MCTS) is a new best-first search method that started a revolution in the field of Computer Go. Parallelizing MCTS is an important way to increase the strength of any Go program. In this article, we discuss three parallelization methods for MCTS: leaf parallelization, root parallelization, and tree parallelization. To be effective tree parallelization requires two techniques: adequately handling of (1) local mutexesand (2) virtual loss. Experiments in 13×13 Go reveal that in the program Mango root parallelization may lead to the best results for a specific time setting and specific program parameters. However, as soon as the selection mechanism is able to handle more adequately the balance of exploitation and exploration, tree parallelization should have attention too and could become a second choice for parallelizing MCTS. Preliminary experiments on the smaller 9×9 board provide promising prospects for tree parallelization.

Single-Player Monte-Carlo Tree Search

Classical methods such as A* and IDA* are a popular and successful choice for one-player games. However, they fail without an accurate admissible evaluation function. In this paper we investigate whether Monte-Carlo Tree Search (MCTS) is an interesting alternative for one-player games where A* and IDA* methods do not perform well. Therefore, we propose a new MCTS variant, called Single-Player Monte-Carlo Tree Search (SP-MCTS). The selection and backpropagation strategy in SP-MCTS are different from standard MCTS. Moreover, SP-MCTS makes use of a straightforward Meta-Search extension. We tested the method on the puzzle SameGame. It turned out that our SP-MCTS program gained the highest score so far on the standardized test set.

Games, computers and artificial intelligence

In 1950, Claude Shannon published his seminal paper on computer chess [23]. It was the dawn of the computer age; ENIAC was only a few years old, and visionary people like Shannon and Alan Turing could see the tremendous potential for the technology. Many computers in that era were used for military applications, typically ballistic calculations for missiles. In contrast, games seemed to be a natural application for computers, and one that an average person could relate to. A desire to use computers for applications that would attract public attention motivated Arthur Samuel to begin his 25-year quest to build a strong checkers-playing program [18,20]. The first working checkers program appeared in 1952 [27], and chess programs followed shortly thereafter [10]. The early efforts of Shannon, Samuel, Turing, Allan Newell, Herbert Simon, and others generated considerable interest in researching computer performance at games. Developing game-playing programs became a major research area in the fledgling field of artificial intelligence (AI). Indeed, building a world-championship-caliber chess program was one of the original “grand challenge” applications for AI. At the time, few realized the difficulty of creating programs that exhibited human-level “intelligence”, and the early days of AI were plagued by over-optimistic predictions [25]. In the 1970s and 1980s, computer-games research concentrated on chess and the socalled brute-force approach. The success of the Northwestern University chess program (the CHESS series of programs) [26] showed there was a strong correlation between search speed and chess-program performance. This was later quantified by Ken Thompson [29]. The consequence was a prolonged period of games-development activity largely devoted to building faster search engines, at the expense of doing mainstream research. The importance of developing a high-performance chess program seemed to fade.

An Integrated View on Rules and Principles

In the law, it is generally acknowledged that there are intuitive differences between reasoning with rules and reasoning with principles. For instance, a rule seems to lead directly to its conclusion if its condition is satisfied, while a principle seems to lead merely to a reason for its conclusion. However, the implications of these intuitive differences for the logical status of rules and principles remain controversial.A radical opinion has been put forward by Dworkin (1978). The intuitive differences led him to argue for a strict logical distinction between rules and principles. Ever since, there has been a controversy whether the intuitive differences between rules and principles require a strict logical distinction between the two. For instance, Soeteman (1991) disagrees with Dworkins opinion, and argues that rules and principles cannot be strictly distinguished, and do not have a different logical structure.In this paper, we claim that the differences between rules and principles are merely a matter of degree. We give an integrated view on rules and principles in which rules and principles have the same logical structure, but different behavior in reasoning. In this view, both rules and principles are considered to consist of a condition and a conclusion. The observed differences between rules and principles are, in our view, the result of different types of relations that they have with other rules and principles. In the integrated view, typical rules and typical principles are the extremes of a spectrum.We support our claim by giving an explicit formalization of our integrated view using the recently developed formal tools provided by Reason-Based Logic. As an application of our view on rules and principles, we give three ways of reconstructing reasoning by analogy.

SCAN: a scalable model of attentional selection

This paper describes the SCAN (Signal Channelling Attentional Network) model, a scalable neural network model for attentional scanning. The building block of SCAN is a gating lattice, a sparsely-connected neural network defined as a special case of the Ising lattice from statistical mechanics. The process of spatial selection through covert attention is interpreted as a biological solution to the problem of translation-invariant pattern processing. In SCAN, a sequence of pattern translations combines active selection with translation-invariant processing. Selected patterns are channelled through a gating network, formed by a hierarchical fractal structure of gating lattices, and mapped onto an output window. We show how the incorporation of an expectation-generating classifier network (e.g. Carpenter and Grossbergs ART network) into SCAN allows attentional selection to be driven by expectation. Simulation studies show the SCAN model to be capable of attending and identifying object patterns that are part of a realistically sized natural image. Copyright 1997 Elsevier Science Ltd.

Games as personality profiling tools

In this paper we investigate whether a personality profile can be determined by observing a players behavior in a game. Five personality traits are used to define a personality profile. They are adopted from the Five Factor Model of personality. The five traits are measured by the NEO-PI-R questionnaire. For our purpose, we developed a game module for the game Neverwinter Nights. The module automatically stores a players behavioral data. Experimental trials were run measuring the behavior of 44 participants. The experiment produced game behavior scores for 275 game variables per player. Correlation analysis shows relationships between all five personality traits and the video game data. From these results, we may conclude that a video game can be used to create an adequate personality profile of a player.

Discovering the Visual Signature of Painters

Recent developments in image classification have focused on efficient preprocessing of visual data to improve the performances of neural networks and other learning algorithms when dealing with content-based classification tasks. Given the high dimensionality and redundancy of visual data, the primary goal of preprocessing is to transfer the original data to a low-dimensional representation that preserves the information relevant for the classification. This contribution reviews modern preprocessing (dimension-reduction) techniques and discusses their advantages and disadvantages. The performance of the techniques is assessed on a difficult painting-classification task that requires painter-specific features to be retained in the low-dimensional representation. Evaluation of the results shows that domain-specific knowledge provides a rough albeit indispensable guideline for determining the appropriate type of preprocessing. Furthermore, the evaluation shows that neural-network techniques are most suitable for executing and fine-tuning the preprocessing and subsequent classification. It is argued that further improvements can be gained by the use of a content-based attentional selection procedure. Our conclusion is that preprocessing should be tailored to the task at hand by combining domain knowledge with neural-network techniques, and that within fifty years the visual signature of painters is as recognizable as is any handwritten signature.