Matej Guid

Using Heuristic-Search Based Engines for Estimating Human Skill at Chess

Establishing heuristic-search based chess programs as appropriate tools for estimating human skill levels at chess may seem impossible due to the following issues: the programs’ evaluations and decisions tend to change with the depth of search and with the program used. In this research, we provide an analysis of the differences between heuristic-search based programs in estimating chess skill. We used four different chess programs to perform analyses of large data sets of recorded human decisions, and obtained very similar rankings of skill-based performances of selected chess players using any of these programs at various levels of search. A conclusion is that, given two chess players, all the programs unanimously rank one player to be clearly stronger than the other, or all the programs assess their strengths to be similar. We also repeated our earlier analysis with the program CRAFTY of World Chess Champions with currently one of the strongest chess programs, RYBKA 32, and obtained qualitatively very similar results as with CRAFTY. This speaks in favour of computer heuristic search being adequate for estimating skill levels of chess players, despite the above stated issues.

Evaluating the Aesthetics of Endgame Studies: A Computational Model of Human Aesthetic Perception

In this paper, we explain how an existing computational aesthetics model for three-move mate problems was improved and adapted to suit the domain of chess endgame studies. Studies are typically longer and more “sophisticated” in terms of their perceived aesthetics or beauty. They are therefore likely a better test of the capability of machines to evaluate beauty in the game. Based on current validation methods for an aesthetics model such as this, the experimental results confirm that the adaptation was successful. In the first experiment, the new model enabled a computer program to distinguish correctly between composed studies and positions with sequences resembling studies taken from real games. In the second, the computational aesthetic evaluations were shown to correlate positively and well with human expert aesthetic assessment. The new model encompasses the previous three-mover one and can be used to evaluate beauty as perceived by humans in both domains. This technology pushes the boundaries of computational chess and can be of benefit to human players, composers, and judges. To some extent, it may also contribute to our understanding of the psychology of human aesthetic perception and the “mechanics” of human creativity in composing problems and studies.

Deriving concepts and strategies from chess tablebases

Complete tablebases, indicating best moves for every position, exist for chess endgames. There is no doubt that tablebases contain a wealth of knowledge, however, mining for this knowledge, manually or automatically, proved as extremely difficult. Recently, we developed an approach that combines specialized minimax search with the argument-based machine learning (ABML) paradigm. In this paper, we put this approach to test in an attempt to elicit human-understandable knowledge from tablebases. Specifically, we semi-automatically synthesize knowledge from the KBNK tablebase for teaching the difficult king, bishop, and knight versus the lone king endgame.

Goal-Oriented conceptualization of procedural knowledge

Conceptualizing procedural knowledge is one of the most challenging tasks of building systems for intelligent tutoring. We present an algorithm that enables teachers to accomplish this task semi automatically. We used the algorithm on a difficult king, bishop, and knight versus the lone king (KBNK) chess endgame, and obtained concepts that could serve as textbook instructions. A pilot experiment with students and a separate evaluation of the instructions by experienced chess trainers were deemed very positive.

Learning Positional Features for Annotating Chess Games: A Case Study

By developing an intelligent computer system that will provide commentary of chess moves in a comprehensible, user-friendly, and instructive way, we are trying to use the power demonstrated by the current chess engines for tutoring chess and for annotating chess games. In this paper, we point out certain differences between the computer programs which are specialized for playing chess and our program which is aimed at providing quality commentary. Through a case study, we present an application of argument-based machine learning, which combines the techniques of machine learning and expert knowledge, to the construction of more complex positional features, in order to provide our annotating system with an ability to comment on various positional intricacies of positions in the game of chess.

Search-Based Estimation of Problem Difficulty for Humans

The research question addressed in this paper is: Given a problem, can we automatically predict how difficult the problem will be to solve by humans? We focus our investigation on problems in which the difficulty arises from the combinatorial complexity of problems. We propose a measure of difficulty that is based on modeling the problem solving effort as search among alternatives and the relations among alternative solutions. In experiments in the chess domain, using data obtained from very strong human players, this measure was shown at a high level of statistical significance to be adequate as a genuine measure of difficulty for humans.

ABML knowledge refinement loop: a case study

Argument Based Machine Learning (ABML) was recently demonstrated to offer significant benefits for knowledge elicitation. In knowledge acquisition, ABML is used by a domain expert in the so-called ABML knowledge refinement loop. This draws the experts attention to the most critical parts of the current knowledge base, and helps the expert to argue about critical concrete cases in terms of the experts own understanding of such cases. Knowledge elicited through ABML refinement loop is therefore more consistent with experts knowledge and thus leads to more comprehensible models in comparison with other ways of knowledge acquisition with machine learning from examples. Whereas the ABML learning method has been described elsewhere, in this paper we concentrate on detailed mechanisms of the ABML knowledge refinement loop. We illustrate these mechanisms with examples from a case study in the acquisition of neurological knowledge, and provide quantitative results that demonstrate how the model evolving through the ABML loop becomes increasingly more consistent with the experts knowledge during the process.

Development of a Program for Playing Progressive Chess

We present the design of a computer program for playing Progressive Chess. In this game, players play progressively longer series of moves rather than just making one move per turn. Our program follows the generally recommended strategy for this game, which consists of three phases: looking for possibilities to checkmate the opponent, playing generally good moves when no checkmate can be found, and preventing checkmates from the opponent. In this paper, we focus on efficiently searching for checkmates, putting to test various heuristics for guiding the search. We also present the findings of self-play experiments between different versions of the program.

Influence of Search Depth on Position Evaluation

By using a well-known chess program and a large data set of chess positions from real games we demonstrate empirically that with increasing search depth backed-up evaluations of won positions tend to increase, while backed-up evaluations of lost positions tend to decrease. We show three implications of this phenomenon in practice and in the theory of computer game playing. First, we show that heuristic evaluations obtained by searching to different search depths are not directly comparable. Second, we show that fewer decision changes with deeper search are a direct consequence of this property of heuristic evaluation functions. Third, we demonstrate that knowing this property may be used to develop a method for detecting fortresses in chess, which is an unsolved task in computer chess.

A program for Progressive chess

In Progressive chess, rather than just making one move per turn, players play progressively longer series of moves. Combinatorial complexity generated by many sequential moves represents a difficult challenge for classic search algorithms. In this article, we present the design of a state-of-the-art program for Progressive chess. The program follows the generally recommended strategy for this game, which consists of three phases: looking for possibilities to checkmate the opponent, playing sequences of generally good moves when checkmate is not available, and preventing checkmates from the opponent. For efficient and effective checkmate search we considered two versions of the A* algorithm, and developed five different heuristics for guiding the search. For finding promising sequences of moves we developed another set of heuristics, and combined the A* algorithm with minimax search, in order to fight the combinatorial complexity. We constructed an opening book, and designed specialized heuristics for playing Progressive chess endgames. An application with a graphical user interface was implemented in order to enable human players to play Progressive chess against the computer, and to use the computer to analyze their games. The program performed excellently in experiments with checkmate search, and won both mini-matches against a human chess master. We also present the findings of self-play experiments between different versions of the program.