Darse Billings

The challenge of poker

Poker is an interesting test-bed for artificial intelligence research. It is a game of imperfect information, where multiple competing agents must deal with probabilistic knowledge, risk assessment, and possible deception, not unlike decisions made in the real world. Opponent modeling is another difficult problem in decision-making applications, and it is essential to achieving high performance in poker. This paper describes the design considerations and architecture of the poker program Poki. In addition to methods for hand evaluation and betting strategy, Poki uses learning techniques to construct statistical models of each opponent, and dynamically adapts to exploit observed patterns and tendencies. The result is a program capable of playing reasonably strong poker, but there remains considerable research to be done to play at world-class level. Copyright 2001 Elsevier Science B.V.

Game-Tree search with adaptation in stochastic imperfect-information games

Building a high-performance poker-playing program is a challenging project. The best program to date, PsOpti, uses game theory to solve a simplified version of the game. Although the program plays reasonably well, it is oblivious to the opponents weaknesses and biases. Modeling the opponent to exploit predictability is critical to success at poker. This paper introduces Vexbot, a program that uses a game-tree search algorithm to compute the expected value of each betting option, and does real-time opponent modeling to improve its evaluation function estimates. The result is a program that defeats PsOpti convincingly, and poses a much tougher challenge for strong human players.

Poker as Testbed for AI Research

For years, games researchers have used chess, checkers and other board games as a testbed for artificial intelligence research. The success of world-championship-caliber programs for these games has resulted in a number of interesting games being overlooked. Specifically, we show that poker can serve as an interesting testbed for machine intelligence research related to decision making problems. Poker is a game of imperfect knowledge, where multiple competing agents must deal with risk management, agent modeling, unreliable information and deception, much like decision-making applications in the real world. The heuristic search and evaluation methods successfully employed in chess are not helpful here. This paper outlines the difficulty of playing strong poker, and describes our first steps towards building a world-class poker-playing program.

Search and Knowledge in Lines of Action

This paper describes the design and development of two world-class Lines of Action game-playing programs: YL, a three time Computer Olympiad gold-medal winner, and Mona, which has dominated international e-mail correspondence play. The underlying design philosophy of the two programs is very different: the former emphasizes fast and efficient search, whereas the latter focuses on a sophisticated but relatively slow evaluation of each board position. In addition to providing a technical description of each program, we explore some long-standing questions on the trade-offs between search and knowledge. These experimental results confirm the conclusions made by earlier researchers in the domain of chess, thus showing that the trends are not game-specific. In particular, we see diminishing returns with additional search depth, and observe that the knowledge level of a program has a significant impact on the results of such experiments.

Abstracting Imperfect Information Game Trees

Modern game programs have achieved spectacular success in many games, such as checkers, Othello, and chess. In contrast, games with hidden information, like poker and bridge, have a fundamentally different structure, and game-playing programs have not had as much success to date.