Hung-Jui Chang

Convergence and correctness analysis of Monte-Carlo tree search algorithms: A case study of 2 by 4 Chinese dark chess

The convergence and correctness of the UCT algorithm is a hot research problem. Previous research has shown the convergence of UCT-based algorithm on simultaneous turns or random turns games, but little is known for traditional alternating turns games. In this paper, we analyze the performance of the UCT algorithm in a 2-player imperfect information alternating turns game, 2 × 4 Chinese dark chess (CDC). The performance of the UCT algorithm is measured by the correctness rate and convergence speed. The correctness is defined by the percentage that the UCT algorithm outputs the same move with the one having the best game theoretic value. The convergence speed is defined by the entropy of a set of moves, which are output by the UCT algorithm using the same number of iterations. Experimental result shows the convergence of the UCT algorithm in the CDC, which can also be applied to explain the existence of diminishing returns in the UCT algorithm. Experimental result also shows an UCT algorithm does not always output moves with the best game theoretic value, but these with the highest chance of winning.

Game Tree Search with Adaptive Resolution

In this paper, we use an adaptive resolution R to enhance the min-max search with alpha-beta pruning technique, and show that the value returned by the modified algorithm, called Negascout-with-resolution, differs from that of the original version by at most R. Guidelines are given to explain how the resolution should be chosen to obtain the best possible outcome. Our experimental results demonstrate that Negascout-with-resolution yields a significant performance improvement over the original algorithm on the domains of random trees and real game trees in Chinese chess.

Advanced Meta-knowledge for Chinese Chess Endgame Knowledge Bases

The proper evaluation of a possible transition from the middle game to the endgame is an important research issue. We constructed an endgame knowledge base that consists of a large set of endgame heuristics that supports the evaluation. Moreover, a general graph model is proposed to resolve conflicts between two competing material combinations. However, it turned out to be difficult to find such competing material combinations. We need better meta-knowledge rules to find more potential conflicts. In this article, we propose five meta-knowledge rules for Chinese chess. Two examples of meta-knowledge rules are piece exchanges and pawn exchanges. The meta-knowledge rules are inducted from real games played by masters. The heuristics so found are endowed with confidence factors to show their chances of being correct. Using this method, 20% of a previous constructed body of endgame knowledge, consisting of 124, 747 material combinations, was found to be erroneous. About 82.13% of these heuristic errors are auto-corrected using our algorithm. By using the corrected knowledge base, the strength of our game-playing program, CONTEMPLATION, is clearly improved according to self-play tests.

Multilevel Inference in Chinese Chess Endgame Knowledge Bases

In Chinese chess, retrograde analysis can be used to solve complex elementary (i.e., fundamental) endgames and to provide perfect play. However, there are still many practical endgames pending to be analysed due to problems related to the complex playing rules. Of course, there is heuristic endgame knowledge for the evaluation functions. This knowledge is often applied to the complex endgames or the real endgames to improve the playing strength. One crucial problem is to choose relatively advantageous endgames by selecting appropriate piece exchanges. For this problem, we designed a Chinese chess endgame knowledge-based system with a large set of endgame heuristics, called an endgame knowledge base. We use this knowledge base in our program, CONTEMPLATION. To maintain the quality of the constructed knowledge base, it is important to detect and resolve conflicts between the heuristics. A conflict-resolution method enables Chinese chess experts to correct erroneous entries by using knowledge about two endgames that differ by precisely one piece. The problem involves detecting potential errors so that a human expert can easily revise and improve the reliability of the knowledge base. In this article, we introduce two major enhancements to the above method. First, we propose a general graph model to handle the heuristics when the endgames involved are differing in more than one piece. Second, we add a confidence-factor parameter to encode the probability that a heuristic may be true. Such heuristics are often used in real games when pieces are exchanged. The resulting graph model is effective in maintaining the consistency of predefined meta-knowledge, and thus improves the overall quality significantly. The results of the experiments on self-play tests demonstrate that the derived knowledge base improves the playing strength of CONTEMPLATION.

The Impact of Household Structures on Pandemic Influenza Vaccination Priority

The household structure is an important aspect of population based simulation. How to generate a mock population with specific household structure characteristics is thus an important question. The network structure is one of the dominant factors for contact-based disease transmission. And household structure is the most important source of close contact among small groups. We identify the percentage of elderly-children households as an important character and study the process to generate mock population with specified percentage of elderly-children households. The generated mock populations are fed into the agent-based simulation module to study the impact of household structure on vaccination policy.

Learning from Simulated World - Surrogates Construction with Deep Neural Network.

The deep learning approach has been applied to many domains with success. We use deep learning to construct the surrogate function to speed up simulation based optimization in epidemiology. The simulator is an agentbased stochastic model for influenza and the optimization problem is to find vaccination strategy to minimize the number of infected cases. The optimizer is a genetic algorithm and the fitness function is the simulation program. The simulation is the bottleneck of the optimization process. An attempt to use the surrogate function with table lookup and interpolation was reported before. The preliminary results show that the surrogate constructed by deep learning approach outperforms the interpolation based one, as long as similar cases of the testing set have been available in the training set. The average of the absolute value of relative error is less than 0.7 percent, which is quite close to the intrinsic limitation of the stochastic variation of the simulation software 0.2 percent, and the rank coefficients are all above 0.99 for cases we studied. The vaccination strategy recommended is still to vaccine the school age children first which is consistent with the previous studies. The preliminary results are encouraging and it should be a worthy effort to use machine learning approach to explore the vast parameter space of simulation models in epidemiology.

Validating and Fine-Tuning of Game Evaluation Functions Using Endgame Databases

The evaluation function and search algorithm are the two main components of almost all game playing programs. A good evaluation function is carefully designed to assess a position by considering the location and the material value of all pieces on board. Normally, an evaluation function f is manually designed, which requires a large amount of expert knowledge. Usually, f must be able to evaluate any position. Theoretically, a huge table that stores all the pre-computed scores for every position can perfectly represent any position. However, it is space-efficient to encode f, which is far from being perfect. On the other hand, endgame databases provide game theoretical values for all legal positions when the total number of pieces remains is small, say within 5 or 6 for Chinese dark chess (CDC). However, only a selected number of endgame databases are available. Furthermore, the size of an endgame database is huge, e.g., from megabytes to gigabytes. We construct a scheme to use the information from endgame databases to validate and fine-tune a manually designed evaluation function. Our method abstracts critical information from a huge database and then validates f on positions when they are contained in an endgame database. Using this information, we then discover meta knowledge to fine-tune and revise f so that f better evaluates a position even when f is fed with positions containing many pieces. Experimental results show that our approach is successful.

A comparison between a deterministic, compartmental model and an individual based-stochastic model for simulating the transmission dynamics of pandemic influenza

Simulation models are often used in the research area of epidemiology to understand characteristics of disease outbreaks. As a result, they are used by authorities to better design intervention methods and to better plan the allocation of medical resources. Previous work make use of many different types of simulation models with an agent-based model, e.g., Taiwan simulation system, and an equation-based model, e.g., AsiaFluCap simulation system, being the two most popular ones. Some comparison studies has been attempted in the past to understand the limits, efficiency, and usability of some model. However, there was little studies to justify why one model is used instead of the other. In this paper, instead of studying the two most popular models one by one, we try to do a comparative study between these two most popular ones. By observing that one model can outperform the other in some cases, and vice versa, we hence study conditions that which one should be used. Furthermore, previous studies show little results in the issue of allocating medical resources. Our paper studies and compares the two models using medical resources allocation as one of our primary concerns. As a conclusion, we come out with a general guideline to help model designers to pick one that fits the given needs better.