Analysis of Evolutionary Program Synthesis for Card Games
AAnalysis of Evolutionary Program Synthesis for Card Games
Rohan Saha ∗ , Cassidy Pirlot ∗ { rsaha , pirlot } @ualberta.caJanuary 12, 2021 Abstract
In this report, we inspect the application of an evolutionary approach to the game of Rack’o,which is a card game revolving around the notion of decision making. We first apply the evolu-tionary technique for obtaining a set of rules over many generations and then compare them witha script written by a human player. A high level domain specific language is used that deter-mines which the sets of rules are synthesized. We report the results by providing a comprehensiveanalysis of the set of rules and their implications.
A genetic algorithm is a search heuristic that aims to find optimal solutions through ideas found inbiology. This includes concepts such as survival of the fittest, mutations, cross-breeding, and more, inother words, an evolutionary approach. We previously saw in assignment 1, the performance of suchan algorithm in a smaller version of the game CAN’T STOP, here we aim to evaluate the performanceof it in the game RACK’O.Evolutionary approach is a method where the goal is to find a solution to a problem iterativelygiven a initial value. In the context of program synthesis, evolutionary approaches are used togenerate a set of rules that is consistent with the game mechanics and this set of rules is expected toperform better than other scripts in the same search space. The set of rules that is obtained using afitness function that measures how good the set of rules are given a state of the game. We chose toinvestigate the area of evolutionary approach in program synthesis because it is an interesting methodto generate strategies and research using evolutionary approach spans over a multitude of programsynthesis problems such as program sketching[1] and guided search for synthesizing programs withhigh complexity[2].As a overview, we first explain the game and its mechanics in section 2, the structure of theproject in 3, the domain specific language in 4, the design of the experiment in section 5, the results * - Both authors have equal contributions. Ordering was decided over a coin toss on a Google Meet. a r X i v : . [ c s . A I] J a n nd findings in section 6.1, and finally conclude in section 7. Rack’O is a card game where the objective for a player is to arrange the cards in your hand inascending order. The number of players can range from two to four. The game has two decks ofcards, a discard pile and a Rack’o deck. Initially, all the cards are shuffled randomly to create theRack’o deck. Then each player is dealt ten cards from the Rack’o deck, which is known as the player’shand. The discard pile is started by turning over the top card from the Rack’o deck. Each player,during their turn, picks up a card from the Rack’o deck, and decides to either replace a card fromtheir hand or does not. The player can also pick up the card from the discard pile and decide towhether replace a card in hand or not. This means that it may be possible that the player decides topass.In the project, a smaller version of the game is used for simplicity. Forty total cards are usedinstead of sixty and each player is dealt five cards instead of ten. Also, the game is designed for twoplayers instead of a maximum of four. These changes are considered due to computational restrictions,but the project can easily be expanded to the full version of the game. • DSL.py-
This file contains our domain specific language as well as our context free grammarwhich uses if’s , and’s , and the names of the functions from our DSL. • Evaluation.py-
This file has all of the functions necessary to preform the genetic algorithm.This includes: – Evaluation
This function plays takes in two instantiated scripts and has them play Rack’oagainst each other n times and returns the amount of times each player won as well asthe relative rate of winning for each payer (the amount of times they won over n for eachplayer). – Eval
This takes in a dictionary of scripts as the keys with their corresponding fitnessscores as the values as well as an integer m. It causes each script in the dictionary to playin the Evaluation function, as both player one and two, m times and return the updateddictionary. – Elite
This takes in a dictionary scripts with their fitness scores as values and an integer k.This function compares the fitness scores of the scripts and returns a reduced dictionaryof the best k scripts according to the fitness scores. – Tournament
This function takes in the dictionary as well as an integer t. It randomlytakes t scripts from the dictionary and returns the best two scripts according to the fitness2core. – GenerateSplit
Takes in two ’parent’ scripts and returns a random crossover strategy ina string. – crossover
This function creates and returns the script that the GenerateSplit functionreturns. – mutate
Takes in a script mutates, instantiates, and returns it. – RemovedUnused
This function takes in a single script and looks at if a certain rule thatthe script contains has ever been used. If it has not then it removes this rule. The adjustedscript is returned. – EZS
This is our genetic algorithm which takes in a generation number, population size,the number of desired elites, and the size of the tournament. This returns our generatedscript. • Script.py-
Contains the outline in which our scripts get generated as. • OurScript.py-
This is the script we made ourselves (more on this later). • Game.py-
This contains functions for how the game is played including shuffling, dealing,seeing available moves that can be made, etc. These functions are used in the Evaluationfunction in Evaluation.py.
The domain specific language(DSL) represents the set of template functions or methods that definethe rules a player can play during the game. The following rules were included in the DSL whichwere used to generate a script.The parameters used are as follows: • action : the resulting hand after a play is made on the players hand. • index : represents the index in a players hand that we are currently concerned with. • hand : the current hand of a player • number : possible number from the Rack’o deck (0-39)1. isBigger(action, index, hand) : Returns true if the card picked up from either the Rack’odeck or discard pile, when swapped with a card at the index in the player’s hand, is bigger thanthe card below it, else returns false. 3. isSmaller(action, index, hand) : Returns true if the card picked up from either the Rack’odeck or discard pile, when swapped with a card at the index in the player’s hand, is smallerthan the card below it, else returns false.3. givesRacko(action) : Returns true if the player achieves Rack’o by taking the action.4. hasRacko(hand) : Returns true if the player has all cards arranged in ascending order.5. isCardBetweenNumbers(action, number, number, index, hand) : Returns true if thecard picked from either the Rack’o deck and discard pile, when placed in the index, is in betweenthe two specified numbers.Using the rules from the DSL, at each generation, a set of random scripts are created initially andthen the best performing scripts are passed on to the next generation using various genetic operationssuch as crossover and mutation. We ran the algorithm multiple times with differing EZS parameters. Each time 100 games were playedto compare two scripts and the scripts were compared 3 times as player 1 each. The different casesare as follows:1. Population: 10, Generations: 4, Elites: 7, Tournaments: 52. Population: 20, Generations: 6, Elites: 7, Tournaments: 73. Population: 30, Generations: 8, Elites: 10, Tournaments: 10
To test how well the genetic algorithm performed we consider testing it against a script that mimicshow we would play the game given the DSL. We play by placing the drawn card in a chosen slot if itfits in the corresponding interval for that slot. The (exhaustive) intervals are all of equal length andcontain larger values for the higher index slots. The code is found below: from Player import Player import random from Game import Game from DSL import DSL class OurScript ( Player ): def __init__ ( self ): self . _id = 1234567 self . _strategies = [ ’ DSL . givesRacko (a) ’, ’ DSL . isCardBetweenNumbers (a ,1 ,8 ,0 , Game . getRack () ) ’, ’ DSL . isCardBetweenNumbers (a ,9 ,16 ,1 , Game . getRack () ) ’, ’ DSL . isCardBetweenNumbers (a ,17 ,25 ,2 , Game . getRack () ) ’, ’ DSL . isCardBetweenNumbers (a ,26 ,34 ,3 , Game . getRack () ) ’, ’ DSL . isCardBetweenNumbers (a ,35 ,40 ,4 , Game . getRack () ) ’] self . _counter_calls = [] for i in range (5) : self . _counter_calls . append (0) def get_counter_calls ( self ): return self . _counter_calls def get_action ( self , Game , card , hand ): actions = Game . available_moves ( card , hand ) for a in actions : if DSL . givesRacko (a): self . _counter_calls [0] += 1 return a if DSL . isCardBetweenNumbers ( a , 1 , 8 , 0 , Game . getRack () ): self . _counter_calls [0] += 1 return a if DSL . isCardBetweenNumbers ( a , 9 , 16 , 1, Game . getRack () ): self . _counter_calls [1] += 1 return a if DSL . isCardBetweenNumbers ( a , 17 , 25 , 2 , Game . getRack () ): self . _counter_calls [2] += 1 return a if DSL . isCardBetweenNumbers ( a , 26 , 34 , 3 , Game . getRack () ): self . _counter_calls [3] += 1 return a if DSL . isCardBetweenNumbers ( a , 35 , 40 , 4 , Game . getRack () ): self . _counter_calls [4] += 1 return a return actions [0]
5e tested our script against the generated script with the evaluation function found in Evalua-tion.py letting them alternate being player one in case there is some sort of advantage in being inthat position.
For case 1 the simulation ran for 10-15 minutes the resulting script had rules: • ‘DSL.givesRacko(a)’ • ‘DSL.isCardBetweenNumbers(a, 37 , 39 , 3 )’ • ‘DSL.isCardBetweenNumbers(a, 25 , 29 , 0 )’ • ‘DSL.hasRacko(Game.getRack())’ • ‘DSL.isCardBetweenNumbers(a, 34 , 20 , 4 )’ • ‘DSL.isSmaller(a, 1 , Game.getRack() )’ • ‘DSL.isSmaller(a, 2 , Game.getRack() )’Case 2 ran for just over an hour and resulted in a script with: • ‘DSL.isSmaller(a, 0 , Game.getRack() )’ • ‘DSL.isCardBetweenNumbers(a, 39 , 26 , 0 )’ • ‘DSL.isBigger(a, 3 , Game.getRack() )’ • ‘DSL.isSmaller(a, 4 , Game.getRack() )’ • ‘DSL.isCardBetweenNumbers(a, 9 , 13 , 0 )’ • ‘DSL.isSmaller(a, 3 , Game.getRack() )’ • ‘DSL.isCardBetweenNumbers(a, 37 , 18 , 3 )’ • ‘DSL.isCardBetweenNumbers(a, 33 , 8 , 4 )’ • ‘DSL.isCardBetweenNumbers(a, 3 , 31 , 1 )’ • ‘DSL.isCardBetweenNumbers(a, 15 , 37 , 3 )’ • ‘DSL.isBigger(a, 0 , Game.getRack() )’ • ‘DSL.isBigger(a, 1 , Game.getRack() )’ 6 ‘DSL.hasRacko(Game.getRack())’ • ‘DSL.isSmaller(a, 2 , Game.getRack() )’ • ‘DSL.isBigger(a, 2 Game.getRack() )’ • ‘DSL.givesRacko(a)’ • ‘DSL.isCardBetweenNumbers(a, 9 , 28 , 0 )’Case 3 ran for just over 2 hours and resulted in a script with: • ‘DSL.isBigger(a, 2 , Game.getRack() )’ • ‘DSL.givesRacko(a)’ • ‘DSL.hasRacko(Game.getRack())’ • ‘DSL.isCardBetweenNumbers(a, 21 , 12 , 3 )’ • ‘DSL.isCardBetweenNumbers(a, 26 , 33 , 3 )’ • ‘DSL.isCardBetweenNumbers(a, 4 , 34 , 4 )’ • ‘DSL.isBigger(a, 1 , Game.getRack() )’ • ‘DSL.isSmaller(a, 3 , Game.getRack() )’ • ‘DSL.isCardBetweenNumbers(a, 12 , 3 , 2 )’ We played Rack’o with the resulting scripts from each of the three cases against our script that wewrote. First of all, it must be noted that though the DSL may seem limited, the search space isactually very large because the parameters of the functions can have a wide range of values. Thismakes it difficult for the evolutionary approach to find a script that is globally optimal. It is oftennecessary to have an extremely large population with many generations to obtain such an optimalscript but due to computational restrictions, we only run the algorithm for the three cases explainedin section 5.1.Let’s now have a look the rules generated from the evolutionary approach. We can see that foreach case, the rule, givesRacko(a) is present. This indicates that it is advisable to play an action ifit helps to achieve Rack’o. This is fairly obvious and is an easy strategy to follow. The other rulethat is present in the best script for all the three cases is the isCardBetweenNumbers(...) function.This is a good strategy because the knowledge of the cards in the current hand is being used tocheck whether the newly swapped card will help to have all cards in ascending order. We found that7he other strategies present do not contribute as much to performance as this will be evident whileobserving the results given below.Figure 1: Percentage wins of the best script from the Evolutionary algorithm vs OurScript for allcasesWe see that the generated scripts did not perform very well against our script. However, we areimpressed that they were able to win any games at all. We were expecting that as we increased ourpopulation size and generations that there would be improvement against our script, which we onlysee marginally. This may be because there was one strategy that was exceptionally stronger than theothers (isBetweenNumbers) and our script was made entirely of that one function.All in all, from the results and the generated scripts, the functions givesRacko(...) and isCardBe-tweenNumbers(...), where the ellipses are the arguments, are better than the other functions in the8SL. This is because preference is given to the actions that help to achieve Rack’o and to cards whenpicked up from the either the discard pile or Rack’o deck and swapped that may help to achieveRack’o in the future.
In this project, we investigated the applicability of the evolutionary approach with respect to programsynthesis. Unlike assignment 1, we looked at how an evolutionary approach would be feasible in thedomain of card games. We used a simplified version of Rack’o and analyzed the rules generated fromthe evolutionary algorithm.We ran the evolutionary algorithm for different population sizes and compared the generatedrules with a custom script containing the functions from the DSL. We found that our own scriptoutperformed the scripts generated from the EZS almost all the time. This was because our script wascarefully tailored to contain specific arguments to the strategies and thus facilitated the arrangementof cards in ascending order. On the other hand, the scripts generated from the evolutionary approachhad more stochasticity in terms of the values of the parameters. Overall, it can be observed that evenif the functions are same, the likelihood of achieving Rack’o is highly pertinent on the cards in thecurrent hand and the card selected from the Rack’o deck or the discard pile. In addition, from suchobservations, it is advisable to understand the game mechanics before formulating a strategy, whichwill elevate the chances of winning the game.In the future, stronger functions in the DSL would be used and even larger population sizes andgenerations should be used as well as increasing the amount of games played during each evaluation.We suspect that because our DSL has a large space, due to having to include all numbers 0-39, thatit would take longer for our scripts to explore and thus converge to an optimal script.Evaluating such computational methods for synthesizing scripts help to understand the gamemechanics better and devise better strategies that would contribute to the evolution of better andmore sophisticated card games.
The code for the project is given in the following GitHub repository:https://github.com/simpleParadox/CMPUT-659-Project.
References [1] “Evolutionary program sketching,” in
Lecture Notes in Computer Science (including subseriesLecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , vol. 10196 LNCS,Springer Verlag, 2017, pp. 3–18, isbn : 9783319556956. doi : .92] R. Sasanka and K. Krommydas, “An Evolutionary Framework for Automatic and Guided Dis-covery of Algorithms,” 2019. arXiv: . [Online]. Available: http://arxiv.org/abs/1904.02830http://arxiv.org/abs/1904.02830