Wendy Ashlock

Fingerprint Analysis of the Noisy Prisoner's Dilemma Using a Finite-State Representation

Fingerprinting is a technique that permits the identification of strategies for playing a game without doing detailed hand analysis. In this study the evolution of strategies for playing the iterated prisoners dilemma in the presence of noise was analyzed using fingerprinting and other techniques. Agents were evolved for 6400 generations taking samples at eight exponentially-spaced epochs with noise levels of 0, 1, and 5 percent. Populations were tested for probability of cooperative play, for competitive ability against agents evolved with different noise levels, for competitive ability against agents from other epochs, and for their distribution of strategy types. The ability of agents in noisy environments to cooperate was significantly enhanced over evolutionary time with substantial gains in cooperation made after the 3000 generation. Also, evolution in the presence of noise was found to significantly improve an agents competitive ability. Agents evolved for a longer time tended to beat agents evolved for a shorter time, though there were some intriguing exceptions. And, populations evolved in the presence of noise had significantly different strategy distributions than populations evolved without noise.

Changes in Prisoner’s Dilemma Strategies Over Evolutionary Time With Different Population Sizes

Prisoners dilemma is a simple game used for studying cooperation and conflict. This study evolves Prisoners dilemma strategies represented by 20-state finite state machines. The resulting strategies are difficult to analyze. It is not obvious looking at a finite state diagram how a machine will behave, and many different machines can represent the same strategy. This study uses a technique called fingerprinting to characterize the strategies. Thirty runs were done for each of three different population sizes for up to 65,536 generations and saved at different stages of evolution. A large diversity of strategies were found. Using different population sizes resulted in finding different strategies and finding common strategies in different proportions. Four strategies were found much more frequently than any others: tit-for-tat, always-defect, and two strategies defined in the study and named Fortress3 and Fortress4. A neighbor-joining technique was used to characterize the fifty most frequently found strategies, and they were found to fall into five distinct groups. The distribution of strategies was found to change over evolutionary time with tit-for-tat and always-defect found more often than other strategies in early evolution, and Fortress3 and Fortress4 becoming important later.

Using Very Small Population Sizes in Genetic Programming

This paper examines the use of very small (4-7) population sizes in genetic programming. When using exploitive operators, this results in hillclimbing; when using exploratory operators this results in genetic drift. The end result is a different way of searching the space which gives insight into the fitness landscape and the nature of the variation operators used. This study compares the use of very small population sizes with the use of population sizes up to 1000 for three genetic programming problems: 4-parity using parse trees, Tartarus using ISAc lists, and several versions of plus-one-recall-store (PORS) using parse trees. For 4-parity and Tartarus with 60 ISAc nodes, algorithms with very small population sizes found more solutions faster. For PORS, the effect was less pronounced: more solutions were found, but the algorithm was faster only than when using slightly larger populations. For Tartarus with 30 ISAc nodes, no effect was detected.

Fingerprint analysis of the noisy prisoner’s dilemma

Fingerprinting is a technique that permits automatic classification of strategies for playing a game. In this paper, the evolution of strategies for playing the iterated prisoners dilemma (IPD) at three different noise levels is analyzed using fingerprinting and other techniques including a novel quantity, evolutionary velocity, derived from fingerprinting. The results are at odds with initial expectations and permit the detection of a critical difference in the evolution of agents with and without noise. Noise during fitness evaluation places a larger fraction of an agents genome under selective pressure, resulting in substantially more efficient training. In this case, efficiency is the production of superior competitive ability at a lower evolutionary velocity. Prisoners dilemma playing agents are evolved for 6400 generations, taking samples at eight exponentially spaced epochs. This permits assessment of the change in populations over long evolutionary time. Agents are evaluated for competitive ability between those evolved for different lengths of time and between those evolved using distinct noise levels. The presence of noise during agent training is found to convey a commanding competitive advantage. A novel analysis is done in which a tournament is run with no two agents from the same evolutionary line and one third of agents from each noise level studied. This analysis simulates contributed agent tournaments without any genetic relation between agents. It is found that in early epochs the agents evolved without noise have the best average tournament rank, but that in later epochs they have the worst.

Why Some Representations Are More Cooperative Than Others For Prisoner's Dilemma

In the work of D. Ashlock et al. (2006) it was shown that the representation used has a large impact on the cooperativeness of evolved prisoners dilemma strategies. Why is this? This paper examines the look-up table representation, the finite state machine representation, and the neural net representation to gain insight into this somewhat surprising result. A tool called a prisoners dilemma fingerprint is used to compare the strategies produced by the different representations, and a Voronoi tiling (based on which of 12 reference strategies is the closest neighbor) of the strategy space is done. The initial random populations are shown to have significantly different distributions, and the evolved populations are shown to favor different parts of the strategy space

Evolved Features for DNA Sequence Classification and Their Fitness Landscapes

A key problem in genomics is the classification and annotation of sequences in a genome. A major challenge is identifying good sequence features. Evolutionary algorithms have the potential to search a large space of features and automatically generate useful ones. This paper proposes a two-stage method that generates features using multiple replicates of a genetic algorithm operating on an augmented finite state machine, called a side effect machine (SEM), and then selects a small diverse feature set using several methods, including a novel method called dissimilarity clustering. We apply our method to three problems related to transposable elements and compare the results to those using k-mer features. We are able to produce a small set of interesting and comprehensible features that create random forest classifiers more accurate and less prone to overfitting than those created using k-mer features. We analyze the SEM fitness landscapes and discuss the use of different fitness functions.

From competition to cooperation: Co-evolution in a rewards continuum

In this study the hypothesis that zero-sum (i.e strictly competitive) games are more difficult targets for co-evolution than non-zero-sum (i.e. games that are not strictly competitive nor strictly cooperative) games is examined. Our method is to compare the co-evolutionary behavior of a three move zero-sum game (rock paper scissors) with that of a three move non-zero-sum game (coordination prisoners dilemma) as well as with intermediate games obtained using weighted averages of the gamess payoff matrices. The games are compared by examining the way use of moves evolves, by using transitivity measures on evolved agents, by estimating the complexity of the agents and by checking for non-local adaptation. Two different agent representations, finite state machines with 8 and 64 states, are used. Unexpectedly, these two representations are found to have large, qualitative differences. The results support the hypothesis that co-evolving good strategies for zero-sum games is more difficult than for non-zero-sum games. Many of the measurements used to compare different games are found to exhibit a nonlinear responses to the change in payoff matrix.

Distinguishing Endogenous Retroviral LTRs from SINE Elements Using Features Extracted from Evolved Side Effect Machines

Side effect machines produce features for classifiers that distinguish different types of DNA sequences. They have the, as yet unexploited, potential to give insight into biological features of the sequences. We introduce several innovations to the production and use of side effect machine sequence features. We compare the results of using consensus sequences and genomic sequences for training classifiers and find that more accurate results can be obtained using genomic sequences. Surprisingly, we were even able to build a classifier that distinguished consensus sequences from genomic sequences with high accuracy, suggesting that consensus sequences are not always representative of their genomic counterparts. We apply our techniques to the problem of distinguishing two types of transposable elements, solo LTRs and SINEs. Identifying these sequences is important because they affect gene expression, genome structure, and genetic diversity, and they serve as genetic markers. They are of similar length, neither codes for protein, and both have many nearly identical copies throughout the genome. Being able to efficiently and automatically distinguish them will aid efforts to improve annotations of genomes. Our approach reveals structural characteristics of the sequences of potential interest to biologists.

Transience in the simulation of ring species

Biological ring species theoretically develop when an ancestral population expands around a geographic barrier and differentiates until terminal populations come back into contact. Adjacent populations are fertile; fertility declines with distance, and the terminal populations are not fertile. This study uses evolutionary algorithms to attempt to create artificial ring species using grid robots performing the Tartarus task with ISAc lists and string genes solving the Self Avoiding Walk (SAW) problem. Three experiments are done with the Tartarus robots. Fertility is shown to decrease with distance, but not to the extent that ring species are formed. Two experiments are done with SAW. These experiments produce sub-populations which satisfy all the criteria for biological ring species at the point in time when the ring closes. As evolution continues, the relationship between fertility and distance continues, but the terminal populations do not remain infertile. In addition, on both problems, record scores are achieved, suggesting that this model of evolution is a good optimizer for multi-optima problems like Tartarus and SAW which have many deceptive suboptima.

Dissecting relative contributions of cis- and trans-determinants to nucleosome distribution by comparing Tetrahymena macronuclear and micronuclear chromatin

The ciliate protozoan Tetrahymena thermophila contains two types of structurally and functionally differentiated nuclei: the transcriptionally active somatic macronucleus (MAC) and the transcriptionally silent germ-line micronucleus (MIC). Here, we demonstrate that MAC features well-positioned nucleosomes downstream of transcription start sites and flanking splice sites. Transcription-associated trans-determinants promote nucleosome positioning in MAC. By contrast, nucleosomes in MIC are dramatically delocalized. Nucleosome occupancy in MAC and MIC are nonetheless highly correlated with each other, as well as with in vitro reconstitution and predictions based upon DNA sequence features, revealing unexpectedly strong contributions from cis-determinants. In particular, well-positioned nucleosomes are often matched with GC content oscillations. As many nucleosomes are coordinately accommodated by both cis- and trans-determinants, we propose that their distribution is shaped by the impact of these nucleosomes on the mutational and transcriptional landscape, and driven by evolutionary selection.