Anthony Bucci

On identifying global optima in cooperative coevolution

When applied to optimization problems, Cooperative Coevolutionary Algorithms(CCEA) have been observed to exhibit a behavior called relative overgeneralization. Roughly, they tend to identify local optima with large basins of attraction which may or may not correspond to global optima. A question which arises is whether one can modify the algorithm to promote the discovery of global optima. We argue that a mechanism from Pareto coevolution can achieve this end. We observe that in CCEAs candidate individuals from one population are used as tests or measurements of individuals in other populations; by treating individuals as tests in this way, a finer-grained comparison can be made among candidate individuals. This finer-grained view permits an algorithm to see when two candidates are differently capable, even when ones evident value is higher than the others. By modifying an existing CCEA to compare individuals using Pareto dominance we have produced an algorithm which reliably finds global optima. We demonstrate the algorithm on two Maximum of Two Quadratics problems and discuss why it works.

Automated Extraction of Problem Structure

Most problems studied in artificial intelligence possess some form of structure, but a precise way to define such structure is so far lacking. We investigate how the notion of problem structure can be made precise, and propose a formal definition of problem structure. The definition is applicable to problems in which the quality of candidate solutions is evaluated by means of a series of tests. This specifies a wide range of problems: tests can be examples in classification, test sequences for a sorting network, or opponents for board games. Based on our definition of problem structure, we provide an automatic procedure for problem structure extraction, and results of proof-of-concept experiments. The definition of problem structure assigns a precise meaning to the notion of the underlying objectives of a problem, a concept which has been used to explain how one can evaluate individuals in a coevolutionary setting. The ability to analyze and represent problem structure may yield new insight into existing problems, and benefit the design of algorithms for learning and search.

DECA: dimension extracting coevolutionary algorithm

Coevolution has often been based on averaged outcomes, resulting in unstable evaluation. Several theoretical approaches have used archives to provide stable evaluation. However, the number of tests required by some of these approaches can be prohibitive of practical applications. Recent work has shown the existence of a set of underlying objectives which compress evaluation information into a potentially small set of dimensions. We consider whether these underlying objectives can be approximated online, and used for evaluation in a coevolution algorithm. The Dimension Extracting Coevolutionary Algorithm (DECA) is compared to several recent reliable coevolution algorithms on a Numbers game problem, and found to perform efficiently. Application to the more realistic Tartarus problem is shown to be feasible. Implications for current coevolution research are discussed.

Focusing versus intransitivity: geometrical aspects of co-evolution

Recently, a minimal domain dubbed the numbers game has been proposed to illustrate well-known issues in co-evolutionary dynamics. The domain permits controlled introduction of features like intransitivity, allowing researchers to understand failings of a co-evolutionary algorithm in terms of the domain. In this paper, we show theoretically that a large class of co-evolution problems closely resemble this minimal domain. In particular, all the problems in this class can be embedded into an ordered, n-dimensional Euclidean space, and so can be construed as greater-than games. Thus, conclusions derived using the numbers game are more widely applicable than might be presumed. In light of this observation, we present a simple algorithm aimed at remedying focusing problems and relativism in the numbers game. With it we show empirically that, contrary to expectations, focusing in transitive games can be more troublesome for co-evolutionary algorithms than intransitivity. Practitioners should therefore be just as wary of focusing issues in application domains.

Thoughts on solution concepts

This paper explores connections between Ficicis notion of solution concept and order theory. Ficici postulates that algorithms should ascend an order called weak preference; thus, understanding this order is important to questions of designing algorithms. We observe that the weak preference order is closely related to the pullback of the so-called lower ordering on subsets of an ordered set. The latter can, in turn, be represented as the pullback of the subset ordering of a certain powerset. Taken together, these two observations represent the weak preference ordering in a more simple and concrete form as a subset ordering. We utilize this representation to show that algorithms which ascend the weak preference ordering are vulnerable to a kind of bloating problem. Since this kind of bloat has been observed several times in practice, we hypothesize that ascending weak preference may be the cause. Finally, we show that monotonic solution concepts are convex in the order-theoretic sense. We conclude by speculating that monotonic solution concepts might be derivable from non-monotonic ones by taking convex hull. Since several intuitive solution concepts like average fitness are not monotonic, there is practical value in creating monotonic solution concepts from non-monotonic ones.

Advanced tutorial on coevolution

The advanced tutorial on coevolution continues the topics covered in the introductory coevolution tutorial with a view towards research conducted in the last eight years. We will explore two themes which have recently been identified: interaction, which is where the context-sensitive nature of evaluation manifests itself; and elaboration, which represents the goal of accumulating capabilities. Evolutionary game theory and the order theory used in Pareto coevolution will be covered as examples of the study of interaction. NeuroEvolution of Augmenting Topologies (NEAT) in particular and monotonic solution concepts more generally treat questions of elaboration. Further topics to be covered include: EGT and dynamical systems studies of cooperative coevolutionary algorithms; archive methods; dimension extraction; and the estimation-exploration algorithm.

Objective Set Compression

We consider a class of optimization problems wherein the quality of candidate solutions is estimated by their performance on a number of tests. Classifier induction, function regression, and certain types of reinforcement learning, including problems often attacked with coevolutionary algorithms, can all be seen as members of this class. Traditional approaches to such test-based problems use a single objective function that aggregates the scores obtained on the tests. Recent work, by contrast, argues that useful finer-grained distinctions between candidate solutions are obtained when each test is treated as a separate objective, and that algorithms employing such multiobjective comparisons show favourable behaviour relative to those which do not. Unfortunately, the number of tests can be very large. Since it is well-known that high-dimensional multiobjective optimization problems are difficult to handle in practice, the question arises whether the multiobjective treatment of test-based problems is feasible. To begin addressing this question, we examine a method for reducing the number of dimensions without sacrificing the favorable properties of the multiobjective approach. Our method, which is a form of dimension extraction, finds underlying objectives implicit in test-based problems. Essentially, the method proceeds by placing the tests along the minimal number of coordinate axes that still preserve ordering information among the candidate solutions. Application of the method to the strategy set for several instances of the game of Nim suggest the technique has significant practical benefits: a type of compression of the set of objectives is observed in all tested instances. Surprisingly, we also find that the information contained in the arrangement of tests on the coordinate axes reveals important information about the structure of the underlying problem.

Analysis of coevolution for worst-case optimization

The problem of finding entities with the best worst-case performance across multiple scenarios arises in domains ranging from job shop scheduling to designing physical artifacts. In spite of previous successful applications of evolutionary computation techniques, particularly coevolution, to such domains, little work has examined utilizing coevolution for optimizing worst-case behavior. Previous work assesses certain algorithm mechanisms using aggregate performance on test problems. We examine fitness and population trajectories of individual algorithm runs, making two observations: first, that aggregate plots wash out important effects that call into question what these algorithms can produce; and second, that none of the mechanisms is generally better than the rest. More importantly, our dynamics analysis explains how the interplay of algorithm properties and problem properties influences performance. These contributions argue in favor of a reassessment of what makes for a good worst-case coevolutionary algorithm and suggest how to design one.

On the practicality of optimal output mechanisms for co-optimization algorithms

Co-optimization problems involve one or more search spaces and a means of assessing interactions between entities in these spaces. Assessing a potential solution requires aggregating in some way the outcomes of a very large or infinite number of such interactions. This layer of complexity presents difficulties for algorithm design that are not encountered in ordinary optimization. For example, what a co-optimization algorithm should output is not at all obvious. Theoretical research has shown that some output selection mechanisms yield better overall performance than others and described an optimal mechanism. This mechanism was shown to be strictly better than a greedy method in common use, but appeared prohibitive from a practical standpoint. In this paper we exhibit the optimal output mechanism for a particular class of co-optimization problems and a certain definition of better overall performance, and provide quantitative characterizations of domains for which this optimal mechanism becomes straightforward to implement. We conclude with a discussion of potential extensions of this work to other problem classes and other views on performance.