Matthew J. Streeter

What's AI done for me lately? Genetic programming's human-competitive results

The automated problem-solving technique of genetic programming has generated at least 36 human-competitive results. In six cases, it automatically duplicated the functionality of inventions patented after January 2000.

The root causes of code growth in genetic programming

This paper discusses the underlying pressures responsible for code growth in genetic programming, and shows how an understanding of these pressures can be used to use to eliminate code growth while simultaneously improving performance. We begin with a discussion of two distinct components of code growth and the extent to which each component is relevant in practice. We then define the concept of resilience in GP trees, and show that the buildup of resilience is essential for code growth. We present simple modifications to the selection procedures used by GP that eliminate bloat without hurting performance. Finally, we show that eliminating bloat can improve the performance of genetic programming by a factor that increases as the problem is scaled in difficulty.

Routine high-return human-competitive automated problem-solving by means of genetic programming

Genetic programming is a systematic method for getting computers to automatically solve problems. Genetic programming starts from a high-level statement of what needs to be done and automatically creates a computer program to solve the problem by means of a simulated evolutionary process. The paper demonstrates that genetic programming (1) now routinely delivers high-return human-competitive machine intelligence; (2) is an automated invention machine; (3) can automatically create a general solution to a problem in the form of a parameterized topology and (4) has delivered a progression of qualitatively more substantial results in synchrony with five approximately order-of-magnitude increases in the expenditure of computer time. These points are illustrated by a group of recent results involving the automatic synthesis of the topology and sizing of analog electrical circuits, the automatic synthesis of placement and routing of circuits, and the automatic synthesis of controllers as well as references to work involving the automatic synthesis of antennas, networks of chemical reactions (metabolic pathways), genetic networks, mathematical algorithms, and protein classifiers.

Upper Bounds on the Time and Space Complexity of Optimizing Additively Separable Functions

We present upper bounds on the time and space complexity of finding the global optimum of additively separable functions, a class of functions that has been studied extensively in the evolutionary computation literature. The algorithm presented uses efficient linkage discovery in conjunction with local search. Using our algorithm, the global optimum of an order-k additively separable function defined on strings of length l can be found using O(l ln(l)2 k ) function evaluations, a bound which is lower than all those that have previously been reported.

A simple distribution-free approach to the max k-armed bandit problem

The max k-armed bandit problem is a recently-introduced online optimization problem with practical applications to heuristic search. Given a set of k slot machines, each yielding payoff from a fixed (but unknown) distribution, we wish to allocate trials to the machines so as to maximize the maximum payoff received over a series of n trials. Previous work on the max k-armed bandit problem has assumed that payoffs are drawn from generalized extreme value (GEV) distributions. In this paper we present a simple algorithm, based on an algorithm for the classical k-armed bandit problem, that solves the max k-armed bandit problem effectively without making strong distributional assumptions. We demonstrate the effectiveness of our approach by applying it to the task of selecting among priority dispatching rules for the resource-constrained project scheduling problem with maximal time lags (RCPSP/max).

Two broad classes of functions for which a no free lunch result does not hold

We identify classes of functions for which a No Free Lunch result does and does not hold, with particular emphasis on the relationship between No Free Lunch and problem description length. We show that a NFL result does not apply to a set of functions when the description length of the functions is sufficiently bounded. We consider sets of functions with non-uniform associated probability distributions, and show that a NFL result does not hold if the probabilities are assigned according either to description length or to a Solomonoff-Levin distribution. We close with a discussion of the conditions under which NFL can apply to sets containing an infinite number of functions.

Toward Automated Design of Industrial-Strength Analog Circuits by Means of Genetic Programming

It has been previously established that genetic programming can be used as an automated invention machine to synthesize designs for complex structures. In particular, genetic programming has automatically synthesized structures that infringe, improve upon, or duplicate the functionality of 21 previously patented inventions (including six 21st-century patented analog electrical circuits) and has also generated two patentable new inventions (controllers). There are seven promising factors suggesting that these previous results can be extended to deliver industrial-strength automated design of analog circuits, but two countervailing factors. This chapter explores the question of whether the seven promising factors can overcome the two countervailing factors by reviewing progress on an ongoing project in which we are employing genetic programming to synthesize an amplifier circuit. The work involves a multiobjective fitness measure consisting of 16 different elements measured by five different test fixtures. The chapter describes five ways of using general domain knowledge applicable to all analog circuits, two ways for employing problem-specific knowledge, four ways of improving on previously published genetic programming techniques, and four ways of grappling with the multi-objective fitness measures associated with real-world design problems.

Automatic synthesis using genetic programming of an improved general-purpose controller for industrially representative plants

Most real-world controllers are composed of proportional, integrative, and derivative Signal processing blocks. The so-called PID controller was invented and patented by A. Callender and A.B. Stevenson (1939). Later J.G. Ziegler and N.B. Nichols (1942) developed mathematical rules for automatically selecting the parameter values for PID controllers. In their influential book, K.J. Astrom and T. Hagglund (1995) developed a world-beating PID controller that outperforms the 1942 Ziegler-Nichols rules on an industrially representative set of plants. In this paper, we approached the problem of automatic synthesis of a controller using genetic programming without requiring in advance that the topology of the plant be the conventional PID topology. We present a genetically evolved controller that outperforms the automatic tuning rules developed by Astrom and Hagglund in 1995 for the industrially representative set of plants specified by Astrom and Hagglund.

Automated Discovery of Numerical Approximation Formulae via Genetic Programming

This paper describes the use of genetic programming to perform automated discovery of numerical approximation formulae. We present results involving rediscovery of known approximations for Harmonic numbers, discovery of rational polynomial approximations for functions of one or more variables, and refinement of existing approximations through both approximation of their error function and incorporation of the approximation as a program tree in the initial GP population. Evolved rational polynomial approximations are compared to Padé approximations obtained through the Maple symbolic mathematics package. We find that approximations evolved by GP can be superior to Padé approximations given certain tradeoffs between approximation cost and accuracy, and that GP is able to evolve approximations in circumstances where the Padé approximation technique cannot be applied. We conclude that genetic programming is a powerful and effective approach that complements but does not replace existing techniques from numerical analysis.

NVIS: an interactive visualization tool for neural networks

This paper presents NVIS, an interactive graphical tool used to examine the weights, topology, and activations of a single artificial neural networks (ANN), as well as the genealogical relationships between members of a population of ANNs as they evolve under an evolutionary algorithm. NVIS is unique in its depiction of nodal activation values, its usage of family tree diagrams to indicate the origin of individual networks, and the degree of interactivity it allows the user while the learning process takes place. The authors have made use of these feature to obtain insights into both the workings of single neural networks and the evolutionary process, based upon which we consider NVIS to be an effective visualization tool of value to designers, users, and students of ANNs.