John H. Holland

Genetic Algorithms and Machine Learning

There is no a priori reason why machine learning must borrow from nature. A field could exist, complete with well-defined algorithms, data structures, and theories of learning, without once referring to organisms, cognitive or genetic structures, and psychological or evolutionary theories. Yet at the end of the day, with the position papers written, the computers plugged in, and the programs debugged, a learning edifice devoid of natural metaphor would lack something. It would ignore the fact that all these creations have become possible only after three billion years of evolution on this planet. It would miss the point that the very ideas of adaptation and learning are concepts invented by the most recent representatives of the species Homo sapiens from the careful observation of themselves and life around them. It would miss the point that natural examples of learning and adaptation are treasure troves of robust procedures and structures. Fortunately, the field of machine learning does rely upon natures bounty for both inspiration and mechanism. Many machine learning systems now borrow heavily from current thinking in cognitive science, and rekindled interest in neural networks and connectionism is evidence of serious mechanistic and philosophical currents running through the field. Another area where natural example has been tapped is in work on genetic algorithms (GAs) and genetics-based machine learning. Rooted in the early cybernetics movement (Holland, 1962), progress has been made in both theory (Holland, 1975; Holland, Holyoak, Nisbett, & Thagard, 1986) and application (Goldberg, 1989; Grefenstette, 1985, 1987) to the point where genetics-based systems are finding their way into everyday commercial use (Davis & Coombs, 1987; Fourman, 1985).

Classifier systems and genetic algorithms

Classifier systems are massively parallel, message-passing, rule-based systems that learn through credit assignment (the bucket brigade algorithm) and rule discovery (the genetic algorithm). They typically operate in environments that exhibit one or more of the following characteristics: (1) perpetually novel events accompanied by large amounts of noisy or irrelevant data; (2) continual, often real-time, requirements for action; (3) implicitly or inexactly defined goals; and (4) sparse payoff or reinforcement obtainable only through long action sequences. Classifier systems are designed to absorb new information continuously from such environments, devising sets of competing hypotheses (expressed as rules) without disturbing significantly capabilities already acquired. This paper reviews the definition, theory, and extant applications of classifier systems, comparing them with other machine learning techniques, and closing with a discussion of advantages, problems, and possible extensions of classifier systems.

Cognitive systems based on adaptive algorithms

The type of cognitive system (CS) studied here has four basic parts: (1) a set of interacting elementary productions, called classifiers, (2) a performance algorithm that directs the action of the system in the environment, (3) a simple learning algorithm that keeps a record of each classifiers success in bringing about rewards, and (4) a more complex learning algorithm, called the genetic algorithm, that modifies the set of classifiers so that variants of good classifiers persist and new, potentially better ones are created in a provably efficient manner.

Genetic Algorithms and the Optimal Allocation of Trials

This study gives a formal setting to the difficult optimization problems characterized by the conjunction of (1) substantial complexity and initial uncertainty, (2) the necessity of acquiring new i...

Building Blocks, Cohort Genetic Algorithms, and Hyperplane-Defined Functions

Building blocks are a ubiquitous feature at all levels of human understanding, from perception through science and innovation. Genetic algorithms are designed to exploit this prevalence. A new, more robust class of genetic algorithms, cohort genetic algorithms (cGAs), provides substantial advantages in exploring search spaces for building blocks while exploiting building blocks already found. To test these capabilities, a new, general class of test functions, the hyperplane-defined functions (hdfs), has been designed. Hdfs offer the means of tracing the origin of each advance in performance; at the same time hdfs are resistant to reverse engineering, so that algorithms cannot be designed to take advantage of the characteristics of particular examples.

COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1

The type of cognitive system (CS) studied here has four basic parts: (1) a set of interacting elementary productions, called lassifiers, (2) a performance algorithm that directs the action of the system in the environment, (3) a simple learning algorithm that keeps a record of each classifiers success in bringing about rewards, and (4) a more complex learning algorithm, called the genetic algorithm, that modifies the set of classifiers so that variants of good classifiers persist and new, potentially better ones are created in a provably efficient manner. Two “proof-of-principle” experiments are reported. One experiment shows CSs performance in a maze when it has only the ability to adjust the predictions about ensuing rewards of classifiers (similar to adjusting the “weight” of each classifier) vs. when the power of the genetic algorithm is added. Criterion was achieved an order of magnitude more rapidly when the genetic algorithm was operative. A second experiment examines transfer of learning. Placed in a more difficult maze, CS with experience in the simpler maze reaches criterion an order of magnitude more rapidly than CS without prior experience.

Concerning the emergence of tag-mediated lookahead in classifier systems

Abstract This paper, after a general introduction to the area, discusses the architecture and learning algorithms that permit automatic parallel, distributed lookahead to emerge in classifier systems. Simple additions to a “standard” classifier system suffice, principally a new register called the virtual strength register, and a provision to use the bucket brigade credit assignment algorithm in “virtual” mode to modify values in this register. With these additions, current actions are decided on the basis of the expected values associated with the “lookahead cones” of possible alternatives.

Genetic Algorithms and Adaptation

Genetics provides us with a canonical example of a complex search through a space of ill-defined possibilities. The basic problem is one of manipulating representations — the chromosomes — so as to search out and generate useful organization — the functional properties of the organism.

Iterative circuit computers

The paper first discusses an example of a computer, intended as a prototype of a practical computer, having an iterative structure and capable of processing arbitrarily many words of stored data at the same time, each by a different sub-program if desired. Next a mathematical characterization is given of a broad class of computers satisfying the conditions just stated. Finally the characterization is related to a program aimed at establishing a theory of adaptive systems via the concept of automaton generators.

Exploring the evolution of complexity in signaling networks

Signaling networks are exemplified by systems as diverse as biological cells, economic markets, and the Web. After a discussion of some general characteristics of signaling networks, this article explores the adaptive evolution of complexity in a simple model of a signaling network. The article closes with a discussion of broader questions concerning the evolution of signaling networks.