James P. Crutchfield

The calculi of emergence: computation, dynamics and induction

Abstract Defining structure and detecting the emergence of complexity in nature are inherently subjective, though essential, scientific activities. Despite the difficulties, these problems can be analyzed in terms of how model-building observers infer from measurements the computational capabilities embedded in nonlinear processes. An observers notion of what is ordered, what is random, and what is complex in its environment depends directly on its computational resources: the amount of raw measurement data, of memory, and of time available for estimation and inference. The discovery of structure in an environment depends more critically and subtlely though on how those resources are organized. The descriptive power of the observers chosen (or implicit) computational model class, for example, can be an overwhelming determinant in finding regularity in data. This paper presents an overview of an inductive framework-hierarchical ϵ-machine reconstruction—in which the emergence of complexity is associated with the innovation of new computational model classes. Complexity metrics for detecting structure and quantifying emergence, along with an analysis of the constraints on the dynamics of innovation, are outlined. Illustrative examples are drawn from the onset of unpredictability in nonlinear systems, finitary nondeterministic processes, and cellular automata pattern recognition. They demonstrate how finite inference resources drive the innovation of new structures and so lead to the emergence of complexity.

Neutral evolution of mutational robustness.

We introduce and analyze a general model of a population evolving over a network of selectively neutral genotypes. We show that the populations limit distribution on the neutral network is solely determined by the network topology and given by the principal eigenvector of the networks adjacency matrix. Moreover, the average number of neutral mutant neighbors per individual is given by the matrix spectral radius. These results quantify the extent to which populations evolve mutational robustness-the insensitivity of the phenotype to mutations-and thus reduce genetic load. Because the average neutrality is independent of evolutionary parameters-such as mutation rate, population size, and selective advantage-one can infer global statistics of neutral network topology by using simple population data available from in vitro or in vivo evolution. Populations evolving on neutral networks of RNA secondary structures show excellent agreement with our theoretical predictions.

Quantum automata and quantum grammars

To study quantum computation, it might be helpful to generalize structures from language and automata theory to the quantum case. To that end, we propose quantum versions of finite-state and push-down automata, and regular and context-free grammars. We find analogs of several classical theorems, including pumping lemmas, closure properties, rational and algebraic generating functions, and Greibach normal form. We also show that there are quantum context-free languages that are not context-free.

Evolving cellular automata to perform computations: mechanisms and impediments

Abstract We present results from experiments in which a genetic algorithm (GA) was used to evolve cellular automata (CAs) to perform a particular computational task - one-dimensional density classification. We look in detail at the evolutionary mechanisms producing the GAs behavior on this task and the impediments faced by the GA. In particular, we identify four “epochs of innovation” in which new CA strategies for solving the problem are discovered by the GA, describe how these strategies are implemented in CA rule tables, and identify the GA mechanisms underlying their discovery. The epochs are characterized by a breaking of the tasks symmetries on the part of the GA. The symmetry breaking results in a short-term fitness gain but ultimately prevents the discovery of the most highly fit strategies. We discuss the extent to which symmetry breaking and other impediments are general phenomena in any GA search.

Fluctuations and simple chaotic dynamics

Abstract We describe the effects of fluctuations on the period-doubling bifurcation to chaos. We study the dynamics of maps of the interval in the absence of noise and numerically verify the scaling behavior of the Lyapunov characteristic exponent near the transition to chaos. As previously shown, fluctuations produce a gap in the period-doubling bifurcation sequence. We show that this implies a scaling behavior for the chaotic threshold and determine the associated critical exponent. By considering fluctuations as a disordering field on the deterministic dynamics, we obtain scaling relations between various critical exponents relating the effect of noise on the Lyapunov characteristic exponent. A rule is developed to explain the effects of additive noise at fixed parameter value from the deterministic dynamics at nearby parameter values.

Computational Mechanics: Pattern and Prediction, Structure and Simplicity

Computational mechanics, an approach to structural complexity, defines a processs causal states and gives a procedure for finding them. We show that the causal-state representation—an ∈-machine—is the minimal one consistent with accurate prediction. We establish several results on ∈-machine optimality and uniqueness and on how ∈-machines compare to alternative representations. Further results relate measures of randomness and structural complexity obtained from ∈-machines to those from ergodic and information theories.

Regularities Unseen, Randomness Observed: Levels of Entropy Convergence

We study how the Shannon entropy of sequences produced by an information source converges to the sources entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to stochastic and deterministic processes by using successive derivatives of the Shannon entropy growth curve. This leads, in turn, to natural measures of apparent memory stored in a source and the amounts of information that must be extracted from observations of a source in order for it to be optimally predicted and for an observer to synchronize to it. To measure the difficulty of synchronization, we define the transient information and prove that, for Markov processes, it is related to the total uncertainty experienced while synchronizing to a process. One consequence of ignoring a processs structural properties is that the missed regularities are converted to apparent randomness. We demonstrate that this problem arises particularly for settings where one has access only to short measurement sequences. Numerically and analytically, we determine the Shannon entropy growth curve, and related quantities, for a range of stochastic and deterministic processes. We conclude by looking at the relationships between a processs entropy convergence behavior and its underlying computational structure.

Noise phenomena in Josephson junctions

We suggest that the reported noise‐rise phenomenon observed in Josephson oscillators can be understood in terms of the full nonlinear and deterministic junction dynamics. We show that the drive damped pendulum equation describing the junction behavior exhibits chaotic solutions associated with the appearance of strange attractors in phase space. These results are relevant to the general problem of turbulent behavior of anharmonic systems.

Symbolic dynamics of noisy chaos

Abstract One model of randomness observed in physical systems is that low-dimensional deterministic chaotic attractors underly the observations. A phenomenological theory of chaotic dynamics requires an accounting of the information flow from the observed system to the observer, the amount of information available in observations, and just how this information affects predictions of the systems future behavior. In an effort to develop such a description, we discuss the information theory of highly discretized observations of random behavior. Metric entropy and topological entropy are well-defined invariant measures of such an attractors “level of chaos”, and are computable using symbolic dynamics. Real physical systems that display low dimensional dynamics are, however, inevitably coupled to high-dimensional randomless, e.g. thermal noise. We investigate the effects of such fluctuations coupled to deterministic chaotic systems, in particular, the metric entropys response to the fluctuations. We find that the entropy increases with a power law in the noise level, and that the convergence of the entropy and the effect of fluctuations can be cast as a scaling theory. We also argue that in addition to the metric entropy, there is a second scaling invariant quantity that characterizes a deterministic system with added fluctuations: I0, the maximum average information obtainable about the initial condition that produces a particular sequence of measurements (or symbols).

A Genetic Algorithm Discovers Particle-Based Computation in Cellular Automata

How does evolution produce sophisticated emergent computation in systems composed of simple components limited to local interactions? To model such a process, we used a genetic algorithm (GA) to evolve cellular automata to perform a computational task requiring globally-coordinated information processing. On most runs a class of relatively unsophisticated strategies was evolved, but on a subset of runs a number of quite sophisticated strategies was discovered. We analyze the emergent logic underlying these strategies in terms of information processing performed by “particles” in space-time, and we describe in detail the generational progression of the GA evolution of these strategies. Our analysis is a preliminary step in understanding the general mechanisms by which sophisticated emergent computational capabilities can be automatically produced in decentralized multiprocessor systems.