Compression-based investigation of the dynamical properties of cellular automata and other systems
CCompression-Based Investigation of the Dynamical Properties of Cellular Automata and Other Systems
Hector Zenil
Laboratoire d’Informatique Fondamentale de Lille (USTL/CNRS) and Institut d’Histoire et de Philosophie des Sciences et des Techniques (CNRS/Paris 1/ENS Ulm)[email protected]
A method for studying the qualitative dynamical properties of abstractcomputing machines based on the approximation of their program-sizecomplexity using a general lossless compression algorithm is presented.It is shown that the compression-based approach classifies cellularautomata into clusters according to their heuristic behavior. These clus-ters show a correspondence with Wolfram’s main classes of cellularautomata behavior. A Gray code-based numbering scheme is developedfor distinguishing initial conditions. A compression-based method toestimate a characteristic exponent for detecting phase transitions andmeasuring the resiliency or sensitivity of a system to its initial condi-tions is also proposed. A conjecture regarding the capability of a systemto reach computational universality related to the values of this phasetransition coefficient is formulated. These ideas constitute a compres-sion-based framework for investigating the dynamical properties of cel-lular automata and other systems.
1. Introduction
Previous investigations of the dynamical properties of cellularautomata have involved compression in one form or another. In thispaper, we take the direct approach, experimentally studying the rela-tionship between properties of dynamics and their compression. Cellular automata were first introduced by J. von Neumann [1] asa mathematical model for biological self-replication phenomena. Theyhave since played a basic role in understanding and explaining vari-ous complex physical, social, chemical, and biological phenomena.Using extensive computer simulation S. Wolfram [2] classified cellularautomata into four classes according to the qualitative behavior oftheir evolution. This classification has been further investigated andverified by G. Braga et al. [3], followed by more detailed verificationsand investigations of classes 1, 2, and 3 in [4, 5]. Other formal approaches to the problem of classifying cellularautomata have also been attempted, with some success. Of these,some are based on the structure of attractors or other topological clas-sifications [6, 7], others use probabilistic approaches [8] or involvelooking at whether a cellular automaton falls into some chaotic attrac-tor or an undecidable class [9 | K ) of the ruletable of a cellular automaton [12]. It has also been shown that rulesthat in certain conditions belong to one class may belong to anotherwhen starting from a different set up. This has been the case of rule40, simple and therefore in class 1 when starting from a 0-finite con-figuration but chaotic [13] when starting from certain randomconfigurations. Complex Systems, © 2010 Complex Systems Publications, Inc. ther formal approaches to the problem of classifying cellularautomata have also been attempted, with some success. Of these,some are based on the structure of attractors or other topological clas-sifications [6, 7], others use probabilistic approaches [8] or involvelooking at whether a cellular automaton falls into some chaotic attrac-tor or an undecidable class [9 | K ) of the ruletable of a cellular automaton [12]. It has also been shown that rulesthat in certain conditions belong to one class may belong to anotherwhen starting from a different set up. This has been the case of rule40, simple and therefore in class 1 when starting from a 0-finite con-figuration but chaotic [13] when starting from certain randomconfigurations.Compression-based mathematical characterizations and techniquesfor classifying and clustering have been suggested and successfully de-veloped in areas as diverse as languages, literature, genomics, music,and astronomy. A good introduction can be found in [14]. Compres-sion is a powerful tool for pattern recognition and has often beenused for classification and clustering. Lempel | Ziv (LZ)-like data com-pressors have been proven to be universally optimal and are thereforegood candidates as approximators of the program-size complexity ofstrings.The program-size complexity [15] K u H s L of a string s with respectto a universal Turing machine U is defined as the binary length of theshortest program p that produces as output the string s . Or, as a Mathematica expression: K u H s L = min H Length @ p DL , U H p L = s < However, a drawback of K is that it is an uncomputable function.In general, the only way to approach K is by compressibility methods.Essentially, the program-size complexity of a string is the ultimatecompressed version of that string.As an attempt to capture and systematically study the behavior ofabstract machines, our experimental approach consists of calculatingthe program-size complexity of the output of the evolution of a cellu-lar automaton. This is done following methods of extended computa-tion, enumerating, and exhaustively running the systems as suggestedin [2].A cellular automaton is a collection of cells on a grid of specifiedshape that evolves through a number of discrete time steps accordingto a set of rules based on the states of neighboring cells. The rules areapplied iteratively for as many time steps as desired. The number ofcolors (or distinct states) k of a cellular automaton is a non-negative in-teger. In addition to the grid on which a cellular automaton lives andthe colors its cells may assume, the neighborhood over which cells af-fect one another must also be specified. The simplest choice is a near-est-neighbor rule, in which only cells directly adjacent to a given cellmay be affected at each time step. The simplest type of cellular automa-ton is then a binary, nearest-neighbor, one-dimensional automaton(called elementary by Wolfram). There are 256 such automata, eachof which can be indexed by a unique binary number whose decimalrepresentation is known as the rule for the particular automaton. H. Zenil
Complex Systems,19
Complex Systems,19 © 2010 Complex Systems Publications, Inc. cellular automaton is a collection of cells on a grid of specifiedshape that evolves through a number of discrete time steps accordingto a set of rules based on the states of neighboring cells. The rules areapplied iteratively for as many time steps as desired. The number ofcolors (or distinct states) k of a cellular automaton is a non-negative in-teger. In addition to the grid on which a cellular automaton lives andthe colors its cells may assume, the neighborhood over which cells af-fect one another must also be specified. The simplest choice is a near-est-neighbor rule, in which only cells directly adjacent to a given cellmay be affected at each time step. The simplest type of cellular automa-ton is then a binary, nearest-neighbor, one-dimensional automaton(called elementary by Wolfram). There are 256 such automata, eachof which can be indexed by a unique binary number whose decimalrepresentation is known as the rule for the particular automaton.Regardless of the apparent simplicity of their formal description,cellular automata are capable of displaying a wide range of interestingand different dynamical properties as thoroughly investigated by Wol-fram in [2]. The problem of classification is a central topic in cellularautomata theory.Wolfram identifies and classifies cellular automata (and other dis-crete systems) as displaying these four different classes of behavior.
1. A fixed, homogeneous state is eventually reached (e.g., rules 0, 8, 136).2. A pattern consisting of separated periodic regions is produced (e.g.,rules 4, 37, 56, 73).3. A chaotic, aperiodic pattern is produced (e.g., rules 18, 45, 146).4. Complex, localized structures are generated (e.g., rule 110).
2. Compression-Based Classification
The method consists of compressing the evolution of a cellular au-tomaton up to a certain number of steps. The
Mathematica function
Compress [16] gives a compressed representation of an expression asa string. It uses a C language implementation of a “deflate” compliantcompressor and decompressor available within the zlib package. Thedeflate lossless compression algorithm, independent of CPU type,operating system, file system, and character set compresses data usinga combination of the LZ algorithm and Huffman coding [17 | | Ziv | Welch (LZW) algorithm. The same algorithm is thebasis of the widely used gzip data compression software. Data com-pression is generally achieved through two steps: † The matching and replacement of duplicate strings with pointers. † Replacing symbols with new, weighted symbols based on frequency ofuse.
The difference in length between the compressed and uncompressedforms of the output of a cellular automaton is a good approximationof its program-size complexity. In most cases, the length of the com-pressed form levels off, indicating that the cellular automaton outputis repetitive and can easily be described. However, in cases like rules30, 45, 110, or 73 the length of the compressed form grows rapidly,corresponding to the apparent randomness and lack of structure inthe display.
Compression-Based Investigation of CAs and Other Systems Complex Systems, © 2010 Complex Systems Publications, Inc. .1.1 Classification Parameters There are two main parameters that play a role when classifying cellu-lar automata: the initial configuration and the number of steps. Classi-fying cellular automata can begin by starting all with a single blackcell. Some of them, such as rule 30, will immediately show their fullrichness in dynamical terms, while others might produce very differ-ent behavior when starting with another initial configuration. Bothtypes might produce different classifications. We first explore the caseof starting with a single black cell and then proceed to consider theother case for detecting phase transitions.An illustration of the evolution of rules 95, 82, 50, and 30 isshown in Figure 1, together with the compressed and uncompressedlengths they produce, each starting from a single black cell movingthrough time (number of steps).As shown in Figure 1, the compressed lengths of simple cellular au-tomata do not approach the uncompressed lengths and stay flat orgrow linearly, while the length of the compressed form approachesthe length of the uncompressed form for rules such as 30.Cellular automata can be classified by sorting their compressedlengths as an approximation to their program-size complexity. In Fig-ure 2, c is the compressed length of the evolution of the cellular au-tomaton up to the first 200 steps (although the pictures only show thefirst 60). Early in 2007 I wrote a program using Mathematica H. Zenil
Complex Systems,19
Complex Systems,19 © 2010 Complex Systems Publications, Inc. ule 95 0 10 20 30 40 steps length steps length Rule 82 0 10 20 30 40 steps length steps length
Rule 50 0 10 20 30 40 steps length steps length
Rule 30 0 10 20 30 40 steps length steps length
Figure 1.
Evolution of rules 95, 82, 50, and 30 together with the compressed(dashed line) and uncompressed (solid line) lengths. rule 255 c = rule 247 c = rule 232 c = rule 224 c = rule 223 c = rule 215 c = rule 200 c = rule 192 c = rule 191 c = rule 183 c = rule 168 c = rule 160 c = rule 159 c = rule 151 c = rule 136 c = rule 128 c = rule 127 c = rule 119 c = Figure 2.
Complete compression-based classification of the elementary cellularautomata ( continues ). Compression-Based Investigation of CAs and Other Systems Complex Systems, © 2010 Complex Systems Publications, Inc. ule 104 c = rule 96 c = rule 95 c = rule 87 c = rule 72 c = rule 64 c = rule 63 c = rule 55 c = rule 40 c = rule 32 c = rule 31 c = rule 23 c = rule 8 c = rule 0 c = rule 253 c = rule 251 c = rule 239 c = rule 21 c = rule 19 c = rule 7 c = rule 237 c = rule 249 c = rule 235 c = rule 233 c = rule 221 c = rule 205 c = rule 236 c = rule 228 c = rule 219 c = rule 217 c = rule 207 c = rule 204 c = rule 203 c = rule 201 c = rule 196 c = rule 172 c = rule 164 c = rule 140 c = rule 132 c = rule 108 c = rule 100 c = rule 76 c = rule 68 c = rule 51 c = rule 44 c = rule 36 c = rule 12 c = rule 4 c = rule 29 c = rule 71 c = rule 123 c = rule 33 c = rule 5 c = rule 1 c = rule 91 c = rule 37 c = rule 83 c = rule 53 c = rule 234 c = rule 226 c = rule 202 c = rule 194 c = rule 189 c = rule 187 c = rule 175 c = rule 170 c = rule 162 c = rule 138 c = rule 130 c = rule 106 c = rule 98 c = rule 85 c = Figure 2. ( continued ) H. Zenil
Complex Systems,19
Complex Systems,19 © 2010 Complex Systems Publications, Inc. ule 74 c = rule 66 c = rule 42 c = rule 39 c = rule 34 c = rule 27 c = rule 10 c = rule 2 c = rule 248 c = rule 245 c = rule 243 c = rule 240 c = rule 231 c = rule 216 c = rule 208 c = rule 184 c = rule 176 c = rule 174 c = rule 173 c = rule 152 c = rule 144 c = rule 143 c = rule 142 c = rule 120 c = rule 117 c = rule 113 c = rule 112 c = rule 88 c = rule 81 c = rule 80 c = rule 56 c = rule 48 c = rule 46 c = rule 24 c = rule 16 c = rule 15 c = rule 14 c = rule 244 c = rule 241 c = rule 229 c = rule 227 c = rule 213 c = rule 212 c = rule 209 c = rule 185 c = rule 171 c = rule 139 c = rule 116 c = rule 84 c = rule 47 c = rule 43 c = rule 11 c = rule 49 c = rule 17 c = rule 211 c = rule 180 c = rule 166 c = rule 155 c = rule 148 c = rule 134 c = rule 115 c = rule 59 c = rule 52 c = rule 38 c = rule 35 c = rule 20 c = rule 6 c = rule 3 c = rule 41 c = rule 61 c = rule 125 c = rule 111 c = Figure 2. ( continued ) Compression-Based Investigation of CAs and Other Systems Complex Systems, © 2010 Complex Systems Publications, Inc. ule 103 c = rule 67 c = rule 65 c = rule 25 c = rule 9 c = rule 97 c = rule 107 c = rule 121 c = rule 141 c = rule 78 c = rule 252 c = rule 220 c = rule 79 c = rule 13 c = rule 238 c = rule 206 c = rule 69 c = rule 93 c = rule 254 c = rule 222 c = rule 157 c = rule 198 c = rule 70 c = rule 163 c = rule 199 c = rule 250 c = rule 242 c = rule 186 c = rule 179 c = rule 178 c = rule 156 c = rule 122 c = rule 114 c = rule 77 c = rule 58 c = rule 50 c = rule 28 c = rule 188 c = rule 230 c = rule 225 c = rule 197 c = rule 246 c = rule 92 c = rule 190 c = rule 169 c = rule 147 c = rule 54 c = rule 158 c = rule 177 c = rule 214 c = rule 133 c = rule 94 c = rule 99 c = rule 57 c = rule 195 c = rule 60 c = rule 153 c = rule 102 c = rule 167 c = rule 181 c = rule 218 c = rule 210 c = rule 165 c = rule 154 c = rule 146 c = rule 90 c = rule 82 c = rule 26 c = rule 18 c = rule 22 c = rule 118 c = rule 145 c = Figure 2. ( continued ) H. Zenil
Complex Systems,19
Complex Systems,19 © 2010 Complex Systems Publications, Inc. ule 161 c = rule 129 c = rule 126 c = rule 182 c = rule 131 c = rule 62 c = rule 150 c = rule 105 c = rule 109 c = rule 124 c = rule 193 c = rule 110 c = rule 137 c = rule 73 c = rule 45 c = rule 89 c = rule 75 c = rule 101 c = rule 135 c = rule 30 c = rule 149 c = rule 86 c = Figure 2. ( continued )
3. Compression-Based Clustering3.1 2-Clusters Plot
By finding neighboring clusters of compressed lengths cellular au-tomata can be grouped by their program-size complexity. Treatingpairs of elements as being less similar when their distances are largerusing an Euclidean distance function, a 2-clusters plot was able toseparate cellular automata that clearly fall into Wolfram’s classes 3and 4 from the rest, dividing complex and random-looking cellular au-tomata from trivial and nested ones as shown in Figures 3 through 5.
Figure 3.
Partitioning elementary cellular automata into clusters by compres-sion length.
Compression-Based Investigation of CAs and Other Systems Complex Systems, © 2010 Complex Systems Publications, Inc. lasses 1 and 20 1 2 3 4 5 6 7 8 9 10 11 12 13 1415 16 17 18 19 20 21 22 23 24 25 26 27 28 2931 32 33 34 35 36 37 38 39 40 41 42 43 44 4647 48 49 50 51 52 53 54 55 56 57 58 59 60 6162 63 64 65 66 67 68 69 70 71 72 74 76 77 7879 80 81 82 83 84 85 87 88 90 91 92 93 94 9596 97 98 99 100 102 103 104 105 106 107 108 109 111 112113 114 115 116 117 118 119 120 121 122 123 125 126 127 128129 130 131 132 133 134 136 138 139 140 141 142 143 144 145146 147 148 150 151 152 153 154 155 156 157 158 159 160 161162 163 164 165 166 167 168 169 170 171 172 173 174 175 176177 178 179 180 181 182 183 184 185 186 187 188 189 190 191192 194 195 196 197 198 199 200 201 202 203 204 205 206 207208 209 210 211 212 213 214 215 216 217 218 219 220 221 222223 224 225 226 227 228 229 230 231 232 233 234 235 236 237238 239 240 241 242 243 244 245 246 247 248 249 250 251 252253 254 255 Classes 3 and 430 45 7375 86 89101 110 124135 137 149193 Figure 4.
Elementary cellular automata clusters by compressibility. rule 124 c = rule 193 c = rule 110 c = rule 137 c = rule 73 c = rule 45 c = rule 89 c = rule 75 c = rule 101 c = rule 135 c = rule 30 c = rule 149 c = rule 86 c = Figure 5.
Elementary cellular automata classes 3 and 4.
A second application of the clustering algorithm splits the originalclasses 3 and 4 into clusters linking automata by their qualitative prop-erties as shown in Figure 6.
By following the same technique, we were able to identify 3-colornearest-neighbor cellular automata in classes 3 and 4 as shown in Fig-ure 7. H. Zenil
Complex Systems,19
Complex Systems,19 © 2010 Complex Systems Publications, Inc. H a L H b L Figure 6. (a) Breaking class 3 and 4 clusters. (b) Splitting classes 3 and 4 bynearest compression lengths . rule1282406871585 c = rule2854645248671 c = rule2854645248671 c = rule3723989953493 c = rule3881084868256 c = rule1131934307315 c = rule3771787467781 c = rule2448480479333 c = rule5519825353591 c = rule2805651523468 c = rule5141921105231 c = rule1279861140687 c = rule573831174454 c = rule6514965490414 c = rule4478058683884 c = rule1013995622274 c = rule940433994538 c = rule6300443157836 c = rule3363296031903 c = rule5413082599859 c = rule2198580087674 c = rule1790219388192 c = rule6589500597925 c = rule3272263785434 c = Figure 7.
Compression-Based Investigation of CAs and Other Systems Complex Systems, © 2010 Complex Systems Publications, Inc. .2.2 2-State 3-Color Turing Machines The exploration of Turing machines is considerably more difficult forthree main reasons.
1. The spaces of Turing machines for the shortest states and colors aremuch larger than the shortest spaces of cellular automata.2. Turing machines with nontrivial dynamical properties are very rarecompared to the size of the space defined by the number of states andcolors, and therefore larger samples are necessary.3. Turing machines evolve much more slowly than cellular automata, solonger runtimes are also necessary.
The best compression-based results for identifying nontrivial Tur-ing machines were obtained by applying the technique to the numberof states a Turing machine is able to reach after a certain number ofsteps rather than to the output itself, unlike for cellular automata.The complexity of a Turing machine is deeply determined by the num-ber of states the machine is capable of reaching from an initial config-uration, and by looking at its complexity the technique distinguishedthe nontrivial machines from the most trivial. Figure 8 shows a sam-ple of Turing machines found by applying the compression-basedmethod.
4. Compression-Based Phase Transition Detection
A phase transition can be defined as a discontinuous change in the dy-namical behavior of a system when a parameter associated with thesystem, such as its initial configuration, is varied.It is conventional to distinguish two kinds of phase transitions,often called first- and higher-order. As described in [2], one feature offirst-order transitions is that as soon as the transition is passed, thewhole system always switches completely from one state to the other.However, higher-order transitions are gradual or recurrent. On oneside of the transition a system is typically completely ordered or disor-dered. But when the transition is passed, the system does not changefrom then on to either one or another state. Instead, its order in-creases or decreases more or less gradually or recurrently as the pa-rameter is varied. Typically the presence of order is signaled by thebreaking of some kind of symmetry; for example, two rules exploredin this section (rules 22 and 109) were found to be highly disturbedwith recurrent phase transitions due to a symmetry breaking whenstarting with certain initial configurations. H. Zenil
Complex Systems,19
Complex Systems,19 © 2010 Complex Systems Publications, Inc. ule2646854 rule2546704 rule2319387 rule2168918 rule2052110 rule1610809 rule1509320 rule1338891 rule1282505 rule1153862 rule1098542 rule988475 rule909555 rule843827 rule764447 rule745570 rule743859 rule662254 rule660951 rule616227 rule600046 rule575259 rule571529 rule518008 rule434006 rule326309 rule324869 rule140458 rule99848 rule99266 Figure 8.
Compression-based search for nontrivial 2-state 3-color Turingmachines.
Ideally, one should feed a system with a natural sequence of initialconfigurations of gradually increasing complexity. Doing so assuresthat qualitative changes in the evolution of the system are not at-tributable to discontinuities in its set of initial conditions.The reflected binary code, also known as the “Gros | Gray code” orsimply the “Gray code” (after Louis Gros and Frank Gray), is a bi-nary numeral system where two successive values differ by only onebit. To explore the qualitative behavior of a cellular automaton whenstarting from different initial configurations, the optimal method is tofollow a Gros | Gray encoding enumeration in order to avoid any unde-sirable “jumps” attributable to the system’s having been fed with dis-continuous initial configurations. By following the Gros | Gray code,an optimal numbering scheme was devised so that two consecutive ini-tial conditions differ only by the simplest change (one bit).
Compression-Based Investigation of CAs and Other Systems Complex Systems, © 2010 Complex Systems Publications, Inc. he reflected binary code, also known as the “Gros | Gray code” orsimply the “Gray code” (after Louis Gros and Frank Gray), is a bi-nary numeral system where two successive values differ by only onebit. To explore the qualitative behavior of a cellular automaton whenstarting from different initial configurations, the optimal method is tofollow a Gros | Gray encoding enumeration in order to avoid any unde-sirable “jumps” attributable to the system’s having been fed with dis-continuous initial configurations. By following the Gros | Gray code,an optimal numbering scheme was devised so that two consecutive ini-tial conditions differ only by the simplest change (one bit).
GrosGrayCodeDerivate and
GrosGrayCodeIntegrate implementthe methods described in [22].
GrosGrayCodeDerivate @ n_Integer D : = Prepend @ Mod @ Ò @@ DD + Ò @@ DD , 2 D & ê ü Partition @ Ò , 2, 1 D , Ò @@ DDD & ü IntegerDigits @ n, 2 D GrosGrayCodeIntegrate @ l_List D : = FromDigits @ Mod @ Ò , 2 D & ê ü Accumulate @ l D , 2 D The function
InitialConfiguration implements the optimal num-bering scheme of initial conditions for cellular automata based on theGros | Gray code, minimizing the Damerau | Levenshtein distance.
Figure 9.
First 11 elements of the Gros | Gray code.
GrosGrayCodeIntegrate is the reverse function of
GrosGray Ö CodeDerivate . It retrieves the element number of an element in Gros | Gray’s code, that is, the composition of
GrosGrayCodeDerivate and
GrosGrayCodeIntegrate is the identity function.
Table @ GrosGrayCodeIntegrate @ GrosGrayCodeDerivate @ n DD , n, 0, 10 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 < The Damerau | Levenshtein distance between two vectors u and v gives the number of one-element deletions, insertions, substitutions,and transpositions required to transform u into v . It can be verifiedthat the distance between any two adjacent elements in the Gros | Graycode is always 1. H. Zenil Complex Systems,19 Complex Systems,19 © 2010 Complex Systems Publications, Inc. he Damerau | Levenshtein distance between two vectors u and v gives the number of one-element deletions, insertions, substitutions,and transpositions required to transform u into v . It can be verifiedthat the distance between any two adjacent elements in the Gros | Graycode is always 1. DamerauLevenshteinDistance @ Ò P T , Ò P TD & ê ü Partition @ Table @ GrosGrayCodeDerivate @ n D , n, 0, 10 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 < The simplest, not completely trivial, initial configuration of a cellu-lar automaton is the typical single black cell that can be denoted (asin Mathematica ) by {{1}, 0}, meaning a single black cell (1) on a back-ground of whites (0). Preserving an “empty” background leaves the re-gion that must be varied consisting only of the nonwhite portion ofthe initial configuration. However, when surrounded with zeroes, ini-tial configurations may be the same for cellular automata. For exam-ple, the initial configuration {0, 1, 0} is exactly the same as {1} be-cause the cellular automaton background is already filled with zeroes.Therefore, valid different initial configurations for cellular automatashould always be wrapped in 1s. InitialCondition @ n_Integer D : = If @ n === Last @ Ò D< , Ò D & ü Append @ GrosGrayCodeDerivate @ n D , 1 D Figure 10. Sequence of the first 11 binary initial configurations for cellularautomata based on the Gros | Gray code. InitialConditionNumber is the reverse function for retrieving thenumber of an initial configuration given an initial configuration ac-cording to the numbering scheme devised herein. InitialConditionNumber @ l_List D : = GrosGrayCodeIntegrate @ Most @ l DD For example, the thirty-second initial condition is InitialConditionNumber @ InitialCondition @ DD An interesting example is the elementary cellular automaton rule22, which behaves as a fractal in Wolfram’s class 2 for some segmentof initial configurations, followed by phase transitions of moredisordered evolutions of the type of class 3. Compression-Based Investigation of CAs and Other Systems Complex Systems, © 2010 Complex Systems Publications, Inc. n interesting example is the elementary cellular automaton rule22, which behaves as a fractal in Wolfram’s class 2 for some segmentof initial configurations, followed by phase transitions of moredisordered evolutions of the type of class 3. Two one-dimensional elementary cellular automata that show dis-crete changes in behavior when the properties of their initial condi-tions are continuously changed are shown in Figures 11 and 12.Data points are joined for clarity only. It can be seen that up to theinitial configuration number 20 there are clear spikes at initial configu-rations 8, 14, 17, and 20 indicating four abrupt phase transitions.For clarity, the background of the evolution of rule 109 in Fig-ure ! 12 was cleaned up. Clear phase transitions are detected at initialconfiguration numbers 2, 3, 11, and 13, together with weaker behav-ior changes at initial configuration numbers 15, 16, and 20 that onlyoccur on one side of the cellular automaton and therefore show spikesfiring at half the length.Comparison of the sequences of the compressed lengths of six dif-ferent elementary cellular automata following the initial configurationnumbering scheme up to the first 2 = 128 initial configurations up to150 steps each is shown in Figure 13.The differences between the compressed versions provide informa-tion on the changes in behavior up to a given number of steps of a sys-tem starting from different initial conditions. The normalization di-vided by the number of steps provides the necessary stability to keepthe increase of complexity on account of the increase of size due tolonger runtimes out of the main equation. In other words, the pro-gram-size complexity accumulated due to longer runtimes is sub-tracted in time from the approximated program-size complexity ofthe system itself.The method given can also be used to precompute the initial config-urations of a cellular automaton space conducting the search for inter-esting behaviors and speeding up the study of qualitative dynamicalproperties. For example, interesting initial configurations to look atfor rules 22 and 109 are those detected in Figure 13 showing clearphase transitions.One open question is whether there are first-order phase transi-tions (when following a “natural” initial condition enumeration) inelementary cellular automata. Our method was only capable of detect-ing higher-order phase transitions up to the steps and initial condi-tions explored herein. H. Zenil Complex Systems,19 Complex Systems,19 © 2010 Complex Systems Publications, Inc. ule 22 Evolutioninit 1 init 2 init 3 init 4init 5 init 6 init 7 init 8init 9 init 10 init 11 init 12init 13 init 14 init 15 init 16init 17 init 18 init 19 init 20 ÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊ initconf compressedlength Sequence of Compressed Lengths Figure 11. Rule 22 phase transition. Compression-Based Investigation of CAs and Other Systems Complex Systems, © 2010 Complex Systems Publications, Inc. ule 109 Evolutioninit 1 init 2 init 3 init 4init 5 init 6 init 7 init 8init 9 init 10 init 11 init 12init 13 init 14 init 15 init 16init 17 init 18 init 19 init 20 ÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊ initconf compressedlength Sequence of Compressed Lengths Figure 12. Rule 109 phase transition. H. Zenil Complex Systems,19 Complex Systems,19 © 2010 Complex Systems Publications, Inc. 20 40 60 80 100 120 initconf compressedlength Rule 0 0 20 40 60 80 100 120 initconf compressedlength Rule 2500 20 40 60 80 100 120 initconf compressedlength Rule 22 0 20 40 60 80 100 120 initconf compressedlength Rule 1090 20 40 60 80 100 120 initconf compressedlength Rule 45 0 20 40 60 80 100 120 initconf compressedlength Rule 30 Figure 13. Sequence of compressed lengths for six elementary cellularautomata. 5. Compression-Based Numerical Computation of a Characteristic Exponent A fundamental property of chaotic behavior is the sensitivity to smallchanges in the initial conditions. Lyapunov characteristic exponentsquantify this qualitative behavior by measuring the mean rate of diver-gence of initially neighboring trajectories. A characteristic exponentas a measure usually has the advantage of keeping systems with no sig-nificant phase transitions close to a constant value, while those withsignificant phase transitions are distinguished by a linear growth thatcharacterizes instability in the system. Whether a system has a phasetransition is an undecidable property of systems in general. However,the characteristic exponent is an effective method of calculation, evenwith no prior knowledge of the generating function of the system. Compression-Based Investigation of CAs and Other Systems Complex Systems, © 2010 Complex Systems Publications, Inc. fundamental property of chaotic behavior is the sensitivity to smallchanges in the initial conditions. Lyapunov characteristic exponentsquantify this qualitative behavior by measuring the mean rate of diver-gence of initially neighboring trajectories. A characteristic exponentas a measure usually has the advantage of keeping systems with no sig-nificant phase transitions close to a constant value, while those withsignificant phase transitions are distinguished by a linear growth thatcharacterizes instability in the system. Whether a system has a phasetransition is an undecidable property of systems in general. However,the characteristic exponent is an effective method of calculation, evenwith no prior knowledge of the generating function of the system.The technique described herein consists of comparing the mean ofthe divergence in time of the compressed lengths of the output of a sys-tem running over a sequence of small changes to the initial conditionsover small intervals of time. The procedure yields a sequence of valuesnormalized by the runtime and the derivative of the function that bestfits the sequence. Just as with Wolfram’s method described in [2] forthe calculation of the Lyapunov exponents of a cellular automaton,the divergence in time is measured by the differences in space-time ofthe patterns produced by the system. But unlike the calculation ofLyapunov exponents, this will be done by measuring the distance be-tween the compressed regions of the evolution of a cellular automa-ton when starting from different initial configurations. After normal-ization, we will be able to evaluate a stable characteristic exponentand therefore characterize its degree of sensitivity.We want to examine the relative behavior of region evolutionswhen starting from adjacent initial configurations. Since the systemsfor which we are introducing this method are discrete, the regular pa-rameter separating the initial conditions in a continuous system whencalculating the Lyapunov exponent of a system can be replaced by theimmediate successor of an initial condition following an optimal enu-meration like the one described in Section 4 based on the Gros | Graycode.Let the characteristic exponent c nt be defined as the mean of theabsolute values of the differences of the compressed lengths of the out-puts of the system M running over the initial segment of initial con-ditions i j with j = 1, … , n < following the numbering scheme devisedearlier and running for t steps, as follows: c nt = † C H M t H i LL - C H M t H i LL§ + ! + † C H M t H i n - LL - C H M t H i n LL§ t H n - L The division by t acts as a normalization parameter in order tokeep the runtime of the different values of c nt as independent as possi-ble among the systems. However, as already noted, normalization canalso be achieved by dividing by the “volume” of the region (the space-time diagram) generated by a system (in the case of a one-dimensionalcellular automaton the area, i.e., the number of affected cells ~ usuallythe characteristic cone). The mean of the absolute values can also bereplaced by the maximum of the absolute values in order to maximizethe differences depending on the type of dynamical features beingintensified.Let us define a phase transition sequence as the sequence of charac-teristic exponents for a system M running for longer runtimes asshown in Figure 14. H. Zenil Complex Systems,19 Complex Systems,19 © 2010 Complex Systems Publications, Inc. ä H c nt L rule ‡ ‡ ‡ ‡ ‡ ‡ Figure 14. Phase characteristic exponent sequence s I c nt H M LM . The general rule for t = 200 and n = 40 is that if the characteristicexponent c nt is greater than 1 for large enough values of n and t , then c nt has a phase transition. Otherwise it does not. Table 1 shows thecalculation of the characteristic exponents of some elementary cellularautomata. S c f H S c L8 < + < - + < - + < + < + < + < + < + < + < + < - < - + < + < - Table 1. Compression-Based Investigation of CAs and Other Systems Complex Systems, © 2010 Complex Systems Publications, Inc. .1 Regression Analysis Let S c = S I c nt M for a fixed n and t . The line that better fits the growthof a sequence S c can be found by calculating the linear combinationthat minimizes the sum of the squares of the deviations of the ele-ments. Let f H S c L denote the line that fits the sequence S c by finding theleast-squares as shown in Figure 15. Data points in S c Fitting lines f H S c L Figure 15. The derivatives of a phase transition function are therefore stableindicators of the degree of the qualitative change in behavior of thesystems. The larger the derivative, the larger the significance. Let C de-note the transition coefficient defined as C = f £ H S c L . Table 2 illustratesthe calculated transition coefficients for a few elementary cellular au-tomata rules. After calculating the transition coefficient, we can calculate the first10 most interesting initial conditions for elementary cellular automatawith transition coefficients greater than 1. Those listed in Table 3were calculated up to 600 steps in blocks of 50. H. Zenil Complex Systems,19 Complex Systems,19 © 2010 Complex Systems Publications, Inc. CA Rule H r L C H ECA r L 22 2.5151 2.3109 1.473 1.3133 0.49183 0.3054 0.27110 0.2797 0.2241 0.21147 0.1845 0.0261 - - Table 2. Phase transition coefficient.ECA Rule Initial Configuration Number151 17, 18, 20, 22, 26, 34, 37, 41, 44, 46 < 8, 14, 17, 20, 23, 24, 26, 27, 28, 29 < 10, 12, 15, 16, 21, 24, 28, 30, 43, 45 < 2, 3, 11, 13, 20, 24, 26, 28, 32, 33 < 4, 6, 8, 10, 14, 16, 18, 19, 22, 25 < 10, 16, 20, 23, 24, 26, 28, 29, 30, 32 < Table 3. Elementary cellular automata that are the most sensitive to initialconfigurations. The coefficient C has positive values if the system is sensitive to theinitial configurations. The larger the positive values, the more sensi-tive the system is to initial configurations. C has negative values or isclose to 0 if it is highly homogeneous. Irregular behavior yields nonlin-ear growth, leading to a positive exponent. For elementary cellular automata, it was found that n = 40 and t = 200 were runtime values large enough to detect and distinguishcellular automata having clear phase transitions. It is also the casethat systems showing no quick phase transition have an asymptoticprobability 1 of having a transition at a later time. In other words, asystem with a phase transition either has the transition very early intime or is unlikely to ever have one later, as can be theoreticallypredicted from an algorithmic theoretical argument. (A phase transi-tion is undecidable and can be seen as a reachability problem equiva-lent to the halting problem, hence a system powerful enough to haltwhen reaching a phase transition, as calculated earlier, has an[effective] density zero [23].) The same transition coefficient can alsobe seen as a homogeneity measure. At the right granular level, ran-domness shares informational properties with trivial systems. Like atrivial system, a random system is incapable of transmitting or carry-ing out information. The characteristic exponent relates these twobehaviors in an interesting way since the granularity of a randomsystem for a runtime large enough is close to the dynamical state ofbeing in a stable configuration according to this measure. One can seethat rule 110 is better classified, certainly because it has some struc-ture and is less homogeneous in time, unlike rule 30 and of courserule 1. Rule 30, like rule 1, changes its compressed output from onestep to the other at a lower rate or not at all. While the top of the clas-sification and the gap between them are more significant because theyshow a qualitative change in their evolution, the bottom is classifiedby its lack of changes. In other words, while rules like 22 and 151exhibit more changes when starting from different initial classifica-tions, rules such as 30 and 1 always look alike. Compression-Based Investigation of CAs and Other Systems Complex Systems, © 2010 Complex Systems Publications, Inc. or elementary cellular automata, it was found that n = 40 and t = 200 were runtime values large enough to detect and distinguishcellular automata having clear phase transitions. It is also the casethat systems showing no quick phase transition have an asymptoticprobability 1 of having a transition at a later time. In other words, asystem with a phase transition either has the transition very early intime or is unlikely to ever have one later, as can be theoreticallypredicted from an algorithmic theoretical argument. (A phase transi-tion is undecidable and can be seen as a reachability problem equiva-lent to the halting problem, hence a system powerful enough to haltwhen reaching a phase transition, as calculated earlier, has an[effective] density zero [23].) The same transition coefficient can alsobe seen as a homogeneity measure. At the right granular level, ran-domness shares informational properties with trivial systems. Like atrivial system, a random system is incapable of transmitting or carry-ing out information. The characteristic exponent relates these twobehaviors in an interesting way since the granularity of a randomsystem for a runtime large enough is close to the dynamical state ofbeing in a stable configuration according to this measure. One can seethat rule 110 is better classified, certainly because it has some struc-ture and is less homogeneous in time, unlike rule 30 and of courserule 1. Rule 30, like rule 1, changes its compressed output from onestep to the other at a lower rate or not at all. While the top of the clas-sification and the gap between them are more significant because theyshow a qualitative change in their evolution, the bottom is classifiedby its lack of changes. In other words, while rules like 22 and 151exhibit more changes when starting from different initial classifica-tions, rules such as 30 and 1 always look alike.The clusters formed (a different cluster per row) for a few selectedcellular automata rules starting from random initial conditions areshown in Figure 16.The clusters shown in Figure 16 are clearly classifying this small se-lection of cellular automata by the presence of phase transitions(sudden structures). This is also a measure of homogeneity. rule 22 rule 151rule 109 rule 73rule 133 rule 183 rule 147rule 41 rule 97 rule 54rule 110 rule 45 rule 30 rule 1 Figure 16. Clusters found using the phase transition coefficient over a sampleof 14 rules. H. Zenil Complex Systems,19 Complex Systems,19 © 2010 Complex Systems Publications, Inc. rule 22 C = rule 151 C = rule 109 C = rule 73 C = rule 133 C = rule 124 C = rule 94 C = rule 193 C = rule 120 C = rule 183 C = rule 106 C = rule 54 C = rule 110 C = rule 225 C = rule 97 C = rule 137 C = rule 169 C = rule 41 C = rule 121 C = rule 147 C = rule 182 C = rule 161 C = rule 129 C = rule 218 C = Figure 17. Phase transition coefficient classification (top 24) of elementary cel-lular automata (from random initial configurations). rule 180 C =- rule 38 C =- rule 66 C =- rule 34 C =- rule 20 C =- rule 98 C =- rule 184 C =- rule 226 C =- rule 138 C =- rule 14 C =- rule 84 C =- rule 166 C =- rule 212 C =- rule 134 C =- rule 194 C =- rule 142 C =- rule 6 C =- rule 88 C =- rule 86 C =- rule 74 C =- rule 37 C =- rule 91 C =- Figure 18. Phase transition coefficient classification (bottom 22) of elementarycellular automata (from random initial configurations). Since a random system would be expected to produce a homoge-neous stream with no distinguishable patterns as incapable of carry-ing or transmitting any information, both the simplest and the ran-dom systems were classified together at the end of the figure. Compression-Based Investigation of CAs and Other Systems Complex Systems, © 2010 Complex Systems Publications, Inc. ince a random system would be expected to produce a homoge-neous stream with no distinguishable patterns as incapable of carry-ing or transmitting any information, both the simplest and the ran-dom systems were classified together at the end of the figure. Based on this study, we conjecture that a system will be capable ofuniversal computation if it has a large transition coefficient, at leastlarger than zero, say. The inverse, however, should not hold, becausehaving a large transition coefficient by no means implies that the sys-tem will behave with the freedom required of a universal system if itis to emulate any possible computation (a case in point may be rule22, which, despite having the largest transition coefficient, may not beversatile enough for universal computation). We base this conjectureon two facts: 1. The only known universal elementary cellular automata figure at thetop of this classification, and so do candidates such as rule 54 which fig-ures right next to rule 110.2. Universality seems to imply that a system should be capable of beingcontrolled by the inputs, which our classification suggests those at thebottom are not, as all of them look the same no matter what the input,and may not be capable of carrying information through the system to-ward the output. The conjecture also seems to be in agreement with Wolfram’s claimthat rule 30 (as a class 3 elementary cellular automaton) may be, ac-cording to his Principle of Computational Equivalence (PCE), compu-tationally universal. But it may turn out that it is too difficult (maybeimpossible) to control in order to perform a computation because itbehaves too randomly.It is worth mentioning that this method is capable of capturingmany of the subtle different behaviors a cellular automaton is capableof, which are heuristically captured by Wolfram’s classes. The tech-nique does not, however, disprove Wolfram’s principle of irreducibil-ity [2] because it is an a posteriori method. In other words, it is onlyby running the system that the method is capable of revealing the dy-namical properties. This is no different from a manual inspection inthat regard. However, it is of value that the method presented doesidentify a large range of qualitative properties without user interven-tion that other techniques (a priori techniques), including several ana-lytical ones, generally seem to neglect. 6. Conclusion We were able to clearly distinguish the different classes of behaviorstudied by Wolfram. By calculating the compressed lengths of the out-put of cellular automata using a general ompression algorithm wefound two clearly distinguishable main clusters and, upon closer in-spection, two others with clear gaps in between. That we found twomain large clusters seems to support Wolfram’s Principle of Computa-tional Equivalence (PCE) [2], which suggests that there is no essentialdistinction between the classes of systems showing trivial and nestedbehavior and those showing random and complex behavior. We havealso provided a compression-based framework for phase transition de-tection, and a method to calculate an exponent capable of identifyingand measuring the significance of other dynamical properties, such assensitivity to initial conditions, presence of structures, and homogene-ity in space or regularity in time. We have also formulated a conjec-ture with regard to a possible connection between its transition coeffi-cient and the ability of a system to reach computational universality.As can be seen from the experiments presented in this paper, the com-pression-based approach and the tools that have been proposed arehighly effective for classifying, clustering, and detecting several dynami-cal properties of abstract systems. Moreover, the method does not de-pend on the system and can be applied to any abstract computing de-vice or to data coming from any source whatsoever. It can also beused to calculate prior distributions and make predictions regardingthe future evolution of a system. H. Zenil Complex Systems,19 Complex Systems,19 © 2010 Complex Systems Publications, Inc. e were able to clearly distinguish the different classes of behaviorstudied by Wolfram. By calculating the compressed lengths of the out-put of cellular automata using a general ompression algorithm wefound two clearly distinguishable main clusters and, upon closer in-spection, two others with clear gaps in between. That we found twomain large clusters seems to support Wolfram’s Principle of Computa-tional Equivalence (PCE) [2], which suggests that there is no essentialdistinction between the classes of systems showing trivial and nestedbehavior and those showing random and complex behavior. We havealso provided a compression-based framework for phase transition de-tection, and a method to calculate an exponent capable of identifyingand measuring the significance of other dynamical properties, such assensitivity to initial conditions, presence of structures, and homogene-ity in space or regularity in time. We have also formulated a conjec-ture with regard to a possible connection between its transition coeffi-cient and the ability of a system to reach computational universality.As can be seen from the experiments presented in this paper, the com-pression-based approach and the tools that have been proposed arehighly effective for classifying, clustering, and detecting several dynam-ical properties of abstract systems. Moreover, the method does not de-pend on the system and can be applied to any abstract computing de-vice or to data coming from any source whatsoever. It can also beused to calculate prior distributions and make predictions regardingthe future evolution of a system. Acknowledgments I would like to thank Jean-Paul Delahaye for his valuable suggestions,and Chiara Basile, Matthew Szudzik, Paul-Jean Letourneau, JamieWilliams, and Todd Rowland for their suggestions, stimulating discus-sions, and disseminating some of the ideas of this paper. References [1] J. von Neumann, Theory of Self-Reproducing Automata , Urbana, IL:University of Illinois Press, 1966.[2] S. Wolfram, A New Kind of Science , Champaign, IL: Wolfram Media,Inc., 2002.[3] G. Braga, G. Cattaneo, P. Flocchini, and G. Mauri, “Complex ChaoticBehavior of a Class of Subshift Cellular Automata,” Complex Systems , (4), 1993 pp. 269 | TheoreticalComputer Science , (1 | | ~ Sierpinski Gasket and Partially Sier-pinski Gasket,” Japan Journal of Industrial and Applied Mathematics , (1), 2001 pp. 59 | Compression-Based Investigation of CAs and Other Systems Complex Systems, © 2010 Complex Systems Publications, Inc. 6] M. Hurley, “Attractors in Cellular Automata,” Ergodic Theory and Dy-namical Systems , (1), 1990 pp. 131 | Physica D:Nonlinear Phenomena , (1 | | Theoretical Computer Science , (1 | | Theoretical Com-puter Science , (1 | | Complex Systems , (2) 1988 pp. 177 | TheoreticalComputer Science , (2), 2003 pp. 365 | Theoretical Computer Science , (1 | | Complex Systems , (3), 2007 pp. 295 | IEEE Trans-actions on Information Theory , (4), 2005 pp. 1523 | Algorithmic Information Theory , 4th printing, Cam-bridge: Cambridge University Press, 1992.[16] “Compress” from Wolfram Mathematica Documentation Center ~ A Wolfram Web Resource. http://reference.wolfram.com/mathematica/ref/Compress.html.[17] J.-L. Gailly and M. Adler. “GZIP Documentation and Sources.” Avail-able as gzip-*.tar. ftp://prep.ai.mit.edu/pub/gnu/gzip.[18] D. A. Huffman, “A Method for the Construction of Minimum-Redun-dancy Codes,” Proceedings of the Institute of Radio Engineers , (9),1952 pp. 1098 | IEEE Transactions on Information Theory , (3), 1977pp. 337 | ~ A Wolfram Web Resource.http://demonstrations.wolfram.com/CellularAutomatonCompressibility.[22] J. P. Delahaye, “Voyageurs et baguenaudiers,” Pour La Science , ,1997 pp. 100 | Advances in Applied Mathematics , (3), 2008 pp. 295 | H. Zenil Complex Systems,19