Paths to Unconventional Computing: Causality in Complexity
aa r X i v : . [ c s . G L ] J un Paths to Unconventional Computing:Causality in Complexity
Hector Zenil ∗ June 28, 2017
Abstract
I describe my path to unconventionality in my exploration of theo-retical and applied aspects of computation towards revealing the algo-rithmic and reprogrammable properties and capabilities of the world,in particular related to applications of algorithmic complexity in re-shaping molecular biology and tackling the challenges of causality inscience.
Keywords— unconventional computing; causality; finite algorithmicrandomness; programmability; algorithmicity; computational biology; algo-rithmic nature; Kolmogorov-Chaitin complexity ∗ Extended version of an invited contribution to a special issue of the journal of Progressin Biophysics & Molecular Biology (Elsevier). spin ), indistinguishable, just as cells on a Tur-ing machine tape are indistinguishable except in terms of the symbol theymay contain (equivalent to reading the spin direction). Moreover, classicalmechanics prescribes full determinism, and the necessity of quantum me-chanics to require or produce true indeterministic randomness is contestedby different interpretations (e.g. Everett’s multiverse).It was not long before I started finding models of computation morepowerful, at least on paper, than Turing-equivalent models. I was, however,able to find very few such models that could be serious contenders, among themany that looked exotic and thus highly unlikely or unrealizable, with someof their proponents ranging from the unconventional to the eccentric. Somepropose trivial modifications of the classical model or go so far as to embracemysticism–a strong belief that their model captures a property of the realworld that no other model can [31]. All models beyond the Turing modelinvolve not only infinite numbers but infinite non-computable numbers [17],that is, numbers whose digits one cannot calculate by mechanistic, Turing-type means.The notion that nature computes should not seem unconventional, but it3pparently does. Indeed, the very fact that we have taken some material fromthe earth’s surface and reshaped it into working electronic digital computerstells us not only that the universe can compute but that it does, and that itcomputes exactly as we instruct it to. The question is whether the universecarries on computations of a similar kind even without our intervention andwith something other than the kind of artificial electronic computers that webuild.Every model in physics is computational and lives in the computationaluniverse [16] (the universe of all possible programs), as we are able to codesuch models in a digital computer, plug in some data as initial conditions andrun them to generate a set of possible outcomes for real physical phenomenawith staggering predictive precision. We do this with ever increasing accu-racy, whether calculating planetary trajectories or forecasting the weather,and such a convergence between simulation and simulated cannot but sug-gest the possibility that the real phenomenon performs the same or a verysimilar computation as the one carried out by the digital computer. We maybe pushed to believe that the inadequacy of such models in predicting longterm weather patterns with absolute precision reflects the limitations of themodels themselves, or else the fundamental unsoundness of computable mod-els as such, but we know that the most salient limitation here is inadequatedata–both in quantitative and qualitative terms–that we can plug into themodel, as we are always limited in our ability to collect data from open envi-ronments, in respect to which we can never attain enough precision withouthaving to simulate the whole universe. But we do know that the more datawe introduce into our models the better they perform.4omputational or not, if anything was clear and not in the least uncon-ventional, it was that the universe was algorithmic in a fundamental way, orat least that in light of successful scientific practice it seemed highly likely tobe so. While this is a highly conventional point of view, many may view sucha claim as being almost as strong as its mechanistic counterpart because, ul-timately, in order to shift the question from computation to algorithms, onemust decide what kind of computer runs the algorithms. However, after myexploration of non-computable models of computation [17], I began my ex-ploration of what I call the algorithmic nature of the world. I wanted tostudy how random the world could be, and what the theory of mathematicalrandomness could tell us about the universe and the kinds of data that couldbe plugged into models, their initial conditions, and the noise attendant uponthe plugging in of the data. This promised to give me a better understandingof whether it was the nature of the data on which a computational model ranthat made it weaker and more limited, or whether it was only the quantityof the data that determined the limitations of computable models. And soI launched out on my strong unconventional path by introducing alterna-tives for measuring and applying algorithmic complexity, leading to excitingdeployments of highly abstract theory in highly applied areas. The basicunits of study in the theory of algorithmic complexity are sequences, andnothing epitomizes a natural sequence better than the DNA. Because mostinformation is in the connections among genes and not the genes themselves,I defined a concept of the graph algorithmic complexity of both labelled andunlabelled graphs [30, 27]. However, this could not have been done if I hadproceeded by using lossless compression as others have [1, 15]. Instead I used5 novel approach based upon algorithmic probability [8, 13] that allowed meto circumvent some of the most serious limitations of compression algorithms.What I first did was to use the theory of algorithmic probability [8,13], a theory that elegantly reconnects computation to classical probabil-ity in a proper way through a theorem called the algorithmic coding the-orem , which for its part establishes that the most frequent outcome of acausal/deterministic system will also be the most simple in the sense of al-gorithmic complexity, that is, the shortest computer program that producessuch an output. So we ran trillions of very small computer programs [2, 12]to build an empirical probability distribution that approximates what in thetheory of algorithmic probability is called the universal distribution [8], usedas the prior in all sorts of Bayesian calculations that require the framing ofhypotheses about generating models or mechanisms that produce an object.Take a binary string of a million 1s. The theory tells us that the most likelymechanistic generator of 1s is not print(1 million 1s) but a recursive programthat iterates over print(1) 1 million times, that is, in this case, the shortestpossible program. For this sequence the most likely and highest causal gen-erating program can easily be guessed, but for most sequences this is verydifficult. However, the more difficult the more random, thus giving us someclues as to the causal content of the object.When I started these approaches I was often discouraged, as I still some-times am, and tempted to turn away from algorithmic complexity because‘its uncomputabilty ’ (the reviewers said), that there is no algorithm to runa computation in every case and expect the result of the algorithmic com-plexity of an object, because the computation may or may not end. But if6e were scared away by uncomputability we would never code anything buttrivial software. We have, however, progressed significantly in our ability towrite all sorts of computer programs, in particular, incredibly powerful lay-ers of sophisticated computer programs of which clearly we cannot control orfully understand as to e.g. know if they will ever get stuck. However, despitefear of the unknowns that uncomputability entails (e.g. the nonexistenceof the perfect antivirus or the nonexistence of error-free software) softwareengineering does not only prevails but has changed the world. We have beentoo fearless of uncomputability when it comes to measuring actual algorith-mic randomness. The truth is that not only are many of these challengespartially circumventable (due to e.g. their semi-computability character)but estimations have been proven to be sound and correspond to theoreticalexpectations [11].Once I had the tools, methods and an unbreakable will, I wanted to knowto what extent the world really ran on a universal mechanical computer, andI came up with measures of algorithmicity [25, 18]: how much the outcome ofa process resembles the way in which outcomes would be sorted according tothe universal distribution, and of programmability [23, 21, 20, 22]: how muchsuch a process can be reprogrammed at will. The more reprogrammable,the more causal, given that a random object cannot be reprogrammed inany practical way other than by changing every possible bit. My colleagues,leading biological and cognitive labs, and I are implementing methods inwhich algorithmic information theory plays a central role, allowing us tosteer and manipulate systems such as cells other than following traditionaltrial-and-error approaches. And we have looked at how the empirical uni-7ersal distribution that we calculated could be plugged back into all sortsof challenges [30, 27, 19, 6, 4, 5, 3, 7] to help with the problem of data col-lection to generate a sound computational framework for model generation.Few, if any researchers could have foreseen that something as theoretical anduncomputable would eventually be put to these kinds of uses.When one takes seriously, however, the dictum that the world is algo-rithmic, one can begin to see seemingly unrelated natural phenomena fromsuch a perspective and devise software engineering approaches to areas suchas the study of human diseases [29]. Cancer is a most interesting case likeother cellular and genetics diseases. Seen as a cell’s computer program gonewrong, the question becomes how to reprogram cancer cells to return themto serving their original purposes, or how to hack their code in order to makethem die (without tampering with the code of non-cancer cells) [29].It turns out that the world (including the natural world) may be morereprogrammable than we expected. By following a Bayesian approach toproving universal computation [28, 9], we recently showed that class bound-aries that seemingly determined the behaviour of computer programs couldeasily be transcended, and that even the simplest of programs could be re-programmed to simulate computer programs of arbitrary complexity. Thisunconventional approach to universality, thinking outside the box, showsthat, after the impossibility results of Turing, Chaitin or Martin-L¨of, proofcan no longer be at the core of some parts of theoretical computer science, andthat a scientific approach based on experimental mathematics is required toanswer certain questions, such as how pervasive Turing-universality is in thecomputational universe. We need more daring, unconventional thinkers who8ould stop fearing uncomputability and carry out this fruitful programme.Niche disciplines that may seem unrelated work hand in hand in theapproaches that I have introduced. Who would have thought that algorith-mic randomness could be connected and make a tangible contribution tomolecular biology in spite of its uncomputable character, and that algorith-mic information theory would equip cognitive scientists with much needednew psychometric tools with which to test and validate long-standing [6]suspicions about the inner workings of the human mind. While unconven-tional computing is about challenging some computational limits, the limitsI challenge are those imposed by axiomatic frameworks and their quest foronly mathematical proofs of ever-increasing abstraction. I rather take proofsfrom mature mathematical areas to seek for their meaning in disparate areasof science, thereby establishing unconventional bridges across conventionalfields.
References [1] Rudi Cilibrasi and Paul MB Vit´anyi. Clustering by compression.
IEEETransactions on Information theory , 51(4):1523–1545, 2005.[2] Jean-Paul Delahaye and Hector Zenil. Numerical evaluation of algorith-mic complexity for short strings: A glance into the innermost structureof randomness.
Applied Mathematics and Computation , 219(1):63–77,2012.[3] Nicolas Gauvrit, Henrik Singmann, Fernando Soler-Toscano, and Hec-tor Zenil. Algorithmic complexity for psychology: a user-friendly imple-9entation of the coding theorem method.
Behavior research methods ,48(1):314–329, 2016.[4] Nicolas Gauvrit, Fernando Soler-Toscano, and Hector Zenil. Naturalscene statistics mediate the perception of image complexity.
Visual Cog-nition , 22(8):1084–1091, 2014.[5] Nicolas Gauvrit, Hector Zenil, Jean-Paul Delahaye, and Fernando Soler-Toscano. Algorithmic complexity for short binary strings applied topsychology: a primer.
Behavior research methods , 46(3):732–744, 2014.[6] Nicolas Gauvrit, Hector Zenil, Fernando Soler-Toscano, Jean-Paul De-lahaye, and Peter Brugger. Human behavioral complexity peaks at age25.
PLOS Computational Biology , 13(4):e1005408, 2017.[7] Nicolas Gauvrit, Hector Zenil, and Jesper Tegn´er. The information-theoretic and algorithmic approach to human, animal and artificial cog-nition. In G. Dodig-Crnkovic and R. Giovagnoli, editors,
Representationand reality: Humans, animals and machines . Springer, 2017.[8] Leonid A Levin. Laws of information conservation (nongrowth) andaspects of the foundation of probability theory.
Problemy PeredachiInformatsii , 10(3):30–35, 1974.[9] J¨urgen Riedel and Hector Zenil. Cross-boundary behavioural repro-grammability reveals evidence of pervasive turing universality. arXivpreprint arXiv:1510.01671 , 2015.[10] Claude E. Shannon. A mathematical theory of communication parts iand ii.
Bell System Technical Journal , 27:379–423 and 623–656, 1948.1011] Fernando Soler-Toscano, Hector Zenil, Jean-Paul Delahaye, and NicolasGauvrit. Correspondence and independence of numerical evaluations ofalgorithmic information measures.
Computability , 2(2):125–140, 2013.[12] Fernando Soler-Toscano, Hector Zenil, Jean-Paul Delahaye, and NicolasGauvrit. Calculating kolmogorov complexity from the output frequencydistributions of small turing machines.
PloS one , 9(5):e96223, 2014.[13] Ray J Solomonoff. A formal theory of inductive inference. part i.
Infor-mation and control , 7(1):1–22, 1964.[14] Jesper Tegn´er, Hector Zenil, Narsis A Kiani, Gordon Ball, and DavidGomez-Cabrero. A perspective on bridging scales and design of modelsusing low-dimensional manifolds and data-driven model inference.
Phil.Trans. R. Soc. A , 374(2080):20160144, 2016.[15] Paul MB Vit´anyi and Ming Li. Minimum description length induction,bayesianism, and kolmogorov complexity.
IEEE Transactions on infor-mation theory , 46(2):446–464, 2000.[16] Stephen Wolfram.
A new kind of science . Wolfram Media, Champaign,IL. 2002.[17] H´ector Zenil. Calcul et hyper-calcul, 2006. (m´emoire) fulfilling the dis-sertation requirement for the degree of Masters (Logic) under the direc-tion of Jacques Dubucs at the University of Paris 1 Panth´eon-Sorbonne.[18] Hector Zenil. The world is either algorithmic or mostly random, 2011.winning 3rd. Place in the International Essay Context of the FQXi, 2011.1119] Hector Zenil. From computer runtimes to the length of proofs: Withan algorithmic probabilistic application to waiting times in automatictheorem proving. In M.J. Dinneen, B. Khousainov, and A. Nies, editors,
Computation, Physics and Beyond , pages 223–240. Springer, 2012.[20] Hector Zenil. A behavioural foundation for natural computing and aprogrammability test. In G. Dodig-Crnkovic and R. Giovagnoli, editors,
Computing Nature , pages 87–113. Springer, 2013.[21] Hector Zenil. Programmability: A turing test approach to computation.In L. De Mol and G. Primiero, editors,
Turing in Context, KoninklijkeVlaamse Academie van Belgi¨e voor Wetenschappen en Kunsten (Bel-gian Academy of Sciences and Arts), Contactforum . Belgian Academyof Sciences and Arts, 2014.[22] Hector Zenil. What is nature-like computation? a behavioural approachand a notion of programmability.
Philosophy & Technology , 27(3):399–421, 2014.[23] Hector Zenil. Algorithmicity and programmability in natural computingwith the game of life as in silico case study.
Journal of Experimental &Theoretical Artificial Intelligence , 27(1):109–121, 2015.[24] Hector Zenil. Algorithmic data analytics, small data matters and cor-relation versus causation. In M. Ott, W. Pietsch, and J. Wernecke,editors,
Berechenbarkeit der Welt? Philosophie und Wissenschaft imZeitalter von Big Data (Computability of the World? Philosophy andScience in the Age of Big Data) . Springer Verlag, 2017.1225] Hector Zenil and Jean-Paul Delahaye. On the algorithmic nature of theworld. In G. Dodig-Crnkovic and M. Burgin (eds), editors,
Informationand Computation . World Scientific Publishing Company, 2010.[26] Hector Zenil, Narsis Kiani, and Tegn´er Jesper. Low algorithmic com-plexity entropy-deceiving graphs.
Phys Rev E. (accepted) , 2017.[27] Hector Zenil, Narsis A Kiani, and Jesper Tegn´er. Methods of informationtheory and algorithmic complexity for network biology. In
Seminars incell & developmental biology , volume 51, pages 32–43. Elsevier, 2016.[28] Hector Zenil and J¨urgen Riedel. Asymptotic intrinsic universality andreprogrammability by behavioural emulation. In A. Adamatzky, edi-tor,
Advances in Unconventional Computation , pages 205–220. SpringerVerlag, 2016.[29] Hector Zenil, Angelika Schmidt, and Jesper Tegn´er. Causality, infor-mation and biological computation: An algorithmic software approachto life, disease and the immune system. In S.I. Walker, P.C.W Davies,and G. Ellis, editors,
Information and Causality: From Matter to Life ,pages 244–249. Cambridge University Press, 2017.[30] Hector Zenil, Fernando Soler-Toscano, Kamaludin Dingle, and Ard ALouis. Correlation of automorphism group size and topological prop-erties with program-size complexity evaluations of graphs and com-plex networks.