Simplicity via Provability for Universal Prefix-free Turing Machines
aa r X i v : . [ c s . I T ] J un T. Neary, D. Woods, A.K. Seda and N. Murphy (Eds.):The Complexity of Simple Programs 2008.EPTCS 1, 2009, pp. 16–21, doi:10.4204/EPTCS.1.2 c (cid:13)
C. S. Calude
Simplicity via Provability for Universal Prefix-free TuringMachines
Cristian S. Calude
Department of Computer ScienceUniversity of AucklandPrivate Bag 92019, Auckland, New Zealand
Universality is one of the most important ideas in computability theory. There are various criteria ofsimplicity for universal Turing machines. Probably the most popular one is to count the number ofstates/symbols. This criterion is more complex than it may appear at a first glance. In this note wereview recent results in Algorithmic Information Theory and propose three new criteria of simplicityfor universal prefix-free Turing machines. These criteria refer to the possibility of proving variousnatural properties of such a machine (its universality, for example) in a formal theory, PA or ZFC. Inall cases some, but not all, machines are simple.
Roughly speaking, a universal Turing machine is a Turing machine capable of simulating any otherTuring machine. In Turing’s words:
It can be shown that a single special machine of that type can be made to do the work of all.It could in fact be made to work as a model of any other machine. The special machine maybe called the universal machine.
The first universal Turing machine was constructed by Turing [26, 27]. Shannon [23] studied theproblem of finding the smallest possible universal Turing machine and showed that two symbols weresufficient, if enough states can be used. He also proved that “it is possible to exchange symbols for statesand vice versa (within certain limits) without much change in the product.” Notable universal Turingmachines include the machines constructed by Minsky (7-state 4-symbol) [15], Rogozhin (4-state 6-symbol) [22], Neary–Woods (5-state 5-symbol) [17]. Herken’s book [11] celebrates the first 50 years ofuniversality.Weak forms of universality were proved by Watanabe (4-state 5-symbol) [28], Cook [9] for Wol-fram’s 2-state 5-symbol machine [29], Neary–Woods [16], and Smith [24] for Wolfram’s 2-state 3-symbol machine. A prefix-free Turing machine, shortly, machine, is a Turing machine whose domain is a prefix-free set.In what follows we will be concerned only with machines working on the binary alphabet { , } . A universal machine U is a machine such that for every other machine C there exists a constant c (whichdepends upon U and C ) such that for every program x there exists a program x ′ with | x ′ | ≤ | x | + c such The critique by Pratt [20, 21], the response in [19] and the forthcoming paper by Margenstern [14] show the subtlety of thenotion of universality. .S.Calude 17that U ( x ′ ) = C ( x ) . Universal machines can be effectively constructed. For example, given a computableenumeration of all machines ( C i ) i , the machine U defined by U ( i x ) = C i ( x ) is universal. The domainsof universal machines have interesting computational and coding properties, cf. [7, 6]. By L A we denote the first-order language of arithmetic whose non-logical symbols consist of the constantsymbols 0 and 1, the binary relation symbol < and two binary function symbols + (addition) and · (multiplication). Peano arithmetic, PA, is the first-order theory [12] given by a set of 15 axioms definingdiscretely ordered rings, together with induction axioms for each formula j ( x , y , . . . , y n ) in L A : ∀ y ( j ( , y ) ∧ ∀ x ( j ( x , y ) → j ( x + , y )) → ∀ x ( j ( x , y )) . By PA ⊢ q we mean “there is a proof in PA for q ”.PA is a first-order theory of arithmetic powerful enough to prove many important results in com-putability and complexity theories. For example, there are total computable functions for which PAcannot prove their totality, but PA can prove the totality of every primitive recursive function (and alsoof Ackermann total computable, non-primitive recursive function), see [12].Zermelo–Fraenkel set theory with the axiom of choice, ZFC, is the standard one-sorted first-ordertheory of sets; it is considered the most common foundation of mathematics. In ZFC set membership isa primitive relation. By ZFC ⊢ q we mean “there is a proof in ZFC for q ”.Our metatheory is ZFC. We fix a (relative) interpretation of PA in ZFC according to which eachformula of L A has a translation into a formula of ZFC. By abuse of language we shall use the phrase“sentence of arithmetic” to mean a sentence (a formula with no free variables) of ZFC that is the transla-tion of some formula of PA. The set of bit strings is denoted by S ∗ . If s is a bit string then | s | denotes the length of s . All realswill be in the unit interval. A computably enumerable (shortly, c.e.) real number a is given by anincreasing computable sequence of rationals converging to a . Equivalently, a c.e. real a is the limit ofan increasing primitive recursive sequence of rationals. We will blur the distinction between the real a and the infinite base-two expansion of a , i.e. the infinite bit sequence a a · · · a n · · · ( a n ∈ { , } ) suchthat a = . a a · · · a n · · · . By a ( n ) we denote the string of length n , a a · · · a n .One of the major problems in algorithmic information theory is to define and study (algorithmically)random reals. To this aim one can use the prefix-complexity or constructive measure theory; remarkably,the class of “random reals” obtained with different approaches remains the same.In what follows we will adopt the complexity-theoretic approach. Fix a universal machine U . Theprefix-complexity induced by U is the function H U : S ∗ → N ( N is the set of natural numbers) definedby the formula: H U ( x ) = min {| p | : U ( p ) = x } . One can prove that this complexity is optimal up to anadditive constant in the class of all prefix-complexities { H C : C is a machine } .A c.e. real a is Chaitin-random if there exists a constant c such that for all n ≥ H U ( a ( n )) ≥ n − c .The above definition is invariant with respect to U . Every Chaitin-random real is non-computable, but See more in [1]. The above universal machine, called prefix-universal because universality is obtained by adjunction, isquite particular. There are universal machines which are not prefix-universal. U (Chaitin’s Omega number): W U = (cid:229) U ( x ) < ¥ −| x | . Each Omega number encodes information about halting programs in the most compact way. Forexample, the answers to the following 2 n + − U ( x ) halt?”, for all programs | x | ≤ n , isencoded in the first n digits of W U —an exponential rate of compression. Is this important? For example,to solve the Riemann hypothesis one needs to calculate the first 7,780 bits of a natural Omega number[3]. The following result characterises the class of c.e. Chaitin-random reals: Theorem 1 [8, 5, 13]
The set of c.e. Chaitin-random reals coincides with the set of all halting probabil-ities of all universal machines.
C.e. random reals have been intensively studied in recent years, with many results summarised in[1, 10, 18].
We start with the simple question: Can PA certify the universality of a universal machine?A universal machine U is called simple for PA if PA ⊢ “ U is universal”, i.e. PA can prove that auniversal U , given by its full description, is indeed universal. For illustration, the results in this sectionwill include full proofs.As one might expect, there exist universal machines simple for PA: Theorem 2 [4]
One can effectively construct a universal machine which is simple for PA . Proof.
The set of all machines PA can prove to be prefix-free is c.e., so if ( C i ) i is a computable enumera-tion of provably prefix-free machines, then the machine U defined by U ( i x ) = C i ( x ) has the propertyspecified in the theorem: PA ⊢ “ U is universal”. ✷ However, not all universal machines are simple:
Theorem 3 [4]
One can effectively construct a universal machine which is not simple for PA . Proof.
Let ( f i ) i be a c.e. enumeration of all primitive recursive functions f i : N → S ∗ and ( C i ) i a c.e. enu-meration of all prefix-free machines. Fix a universal prefix-free machine U and consider the computablefunction g : N → N defined by: C g ( i ) ( x ) = (cid:26) U ( x ) , if for some j > , { f i ( ) , f i ( ) , . . . , f i ( j ) } > | x | , ¥ , otherwise . For every i , C g ( i ) is a prefix-free universal machine iff f i ( N ) is infinite (if f i ( N ) is finite, then so is C g ( i ) ). Since the set of all indices of primitive recursive functions with infinite range is not c.e. it followsthat PA cannot prove that for some i , C g ( i ) is universal. ✷ Both results above are true for plain universal machines too. The above proofs work for plain uni-versal machines, but a simpler proof can be given for the negative result. U ( x ) < ¥ means “ U is defined on x ”. .S.Calude 19 Assume that the binary expansion of W U is 0 . w w · · · . For each digit w i we can consider two arithmeticsentences in ZFC, “ w i = w i = Theorem 4 [8]
Assume that
ZFC is arithmetically sound (that is, each sentence of arithmetic proved by
ZFC is true). Then, for every universal machine U ,
ZFC can determine the value of only finitely manybits of the binary expansion of W U , and one can calculate a bound on the number of bits of W U which ZFC can determine. Actually, we can precisely describe the“moment” ZFC fails to prove any bit of W U : Theorem 5 [2]
Assume that
ZFC is arithmetically sound. Let i ≥ and consider the c.e. random real a = . a . . . a i − a i a i + . . . , where a = . . . = a i − = , a i = . Then, we can effectively construct a universal machine U (depending upon
ZFC and a ) such that PA proves the universality of U , ZFC can determine at most i initial bits of W U and a = W U . In other words, the moment the first 0 appears (and this is always the case because a is random) ZFCcannot prove anything about the values of the remaining bits.By taking a < / Theorem 6 [25]
One can effectively construct a universal machine U such that
ZFC (if arithmeticallysound) cannot determine any bit of W U . We say that a universal machine is n–simple for
ZFC if ZFC can prove at most n digits of the binaryexpansion of W U . In view of Theorem 5, for every n ≥ n–simple for ZFC. By Theorem 6 there exists a universal machine which is not 1–simple for ZFC.
We first express Chaitin randomness in PA. A c.e. real a is provably Chaitin-random if there exists auniversal machine simple for PA and a constant c such that PA ⊢ “ ∀ n ( H U ( a ( n )) ≥ n − c ) ”.In this context it is natural to ask the question: Which universal machines U “reveal” to PA that W U is Chaitin-random? Theorem 7 [4]
The halting probability of a universal machine simple for PA is provably Chaitin-random .In fact, Theorem 1 can be proved in PA: Theorem 8 [4]
The set of c.e. provably Chaitin-random reals coincides with the set of all halting prob-abilities of all universal machines simple for PA . Based on Theorem 7 we define another (seemingly more general) notion of randomness in PA. A c.e.real is provably-random (in PA) if there is a universal machine simple for PA and PA ⊢ “ W U = a ”. This means that ZFC can prove only finitely many sentences of the form “ w i = w i =
1” and one can calculate a natural N such that no sentence of the above type with i ≥ N can be proved in ZFC. Theorem 6 was obtained before Theorem 5.
Theorem 9 [4]
A c.e. real is provably-random iff it is provably Chaitin-random.
In contrast with the case of finite random strings where ZFC (hence PA) cannot prove the randomnessof more than finitely many strings, for c.e. reals we have:
Theorem 10 [4]
Every c.e. random real is provably-random.
We say that a universal machine U is PA– simple for randomness if PA ⊢ “ W U is random.” In view ofthe Theorem 10 we get: Corollary 11
For every c.e. random real a there exists a PA –simple for randomness universal machineU such that a = W U . However,
Theorem 12
There exists a universal machine which is not PA –simple for randomness. We have used some recent results in Algorithmic Information Theory to introduce three new criteria ofsimplicity for universal machines based on their “openness” in revealing information to a formal system,PA or ZFC. The type of encoding is essential for these criteria. This point of view might be useful inother contexts, specifically in automatic theorem proving. It would be interesting to “actually construct”the universal machines discussed in this paper.
Acknowledgement
I thank D. Woods whose invitation to CSP08 stimulated these thoughts and H. Zenil who helped mewith recent references. I am indebted to the anonymous referees for their comments which substantiallyimproved the presentation.
References [1] C. S. Calude.
Information and Randomness. An Algorithmic Perspective , 2nd Edition, Revised and Extended,Springer Verlag, Berlin, 2002.[2] C. S. Calude. Chaitin W numbers, Solovay machines and incompleteness, Theoret. Comput. Sci.
284 (2002),269–277.[3] C. S. Calude, Elena Calude, M. J. Dinneen. A new measure of the difficulty of problems,
Journal for Multiple-Valued Logic and Soft Computing
12 (2006), 285–307.[4] C. S. Calude, N. J. Hay. Every Computably Enumerable Random Real Is Provably Computably EnumerableRandom,
CDMTCS Research Report W num-bers, in: M. Morvan, C. Meinel, D. Krob (eds.), Proceedings of the 15th Symposium on Theoretical Aspectsof Computer Science (Paris) , Springer–Verlag, Berlin, 1998, 596–606. Full paper in
Theoret. Comput. Sci.
255 (2001), 125–149.[6] C. S. Calude, A. Nies, L. Staiger, F. Stephan. Universal recursively enumerable sets of strings, in M. Ito, M.Toyama (eds.).
Developments in Language Theory (DLT’08) , Lectures Notes in Comput. Sci. 5257, Springer-Verlag, Berlin, 2008, 170–182. .S.Calude 21 [7] C. S. Calude, L. Staiger. On universal computably enumerable prefix codes,
Mathematical Structures inComputer Science
19 (2009), 45–57.[8] G. J. Chaitin. A theory of program size formally identical to information theory,
J. Assoc. Comput. Mach.
Complex Systems
Algorithmic Randomness and Complexity , Springer, Heidelberg, 2008.[11] R. Herken.
The Universal Turing Machine: A Half-Century Survey , Oxford University Press, Oxford, 1992.[12] R. Kaye.
Models of Peano Arithmetic , Oxford Press, Oxford, 1991.[13] A. Kuˇcera, T. A. Slaman. Randomness and recursive enumerability,
SIAM J. Comput.
31, 1 (2001), 199-211.[14] M. Margenstern. Turing machines with two letters and two states,
Complex Systems , to appear.[15] M. Minsky. Size and structure of universal Turing machines using Tag systems, in
Recursive Function Theory,Proc. Symp. Pure Mathematics , AMS, Providence RI, 5, 1962, 229–238.[16] D. Woods, T. Neary. Small semi-weakly universal Turing machines,
Fundamenta Informaticae , 91(1):179-195, 2009.[17] T. Neary, D. Woods. Four small universal Turing machines,
Fundamenta Informaticae , 91(1):123-144, 2009.[18] A. Nies.
Computability and Randomness . Oxford University Press, 2009.[19] NKS Forum, http://forum.wolframscience.com/showthread.php?s=&threadid=1472 .[20] V. Pratt. Simple Turing machines, universality, encodings, etc., http://cs.nyu.edu/pipermail/fom/2007-October/012156.html .[21] V. Pratt. Definition of universal Turing machine, http://cs.nyu.edu/pipermail/fom/2007-October/012148.html .[22] Y. Rogozhin. A universal Turing machine with 22 states and 2 symbols,
Romanian Journal of InformationScience and Technology
Automata Studies, Princeton , PrincetonUniversity Press, NJ, 1956, 157–165.[24] A. Smith. Wolframs 2,3 Turing machine is universal,
Complex Systems , to appear.[25] R. M. Solovay. A version of W for which ZFC can not predict a single bit, in C. S. Calude, G. P˘aun (eds.).
Finite Versus Infinite. Contributions to an Eternal Dilemma , Springer-Verlag, London, 2000, 323–334.[26] A. Turing. On computable numbers, with an application to the Entscheidungsproblem,
Proceedings of theLondon Mathematical Society
42 (2) (1936), 230–265.[27] A. Turing. On computable numbers, with an application to the Entscheidungsproblem: A correc-tion,
Proceedings of the London Mathematical Society
2, 43 (1937), 544–546.[28] S. Watanabe. 4-symbol 5-state universal Turing machine,
Information Processing Society of Japan Magazine
13 (9) (1972), 588–592.[29] S. Wolfram.
A New Kind of Science , Wolfram Research, 2002, 706–714.[30] Wolfram 2,3 Turing Machine, http://demonstrations.wolfram.com/TheWolfram23TuringMachine/http://demonstrations.wolfram.com/TheWolfram23TuringMachine/