Logical depth for reversible Turing machines with an application to the rate of decrease in logical depth for general Turing machines
aa r X i v : . [ c s . CC ] A ug Logical depth for reversible Turing machines withan application to the rate of decrease in logicaldepth for general Turing machines
Paul M.B. Vit´anyi ∗ August 29, 2019
Abstract
The logical depth of a reversible
Turing machine equals the shortestrunning time of a shortest program for it. This is applied to show that theresult in [1] is valid notwithstanding the error noted in Corrigendum [7].keywords: Logical depth, Kolmogorov complexity, compression
A book on number theory is difficult, or ‘deep.’ The book lists a number ofdifficult theorems of number theory. However, it has very low Kolmogorovcomplexity, since all theorems are derivable from the initial few definitions. Ourestimate of the difficulty, or ‘depth,’ of the book is based on the fact that ittakes a long time to reproduce the book from part of the information in it. Theexistence of a deep book is itself evidence of some long evolution preceding it.The logical depth of a (finite) string is related to complexity with boundedresources and measures the tradeoff between program sizes and running times.Computing a string x from one of its shortest programs may take a very longtime, but computing the same string from a simple “ print(x) ” program oflength about | x | bits takes very little time.Logical depth as defined in [4] for a string comes in two versions: one basedon the compressibility of programs of prefix Turing machines and the other usingthe ratio between algorithmic probabilities with and without time limits. Sinceboth are approximately the same ([5, Theorem 7.7.1] based on [4, Lemma 3]) itis no restriction to use the compressibility version.The used notions of computability, resource-bounded computation time, self-delimiting strings, big-O notation, and Kolmogorov complexity are well-knownand the properties, notations, are treated in [5]. ∗ CWI and University of Amsterdam. Address: CWI, Science Park 123, 1098XG Amster-dam, The Netherlands. Email: [email protected] Preliminaries
All Turing machines in this paper are prefix Turing machines. A prefix Turingmachine is a Turing machine with a one-way read-only program tape, an aux-iliary tape, one or more work tapes and an output tape. All tapes are linearone-way infinite and divided into cells capable of containing one symbol out ofa finite set. Initially the program tape is inscribed with an infinite sequence of0’s and 1’s and the head is scanning the leftmost cell. When the computationterminates the sequence of bits scanned on the input tape is the program . Forevery fixed finite contents of the auxiliary tape the set of programs for such amachine is a prefix code (no program is a proper prefix of another program).Let T , T , . . . be the standard enumeration of prefix Turing machines. A uni-versal prefix Turing machine simulates every prefix Turing machine given itsindex number. We also require it to be optimal which means that the simula-tion program is as short as possible. We choose a reference optimal universalprefix Turing machine and call it U .The prefix Kolmogorov complexity is based on the prefix Turing machinesimilar to the (plain) Kolmogorov complexity based on the (plain) Turing ma-chine. Let x, y be finite binary strings. The prefix Kolmogorov complexity K ( x | y )of x with auxiliary y is defined by K ( x | y ) = min p {| p | : U ( p, y ) = x } . If x is a binary string of length n then K ( x | y ) ≤ n + O (log n ). Restrictingthe computation time resource is indicated by a superscript giving the allowednumber of steps, usually denoted by d . The notation U d ( p, y ) = x means that U ( p, y ) = x within d steps. If the auxiliary string y is the empty string ǫ , then weusually drop it. Similarly, we write U ( p ) for U ( p, ǫ ). The string x ∗ is a shortestprogram for x if U ( x ∗ ) = x and K ( x ) = | x ∗ | . A string x is b - incompressible if | x ∗ | ≥ | x | − b . A Turing machine behaves according to a finite list of rules. These rules deter-mine, from the current state of the finite control and the symbol contained inthe cell under scan, the operation to be performed next and the state to enterat the end of the next operation execution.The device is ( forward ) deterministic . Not every possible combination ofthe first two elements has to be in the set; in this way we permit the device toperform no operation. In this case we say that the device halts . Hence, we candefine a Turing machine by a transition function . Definition . A reversible Turing machine [3, 2] is a Turing machine thatis forward deterministic (any Turing mchine as defined is) but also backwarddeterministic , that is, the transition function has a single-valued inverse. The2etails of the formal definition are intricate [3, 2] and need not concern us here.This definition extends in the obvious manner to multitape Turing machines.In [3] for every 1-tape ordinary Turing machine T a 3-tape reversible Turingmachine T rev is constructed that emulates T in linear time such that with input p the output is T rev ( p ) = ( p, T ( p )). The reversible Turing machine that emulates U is called U rev . Definition . Let x be a string and b a nonnegative integer. The logical depth of x at significance level b , isdepth b ( x ) = min (cid:8) d : p ∈ { , } ∗ ∧ U d ( p ) = x ∧ | p | ≤ K ( p ) + b (cid:9) , the least number of steps to compute x by a b -incompressible program. Theorem . The logical depth of a string x at significance level b ∈ N forreversible Turing machines is equal todepth b ( x ) = min (cid:8) d : p ∈ { , } ∗ ∧ U d rev ( p ) = ( p, x ) ∧ | p | ≤ K ( x ) + b (cid:9) , the least number of steps to compute x by U rev from a program of length at most K ( x ) + b .Proof. Since a reversible Turing machine is backwards deterministic, and anincompressible program cannot be computed from a shorter program, the lengthof an incompressible program for x can only be the length of a shortest programfor x . The logical depth at significance b is then the least number of steps tocompute x by U rev from a program p of length K ( x ) + b . In [1, Section 4 ] it is assumed that, for all x ∈ { , } ∗ , the string x ∗ is the onlyincompressible string such that U ( x ∗ ) = x . That is, logical depth accordingto Theorem 1 is used. However, this assumption is wrong for general Turingmachines in that for many x there may be an incompressible string p with | x | ≥ | p | > | x ∗ | such that U ( p ) = x . The computation of U ( p ) = x may befaster than that of U ( x ∗ ) = x . For example, the function from x ∈ { , } ∗ to the least number of steps in a computation U ( p ) = x for an incompressiblestring p may be computable. The argument in the paper is, however, correctfor the set of reversible Turing machines. These Turing machines are a subsetof the set of all Turing machines [3, 2] and emulate them in linear time. Thisimplies the correctness of [1, Theorem 2] as we shall show. Lemma . Let ψ be defined by ψ ( n ) = max | x | = n min d { d : U d rev ( x ∗ ) = ( x ∗ , x ) } . Then ψ is not computable and grows faster than any computable function. roof. If a function ψ as in the lemma were computable, then for an x of length n we could run U rev emulating U [3] forward for ψ ( n ) steps on all programs oflength n + O (log n ). Among those programs that halt within ψ ( n ) steps, we couldselect the programs p which output ( p, x ). Subsequently, we could select fromthat set a program p of minimum length, say x ∗ . Such a program x ∗ has length K ( x ) since U rev is emulating U . This would imply that K would be computable.But the function K is incomputable [6, 5]: contradiction. Therefore ψ cannotbe computable. Since this holds for every function majoring ψ , the function ψ must grow faster than any computable function. Corollary . The set of reversible Turing machines is a subset of the set of allTuring machines. The emulation of U ( p ) by U rev ( p ) is linear time for all binaryinputs p by [3]. Therefore, replacing in the lemma U rev by U changes ψ ( n ) to φ ( n ) = Ω( ψ ( n )). Hence the lemma holds with ψ replaced by φ and U rev by U . This gives us [1, Lemma 1] and therefore [1, Theorem 2] (the Busy Beaverupper bound is proved as it is in [1]): Theorem . The function f ( n ) = max | x | = n, ≤ b ≤ n { x : depth b ( x ) − depth b +1 ( x ) } grows faster than any computable function but not as fast as the Busy Beaverfunction. References [1] L.F. Antunes, A. Souto, and P.M.B. Vit´anyi, On the Rate of Decrease in LogicalDepth,
Theor. Comput. Sci. , 702(2017), 60–64.[2] H.B. Axelsen and R. Gl¨uck, A simple and efficient universal reversible Turingmachine,
Proc. 5th Int. Conf. Language and Automata Theory and Applications ,Lecture notes in computer science, Vol. 6638, Springer, 2011, 117–128.[3] C.H. Bennett, Logical reversibility of computation,
IBM J. Research and Develop-ment , 17:6(1973), 525–532.[4] C.H. Bennett, Logical depth and physical complexity, pages 227–257 in
The Uni-versal Turing Machine A Half-Century Survey , R. Herken Ed., Oxford UniversityPress, 1988.[5] M. Li and P.M.B. Vit´anyi,
An Introduction to Kolmogorov Complexity and ItsApplications , Springer, New York, 2008.[6] A.N. Kolmogorov, Three approaches to the quantitative definition of information.
Problems Inform. Transmission , 1:1(1965), 1–7.[7] P.M.B. Vit´anyi, Corrigendum to “On the rate of decrease in logical depth” byL.F. Antunes, A. Souto, and P.M.B. Vit´anyi [Theoret. Comput. Sci. 702 (2017)60–64],
Theoret. Comput. Sci. , https://doi.org/10.1016/j.tcs.2018.07.009 (In Press,Corrected Proof)., https://doi.org/10.1016/j.tcs.2018.07.009 (In Press,Corrected Proof).