aa r X i v : . [ c s . CC ] J u l Fundamentals of Computing
Leonid A. Levin:
Abstract
These are notes for the course CS-172 I first taught in the Fall 1986 at UC Berkeley and subsequentlyat Boston University. The goal was to introduce the undergraduates to basic concepts of Theory ofComputation and to provoke their interest in further study. Model-dependent effects were systematicallyignored. Concrete computational problems were considered only as illustrations of general principles.The notes are skeletal: they do have (terse) proofs, but exercises, references, intuitive comments andexamples are missing or inadequate. The notes can be used by an instructor designing a course or bystudents who either know the material and want to refresh the memory or are exceptionally bright andhave access to an instructor for questions. Each subsection takes about a week of the course. Versions ofthese notes appeared in [Levin 91].
Acknowledgments.
I am grateful to the University of California at Berkeley, its MacKey Professorship fundand Manuel Blum who made possible for me to teach this course. The opportunity to attend lectures of M. Blumand Richard Karp and many ideas of my colleagues at BU and MIT were very beneficial for my lectures. I am alsograteful to the California Institute of Technology for a semester with light teaching load in a stimulating environmentenabling me to rewrite the students’ notes. NSF grants
Contents
Copyright c (cid:13) Fundamentals of Computing Leonid A. Levin
Sections 1,2 study deterministic computations. Non-deterministic aspects of computations (inputs, interac-tion, errors, randomization, etc.) are crucial and challenging in advanced theory and practice. Defining themas an extension of deterministic computations is simple. The latter, however, while simpler conceptually,require elaborate models for definition. These models may be sophisticated if we need a precise measure-ment of all required resources. However, if we only need to define what is computable and get a very roughmagnitude of the needed resources, all reasonable models turn out equivalent, even to the simplest ones. Wewill pay significant attention to this surprising and important fact. The simplest models are most useful forproving negative results and the strongest ones for positive results.We start with terminology common to all models, gradually making it more specific to those we actuallystudy. We represent computations as graphs: the edges reflect various relations between nodes ( events ).Nodes, edges have attributes: labels, states, colors, parameters, etc. (affecting the computation or its analysis).
Causal edges run from each event to all events essential for its occurrence or attributes. They form a directedacyclic graph (though cycles may be added artificially to mark the external input parts of the computation).We will study only synchronous computations. Their nodes have a time parameter. It reflects logicalsteps, not necessarily a precise value of any physical clock. Causal edges only span short (typically, ≤ parent . Pointer edges connectthe parent of each event to all its other possible causes and reflect connections that allow simultaneousevents to interact and have a joint effect. Pointers with the same source have different labels. The (labeled)subgraph of events/edges at a given time is an instant memory configuration of the model.Each non-terminal configuration has active nodes/edges around which it may change. The models withonly a small active area at any step of the computation are sequential . Others are called parallel . Complexity.
The following measures of computing resources of a machine A on input x will be used throughout: Time : The greatest depth D A ( x ) of causal chains is the number of computation steps. The volume V A ( x ) is the combined number of active edges during all steps. Time T A ( x ) is used (depending on the context) aseither depth or volume, which are close for sequential models. Note that time complexity is robust only upto a constant factor: a machine can be modified into a new one with a larger alphabet of labels, representingseveral locations in one. It would produce identical results in a fraction of time and space (provided that thetime limits suffice for transforming the input and output into the other alphabet). Space : S A ( x ) or S A ( x ) of a synchronous computation is the greatest (over time) size of its configurations.Sometimes excluded are nodes/edges unchanged since the input. Growth Rates (typically expressed as functions of bit length n = k x, y k of input/output x / y ): O, Ω: f ( n ) = O ( g ( n )) ⇐⇒ g ( n ) = Ω( f ( n )) ⇐⇒ sup n f ( n ) g ( n ) < ∞ . o, ω : f ( n ) = o ( g ( n )) ⇐⇒ g ( n ) = ω ( f ( n )) ⇐⇒ lim n →∞ f ( n ) g ( n ) = 0.Θ : f ( n ) = Θ( g ( n )) ⇐⇒ ( f ( n ) = O ( g ( n )) and g ( n ) = O ( f ( n ))).Here are a few examples of frequently appearing growth rates: negligible (log n ) O (1) ; moderate n Θ(1) (called polynomial or P, like in P-time); infeasible: 2 n Ω(1) , also n ! = ( n/e ) n p π (2 n +1 /
3) + ε/n , ε ∈ [0 , . The reason for ruling out exponential (and neglecting logarithmic) rates is that the visible Universe istoo small to accommodate exponents. Its radius is about 46.5 giga-light-years ∼ Plank units. A systemof ≫ R . atoms packed in R Plank Units radius collapses rapidly, be it Universe-sized or a neutron star. Sothe number of atoms is < ≪ ≪ This is a customary but somewhat misleading notation. The clear notations would be like f ( n ) ∈ O ( g ( n )) A rougher estimate follows by computing ln n ! = ( n + .
5) ln n − n + O (1) using that | P ni =2 g ( i ) − R n + . . g ( t ) d t | ≤ v , where v is the total variation of g ′ /
8. So v < /
12 for monotone g ′ ( t ) = ln ′ ( t ) = 1 /t and the O (1) is 1 . − ln 1 .
5) + ε , ε ∈ [0 , . eonid A. Levin Fundamentals of Computing Rigid computations have another node parameter: location or cell . Combined with time, it designates theevent uniquely. Locations have structure or proximity edges between them. They (or their short chains)indicate all neighbors of a node to which pointers may be directed. Cellular Automata (CA).
CA are a parallel rigid model. Its sequential restriction is the
Turing Machine (TM) . The configurationof CA is a (possibly multi-dimensional) grid with a finite, independent of the grid size, alphabet of states tolabel the events. The states include, among other values, pointers to the grid neighbors. At each step of thecomputation, the state of each cell can change as prescribed by a transition function (also called program)applied to the previous states of the cell and its pointed-to neighbors. The initial state of the cells is theinput for the CA. All subsequent states are determined by the transition function.An example of a possible application of CA is a VLSI (very large scale integration) chip represented asa grid of cells connected by wires (chains of cells) of different lengths. The propagation of signals along thewires is simulated by changing the state of the wire cells step by step. The clock interval can be set to thetime the signals propagate through the longest wire. This way the delays affect the simulation implicitly.
An example: the Game of Life (GL).
GL is a plane grid of cells, each holds a 1-bit state (dead/alive)and pointers to the 8 adjacent cells. A cell remains dead or alive if the number i of its live neighbors is 2.It becomes (or stays) alive if i =3. In all other cases it dies (of overcrowding or loneliness) or stays dead.A simulation of a machine M by M is a correspondence between memory configurations of M and M which is preserved during the computation (may be with some time dilation). Such constructions showthat the computation of M on any input x can be performed by M as well. GL can simulate any CA (seea sketch of an ingenious proof in the last section of [Berlekamp, Conway, Guy 82]) in this formal sense:We fix space and time periods a, b . Cells ( i, j ) of GL are mapped to cell ( ⌊ i/a ⌋ , ⌊ j/a ⌋ ) of CA M (com-pressing a × a blocks). We represent cell states of M by states of a × a blocks of GL . This correspondenceis preserved after any number t steps of M and bt steps of GL regardless of the starting configuration. Turing Machines (TMs).
TM is a minimal CA . Its configuration – tape – is an infinite to the right chain of cells.Each state of a cell has a pointer to one of the cell’s two adjacent neighbors. No adjacent cells can bothpoint away from each other. Only the two cells pointing at each other are active , i.e. can change state.The cell that just turned its pointer is the TM’s moving head working on the tape symbol - its target.The input is an array of non-blank cells (only one is rightward) followed by blanks at the right.Another type of CA represents a TM A with several non-communicating heads. At most O (1) heads fitin a cell. They can vanish, split, or merge only in the first cell (which, thus, controls the number of activecells). The input x makes an unchangeable “ink” part of each cell’s state. The rest of the cell’s state is in“pencil” and can be changed by A . The computation halts when all heads drop off. The output A ( x ) isthe pencil part of the tape’s state. This model has convenient theoretical features. E.g. with linear (in T )number ( k p k T ) of state changes (volume) one can solve the Bounded Halting Problem H ( p, x, T ): findout whether the machine with a program p stops on an input x within volume T of computation (see 2.3). Problem:
Find a method to transform any given multi-head TM A into another one B such that the valueof the output of B ( x ) (as a binary integer) and the volumes of computation of A ( x ) and of B ( x ) are allequal within a constant factor (for all inputs x ). Hint: B -cells may have a field to simulate A and maintain(in other fields) two binary counters h (with Θ(1) density of heads) for the number of heads of A and v for A ’s volume. Their least significant digits are at the leftmost cell. h adds its most significant digit to thesame position in v at each step of A . To move the carry 1 on v a head is borrowed from h . These 1-headsmove right in v till they empty their carry into its 0 digit. Then empty 0-heads move back to h in a separatefield/track, possibly first continuing right to find a free slot in this return track. (The heads area in v extendsto k -th cell only by dropping the carry there, with frequency O (2 − k ). Then it shrinks to O (1) in O ( k ) stepssince heads enter it slower than they move away.) Borrowed or returned heads make low or high head-densityareas in h which shift left until absorbed at the leftmost cell. Fundamentals of Computing Leonid A. Levin
The memory configuration of a
Pointer Machine (PM) , called pointer graph , is a finite directed labeledmultigraph. One node R is marked as root and has directed paths to all nodes. Nodes can see and changethe configuration of their out-neighborhood of constant (2 suffices) depth. Edges ( pointers ) are labeled with colors from a finite alphabet common to all graphs handled by a given program. The pointers coming outof a node must have different colors (which bounds the outdegree). Some colors are designated as working and not used in inputs/outputs. One of them is called active , as also are pointers carrying it and nodesseeing them. Active pointers must have inverses, form a tree to the root, and can be dropped only in leaves.All active nodes each step execute an identical program . At its first pulling stage, node A acquirescopies of all pointers of its children using “composite” colors: e.g., for a two-pointer path ( A, B, C ) colored x, y , the new pointer (
A, C ) is colored xy , or an existing z -colored pointer ( A, C ) is recolored { z, xy } . A alsospawns a new node with pointers to and from it. Next, A transforms the colors of its set of pointers, dropsthe pointers left with composite colors, and vanishes if no pointers are left. Nodes with no path from the rootare forever invisible and considered dropped. The computation is initiated by inserting an active loop-edgeinto the root. When no active pointers remain, the graph, with all working colors dropped, is the output. Problem:
Design a PM transforming the input graph into the same one with two extra pointers fromeach node: to its parent in a BFS spanning tree and to the root. Hint: Nodes with no path to the root cannever be activated. but can be copied with pointers, copies connected to the root, the original input removed.PM can be parallel, PPM [Barzdin’, Kalnin’s 74] or sequential, SPM . SPM differ in that only pointersto the root, their sources, and nodes that have pointers with inverses to these sources can be active.A Kolmogorov or Kolmogorov-Uspenskii
Machine (KM) [Kolmogorov, Uspenskii 58], is a specialcase of Pointer Machine [Schoenhage 80] with the restriction that all pointers have inverses. This implies thebounded in/out-degree of the graph which we further assume to be constant.
Fixed Connection
Machine (FCM) is a variant of the PKM with the restriction that pointers oncecreated cannot be removed, only re-colored. So when the memory limits are reached, the pointer structurefreezes, and the computation can be continued only by changing the colors of the pointers.PPM is the most powerful model we consider: it can simulate the others in the same space/time. E.g.,cellular automata make a simple special case of a PPM which restricts the Pointer Graph to be a grid.
Problem.
Design a machine of each model (TM, CA, KM, PPM) which determines if an input string x has a form ww , w ∈ { a, b } ∗ . Analyze time (depth) and space. KM/PPM takes input x in the form of colorsof edges in a chain of nodes, with root linked to both ends. The PPM nodes also have pointers to the root.Below are hints for TM,SPM,CA. The space is O ( k x k ) in all three cases. Turing and Pointer Machines.
TM first finds the middle of ww by capitalizing the letters at both endsone by one. Then it compares letter by letter the two halves, lowering their case. The complexity is: T ( x ) = O ( k x k ). SPM acts similarly, except that the root keeps and updates the pointers to the borders betweenthe upper and lower case substrings. This allows constant time access to these borders. So, T ( x )= O ( k x k ). Cellular Automata.
The computation starts with the leftmost cell sending right two signals. Reachingthe end the first signal turns back. The second signal propagates three times slower, so they meet in themiddle of ww and disappear. While alive, the second signal copies the input field i of each cell into a specialfield c . The c symbols will try to move right whenever the next cell’s c field is blank. So the chain of thesesymbols alternating with blanks will start moving right from the middle of ww . Upon reaching the end theywill push the blanks out and pack themselves back into a copy of the left half of ww shifted right. When a c symbol does not have a blank at the right to move to, it compares itself with the i field of the same cell. Ifthey differ, a signal is generated which halts all activity and rejects x . If all comparisons are successful, thelast c generates the accepting signal. The depth is: T ( x ) = O ( k x k ). eonid A. Levin Fundamentals of Computing We have considered several models of computation. We will see now how the simplest of them – TuringMachine – can simulate all others: these powerful machines can compute no more functions than TM.
Church-Turing Thesis is a generalization of this conclusion: TMs can compute every function computablein any thinkable physical model of computation. This is not a math theorem because the notion of modelis not formally specified. But the long history of studying ways to design real and ideal computing devicesmakes it very convincing. Moreover, this Thesis has a stronger
Polynomial Time version which boundsthe volume of computation required by that TM simulation by a polynomial of the volume used by the othermodels. Both forms of the Thesis play a significant role in foundations of Computer Science.
PKM Simulation of PPM.
For convenience, we assume all PPM nodes have pointers to root. PPMconfiguration is represented in PKM with extra colors l, r, u used in a u -colored binary tree added to eachnode X so that all (unlimited in number) PPM pointers to X are reconnected to its leaves, and inverses,colored l, r , added to all pointers. The number of pointers increases at most 4 times. To simulate PPM, X gets a binary name formed by the l, r colors on its path through the root tree, and broadcasts it downits own tree. For pulling stage X extends its tree to double depth and merges (with combined colors) itsown pointers to nodes with identical names. Then X re-colors its pointers as PPM program requires andrebalances its tree. This simulation of a PPM step takes polylogarithmic parallel time. TM Simulation of PKM.
We assume the PKM keeps a constant degree and a roughly balanced roottree (to yield short node names as described above). TM tape reflects its configuration as the list of allpointers sorted by source name, then by color. The TM’s transition table reflects the PKM program. Tosimulate PKM’s pulling stage TM creates a copy of each pointer and sorts copies by their sinks. Now eachpointer, located at source, has its copy near its sink. So both components of 2-pointer paths are nearby: thespecial double-colored pointers can be created and moved to their sources by resorting on the source names.The re-coloring stage is straightforward, as all relevant pointers having the same source are located together.Once the root has no active pointers, the Turing machine stops and its tape represents the PKM output. If aPPM computes a function f ( x ) in t ( x ) steps, using s ( x ) nodes, the simulating TM uses space S = O ( s log s ),( O (log s ) bits for each of O ( s ) pointers) and time T = O ( S ) t , as TM sorting takes quadratic time. Squaring matters !
TM cannot outperform Bubble Sort. Is its quadratic overhead a big deal? In a short time allsilicon gates on your PC run, say, X =10 ∼ . clock cycles combined. Silicon parameters double almost annually.Decades may bring micron-thin things that can sail sunlight in space in clouds of great computing and physical (lightbeam) power. Centuries may turn them into a Dyson Sphere enveloping the solar system. Still, the power of suchan ultimate computer is limited by the number of photons the Sun emits per second: Y ∼ . = X . Giga-years mayturn much of the known universe into a computer, but its might is still limited by its total entropy 2 . = Y . Faster PPM Simulations.
Parallel Bubble-Sort on CA or Merge-Sort on sequential FCM take nearlylinear time. Parallel FCM can do much better [Ofman 65]. It represents and updates pointer graphs as theabove TM. All steps are straightforward to do locally in parallel polylog time except sorting of pointers. Weneed to create a fixed connection sorting network. Sophisticated networks sort arbitrary arrays of n integersin O (log n ) parallel steps. We need only a simpler polylog method. Merge-Sort splits an array of two or moreentries in two halves and sorts each recursively. Batcher-Merge combines two sorted lists in O (log n ) steps. Batcher Merge. A bitonic cycle is the combination of two sorted arrays (one may be shorter),connected by max-to-max and min-to-min entries. Entries in a contiguous half ( high-half ) of the cycleare ≥ than all entries in the other ( low ) half. Each half (with its ends connected) forms a bitonic cycle itself.A flip of an array entry is one with the highest address bit flipped. Its shift has the highest bit of itsaddress cycle-shifted to the end. Linking nodes in a 2 k -nodes array to their flips and shifts forms a ShuffleExchange graph. We merge-sort two sorted arrays given as a bitonic cycle on such a graph as follows.Comparing each entry with its flip (half-a-cycle away), and switching if wrongly ordered, fits the high andlow halves into respectively the first and last halves of the array by shifting the dislocated segment of each(thus rotating each cycle). This repeats for each half recursively (decrementing k via graph’s shift edges). Fundamentals of Computing Leonid A. Levin
The first computers were hardware-programmable. To change the function computed, one had to reconnectthe wires or even build a new computer. John von Neumann suggested using Turing’s Universal Algorithm.The function computed can be then specified by just giving its description (program) as part of the inputrather than by changing the hardware. This was a radical idea, since in the classical mathematics universalfunctions do not exist (as we will see in Sec. 2.2).Let R be the class of all TM-computable functions: total (defined for all inputs) and partial (which maydiverge). Surprisingly, there is a universal function u in R . For any Turing Machine M that computes f ∈ R in time T and space S , u uses a program m of length c listing the commands and initial head state of M .Then u ( mx ) simulates M ( x ) in time c T and space S + c . It operates in cycles, each simulating one step of M ( x ). After i steps of M ( x ), let s i be the head’s state, l i be the left from it part of the tape; r i be the rest ofthe tape. After i cycles u ( mx ) has the tape configuration t i = l i ms i r i and looks up m to find the commandcorresponding to the state s i and the first symbol of r i . It modifies t i accordingly. When M ( x ) halts, u ( mx )erases the (penciled) ms i from the tape and halts too. Universal Multi-head TM works similarly but canalso determine in time O ( t ( x )) whether it halts in t steps (given x, t ( x ) and an appropriate program). Problem.
Design a universal multi-head TM with a constant factor overhead.Hint: When heads split or merge in the first cell, the room u needs for their programscreates sparse or dense content regions that propagate right (sparse faster).We now describe in detail a simpler but slower universal [Ikeno 58] TM U . It simulates any other TM M that uses only 0 , M lacks the blanksymbol that usually marks the end of input. Thus input needs to be given in some prefixless form, e.g. witha padding that encodes input length l = k x k as a binary string preceded by a string of 2 k l k zeros. In the cellscarrying this padding, two counters are initiated that monitor the distance to both ends of the used partof M ’s tape (initially the input). M ’s head moving on the tape pulls these counters along and keeps themupdated. When the right end of the used tape is reached, any subsequent characters are treated as blanks. U has 11 states + 6 symbols; its transition table is at the right.It shows the states and tape digits only when changed, exceptthat the prime is always shown. The head is on the tape: lowercase states look left, upper case – right. The external choice, halt,etc. commands are special states for M ; for U they are shownas A/B or =. U works in cycles, simulating one transition of M each. The tape is infinite to the right (the leftward head in theleftmost cell halts). It consist of 0 / M ’s tape. The other segmentsdescribe one transition each: a command ( s, b ) → ( s ′ , b ′ , d ) for M to change state s to s ′ , tape bit b to b ′ and turn left or right. 1 0 * 1’ 0’ *’A f f e0B F F e1f,F b* a* F cD d’ – e’E = – ’ ’ ’ e’d ’ ’ ’ ’ Db ’ ’ ’ a’ Da ’ ’ ’ b’ F E’c ’ ’ = F E’e ’ ’ ’ B A A/BThe transition segments are sorted in order of ( s, b ) and never change, except for priming. Each transitionis represented as ∗ Sdb , where b is the bit to write, d the direction R =0 /L =1 to turn. S points to the nextstate represented as 1 k , if it is k segments to the left, or 0 k (if to the right). Each cycle starts with U ’s headin state F or f , located at the site of M ’s head. Primed are the digits of S in the prior command and alldigits to their left. An example of the configuration: ∗ ′ ′ ′ ′ ′ ′ ∗ ′ ′ ′ ′ ′ ∗ ∗ . . . ∗
00 head 00 . . .U first reads the bit of an M ’s cell changing the state from F or f to a/b , puts a * there, moves left tothe primed state segment S , finds from it the command segment and moves there. It does this by repeatedlypriming nearest unprimed * and 1s of S (or unpriming 0s) while alternating the states c/F or d/D . When S is exhausted, the target segment k S k + b stars away is reached. Then U reads (changing state from e to A or B ) the rightmost symbol b ′ of the command, copies it at the * in the M area, goes back, reads the nextsymbol d , returns to the just overwritten (and first unprimed) cell of M area and turns left or right. As CA, M and U have in each cell three standard bits: present and previous pointer directions and a “content” bitto store M’s symbol. In addition U needs just one “trit” of its own! See its simulator at eonid A. Levin Fundamentals of Computing Universal and Complete Functions.
Notations: Let us choose a special mark and after its k -th occurrence, break any string x into Prefix k ( x ) andSuffix k ( x ). Let f + ( x ) be f (Prefix k ( x ) x ) and f − ( x ) be f (Suffix k ( x )). We say u k - simulates f iff for some p =Prefix k ( p ) and all s , u ( ps ) = f ( s ). The prefix can be intuitively viewed as a program which simulatingfunction u applies to the suffix (input). We also consider a symmetric variant of relation “ k -simulate” whichmakes some proofs easier. Namely, u k - intersects f iff u ( ps ) = f ( ps ) for some prefix k p and all s . E.g.,length preserving functions can intersect but cannot simulate one another.We call universal for a class F , any u which k -simulates all functions in F for a fixed k . When F contains f − , f + for each f ∈ F , universality is equivalent to (or implies, if only f + ∈ F ) completeness : uk -intersects all f ∈ F . Indeed, u k -simulates f iff it k -intersects f − ; u k -intersects f if it k -simulates f + . Problem:
Describe explicitly a function, complete for the class of all linear (e.g., 5 x or 23 x ) functions.A negation of a (partial or total) function f is the total predicate ¬ f which yields 1 iff f ( x )=0 andyields 0 otherwise. Obviously, no closed under negation class of functions contains a complete one. So, thereis no universal function in the class of all (computable or not) predicates. This is the well known CantorTheorem that the set of all sets of strings (as well as the sets of all functions, reals etc.) is not countable. Goedel’s Theorem.
There is no complete function among the total computable ones, as this class is closed under negation.So the universal in R function u (and u = ( u mod 2)) has no total computable extensions.Formal proof systems are computable functions A ( P ) which check if P is an acceptable proof and outputthe proven statement. ⊢ s means s = A ( P ) for some P . A is rich iff it allows computable translations s x of statements “ u ( x ) = 0,” provable whenever true, and refutable ( ⊢ ¬ s x ), whenever u ( x ) = 1. A is consistent iff at most one of any such pair s x , ¬ s x is provable, and complete iff at least one of themalways (even when u ( x ) diverges) is. Rich consistent and complete formal systems cannot exist, since theywould provide an obvious total extension u A of u (by exhaustive search for P to prove or refute s x ). Thisis the famous Goedel’s Theorem – one of the shocking surprises of the 20th century science. (Here A is anyextension of the formal Peano Arithmetic; we skip the details of its formalization and proof of richness.) Recursive Functions.
Another byproduct is that the Halting (of u ( x )) Problem would yield a totalextension of u and, thus, is not computable. This is the source of many other uncomputability results.Another source is an elegant Fixed Point
Theorem by S. Kleene: any total computable transformation A of programs (prefixes) maps some program into an equivalent one. Indeed, the complete/universal u ( ps )intersects computable u ( A ( p ) s ). This implies, e.g., Rice theorem: the only computable invariant (i.e. thesame on programs computing the same functions) property of programs is constant ( exercise ).Computable (partial and total) functions are also called recursive (due to an alternative definition).Their ranges (and, equivalently, domains) are called (recursively) enumerable or r.e. sets. An r.e. set withan r.e. complement is called recursive (as is its yes/no characteristic function) or decidable . A function isrecursive iff its graph is r.e. An r.e. graph of a total function is recursive. Each infinite r.e. set is the rangeof an injective total recursive function (“enumerating” it, hence the name r.e.).We can reduce membership problem of a set A to the one of a set B by finding a recursive function f s.t. x ∈ A ⇐⇒ f ( x ) ∈ B . Then A is called m- (or many-to-1- ) reducible to B . A more complex Turing reduction is given by an algorithm which, starting from input x , interacts with B by generating strings s and receiving answers to s ∈ ? B questions. Eventually it stops and tells if x ∈ A . R.e. sets (like HaltingProblem) to which all r.e. sets can be m-reduced are called r.e.-complete. One can show a set r.e.-complete(and, thus, undecidable) by reducing the Halting Problem to it. So Ju.Matijasevich proved r.e.-completenessof Diophantine Equations Problem: given a multivariate polynomial of degree 4 with integer coefficients, findif it has integer roots. The above (and related) concepts and facts are broadly used in Theory of Algorithmsand should be learned from any standard text, e.g., [Rogers 67]. A closer look at this proof reveals another famous Goedel theorem: Consistency C of A (expressible in A as divergence ofthe search for contradictions) is itself an example of unprovable ¬ s x . Indeed, u intersects 1 − u A for some prefix a . C impliesthat u A extends u and, thus, u ( a ) , u A ( a ) both diverge. So, C ⇒ ¬ s a . This proof can be formalized in Peano Arithmetic, thus ⊢ C ⇒ ⊢ ¬ s a . But ⊢ ¬ s a implies u A ( a ) converges, so ⊢ C contradicts C : Consistency of A is provable in A if and only if false ! Fundamentals of Computing Leonid A. Levin
The t -restriction u t of u aborts and outputs 1 if u ( x ) does not halt within t ( x ) steps, i.e. u t computesthe t -Bounded Halting Problem ( t -BHP) . It remains complete for the closed under negation class offunctions computable in o ( t ( x )) steps. ( O ( k p k ) overhead is absorbed by o (1) and padding p .) So, u t is not inthe class, i.e. cannot be computed in time o ( t ( x )) [Tseitin 56]. (And neither can be any function agreeing with t -BHP on a dense (i.e. having strings with each prefix) subset.) E.g. 2 k x k -BHP requires exponential time.However for some trivial input programs the BHT can obviously be answered by a fast algorithm. Thefollowing theorem provides another function P f ( x ) (which can be made a predicate) for which there is onlya finite number of such trivial inputs. We state the theorem for the volume of computation of Multi-HeadTuring Machine. It can be reformulated in terms of time of Pointer Machine and space (or, with smalleraccuracy, time) of regular Turing Machine. Definition:
A function f ( x ) is constructible if it can be computed in volume V ( x ) = O ( f ( x )).Here are two examples: 2 k x k is constructible, as V ( x ) = O ( k x k log k x k ) ≪ k x k .Yet, 2 k x k + h ( x ), where h ( x ) is 0 or 1, depending on whether U ( x ) halts within 3 k x k steps, is not. Compression Theorem [Rabin 59].
For any constructible function f , there exists a function P f suchthat for all functions t , the following two statements are equivalent:1. There exists an algorithm A such that A ( x ) computes P f ( x ) in volume t ( x ) for all inputs x .2. t is constructible and f ( x ) = O ( t ( x )). Proof.
Let t -bounded Kolmogorov Complexity K t ( i | x ) of i given x be the length of the shortest pro-gram p for the Universal Multi-Head Turing Machine transforming x into i with < t volume of computation.Let P f ( x ) be the smallest i , with 2 K t ( i | x ) > log( f ( x ) | t ) for all t . P f is computed in volume f by generatingall i of low complexity, sorting them and taking the first missing. It satisfies the Theorem, since computing i = P f ( x ) faster would violate the complexity bound defining it. (Some extra efforts can make P boolean.) Speed-up Theorem [Blum 67].
There exists a total computable predicate P such that for any algorithmcomputing P ( x ) in volume t ( x ), there exists another algorithm doing it in volume O (log t ( x )).Though stated here for exponential speed-up, this theorem remains true with log replaced by any computableunbounded monotone function. In other words, there is no even nearly optimal algorithm to compute P . The general case.
So, the complexity of some predicates P cannot be characterized by a single con-structible function f , as in Compression Theorem. However, the Compression Theorem remains true (withharder proof) if the requirement that f is constructible is dropped (replaced with being computable). In this form it is general enough so that every computable predicate (or function) P satisfies the statementof the theorem with an appropriate computable function f . There is no contradiction with Blum’s Speed-up,since the complexity f (not constructible itself) cannot be reached. See a review in [Seiferas, Meyer 95]. The proof stands if constructibility of f is weakened to being semi-constructible , i.e. one with an algorithm A ( n, x ) runningin volume O ( n ) and such that A ( n, x )= f ( x ) if n>f ( x ). The sets of programs t whose volumes (where finite) satisfy either(1) or (2) of the Theorem (for computable P, f ) are in Σ (i.e. defined with 2 quantifiers). Both generate monotone classes ofconstructible functions closed under min( t , t ) /
2. Then any such class is shown to be the Ω( f ) for some semi-constructible f . eonid A. Levin Fundamentals of Computing In this section we consider a more interesting provably intractable problem: playing games with full informa-tion, two players and zero sum. We will see that even for some simple games there cannot be a much moreefficient algorithm than exhaustive search through all possible configurations.The rules of an n -player game G are set by families f, v of information and value functions anda transition rule r . Each player i ∈ I at each step participates in transforming a configuration (gameposition) x ∈ C into the new configuration r ( x, m ) , m : I → M by choosing a move m i = m ( i ) based onlyon his knowledge f i ( x ) of x . The game proceeds until a terminal configurations t ∈ T ⊂ C is reached. Then v i ( t ) is the loss or gain of the i -th player. Our games will have zero sum P v i ( t ) = 0 and full information: f i ( x ) = x , r ( x, m ) = r ′ ( x, m a ( x ) ), where a ( x ) points to the active player. We consider binary, two-players,no-draw games, taking C ⊂ Z , M ⊂ Z , I = {± } , a ( x ) = sign ( x ), v i ( t ) = a ( t ) i , and | r ( x, m ) | < | x | .An example of such games is chess. Examples of games without full information are card games, whereonly a part f i ( x ) (player’s own hand) of the position x is known. Each player may have a strategy providinga move for each position. A strategy S is winning if it guarantees victory whatever the opponent does, evenif he knows S . We can extend v on T to V on all positions with a winning strategy for one side so that a ( x ) V ( x ) = sup m { a ( x ) V ( r ( x, m )) } . (sup {} taken as − Evaluating or solving a game, means computing V . This ability is close to the ability to find a goodmove in a modified game. Indeed, modify a game G into G ′ by adding a preliminary stage to it.At this stage the player A offers a starting position for G and her opponent B chooses which side to play.Then A may either start playing G or decrement the counter of unused positions and offer another one.Obviously, B wins if he can determine the winning side of every position. If he cannot while A can, A wins.Also, any game can be modified into one with two moves: M ⊂ { , } by breaking a string move into severalbit-moves. (A position of the new game consists of a position x of the old one and a prefix y of a move.The active player keeps adding bits to y until m is complete and the next position generated by r ( x, m ).)Evaluating such games is obviously sufficient for choosing the right move. Theorem.
Each position of any full information game has a winning strategy for one side. (This theorem [Neumann, Morgenstern 44] fails for games with partial information: either playermay lose if his strategy is known to the adversary. E.g.: 1. Blackjack (21); 2. Each player picks a bit;their equality determines the winner.) The game can be solved by playing all strategies against each other.There are 2 n positions of length n , (2 n ) n = 2 n × n strategies and 2 n × n +1 pairs of them. For a 5-bit gamethat is 2 . The proof of this Theorem gives a much faster (but still exponential time!) strategy. Proof.
Make the graph of all ≤k x k -bit positions and moves; Set V = 0; reset V = v on T .Repeat until idle: If V ( x ) = 0, set V ( x ) = a ( x ) sup m { a ( x ) V ( r ( x, m )) } .The procedure stops with empty V − (0) since | r ( x, m ) | < | x | in our games keep decreasing.Games may be categorized by the difficulty to compute r . We will consider only r computable inlinear space O ( k x k ). Then, the 2 k x k possible moves can be computed in exponential time, say 2 k x k .The algorithm tries each move in each step. Thus, its total running time is 2 k x k +1 : extremely slow(2 for a 13-byte game) but still much faster than the previous (double exponential) algorithm. Problem: the Match Game.
Consider 3 boxes with 3 matches each: ! ! ! ! ! ! ! ! ! .The players alternate turns taking any positive number of matches from a single box. One cannot leave thetable empty. Use the above algorithm to evaluate all positions and list the evaluations after each its cycle.
Problem:
Modify the chess game by giving one side the right to make (if it chooses to) an extramove out of turn during the first 10 moves. Prove that this side have a non-loosing strategy.0
Fundamentals of Computing Leonid A. Levin
A simple example of a full information game is
Linear Chess , played on a finite linear board. Each piecehas a 1-byte type, including loyalty to one of two sides: W (weak) or S (shy), gender
M/F and a 6-bit rank .All cells of the board are filled and all W’s are always on the left of all S’s. Changes occur only at the active border where W and S meet (and fight). The winner of a fight is determined by the following Gender Rules:1. If S and W are of the same sex, W (being weaker) loses.2. If S and W are of different sexes, S gets confused and loses.The party of a winning piece A replaces the loser’s piece B by its own piece C. The choice of C isrestricted by the table of rules listing all allowed triples (ABC). We will see that this game cannot be solvedin a subexponential time. We first prove that (see [Chandra, Kozen, Stockmeyer 81]) for an artificial game.Then we reduce this
Halting Game to Linear Chess showing that any fast algorithm to solve Linear Chess,could be used to solve Halting Game, thus requiring exponential time. For Exp-Time Completeness of regular(but n × n ) Chess, Go, Checkers see: [Fraenkel, Lichtenstein 81, Robson 83, 84]. Exptime Complete Halting Game.
We use a universal Turing Machine u (defined as 1-pointer cellular automata) which halts only by its headrolling off of the tape’s left end, leaving a blank. Bounded Halting Problem BHP( x ) determines if u ( x ) stops(i.e. the leftmost tape cell points left) within 2 k x k steps. This cannot be done in o (2 k x k ) steps.We now convert u into the Halting Game.The players are: L claiming u ( x ) halts in time (and shouldhave winning strategy iff this is true); His opponent is S . The board has four parts: the diagram, the input x to u , positiveintegers p (position) and t (time in the execution of u ( x )): p t A t +1 x B − B B +1 tp − p p +1The diagram shows the states A of cell p at time t +1 and B s , s ∈{ , ± } of cells p + s , at time t . A, B include the pointer direction; B may be replaced by “?”. Some board configurations are illegal: if (1) two of B s point away from each other, or (2) A differs from the result prescribed by the transition rules for B s , or(3) t = 1, while ( B s ) = x p + s . (At t = 1, u ( x ) is just starting, so its tape has the input x at the left, the headin the initial state at the end with blanks leading off to the right.) Here are the Game Rules:
The game starts in the configuration shown below. L moves first, replacing the ?s with symbols thatclaim to reflect the state of cells p + s at step t of u ( x ). S in its move chooses s , copies B s into A , fills all B with ?s, adds s to p , and decrements t .Start: p = 0 t = 2 k x k ← input x ? ? ? L puts: a t +1 b − b b +1 t S puts: b s t ? ? ? t − L may lie (i.e fill in “?” distorting the actual computation of u ( x )), as long as he is consistentwith the above “local” rules. All S can do is to check the two consecutive board configurations. He cannotrefer to past moves or to actual computation of u ( x ) as an evidence of L ’s violation. Strategy: If u ( x ) does indeed halt within 2 k x k steps, then the initial configuration is true to the compu-tation of u ( x ). Then L has an obvious (though hard to compute) winning strategy: just tell truly (and thusalways consistently) what actually happens in the computation. S will lose when t =1 and cannot decreaseany more. If the initial configuration is a lie, S can force L to lie all the way down to t = 1. How?If the upper box a of a legal configuration is false then the lower boxes b s cannot all be true, since therules of u determine a uniquely from them. If S correctly points the false b s and brings it to the top on hismove, then L is forced to keep on lying. At time t =1 the lie is exposed: the configuration doesn’t match theactual input string x , i.e. is illegal.Solving this game amounts to deciding correctness of the initial configuration, i.e. u ( x ) halting in 2 k x k steps: impossible in time o (2 k x k ). This Halting Game is artificial, still has a BHP flavor, though it does notrefer to exponents. We now reduce it to a nicer game (Linear Chess) to prove it exponentially hard, too. eonid A. Levin Fundamentals of Computing To reduce (see definition in sec. 2.2) Halting game to Linear Chess we introduce a few concepts.A non-deterministic Turing Machine (NTM) is a TM that sometimes offers a (restricted) transitionchoice, made by a driver , a function (of the TM configuration) of unrestricted complexity. A deterministic(ordinary) TM M accepts a string x if M ( x )=yes; an NTM M does if there exists a driver d s.t. M d ( x )=yes.NTM represent single player games – puzzles – with a simple transition rule, e.g., Rubik’s Cube.One can compute the winning strategy in exponential time by exhaustive search of all d . Home Work:
Prove all such games have P-time winning strategies, or show some have not.Will get you grade A for the course, $1,000,000 Award and a senior faculty rank at a school of your choice.The alternating
TM (ATM) is a variation of the NTM driven by two alternating drivers (players) l, r . A string is accepted if there is l such that for any r : M l,r ( x )= yes. Our games could be viewed asATM returning the result of the game in linear space but possibly exponential time, M prompts l and r alternatingly to choose their moves (in several steps if the move is specified by several bits) and computesthe resulting position, until a winner emerges. Accepted strings describe winning positions. Linear Chess Simulation of TM-Games.
The simulation first represents the Halting Game as an ATMcomputation simulated by the Ikeno TM (2.1) (using the “A/B” command for players’ input). The UTM isviewed as an array of 1-pointer cellular automata: Weak cells as rightward, Shy leftward. Upon termination,the TM head is set to move to the end of the tape, eliminating all loser pieces.This is viewed as a game of , a variant of Linear Chess, where the table, not the “GenderRule,” determine the victorious piece, and not only the vanquished piece is replaced, but also the winningpiece may be “promoted to” another type of the same side. The types are states of Ikeno TM showing Loyalty(pointer direction) d ∈{ W, S } , gender g (= previous d ), and 6/6/6/5 ranks (trit t ∈{ , , ∗} with ′ bit p ).Each 1dC transition is simulated in several Linear Chess stages. Let L,R be the two pieces active in 1dC.In odd stages L (in even stages R) changes gender while turning pointer twice. The last stage turns pointeronly once and possibly changes gender. In the first stage L appends its rank with R’s p bit. All other stagesreplace old 1dC rank with the new one. R appends its old t bit (only if t = ∗ ) to its new rank. Subsequentstages drop both old bits, marking L instead if it is the new 1dC head. Up to 4 more stages are used to exitany mismatch with 1dC new d, g bits. Space-Time Trade-off.
Deterministic linear space computations are games where any position has atmost one (and easily computable) move. We know no general superlinear lower bound or subexponentialupper bound for time required to determine their outcome. This is a big open problem.Recall that on a parallel machine: time is the number of steps untilthe last processor halts; space is the amount of memory used; vol-ume is the combined number of steps of all processors. “
Small ” willrefer to values bounded by a polynomial of the input length; “ large ”to exponential. Let us call computations narrow if either time or space are polynomial, and compact if both (and, thus, volume too)are. An open question:
Do all exponential volume algorithms (e.g.,one solving Linear Chess) allow an equivalent narrow computation? ✲ space ❄ time largetime,smallspace small time, large spacenarrow computations Alternatively, can every narrow computation be converted into a compact one?
This isequivalent to the existence of a P-time algorithm for solving any fast game, i.e. a game with a P-timetransition rule and a move counter limiting the number of moves to a polynomial. The sec. 3.1 algorithmcan be implemented in parallel P-time for such games. Converse also holds, similarly to the Halting Game.[Stockmeyer, Meyer 73] solve compact games in P-space: With M ⊂{ , } run depth-first search on thetree of all games – sequences of moves. On exiting each node it is marked as the active player’s win ifsome move leads to a child so marked; else as his opponent’s. Children’s marks are then erased. Conversely,compact games can simulate any P-space algorithms. Player A declares the result of the space- k , time-2 k computation. If he lies, player B asks him to declare the memory state in the middle of that time interval,and so by a k-step binary search catches A’s lie on a mismatch of states at two adjacent times. This hassome flavor of trade-offs such as saving time at the expense of space in dynamic programming.Thus, fast games (i.e. compact alternating computations) correspond to narrow deterministic computa-tions; general games (i.e. narrow alternating computations) correspond to large deterministic ones.2 Fundamentals of Computing Leonid A. Levin
Consider a P-time function F . For convenience, assume k F ( x ) k = k x k , (often k F ( x ) k = k x k Θ(1) suffices).Inverting F means finding, for a given y , at least one x ∈ F − ( y ), i.e. such that F ( x ) = y .We may try all possible x for F ( x ) = y . Assume F runs in linear time on a Pointer Machine. What is thecost of inverting F ? The space used is k x k + k y k +space F ( x ) = O ( k x k ). But time is O ( k x k k x k ): absolutelyinfeasible. No method is currently proven much better in the worst case. And neither could we prove someinversion problems to require super-linear time. This is the sad present state of Computer Science! An Example: Factoring.
Let F ( x , x ) = x x be the product of integers. For simplicity, assume x , x are primes. A fast algorithm in sec. 5.1 determines if an integer is prime. If not, no factor is given, only itsexistence. To invert F means to factor F ( x ). The density of n -bit primes is ≈ / ( n ln 2). So, factoring byexhaustive search takes exponential time! In fact, even the best known algorithms for this ancient problemrun in time about 2 √ k y k , despite centuries of efforts by most brilliant people. The task is now commonlybelieved infeasible and the security of many famous cryptographic schemes depends on this unproven faith. One-Way Functions: F : x → y are those easy to compute ( x y ) and hard to invert ( y x ) formost x . Even their existence is sort of a religious belief in Computer Theory. It is unproven, though manyfunctions seem to be one-way. Some functions, however, are proven to be one-way, IFF one-way functionsEXIST. Many theories and applications are based on this hypothetical existence. Search and NP Problems.
Let us compare the inversion problems with another type – the search problems specified by computable intime k x k O (1) relations P ( x, w ): given x , find w s.t. P ( x, w ). Any inversion problem is a search problem andany search problem can be restated as an inversion problem. E.g., finding a Hamiltonian cycle C in a graph G , can be stated as inverting a f ( G, C ), which outputs G, . . . C is in fact a Hamiltonian cycle of G .Otherwise, f ( G, C ) = 0 . . . w (called witness ) exist, and (b) aconstructive problem: actually find w . A time bound for solving one of these types of problems gives a similarbound for the other. Decision from construction is obvious. Conversely, each relation P can be extended to P ′ (( x, y ) , w ) = P ( x, w )&( w Turing Machine (sec. 3.3). All three classes of languages– search, inversion and NP – coincide (NP ⇐⇒ search is straightforward).Interestingly, polynomial space bounded deterministic and non-deterministic TMs have equivalent power.It is easy to modify TM to have a unique accepting configuration. Any acceptable string will be acceptedin time 2 s , where s is the space bound. Then we need to check A ( x, w, s, k ): whether the TM can be drivenfrom the configuration x to w in time < k and space s . For this we need for every z , to check A ( x, z, s, k − A ( z, w, s, k − t k ≤ t k − + k z k + O (1). So, t k = O ( sk ) = O ( s ) [Savitch 70].Search problems are games with P-time transition rules and one move duration. A great hierarchy ofproblems results from allowing more moves and/or other complexity bounds for transition rules. eonid A. Levin Fundamentals of Computing We discussed the (equivalent) inversion, search, and NP types of problems. Nobody knows whether all suchproblems are solvable in P-time (i.e. belong to P). This question (called P=?NP) is probably the most famousone in Theoretical Computer Science. All such problems are solvable in exponential time but it is unknownwhether any better algorithms generally exist. For many problems the task of finding an efficient algorithmmay seem hopeless, while similar or slightly modified problems have been solved. Examples:1. Linear Programming: Given integer n × m matrix A and vector b , find a rational vector x with Ax < b .Note, if n and entries in A have ≤ k -bits and x exists then an O ( nk )-bit x exists, too.Solution: The Dantzig’s Simplex algorithm finds x quickly for many A .Some A , however, take exponential time. After long frustrating efforts, a worst caseP-time Ellipsoid Algorithm was finally found in [Yudin and A.S. Nemirovsky 76].2. Primality test: Determine whether a given integer p has a factor?Solution: A bad (exponential time) way is to try all 2 k p k possible integer factors of p .More sophisticated algorithms, however, run fast (see section 5.1).3. Graph Isomorphism Problem: Are two given graphs G , G , isomorphic?I.e., can the vertices of G be re-numbered so that it becomes equal G ?Solution: Checking all n ! enumerations of vertices is impractical(for n = 100, this exceeds the number of atoms in the known Universe).[Luks 80] found an O ( n d ) steps algorithm where d is the degree. This is a P-time for d = O (1).4. Independent Edges (Matching):Find a given number of independent (i.e., not sharing nodes) edges in a given graph.Solution: Max flow algorithm solves a bipartite graph case.The general case is solved with a more sophisticated algorithm by J. Edmonds.Many other problems have been battled for decades or centuries and no P-time solution has been found.Even modifications of the previous four examples have no known answers:1. Linear Programming: All known solutions produce rational x .No reasonable algorithm is known to find integer x .2. Factoring: Given an integer, find a factor. Can be done in about exponential time n √ n .Seems very hard: Centuries of quest for fast algorithm were unsuccessful.3. Sub-graph isomorphism: In a more general case of finding isomorphisms of a graphto a part of another, no P-time solution has been found, even for O (1)-degree graphs.4. Independent Nodes: Find k independent (i.e., not sharing edges) nodes in a given graph.No P-time solution is known.We learned the proofs that Linear Chess and some other games have exponential complexity. None ofthe above or any other search/inversion/NP problem, however, have been proven to require super-P-time.When, therefore, do we stop looking for an efficient solution? NP-Completeness theory is an attempt to answer this question.See results by S.Cook, R.Karp, L.Levin, and others surveyed in [Garey, Johnson 79, Trakhtenbrot 84].A P-time function f reduces one NP-predicate p ( x ) to p ( x ) iff p ( x ) = p ( f ( x )), for all x . p is NP-complete if all NP problems can be reduced to it. Thus, each NP-complete problem is at least as worst-casehard as all other NP problems. This may be a good reason to give up on fast algorithms for it. Any P-timealgorithm for one NP-complete problem would yield one for all other NP (or inversion, or search) problems.No such solution has been discovered yet and this is left as a homework (10 years deadline).Faced with an NP-complete problem we can sometimes restate it, find a similar one which is easier(possibly with additional tools) but still gives the information we really want. We will do this in Sec. 5.1 forfactoring. Now we proceed with an example of NP-completeness.4 Fundamentals of Computing Leonid A. Levin Tiling Problem. Invert the function which, given a tiled square, outputs its first rowand the list of tiles used. A tile is one of the 26 possible squares containing a Latin letterat each corner. Two tiles may be placed next to each other if the letters on the sharedside match. (See an example at the right.) We now reduce to Tiling any search problem:given x , find w satisfying a P-time computable property P ( x, w ). a xm r x cr zm rn s r zs z Padding Argument. First, we need to reduce it to some “standard” NP problem. An obvious candidateis the problem “Is there w : U ( v, w ) ?”, where U is the universal Turing Machine, simulating P ( x, w ) for v = px . But U does not run in P-time, so we must restrict U to u which stops within some P-time limit.How to make this fixed degree limit sufficient to simulate any polynomial (even of higher degree) time P ?Let the TM u ( v, w ) for v =00 . . . px simulate k v k steps of U ( px, w )= P ( x, w ). If the padding of 0’s in v is sufficiently long, u will have enough time to simulate P , even though u runs in quadratic time, while P ’stime limit may be, say, cube (of a shorter “un-padded” string). So the NP problem P ( x, w ) is reduced to u ( v, w ) by mapping instances x into f ( x ) = 0 . . . px = v , with k v k determined by the time limit for P .Notice that program p for P is fixed. So, if some NP problem cannot be solved in P-time then neither canbe the problem ∃ ? w : u ( v, w ). Equivalently, if the latter IS solvable in P-time then so is any search problem.We do not know which of these alternatives is true. It remains to reduce the search problem u to Tiling. The Reduction. We compute u ( v, w ) (where v = 00 . . . px ) by a TM represented as an array of 1-pointer cellular automata that runs for k v k steps and stops if w does NOT solve the relation P . Otherwiseit enters an infinite loop. An instance x has a solution iff u ( v, w ) runs forever for some w and v = 0 . . . px .Here is the space-time diagram of computation of u ( v, w ). We set n to u ’s time (and space) k v k . Each row in this table represents theconfiguration of u in the respective moment of time. The solution w is filled in at the second step below a special symbol ”?”. If a table isfilled in wrongly, i.e. doesn’t reflect any actual computation, then itmust have four cells sharing a corner that couldn’t possibly appearin the computation of u on any input. ✲ space: n = k v k ❄ time v ?. . . ? v w T ... ... ... T n Proof. As the input v and the guessed solution w are the same in both the right and the wrong tables,the first 2 rows agree. The actual computation starts on the third row. Obviously, in the first mismatchingrow a transition of some cell from the previous row is wrong. This is visible from the state in both rows ofthis cell and the cell it points to, resulting in an impossible combination of four cells sharing a corner.For a given x , the existence of w satisfying P ( x, w ) is equivalent to the existence ofa table with the prescribed first row, no halting state, and permissible patterns of eachfour adjacent squares (cells). Converting the table into the Tiling Problem :The cells in the table are separated by “—” ; the tiles by “...”; Cut each cell into 4 partsby a vertical and a horizontal lines through its center and copy cell’s content in eachpart. Combine into a tile each four parts sharing a corner of 4 cells. If these cells arepermissible in the table, then so is the respective tile. u vv xSo, any P-time algorithm extending a given first row to the whole table of matching tiles from a givenset could be used to solve any NP problem by converting it to Tiling as shown. Problem: Find a polynomial time algorithm for n × log n Tiling Problem. eonid A. Levin Fundamentals of Computing The factoring problem seems very hard. But to test if a number has factors turns out to be much easier thanto find them. It also helps if we supply the computer with a coin-flipping device. See: [Rabin 80, Miller 76,Solovay, Strassen 77]. We now consider a Monte Carlo algorithm, i.e. one that with high probability rejectsany composite number, but never a prime. Residue Arithmetic. p | x means p divides x . x ≡ y (mod p ) means p | ( x − y ). y = ( x mod p ) denotes theresidue of x when divided by p , i.e. x ≡ y ∈ [0 , p − , p − 1] via shifting by an appropriate multiple of p . E.g., − x means p − x for residues mod p . We use ± x to mean either x or − x .The Euclidean Algorithm finds gcd( x, y ) – the greatest (and divisible by any other) common divisor of x and y : gcd( x, 0) = x ; gcd( x, y ) = gcd( y, ( x mod y )), for y > 0. By induction, g = gcd( x, y )= A ∗ x − B ∗ y ,where integers A =( g/x mod y ) and B =( g/y mod x ) are produced as a byproduct of Euclid’s Algorithm. Thisallows division ( mod p ) by any r coprime with p , (i.e. gcd( r, p )=1), and operations + , − , ∗ , / obey all usualarithmetical laws. We will need to compute ( x q mod p ) in polynomial time. We cannot do q> k q k multipli-cations. Instead we compute all numbers x i = ( x i − mod p ) = ( x i mod p ) , i < k q k . Then we represent q inbinary, i.e. as a sum of powers of 2 and multiply mod p the needed x i ’s. Fermat Test. The Little Fermat Theorem for every prime p and x ∈ [1 , p − 1] says: x ( p − ≡ p ).Indeed, the sequence ( xi mod p ) is a permutation of 1 , . . . , p − 1. So, 1 ≡ ( Q i
For each y and prime p, x ≡ y (mod p ) has at most one pair of solutions ± x . Proof.