List approximation for increasing Kolmogorov complexity
aa r X i v : . [ c s . CC ] F e b List approximation for increasing Kolmogorovcomplexity
Marius Zimand ∗ July 11, 2018
Abstract
It is impossible to effectively modify a string in order to increase itsKolmogorov complexity. But is it possible to construct a few strings,not longer than the input string, so that most of them have largercomplexity? We show that the answer is yes. We present an algorithmthat on input a string x of length n returns a list with O ( n ) manystrings, all of length n , such that 99% of them are more complex than x , provided the complexity of x is less than n − log log n − O (1). Weobtain similar results for other parameters, including a polynomial-time construction. The Kolmogorov complexity of a binary string x , denoted C ( x ), is the min-imal description length of x , i.e., it is the length of a shortest program (in afixed universal programming system) that prints x . We analyze the possibil-ity of modifying a string in an effective way in order to obtain a string withhigher complexity, without increasing its length. Strings with high complex-ity exhibit good randomness properties and are potentially useful becausethey can be employed in lieu of random bits in probabilistic algorithms. It iscommon to define the randomness deficiency of x as the difference | x | − C ( x )(where | x | is the length of x ), and to say that the smaller the randomnessdeficiency is, the more random is the string. In this sense, we want tomodify a string so that it becomes “more” random. As stated, the abovetask is impossible because clearly any effective modification cannot increaseKolmogorov complexity (at least not by more than a constant): If f is a ∗ Department of Computer and Information Sciences, Towson University, Baltimore,MD. http://orion.towson.edu/˜mzimand C ( f ( x )) ≤ C ( x ) + O (1), for every x . Consequentlywe have to settle for a weaker solution, and the one we consider is that oflist-approximation. List approximation consists in the construction of a listof objects guaranteed to contain at least one element having the desiredproperty. Actually, we try to obtain a stronger type of list approximation,in which, not just one, but most of the elements in the list have the desiredproperty. More precisely, we study the following question: Question.
Is there a computable function which takes as input a string x and outputs a short list of strings, which are not longer than x , such thatmost of the list’s elements have complexity greater than C ( x )?Without the restriction that the length is not increased, the problem iseasy to solve by appending a random string (see the discussion in Section 2).The restriction not only makes the problem interesting, but also amenableto applications in which the input string and the modified strings need tobe in a given finite set. The solution that we give can be readily adjustedto handle this case.The problem of increasing Kolmogorov complexity has been studied be-fore by Buhrman, Fortnow, Newman, and Vereshchagin [3]. They show thatthere exists a polynomial-time computable f that on input x of length n returns a list of strings, all having length n , such that if C ( x ) < n , thenthere exists y in the list with C ( y ) > C ( x ) (this is Theorem 14 in [3]). Inthe case of complexity conditioned by the string length, they show that it iseven possible to compute in polynomial time a list of constant size. That is f ( x ) is a list with O (1)-many strings of length n and if C ( x | n ) < n , thenit contains a string y with C ( y | n ) > C ( x | n ) (this is Theorem 11 in [3]).As indicated above we are after a stronger type of list approximation:We want on input x and δ > x with the property that a fraction of (1 − δ ) of its elementshave complexity larger than that of x . There are several parameters toconsider. The first one is the size of the list. The shorter is the list, thebetter is the approximation. Next, the increasing-complexity procedure thatwe seek will not work for all strings x . Recall that C ( x ) ≤ | x | + O (1)and if x is a string of maximal complexity at its length, then there simplyis no string of larger complexity at its length. In general, for strings x that have complexity close to | x | , it is difficult to increase their complexity.Thus, a second parameter is the bound on the complexity of x for which theincreasing-complexity procedure succeeds. The closer this bound is to | x | ,the better is the procedure. The third parameter is the complexity of theprocedure. The procedure is required to be computable, but it is preferable2f it is computable in polynomial time.We show the following three results, each one beating the other twowith respect to one of these three parameters. The first result exhibits acomputable list-approximation for increasing Kolmogorov complexity thatworks for any x with complexity C ( x ) < | x | − log log | x | − O (1). Theorem 1.1 (Computable list of polynomial size for increasing Kolmogorovcomplexity) . There exists a computable function f that on input x ∈ { , } ∗ and a rational number δ > , returns a list of strings of length | x | with thefollowing properties:1. The size of the list is O ( | x | )poly(1 /δ ) ,2. If C ( x ) < | x | − log log | x | − O (1) , then (1 − δ ) fraction of the elementsin the list f ( x ) have Kolmogorov complexity larger than C ( x ) (wherethe constant hidden in O (1) depends on δ ). Note.
In a previous version of this work (Proceedings STACS 2017) itis claimed that the above theorem holds for all strings x with C ( x ) < x .The proof had a bug, and we can only prove the version above which holdsfor all x with C ( x ) < | x | − log log | x | − O (1).Whether the bound C ( x ) < | x | − log log | x | − O (1) can be improvedremains open. In the next result, we improve the list size, making it linearin | x | (for constant δ ). The price is that the procedure works only for strings x with a slightly lower complexity. Theorem 1.2 (Computable list of linear size for increasing Kolmogorovcomplexity) . There exists a computable function f that on input x ∈ { , } ∗ and a rational number δ > , returns a list of strings of length | x | with thefollowing properties:1. The size of the list is O ( | x | )poly(1 /δ ) ,2. If C ( x ) < | x | − log | x | − O (1) , then (1 − δ ) fraction of the elements inthe list f ( x ) have Kolmogorov complexity larger than C ( x ) (where theconstant hidden in O (1) depends on δ ). Further reducing the list size remains an interesting open question. Wecould not establish a lower bound, and, as far as we currently know, it ispossible that even constant list size may be achievable.In the next result, the complexity-increasing procedure runs in polyno-mial time in the following sense. The size of the list is only quasi-polynomial,but each string in the list is computed in polynomial time.3 heorem 1.3 (Polynomial-time computable list for increasing Kolmogorovcomplexity) . There exists a function f that on input x ∈ { , } ∗ and aconstant rational number δ > , returns a list of strings of length | x | withthe following properties:1. The size of the list is bounded by O (log | x | ) ,2. If C ( x ) < | x | − O (log | x | ) , then (1 − δ ) fraction of the elements in thelist f ( x ) have Kolmogorov complexity larger than C ( x ) , and3. The function f is computable in polynomial time in the following sense:there is a polynomial time algorithm that on input x, i computes the i -th element in the list f ( x ) . Note that the procedure in Theorem 1.3 can be readily converted into apolynomial-time probabilistic algorithm, which uses O (log | x | ) random bitsto pick at random which element from the list to return.This paper is inspired by recent list approximation results regarding an-other problem in Kolmogorov complexity, namely the construction of shortprograms (or descriptions) for strings. We recall the standard setup forKolmogorov complexity, which we also use here. We fix an universal Turingmachine U . The universality of U means that for any Turing machine M ,there exists a computable “translator” function t , such that for all strings p , M ( p ) = U ( t ( p )) and | t ( p ) | ≤ | p | + O (1). For the polynomial-time construc-tions we also require that t is polynomial-time computable. If U ( p ) = x , wesay that p is a program (or description ) for x . The Kolmogorov complexityof the string x is C ( x ) = min {| p | | p is a program for x } . If p is a programfor x and | p | ≤ C ( x ) + c , we say that p is a c -short program for x . Using aBerry paradox argument, it is easy to see that it is impossible to effectivelyconstruct a shortest program for x (or, even a, say, n/ x ). Remarkably, Bauwens et al. [1] show that effective list approximation forshort programs is possible: There is an algorithm that, for some constant c ,on input x , returns a list with O ( | x | ) many strings guaranteed to contain a c -short program for x . They also show a lower bound: The quadratic size ofthe list is minimal up to constant factors. Teutsch [8] presents a polynomial-time algorithm with similar parameters, except that the list size is largerthan quadratic, but still polynomial. The currently shortest list size for apolynomial time list approximation is given by Zimand [11]. Closer to thestronger type of list approximation in this paper, are the probabilistic listapproximation results for short programs from Bauwens and Zimand [2] andZimand [12]. A polynomial-time probabilistic algorithm from [2], on input( x, k ) returns a string p of length bounded by k + O (log n ) such that, if4he promise k = C ( x ) holds, then, with 0 .
99 probability, p is a program for x . In [12], it is shown that the promise can be relaxed to k ≥ C ( x ). Thesurvey paper [9] presents most of these results. In this paper, we build onthe techniques in [2, 12]. We start by explaining why an approach that probably first comes to mindcannot lead to a result with good parameters, such as those obtained inTheorem 1.1 with a more complicated argument.Given that we want to modify a string x so that it becomes more complex,which in a sense means more random, a simple idea is to just append arandom string z to x . Indeed, if we consider strings z of length c , then C ( xz ) > C ( x ) + c/
2, for most strings z , provided c is large enough. Let ussee why this is true. Let k = C ( x ) and let z be a string that satisfies theopposite inequality, that is C ( xz ) ≤ C ( x ) + c/ , (1)Given a shortest program for xz and a self-delimited representation of theinteger c , which is 2 log c bits long, we obtain a description of x with at most k + c/ c bits. Note that from different z ’s satisfying (1), we obtainin this way distinct ( c/ c )-short programs for x . By a theorem ofChaitin [4] (also presented as Lemma 3.4.2 in [5]), for any d , the number of d -short programs for x is bounded by O (2 d ). Thus the number of strings z satisfying (1) is bounded by O (2 c/ c ). Since for large c , O (2 c/ c )is much smaller than 2 c , it follows that most strings z of length c satisfy theclaimed inequality (the opposite of (1)). Therefore, in this way we can obtaina list with a constant number of strings and most of them have complexitylarger than C ( x ). The problem with appending a random z to x , is thatthis operation not only increases complexity (which is something we want)but also increases length (which is something we don’t want). The naturalway to get around this problem is to first compress x to close to minimaldescription length using the probabilistic algorithms from [2, 12] described inthe Introduction, and then to append z . However, the algorithms from [2, 12]compress x to length C ( x ) + O (log n ), where n is the length of x . Afterappending a random z of length c , we obtain a string of length C ( x ) + O (log n ) + c , and for this to be n (so that length is not increased), we need C ( x ) ≤ n − O (log n ) − c . Thus, in this way we cannot obtain a procedure5hat works for all x with C ( x ) < n − log log n − O (1), such as the one fromTheorem 1.1.Our solution is based on a more elaborate construction. The centerpieceis a type of bipartite graph with a low congestion property. Once we havethe graph, we view x as a left node, and the list f ( x ) consists of some ofthe nodes at distance 2 in the graph from x . (A side remark: Buhrmanet al. [3] use graphs as well, namely constant-degree expanders, and theyobtain the lists also as the set of neighbors at some given distance.) In ourgraph, the left side is L = { , } n , the set of n -bit strings, the right side is R = { , } m , the set of m -bit strings, and each left node has degree D . Thegraphs also depend on three parameters ǫ, ∆, and t , and for our discussionit is convenient to also use δ = ǫ / and s = δ · ∆. The graphs that we needhave two properties. The first one is a low congestion requirement whichdemands that for every subset B of left nodes of size at most 2 t , (1 − δ )fraction of nodes in B share (1 − δ ) fraction of their right neighbors with atmost s other nodes in B . The second property is that each right node hasat least ∆ neighbors.Let us now see how to use such graphs to increase Kolmogorov com-plexity in the list-approximation sense. Suppose we have a graph G withthe above properties for the parameters n, δ, ∆ , D, s , and t . We claim thatfor each x of length n and with complexity C ( x ) < t , we can obtain a listwith D · ∆ many strings, all having length n , such that at least a fractionof (1 − δ ) of the strings in the list have complexity larger than C ( x ). In-deed, let x be a string of length n with C ( x ) = k < t . Consider the set B = { x ′ ∈ { , } n | C ( x ′ ) ≤ k } . Note that the size of B is bounded by2 t . A node that does not have the low-congestion property is said to be δ -BAD( B ). By the low-congestion of G , there are at most δ | B | elements in B that are δ -BAD( B ). It can be shown that x is not δ -BAD( B ). The reasonis, essentially, that the strings that are δ -BAD( B ) can be enumerated andthey make a small fraction of B and therefore can be described with lessthan k bits. Now, to construct the list, we view x as a left node in G and we“go-right-then-go-left.” This means that we first “go-right,” i.e., we take allthe D neighbors of x , and for each such neighbor y we “go-left,” i.e., we take∆ of the y ’s neighbors and put them in the list. Since x is not δ -BAD( B ),(1 − δ ) D of its neighbors have at most s = δ · ∆ elements in B . Overall, only2 δ · D · ∆ of the strings in the list can be in B , and so at least a fraction More formally, for all B ⊆ L with | B | ≤ t , for all x ∈ B , except at most δ | B | elements, all neighbors y of x , except at most δD , have deg B ( y ) ≤ s , where deg B ( y ) isthe number of y ’s neighbors that are in B .
6f (1 − δ ) of the strings in the list have complexity larger than k = C ( x ).Our claim is proved.For our main results (Theorem 1.1, Theorem 1.2, and Theorem 1.3),we need graphs with the above properties for different settings of parame-ters. Such graphs can be obtained from randomness extractors, which havebeen extensively studied in the theory of pseudo-randomness (for example,see Vadhan’s monograph [10]). The graphs required by Theorem 1.1 andTheorem 1.2 are constructed using the probabilistic method in Lemma 3.2,and the graph required by Theorem 1.3 is obtained in Lemma 3.3 from arandomness extractor of Raz, Reingold, and Vadhan [7]. We define here formally the type of graphs that we need. We work withfamilies of graphs G n = ( L, R, E ⊆ L × R ), indexed by n , which have thefollowing structure:1. Vertices are labeled with binary strings: L = { , } n , R = { , } n − a ,where we view L as the set of left nodes, and R as the set of rightnodes. The parameter a can be positive or negative, and in absolutevalue is typically small (less than poly(log n)).2. All left nodes have the same degree D , D = 2 d is a power of two, andthe edges outgoing from a left node x are labeled with binary stringsof length d .3. We allow multiple edges between two nodes. For a node x , we write N ( x ) for the multiset of x ’s neighbors, each element being taken withthe multiplicity equal to the number of edges from x landing into it.A bipartite graph of this type can be viewed as a function EXT : { , } n ×{ , } d → { , } n − a , where EXT(x , y) = z iff there is an edge between x and z labeled y . We want EXT to yield a ( k, ǫ ) randomness extractorwhenever we consider the modified function EXT k which on input ( x, y )returns EXT(x , y) from which we cut the last n − k bits. Note that theoutput of EXT k has k − a bits.From the function EXT k , we go back to the graph representation, and weobtain the “prefix” bipartite graph G n,k = ( L = { , } n , R k = { , } k − a , E k ⊆ L × R k ), where in G n,k we merge the right nodes of G n that have the sameprefix of length k − a . Since we allow multiple edges between nodes, theleft degrees in the prefix graph do not change. However, right degrees may7hange, and as m k gets smaller, right degrees typically get larger due tomerging.The requirement that G n,k is a ( k, ǫ ) randomness extractor means thatfor every subset B ⊆ L of size | B | ≥ k , for every A ⊆ R k , (cid:12)(cid:12)(cid:12)(cid:12) | E k ( B, A ) || B | × D − | A || R k | (cid:12)(cid:12)(cid:12)(cid:12) ≤ ǫ, (2)where E k ( B, A ) is the set of edges between B and A in G n,k .We also want to have the guarantee that each right node in G n,t hasdegree at least ∆, where ∆ and t are parameters.Accordingly, we have the following definition. Definition 3.1.
A graph G n = ( L, R, E ⊆ L × R ) as above is ( ǫ, ∆ , t ) -balanced if the following requierments hold:1. For every k ∈ { , . . . , n } , let G n,k be the graph corresponding to EXT k described above. We require that, for every k ∈ { , . . . , n } , G n,k is a ( k, ǫ ) extractor, i.e., G n,k has the property in Equation (2).2. In the graph G n,t , every right node with non-zero degree has degree atleast ∆ . In our applications, we need balanced graphs in which the neighbors ofa given node can be found effectively, or even in time that is polynomial in n . As usual, we consider families of graphs ( G n ) n ≥ , and we say that sucha family is computable if there is an algorithm that on input ( x, y ), where x is a left node, and y is the label of an edge outgoing from x , outputs z ,where z is the right node where the edge y lands. If the algorithm runsin time polynomial in n , we say that the family ( G n ) n ≥ is explicit . Forpolynomial-time list approximation, we actually need a stronger propertywhich essentially states that going from right to left can also be done inpolynomial time (see the “Moreover...” part in Lemma 3.3).The following two lemmas provide the balanced graphs that are used inthe proofs of the main result as explained in the proof overview in Section 2. Lemma 3.2. (a) For every sufficiently large positive integer n and everyrational ǫ > , there is constant c and a computable ( ǫ, ∆ , t ) -balanced graph G n = ( L = { , } n , R = { , } n , E ⊆ L × R ) , with left degree D = 2 d = O ( n · (1 /ǫ ) ) , ∆ = 2(1 /ǫ ) / D and t = n − log log n − c .(b) There exists a constant c such that for every sufficiently large positiveinteger n and every rational ǫ > , there is a constant c ′ and a computable ( ǫ, ∆ , t ) -balanced graph G n = ( L = { , } n , R = { , } m , E ⊆ L × R ) , with eft degree D = 2 d = O ( n · (1 /ǫ ) ) , m = n + d − /ǫ ) − c , ∆ =2(1 /ǫ ) / D | L | / | R | = O (1) and t = n − log n − c ′ . The proof of Lemma 3.2 is by the standard probabilistic method, and ispresented in Section 5.1.
Lemma 3.3.
There exists a constant c such that for every positive integer n and for every rational ǫ > , there is an explicit ( ǫ, ∆ , t ) -balanced graph G n = ( L = { , } n , R = { , } m , E ⊆ L × R ) , with left degree D = 2 d ,for d = O (log ( n ) log (1 /ǫ )) , m = n − cd , ∆ = 2(1 /ǫ ) / · D c +1 and t = n − (log ∆ − cd ) .Moreover, there is an algorithm that on input ( z, y ) (and n ), where z ∈ R = { , } m and y ∈ { , } d computes a list of ∆ left neighbors of z reachablefrom z by edges labeled y , or NIL if there are less than ∆ such neighbors.This algorithm computes the list implicitly, in the sense that given an index i , it returns the i -th element in the list in time polynomial in n and log i . The proof of Lemma 3.3 is based on a randomness extractor of Raz,Reingold, and Vadhan [7] and is presented in Section 5.2.Let us now proceed to the proofs of Theorem 1.1, Theorem 1.2, andTheorem 1.3.
The theorems have essentially identical proofs, except that balanced graphswith different parameters are used. The following lemma shows a generictransformation of a balanced graph into a function that on input x producesa list so that most of its elements have complexity larger than C ( x ). Lemma 4.1.
Suppose that for every constant δ > , there is t = t ( n ) , a = a ( n ) , and a computable (respectively, explicit and satisfying the propertystated in the “moreover” part of Lemma 3.3) ( δ , ∆ , t ) - balanced graph G n =( L n = { , } n , R n = { , } n − a , E n ⊆ L n × R n ) , with ∆ = 2(1 /δ ) · D · a ,where D is the left degree.Then there exists a computable (respectively, polynomial-time computable)function f that on input a string x and a rational number δ > returns alist containing strings of length | x | and1. The size of the list is /δ ) D a , . If C ( x ) ≤ t , then (1 − δ ) of the elements in the list have complexitylarger than C ( x ) .Proof of Lemma 4.1. We can assume without loss of generality that 1 /δ is sufficiently large(for the following arguments to be valid) and also that it is a power of 2.Let ǫ = δ . Let x be a binary string of length n , with complexity C ( x ) = k .We assume that k ≤ t . We explain how to compute the list f ( x ), with theproperty stipulated in the theorem’s statement.We take G n to be the ( ǫ, ∆ , t )-balanced graph with left nodes of length n promised by the hypothesis. Let G n,t be the “prefix” graph obtained from G n by cutting the last n − t bits in the labels of right nodes (thus preservingthe prefix of length t − a in the labels).The list f ( x ) is computed in two steps:1. First, we view x as a left node in G n,t and take N ( x ), the multiset ofall neighbors of x in G n,t .2. Secondly, for each p in N ( x ), we take A p to be a set of ∆ neighborsof p in G n,t (say, the first ∆ ones in some canonical order). We set f ( x ) = S p ∈ N ( x ) A p (if p appears n p times in N ( x ), we take A p in theunion also n p times; note that f ( x ) is a multiset).Note that all the elements in the list have length n , and the size of the listis | f ( x ) | = ∆ · D = (1 /δ ) D a .The rest of the proof is dedicated to showing that the list f ( x ) satisfiesthe second item in the statement. Let B n.k = { x ′ ∈ { , } n | C ( x ′ ) ≤ k } , and let S n,k = ⌊ log | B n,k |⌋ . Thus, 2 S n,k ≤ | B n,k | < S n,k +1 . Later we willuse the fact that S n,k ≤ k ≤ t. (3)We want to use the properties of extractors for sources with min-entropy S n,k and therefore we consider the graph G n,S n,k , which is obtained, as wehave explained above, from G n by taking the prefixes of right nodes of length S n,k − a . To simplify notation, we use G instead of G n,S n,k . The set of leftnodes in G is L = { , } n and the set of right nodes in G is R = { , } m , for m = S n,k − a .We view B n,k as a subset of the left nodes in G . Let us introduce somehelpful terminology. In the following all the graph concepts (left node, rightnode, edge, neighbor) refer to the graph G . We say that a right node z in10 is (1 /ǫ )-light if it has at most (1 /ǫ ) · | B n,k |· D | R | neighbors in B n,k . A nodethat is not (1 /ǫ )-light is said to be (1 /ǫ )-heavy. Note that(1 /ǫ ) · | B n,k | · D | R | ≤ (1 /ǫ ) 2 S n,k +1 · D S n.k · − a = δ ∆ , and thus an (1 /ǫ )-light node has at most δ ∆ many neighbors in B n,k .We also say that a left node in B n,k is δ -BAD with respect to B n,k if atleast a δ fraction of the D edges outgoing from it land in right neighborsthat are (1 /ǫ )-heavy. Let δ -BAD( B n,k ) be the set of nodes that are δ -BADwith respect to B n,k .We show the following claim. Claim 4.2.
At most a δ fraction of the nodes in B n,k are δ -BAD withrespect to B n,k .(In other words: for every x ′ in B n,k , except at most a δ fraction, atleast a (1 − δ ) fraction of the edges going out from x ′ in G land in rightnodes that have at most ∆ ′ neighbors with complexity at most k .) We defer for later the proof of Claim 4.2, and continue the proof of thetheorem.For any positive integer k , let B k = { x ′ | C ( x ′ ) ≤ k and k ≤ t ( | x ′ | ) } . Let I k = { n | k ≤ t ( n ) } . Note that | B k | = P n ∈ I k | B n,k | . Let x ′ ∈ B k , andlet n ′ = | x ′ | . We say that x ′ is δ -BAD with respect to B k if in G n ′ , x ′ is δ -BAD with respect to B n ′ ,k . We denote δ -BAD( B k ) the set of nodes thatare δ -BAD with respect to B k . We upper bound the size of δ -BAD( B k ): | δ -BAD( B k ) | = P n ′ ∈ I k | δ -BAD( B n ′ ,k ) |≤ P n ′ ∈ I k δ · | B n ′ ,k | (by Claim (4.2))= 2 δ P n ∈ I k | B n ′ ,k | = 2 δ | B k |≤ δ · k +1 . Note that the set δ -BAD( B k ) can be enumerated given k and δ . Thereforea node x ′ that is δ -BAD with respect to B k can be described by k , δ and itsordinal in the enumeration of the set δ -BAD( B k ). We write the ordinal onexactly k +2 − log(1 /δ ) bits and δ in a self-delimited way on 2 log log(1 /δ ) bits(recall that 1 /δ is a power of 2), so that k can be inferred from the ordinal11nd δ . It follows that if x ′ is δ -BAD with respect to B k , then, provided 1 /δ is sufficiently large, C ( x ′ ) ≤ k + 2 − log(1 /δ ) + 2 log log(1 /δ ) + O (1) < k. (4)Now, recall our string x ∈ { , } n which has complexity C ( x ) = k . Theinequality (4) implies that x cannot be δ -BAD with respect to B k , whichmeans that (1 − δ ) of the edges going out from x land in neighbors in G having at most δ ∆ neighbors in B k . The same is true if we replace G by G n,t , because, by the inequality (3), right nodes in G are prefixes of rightnodes in G n,t .Now suppose we pick at random a neighbor p of x in G n,t , and then finda set A p of ∆ neighbors of p in G n,t . Then with probability 1 − δ , only afraction of δ of the elements of A p can be in B k . Recall that we have definedthe list f ( x ) to be f ( x ) = [ p neighbor of x in G n,t A p . It follows that (1 − δ ) of elements in f ( x ) have complexity larger than C ( x )and this ends the proof.It remains to prove Claim 4.2. Proof of Claim 4.2.
Let A be the set of right nodes that are (1 /ǫ )-heavy.Then | A | ≤ ǫ | R | . Indeed the number of edges between B n,k and A is at least | A |· (1 /ǫ ) · | B n,k |· D | R | (by the definition of (1 /ǫ )-heavy), but at the same time the total number ofedges between B n,k and R is | B n,k | · D (because each left node has degree D ).Next we show that | δ -BAD( B n.k ) | ≤ δ | B n,k | . (5)For this, note that G is a ( S n,k , ǫ ) randomness extractor and B n,k has sizeat least 2 S n,k . Therefore by the property (2) of extractors, | E ( B n,k , A ) || B n,k | · D ≤ | A || R | + ǫ ≤ ǫ.
12n the other hand the number of edges linking B n,k and A is at least thenumber of edges linking δ -BAD( B n,k ) and A and this number is at least | δ -BAD( B n,k | · δD . Thus, | E ( B n,k , A ) | ≥ | δ -BAD( B n,k ) | · δD. Combining the last two inequalities, we obtain | δ -BAD( B n,k ) || B n,k | ≤ ǫ · δ = 2 δ. End of the proofs of Claim 4.2 and of Lemma 4.1.
Theorem 1.1, Theorem 1.2 , and Theorem 1.3 are obtained by plugging,respectively, into the above lemma the balanced graphs from Lemma 3.2(a), Lemma 3.2 (b) and Lemma 3.3, with parameter ǫ = δ in every case. We first prove part (b), since we can give a self-contained and elementaryargument . We use the probabilistic method. For some constant c that willbe fixed later, we consider a random function EXT : { , } n × { , } d →{ , } n+d − /ǫ ) − c . We show the following two claims, which imply that arandom function has the desired properties with positive probability. Sincethe properties can be checked effectively, we can find a graph as stipulatedin part (b) by exhaustive search. We use the notation from Definition 3.1and from the paragraph preceding it. Claim 5.1.
For some constant c , with probability ≥ / , it holds that for ev-ery k ∈ { , . . . , n } , in the bipartite graph G n,k = { L, R k , E k ⊆ L × R k } , every B ⊆ L = { , } n of size | B | ≥ k , and every A ⊆ R k = { , } k + d − /ǫ ) − c satisfy (cid:12)(cid:12)(cid:12)(cid:12) | E k ( B, A ) || B | × D − | A || R k | (cid:12)(cid:12)(cid:12)(cid:12) ≤ ǫ. (6) Claim 5.2.
For every sufficiently large positive integer n , with probability ≥ / , every right node in the graph G n,n − log n − log log n has degree at least ∆ . Proof of Claim 5.1. First we fix k ∈ { , . . . , n } and let K = 2 k and N = 2 n . Let us consider B ⊆ { , } n of size | B | ≥ K , and A ⊆ R k . For13 fixed x ∈ B and y ∈ { , } d , the probability that EXT k (x , y) is in A is | A | / | R k | . By the Chernoff bounds,Prob (cid:20)(cid:12)(cid:12)(cid:12)(cid:12) | E k (B , A) || B | × D − | A || R k | (cid:12)(cid:12)(cid:12)(cid:12) > ǫ (cid:21) ≤ − Ω(K · D · ǫ ) . The probability that relation (6) fails for a fixed k , some B ⊆ { , } k ofsize | B | ≥ K and some A ⊆ R k is bounded by 2 K · D · ǫ · − c · (cid:0) NK (cid:1) · − Ω( K · D · ǫ ) ,because A can be chosen in 2 K · D · ǫ · − c ways, and we can consider that B has size exactly K and there are (cid:0) NK (cid:1) possible choices of such B ’s. If D = Ω(( n − k ) /ǫ ) and c is sufficiently large, the above probability is muchless than (1 / − k . Therefore the probability that relation (6) fails for some k , some B and some A is less than 1 / Proof of Claim 5.2.
We use a standard “coupon collector” argument.Let t = n − log n − c ′ , where c ′ is a constant that will be fixed later. Let N = 2 n and C = 2 c , where c is the constant for which Claim 5.1 holds..We work in the bipartite graph G n,n − log n − c ′ = ( L, R, E ⊆ L × R ) in whichevery left node has degree D = 2 d , L = { , } n , and R = { , } m , where m = ( n − log n − c ′ ) + ( d − /ǫ ) − c ). For a left node x , an edgelabeled y ∈ { , } d and a right node z , we say that ( x, y ) hits z if the y -labeled edge outgoing from x lands in z . We want to show that with highprobability each z is hit at least ∆ times. Let us order { , } n × { , } d in,say, lexicographical order { ( x, y ) < ( x, y ) < . . . < ( x, y ) ND } . We define∆ groups of “shooting” at R by taking ( x, y ) , . . . , ( x, y ) r in the first group,( x, y ) r +1 , . . . , ( x, y ) r in the second group, and so on with r left nodes in eachgroup, where r will be fixed later. The probability that a fixed z is not hitby some ( x, y ) i is (1 − / | R | ) ≤ e − / | R | . The probability that a fixed z is nothit by any element in a given group is at most e − r/ | R | and the probabilitythat there exists some z ∈ R that is not hit by a given group is bounded by | R | e − r/ | R | . We take r = | R | (ln | R | + ln(4∆)), and the above probability isbounded by 1 / (4∆). Therefore, the probability that some z in R is not hitby some group in the set of ∆ groups is at most 1 /
4. Note that, for someappropriate choice of c ′ , r · ∆ ≤ N D , provided n is large enough, and thusall the groups fit into { , } n × { , } d . Part (a).
Similarly to part (b), we take a random function EXT : { , } n × { , } d → { , } n . The analogue of Claim 5.1 holds true in thesame way since the only modification is that this time R k = { , } k . The“coupon collector” argument needed to prove the analogue of Claim 5.2 isdone as follows. We consider the graph G n,n − log log n − c for some constant14 that will be fixed later. This graph is obtained from the above functionEXT as explained in Definition 3.1. The graph G n,n − log log n − c is a bipartitegraph with left side L = { , } n , right side R ′ = { , } n − log log n − c and eachleft node has degree D = 2 d . We show that with probability ≥ /
4, everyright node in G n,n − log log n − c has degree at least ∆. The random processconsists of drawing for each x ∈ L and edge y ∈ { , } d a random elementfrom R ′ . Thus we draw at random N D times, with replacement, from aset with | R | ′ many “coupons.” Newman and Shepp [6] have shown that toobtain at least h times each coupon from a set of p coupons, the expectednumber of draws is p log p + ( h − p log log p + o ( p ). By Markov’s inequality,if the number of draws is 4 times the expected value, we collect each coupon p times with probability 3 /
4. In our case, we have p = 2 n − log log n − c and h = ∆ and it can be checked readily that, for an appropriate choice of theconstant c , 4( p log p + ( h − p log log p + o ( p )) < N D , provided n is largeenough. End of the proof of Lemma 3.2.
The construction relies on the randomness extractor of Raz, Reingold, andVadhan [7].
Theorem 5.3 (Theorem 22, (2) in [7]) . There exists a function
EXT : { , } n × { , } d → { , } m , computable in time polynomial in n , with thefollowing properties:(1) d = O (log ( n ) log (1 /ǫ )) ,(2) m = n − c · d , for some constant c ,(3) For every k ≤ n , the function EXT k obtained by computing EXT andcutting the last n − k bits of the output is a ( k, ǫ ) extractor,(4) For every y ∈ { , } d ( n ) , the function f y ( x ) = EXT(x , y) is a linearfunction from (GF[2]) n to (GF[2]) m (where we view x ∈ { , } n as anelement of (GF[2]) n in the natural way). In other words, EXT(x , y) =A y · x , where A y is an m -by- n matrix with entries in GF[2] , computablefrom y in time polynomial in n .Note. Item (4) is not explicitly stated in [7], so we provide here a short ex-planation. The construction given in [7] of EXT : { , } n ×{ , } d → { , } m ,views x as the specification of a function u x ( · , · ) of two variables (in a way15hat we present below), defines some functions g ( y ) , h ( y ) , . . . , g m ( y ) , h m ( y ),each one computable in time polynomial in n , and then setsEXT(x , y) = u x (g (y) , h (y)) , . . . , u x (g m (y) , h m (y)) , (7)i.e., the i -th bit is u x ( g i ( y ) , h i ( y )). Thus, it is enough to check that f v,w ( x ) = u x ( v, w ) is linear in x . Let us now describe u x . The characteristic sequenceof u x is the Reed-Solomon code of x . More precisely, for some s , x isviewed as a polynomial p x over the field GF[2 s ]. The elements of GF[2 s ]are viewed as s -dimensional vectors over GF[2] in the natural way. Notethat in this view the evaluation of p x at point v is a linear transformationof x , i.e., p x ( v ) = A v x for some s -by- n matrix A v with entries from GF[2].Finally, u x ( v, w ) is defined as the inner product w · p x ( v ) and therefore u x ( v, w ) = ( wA v ) x , and thus it is a linear function in x . Now we plug h i ( y )as w and g i ( y ) as v , and we build the matrix A y , by taking its i -th row tobe h i ( y ) A g i ( y ) . Using the Equation (7), we obtain item (4) in the theorem.Now let us proceed to the actual proof of Lemma 3.3. The functionEXT from Theorem 5.3 defines the explicit bipartite graph G n . Let t = n − (log ∆ − c · d ). By removing the last n − t bits in each right node weobtain the graph G n,t . We only need to check that in the bipartite graph G n,t = ( L t = { , } n , R t = { , } m t , E t ⊆ L t × R t ) (where m t = n − log ∆),every right node with non-zero degree has degree at least ∆. This followseasily from the linearity of EXT t (x , y) defined to be EXT(x , y) from whichwe cut the last n − t bits.Indeed, let z in { , } m t be a right node with non-zero degree. Thismeans that there exist x and y such that EXT t (x , y) = z. Since the function f y ( x ) = EXT t (x , y) is linear in x , it follows that { x ′ | EXT(x ′ , y) = z } = { x ′ | A y · x ′ = z } (i.e., the preimage of z ) is an affine space over GF[2] withdimension at least n − m t = log ∆, and therefore z has degree at least ∆.Moreover, given y , we can find ∆ preimages of z in time polynomial in n ,by solving the linear system. The author is grateful to Bruno Bauwens for his insightful observations. Heis also grateful to Nikolay Vereshchagin for pointing out an error in an earlierversion. 16 eferences [1] B. Bauwens, A. Makhlin, N. Vereshchagin, and M. Zimand. Shortlists with short programs in short time. In
Proceedings of 28th IEEEConference on Computational Complexity, Stanford, California, USA ,2013.[2] Bruno Bauwens and Marius Zimand. Linear list-approximationfor short programs (or the power of a few random bits).In
IEEE 29th Conference on Computational Complexity, CCC2014, Vancouver, BC, Canada, June 11-13, 2014 , pages 241–247. IEEE, 2014. URL: http://dx.doi.org/10.1109/CCC.2014.32 , doi:10.1109/CCC.2014.32 .[3] H. Buhrman, L. Fortnow, I. Newman, and N. Vereshchagin. IncreasingKolmogorov complexity. In Proceedings of the 22nd Annual Symposiumon Theoretical Aspects of Computer Science , pages 412–421, Berlin,2005. Springer-Verlag
Lecture Notes in Computer Science .[4] Gregory J. Chaitin. Information-theoretic characterizations of recursiveinfinite strings.
Theor. Comput. Sci. , 2(1):45–48, 1976.[5] R. Downey and D. Hirschfeldt.
Algorithmic randomness and complexity .Springer Verlag, 2010.[6] D. Newman and L. Shepp. The Double Dixie Cup Problem.
AmericanMathematical Monthly . 67:58–61, 1960.[7] Ran Raz, Omer Reingold, and Salil P. Vadhan. Extract-ing all the randomness and reducing the error in Tre-visan’s extractors.
J. Comput. Syst. Sci. , 65(1):97–128,2002. URL: http://dx.doi.org/10.1006/jcss.2002.1824 , doi:10.1006/jcss.2002.1824 .[8] Jason Teutsch. Short lists for shortest descriptions inshort time. Computational Complexity , 23(4):565–583, 2014.URL: http://dx.doi.org/10.1007/s00037-014-0090-3 , doi:10.1007/s00037-014-0090-3 .[9] Jason Teutsch and Marius Zimand. A brief on short descriptions. SIGACT News , 47(1):42–67, March 2016.[10] Salil P. Vadhan. Pseudorandomness.
Foundations andTrends in Theoretical Computer Science , 7(1-3):1–336,2012. URL: http://dx.doi.org/10.1561/0400000010 , doi:10.1561/0400000010 . 1711] Marius Zimand. Short lists with short programs in shorttime - A short proof. In Arnold Beckmann, Erzs´ebet Csuhaj-Varj´u, and Klaus Meer, editors, Language, Life, Limits -10th Conference on Computability in Europe, CiE 2014, Bu-dapest, Hungary, June 23-27, 2014. Proceedings , volume 8493of
Lecture Notes in Computer Science , pages 403–408. Springer,2014. URL: http://dx.doi.org/10.1007/978-3-319-08019-2_42 , doi:10.1007/978-3-319-08019-2_42 .[12] Marius Zimand. Kolmogorov complexity version of Slepian-Wolf coding. CoRR , abs/1511.03602, 2015. URL: http://arxiv.org/abs/1511.03602http://arxiv.org/abs/1511.03602