Block Rigidity: Strong Multiplayer Parallel Repetition implies Super-Linear Lower Bounds for Turing Machines
aa r X i v : . [ c s . CC ] N ov Block Rigidity: Strong Multiplayer Parallel Repetitionimplies Super-Linear Lower Bounds for Turing Machines
Kunal Mittal ∗ Ran Raz ∗ Abstract
We prove that a sufficiently strong parallel repetition theorem for a special caseof multiplayer (multiprover) games implies super-linear lower bounds for multi-tapeTuring machines with advice. To the best of our knowledge, this is the first connectionbetween parallel repetition and lower bounds for time complexity and the first majorpotential implication of a parallel repetition theorem with more than two players.Along the way to proving this result, we define and initiate a study of block rigidity ,a weakening of Valiant’s notion of rigidity [Val77]. While rigidity was originally definedfor matrices, or, equivalently, for (multi-output) linear functions, we extend and studyboth rigidity and block rigidity for general (multi-output) functions. Using techniques ofPaul, Pippenger, Szemer´edi and Trotter [PPST83], we show that a block-rigid functioncannot be computed by multi-tape Turing machines that run in linear (or slightly super-linear) time, even in the non-uniform setting, where the machine gets an arbitrary advicetape.We then describe a class of multiplayer games, such that, a sufficiently strong parallelrepetition theorem for that class of games implies an explicit block-rigid function. Thegames in that class have the following property that may be of independent interest:for every random string for the verifier (which, in particular, determines the vector ofqueries to the players), there is a unique correct answer for each of the players, andthe verifier accepts if and only if all answers are correct. We refer to such games as independent games . The theorem that we need is that parallel repetition reduces thevalue of games in this class from v to v Ω( n ) , where n is the number of repetitions.As another application of block rigidity, we show conditional size-depth tradeoffsfor boolean circuits, where the gates compute arbitrary functions over large sets. ∗ Department of Computer Science, Princeton University. Research supported by the Simons Collaborationon Algorithms and Geometry, by a Simons Investigator Award and by the National Science Foundation grantsNo. CCF-1714779, CCF-2007462. Introduction
We study relations between three seemingly unrelated topics: parallel repetition of multi-player games, a variant of Valiant’s notion of rigidity, that we refer to as block rigidity, andproving super-linear lower bounds for Turing machines with advice.
Deterministic multi-tape Turing machines are the standard model of computation for definingtime-complexity classes. Lower bounds for the running time of such machines are known bythe time-hierarchy theorem [HS65], using a diagonalization argument. Moreover, the seminalwork of Paul, Pippenger, Szemer´edi and Trotter gives a separation of non-deterministic lineartime from deterministic linear time, for multi-tape Turing machines [PPST83] (using ideasfrom [HPV77] and [PR80]). That is, it shows that DTIME( n ) ( NTIME( n ). This result hasbeen slightly improved to give DTIME( n p log ∗ n ) ( NTIME( n p log ∗ n ) [San01].However, the above mentioned lower bounds do not hold in the non-uniform setting,where the machines are allowed to use arbitrary advice depending on the length of the input.In the non-uniform setting, no super-linear lower bound is known for the running time ofdeterministic multi-tape Turing machines. Moreover, such bounds are not known even for amulti-output function. The concept of matrix rigidity was introduced by Valiant as a means to prove super-linearlower bounds against circuits of logarithmic depth [Val77]. Since then, it has also foundapplications in communication complexity [Raz89] (see also [Wun12, Lok01]). We extendValiant’s notion of rigid matrices to the concept of rigid functions , and further to block-rigid functions . We believe that these notions are of independent interest. We note thatblock-rigidity is a weaker condition than rigidity and hence it may be easier to find explicitblock-rigid functions. Further, our result gives a new application of rigidity.Over a field F , a matrix A ∈ F n × n is said to be an ( r, s )-rigid matrix if it is not possibleto reduce the rank of A to at most r , by changing at most s entries in each row of A . Valiantshowed that if A ∈ F n × n is ( ǫn, n ǫ )-rigid for some constant ǫ >
0, then A is not computableby a linear-circuit of logarithmic depth and linear size. As in many problems in complexity,the challenge is to find explicit rigid matrices. By explicit, we mean that a polynomial timedeterministic Turing machine should be able to output a rigid matrix A ∈ F n × n on input 1 n .The best known bounds on explicit rigid matrices are far from what is needed to get super-linear circuit lower bounds (see [Fri93, SS97, SSS97, Lok00, Lok06, KLPS14, AW17, GT18,AC19, BHPT20]).We extend the above definition to functions f : { , } n → { , } n , by saying that f is not an ( r, s )-rigid function if there exists a subset X ⊆ { , } n of size at least 2 n − r , such that over X , each output bit of f can be written as a function of some s input bits (see Definition 7).By a simple counting argument (see Proposition 8), it follows that random functions are rigidwith good probability.We further extend this definition to what we call block-rigid functions (see Definition 11).For this, we’ll consider vectors x ∈ { , } nk , which are thought of as composed of k blocks,each of size n . We say that a function f : { , } nk → { , } nk is not an ( r, s )-block-rigidfunction, if there exists a subset X ⊆ { , } nk of size at least 2 nk − r , such that over X , each output block of f can be written as a function of some s input blocks .2e conjecture that it is possible to obtain large block-rigid functions, using smaller rigidfunctions. For a function f : { , } k → { , } k , we define the function f ⊗ n : { , } nk →{ , } nk as follows. For each x = ( x ij ) i ∈ [ n ] ,j ∈ [ k ] ∈ { , } nk , we define f ⊗ n ( x ) to be the vectorobtained by applying f to ( x i , . . . , x ik ), in place for each i ∈ [ n ] (see Definition 13). Conjecture 1.
There exists a universal constant c > such that the following is true. Let f : { , } k → { , } k be an ( r, s ) -rigid function, and n ∈ N . Then, f ⊗ n : { , } nk → { , } nk is a ( cnr, cs ) -block-rigid function. We prove the following theorem. It is restated and proved as Theorem 28 in Section 5.
Theorem 2.
Let t : N → N be any function such that t ( n ) = ω ( n ) . Assuming Conjecture ,there exists an (explicitly given) function f : { , } ∗ → { , } ∗ such that1. On inputs x of length n bits, the output f ( x ) is of length at most n bits.2. The function f is computable by a multi-tape deterministic Turing machine that runsin time O ( t ( n )) on inputs of length n .3. The function f is not computable by any multi-tape deterministic Turing machine thattakes advice and runs in time O ( n ) on inputs of length n . More generally, we show that families of block-rigid functions cannot be computed bynon-uniform Turing machines running in linear-time. This makes it interesting to find suchfamilies that are computable in polynomial time. The following theorem is restated as The-orem 29.
Theorem 3.
Let k : N → N be a function such that k ( n ) = ω (1) and k ( n ) = 2 o ( n ) , and f n : { , } nk ( n ) → { , } nk ( n ) be a family of ( ǫnk ( n ) , ǫk ( n )) -block-rigid functions, for someconstant ǫ > . Let M be any multi-tape deterministic linear-time Turing machine that takesadvice. Then, there exists n ∈ N , and x ∈ { , } nk ( n ) , such that M ( x ) = f n ( x ) . As another application, based on Conjecture 1, we show size-depth tradeoffs for booleancircuits, where the gates compute arbitrary functions over large (with respect to the inputsize) sets (see Section 6).
In a k -player game G , questions ( x , . . . , x k ) are chosen from some joint distribution µ . Foreach j ∈ [ k ], player j is given x j and gives an answer a j that depends only on x j . The playersare said to win if their answers satisfy a fixed predicate V ( x , . . . , x k , a , . . . , a k ). We notethat V might be randomized, that is, it might depend on some random string that is sampledindependently of ( x , . . . , x k ). The value of the game val( G ) is defined to be the maximumwinning probability over the possible strategies of the players.It is natural to consider the parallel repetition G ⊗ n of such a game G . Now, the questions( x ( i )1 , . . . , x ( i ) k ) are chosen from µ , independently for each i ∈ [ n ]. For each j ∈ [ k ], player j is given ( x (1) j , . . . , x ( n ) j ) and gives answers ( a (1) j , . . . , a ( n ) j ). The players are said to win if theanswers satisfy V ( x ( i )1 , . . . , x ( i ) k , a ( i )1 , . . . , a ( i ) k ) for every i ∈ [ n ]. The value of the game val( G ⊗ n )is defined to be the maximum winning probability over the possible strategies of the players.Note that the players are allowed to correlate their answers to different repetitions of thegame. 3arallel repetition of games was first studied in [FRS94], owing to its relation withmultiprover interactive proofs [BOGKW88]. It was hoped that the value val( G ⊗ n ) ofthe repeated game goes down as val( G ) n . However, this is not the case, as shown in[For89, Fei91, FV02, Raz11].A lot is known about parallel repetition of 2-player games. The, so called, parallel rep-etition theorem, first proved by Raz [Raz98] and further simplified and improved by Holen-stein [Hol09], shows that if val( G ) <
1, then val( G ⊗ n ) ≤ − Ω( n/s ) , where s is the length of theanswers given by the players. The bounds in this theorem were later made tight even for thecase when the initial game has small value (see [DS14] and [BG15]).Much less is known for k -player games with k ≥
3. Verbitsky [Ver96] showed that ifval( G ) <
1, then the value of the the repeated game goes down to zero as n grows larger. Theresult shows a very weak rate of decay, approximately equal to α ( n ) , where α is the inverse-Ackermann function, owing to the use of the density Hales-Jewett theorem (see [FK91] and[Pol12]). A recent result by Dinur, Harsha, Venkat and Yuen [DHVY17] shows exponentialdecay, but only in the special case of what they call expanding games . This approach failswhen the input to the players have strong correlations.In this paper (see Section 4), we show that a sufficiently strong parallel repetition theoremfor multiplayer games implies Conjecture 1. The following theorem is proved formally asTheorem 19 in Section 4. Theorem 4.
There exists a family {G S ,k } of k -player games (where S is some parameter),such that a strong parallel repetition theorem for all games in {G S ,k } implies Conjecture 1. Although the games in this family do not fit into the framework of [DHVY17], they satisfysome very special properties. Every k -player game in the family satisfies the following:1. The questions to the k -players are chosen as follows: First, k bits, x , . . . , x k ∈ { , } ,are drawn uniformly and independently. Each of the k -players sees some subset of these k -bits.2. The predicate V satisfies the condition that on fixing the bits x , . . . , x k , there is aunique accepting answer for each player (independently of all other answers) and theverifier accepts if every player answers with the accepting answer. We refer to gamesthat satisfy this property as independent games .We believe that these properties may allow us to prove strong upper bounds on the valueof parallel repetition of such games, despite our lack of understanding of multiplayer parallelrepetition. The bounds that we need are that parallel repetition reduces the value of suchgames from v to v Ω( n ) , where n is the number of repetitions (as is proved in [DS14] and [BG15]for 2-player games).
1. The main open problem is to make progress towards proving Conjecture 1, possiblyusing the framework of parallel repetition. The remarks after Theorem 28 mentionsome weaker statements that suffice for our applications. The examples of matrix-transpose and matrix-product in Section 8 also serve as interesting problems.2. Our techniques, which are based on [PPST83], heavily exploit the fact that the Turingmachines have one-dimensional tapes. Time-space lower bounds for satisfiability in4he case of multi-tape Turing machines with random access [FLvMV05], and Turingmachines with one d -dimensional tape [vMR05], are known. Extending such results tothe non-uniform setting is an interesting open problem.3. The question of whether a rigid-matrix A ∈ F n × n is rigid when seen as a function A : { , } n → { , } n is very interesting (see Section 7). This question is closely relatedto a Conjecture of Jukna and Schnitger [JS11] on the linearization of depth-2 circuits.This is also related to the question of whether data structures for linear problems canbe optimally linearized (see [DGW19]). We note that there are known examples oflinear problems for which the best known data-structures are non-linear, without anyknown linear data-structure achieving the same bounds (see [KU11]). Let N = { , , , . . . } be the set of all natural numbers. For any k ∈ N , we use [ k ] to denotethe set { , , . . . , k } .We use F to denote an arbitrary finite field, and F to denote the finite field on twoelements.Let x ∈ { , } k . For i ∈ [ k ], we use x i to denote the i th coordinate of x . For S ⊆ [ k ], wedenote by x | S the vector ( x i ) i ∈ S , which is the restriction of x to coordinates in S .We also consider vectors x ∈ { , } nk , for some n ∈ N . We think of these as composedof k blocks, each consisting of a vector in { , } n . That is, x = ( x ij ) i ∈ [ n ] ,j ∈ [ k ] . By abuse ofnotation, for S ⊆ [ k ], we denote by x | S the vector ( x ij ) i ∈ [ n ] ,j ∈ S , which is the restriction of x to the blocks indexed by S .Let A ∈ F nk × nk be an nk × nk matrix. We think of A as a block-matrix consisting of k blocks, each block being an n × n matrix. That is, A = ( A ij ) i,j ∈ [ k ] , where for all i, j ∈ [ k ], A ij ∈ F n × n . For each i ∈ [ k ], we call ( A ij ) j ∈ [ k ] the i th block-row of A .For every n ∈ N , we define log ∗ n = min { ℓ ∈ N ∪ { } : log log . . . log | {z } ℓ times n ≤ } . The concept of matrix rigidity was introduced by Valiant [Val77]. It is defined as follows.
Definition 5.
A matrix A ∈ F n × n is said to be an ( r, s ) -rigid matrix if it cannot be writtenas A = B + C , where B has rank at most r , and C has at most s non-zero entries in eachrow. Valiant [Val77] showed the existence of rigid matrices by a simple counting argument. Forthe sake of completeness, we include this proof.
Proposition 6.
For any constant < ǫ ≤ , and any n ∈ N , there exists a matrix A ∈ F n × n that is an ( ǫn, ǫn ) -rigid matrix.Proof. Fix any 0 < ǫ ≤ . We bound the number of n × n matrices that are not ( ǫn, ǫn )-rigidmatrices. 5. Any n × n matrix with rank at most r can be written as the product of an n × r and an r × n matrix. Hence, the number of matrices of rank at most ǫn is at most | F | ǫn ≤ | F | n .2. The number of matrices that have at most ǫn non-zero entries in each row is at most (cid:18)(cid:18) nǫn (cid:19) | F | ǫn (cid:19) n ≤ (cid:16) eǫ (cid:17) ǫn | F | ǫn = | F | ǫn (1+log | F | eǫ ) < | F | n . We used the binomial estimate (cid:0) nr (cid:1) ≤ (cid:0) enr (cid:1) r .Since each matrix that is not an ( r, s )-rigid matrix can be written as the sum of a matrixwith rank at most r , and a matrix with at most s non-zero entries in each row, the number ofmatrices that are not ( ǫn, ǫn )-rigid matrices is strictly less than | F | n · | F | n = | F | n , whichis the total number of n × n matrices.Observe that a matrix A ∈ F n × n is not an ( r, s )-rigid matrix if and only if there is asubspace X ⊆ F n of dimension at least n − r , and a matrix C with at most s non-zero entriesin each row, such that Ax = Cx for all x ∈ X .We use this formulation to extend the concept of rigidity to general functions. Definition 7.
A function f : { , } n → { , } n is said to be an ( r, s ) -rigid function iffor every subset X ⊆ { , } n of size at least n − r , and subsets S , . . . , S n ⊆ [ n ] of size s , and functions g , . . . , g n : { , } s → { , } , there exists x ∈ X such that f ( x ) =( g ( x | S ) , g ( x | S ) , . . . , g n ( x | S n )) . Using a similar counting argument as in Proposition 6, we show the existence of rigidfunctions.
Proposition 8.
For any constant < ǫ ≤ , and any (large enough) integer n , there existsa function f : { , } n → { , } n that is an ( ǫn, ǫn ) -rigid function.Proof. Fix any constant 0 < ǫ ≤ , and any integer n ≥ ǫ . We count the number of functions f : { , } n → { , } n that are not ( ǫn, ǫn )-rigid functions.1. Note that in Definition 7, it is without loss of generality to assume that | X | = 2 n − ǫn .The number subsets X ⊆ { , } n of size 2 n − ǫn is (cid:0) n n − ǫn (cid:1) ≤ ( e ǫn ) n − ǫn < ǫn n − ǫn ≤ n n − ǫn .
2. The number of subsets S , . . . , S n ⊆ [ n ] of size ǫn is at most (cid:0) nǫn (cid:1) n < n ǫn < n n − ǫn .3. The number of functions g , . . . , g n : { , } ǫn → { , } is at most 2 n ǫn < n n − ǫn .4. The number of choices for values of f on { , } n \ X is at most 2 n (2 n −| X | ) ≤ n (2 n − n − ǫn ) .Hence, the total number of functions f : { , } n → { , } n that are not ( ǫn, ǫn )-rigid functionsis strictly less than (cid:16) n n − ǫn (cid:17) n n − n n − ǫn < n n . .2 Block Rigidity In this section, we introduce the notion of block rigidity.
Definition 9.
A matrix A ∈ F nk × nk is said to be an ( r, s ) -block-rigid matrix if it cannot bewritten as A = B + C , where B has rank at most r , and C has at most s non-zero matricesin each block-row. Observe that if A ∈ F nk × nk is an ( r, ns )-rigid matrix, then it is also ( r, s )-block-rigidmatrix. Combining this with Proposition 6, we get the following. Observation 10.
For any constant < ǫ ≤ , and positive integers n, k , there exists an ( ǫnk, ǫk ) -block-rigid matrix A ∈ F nk × nk . Following the definition of rigid-functions in Section 3.1, we define block-rigid functionsas follows.
Definition 11.
A function f : { , } nk → { , } nk is said to be an ( r, s ) -block-rigid func-tion if for every subset X ⊆ { , } nk of size at least nk − r , and subsets S , . . . , S k ⊆ [ k ] of size s , and functions g , . . . , g k : { , } ns → { , } n , there exists x ∈ X such that f ( x ) = ( g ( x | S ) , g ( x | S ) , . . . , g k ( x | S k )) . Observe that if f : { , } nk → { , } nk is an ( r, ns )-rigid function, then it is also ( r, s )-block-rigid function. Combining this with Proposition 8, we get the following. Observation 12.
For any constant < ǫ ≤ , and (large enough) integers n, k , there existsan ( ǫnk, ǫk ) -block-rigid function f : { , } nk → { , } nk . Note that n = 1 in the definition of block-rigid matrices (functions) gives the usualdefinition of rigid matrices (functions). For our applications, we will mostly be interested inthe case when n is much larger than k . A natural question is whether there is a way to amplify rigidity. That is, given a rigidmatrix (function), is there a way to obtain a larger matrix (function) which is rigid, or evenblock-rigid.
Definition 13.
Let f : { , } k → { , } k be any function. Define f ⊗ n : { , } nk → { , } nk as following. Let x = ( x ij ) i ∈ [ n ] ,j ∈ [ k ] ∈ { , } nk and i ∈ [ n ] , j ∈ [ k ] . The ( i, j ) th coordinate of f ⊗ n ( x ) is defined to be the j th coordinate of f ( x i , x i , . . . , x ik ) . Basically, applying f ⊗ n on x ∈ { , } nk is the same as applying f on ( x i , x i , . . . , x ik ),in place for each i ∈ [ n ]. For a linear function given by matrix A ∈ F k × k , this operationcorresponds to A ⊗ I n , where I n is the n × n identity matrix, and ⊗ denotes the Kroneckerproduct of matrices.It is easy to see that if f is not rigid, then f ⊗ n is not block-rigid. Observation 14.
Suppose f : { , } k → { , } k is not an ( r, s ) -rigid function. Then f ⊗ n isnot an ( nr, s ) -block-rigid function. The converse of Observation 14 is more interesting. We believe that it is true, and restateConjecture 1 below.
Conjecture 15.
There exists a universal constant c > such that the following is true. Let f : { , } k → { , } k be an ( r, s ) -rigid function, and n ∈ N . Then, f ⊗ n : { , } nk → { , } nk is a ( cnr, cs ) -block-rigid function. Parallel Repetition
In this section, we show an approach to prove Conjecture 15 regarding rigidity amplification.This is based on proving a strong parallel repetition theorem for a k -player game.Fix some k ∈ N , a function f : { , } k → { , } k , an integer 1 ≤ s < k , and S =( S , . . . , S k ), where each S i ⊆ [ k ] is of size s . We define a k -player game G S as follows:The k -players choose functions g , . . . , g k : { , } s → { , } , which we call a strategy.A verifier chooses x , . . . , x k ∈ { , } uniformly and independently. Let x = ( x , . . . , x k ) ∈{ , } k . For each j ∈ [ k ], Player j is given the input x | S j , and they answer a j = g j ( x | S j ) ∈{ , } . The verifier accepts if and only if f ( x ) = ( a , . . . , a k ). The goal of the players is tomaximize the winning probability. Formally, the value of the game is defined asval( G S ) := max g ,...,g k Pr x ,...,x k ∈{ , } [ f ( x ) = ( g ( x | S ) , . . . , g k ( x | S k ))] . The n -fold repetition of G S , denoted by G ⊗ n S is defined as follows. The players choose astrategy g , . . . , g k : { , } ns → { , } n . The verifier chooses x , . . . , x k ∈ { , } n uniformlyand independently. Let x = ( x , . . . , x k ) ∈ { , } nk . Player j is given the input x | S j , andthey answer a j = g j ( x | S j ) ∈ { , } n . The verifier accepts if and only if f ⊗ n ( x ) = ( a , . . . , a k ).That is, for each i ∈ [ n ], j ∈ [ k ], the j th bit of f ( x i , . . . , x ik ) equals the i th bit of a j . Thevalue of this repeated game isval( G ⊗ n S ) := max g ,...,g k Pr x ,...,x k ∈{ , } n (cid:2) f ⊗ n ( x ) = ( g ( x | S ) , . . . , g k ( x | S k )) (cid:3) . From Definition 11, we get the following:
Observation 16.
Let f : { , } k → { , } k be a function, and n ∈ N . Then, f ⊗ n isan ( r, s ) -block-rigid function if and only if for every S = ( S , . . . , S k ) with set sizes as s ,val ( G ⊗ n S ) < − r .Proof. Let f ⊗ n be an ( r, s )-block-rigid function. Suppose, for the sake of contra-diction, that S = ( S , . . . , S k ) is such that val( G ⊗ n S ) ≥ − r . Let the functions g , . . . , g k : { , } ns → { , } n be an optimal strategy for the players. Define X := n x ∈ { , } nk | f ⊗ n ( x ) = ( g ( x | S ) , . . . , g k ( x | S k )) o . Then, | X | = val( G S ) · nk ≥ nk − r , whichcontradicts the block rigidity of f ⊗ n .Conversely, suppose that f ⊗ n is not ( r, s )-block-rigid. Then, there exists X ⊆ { , } nk with | X | ≥ nk − r , subsets S , . . . , S k ⊆ [ k ] of size s , and functions g , . . . , g k : { , } ns →{ , } n , such that for all x ∈ X , f ⊗ n ( x ) = ( g ( x | S ) , . . . , g ( x | S k )). Let S = ( S , . . . , S k ), andsuppose the players use strategy g , . . . , g k . Then, val( G ⊗ n S ) ≥ | X | · − nk ≥ − r .In particular, for n = 1, Observation 16 gives the following: Observation 17.
A function f : { , } k → { , } k is an ( r, s ) -rigid-function if and only iffor every S = ( S , . . . , S k ) with set sizes as s , val ( G S ) < − r . We conjecture the following strong parallel repetition theorem.
Conjecture 18.
There exists a constant c > such that the following is true. Let f : { , } k → { , } k be any function, and S = ( S , . . . , S k ) be such that for each i ∈ [ k ] , S i ⊆ [ k ] is of size s . Then, for all n ∈ N , val ( G ⊗ n S ) ≤ ( val ( G S )) cn . Combining Observation 16 and 17, we get the following:8 heorem 19.
Conjecture 18 = ⇒ Conjecture 15.
Remarks. (i) By looking only at some particular player, it can be shown that ifval ( G S ) < , then val ( G ⊗ n S ) ≤ − Ω( n ) . In fact, such a result holds for all independentgames . The harder part seems to be showing strong parallel repetition when the initialgame has small value.(ii) Observe that the game G S has a randomized predicate in the case ∪ kj =1 S j = [ k ] . Thiscondition can be removed (even for general independent games) by introducing a newplayer. This player is given the random string used by the verifier, and is always requiredto answer a single bit equal to zero. This maintains the independent game property, andensures that the predicate used by the verifier is a deterministic function of the vectorof input queries to the players. In this section, we show a conditional super-linear lower bound for multi-tape deterministicTuring machines that can take advice.Without loss of generality, we only consider machines that have a separate read-only inputtape. We assume that the advice string, which is a function of the input length, is writtenon a separate advice tape at the beginning of computation. We are interested in machinesthat compute multi-output functions. For this, we assume that at the end of computation,the machine writes the entire output on a separate write-only output tape, and then halts.We consider the following problem.
Definition 20.
Let k : N → N be a function. We define the problem Tensor k as follows:- Input: ( f, x ) , where f : { , } k → { , } k is a function, and x ∈ { , } nk , for some n ∈ N and k = k ( n ) .- Output: f ⊗ n ( x ) ∈ { , } nk .The function f is given as input in the form of its entire truth table. The input x =( x ij ) i ∈ [ n ] ,j ∈ [ k ] is given in the order ( x , . . . , x n , x , . . . , x n , . . . , x k , . . . , x nk ) . The totallength of the input is m ( n ) := 2 k k + nk . We observe that if the function k : N → N grows very slowly with n , the problem Tensor k can be solved by a deterministic Turing machine in slightly super-linear time. Observation 21.
Let k : N → N be a function. There exists a deterministic Turing machinethat solves the problem Tensor k in time O ( nk k ) , on input ( f, x ) of length nk + 2 k k , where k = k ( n ) .Proof. We note that applying f ⊗ n on x = ( x ij ) i ∈ [ n ] ,j ∈ [ k ] is the same as applying f on( x i , x i , . . . , x ik ), in place for each i ∈ [ n ]. A Turing machine can do the following:1. Find n and k , using the fact that the description of f is of length 2 k k , and that of x isof length nk .2. Rearrange the input so that for each i ∈ [ n ], the part ( x i , x i , . . . , x ik ) is writtenconsecutively on the tape. 9. Using the truth table of f , compute the output for each such part.4. Rearrange the entire output back to the desired form.The total time taken is O ( nk + nk + nk k + nk ) = O ( nk k ).We now state and prove the main technical theorem of this section. Theorem 22.
Let k : N → N be a function such that k ( n ) = ω (1) and k ( n ) = o (log n ) . Let m = m ( n ) := 2 k k + nk , where k = k ( n ) .Suppose M is a deterministic multi-tape Turing machine that takes advice, and runsin linear time in the length of its input. Assuming Conjecture 15, the machine M doesnot solve the problem Tensor k correctly for all inputs. That is, there exists n ∈ N , and y = ( f, x ) ∈ { , } m such that M ( y ) = f ⊗ n ( x ) . There are two main technical ideas that will be useful to us. The first is the notion of block-respecting Turing machines, defined by Hopcroft, Paul and Valiant [HPV77]. The secondis a graph theoretic result, which was proven by Paul, Pippenger, Szemer´edi and Trotter[PPST83], and was used to show a separation between deterministic and non-deterministiclinear time.
Definition 23.
Let M be a Turing machine, and let b : N → N be a function. Partition thecomputation of M , on any input y of length m , into time segments of length b ( m ) , with thelast segment having length at most b ( m ) . Also, partition each of the tapes of M into blocks ,each consisting of b ( m ) contiguous cells.We say that M is block-respecting with respect to block size b , if on inputs of length m ,the tape heads of M cross blocks only at times that are integer multiples of b ( m ) . Lemma 24. [HPV77] Let t : N → N be a function, and M be a multi-tape deterministicTuring machine running in time t ( m ) on inputs of length m . Let b : N → N be a functionsuch that b ( m ) is computable from m by a multi-tape deterministic Turing machine runningin time O ( t ( m )) . Then, the language recognized by M is also recognized by a multi-tapedeterministic Turing Machine M ′ , which runs in time O ( t ( m )) , and is block-respecting withrespect to b . The rest of this section is devoted to the proof of Theorem 22. Let k : N → N be afunction such that k ( n ) = ω (1) and k ( n ) = o (log n ). Let m = m ( n ) := 2 k k + nk , where k = k ( n ).Suppose that M is a deterministic multi-tape Turing Machine, which on input y = ( f, x ) ∈{ , } m , takes advice, runs in time O ( m ), and outputs f ⊗ n ( x ).Let b : N → N be a function such that b ( m ) = n . By our assumption that k ( n ) = o (log n ),we can assume that the input y = ( f, x ) ∈ { , } m consists of k + 1 blocks, where the firstblock contains f (possibly padded by blank symbols to the left), and the remaining k blockscontain x . By Lemma 24, we can further assume that M is block-respecting with respect to b . Note that we can assume n to be a part of the advice, and hence we don’t need to careabout the computability of b .For an input y = ( f, x ) ∈ { , } m , the number of time segments for which M runs on x isat most O ( m ) b ( m ) = O ( nk ) n = O ( k ) := a ( n ).We define the computation graph G M ( y ), for input y ∈ { , } m , as follows. Definition 25.
The vertex set of G M ( y ) is defined to be V M ( y ) = { , . . . , a ( n ) } . For each ≤ i < j ≤ a ( n ) , the edge set E M ( y ) has the edge ( i, j ) , if either i) j = i + 1 , or(ii) there is some tape, such that, during the computation on y , M visits the same blockon that tape in both time-segments i and j , and never visits that block during anytime-segment strictly between i and j . In a directed acyclic graph G , we say that a vertex u is a predecessor of a vertex v , ifthere exists a directed path from u to v . Lemma 26. [PPST83] For every y , the graph G M ( y ) satisfies the following:1. Each vertex in G M ( y ) has degree O (1) .2. There exists a set of vertices J ⊂ V M ( y ) in G M ( y ) , of size O (cid:16) a ( n )log ∗ a ( n ) (cid:17) such that everyvertex of G M ( y ) has at most O (cid:16) a ( n )log ∗ a ( n ) (cid:17) many predecessors in the induced subgraph onthe vertex set V M ( y ) \ J .We note that the constants here might depend on the number of tapes of M . Lemma 27.
Let ǫ > be any constant and Y ⊆ { , } m be any subset of the inputs. Forall (large enough) n , there exists a subset Y ⊆ Y of size | Y | ≥ |Y | · − ǫnk , and subsets S , . . . , S k ⊆ [ k ] of size ǫk , such that for each y = ( f, x ) ∈ Y , and each i ∈ [ k ] , the i th block(of length n ) of f ⊗ n ( x ) can be written as a function of x | S i and the truth-table of f .Proof. For input y = ( f, x ) ∈ { , } m , let J ( y ) ⊂ V M ( y ) be a set as in Lemma 26.Let C ( y ) denote the following information about the computation of M :(i) The internal state of M at the end of each time-segment.(ii) The position of all tape heads at the end of each time-segment.(iii) For each time segment in J ( y ), and for each tape of M , the final transcription (of length n ) of the block that was visited on this tape during this segment.Let g : Y → { , } ∗ be the function given by g ( y ) = ( G M ( y ) , J ( y ) , C ( y )). Observe thatthe output of g can be described using O (cid:16) k log k + nk log ∗ k (cid:17) bits. By our assumption that k ( n ) = ω (1) and k ( n ) = o (log n ), we have that for large n , this is at most ǫnk bits. Hence,there exists a set Y ⊆ Y of size | Y | ≥ |Y | · − ǫnk , such that for each y ∈ Y , g ( y ) takes onsome fixed value ( G = ( V, E ) , J, C ).Now, consider any y = ( f, x ) ∈ Y . The machine writes the k blocks of the output f ⊗ n ( x ) on the output tape in the last k time segments before halting. For each of these timesegments, the corresponding vertex in G has at most O (cid:16) k log ∗ k (cid:17) ≤ ǫk predecessors in theinduced subgraph on V \ J . These further correspond to at most ǫk distinct blocks of y thatare visited (on the input tape) during these predecessor time segments. Since the relevantblock transcriptions at the end of time segments for vertices in J are fixed in C , each outputblock can be written as a function of at most ǫk blocks of y . For the i th block of output,without loss of generality, this includes the first block of y , which contains the truth table of f , and blocks of x which indexed by some subset S i ⊆ [ k ] of size ǫk .11 roof of Theorem 22. Let δ = . Fix some sufficiently large n , and a ( δk, δk )-rigid function f : { , } k → { , } k . The existence of such a function is guaranteed by Proposition 8. ByConjecture 15, the function f ⊗ n is an ( ǫnk, ǫk )-block-rigid function for some constant ǫ > Y = n ( f , x ) : x ∈ { , } nk o , shows that f ⊗ n is not an( ǫnk, ǫk )-block-rigid function for any constant ǫ > Theorem 28.
Let t : N → N be any function such that t ( n ) = ω ( n ) . Assuming Conjecture , there exists a function f : { , } ∗ → { , } ∗ such that1. On inputs x of length n bits, the output f ( x ) is of length at most n bits.2. The function f is computable by a multi-tape deterministic Turing machine that runsin time O ( t ( n )) on inputs of length n .3. The function f is not computable by any multi-tape deterministic Turing machine thattakes advice and runs in time O ( n ) on inputs of length n .Proof. Let k : N → N be a function such that k ( n ) = ω (1), k ( n ) = o (log n ), and nk k ≤ t (2 k k + nk ). The theorem then follows from Observation 21 and Theorem 22. Remarks. (i) We note that for the proof of Theorem 22, it suffices to find, for infinitelymany n , a single function f : { , } k ( n ) → { , } k ( n ) such that f ⊗ n is an ( ǫnk, ǫk ) -block-rigid function, where ǫ > is a constant. This would show that M cannot give thecorrect answer to Tensor k for inputs of the form ( f, x ) , where x ∈ { , } nk .(ii) For the proof of Theorem 22, it can be shown that it suffices for the following conditionto hold for infinitely many n , and some constant ǫ > . Let S , . . . , S k ⊆ [ k ] be fixedsets of size ǫk , and f : { , } k → { , } k be a function chosen uniformly at random.Then, with probability at least − − ω ( k log k ) , the function f ⊗ n is an ( ǫnk, ǫk ) -block-rigidfunction against the fixed sets S , . . . , S k . We note that the probability here is not goodenough to be able to union bound over S , . . . , S k and get a single function as mentionedin the previous remark. Essentially the same argument as that of Theorem 22 also proves Theorem 3, which werestate below.
Theorem 29.
Let k : N → N be a function such that k ( n ) = ω (1) and k ( n ) = 2 o ( n ) , and f n : { , } nk ( n ) → { , } nk ( n ) be a family of ( ǫnk ( n ) , ǫk ( n )) -block-rigid functions, for someconstant ǫ > . Let M be any multi-tape deterministic linear-time Turing machine that takesadvice. Then, there exists n ∈ N , and x ∈ { , } nk ( n ) , such that M ( x ) = f n ( x ) . The above theorem makes it interesting to find families of block-rigid functions that arecomputable in polynomial time.
In this section, we will consider boolean circuits over a set F . These are directed acyclicgraphs with each node v labelled either as an input node or by an arbitrary function g v : F × F → F . The input nodes have in-degree 0 and all other nodes have in-degree 2. Some12odes are further labelled as output nodes, and they compute the outputs (in the usualmanner), when the inputs are from the set F . The size of the circuit is defined to be thenumber of edges in the graph. The depth of the circuit is defined to be the length of a longestdirected path from an input node to an output node.Valiant [Val77] showed that if A ∈ F n × n is an ( ǫn, n ǫ )-rigid matrix for some constant ǫ > O ( n )-size and O (log n )-depthlinear circuit over F . By a linear circuit, we mean that each gate computes a linear function(over F ) of its inputs. A similar argument can be used to prove the following. Lemma 30. [Val77] Suppose f : { , } nk → { , } nk is an ( ǫnk, k ǫ ) -block-rigid function, forsome constant ǫ > . Then, the function g : ( { , } n ) k → ( { , } n ) k corresponding to f cannot be computed by an O ( k ) -size and O (log k ) -depth circuit over the set F = { , } n . Theorem 31.
Let k : N → N be a function such that k ( n ) = ω (1) and k ( n ) = o (log n ) . Let m = m ( n ) := 2 k k + nk , where k = k ( n ) .Assuming Conjecture 15, the problem Tensor k is not solvable by O ( k ) -size and O (log k ) -depth circuits over the set F = { , } n . Here, the input ( f, x ) to the circuit is given in theform of k + 1 elements in { , } n , the first one being the truth table of f , and the remaining k being the blocks of x .Proof. Let δ = . Fix some large n , and a ( δk, δk )-rigid function f : { , } k → { , } k ,where k = k ( n ). The existence of such a function is guaranteed by Proposition 8. AssumingConjecture 15, the function f ⊗ n is an ( ǫnk, ǫk )-block-rigid function, for some universal con-stant ǫ >
0. By Lemma 30, the corresponding function on ( { , } n ) k cannot be computedby an O ( k )-size and O (log k )-depth circuit over F = { , } n . Since f can be hard-wired inany circuit solving Tensor k , we have the desired result. A natural question to ask is whether the functions corresponding to rigid matrices are rigidfunctions or not.
Conjecture 32.
There exists a universal constant c > such that whenever A ∈ F n × n is an ( r, s ) -rigid matrix, the corresponding function A : F n → F n is a ( cr, cs ) -rigid function. We show that a positive answer to the above resolves a closely related conjecture by Juknaand Schnitger [JS11].
Definition 33.
Consider a depth-2 circuit, with x = ( x , . . . , x n ) as the input variables, w gates in the middle layer, computing boolean functions h , . . . , h w and m output gates,computing boolean functions g , . . . , g m . The circuit computes a function f = ( f , . . . , f m ) : F n → F m satisfying f i ( x . . . , x n ) = g i ( x, h ( x ) , . . . , h w ( x )) , for each i ∈ [ m ] . The width ofthe circuit is defined to be w . The degree of the circuit is defined to be the maximum over allgates g i , of the number of wires going directly from the inputs x , . . . , x n to g i . We remark that Lemma 27 essentially shows that any function computable by a determin-istic linear-time Turing Machine has a depth-2 circuit of small width and small ‘block-degree’.
Conjecture 34. [JS11] Suppose f : F n → F n is a linear function computable by a depth-2circuit with width w and degree d . Then, f is computable by a depth-2 circuit, with width O ( w ) , and degree O ( d ) , each of whose gates compute a linear function. bservation 35. Conjecture 32 = ⇒ Conjecture 34.Proof.
Suppose f : F n → F n is a linear function computable by a depth-2 circuit with width w and degree d . Then, there exists a set X ⊆ F n of size at least 2 n − w , such that for each x ∈ X , the value of the functions computed by the gates in the middle layer is the same.Hence, for each x ∈ X , each element of f ( x ) can be written as a function of at most d elements of x . This shows that f is not an ( w, d )-rigid function. Assuming Conjecture 32,the matrix A ∈ F n × n for the function f is not a ( cw, cd )-rigid matrix, for some constant c > A = B + C , where the rank of B is at most cw , and C has at most cd non-zero entriesin each row. Now, there exist matrices B ∈ F n × cw and B ∈ F cw × n , such that B = B B .Then, f is computable by a linear depth-2 circuit with width cw and degree cd , where themiddle layer computes output of the function corresponding to B . In this section, we state some well known problems related to matrices. It seems interestingto study the block-rigidity of the these functions.
The matrix-transpose problem is described as follows:-
Input:
A matrix X ∈ F n × n as a vector of length n bits, in row-major order, for some n ∈ N .- Output:
The matrix X column-major order (or equivalently, the transpose of X inrow-major order).It is well known (see [DMS91] for a short proof) that the above problem can be solved ona 2-tape Turing machine in time O ( N log N ), on inputs of length N = n . We believe thatthis cannot be solved by Turing machines in linear-time, and that the notion of block-rigiditymight be a viable approach to prove this. Next, we observe some structural details aboutthe problem.The matrix-transpose problem computes a linear function, whose N × N matrix A oninputs of length N = n is described as follows. For each i, j ∈ [ n ], let e ij ∈ F n × n denote thematrix whose ( i, j ) th entry is 1 and rest of the entries are zero. The matrix A is an N × N matrix made up of n blocks, with the ( i, j ) th block equal to e ji .Using a similar argument as in Observation 16, one can show that the value of the followinggame captures the block-rigidity of the matrix-transpose function. Fix integers n ∈ N ,1 ≤ s < n , and a collection S = ( S , . . . , S n ), where each S i ⊆ [ n ] is of size s . We define an n -player game G S as follows: A verifier chooses a matrix X ∈ F n × n , with each entry beingchosen uniformly and independently. For each j ∈ [ n ], player j is given the rows of the matrixindexed by S j , and they answer y j ∈ F n . The verifier accepts if and only if for each j ∈ [ n ], y j equals the j th column of X . Conjecture 36.
There exists a constant c > such that the function given by the matrix A is a ( cn , cn ) -block-rigid function. Equivalently, for each collection S with set sizes as cn ,the value of the game G S is at most − cn .
14e note that the above game is of independent interest from a combinatorial point ofview as well. Basically, it asks whether there exists a large family of n × n matrices, in whicheach column can be represented as some function of a small fraction of the rows. The problemof whether the matrix A is a block-rigid matrix is also interesting. This corresponds to theplayers in the above game using strategies which are linear functions. The matrix-product problem is described as follows:-
Input:
Matrices
X, Y ∈ F n × n as vectors of length n bits, in row-major order, for some n ∈ N .- Output:
The matrix Z = XY in row-major order.The block-rigidity of the matrix-product function is captured by the following game: Fixintegers n ∈ N , 1 ≤ s < n , and collections S = ( S , . . . , S n ), T = ( T , . . . , T n ) where each S i , T i ⊆ [ n ] is of size s . We define a n -player game G S , T as follows: A verifier chooses matrices X, Y ∈ F n × n , with each entry being chosen uniformly and independently. For each j ∈ [ n ],player j is given the rows of the matrices X and Y indexed by S j and T j respectively, andthey answer y j ∈ F n . The verifier accepts if and only if for each j ∈ [ n ], y j equals the j th rowof XY . Conjecture 37.
There exists a constant c > such that for each S , T with set sizes as cn ,the value of the game G S , T is at most − cn . One may change the row-major order for some (or all) of the matrices to column-majororder. It is easy to modify the above game in such a case.
References [AC19] Josh Alman and Lijie Chen. Efficient construction of rigid matrices using anNP oracle. In
FOCS , pages 1034–1055, 2019. 2[AW17] Josh Alman and Ryan Williams. Probabilistic rank and matrix rigidity. In
STOC , pages 641–652, 2017. 2[BG15] Mark Braverman and Ankit Garg. Small value parallel repetition for generalgames. In
STOC , pages 335–340, 2015. 4[BHPT20] Amey Bhangale, Prahladh Harsha, Orr Paradise, and Avishay Tal. Rigid ma-trices from rectangular PCPs. In
FOCS , 2020. accepted. 2[BOGKW88] Michael Ben-Or, Shafi Goldwasser, Joe Kilian, and Avi Wigderson. Multi-prover interactive proofs: How to remove intractability assumptions. In
STOC ,pages 113–131, 1988. 4[DGW19] Zeev Dvir, Alexander Golovnev, and Omri Weinstein. Static data structurelower bounds imply rigidity. In
STOC , pages 967–978, 2019. 515DHVY17] Irit Dinur, Prahladh Harsha, Rakesh Venkat, and Henry Yuen. Multiplayerparallel repetition for expanding games. In
ITCS , volume 67 of
LIPIcs , pagesArt. No. 37, 16. 2017. 4[DMS91] Martin Dietzfelbinger, Wolfgang Maass, and Georg Schnitger. The complexityof matrix transposition on one-tape off-line Turing machines.
Theoret. Comput.Sci. , 82:113–129, 1991. 14[DS14] Irit Dinur and David Steurer. Analytical approach to parallel repetition. In
STOC , pages 624–633, 2014. 4[Fei91] Uriel Feige. On the success probability of the two provers in one-round proofsystems. In
Structure in Complexity Theory Conference , pages 116–123, 1991.4[FK91] H. Furstenberg and Y. Katznelson. A density version of the Hales-Jewetttheorem.
J. Anal. Math. , 57:64–119, 1991. 4[FLvMV05] Lance Fortnow, Richard Lipton, Dieter van Melkebeek, and Anastasios Viglas.Time-space lower bounds for satisfiability.
J. ACM , 52(6):835–865, 2005. 5[For89] Lance Jeremy Fortnow.
Complexity theoretic aspects of interactive proof sys-tems . 1989. Ph.D. Thesis, MIT. 4[Fri93] Joel Friedman. A note on matrix rigidity.
Combinatorica , 13(2):235–239, 1993.2[FRS94] Lance Fortnow, John Rompel, and Michael Sipser. On the power of multi-prover interactive protocols.
Theoret. Comput. Sci. , 134(2):545–557, 1994. 4[FV02] Uriel Feige and Oleg Verbitsky. Error reduction by parallel repetition – anegative result.
Combinatorica , 22(4):461–478, 2002. 4[GT18] Oded Goldreich and Avishay Tal. Matrix rigidity of random Toeplitz matrices.
Comput. Complexity , 27(2):305–350, 2018. (also in STOC 2016). 2[Hol09] Thomas Holenstein. Parallel repetition: simplifications and the no-signalingcase.
Theory Comput. , 5:141–172, 2009. (also in STOC 2007). 4[HPV77] John Hopcroft, Wolfgang Paul, and Leslie Valiant. On time versus space.
J.Assoc. Comput. Mach. , 24(2):332–337, 1977. (also in FOCS 1975). 2, 10[HS65] J. Hartmanis and R. E. Stearns. On the computational complexity of algo-rithms.
Trans. Amer. Math. Soc. , 117:285–306, 1965. 2[JS11] Stasys Jukna and Georg Schnitger. Min-rank conjecture for log-depth circuits.
J. Comput. System Sci. , 77(6):1023–1038, 2011. 5, 13[KLPS14] Abhinav Kumar, Satyanarayana V. Lokam, Vijay M. Patankar, and M. N. Jay-alal Sarma. Using elimination theory to construct rigid matrices.
Comput.Complexity , 23(4):531–563, 2014. (also in FSTTCS 2009). 216KU11] Kiran S. Kedlaya and Christopher Umans. Fast polynomial factorization andmodular composition.
SIAM J. Comput. , 40(6):1767–1802, 2011. (also inSTOC 2008). 5[Lok00] Satyanarayana V. Lokam. On the rigidity of Vandermonde matrices.
Theoret.Comput. Sci. , 237(1-2):477–483, 2000. 2[Lok01] Satyanarayana V. Lokam. Spectral methods for matrix rigidity with appli-cations to size-depth trade-offs and communication complexity.
J. Comput.System Sci. , 63(3):449–473, 2001. 2[Lok06] Satyanarayana V. Lokam. Quadratic lower bounds on matrix rigidity. In
Theoryand Applications of Models of Computation , pages 295–307, 2006. 2[Pol12] D. H. J. Polymath. A new proof of the density Hales-Jewett theorem.
Ann. ofMath. (2) , 175(3):1283–1327, 2012. 4[PPST83] Wolfgang J. Paul, Nicholas Pippenger, Endre Szemeredi, and William T. Trot-ter. On determinism versus non-determinism and related problems. In
FOCS ,pages 429–438, 1983. 1, 2, 4, 10, 11[PR80] Wolfgang Paul and R¨udiger Reischuk. On alternation. II. A graph-theoreticapproach to determinism versus nondeterminism.
Acta Inform. , 14(4):391–403,1980. 2[Raz89] Alexander Razborov. On rigid matrices (in russian). Unpublished Manuscript,1989. 2[Raz98] Ran Raz. A parallel repetition theorem.
SIAM J. Comput. , 27(3):763–803,1998. (also in STOC 1995). 4[Raz11] Ran Raz. A counterexample to strong parallel repetition.
SIAM J. Comput. ,40(3):771–777, 2011. (also in FOCS 2008). 4[San01] Rahul Santhanam. On separators, segregators and time versus space. In
CCC ,pages 286–294, 2001. 2[SS97] Victor Shoup and Roman Smolensky. Lower bounds for polynomial evaluationand interpolation problems.
Comput. Complexity , 6(4):301–311, 1996/97. 2[SSS97] M. A. Shokrollahi, D. A. Spielman, and V. Stemann. A remark on matrixrigidity.
Inform. Process. Lett. , 64(6):283–285, 1997. 2[Val77] Leslie G. Valiant. Graph-theoretic arguments in low-level complexity. In
MFCS ,volume 53 of
Lecture Notes in Computer Science , pages 162–176, 1977. 1, 2,5, 13[Ver96] Oleg Verbitsky. Towards the parallel repetition conjecture.
Theoret. Comput.Sci. , 157(2):277–282, 1996. 4[vMR05] Dieter van Melkebeek and Ran Raz. A time lower bound for satisfiability.
Theoret. Comput. Sci. , 348(2-3):311–320, 2005. (also in ICALP 2004). 5[Wun12] Henning Wunderlich. On a theorem of Razborov.