Expressive power of linear algebra query languages
Floris Geerts, Thomas Muñoz, Cristian Riveros, Domagoj Vrgoč
EExpressive power of linear algebra query languages
Floris Geerts
University of [email protected]
Thomas Muñoz
PUC Chile and IMFD [email protected]
Cristian Riveros
PUC Chile and IMFD [email protected]
Domagoj Vrgoč
PUC Chile and IMFD [email protected]
ABSTRACT
Linear algebra algorithms often require some sort of iteration orrecursion as is illustrated by standard algorithms for Gaussian elimi-nation, matrix inversion, and transitive closure. A key characteristicshared by these algorithms is that they allow looping for a numberof steps that is bounded by the matrix dimension. In this paper weextend the matrix query language
MATLANG with this type of recur-sion, and show that this suffices to express classical linear algebraalgorithms. We study the expressive power of this language andshow that it naturally corresponds to arithmetic circuit families,which are often said to capture linear algebra. Furthermore, weanalyze several sub-fragments of our language, and show that theirexpressive power is closely tied to logical formalisms on semiring-annotated relations.
Linear algebra-based algorithms have become a key componentin data analytic workflows. As such, there is a growing interest inthe database community to integrate linear algebra functionalitiesinto relational database management systems [5, 23, 25–27]. Inparticular, from a query language perspective, several proposalshave recently been put forward to unify relational algebra and linearalgebra. Two notable examples of this are:
LARA [22], a minimalisticlanguage in which a number of atomic operations on associativetables are proposed, and
MATLANG , a query language for matrices [7].Both
LARA and
MATLANG have been studied by the database theorycommunity, showing interesting connections to relational algebraand logic. For example, fragments of
LARA are known to capturefirst-order logic with aggregation [4], and
MATLANG has been re-cently shown to be equivalent to a restricted version of the (posi-tive) relational algebra on 𝐾 -relations, RA + 𝐾 [8], where 𝐾 denotes asemiring. On the other hand, some standard constructions in linearalgebra are out of reach for these languages. For instance, it wasshown that under standard complexity-theoretic assumptions, LARA can not compute the inverse of a matrix or its determinant [4], andoperations such as the transitive closure of a matrix are known tobe inexpressible in
MATLANG [7]. Given that these are fundamentalconstructs in linear algebra, one might wonder how to extend
LARA or MATLANG in order to allow expressing such properties.One approach would be to add these constructions explicitly tothe language. Indeed, this was done for
MATLANG in [7], and
LARA in [4]. In these works, the authors have extended the core languagewith the trace, the inverse, the determinant, or the eigenvectorsoperators and study the expressive power of the result. However,one can argue that there is nothing special in these operators, apart they have been used historically in linear algebra textbooks andthey extend the expressibility of the core language. The questionhere is whether these new operators form a sound and naturalchoice to extend the core language, or are they just some particularqueries that we would like to support.In this paper we take a more principled approach by studyingwhat are the atomic operations needed to define standard linearalgebra algorithms. Inspecting any linear algebra textbook, onesees that most linear algebra procedures heavily rely on the use offor-loops in which iterations happen over the dimensions of thematrices involved. To illustrate this, let us consider the example ofcomputing the transitive closure of a graph. This can be done usinga modification of the Floyd-Warshall algorithm [10], which takesas its input an 𝑛 × 𝑛 adjacency matrix 𝐴 representing our graph,and operates according to the following pseudo-code: for 𝑘 = ..𝑛 dofor 𝑖 = ..𝑛 dofor 𝑗 = ..𝑛 do 𝐴 [ 𝑖, 𝑗 ] : = 𝐴 [ 𝑖, 𝑗 ] + 𝐴 [ 𝑖, 𝑘 ] · 𝐴 [ 𝑘, 𝑗 ] After executing the algorithm, all of the non zero entries signify anedge in the (irreflexive) transitive closure graph.By examining standard linear algebra algorithms such as Gauss-ian elimination, 𝐿𝑈 -decomposition, computing the inverse of amatrix, or its determinant, we can readily see that this pattern con-tinues. Namely, we observe that there are two main components tosuch algorithms: (i) the ability to iterate up to the matrix dimension;and (ii) the ability to access a particular position in our matrix. Inorder to allow this behavior in a query language, we propose toextend MATLANG with limited recursion in the form of for-loops,resulting in the language for - MATLANG . To simulate the two com-ponents of standard linear algebra algorithms in a natural way, wesimulate a loop of the form for 𝑖 = ..𝑛 do by leveraging canonicalvectors. In other words, we use the canonical vectors 𝑏 = ( , , . . . ) , 𝑏 = ( , , . . . ) , . . . , to access specific rows and columns, and iterateover these vectors. In this way, we obtain a language able to com-pute important linear algebra operators such as 𝐿𝑈 -decomposition,determinant, matrix inverse, among other things.Of course, a natural question to ask now is whether this reallyresults in a language suitable for linear algebra? We argue that thecorrect way to approach this question is to compare our languageto arithmetic circuits, which have been shown to capture the vastmajority of existing matrix algorithms, from basic ones such ascomputing the determinant and the inverse, to complex proceduressuch as discrete Fourier transformation, and Strassen’s algorithm a r X i v : . [ c s . CC ] O c t onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč (see [1, 30] for an overview of the area), and can therefore be con-sidered to effectively capture linear algebra. In the main technicalresult of this paper, we show that for - MATLANG indeed computesthe same class of functions over matrices as the ones computed byarithmetic circuit families of bounded degree. As a consequence, for - MATLANG inherits all expressiveness properties of circuits, andthus can simulate any linear algebra algorithm definable by circuits.Having established that for - MATLANG indeed provides a goodbasis for a linear algebra language, we move to a more fine-grainedanalysis of the expressiveness of its different fragments. For this,we aim to provide a connection with logical formalisms, similarlyas was done by linking
LARA and
MATLANG to first-order logic withaggregates [4, 7]. As we show, capturing different logics correspondto restricting how matrix variables are updated in each iteration ofthe for-loops allowed in for - MATLANG . For instance, if we only allowto add some temporary result to a variable in each iteration (in-stead of rewriting it completely like in any programming language),we obtain a language, called sum - MATLANG , which is equivalent to RA + 𝐾 , directly extending an analogous result shown for MATLANG ,mentioned earlier [8]. We then study updating matrix variablesbased on another standard linear algebra operator, the Hadamardproduct, resulting in a fragment called FO - MATLANG , which we showto be equivalent to weighted logics [13]. Finally, in prod - MATLANG we update the variables based on the standard matrix product, andlink this fragment to the ones discussed previously.
Contribution and outline. • After we recall
MATLANG in Section 2, we show in Section 3 howfor-loops can be added to
MATLANG in a natural way. We alsoobserve that for - MATLANG strictly extends
MATLANG . In addi-tion, we discuss some design decisions behind the definition of for - MATLANG , noting that our use of canonical vectors resultsin the availability of an order relation. • In Section 4 we show that for - MATLANG can compute importantlinear algebra algorithms in a natural way. We provide expres-sions in for - MATLANG for LU decomposition (used to solve linearsystems of equations), the determinant and matrix inversion. • More generally, in Section 5 we report our main technical con-tribution. We show that every uniform arithmetic circuits ofpolynomial degree correspond to a for - MATLANG expression,and vice versa, when a for - MATLANG expression has polynomialdegree, then there is an equivalent uniform family of arithmeticcircuits. As a consequence, for - MATLANG inherits all expressive-ness properties of such circuits. • Finally, in Section 6 we generalize the semantics of for - MATLANG to matrices with values in a semiring 𝐾 , and show that two nat-ural fragment of for - MATLANG , sum - MATLANG , and FO - MATLANG ,are equivalent to the (positive) relational algebra and weightedlogics on binary 𝐾 -relations, respectively. We also briefly com-ment on a minimal fragment of for - MATLANG , based on prod - MATLANG , that is able to compute matrix inversion.Due to space limitations, most proofs are referred to the appendix.
Related work.
We already mentioned
LARA [22] and
MATLANG [7]whose expressive power was further analyzed in [4, 8, 15, 16]. Ex-tensions of
SQL for matrix manipulations are reported in [27]. Mostrelevant is [23] in which a recursion mechanism is added to
SQL which resembles for-loops. The expressive power of this extension is unknown, however. Classical logics with aggregation [20] andfixed-point logics with counting [19] can also be used for linearalgebra. More generally, for the descriptive complexity of linearalgebra we refer to [12, 21]. Most of these works require to encodereal numbers inside relations, whereas we treat real numbers asatomic values. We refer to relevant papers related to arithmeticcircuits and logical formalisms on semiring-annotated relations inthe corresponding sections later in the paper.
We start by recalling the matrix query language
MATLANG , intro-duced in [7], which serves as our starting point.
Syntax.
Let V = { 𝑉 , 𝑉 , . . . } be a countably infinite set of matrixvariables and F = (cid:208) 𝑘 > F 𝑘 with F 𝑘 a set of functions of the form 𝑓 : R 𝑘 → R , where R denotes the set of real numbers. The syntaxof MATLANG expressions is defined by the following grammar : 𝑒 :: = 𝑉 ∈ V (matrix variable) | 𝑒 𝑇 (transpose) | ( 𝑒 ) (one-vector) | diag ( 𝑒 ) (diagonalization of a vector) | 𝑒 · 𝑒 (matrix multiplication) | 𝑒 + 𝑒 (matrix addition) | 𝑒 × 𝑒 (scalar multiplication) | 𝑓 ( 𝑒 , . . . , 𝑒 𝑘 ) (pointwise application of 𝑓 ∈ F 𝑘 ). MATLANG is parametrized by a collection of functions F but inthe remainder of the paper we only make this dependence explicit,and write MATLANG [F ] , for some set F of functions, when thesefunctions are crucial for some results to hold. When we simplywrite MATLANG , we mean that any function can be used (includingnot using any function at all).
Schemas and typing.
To define the semantics of
MATLANG expres-sions we need a notion of schema and well-typedness of expressions.A
MATLANG schema S is a pair S = (M , size ) , where M ⊂ V isa finite set of matrix variables, and size : M ↦→
Symb × Symb isa function that maps each matrix variable in M to a pair of sizesymbols . The size function helps us determine whether certain ma-trix operations, such as matrix multiplication, can be performed formatrices adhering to a schema. We denote size symbols by Greekletters 𝛼, 𝛽,𝛾 . We also assume that ∈ Symb . To help us determinewhether a
MATLANG expression can always be evaluated, we definethe type of an expression 𝑒 , with respect to a schema S , denoted by type S ( 𝑒 ) , inductively as follows: • type S ( 𝑉 ) : = size ( 𝑉 ) , for a matrix variable 𝑉 ∈ M ; • type S ( 𝑒 𝑇 ) : = ( 𝛽, 𝛼 ) if type S ( 𝑒 ) = ( 𝛼, 𝛽 ) ; • type S ( ( 𝑒 )) : = ( 𝛼, ) if type S ( 𝑒 ) = ( 𝛼, 𝛽 ) ; • type S ( diag ( 𝑒 )) : = ( 𝛼, 𝛼 ) , if type S ( 𝑒 ) = ( 𝛼, ) ; • type S ( 𝑒 · 𝑒 ) : = ( 𝛼, 𝛾 ) if type S ( 𝑒 ) = ( 𝛼, 𝛽 ) , and type S ( 𝑒 ) = ( 𝛽,𝛾 ) ; • type S ( 𝑒 + 𝑒 ) : = ( 𝛼, 𝛽 ) if type S ( 𝑒 ) = type S ( 𝑒 ) = ( 𝛼, 𝛽 ) ; The original syntax also permits the operator let 𝑉 = 𝑒 in 𝑒 , which replacesevery occurrence of 𝑉 in 𝑒 with the value of 𝑒 . Since this is just syntactic sugar, weomit this operator. We also explicitly include matrix addition and scalar multiplication,although these can be simulated by pointwise function applications. Finally, we usetransposition instead of conjugate transposition since we work with matrices over R . xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA • type S ( 𝑒 × 𝑒 ) : = ( 𝛼, 𝛽 ) if type S ( 𝑒 ) = ( , ) and type S ( 𝑒 ) = ( 𝛼, 𝛽 ) ; and • type S ( 𝑓 ( 𝑒 , . . . , 𝑒 𝑘 )) : = ( 𝛼, 𝛽 ) , whenever type S ( 𝑒 ) = · · · = type S ( 𝑒 𝑘 ) : = ( 𝛼, 𝛽 ) and 𝑓 ∈ F 𝑘 .We call an expression well-typed according to the schema S , if it hasa defined type. A well-typed expression can be evaluated regardlessof the actual sizes of the matrices assigned to matrix variables, aswe describe next. Semantics.
We use
Mat [ R ] to denote the set of all real matricesand for 𝐴 ∈ Mat [ R ] , dim ( 𝐴 ) ∈ N denotes its dimensions. A( MATLANG ) instance I over a schema S is a pair I = (D , mat ) , where D : Symb ↦→ N assigns a value to each size symbol (and thus inturn dimensions to each matrix variable), and mat : M ↦→
Mat [ R ] assigns a concrete matrix to each matrix variable 𝑉 ∈ M , suchthat dim ( mat ( 𝑉 )) = D( 𝛼 ) × D( 𝛽 ) if size ( 𝑉 ) = ( 𝛼, 𝛽 ) . That is, aninstance tells us the dimensions of each matrix variable, and alsothe concrete matrices assigned to the variable names in M . Weassume that D( ) = , for every instance I . If 𝑒 is a well-typedexpression according to S , then we denote by ⟦ 𝑒 ⟧(I) the matrixobtained by evaluating 𝑒 over I , and define it as follows: • ⟦ 𝑉 ⟧(I) : = mat ( 𝑉 ) , for 𝑉 ∈ M ; • ⟦ 𝑒 𝑇 ⟧(I) : = ⟦ 𝑒 ⟧(I) 𝑇 , where 𝐴 𝑇 is the transpose of matrix 𝐴 ; • ⟦ ( 𝑒 )⟧(I) is a 𝑛 × vector with as all of its entries, where dim (⟦ 𝑒 ⟧(I)) = ( 𝑛, 𝑚 ) ; • ⟦ diag ( 𝑒 )⟧(I) is a diagonal matrix with the vector ⟦ 𝑒 ⟧(I) on its main diagonal, and zero in every other position; • ⟦ 𝑒 · 𝑒 ⟧(I) : = ⟦ 𝑒 ⟧(I) · ⟦ 𝑒 ⟧(I) ; • ⟦ 𝑒 + 𝑒 ⟧(I) : = ⟦ 𝑒 ⟧(I) + ⟦ 𝑒 ⟧(I) ; • ⟦ 𝑒 × 𝑒 ⟧(I) : = 𝑎 × ⟦ 𝑒 ⟧(I) with ⟦ 𝑒 ⟧(I) = [ 𝑎 ] ; and • ⟦ 𝑓 ( 𝑒 , . . . , 𝑒 𝑘 )⟧(I) is a matrix 𝐴 of the same size as ⟦ 𝑒 ⟧(I) ,and where 𝐴 𝑖 𝑗 has the value 𝑓 (⟦ 𝑒 ⟧(I) 𝑖 𝑗 , . . . , ⟦ 𝑒 𝑘 ⟧(I) 𝑖 𝑗 ) .Although MATLANG forms a solid basis for a matrix query language,it is limited in expressive power. Indeed,
MATLANG is subsumed byfirst order logic with aggregates that uses only three variables [7].Hence, no
MATLANG expression exists that can compute the transi-tive closure of a graph (represented by its adjacency matrix) or cancompute the inverse of a matrix. Rather than extending
MATLANG with specific linear algebra operators, such as matrix inversion, wenext introduce a limited form of recursion in
MATLANG . To extend
MATLANG with recursion, we take inspiration from classi-cal linear algebra algorithms, such as those described in [28]. Manyof these algorithms are based on for-loops in which the terminationcondition for each loop is determined by the matrix dimensions.We have seen how the transitive closure of a matrix can be com-puted using for-loops in the Introduction. Here we add this ability to
MATLANG , and show that the resulting language, called for - MATLANG , can compute properties outside of the scope of MATLANG . We seemore advanced examples, such as Gaussian elimination and matrixinversion, later in the paper. for - MATLANG
The syntax of for - MATLANG is defined just as for
MATLANG but withan extra rule in the grammar: for 𝑣, 𝑋 . 𝑒 (canonical for loop, with 𝑣, 𝑋 ∈ V ).Intuitively, 𝑋 is a matrix variable which is iteratively updated ac-cording to the expression 𝑒 . We simulate iterations of the form “ for 𝑖 ∈ [ ..𝑛 ] ” by letting 𝑣 loop over the canonical vectors 𝑏 𝑛 , . . . , 𝑏 𝑛𝑛 ofdimension 𝑛 . Here, 𝑏 𝑛 = [ · · · ] 𝑇 , 𝑏 𝑛 = [ · · · ] 𝑇 , etc. When 𝑛 is clear from the context we simply write 𝑏 , 𝑏 , . . . . In addition,the expression 𝑒 in the rule above may depend on 𝑣 .We next make the semantics precise and start by declaringthe type of loop expressions. Given a schema S , the type of a for - MATLANG expression 𝑒 , denoted type S ( 𝑒 ) , is defined induc-tively as in MATLANG but with following extra rule: • type S ( for 𝑣, 𝑋 . 𝑒 ) : = ( 𝛼, 𝛽 ) , if type S ( 𝑒 ) = type S ( 𝑋 ) = ( 𝛼, 𝛽 ) and type S ( 𝑣 ) = ( 𝛾, ) .We note that S now necessarily includes 𝑣 and 𝑋 as variables andassigns size symbols to them. We also remark that in the definitionof the type of for 𝑣, 𝑋 . 𝑒 , we require that type S ( 𝑋 ) = type S ( 𝑒 ) as this expression updates the content of the variable 𝑋 in eachiteration using the result of 𝑒 . We further restrict the type of 𝑣 tobe a vector, i.e., type S ( 𝑣 ) = ( 𝛾, ) , since 𝑣 will be instantiated withcanonical vectors. A for - MATLANG expression 𝑒 is well-typed overa schema S if its type is defined.For well-typed expressions we next define their semantics. Thisis done in an inductive way, just as for MATLANG . To define thesemantics of for 𝑣, 𝑋 . 𝑒 over an instance I , we need the followingnotation. Let I be an instance and 𝑉 ∈ M . Then I [ 𝑉 : = 𝐴 ] denotesan instance that coincides with I , except that the value of the matrixvariable 𝑉 is given by the matrix 𝐴 . Assume that type S ( 𝑣 ) = ( 𝛾, ) ,and type S ( 𝑒 ) = ( 𝛼, 𝛽 ) and 𝑛 : = D( 𝛾 ) . Then, ⟦ for 𝑣, 𝑋 . 𝑒 ⟧(I) isdefined iteratively, as follows: • Let 𝐴 : = be the zero matrix of size D( 𝛼 ) × D( 𝛽 ) . • For 𝑖 = , . . . 𝑛 , compute 𝐴 𝑖 : = ⟦ 𝑒 ⟧(I [ 𝑣 : = 𝑏 𝑛𝑖 , 𝑋 : = 𝐴 𝑖 − ]) . • Finally, set ⟦ for 𝑣, 𝑋 . 𝑒 ⟧(I) : = 𝐴 𝑛 .For better understanding how for - MATLANG works, we next pro-vide some examples. We start by showing that the one-vector and diag operators are redundant in for - MATLANG . Example 3.1.
We first show how the one-vector operator ( 𝑒 ) canbe expressed using for loops. It suffices to consider the expression 𝑒 : = for 𝑣, 𝑋 . 𝑋 + 𝑣, with type S ( 𝑣 ) = ( 𝛼, ) = type S ( 𝑋 ) if type S ( 𝑒 ) = ( 𝛼, 𝛽 ) . Thisexpression is well-typed and is of type ( 𝛼, ) . When evaluated oversome instance I with 𝑛 = D( 𝛼 ) , ⟦ 𝑒 ⟧(I) is defined as follows.Initially, 𝐴 : = . Then 𝐴 𝑖 : = 𝐴 𝑖 − + 𝑏 𝑛𝑖 , i.e., the 𝑖 th canonical vectoris added to 𝐴 𝑖 − . Finally, ⟦ 𝑒 ⟧(I) : = 𝐴 𝑛 and this now clearlycoincides with ⟦ ( 𝑒 )⟧(I) . □ Example 3.2.
We next show that the diag operator is redundantin for - MATLANG . Indeed, it suffices to consider the expression 𝑒 diag : = for 𝑣, 𝑋 . 𝑋 + ( 𝑣 𝑇 · 𝑒 ) × 𝑣 · 𝑣 𝑇 , where 𝑒 is a for - MATLANG expression of type ( 𝛼, ) . For this ex-pression to be well-typed, 𝑣 has to be a vector variable of type onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč 𝛼 × and 𝑋 a matrix variable of type ( 𝛼, 𝛼 ) . Then, ⟦ 𝑒 diag ⟧(I) is defined as follows. Initially, 𝐴 is the zero matrix of dimen-sion 𝑛 × 𝑛 , where 𝑛 = D( 𝛼 ) . Then, in each iteration 𝑖 ∈ [ ..𝑛 ] , 𝐴 𝑖 : = 𝐴 𝑖 − + (( 𝑏 𝑛𝑖 ) 𝑇 · ⟦ 𝑒 ⟧(I)) × ( 𝑏 𝑛𝑖 · ( 𝑏 𝑛𝑖 ) 𝑇 ) . In other words, 𝐴 𝑖 isobtained by adding the matrix with value (⟦ 𝑒 ⟧(I)) 𝑖 on position ( 𝑖, 𝑖 ) to 𝐴 𝑖 − . Hence, ⟦ 𝑒 diag ⟧(I) : = 𝐴 𝑛 = ⟦ diag ( 𝑒 )⟧(I) . □ These examples illustrate that we can limit for - MATLANG to con-sist of the following “core” operators: transposition, matrix mul-tiplication and addition, scalar multiplication, pointwise functionapplication, and for-loops. More specific, for - MATLANG is definedby the following simplified syntax: 𝑒 :: = 𝑉 | 𝑒 𝑇 | 𝑒 · 𝑒 | 𝑒 + 𝑒 | 𝑒 × 𝑒 | 𝑓 ( 𝑒 , . . . , 𝑒 𝑘 ) | for 𝑣, 𝑋 . 𝑒 Similarly as for
MATLANG , we write for - MATLANG [F ] for some set F of functions when these are required for the task at hand.As a final example, we show that we can compute whether agraph contains a 4- clique using for - MATLANG . Example 3.3.
To test for -cliques it suffices to consider the fol-lowing expression with for-loops nested four times: for 𝑢, 𝑋 . 𝑋 + for 𝑣, 𝑋 . 𝑋 + for 𝑤, 𝑋 . 𝑋 + for 𝑥, 𝑋 . 𝑋 + 𝑢 𝑇 · 𝑉 · 𝑣 · 𝑢 𝑇 · 𝑉 · 𝑤 · 𝑢 𝑇 · 𝑉 · 𝑥 · 𝑣 𝑇 · 𝑉 · 𝑤 · 𝑣 𝑇 · 𝑉 · 𝑥 · 𝑤 𝑇 · 𝑉 · 𝑥 · 𝑔 ( 𝑢, 𝑣, 𝑤, 𝑥 ) with 𝑔 ( 𝑢, 𝑣, 𝑤, 𝑥 ) = 𝑓 ( 𝑢, 𝑣 )· 𝑓 ( 𝑢, 𝑤 )· 𝑓 ( 𝑢, 𝑥 )· 𝑓 ( 𝑣, 𝑤 )· 𝑓 ( 𝑣, 𝑥 )· 𝑓 ( 𝑤, 𝑥 ) and 𝑓 ( 𝑢, 𝑣 ) = − 𝑢 𝑇 · 𝑣 . Note that 𝑓 ( 𝑏 𝑛𝑖 , 𝑏 𝑛𝑗 ) = if 𝑖 ≠ 𝑗 and 𝑓 ( 𝑏 𝑛𝑖 , 𝑏 𝑛𝑗 ) = otherwise. Hence, 𝑔 ( 𝑏 𝑛𝑖 , 𝑏 𝑛𝑗 , 𝑏 𝑛𝑘 , 𝑏 𝑛ℓ ) = if and only ifall 𝑖, 𝑗, 𝑘, 𝑙 are pairwise different. When evaluating the expressionon an instance I such that 𝑉 is assigned to the adjacency matrixof a graph, the expression above evaluates to a non-zero value ifand only if the graph contains a four-clique. □ Given that
MATLANG can not check for 4-cliques [7], we easilyobtain the following.Proposition 3.4.
For any collection of functions F , MATLANG [F ] is properly subsumed by for - MATLANG [F ] . for - MATLANG
Loop Initialization.
As the reader may have observed, in the se-mantics of for-loops we always initialize 𝐴 to the zero matrix (of appropriate dimensions). It is often convenient to start the it-eration given some concrete matrix originating from the resultof evaluation a for - MATLANG expression 𝑒 . To make this explicit,we write for 𝑣, 𝑋 = 𝑒 . 𝑒 and its semantics is defined as abovewith the difference that 𝐴 : = ⟦ 𝑒 ⟧(I) . We observe, however, that for 𝑣, 𝑋 = 𝑒 . 𝑒 can already be expressed in for - MATLANG . In otherwords, we do not loose generality by assuming an initializationof 𝐴 by . The key insight is that in for - MATLANG we can checkduring evaluation whether or not the current canonical vector 𝑏 𝑛𝑖 isequal to the 𝑏 𝑛 . This is due to the fact that for-loops iterate over thecanonical vectors in a fixed order. We discuss this more in the nextparagraph. In particular, we can define a for - MATLANG expression min ( ), which when evaluated on an instance, returns if its input vector is 𝑏 𝑛 , and returns otherwise. Given min ( ), consider nowthe for - MATLANG expression for 𝑣, 𝑋 . min ( 𝑣 ) · 𝑒 ( 𝑣, 𝑋 / 𝑒 ) + ( − min ( 𝑣 )) · 𝑒 ( 𝑣, 𝑋 ) , where we explicitly list 𝑣 and 𝑋 as matrix variables on which 𝑒 po-tentially depends on, and where 𝑒 ( 𝑣, 𝑋 / 𝑒 ) denotes the expressionobtained by replacing every occurrence of 𝑋 in 𝑒 with 𝑒 . Whenevaluating this expression on an instance I , 𝐴 is initial set to thezero matrix, in the first iteration (when 𝑣 = 𝑏 𝑛 and thus min ( 𝑣 ) = )we have 𝐴 = ⟦ 𝑒 ⟧(I [ 𝑣 : = 𝑏 𝑛 , 𝑋 : = ⟦ 𝑒 ⟧(I)]) , and for consecutiveiterations (when only the part related to − min ( 𝑣 ) applies) 𝐴 𝑖 isupdated as before. Clearly, the result of this evaluation is equal to ⟦ for 𝑣, 𝑋 = 𝑒 . 𝑒 ⟧(I) .As an illustration, we consider the Floyd-Warshall algorithmgiven in the Introduction. Example 3.5.
Consider the following expression: 𝑒 𝐹𝑊 : = for 𝑣 𝑘 , 𝑋 = 𝐴. 𝑋 + for 𝑣 𝑖 , 𝑋 . 𝑋 + for 𝑣 𝑗 , 𝑋 . 𝑋 +( 𝑣 𝑇𝑖 · 𝑋 · 𝑣 𝑘 · 𝑣 𝑇𝑘 · 𝑋 · 𝑣 𝑗 ) × 𝑣 𝑖 · 𝑣 𝑇𝑗 The expression 𝑒 𝐹𝑊 simulates the Floyd-Warshall algorithm byupdating the matrix 𝐴 , which is stored in the variable 𝑋 . The innersub-expression here constructs an 𝑛 × 𝑛 matrix that contains onein the position ( 𝑖, 𝑗 ) if and only if one can reach the vertex 𝑗 from 𝑖 by going through 𝑘 , and zero elsewhere. If an instance I assigns to 𝐴 the adjacency matrix of a graph, then ⟦ 𝑒 𝐹𝑊 ⟧(I) will be equal tothe matrix produced by the algorithm given in the Introduction. □ Order.
By introducing for-loops we not only extend
MATLANG withbounded recursion, we also introduce order information. Indeed, thesemantics of the for operator assumes that the canonical vectors 𝑏 , 𝑏 , . . . are accessed in this order. It implies, among other things,that for - MATLANG expressions are not permutation-invariant. Wecan, for example, return the bottom right-most entry in a matrix.Indeed, consider the expression 𝑒 max : = for 𝑣, 𝑋 . 𝑣 which, for itto be well-typed, requires both 𝑣 and 𝑋 to be of type ( 𝛼, ) . Then, ⟦ 𝑒 max ⟧(I) = 𝑏 𝑛𝑛 , for 𝑛 = D( 𝛼 ) , simply because initially, 𝑋 = ,but 𝑋 will be overwritten by 𝑏 𝑛 , 𝑏 𝑛 , . . . , 𝑏 𝑛𝑛 , in this order. Hence,at the end of the evaluation 𝑏 𝑛𝑛 is returned. To extract the bottomright-most entry from a matrix, we now simply use 𝑒 𝑇 max · 𝑉 · 𝑒 max .Although the order is implicit in for - MATLANG , we can explicitlyuse this order in for - MATLANG expressions. More precisely, the orderon canonical vectors is made accessible by using the matrix: 𝑆 ≤ = · · ·
10 1 · · · ... ... . . .
10 0 · · · . We observe that 𝑆 ≤ has the property that 𝑏 𝑇𝑖 · 𝑆 ≤ · 𝑏 𝑗 = , for twocanonical vectors 𝑏 𝑖 and 𝑏 𝑗 of the same dimension, if and only if 𝑖 ≤ 𝑗 . Otherwise, 𝑏 𝑇𝑖 · 𝑆 ≤ · 𝑏 𝑗 = . Interestingly, we can build thematrix 𝑆 ≤ with the following for - MATLANG expression: 𝑒 ≤ = for 𝑣, 𝑋 . 𝑋 + (( 𝑋 · 𝑒 max ) + 𝑣 ) · 𝑣 𝑇 + 𝑣 · 𝑒 𝑇 max , where 𝑒 max is as defined above. The intuition behind this expressionis that by using the last canonical vector 𝑏 𝑛 , as returned by 𝑒 max , xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA we have access to the last column of 𝑋 (via the product 𝑋 · 𝑒 max ).We use this column such that after the 𝑖 -th iteration, this columncontains the 𝑖 -th column of 𝑆 ≤ . This is done by incrementing 𝑋 with 𝑣 · 𝑒 𝑇 max . To construct 𝑆 ≤ , in the 𝑖 -th iteration we further increment 𝑋 with (i) the current last column in 𝑋 (via 𝑋 · 𝑒 max · 𝑣 𝑇 ) whichholds the ( 𝑖 − ) -th column of 𝑆 ≤ ; and (ii) the current canonicalvector (via 𝑣 · 𝑣 𝑇 ). Hence, after iteration 𝑖 , 𝑋 contains the first 𝑖 columns of 𝑆 ≤ and holds the 𝑖 th column of 𝑆 ≤ in its last column. Itis now readily verified that 𝑋 = 𝑆 ≤ after the 𝑛 th iteration.It should be clear that if we can compute 𝑆 ≤ using 𝑒 ≤ , then wecan easily define the following predicates and vectors related withthe order of canonical vectors: • succ ( 𝑢, 𝑣 ) such that succ ( 𝑏 𝑛𝑖 , 𝑏 𝑛𝑗 ) = if 𝑖 ≤ 𝑗 and otherwise.Similarly, we can define succ + ( 𝑢, 𝑣 ) such that succ + ( 𝑏 𝑛𝑖 , 𝑏 𝑛𝑗 ) = if 𝑖 < 𝑗 and otherwise; • min ( 𝑢 ) such that min ( 𝑏 𝑛𝑖 ) = if 𝑖 = and min ( 𝑏 𝑛𝑖 ) = otherwise; • max ( 𝑢 ) such that max ( 𝑏 𝑛𝑖 ) = if 𝑖 = 𝑛 and min ( 𝑏 𝑛𝑖 ) = otherwise; and • 𝑒 min and 𝑒 max such that ⟦ 𝑒 min ⟧(I) = 𝑏 𝑛 and ⟦ 𝑒 max ⟧(I) = 𝑏 𝑛𝑛 , respectively.The definitions of these expressions are detailed in the appendix.Having order information available results in for - MATLANG to bequite expressive. We heavily rely on order information in the nextsections to compute the inverse of matrices and more generally tosimulate low complexity Turing machines and arithmetic circuits.
One of our main motivations to introduce for-loops is to be ableto express classical linear algebra algorithms in a natural way. Wehave seen that for - MATLANG is quite expressive as it can check forcliques, compute the transitive closure, and can even leverage asuccessor relation on canonical vectors. The big question is howexpressive for - MATLANG actually is. We will answer this in the nextsection by connecting for - MATLANG with arithmetic circuits of poly-nomial degree. Through this connection, one can move back andforth between for - MATLANG and arithmetic circuits, and as a conse-quence, anything computable by such a circuit can be computedby for - MATLANG as well. When it comes to specific linear algebraalgorithms, the detour via circuits can often be avoided. Indeed, inthis section we illustrate that for - MATLANG is able to compute LUdecompositions of matrices. These decompositions form the basis ofmany other algorithms, such as solving linear systems of equations.We further show that for - MATLANG is expressive enough to com-pute matrix inversion and the determinant. We recall that matrixinversion and determinant need to be explicitly added as separateoperators in
MATLANG [7] and that the LARA language is unable toinvert matrices under usual complexity-theoretic assumptions [4].
A lower-upper (LU) decomposition factors a matrix 𝐴 as the productof a lower triangular matrix 𝐿 and upper triangular matrix 𝑈 . Thisdecomposition, and more generally LU decomposition with rowpivoting (PLU), underlies many linear algebra algorithms and wenext show that for - MATLANG can compute these decompositions.
LU decomposition by Gaussian elimination.
LU decomposi-tion can be seen as a matrix form of Gaussian elimination in whichthe columns of 𝐴 are reduced, one by one, to obtain the matrix 𝑈 .The reduction of columns of 𝐴 is achieved as follows. Consider thefirst column [ 𝐴 , . . . , 𝐴 𝑛 ] 𝑇 of 𝐴 and define 𝑐 : = [ , 𝛼 , . . . , 𝛼 𝑛 ] 𝑇 with 𝛼 𝑗 : = − 𝐴 𝑗 𝐴 . Let 𝑇 : = 𝐼 + 𝑐 · 𝑏 𝑇 and consider 𝑇 · 𝐴 . Thatis, the 𝑗 th row of 𝑇 · 𝐴 is obtained by multiplying the first row of 𝐴 by 𝛼 𝑗 and adding it to the 𝑗 th row of 𝐴 . As a result, the firstcolumn of 𝑇 · 𝐴 is equal to [ 𝐴 , , . . . , ] 𝑇 , i.e., all of its entriesbelow the diagonal are zero. One then iteratively performs a sim-ilar computation, using a matrix 𝑇 𝑖 : = 𝐼 + 𝑐 𝑖 · 𝑏 𝑇𝑖 , where 𝑐 𝑖 nowdepends on the 𝑖 th column in 𝑇 𝑖 − · · · 𝑇 · 𝐴 . As a consequence, 𝑇 𝑖 · 𝑇 𝑖 − · · · 𝑇 · 𝐴 is upper triangular in its first 𝑖 columns. At theend of this process, 𝑇 𝑛 · · · 𝑇 · 𝐴 = 𝑈 where 𝑈 is the desired uppertriangular matrix. Furthermore, it is easily verified that each 𝑇 𝑖 isinvertible and by defining 𝐿 : = 𝑇 − · · · · · 𝑇 − 𝑛 one obtains a lowertriangular matrix satisfying 𝐴 = 𝐿 · 𝑈 . The above procedure is onlysuccessful when the denominators used in the definition of thevectors 𝑐 𝑖 are non-zero. When this is the case we call a matrix 𝐴 LU-factorizable .In case when such a denominator is zero in one of the reductionsteps, one can remedy this situation by row pivoting . That is, whenthe 𝑖 th entry of the 𝑖 th row in 𝑇 𝑖 − · · · 𝑇 · 𝐴 is zero, one replacesthe 𝑖 th row by 𝑗 th row in this matrix, with 𝑗 > 𝑖 , provided that 𝑖 theentry of the 𝑗 th row is non-zero. If no such row exists, this impliesthat all elements below the diagonal are zero already in column 𝑖 and one can proceed with the next column. One can formulate thisin matrix terms by stating that there exists a permutation matrix 𝑃 , which pivots rows, such that 𝑃 · 𝐴 = 𝐿 · 𝑈 . Any matrix 𝐴 isLU-factorizable with pivoting . Implementing LU decomposition in for - MATLANG . We first as-sume that the input matrices are LU-factorizable. We deal withgeneral matrices later on. To implement the above procedure, weneed to compute the vector 𝑐 𝑖 for each column 𝑖 . We do this in twosteps. First, we extract from our input matrix its 𝑖 th column and setall its upper diagonal entries to zero by means of the expression: col( 𝑉 , 𝑦 ) : = for 𝑣, 𝑋 . succ + ( 𝑦, 𝑣 ) · ( 𝑣 𝑇 · 𝑉 · 𝑦 ) · 𝑣 + 𝑋 .
Indeed, when 𝑉 is assigned to a matrix 𝐴 and 𝑦 to 𝑏 𝑖 , we have that 𝑋 will be initially assigned 𝐴 = and in consecutive iterations, 𝐴 𝑗 = 𝐴 𝑗 − + 𝑏 𝑇𝑗 · 𝐴 · 𝑏 𝑖 if 𝑗 > 𝑖 (because succ + ( 𝑏 𝑖 , 𝑏 𝑗 ) = if 𝑗 > 𝑖 )and 𝐴 𝑗 = 𝐴 𝑗 − otherwise (because succ + ( 𝑏 𝑖 , 𝑏 𝑗 ) = for 𝑗 ≤ 𝑖 ).The result of this evaluation is the desired column vector. Using col( 𝑉 , 𝑦 ) , we can now compute 𝑇 𝑖 by the following expression: reduce( 𝑉 , 𝑦 ) : = 𝑒 Id + 𝑓 / ( col( 𝑉 , 𝑦 ) , −( 𝑦 𝑇 · 𝑉 · 𝑦 ) · ( 𝑦 )) · 𝑦 𝑇 , where 𝑓 / : R → R : ( 𝑥, 𝑦 ) ↦→ 𝑥 / 𝑦 is the division function. When 𝑉 is assigned to 𝐴 and 𝑦 to 𝑏 𝑖 , 𝑓 / ( col( 𝐴, 𝑏 𝑖 ) , −( 𝑏 𝑇𝑖 · 𝐴 · 𝑏 𝑖 ) · ( 𝑏 𝑖 )) is equal to the vector 𝑐 𝑖 used in the definition of 𝑇 𝑖 . To perform thereduction steps for all columns, we consider the expression: 𝑒 𝑈 ( 𝑉 ) : = ( for 𝑦, 𝑋 = 𝑒 Id . reduce( 𝑋 · 𝑉 , 𝑦 ) · 𝑋 ) · 𝑉 .
That is, when 𝑉 is assigned 𝐴 , 𝑋 will be initially 𝐴 = 𝐼 , and then 𝐴 𝑖 = reduce( 𝐴 𝑖 − · 𝐴, 𝑏 𝑖 ) = 𝑇 𝑖 · 𝑇 𝑖 − · · · 𝑇 · 𝐴 , as desired. Weshow in the appendix that, because we can obtain the matrices 𝑇 𝑖 in for - MATLANG and that these are easily invertible, we can also onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč construct an expression 𝑒 𝐿 ( 𝑉 ) which evaluates to 𝐿 when 𝑉 isassigned to 𝐴 . We may thus conclude the following.Proposition 4.1. There exists for - MATLANG [ 𝑓 / ] expressions 𝑒 𝐿 ( 𝑉 ) and 𝑒 𝑈 ( 𝑉 ) such that ⟦ 𝑒 𝐿 ⟧(I) = 𝐿 and ⟦ 𝑒 𝑈 ⟧(I) = 𝑈 form an LU-decomposition of 𝐴 , where mat ( 𝑉 ) = 𝐴 and 𝐴 is LU-factorizable. □ We remark that the proposition holds when division is addedas a function in F in for - MATLANG . When row pivoting is needed,we can also obtain a permutation matrix 𝑃 such that 𝑃 · 𝐴 = 𝐿 · 𝑈 holds by means of an expression in for - MATLANG , provided that weadditionally allow the function 𝑓 > , where 𝑓 > : R → R is suchthat 𝑓 > ( 𝑥 ) : = if 𝑥 > and 𝑓 > ( 𝑥 ) : = otherwise.Proposition 4.2. There exist expressions 𝑒 𝐿 − 𝑃 ( 𝑀 ) and 𝑒 𝑈 ( 𝑀 ) in for - MATLANG [ 𝑓 / , 𝑓 > ] such that 𝐿 − · 𝑃 = ⟦ 𝑒 𝐿 − 𝑃 ⟧(I) and 𝑈 = ⟦ 𝑒 𝑈 ⟧(I) , satisfy 𝐿 − · 𝑃 · 𝐴 = 𝑈 . □ Intuitively, by allowing 𝑓 > we introduce a limited form of if-then-else in for - MATLANG , which is needed to continue re-ducing columns only when the right pivot has been found.
Other key linear algebra operations include the computation of thedeterminant and the inverse of a matrix (if the matrix is invertible).As a consequence of the expressibility in for - MATLANG [ 𝑓 / , 𝑓 > ] ofLU-decompositions with pivoting, it can be shown that the deter-minant and inverse can be expressed as well.However, the results in the next section (connecting for - MATLANG with arithmetic circuits) imply that the determinant and inverseof a matrix can already be defined in for - MATLANG [ 𝑓 / ] . So insteadof using LU decomposition with pivoting for matrix inversion andcomputing the determinant, we provide an alternative solution.More specifically, we rely on Csanky’s algorithm for comput-ing the inverse of a matrix [11]. This algorithm uses the char-acteristic polynomial 𝑝 𝐴 ( 𝑥 ) = det ( 𝑥𝐼 − 𝐴 ) of a matrix. Whenexpanded as a polynomial 𝑝 𝐴 ( 𝑥 ) = (cid:205) 𝑛𝑖 = 𝑐 𝑖 𝑥 𝑖 and it is knownthat 𝐴 − = − 𝑐 𝑛 (cid:205) 𝑛 − 𝑖 = 𝑐 𝑖 𝐴 𝑛 − − 𝑖 if 𝑐 𝑛 ≠ . Furthermore, 𝑐 = , 𝑐 𝑛 = (− ) 𝑛 det ( 𝐴 ) and the coefficients 𝑐 𝑖 of 𝑝 𝐴 ( 𝑥 ) are known tosatisfy the system of equations 𝑆 · 𝑐 = 𝑠 given by: (cid:169)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:171) · · · 𝑆 · · · 𝑆 𝑆 · · · ... ... ... ... ... 𝑆 𝑛 − 𝑆 𝑛 − 𝑆 𝑛 − · · · 𝑆 𝑛 (cid:170)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:172) · (cid:169)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:171) 𝑐 𝑐 𝑐 ...𝑐 𝑛 (cid:170)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:172) = (cid:169)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:171) 𝑆 𝑆 𝑆 ...𝑆 𝑛 (cid:170)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:172) , with 𝑆 𝑖 = tr ( 𝐴 𝑖 ) . We show, in the appendix, that we can constructall ingredients of this system of equations in for - MATLANG [ 𝑓 / ] . Byobserving that the matrix 𝑆 is a lower triangular matrix with non-zero elements on its diagonal, we can write it in the form 𝐷 𝑆 +( 𝑆 − 𝐷 𝑆 ) = 𝐷 𝑆 · ( 𝐼 + 𝐷 − 𝑆 · ( 𝑆 − 𝐷 𝑆 )) with 𝐷 𝑆 the diagonal matrixconsisting of the diagonal entries of 𝑆 . Hence 𝑆 − = ( 𝐼 + 𝐷 − 𝑆 ·( 𝑆 − 𝐷 𝑆 )) − · 𝐷 − 𝑆 . We remark 𝐷 − 𝑆 can simply be obtained byinverting the (non-zero) elements on the diagonal by means of 𝑓 / in for - MATLANG [ 𝑓 / ] . Furthermore, we observe that ( 𝐼 + 𝐷 − 𝑆 ( 𝑆 − 𝐷 𝑆 )) − = (cid:205) 𝑛𝑖 = ( 𝐷 − 𝑆 ( 𝑆 − 𝐷 𝑆 )) 𝑖 which is something we can computein for - MATLANG [ 𝑓 / ] as well. Hence, we can invert 𝑆 and obtain the vector ( 𝑐 , . . . , 𝑐 𝑛 ) 𝑇 as 𝑆 − · 𝑠 . To compute 𝐴 − it now suffices tocompute − 𝑐 𝑛 (cid:205) 𝑛 − 𝑖 = 𝑐 𝑖 𝐴 𝑛 − − 𝑖 . To find the determinant, we compute (− ) 𝑛 𝑐 𝑛 . All this can be done in for - MATLANG [ 𝑓 / ] . We may thusconclude:Proposition 4.3. There are for - MATLANG [ 𝑓 / ] expressions 𝑒 det ( 𝑉 ) and 𝑒 inv ( 𝑉 ) such that ⟦ 𝑒 det ⟧(I) = det ( 𝐴 ) , and ⟦ 𝑒 inv ⟧(I) = 𝐴 − when I assigns 𝑉 to 𝐴 and 𝐴 is invertible. □ In this section we explore the expressive power of for - MATLANG . Given that arithmetic circuits [1] capture most standard linear al-gebra algorithms [29, 30], they seem as a natural candidate forcomparison. Intuitively, an arithmetic circuit is similar to a booleancircuit [3], except that it has gates computing the sum and the prod-uct function, and processes elements of R instead of boolean values.To connect for - MATLANG to arithmetic circuits we need a notionof uniformity of such circuits. After all, a for - MATLANG expressioncan take matrices of arbitrary dimensions as input and we want toavoid having different circuits for each dimension. To handle inputsof different sizes, we thus consider a notion of uniform familiesof arithmetic circuits, defined via a Turing machine generating adescription of the circuit for each input size 𝑛 .What we show in the remainder of this section is that any func-tion 𝑓 which operates on matrices, and is computed by a uniformfamily of arithmetic circuits of bounded degree, can also be com-puted by a for - MATLANG expression, and vice versa. In order to keepthe notation light, we will focus on for - MATLANG schemas over“square matrices” where each variable has type ( 𝛼, 𝛼 ) , ( 𝛼, ) , ( , 𝛼 ) ,or ( , ) , although all of our results hold without these restric-tions as well. In what follows, we will write for - MATLANG to denote for - MATLANG [∅] , i.e. the fragment of our language with no addi-tional pointwise functions. We begin by defining circuits and thenshow how circuit families can be simulated by for - MATLANG . for - MATLANG
Let us first recall the definition of arithmetic circuits. An arithmeticcircuit Φ over a set 𝑋 = { 𝑥 , . . . , 𝑥 𝑛 } of input variables is a directedacyclic labeled graph. The vertices of Φ are called gates and denotedby 𝑔 , . . . , 𝑔 𝑚 ; the edges in Φ are called wires . The children of a gate 𝑔 correspond to all gates 𝑔 ′ such that ( 𝑔, 𝑔 ′ ) is an edge. The parents of 𝑔 correspond to all gates 𝑔 ′ such that ( 𝑔 ′ , 𝑔 ) is an edge. The in-degree ,or a fan-in , of a gate 𝑔 refers to its number of children, and the out-degree to its number of parents. We will not assume any restrictionon the in-degree of a gate, and will thus consider circuits withunbounded fan-in. Gates with in-degree are called input gates andare labeled by either a variable in 𝑋 or a constant or . All othergates are labeled by either + or × , and are referred to as sum gates or product gates , respectively. Gates with out-degree are called output gates . When talking about arithmetic circuits, one usuallyfocuses on circuits with 𝑛 input gates and a single output gate.The size of Φ , denoted by | Φ | , is its number of gates and wires.The depth of Φ , denoted by depth ( Φ ) , is the length of the longestdirected path from any of its output gates to any of the inputgates. The degree of a gate is defined inductively: an input gatehas degree 1, a sum gate has a degree equal to the maximum ofdegrees of its children, and a product gate has a degree equal to the xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA sum of the degrees of its children. When Φ has a single output gate,the degree of Φ , denoted by degree ( Φ ) , is defined as the degree ofits output gate. If Φ has a single output gate and its input gates takevalues from R , then Φ corresponds to a polynomial in R [ 𝑋 ] in anatural way. In this case, the degree of Φ equals the degree of thepolynomial corresponding to Φ . If 𝑎 , . . . , 𝑎 𝑛 are values in R , thenthe result of the circuit on this input is the value computed by thecorresponding polynomial, denoted by Φ ( 𝑎 , . . . , 𝑎 𝑘 ) .In order to handle inputs of different sizes, we use the notionof uniform circuit families. An arithmetic circuit family is a setof arithmetic circuits { Φ 𝑛 | 𝑛 = , , . . . } where Φ 𝑛 has 𝑛 inputvariables and a single output gate. An arithmetic circuit familyis uniform if there exists a LOGSPACE-Turing machine, whichon input 𝑛 , returns an encoding of the arithmetic circuit Φ 𝑛 foreach 𝑛 . We observe that uniform arithmetic circuit families arenecessarily of polynomial size. Another important parameter is thecircuit depth. A circuit family is of logarithmic depth, whenever depth ( Φ 𝑛 ) ∈ O( 𝑙𝑜𝑔 𝑛 ) . We now show that for - MATLANG subsumesuniform arithmetic circuit families that are of logarithmic depth.Theorem 5.1.
For any uniform arithmetic circuit family { Φ 𝑛 | 𝑛 = , , . . . } of logarithmic depth there is a for - MATLANG schema S andan expression 𝑒 Φ using a matrix variable 𝑣 , with type S ( 𝑣 ) = ( 𝛼, ) and type S ( 𝑒 ) = ( , ) , such that for any input values 𝑎 , . . . , 𝑎 𝑛 : • If I = (D , mat ) is a MATLANG instance such that D( 𝛼 ) = 𝑛 and mat ( 𝑣 ) = [ 𝑎 . . . 𝑎 𝑛 ] 𝑇 . • Then ⟦ 𝑒 Φ ⟧(I) = Φ 𝑛 ( 𝑎 , . . . , 𝑎 𝑛 ) . It is important to note that the expression 𝑒 Φ does not changedepending on the input size, meaning that it is uniform in thesame sense as the circuit family being generated by a single Turingmachine. The different input sizes for a for - MATLANG instance arehandled by the typing mechanism of the language.
Proof sketch.
The proof of this Theorem, which is the deepest tech-nical result of the paper, depends crucially on two facts: (i) that anypolynomial time Turing machine working within linear space andproducing linear size output, can be simulated via a for - MATLANG expression; and (ii) that evaluating an arithmetic circuit Φ 𝑛 can bedone using two stacks of depth 𝑛 .Evaluating Φ 𝑛 on input ( 𝑎 , . . . , 𝑎 𝑛 ) can be done in a depth-firstmanner by maintaining two stacks: the gates-stack that tracks thecurrent gate being evaluated, and the values-stack that stores thevalue that is being computed for this gate. The idea behind havingtwo stacks is that whenever the number of items on the gates-stackis higher by one than the number of items on the values-stack, weknow that we are processing a fresh gate, and we have to initializeits current value (to 0 if it is a sum gate, and to 1 if it is a product gate),and push it to the values-stack. We then proceed by processing thechildren of the head of the gates-stack one by one, and aggregatethe results using sum if we are working with a sum gate, and byusing product otherwise.In order to access the information about the gate we are process-ing (such as whether it is a sum or a product gate, the list of itschildren, etc.) we use the uniformity of our circuit family. Namely,we know that we can generate the circuit Φ 𝑛 with a LOGSPACE-Turing machine 𝑀 Φ by running it on the input 𝑛 . Using this ma-chine, we can in fact compute all the information needed to runthe two-stack algorithms described above. For instance, we can construct a LOGSPACE machine that checks, given two gates 𝑔 and 𝑔 , whether 𝑔 is a child of 𝑔 . Similarly, we can construct amachine that, given 𝑔 and 𝑔 tells us whether 𝑔 is the final childof 𝑔 , or the one that produces the following child of 𝑔 (accordingto the ordering given by the machine 𝑀 Φ ). Defining these machinesbased of 𝑀 Φ is similar to the algorithm for the composition oftwo LOGSPACE transducers, and is commonly used to evaluatearithmetic circuits [1].To simulate the circuit evaluation algorithm that uses two stacks,in for - MATLANG we can use a binary matrix of size 𝑛 × 𝑛 , where 𝑛 is the number of inputs. The idea here is that the gates-stackcorresponds to the first 𝑛 − columns of the matrix, with eachgate being encoded as a binary number in positions , . . . , 𝑛 − ofa row. The remaining three columns are reserved for the values-stack, the number of elements on the gates stack, and the number ofelements on the values stack, respectively. The number of elementsis encoded as a canonical vector of size 𝑛 . Here we crucially dependon the fact that the circuit is of logarithmic depth, and therefore thesize of the two stacks is bounded by 𝑛 (apart from the portion beforethe asymptotic bound kicks-in, which can be hard-coded into theexpression 𝑒 Φ ). Similarly, given that the circuits are of polynomialsize, we can assume that gate ids can be encoded into 𝑛 − bits.This matrix is then updated in the same way as the two-stackalgorithm. It processes gates one by one, and using the successorrelation for canonical vectors determines whether we have moreelements on the gates stack. In this case, a new value is added to thevalues stack ( if the gate is a sum gate, and otherwise), and theprocess continues. Information about the next child, last child, orinput value, are obtained using the expression which simulates theTuring machine generating this data about the circuit (the machinesused never produce an output longer than their input). Given thatthe size of the circuit is polynomial, say 𝑛 𝑘 , we can initialize thematrix with the output gate only, and run the simulation of thetwo-stack algorithm for 𝑛 𝑘 steps (by iterating 𝑘 times over size 𝑛 canonical vectors). After this, the value in position ( , 𝑛 − ) (thetop of the values stack) holds the final results. □ While Theorem 5.1 gives us an idea on how to simulate arithmeticcircuits, it does not tell us which classes of functions over realnumbers can be computed by for - MATLANG expressions. In order toanswer this question, we note that arithmetic circuits can be usedto compute functions over real numbers. Formally, a circuit family { Φ 𝑛 | 𝑛 = , , . . . } computes a function 𝑓 : (cid:208) 𝑛 ≥ R 𝑛 ↦→ R , if forany 𝑎 , . . . 𝑎 𝑛 ∈ R it holds that Φ 𝑛 ( 𝑎 , . . . , 𝑎 𝑛 ) = 𝑓 ( 𝑎 , . . . , 𝑎 𝑛 ) . Tomake the connection with for - MATLANG , we need to look at circuitfamilies of bounded degree.A circuit family { Φ 𝑛 | 𝑛 = , , . . . } is said to be of polynomial de-gree if degree ( Φ 𝑛 ) ∈ 𝑂 ( 𝑝 ( 𝑛 )) , for some polynomial 𝑝 ( 𝑛 ) . Note thatpolynomial size circuit families are not necessarily of polynomialdegree. An easy corollary of Theorem 5.1 tells us that all functionscomputed by uniform family of circuits of polynomial degree andlogarithmic depth can be simulated using for - MATLANG expressions.However, we can actually drop the restriction on circuit depth dueto the result of Valiant et. al. [32] and Allender et. al. [2] whichsays that any function computed by a uniform circuit family ofpolynomial degree (and polynomial depth), can also be computed onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč by a uniform circuit family of logarithmic depth. Using this fact,we can conclude the following:Corollary 5.2.
For any function 𝑓 computed by a uniform familyof arithmetic circuits of polynomial degree, there is an equivalent for - MATLANG formula 𝑒 𝑓 . Note that there is nothing special about circuits that have asingle output, and both Theorem 5.1 and Corollary 5.2 also hold forfunctions 𝑓 : (cid:208) 𝑛 ≥ R 𝑛 ↦→ R 𝑠 ( 𝑛 ) , where 𝑠 is a polynomial. Namely,in this case, we can assume that circuits for 𝑓 have multiple outputgates, and that the depth reduction procedure of [2] is carried out foreach output gate separately. Similarly, the construction underlyingthe proof of Theorem 5.1 can be performed for each output gateindependently, and later composed into a single output vector. for - MATLANG to circuits
Now that we know that arithmetic circuits can be simulated using for - MATLANG expressions, it is natural to ask whether the sameholds in the other direction. That is, we are asking whether for each for - MATLANG expression 𝑒 over some schema S there is a uniformfamily of arithmetic circuits computing precisely the same resultdepending on the input size.In order to handle the fact that for - MATLANG expressions can pro-duce any matrix, and not just a single value, as their output, we needto consider circuits which have multiple output gates. Similarly,we need to encode matrix inputs of a for - MATLANG expression inour circuits. We will write Φ ( 𝐴 , . . . , 𝐴 𝑘 ) , where Φ is an arithmeticcircuit with multiple output gates, and each 𝐴 𝑖 is a matrix of dimen-sions 𝛼 𝑖 × 𝛽 𝑖 , with 𝛼 𝑖 , 𝛽 𝑖 ∈ { 𝑛, } to denote the input matrices fora circuit Φ . We will also write type ( Φ ) = ( 𝛼, 𝛽 ) , with 𝛼, 𝛽 ∈ { 𝑛, } ,to denote the size of the output matrix for Φ . We call such circuits arithmetic circuits over matrices . When { Φ 𝑛 | 𝑛 = , , . . . } is a uni-form family of arithmetic circuits over matrices, we will assumethat the Turing machine for generating Φ 𝑛 also gives us the infor-mation about how to access a position of each input matrix, andhow to access the positions of the output matrix, as is usually donewhen handling matrices with arithmetic circuits [29]. The notionof degree is extended to be the sum of the degrees of all the outputgates. With this at hand, we can now show the following result.Theorem 5.3. Let 𝑒 be a for - MATLANG expression over a schema S , and let 𝑉 , . . . ,𝑉 𝑘 be the variables of 𝑒 such that type S ( 𝑉 𝑖 ) ∈{( 𝛼, 𝛼 ) , ( 𝛼, ) , ( , 𝛼 ) , ( , )} . Then there exists a uniform arithmeticcircuit family over matrices Φ 𝑛 ( 𝐴 , . . . , 𝐴 𝑘 ) such that: • For any instance I = (D , mat ) such that D( 𝛼 ) = 𝑛 and mat ( 𝑉 𝑖 ) = 𝐴 𝑖 it holds that: • ⟦ 𝑒 ⟧(I) = Φ 𝑛 ( 𝐴 , . . . , 𝐴 𝑘 ) . It is not difficult to see that the proof of Theorem 5.1 can also beextended to support arithmetic circuits over matrices. In order toidentify the class of functions computed by for - MATLANG expres-sions, we need to impose one final restriction: than on the degreeof an expression. Formally, the degree of for - MATLANG expression 𝑒 over a schema S , is the minimum of the degrees of any circuit fam-ily { Φ 𝑛 | 𝑛 = , , . . . } that is equivalent to 𝑒 . That is, the expression 𝑒 is of polynomial degree, whenever there is an equivalent circuitfamily for 𝑒 of a polynomial degree. For example, all for - MATLANG expressions seen so far have polynomial degree. With this defini-tion, we can now identify the class of functions for which arithmeticcircuits and for - MATLANG formulas are equivalent. This is the maintechnical contribution of the paper.Corollary 5.4.
Let 𝑓 be a function with input matrices 𝐴 , . . . , 𝐴 𝑘 of dimensions 𝛼 × 𝛽 , with 𝛼, 𝛽 ∈ { 𝑛, } . Then, 𝑓 is computed by auniform circuit family over matrices of polynomial degree if and onlyif there is a for - MATLANG expression of polynomial degree for 𝑓 . Note that this result crucially depends on the fact that expres-sions in for - MATLANG are of polynomial degree. Some for - MATLANG expression are easily seen to produce results which are not poly-nomial. An example of such an expression is, for instance, 𝑒 exp = for 𝑣, 𝑋 = 𝐴 . 𝑋 · 𝑋 , over a schema S with type S ( 𝑣 ) = ( 𝛾, ) , and type S ( 𝑋 ) = ( , ) . Over an instance which assigns 𝑛 to 𝛾 thisexpression computes the function 𝑎 𝑛 , for 𝐴 = [ 𝑎 ] . Therefore, anatural question to ask then is whether we can determine the de-gree of a for - MATLANG expression. Unfortunately, as we show inthe following proposition this question is in fact undecidable.Proposition 5.5.
Given a for - MATLANG expression 𝑒 over a schema S , it is undecidable to check whether 𝑒 is of polynomial degree. Of course, one might wonder whether it is possible to define asyntactic subclass of for - MATLANG expressions that are of polyno-mial degree and can still express many important linear algebraalgorithms. We identify one such class in Section 6.1, called sum - MATLANG , and in fact show that this class is powerful enough tocapture relational algebra on (binary) 𝐾 -relations. The equivalence of for - MATLANG and arithmetic circuits we proveabove assumes that circuits can only use the sum and productgates (note that even without the sum and the product function, for - MATLANG can simulate these operations via matrix sum/product).However, both arithmetic circuits and expressions in for - MATLANG can be allowed to use a multitude of functions over R . The mostnatural addition to the set of functions is the division operator,which is crucially needed in many linear algebra algorithms, suchas, for instance, Gaussian elimination, or 𝐿𝑈 decomposition (re-call Proposition 4.1). Interestingly, the equivalence in this case stillholds, mainly due to a surprising result which shows that (almostall) divisions can in fact be removed for arithmetic circuits whichallow sum, product, and division gates [1].More precisely, in [6, 24, 31] it was shown that for any function ofthe form 𝑓 = 𝑔 / ℎ , where 𝑔 and ℎ are relatively prime polynomials ofdegree 𝑑 , if 𝑓 is computed by an arithmetic circuit of size 𝑠 , then both 𝑔 and ℎ can be computed by a circuit whose size is polynomial in 𝑠 + 𝑑 . Given that we can postpone the division without affecting thefinal result, this, in essence, tells us that division can be eliminated(pushed to the top of the circuit), and we can work with sum-productcircuits instead. The degree of a circuit for 𝑓 , can then be definedas the maximum of degrees of circuits for 𝑔 and ℎ . Given this fact,we can again use the depth reduction procedure of [2], and extendCorollary 5.4 to circuits with division.Corollary 5.6. Let 𝑓 be a function taking as its input matri-ces 𝐴 , . . . , 𝐴 𝑘 of dimensions 𝛼 × 𝛽 , with 𝛼, 𝛽 ∈ { 𝑛, } . Then, 𝑓 is xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA computed by a uniform circuit family over matrices of polynomialdegree that allows divisions, if and only if there is a for - MATLANG [ 𝑓 / ] expression of polynomial degree for 𝑓 . An interesting line of future work here is to see which additionalfunctions can be added to arithmetic circuits and for - MATLANG formulas, in order to preserve their equivalence. Note that this willcrucially depend on the fact that these functions have to allow thedepth reduction of [2] in order to be supported.
We conclude the paper by zooming in on some special fragments of for - MATLANG and in which matrices can take values from an arbi-trary (commutative) semiring 𝐾 . In particular, we first consider sum - MATLANG , in which iterations can only perform additive updates, andshow that it is equivalent in expressive power to the (positive) rela-tional algebra on 𝐾 -relations. We then extend sum - MATLANG suchthat also updates involving pointwise-multiplication (Hadamardproduct) are allowed. The resulting fragment, FO - MATLANG , is shownto be equivalent in expressive power to weighted logics. Finally, weconsider the fragment prod - MATLANG in which updates involvingsum and matrix multiplication, and possibly order information,is allowed. From the results in Section 4, we infer that the latterfragment suffices to compute matrix inversion. An overview of thefragments and their relationships are depicted in Figure 1.
When defining -cliques and in several other expressions we haveseen so far, we only update 𝑋 by adding some matrix to it. Thisrestricted form of for-loop proved useful throughout the paper, andwe therefore introduce it as a special operator. That is, we define: Σ 𝑣.𝑒 : = for 𝑣, 𝑋 . 𝑋 + 𝑒. We define the subfragment of for - MATLANG , called sum - MATLANG ,to consist of the Σ operator plus the “core” operators in MATLANG ,namely, transposition, matrix multiplication and addition, scalarmultiplication, and pointwise function applications.One property of sum - MATLANG is that it only allows expressions ofpolynomial degree. Indeed, one can easily show that sum - MATLANG can only create matrix entries that are polynomial in the dimension 𝑛 of the expression. More precisely, we can show the following:Proposition 6.1. Every expression in sum - MATLANG is of polyno-mial degree.
Interestingly enough, this restricted version of for-loop alreadyallows us to capture the
MATLANG operators that are not presentin the syntax of sum - MATLANG . More precisely, we see from Ex-amples 3.1 and 3.2 that the one-vector and diag operator are ex-pressible in sum - MATLANG . Combined with the observation that the -clique expression of Example 3.3 is in sum - MATLANG , the followingresult is immediate.Corollary 6.2.
MATLANG is strictly subsumed by sum - MATLANG . What operations over matrices can be defined with sum - MATLANG that is beyond
MATLANG ? In [8], it was shown that
MATLANG is strictlyincluded in the (positive) relational algebra on 𝐾 -relations, denoted by RA + 𝐾 [18]. It thus seems natural to compare the expressivepower of sum - MATLANG with RA + 𝐾 . The main result in this sectionis that sum - MATLANG and RA + 𝐾 are equally expressive over binaryschemas. To make this equivalence precise, we next give the defini-tion of RA + 𝐾 [18] and then show how to connect both formalisms.Let D be a data domain and A a set of attributes. A relationalsignature is a finite subset of A . A relational schema is a function R on finite set of symbols dom (R) such that R( 𝑅 ) is a relationsignature for each 𝑅 ∈ dom (R) . To simplify the notation, fromnow on we write 𝑅 to denote both the symbol 𝑅 and the relationalsignature R( 𝑅 ) . Furthermore, we write 𝑅 ∈ R to say that 𝑅 is asymbol of R . For 𝑅 ∈ R , an 𝑅 -tuple is a function 𝑡 : 𝑅 → D . Wedenote by tuples ( 𝑅 ) the set of all 𝑅 -tuples. Given 𝑋 ⊆ 𝑅 , we denoteby 𝑡 [ 𝑋 ] the restriction of 𝑡 to the set 𝑋 .A semiring ( 𝐾, ⊕ , ⊙ , , ) is an algebraic structure where 𝐾 is anon-empty set, ⊕ and ⊙ are binary operations over 𝐾 , and , ∈ 𝐾 .Furthermore, ⊕ and ⊙ are associative operations, and are theidentities of ⊕ and ⊙ respectively, ⊕ is a commutative operation, ⊙ distributes over ⊕ , and annihilates 𝐾 (i.e. ⊙ 𝑘 = 𝑘 ⊙ = ). Asusual, we assume that all semirings in this paper are commutative,namely, ⊙ is also commutative. We use (cid:201) 𝑋 or (cid:199) 𝑋 for the ⊕ -or ⊙ -operation over all elements in 𝑋 , respectively. Typical exam-ples of semirings are the reals ( R , + , × , , ) , the natural numbers ( N , + , × , , ) , and the boolean semiring ({ , } , ∨ , ∧ , , ) .Fix a semiring ( 𝐾, ⊕ , ⊙ , , ) and a relational schema R . A 𝐾 -relation of 𝑅 ∈ R is a function 𝑟 : tuples ( 𝑅 ) → 𝐾 such that thesupport supp ( 𝑟 ) = { 𝑡 ∈ tuples ( 𝑅 ) | 𝑟 ( 𝑡 ) ≠ } is finite. A 𝐾 -instance J of R is a function that assigns relational signatures of R to 𝐾 -relations. Given 𝑅 ∈ R , we denote by 𝑅 J the 𝐾 -relationassociated to 𝑅 . Recall that 𝑅 J is a function and hence 𝑅 J ( 𝑡 ) isthe value in 𝐾 assigned to 𝑡 . Given a 𝐾 -relation 𝑟 we denote by adom ( 𝑟 ) the active domain of 𝑟 defined as adom ( 𝑟 ) = { 𝑡 ( 𝑎 ) | 𝑡 ∈ supp ( 𝑟 ) ∧ 𝑎 ∈ 𝑅 } . Then the active domain of an 𝐾 -instance J of R is defined as adom (J ) = (cid:208) 𝑅 ∈R adom ( 𝑅 J ) .An RA + 𝐾 expression 𝑄 over R is given by the following syntax: 𝑄 : = 𝑅 | 𝑄 ∪ 𝑄 | 𝜋 𝑋 ( 𝑄 ) | 𝜎 𝑋 ( 𝑄 ) | 𝜌 𝑓 ( 𝑄 ) | 𝑄 ⊲⊳ 𝑄 where 𝑅 ∈ R , 𝑋 ⊆ A is finite, and 𝑓 : 𝑋 → 𝑌 is a one to onemapping with 𝑌 ⊆ A . One can extend the schema R to any ex-pression over R recursively as follows: R( 𝑅 ) = 𝑅 , R( 𝑄 ∪ 𝑄 ′ ) = R( 𝑄 ) , R( 𝜋 𝑋 ( 𝑄 )) = 𝑋 , R( 𝜎 𝑋 ( 𝑄 )) = R( 𝑄 ) , R( 𝜌 𝑓 ( 𝑄 )) = 𝑋 where 𝑓 : 𝑋 → 𝑌 , and R( 𝑄 ⊲⊳ 𝑄 ′ ) = R( 𝑄 ) ∪ R( 𝑄 ′ ) for every expres-sions 𝑄 and 𝑄 ′ . We further assume that any expression 𝑄 satisfiesthe following syntactic restrictions: R( 𝑄 ′ ) = R( 𝑄 ′′ ) whenever 𝑄 = 𝑄 ′ ∪ 𝑄 ′′ , 𝑋 ⊆ R( 𝑄 ′ ) whenever 𝑄 = 𝜋 𝑋 ( 𝑄 ′ ) or 𝑄 = 𝜎 𝑋 ( 𝑄 ′ ) ,and 𝑌 = R( 𝑄 ′ ) whenever 𝑄 = 𝜌 𝑓 ( 𝑄 ′ ) with 𝑓 : 𝑋 → 𝑌 .Given an RA + 𝐾 expression 𝑄 and a 𝐾 -instance J of R , we definethe semantics ⟦ 𝑄 ⟧ J as a 𝐾 -relation of R( 𝑄 ) as follows. For 𝑋 ⊆ A ,let Eq 𝑋 ( 𝑡 ) = when 𝑡 ( 𝑎 ) = 𝑡 ( 𝑏 ) for every 𝑎, 𝑏 ∈ 𝑋 , and Eq 𝑋 ( 𝑡 ) = The algebra used in [8] differs slightly from the one given in [18]. In this paper wework with the original algebra RA + 𝐾 as defined in [18]. onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč otherwise. For every tuple 𝑡 ∈ R( 𝑄 ) :if 𝑄 = 𝑅 , then ⟦ 𝑄 ⟧ J ( 𝑡 ) = 𝑅 J ( 𝑡 ) if 𝑄 = 𝑄 ∪ 𝑄 , then ⟦ 𝑄 ⟧ J ( 𝑡 ) = ⟦ 𝑄 ⟧ J ( 𝑡 ) ⊕ ⟦ 𝑄 ⟧ J ( 𝑡 ) if 𝑄 = 𝜋 𝑋 ( 𝑄 ′ ) , then ⟦ 𝑄 ⟧ J ( 𝑡 ) = (cid:201) 𝑡 ′ : 𝑡 ′ [ 𝑋 ] = 𝑡 ⟦ 𝑄 ′ ⟧ J ( 𝑡 ′ ) if 𝑄 = 𝜎 𝑋 ( 𝑄 ′ ) , then ⟦ 𝑄 ⟧ J ( 𝑡 ) = ⟦ 𝑄 ′ ⟧ J ( 𝑡 ) ⊙ Eq 𝑋 ( 𝑡 ) if 𝑄 = 𝜌 𝑓 ( 𝑄 ′ ) , then ⟦ 𝑄 ⟧ J ( 𝑡 ) = ⟦ 𝑄 ′ ⟧ J ( 𝑡 ◦ 𝑓 ) if 𝑄 = 𝑄 ⊲⊳ 𝑄 , then ⟦ 𝑄 ⟧ J ( 𝑡 ) = ⟦ 𝑄 ⟧ J ( 𝑡 [ 𝑌 ]) ⊙ ⟦ 𝑄 ⟧ J ( 𝑡 [ 𝑍 ]) , where 𝑌 = R( 𝑄 ) and 𝑍 = R( 𝑄 ) . It is important to note that the (cid:201) -operation in the semantics of 𝜋 𝑋 ( 𝑄 ′ ) is well-defined given thatthe support of ⟦ 𝑄 ′ ⟧ J is always finite.We are now ready for comparing sum - MATLANG with RA + 𝐾 . Firstof all, we need to extend sum - MATLANG from R to any semiring. Let Mat [ 𝐾 ] denote the set of all 𝐾 -matrices. Similarly as for MATLANG over R , given a MATLANG schema S , a 𝐾 -instance I over S is apair I = (D , mat ) , where D : Symb ↦→ N assigns a value toeach size symbol, and mat : M ↦→
Mat [ 𝐾 ] assigns a concrete 𝐾 -matrix to each matrix variable. Then it is straightforward to extendthe semantics of MATLANG , for - MATLANG , and sum - MATLANG from ( R , + , × , , ) to ( 𝐾, ⊕ , ⊙ , , ) by switching + with ⊕ and × with ⊙ .The next step to compare sum - MATLANG with RA + 𝐾 is to represent 𝐾 -matrices as 𝐾 -relations. Let S = (M , size ) be a MATLANG schema.On the relational side we have for each size symbol 𝛼 ∈ Symb \ { } ,attributes 𝛼 , row 𝛼 , and col 𝛼 in A . Furthermore, for each 𝑉 ∈ M and 𝛼 ∈ Symb we denote by 𝑅 𝑉 and 𝑅 𝛼 its corresponding relation name,respectively. Then, given S we define the relational schema Rel (S) such that dom ( Rel (S)) = { 𝑅 𝛼 | 𝛼 ∈ Symb } ∪ { 𝑅 𝑉 | 𝑉 ∈ M} whereRel (S)( 𝑅 𝛼 ) = { 𝛼 } and:Rel (S)( 𝑅 𝑉 ) = { row 𝛼 , col 𝛽 } if size ( 𝑉 ) = ( 𝛼, 𝛽 ){ row 𝛼 } if size ( 𝑉 ) = ( 𝛼, ){ col 𝛽 } if size ( 𝑉 ) = ( , 𝛽 ){} if size ( 𝑉 ) = ( , ) . Consider now a matrix instance I = (D , mat ) over S . Let 𝑉 ∈ M with size ( 𝑉 ) = ( 𝛼, 𝛽 ) and let mat ( 𝑉 ) be its corresponding 𝐾 -matrix of dimension D( 𝛼 ) × D( 𝛽 ) . To encode I as a 𝐾 -instance in RA + 𝐾 , we use as data domain D = N \ { } . Then we construct the 𝐾 -instance Rel (I) such that for each 𝑉 ∈ M we define 𝑅 Rel (I) 𝑉 ( 𝑡 ) : = mat ( 𝑉 ) 𝑖 𝑗 whenever 𝑡 ( row 𝛼 ) = 𝑖 ≤ D( 𝛼 ) and 𝑡 ( col 𝛽 ) = 𝑗 ≤D( 𝛽 ) , and otherwise. Furthermore, for each 𝛼 ∈ Symb we de-fine 𝑅 Rel (I) 𝛼 ( 𝑡 ) : = whenever 𝑡 ( 𝛼 ) ≤ D( 𝛼 ) , and otherwise. Inother words, 𝑅 𝛼 and 𝑅 𝛽 encodes the active domain of a matrixvariable 𝑉 with size ( 𝑉 ) = ( 𝛼, 𝛽 ) . Given that the RA + 𝐾 frameworkof [18] represents the “absence” of a tuple in the relation with ,we need to separately encode the indexes in a matrix. This is where 𝑅 Rel (I) 𝛼 and 𝑅 Rel (I) 𝛽 are used for. We are now ready to state the firstconnection between sum - MATLANG and RA + 𝐾 by using the previousencoding. The proof of the proposition below is by induction onthe structure of expressions.Proposition 6.3. For each sum - MATLANG expression 𝑒 over schema S such that S( 𝑒 ) = ( 𝛼, 𝛽 ) with 𝛼 ≠ ≠ 𝛽 , there exists an RA + 𝐾 ex-pression Φ ( 𝑒 ) over relational schema Rel (S) such that Rel (S)( Φ ( 𝑒 )) = { row 𝛼 , row 𝛽 } and such that for any instance I over S , ⟦ 𝑒 ⟧(I) 𝑖,𝑗 = ⟦ Φ ( 𝑒 )⟧ Rel (I) ( 𝑡 ) for tuple 𝑡 ( row 𝛼 ) = 𝑖 and 𝑡 ( col 𝛽 ) = 𝑗 . Similarly for when 𝑒 hasschema S( 𝑒 ) = ( 𝛼, ) , S( 𝑒 ) = ( , 𝛽 ) or S( 𝑒 ) = ( , ) , then Φ ( 𝑒 ) has schema Rel (S)( Φ ( 𝑒 )) = { row 𝛼 } , Rel (S)( Φ ( 𝑒 )) = { col 𝛼 } , orRel (S)( Φ ( 𝑒 )) = {} , respectively. We now move to the other direction. To translate RA + 𝐾 into sum - MATLANG , we must restrict our comparison to RA + 𝐾 over 𝐾 -relationswith at most two attributes. Given that linear algebra works overvector and matrices, it is reasonable to restrict to unary or binaryrelations as input. Note that this is only a restriction on the inputrelations and not on intermediate relations, namely, expressions cancreate relation signatures of arbitrary size from the binary inputrelations. Thus, from now we say that a relational schema R isbinary if | 𝑅 | ≤ for every 𝑅 ∈ R . We also make the assumptionthat there is an (arbitrary) order, denoted by < , on the attributesin A . This is to identify which attributes correspond to rows andcolumns when moving to matrices. Then, given that relations willbe either unary or binary and there is an order on the attributes, wewrite 𝑡 = ( 𝑣 ) or 𝑡 = ( 𝑢, 𝑣 ) to denote a tuple over a unary or binaryrelation 𝑅 , respectively, where 𝑢 and 𝑣 is the value of the first andsecond attribute with respect to < .Consider a binary relational schema R . With each 𝑅 ∈ R weassociate a matrix variable 𝑉 𝑅 such that, if 𝑅 is a binary relationalsignature, then 𝑉 𝑅 represents a (square) matrix, and, if not (i.e. 𝑅 is unary), then 𝑉 𝑅 represents a vector. Formally, fix a symbol 𝛼 ∈ Symb \ { } . Let Mat (R) denote the MATLANG schema (M R , size R ) such that M R = { 𝑉 𝑅 | 𝑅 ∈ R} and size R ( 𝑉 𝑅 ) = ( 𝛼, 𝛼 ) whenever | 𝑅 | = , and size R ( 𝑉 𝑅 ) = ( 𝛼, ) whenever | 𝑅 | = . Take nowa 𝐾 -instance J of R and suppose that adom (J ) = { 𝑑 , . . . , 𝑑 𝑛 } is the active domain of J (the order over adom (J ) is arbitrary).Then we define the matrix instance Mat (J ) = (D J , mat J ) suchthat D J ( 𝛼 ) = 𝑛 , mat J ( 𝑉 𝑅 ) 𝑖,𝑗 = 𝑅 J (( 𝑑 𝑖 , 𝑑 𝑗 )) whenever | 𝑅 | = ,and mat J ( 𝑉 𝑅 ) 𝑖 = 𝑅 J (( 𝑑 𝑖 )) whenever | 𝑅 | = . Note that, althougheach 𝐾 -relation can have a different active domain, we encodethem as square matrices by considering the active domain of the 𝐾 -instance. By again using an inductive proof on the structure of RA + 𝐾 expressions, we obtain the following result.Proposition 6.4. Let R be a binary relational schema. For each RA + 𝐾 expression 𝑄 over R such that |R( 𝑄 )| = , there exists a sum - MATLANG expression Ψ ( 𝑄 ) over MATLANG schema Mat (R) such thatfor any 𝐾 -instance J with adom (J ) = { 𝑑 , . . . , 𝑑 𝑛 } over R , ⟦ 𝑄 ⟧ J (( 𝑑 𝑖 , 𝑑 𝑗 )) = ⟦ Ψ ( 𝑄 )⟧( Mat (J )) 𝑖,𝑗 . Similarly for when |R( 𝑄 )| = , or |R( 𝑄 )| = respectively. It is important to remark that the expression 𝑄 of the previousresult can have intermediate expressions that are not necessarybinary, given that the proposition only restricts that the input rela-tion and the schema of 𝑄 must have arity at most two. We recallfrom [8] that MATLANG corresponds to RA + 𝐾 where intermediate ex-pressions are at most ternary, and this underlies, e.g., the inability of MATLANG to check for -cliques. In sum - MATLANG , we can deal withintermediate relations of arbitrary arity. In fact, each new attributecan be seen to correspond to an application of the Σ operator. Forexample, in the -clique expression, four Σ operators are needed,in analogy to how -clique is expressed in RA + 𝐾 . xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA Given the previous two propositions we derive the followingconclusion which is the first characterization of relational algebrawith a (sub)-fragment of linear algebra.Corollary 6.5. sum - MATLANG and RA + 𝐾 over binary relationalschemas are equally expressive. As a direct consequence, we have that sum - MATLANG cannot com-pute matrix inversion. Indeed, using similar arguments as in [7],i.e., by embedding RA + 𝐾 in (infinitary) first-order logic with count-ing and by leveraging its locality, one can show that sum - MATLANG cannot compute the transitive closure of an adjacency matrix. Bycontrast, the transitive closure can be expressed by means of matrixinversion [7]. We also note that the evaluation of the Σ operator isindependent of the order in which the canonical vectors are con-sidered. This is because ⊕ is commutative. Hence, sum - MATLANG cannot express the order predicates mentioned in Section 3.
Similarly to using sum, we can use other operations to update 𝑋 in the for-loop. The next natural choice is to consider products ofmatrices. In contrast to matrix sum, we have two options: eitherwe can choose to use matrix product or to use the pointwise matrixproduct, also called the Hadamard product. We treat matrix productin the next subsection and first explain here the connection of sumand Hadamard product operators to weighted logics.For the rest of this section, fix a semiring ( 𝐾, ⊕ , ⊙ , , ) . TheHadamard product over 𝐾 -matrices can be defined as the pointwiseapplication of ⊙ between two matrices of the same size. Formally,we define the expression 𝑒 ◦ 𝑒 ′ where 𝑒, 𝑒 ′ are expressions withrespect to S and type S ( 𝑒 ) = type S ( 𝑒 ′ ) for some schema S = (M , size ) . Then the semantics of 𝑒 ◦ 𝑒 ′ is the pointwise applicationof ⊙ , namely, ⟦ 𝑒 ◦ 𝑒 ′ ⟧(I) 𝑖 𝑗 = ⟦ 𝑒 ⟧(I) 𝑖 𝑗 ⊙⟦ 𝑒 ′ ⟧(I) 𝑖 𝑗 for any instance I of S . This enables us to define, similar as for Σ 𝑣 , the pointwise-product quantifier Π ◦ 𝑣 as follows: Π ◦ 𝑣. 𝑒 : = for 𝑣, 𝑋 = . 𝑋 ◦ 𝑒. where is a matrix with the same type as 𝑋 and all entries equal tothe -element of 𝐾 (i.e., we need to initialize 𝑋 accordingly with the ⊙ -operator). We cal FO - MATLANG the subfragment of for - MATLANG that consists of sum - MATLANG extended with Π ◦ 𝑣 . Example 6.6.
Similar to the trace of a matrix, a useful functionin linear algebra is to compute the product of the values on thediagonal. Using the Π ◦ 𝑣 operator, this can be easily expressed: 𝑒 dp ( 𝑉 ) : = Π ◦ 𝑣. 𝑣 𝑇 · 𝑉 · 𝑣. Clearly, the inclusion of this new operator extends the expressivepower to sum - MATLANG . For example, ⟦ 𝑒 dp ⟧(I) can be an exponen-tially large number in the dimension 𝑛 of the input. By contrast, onecan easily show that all expressions in sum - MATLANG can only returnnumbers polynomial in 𝑛 . That is, FO - MATLANG is more expressivethan sum - MATLANG and RA + 𝐾 .To measure the expressive power of FO - MATLANG , we use weightedlogics [13] (WL) as a yardstick. Weighted logics extend monadicsecond-order logic from the boolean semiring to any semiring 𝐾 .Furthermore, it has been used extensively to characterize the ex-pressive power of weighted automata in terms of logic [14]. Weuse here the first-order subfragment of weighted logics to suit our purpose and, moreover, we extend its semantics over weightedstructures (similar as in [17]).A relational vocabulary Γ is a finite collection of relation sym-bols such that each 𝑅 ∈ Γ has an associated arity, denoted by arity ( 𝑅 ) . A 𝐾 -weighted structure over Γ (or just structure) is a pair A = ( 𝐴, { 𝑅 A } 𝑅 ∈ Γ ) such that 𝐴 is a non-empty finite set (i.e. thedomain) and, for each 𝑅 ∈ Γ , 𝑅 A : 𝐴 arity ( 𝑅 ) → 𝐾 is a functionthat associates to each tuple in 𝐴 arity ( 𝑅 ) a weight in 𝐾 .Let 𝑋 be a set of first-order variables. A 𝐾 -weighted logic (WL)formula 𝜑 over Γ is defined by the following syntax: 𝜑 : = 𝑥 = 𝑦 | 𝑅 ( ¯ 𝑥 ) | 𝜑 ⊕ 𝜑 | 𝜑 ⊙ 𝜑 | Σ 𝑥.𝜑 | Π 𝑥.𝜑 where 𝑥, 𝑦 ∈ 𝑋 , 𝑅 ∈ Γ , and ¯ 𝑥 = 𝑥 , . . . , 𝑥 𝑘 is a sequence of variablesin 𝑋 such that 𝑘 = arity ( 𝑅 ) . As usual, we say that 𝑥 is a freevariable of 𝜑 , if 𝑥 is not below Σ 𝑥 or Π 𝑥 quantifiers (e.g. 𝑥 is free in Σ 𝑦.𝑅 ( 𝑥, 𝑦 ) but 𝑦 is not). Given that 𝐾 is fixed, from now on we talkabout structures and formulas without mentioning 𝐾 explicitly.An assignment 𝜎 over a structure A = ( 𝐴, { 𝑅 A } 𝑅 ∈ Γ ) is a func-tion 𝜎 : 𝑋 → 𝐴 . Given 𝑥 ∈ 𝑋 and 𝑎 ∈ 𝐴 , we denote by 𝜎 [ 𝑥 ↦→ 𝑎 ] anew assignment such that 𝜎 [ 𝑥 ↦→ 𝑎 ]( 𝑦 ) = 𝑎 whenever 𝑥 = 𝑦 and 𝜎 [ 𝑥 ↦→ 𝑎 ]( 𝑦 ) = 𝜎 ( 𝑦 ) otherwise. For ¯ 𝑥 = 𝑥 , . . . , 𝑥 𝑘 , we write 𝜎 ( ¯ 𝑥 ) to say 𝜎 ( 𝑥 ) , . . . , 𝜎 ( 𝑥 𝑘 ) . Given a structure A = ( 𝐴, { 𝑅 A } 𝑅 ∈ Γ ) andan assignment 𝜎 , we define the semantics ⟦ 𝜑 ⟧ A ( 𝜎 ) of 𝜑 as follows:if 𝜑 : = 𝑥 = 𝑦 , then ⟦ 𝜑 ⟧ A ( 𝜎 ) = (cid:26) if 𝜎 ( 𝑥 ) = 𝜎 ( 𝑦 ) otherwiseif 𝜑 : = 𝑅 ( ¯ 𝑥 ) , then ⟦ 𝜑 ⟧ A ( 𝜎 ) = 𝑅 A ( 𝜎 ( ¯ 𝑥 )) if 𝜑 : = 𝜑 ⊕ 𝜑 , then ⟦ 𝜑 ⟧ A ( 𝜎 ) = ⟦ 𝜑 ⟧ A ( 𝜎 ) ⊕ ⟦ 𝜑 ⟧ A ( 𝜎 ) if 𝜑 : = 𝜑 ⊙ 𝜑 , then ⟦ 𝜑 ⟧ A ( 𝜎 ) = ⟦ 𝜑 ⟧ A ( 𝜎 ) ⊙ ⟦ 𝜑 ⟧ A ( 𝜎 ) if 𝜑 : = Σ 𝑥. 𝜑 ′ , then ⟦ 𝜑 ⟧ A ( 𝜎 ) = (cid:201) 𝑎 ∈ 𝐴 ⟦ 𝜑 ′ ⟧ A ( 𝜎 [ 𝑥 ↦→ 𝑎 ]) if 𝜑 : = Π 𝑥. 𝜑 ′ , then ⟦ 𝜑 ⟧ A ( 𝜎 ) = (cid:199) 𝑎 ∈ 𝐴 ⟦ 𝜑 ′ ⟧ A ( 𝜎 [ 𝑥 ↦→ 𝑎 ]) When 𝜑 contains no free variables, we omit 𝜎 and write ⟦ 𝜑 ⟧ A instead of ⟦ 𝜑 ⟧ A ( 𝜎 ) .For comparing the expressive power of FO - MATLANG with WL, wehave to show how to encode
MATLANG instances into structures andvice versa. For this, we make two assumptions to put both languagesat the same level: (1) we restrict structures to relation symbols ofarity at most two and (2) we restrict instances to square matrices.The first assumption is for the same reasons as when comparing sum - MATLANG with RA + 𝐾 , and the second assumption is to have acrisp translation between both languages. Indeed, understandingthe relation of FO - MATLANG with WL for non-square matrices isslightly more complicated and we leave this for future work.Let S = (M , size ) be a schema of square matrices, that is, thereexists an 𝛼 such that size ( 𝑉 ) ∈ { , 𝛼 } × { , 𝛼 } for every 𝑉 ∈ M .Define the relational vocabulary WL (S) = { 𝑅 𝑉 | 𝑉 ∈ M} such that arity ( 𝑅 𝑉 ) = if size ( 𝑉 ) = ( 𝛼, 𝛼 ) , arity ( 𝑅 𝑉 ) = if size ( 𝑉 ) ∈{( 𝛼, ) , ( , 𝛼 )} , and arity ( 𝑅 𝑉 ) = otherwise. Then given a ma-trix instance I = (D , mat ) over S define the structure WL (I) = ({ , . . . , 𝑛 } , { 𝑅 I 𝑉 }) such that D( 𝛼 ) = 𝑛 and 𝑅 I 𝑉 ( 𝑖, 𝑗 ) = mat ( 𝑉 ) 𝑖,𝑗 if size ( 𝑉 ) = ( 𝛼, 𝛼 ) , 𝑅 I 𝑉 ( 𝑖 ) = mat ( 𝑉 ) 𝑖 if size ( 𝑉 ) ∈ {( 𝛼, ) , ( , 𝛼 )} ,and 𝑅 I 𝑉 = mat ( 𝑉 ) if size ( 𝑉 ) = ( , ) .To encode weighted structures into matrices and vectors, thestory is similar as for RA + 𝐾 . Let Γ be a relational vocabulary where arity ( 𝑅 ) ≤ . Define Mat ( Γ ) = (M Γ , size Γ ) such that M Γ = { 𝑉 𝑅 | 𝑅 ∈ Γ } and size Γ ( 𝑉 𝑅 ) is equal to ( 𝛼, 𝛼 ) , ( 𝛼, ) , or ( , ) if onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč arity ( 𝑅 ) = , arity ( 𝑅 ) = , or arity ( 𝑅 ) = , respectively, for some 𝛼 ∈ Symb . Similarly, let A = ( 𝐴, { 𝑅 A } 𝑅 ∈ Γ ) be a structure with 𝐴 = { 𝑎 , . . . , 𝑎 𝑛 } , ordered arbitrarily. Then we define the matrixinstance Mat (A) = (D , mat ) such that D( 𝛼 ) = 𝑛 , mat ( 𝑉 𝑅 ) 𝑖,𝑗 = 𝑅 A ( 𝑎 𝑖 , 𝑎 𝑗 ) if arity ( 𝑅 ) = , mat ( 𝑉 𝑅 ) 𝑖 = 𝑅 A ( 𝑎 𝑖 ) if arity ( 𝑅 ) = ,and mat ( 𝑉 𝑅 ) = 𝑅 A otherwise.Let S be a MATLANG schema of square matrices and Γ a relationalvocabulary of relational symbols of arity at most . We can thenshow the equivalence of FO - MATLANG and WL as follows.Proposition 6.7.
Weighted logics over Γ and FO - MATLANG over S have the same expressive power. More specifically, • for each FO - MATLANG expression 𝑒 over S such that S( 𝑒 ) = ( , ) , there exists a WL-formula Φ ( 𝑒 ) over WL (S) such thatfor every instance I of S , ⟦ 𝑒 ⟧(I) = ⟦ Φ ( 𝑒 )⟧ WL (I) . • for each WL-formula 𝜑 over Γ without free variables, there ex-ists a FO - MATLANG expression Ψ ( 𝜑 ) such that for any structure A over Mat ( Γ ) , ⟦ 𝜑 ⟧ A = ⟦ Ψ ( 𝜑 )⟧( Mat (A)) . In a similar way, we can consider a fragment in which sum and theusual product of matrices can be used in for-loops. Formally, for anexpression 𝑒 we define the operator: Π 𝑣. 𝑒 = for 𝑣, 𝑋 = 𝐼 . 𝑋 · 𝑒. where 𝐼 is the identity matrix. We call prod - MATLANG the subfrag-ment of for - MATLANG that consists of sum - MATLANG extended with Π 𝑣 . It is readily verified that Π ◦ 𝑣 can expressed in terms of Π 𝑣 .Furthermore, by contrast to the Hadamard product, matrix multi-plication is a non-commutative operator. As a consequence, onecan formulate expressions that are not invariant under the order inwhich the canonical vectors are processed.Proposition 6.8. Every expression in FO - MATLANG can be definedin prod - MATLANG . Moreover, there exists an expression that uses the Π 𝑣 quantifier that cannot be defined in FO - MATLANG . What is interesting is that sum - MATLANG extended with Π 𝑣 suf-fices to compute the transitive closure, provided that we allow forthe 𝑓 > function. Indeed, one can use the expression 𝑒 TC ( 𝑉 ) : = 𝑓 > (cid:0) Π 𝑣. ( 𝑒 Id + 𝑉 ) (cid:1) for this purpose because ⟦ 𝑒 TC ⟧(I) = 𝑓 > (cid:0) ( 𝐼 + 𝐴 ) 𝑛 (cid:1) when I assigns an 𝑛 × 𝑛 adjacency matrix 𝐴 to 𝑉 , and non-zeroentries in ( 𝐼 + 𝐴 ) 𝑛 coincide with non-zero entries in the transitiveclosure of 𝐴 . Furthermore, if we extend this fragment with accessto the matrix 𝑆 < , defining the (strict) order on canonical vectors,then Csanky’s matrix inversion algorithm becomes expressible (if 𝑓 / is allowed). We leave the study of this fragment and, in particular,the relationship to full for - MATLANG , for future work.Finally, in Figure 1 we show a diagram of all the fragments of for - MATLANG introduced in this section and their correspondingequivalent formalisms.
We proposed for - MATLANG , an extension of
MATLANG with limitedrecursion, and showed that it is able to capture most of linear algebradue to its connection to arithmetic circuits. We further revealedinteresting connections to logics on annotated relations. Our focuswas on language design and expressivity. An interesting direction
MLsum - ML ≡ RA + 𝐾 FO - ML ≡ WL DP prod - ML + 𝑆 < InvDet for - ML ≡ Arithmetic Circuits PLU
Figure 1: Fragments of for - MATLANG and their equivalences.The functions 4Cliqe, DP (diagonal product), Inv, Det,and PLU decomposition are placed in their fragments. for future work relates to efficient evaluation of (fragments) of for - MATLANG . A possible starting point is [9] in which a generalmethodology for communication-optimal algorithms for for-looplinear algebra programs is proposed.
REFERENCES [1] Eric Allender. 2004. Arithmetic circuits and counting complexity classes.
Com-plexity of Computations and Proofs, Quaderni di Matematica
13 (2004), 33–72.[2] Eric Allender, Jia Jiao, Meena Mahajan, and V. Vinay. 1998. Non-CommutativeArithmetic Circuits: Depth Reduction and Size Lower Bounds.
Theor. Comput.Sci.
ICDT . 6:1–6:20.[5] Matthias Boehm, Arun Kumar, and Jun Yang. 2019.
Data Management in MachineLearning Systems . Morgan & Claypool Publishers.[6] Allan Borodin, Joachim von zur Gathem, and John Hopcroft. 1982. Fast parallelmatrix and GCD computations. In
FOCS . 65–71.[7] Robert Brijder, Floris Geerts, Jan Van den Bussche, and Timmy Weerwag. 2019.On the Expressive Power of Query Languages for Matrices.
ACM Trans. DatabaseSyst.
44, 4 (2019), 15:1–15:31.[8] Robert Brijder, Marc Gyssens, and Jan Van den Bussche. 2020. On Matrices and 𝐾 -Relations. In FoIKS . 42–57.[9] Michael Christ, James Demmel, Nicholas Knight, Thomas Scanlon, and Kather-ine A. Yelick. 2013. Communication Lower Bounds and Optimal Algorithms forPrograms that Reference Arrays - Part 1. (2013).[10] Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein.2001.
Introduction to Algorithms (2nd ed.). The MIT Press.[11] L. Csanky. 1976. Fast Parallel Matrix Inversion Algorithms.
SIAM J. Comput.
5, 4(1976), 618–623.[12] Anuj Dawar, Martin Grohe, Bjarki Holm, and Bastian Laubner. 2009. Logics withRank Operators. In
LICS . 113–122.[13] Manfred Droste and Paul Gastin. 2005. Weighted Automata and Weighted Logics.In
ICALP , Vol. 3580. 513–525.[14] Manfred Droste, Werner Kuich, and Heiko Vogler. 2009.
Handbook of weightedautomata . Springer Science & Business Media.[15] Floris Geerts. 2019. On the Expressive Power of Linear Algebra on Graphs. In
ICDT . 7:1–7:19.[16] Floris Geerts. 2020. When Can Matrix Query Languages Discern Matrices?. In
ICDT . 12:1–12:18.[17] Erich Grädel and Val Tannen. 2017. Semiring Provenance for First-Order ModelChecking.
CoRR abs/1712.01980 (2017).[18] Todd J. Green, Gregory Karvounarakis, and Val Tannen. 2007. Provenancesemirings. In
PODS . 31–40.[19] Martin Grohe and Wied Pakusa. 2017. Descriptive complexity of linear equationsystems and applications to propositional proof complexity. In
LICS . 1–12.[20] Lauri Hella, Leonid Libkin, Juha Nurmonen, and Limsoon Wong. 2001. Logicswith Aggregate Operators.
J. ACM
48, 4 (2001), 880–907.[21] Bjarki Holm. 2010.
Descriptive Complexity of Linear Algebra . Ph.D. Dissertation.University of Cambridge.[22] Dylan Hutchison, Bill Howe, and Dan Suciu. 2017. LaraDB: A Minimalist Kernelfor Linear and Relational Algebra Computation. In
BeyondMR . 2:1–2:10.[23] Dimitrije Jankov, Shangyu Luo, Binhang Yuan, Zhuhua Cai, Jia Zou, Chris Jer-maine, and Zekai J. Gao. 2019. Declarative Recursive Computation on an RDBMS. xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA
Proc. VLDB Endow.
12, 7 (2019), 822–835.[24] Erich Kaltofen. 1988. Greatest common divisors of polynomials given by straight-line programs.
Journal of the ACM (JACM)
35, 1 (1988), 231–264.[25] Mahmoud Abo Khamis, Hung Q. Ngo, XuanLong Nguyen, Dan Olteanu, andMaximilian Schleich. 2018. AC/DC: In-Database Learning Thunderstruck. In
DEEM , Sebastian Schelter, Stephan Seufert, and Arun Kumar (Eds.). 8:1–8:10.[26] Andreas Kunft, Alexander Alexandrov, Asterios Katsifodimos, and Volker Markl.2016. Bridging the Gap: Towards Optimization Across Linear and RelationalAlgebra. In
BeyondMR . 1:1–1:4.[27] Shangyu Luo, Zekai J. Gao, Michael Gubanov, Luis L. Perez, and ChristopherJermaine. 2018. Scalable Linear Algebra on a Relational Database System.
SIGMODRec.
47, 1 (2018), 24–31.[28] William H. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery.1992.
Numerical Recipes in C, 2nd Edition . Cambridge University Press.[29] Ran Raz. 2003. On the Complexity of Matrix Product.
SIAM J. Comput.
32, 5(2003), 1356–1369.[30] Amir Shpilka and Amir Yehudayoff. 2010. Arithmetic Circuits: A survey of recentresults and open questions.
Foundations and Trends in Theoretical ComputerScience
5, 3-4 (2010), 207–388.[31] Volker Strassen. 1973. Vermeidung von Divisionen.
Journal für die reine undangewandte Mathematik
264 (1973), 184–202.[32] Leslie G Valiant and Sven Skyum. 1981. Fast parallel computation of polynomialsusing few processors. In
International Symposium on Mathematical Foundationsof Computer Science . Springer, 132–139. onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč
APPENDIXA PRELIMINARIES
We first introduce some additional notations and describe simplifications that will be used later in the appendix.
A.1 Definitions
We sometimes want to iterate over 𝑘 canonical vectors. We define the following shorthand notation: for 𝑣 , . . . , 𝑣 𝑘 , 𝑋 . 𝑒 ( 𝑋, 𝑣 , . . . , 𝑣 𝑛 ) : = for 𝑣 , 𝑋 . 𝑋 + for 𝑣 , 𝑋 = 𝑋 . 𝑋 + for 𝑣 , 𝑋 = 𝑋 . 𝑋 + . . . for 𝑣 𝑘 , 𝑋 𝑘 = 𝑋 𝑘 − . 𝑒 ( 𝑋 𝑘 , 𝑣 , . . . , 𝑣 𝑘 ) . To reference ℓ different vector variables 𝑋 , . . . , 𝑋 ℓ in every iteration and update them in different ways we define: for 𝑣, 𝑋 , . . . , 𝑋 ℓ . ( 𝑒 ( 𝑋 , 𝑣 ) , 𝑒 ( 𝑋 , 𝑣 ) , . . . , 𝑒 𝑙 ( 𝑋 ℓ , 𝑣 )) : = for 𝑣, 𝑋 . 𝑒 ( 𝑋 · 𝑒 min , 𝑣 ) · ( 𝑒 diag ( 𝑒 ( 𝑋 𝑇 )) · 𝑒 min ) 𝑇 + 𝑒 ( 𝑋 · 𝑒 min + , 𝑣 ) · ( 𝑒 diag ( 𝑒 ( 𝑋 𝑇 )) · 𝑒 min + ) 𝑇 + . . . + 𝑒 ℓ ( 𝑋 · 𝑒 max , 𝑣 ) · ( 𝑒 diag ( 𝑒 ( 𝑋 𝑇 )) · 𝑒 max ) 𝑇 We note that for the latter expression to be semantically correct 𝑣 has to be of type 𝛾 × , both 𝑋 𝑖 and 𝑒 𝑖 for 𝑖 = , . . . , ℓ have to be of type 𝛼 × , and 𝑋 has to be of type 𝛼 × 𝛽 , where D( 𝛽 ) = ℓ . Here we use 𝑒 diag ( 𝑒 ( 𝑋 𝑇 )) to compute the 𝛽 × 𝛽 identity and ensure the typing of the 𝑒 min + 𝑖 . When evaluated on an instance I , 𝑒 min , 𝑒 min + 𝑖 evaluate to 𝑏 D ( 𝛽 ) and 𝑏 D ( 𝛽 ) + 𝑖 , respectively, and we show their defining expressions insection B.1. Similarly for 𝑒 max = 𝑏 D ( 𝛽 ) 𝑛 . The combinations of both previous operators results in: for 𝑣 , . . . , 𝑣 𝑘 , 𝑋 , . . . , 𝑋 ℓ . ( 𝑒 ( 𝑋 , 𝑣 , . . . , 𝑣 𝑘 ) , 𝑒 ( 𝑋 , 𝑣 , . . . , 𝑣 𝑘 ) , . . . , 𝑒 ℓ ( 𝑋 ℓ , 𝑣 , . . . , 𝑣 𝑘 )) : = for 𝑣 , . . . , 𝑣 𝑘 , 𝑋 . 𝑒 ′ ( 𝑋, 𝑣 , . . . , 𝑣 𝑘 ) where 𝑒 ′ ( 𝑋, 𝑣 , . . . , 𝑣 𝑘 ) : = 𝑒 ( 𝑋 · 𝑒 min , 𝑣 , . . . , 𝑣 𝑘 ) · ( 𝑒 diag ( 𝑒 ( 𝑋 𝑇 )) · 𝑒 min ) 𝑇 + 𝑒 ( 𝑋 · 𝑒 min + , 𝑣 , . . . , 𝑣 𝑘 ) · ( 𝑒 diag ( 𝑒 ( 𝑋 𝑇 )) · 𝑒 min + ) 𝑇 + . . . + 𝑒 ℓ ( 𝑋 · 𝑒 max , 𝑣 , . . . , 𝑣 𝑘 ) · ( 𝑒 diag ( 𝑒 ( 𝑋 𝑇 )) · 𝑒 max ) 𝑇 (1)(2)(3)It is clear that this expression iterates over 𝑘 canonical vectors and references ℓ independent vectors updating each of them in their particularway. A.2 Simplifications
When showing results based on induction of expressions in for - MATLANG , it is often convenient to assume that function applications 𝑓 ( 𝑒 , . . . , 𝑒 𝑘 ) for 𝑓 ∈ F 𝑘 are restricted to the case when all expressions 𝑒 , . . . , 𝑒 𝑘 have type × . This does not loose generality. Indeed, forgeneral function applications 𝑓 ( 𝑒 , . . . , 𝑒 𝑘 ) , if we have Σ , scalar product and function application on scalars (here denoted by 𝑓 × ), we cansimulate full function application, as follows: 𝑓 ( 𝑒 , . . . , 𝑒 𝑘 ) : = Σ 𝑣 𝑖 Σ 𝑣 𝑗 .𝑓 × ( 𝑣 𝑇𝑖 · 𝑒 · 𝑣 𝑗 , . . . , 𝑣 𝑇𝑖 · 𝑒 𝑘 · 𝑣 𝑗 ) × 𝑣 𝑖 · 𝑣 𝑇𝑗 . Furthermore, it also convenient at times to use the pointwise functions 𝑓 𝑘 ⊙ : R 𝑘 ↦→ R : ( 𝑥 , . . . , 𝑥 𝑘 ) ↦→ 𝑥 × · · · · 𝑥 𝑘 and 𝑓 𝑘 ⊕ : R 𝑘 ↦→ R : ( 𝑥 , . . . , 𝑥 𝑘 ) ↦→ 𝑥 + · · · + 𝑥 𝑘 . In fact, it is readily observed that adding these functions does not extend the expressive power of for - MATLANG :Lemma A.1.
We have that for - MATLANG [∅] ≡ for - MATLANG [{ 𝑓 𝑘 ⊙ , 𝑓 𝑘 ⊕ | 𝑘 ∈ N }] . In fact, this lemma also holds for the smaller fragments we consider. We also observe that having 𝑓 ⊙ : R → R allows us to define scalarmultiplication: 𝑒 × 𝑒 : = 𝑓 ⊙ ( ( 𝑒 ) 𝑇 · 𝑒 · ( 𝑒 ) 𝑇 , 𝑒 ) . Conversely, 𝑓 𝑘 ⊙ can be expressed using scalar multiplication, as can be seen from our simulation of general function applications by pointwisefunction application on scalars. Finally, a notational simplification is that when using scalars 𝑎 ∈ R in our expressions, we write sometimes 𝑎 instead of [ 𝑎 ] . For example, ( − 𝑒 ( 𝑣 ) 𝑇 · 𝑣 ) stands for ([ ] − 𝑒 ( 𝑣 ) 𝑇 · 𝑣 ) . xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA B PROOFS OF SECTION 3B.1 Order predicates
We detail how order information on canonical vectors can be obtained in for - MATLANG . We provide explicit expressions for the operatorsmentioned in Section 3 and furthermore, we also define expressions for operators that will be used in our proofs.To begin with, we can easily obtain the last canonical vector using the expression 𝑒 max : = for 𝑣, 𝑋 . 𝑣. In other words, we simply overwrite 𝑋 with the current canonical vector in each iteration. Hence, at the end, 𝑋 is assigned to the lastcanonical vector.As already mentioned in the main body of the paper, to define an order relation for canonical vectors, we notice that the following matrix: 𝑆 ≤ = · · · . . . . . . .... . . . . . . . . . . . . · · · · · · . has the property that for two canonical vectors 𝑏 𝑖 and 𝑏 𝑗 of the same dimension, 𝑏 𝑇𝑖 · 𝑆 ≤ · 𝑏 𝑗 = (cid:40) if 𝑖 ≤ 𝑗 otherwise . We observe that 𝑆 ≤ can be expressed in for - MATLANG as follows: 𝑆 ≤ : = for 𝑣, 𝑋 . 𝑋 + (cid:0) ( 𝑋 · 𝑒 max ) + 𝑣 (cid:1) · 𝑣 𝑇 + 𝑣 · 𝑒 𝑇 max , where 𝑒 max is as defined above. The intuition behind this expression is that by using the last canonical vector 𝑏 𝑛 , as returned by 𝑒 max , wehave access to the last column of 𝑋 (via the product 𝑋 · 𝑒 max ). We use this column such that after the 𝑖 -th iteration, this column contains the 𝑖 -th column of 𝑆 ≤ . This is done by incrementing 𝑋 with 𝑣 · 𝑒 𝑇 max . To construct 𝑆 ≤ , in the 𝑖 -th iteration we further increment 𝑋 with (i) thecurrent last column in 𝑋 (via 𝑋 · 𝑒 max · 𝑣 𝑇 ) which holds the ( 𝑖 − ) -th column of 𝑆 ≤ ; and (ii) the current canonical vector (via 𝑣 · 𝑣 𝑇 ). Hence,after iteration 𝑖 , 𝑋 contains the first 𝑖 columns of 𝑆 ≤ and holds the 𝑖 th column of 𝑆 ≤ in its last column. It is now readily verified that 𝑋 = 𝑆 ≤ after the 𝑛 th iteration.By defining succ ( 𝑢, 𝑣 ) : = 𝑢 𝑇 · 𝑆 ≤ · 𝑣, we obtain an order relation that allows us to discern whether one canonical vector comes before the other in the order given by 𝑆 ≤ . If wewant a strict order, we can just use the matrix 𝑆 < : = 𝑆 ≤ − 𝑒 Id , where 𝑒 Id is an expression in for - MATLANG which returns the identity matrix(of appropriate dimension). Given this, we define succ + ( 𝑢, 𝑣 ) : = 𝑢 𝑇 · 𝑆 < · 𝑣. from which we can also derive max ( 𝑢 ) : = 𝑢 𝑇 · 𝑒 max . which is an expression that returns the last canonical vector.Interestingly, we can also define the previous relation between canonical vectors. For this, we require the following matrix: Prev = · · · . . . . . . .... . . . . . . . . . . . . · · · · · · , Using this matrix, we have that for a canonical vector 𝑏 𝑖 : Prev · 𝑏 𝑖 = (cid:40) 𝑏 𝑖 − , if 𝑖 > . , if 𝑖 = . where is a vector of zeros of the same type as 𝑏 𝑖 . Notice also that ( 𝑢 ) 𝑇 · Prev · 𝑢 is equal to zero, for a canonical vector 𝑢 , if and only if 𝑢 = 𝑏 is the first canonical vector, and zero otherwise. Therefore the expression min ( 𝑢 ) is defined as min ( 𝑢 ) : = − ( 𝑢 ) 𝑇 · Prev · 𝑢, and, when evaluated over canonical vectors, will result in if and only if 𝑢 = 𝑏 is the first canonical vector. To define the first canonicalvector in the order given by for , we can then write: 𝑒 min : = for 𝑣, 𝑋 . 𝑋 + min ( 𝑣 ) × 𝑣, onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč Finally, we show that
Prev can be defined using the following for - MATLANG expression: 𝑒 Prev : = for 𝑣, 𝑋 . 𝑋 + (cid:0) ( − max ( 𝑣 )) × 𝑣 · 𝑒 𝑇 max − ( 𝑋 · 𝑒 max ) · 𝑒 𝑇 max + ( 𝑋 · 𝑒 max ) · 𝑣 𝑇 (cid:1) . Here, 𝑋 is initialized as and thus in the first iteration we put 𝑏 in the last column of 𝑋 (note that 𝑋 · 𝑒 max is also zero in the first iteration).Next, in iteration two, we add a matrix that has the stored vector 𝑋 · 𝑒 max (the previous canonical vector) in the column indicated by 𝑣 (thecurrent canonical vector) and 𝑣 − 𝑋 · 𝑒 max in the last column, to replace the vector stored. As a consequence, 𝑏 is now stored in the lastcolumn. In the last iteration, we have 𝑏 𝑛 − already in the last column, so no further update of 𝑋 is required.To get the next relation we simply do 𝑒 Next = 𝑒 𝑇 Prev . We have that for a canonical vector 𝑏 𝑖 : Next · 𝑏 𝑖 = (cid:40) 𝑏 𝑖 + , if 𝑖 < 𝑛. , if 𝑖 = 𝑛. In this way, we also can obtain the following operators for a canonical vector 𝑣 : prev ( 𝑣 ) : = 𝑒 Prev · 𝑣. next ( 𝑣 ) : = 𝑒 Next · 𝑣. More generally, we define 𝑒 getPrevMatrix ( 𝑣 ) : = Π 𝑤 . succ ( 𝑤, 𝑣 ) × 𝑒 Prev + ( − succ ( 𝑤, 𝑣 )) × 𝑒 Id 𝑒 getNextMatrix ( 𝑣 ) : = Π 𝑤 . succ ( 𝑤, 𝑣 ) × 𝑒 Next + ( − succ ( 𝑤, 𝑣 )) × 𝑒 Id expressions that, when 𝑣 is interpreted as canonical vector 𝑏 𝑖 , output Prev 𝑖 and Next 𝑖 respectively. Note that Prev 𝑗 · 𝑏 𝑖 = (cid:40) 𝑏 𝑖 − 𝑗 , if 𝑖 > 𝑗. , if 𝑖 ≤ 𝑗. and Next 𝑗 · 𝑏 𝑖 = (cid:40) 𝑏 𝑖 + 𝑗 , if 𝑖 + 𝑗 ≤ 𝑛. , if 𝑖 + 𝑗 > 𝑛. Finally, define 𝑒 min + 𝑖 : = 𝑒 getNextMatrix ( . . . 𝑒 getNextMatrix (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) 𝑖 times ( 𝑒 min )) and 𝑒 max − 𝑖 : = 𝑒 getPrevMatrix ( . . . 𝑒 getPrevMatrix (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) 𝑖 times ( 𝑒 max )) We note that some these expressions were already used in Section A.1.
C PROOFS OF SECTION 4
We next provide more details about how to perform LU-decomposition (without and with pivoting) and to compute the determinant andinverse of a matrix.
C.1 LU-decomposition
We start with LU-decomposition without pivoting. We recall proposition 4.1:Proposition 4.1.
There exists for - MATLANG [ 𝑓 / ] expressions 𝑒 𝐿 ( 𝑉 ) and 𝑒 𝑈 ( 𝑉 ) such that ⟦ 𝑒 𝐿 ⟧(I) = 𝐿 and ⟦ 𝑒 𝑈 ⟧(I) = 𝑈 form an LU-decomposition of 𝐴 , where mat ( 𝑉 ) = 𝐴 and 𝐴 is LU-factorizable. Proof. Let 𝐴 be an LU-factorizable matrix. We already explained how the expression 𝑒 𝑈 ( 𝑉 ) is obtained in the main body of the paper, i.e., 𝑒 𝑈 ( 𝑉 ) : = ( for 𝑦, 𝑋 = 𝑒 Id . reduce( 𝑋 · 𝑉 , 𝑦 ) · 𝑋 ) · 𝑉 .
We recall that 𝑒 𝑈 ( 𝐴 ) = 𝑇 𝑛 · · · · · 𝑇 · 𝐴 with 𝐿 − = 𝑇 𝑛 · · · · · 𝑇 . Let 𝑒 𝐿 − ( 𝑉 ) : = for 𝑦, 𝑋 = 𝑒 Id . reduce( 𝑋 · 𝑉 , 𝑦 ) · 𝑋 . such that 𝑒 U ( 𝑉 ) : = 𝑒 𝐿 − ( 𝑉 ) · 𝑉 .
It now suffices to observe that, since 𝑇 𝑛 = 𝐼 , 𝐿 − = ( 𝐼 − 𝑐 · 𝑏 𝑇 ) · · · ( 𝐼 − 𝑐 𝑛 − · 𝑏 𝑇𝑛 − ) = 𝐼 − 𝑐 · 𝑏 𝑇 − · · · − 𝑐 𝑛 − · 𝑏 𝑇𝑛 − xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA and hence, 𝐿 = ( 𝐼 + 𝑐 · 𝑏 𝑇 ) · · · ( 𝐼 + 𝑐 𝑛 − · 𝑏 𝑇𝑛 − ) = 𝐼 + 𝑐 · 𝑏 𝑇 + · · · + 𝑐 𝑛 − · 𝑏 𝑇𝑛 − . As a consequence, to obtain 𝐿 from 𝐿 − we just need to multiply every entry below the diagonal by − . Since both 𝐿 and 𝐿 − are lowertriangular, this can done by computing 𝐿 = − × 𝐿 − + × 𝐼 . Translated into for - MATLANG , this means that we can define 𝑒 𝐿 ( 𝑉 ) : = − × 𝑒 𝐿 − ( 𝑉 ) + × 𝑒 Id , which concludes the proof of the proposition. □ C.2 LU-decomposition with pivoting
We next consider LU-decomposition with pivoting. We recall proposition 4.2:Proposition 4.2.
There exist expressions 𝑒 𝐿 − 𝑃 ( 𝑉 ) and 𝑒 𝑈 ( 𝑉 ) in for - MATLANG [ 𝑓 / , 𝑓 > ] such that 𝐿 − · 𝑃 = ⟦ 𝑒 𝐿 − 𝑃 ⟧(I) and 𝑈 = ⟦ 𝑒 𝑈 ⟧(I) ,satisfy 𝐿 − · 𝑃 · 𝐴 = 𝑈 , where I is an instance such that mat ( 𝑉 ) = 𝐴 . Proof. We assume that 𝑓 / and 𝑓 > are in F . Let 𝐴 be an arbitrary matrix. By contrast to when 𝐴 is LU-factorizable, during the LU-decomposition process we may need row interchange (pivoting) in each step of the iteration. Let us assume that row interchange is neededimmediately before step 𝑘 , ≤ 𝑘 ≤ 𝑛 . In other words, we now aim to reduce the 𝑘 -th column of 𝐴 𝑘 = 𝑇 𝑘 − · · · 𝑇 · 𝐴 , or 𝐴 𝑘 = 𝐴 if 𝑘 = , butnow 𝐴 𝑘 has a zero pivot, i.e., ( 𝐴 𝑘 ) 𝑘𝑘 = . Let 𝑃 be the matrix that denotes the necessary row interchange. If we know 𝑃 , then to compute 𝑇 𝑘 we need to perform reduce( 𝑃 · 𝑋 · 𝐴, 𝑣 ) in this iteration, where reduce( · , ) is the expression in for - MATLANG reducing a column, asexplained in the main body of the paper. Furthermore, we need to apply the permutation 𝑃 to the current result, resulting in the expression for 𝑣, 𝑋 = 𝐼 . reduce( 𝑃 · 𝑋 · 𝐴, 𝑣 ) · 𝑃 · 𝑋 . We now remark that 𝑃 is a permutation matrix of the form 𝑃 = 𝐼 − 𝑢 · 𝑢 𝑇 and it denotes an interchange(if multiplied by left) of rows 𝑖 and 𝑗 if 𝑢 = ( 𝑏 𝑖 − 𝑏 𝑗 ) . Note that we are performing a row interchange for column 𝑘 and thus 𝑖 = 𝑘 and 𝑗 > 𝑘 − . If no interchange is needed, 𝑖 = 𝑗 = 𝑘 and 𝑃 = 𝐼 . Also note that when 𝑘 = 𝑛 no interchange takes place. Furthermore, if no suitable 𝑏 𝑗 can be found, this implies that no interchange is required as well and we can move on to next column.To find the vector 𝑢 in 𝑃 , we can, for example, find the first entry 𝑗 ≥ 𝑘 in column 𝑘 of 𝐴 𝑘 that holds a non-zero value. More generally, wecan find the first entry in a vector 𝑎 that holds a non-zero value by using the function 𝑓 > . Indeed, consider the following expression: neq( 𝑎, 𝑢 ) : = for 𝑣, 𝑋 . (cid:16) − 𝑒 ( 𝑣 ) 𝑇 · 𝑋 (cid:17) × 𝑓 > (cid:16) ( 𝑣 𝑇 · 𝑎 ) (cid:17) × 𝑣 + max ( 𝑣 ) × (cid:16) − 𝑒 ( 𝑣 ) 𝑇 · 𝑋 (cid:17) × (cid:16) − 𝑓 > (cid:16) ( 𝑣 𝑇 · 𝑎 ) (cid:17)(cid:17) × 𝑢 Here, neq( 𝑎, 𝑢 ) receives two 𝑛 dimensional vectors 𝑎 and 𝑢 and outputs a canonical vector 𝑏 𝑗 such that 𝑎 𝑗 is the first non-zero entry of 𝑎 , or 𝑢 if such non-zero value does not exist in 𝑎 . We check for 𝑓 > ((·) ) in case a negative number is tested. The above expression simply checksin each iteration whether 𝑋 already holds a canonical vector. If so, then 𝑋 is not updated. Otherwise, 𝑋 is replaced by the current canonicalvector 𝑏 𝑗 if and only if 𝑏 𝑇𝑗 · 𝑎 is non-zero. Furthermore, when the final canonical vector is considered and 𝑋 does not hold a canonical vectoryet and 𝑏 𝑇𝑛 · 𝑎 is zero, the vector 𝑢 is returned.We use neq( 𝑎, 𝑢 ) to find a pivot for a specific column. Let us assume again that we want to find a pivot in column 𝑘 of 𝐴 𝑘 . We canthen first make all entries in that column, with indexes smaller or equal to 𝑘 , zero, just as we did by means of col( · , · ) in the definition of reduce( · , ) . Except, now we also need to make the 𝑘 the entry zero as well. Let us denote by col eq ( · , · ) the operation col( · , · ) , as defined inthe main body of the paper, but using succ instead of succ + (to include the 𝑘 entry). Given this, we can construct 𝑃 = 𝐼 − 𝑢 · 𝑢 𝑇 as follows: 𝑒 𝑃 𝑢 ( 𝐴, 𝑢 ) : = 𝑒 Id − (cid:2) 𝑢 − neq(col eq ( 𝐴, 𝑢 ) , 𝑢 ) (cid:3) · (cid:2) 𝑢 − neq(col eq ( 𝐴, 𝑢 ) , 𝑢 ) (cid:3) 𝑇 . From the explanations given above, it should be clear that 𝑒 𝑃 𝑢 ( 𝐴, 𝑢 ) computes the necessary permutation matrix of 𝐴 𝑘 for the columnindicated by 𝑢 , or 𝐼 if no permutation is needed, or if such permutation does not exist (so we skip the current column). Also, we have tomodify the reduce( 𝑉 , 𝑦 ) operators, as follows: reduce( 𝑉 , 𝑦 ) : = 𝑒 Id + 𝑓 > (cid:16) ( 𝑦 𝑇 · 𝑉 · 𝑦 ) (cid:17) × 𝑓 / ( col( 𝑉 , 𝑦 ) , (cid:104) −( 𝑦 𝑇 · 𝑉 · 𝑦 ) × 𝑒 ( 𝑦 ) + (cid:16) − 𝑓 > (cid:16) ( 𝑦 𝑇 · 𝑉 · 𝑦 ) (cid:17)(cid:17) × 𝑒 ( 𝑦 ) (cid:105) ) · 𝑦 𝑇 , so that when 𝑉 is interpreted by a matrix 𝐵 and 𝑦 = 𝑏 𝑖 , it returns 𝐼 + 𝑐 𝑖 𝑏 𝑇𝑖 if 𝐵 𝑖𝑖 is not zero. If 𝐵 𝑖𝑖 = then we divide col( 𝐵, 𝑏 𝑖 ) by 𝑒 ( 𝑏 𝑖 ) (sowe don’t get undefined ), but we don’t add 𝑐 𝑖 𝑏 𝑇𝑖 precisely because 𝐵 𝑖𝑖 = , and return the identity so nothing happens. We check for 𝑓 > ((·) ) in case a negative number is tested.Finally, we define 𝑒 𝐿 − 𝑃 ( 𝑉 ) : = for 𝑣, 𝑋 = 𝑒 Id . reduce( 𝑒 𝑃 𝑣 ( 𝑋 · 𝑉 , 𝑣 ) · 𝑋 · 𝑉 , 𝑣 ) · 𝑒 𝑃 𝑣 ( 𝑋 · 𝑉 , 𝑣 ) · 𝑋 and 𝑒 U ( 𝑉 ) : = 𝑒 𝐿 − 𝑃 ( 𝑉 ) · 𝑉 as the desired expressions.As a final observation, in the definition of 𝑒 𝐿 − 𝑃 ( 𝑉 ) we interlaced permutation matrices with the 𝑇 𝑖 ’s. More specifically, 𝐴 𝑘 = 𝑇 𝑘 · 𝑃 · 𝑇 𝑘 − · · · 𝑇 · 𝐴 . We observe, however, that for ℓ ≤ 𝑘 − and 𝑇 ℓ = 𝐼 − 𝑐 ℓ · 𝑏 𝑇ℓ , we have that 𝑏 𝑇ℓ · 𝑃 = 𝑏 ℓ because 𝑏 ℓ has zeroes in positions in therows involved in the row exchange 𝑃 . Also, note that 𝑃 = 𝐼 and thus 𝑃 · 𝑇 ℓ · 𝑃 = 𝑃 − 𝑃 · 𝑐 ℓ · 𝑏 𝑇ℓ · 𝑃 = 𝐼 − (cid:98) 𝑐 ℓ · 𝑏 𝑇𝑙 = (cid:98) 𝑇 ℓ . onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč As a consequence, 𝑇 𝑘 · 𝑃 · 𝑇 𝑘 − · · · 𝑇 = 𝑇 𝑘 · 𝑃 · 𝑇 𝑘 − · 𝑃 · 𝑇 𝑘 − · 𝑃 · · · 𝑃 · 𝑇 · 𝑃 = 𝑇 𝑘 · ( 𝑃 · 𝑇 𝑘 − · 𝑃 ) · · · ( 𝑃 · 𝑇 · 𝑃 ) · 𝑃 = (cid:98) 𝑇 𝑘 − · · · (cid:98) 𝑇 · 𝑃, and thus we may assume that 𝑃 occurs at the end. Hence, we obtain 𝐿 − · 𝑃 · 𝐴 = 𝑈 . □ C.3 Determinant and inverse
We next turn our attention to computing the inverse and determinant of a matrix. To show Proposition 4.3 we first show that it holds whenconsidering non-singular lower or upper triangular matrices.Lemma C.1.
There are for - MATLANG [ 𝑓 / ] expressions 𝑒 upperDiagInv ( 𝑉 ) and 𝑒 lowerDiagInv ( 𝑉 ) such that ⟦ 𝑒 upperDiagInv ⟧(I) = 𝐴 − when I assigns 𝑉 to an invertible upper triangular matrix 𝐴 and ⟦ 𝑒 lowerDiagInv ⟧(I) = 𝐴 − when I assigns 𝑉 to an invertible lower triangular matrix 𝐴 . Proof. We start by considering the expression: 𝑒 ps ( 𝑉 ) : = 𝑒 Id + Σ 𝑣. Π 𝑤 . [ succ ( 𝑤, 𝑣 ) × 𝑉 + ( − succ ( 𝑤, 𝑣 )) × 𝑒 Id ] . Here, 𝑒 ps ( 𝐴 ) results in 𝐼 + 𝐴 + 𝐴 + · · · + 𝐴 𝑛 for any matrix 𝐴 . In the expression, the outer loop defines which power we compute. That is,when 𝑣 is the 𝑖 th canonical vector, we compute 𝐴 𝑖 . Computing 𝐴 𝑖 is achieved via the inner product loop, which uses succ ( 𝑤, 𝑣 ) to determinewhether 𝑤 comes before or is 𝑣 in the ordering of canonical vectors. When this is the case, we multiply the current result by 𝐴 , and when 𝑤 is greater than 𝑣 , we use the identity as not to affect the already computed result. We add the identity at the end.Now, let 𝐴 be an 𝑛 × 𝑛 matrix that is upper triangular and let 𝐷 𝐴 be the matrix consisting of the diagonal elements of 𝐴 , i.e., 𝐷 𝐴 = 𝑎 · · · · · · 𝑎 · · · . . . ... ...... · · · · · · 𝑎 𝑛𝑛 . We can compute 𝐷 𝐴 by the expression: 𝑒 getDiag ( 𝑉 ) : = Σ 𝑣. ( 𝑣 𝑇 · 𝑉 · 𝑣 ) × 𝑣 · 𝑣 𝑇 . Let 𝑇 = 𝐴 − 𝐷 𝐴 , then 𝐴 − = [ 𝐷 𝐴 + 𝑇 ] − = (cid:104) 𝐷 𝐴 (cid:16) 𝐼 + 𝐷 − 𝐴 𝑇 (cid:17)(cid:105) − = (cid:16) 𝐼 + 𝐷 − 𝐴 𝑇 (cid:17) − 𝐷 − 𝐴 . We now observe that 𝐷 − 𝐴 simply consists of the inverses of the elements on the diagonal. This can be expressed, as follows: 𝑒 diagInverse ( 𝑉 ) : = Σ 𝑣.𝑓 / ( , 𝑣 𝑇 · 𝑉 · 𝑣 ) × 𝑣 · 𝑣 𝑇 = Σ 𝑣.𝑓 / ( , 𝑣 𝑇 · 𝑉 · 𝑣 ) × 𝑣 · 𝑣 𝑇 , Where 𝑓 / is the division function. In the last equality we take advantage of the fact that the diagonals of 𝐴 and 𝐷 𝐴 are the same.We now focus on the computation of (cid:16) 𝐼 + 𝐷 − 𝐴 · 𝑇 (cid:17) − . First, by construction, 𝐷 − 𝐴 · 𝑇 is strictly upper triangular and thus nilpotent, suchthat (cid:16) 𝐷 − 𝐴 · 𝑇 (cid:17) 𝑛 = , where 𝑛 is the dimension of 𝐴 . Recall the following algebraic identity ( + 𝑥 ) (cid:32) 𝑚 ∑︁ 𝑖 = (− 𝑥 ) 𝑖 (cid:33) = − (− 𝑥 ) 𝑚 + . By choosing 𝑚 = 𝑛 − and applying it to 𝑥 = 𝐷 − 𝐴 · 𝑇 , we have (cid:16) 𝐼 + 𝐷 − 𝐴 · 𝑇 (cid:17) (cid:32) 𝑛 − ∑︁ 𝑖 = (− 𝐷 − 𝐴 · 𝑇 ) 𝑖 (cid:33) = 𝐼 − (cid:16) − 𝐷 − 𝐴 · 𝑇 (cid:17) 𝑛 = 𝐼 .
Hence, (cid:16) 𝐼 + 𝐷 − 𝐴 · 𝑇 (cid:17) − = 𝑛 − ∑︁ 𝑖 = (− 𝐷 − 𝐴 · 𝑇 ) 𝑖 = 𝑛 ∑︁ 𝑖 = (− 𝐷 − 𝐴 · 𝑇 ) 𝑖 . We now observe that 𝑒 ps (− × 𝐷 − 𝐴 · 𝑇 ) = 𝑛 ∑︁ 𝑖 = (− 𝐷 − 𝐴 · 𝑇 ) 𝑖 = (cid:16) 𝐼 + 𝐷 − 𝐴 · 𝑇 (cid:17) − , and thus 𝐴 − = 𝑒 ps (cid:0) − × (cid:2) 𝑒 diagInverse ( 𝐴 )( 𝐴 − 𝑒 getDiag ( 𝐴 )) (cid:3)(cid:1) 𝑒 diagInverse ( 𝐴 ) . Seeing this as an expression: 𝑒 upperDiagInv ( 𝑉 ) : = 𝑒 ps (cid:0) − × (cid:2) 𝑒 diagInverse ( 𝑉 )( 𝑉 − 𝑒 getDiag ( 𝑉 )) (cid:3)(cid:1) 𝑒 diagInverse ( 𝑉 ) , xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA we see that when interpreting 𝑉 as an upper triangular invertible matrix, 𝑒 upperDiagInv ( 𝐴 ) evaluates to 𝐴 − .To deal with invertible lower triangular matrices 𝐴 , we observe that (cid:0) 𝐴 − (cid:1) 𝑇 = (cid:16) 𝐴 𝑇 (cid:17) − and 𝐴 𝑇 is upper triangular. Hence, it suffices todefine 𝑒 lowerDiagInv ( 𝑉 ) : = 𝑒 upperDiagInv ( 𝑉 𝑇 ) 𝑇 . This concludes the proof of the lemma. □ We are now ready to prove proposition 4.3. We recall:Proposition 4.3.
There are for - MATLANG [ 𝑓 / ] expressions 𝑒 det ( 𝑉 ) and 𝑒 inv ( 𝑉 ) such that ⟦ 𝑒 det ⟧(I) = det ( 𝐴 ) , and ⟦ 𝑒 inv ⟧(I) = 𝐴 − when I assigns 𝑉 to 𝐴 and 𝐴 is invertible. Proof. Let 𝐴 be an 𝑛 × 𝑛 matrix. As mentioned in the main body of the paper, we will implement Csanky’s algorithm. Let 𝑝 𝐴 ( 𝑥 ) : = det ( 𝑥𝐼 − 𝐴 ) denote characteristic polynomial of 𝐴 . We write 𝑝 𝐴 ( 𝑥 ) = + (cid:205) 𝑛𝑖 = 𝑐 𝑖 𝑥 𝑖 and let 𝑆 𝑖 : = 𝑖 + tr ( 𝐴 𝑖 ) with tr (·) the trace operatorwhich sums up the diagonal elements of a matrix. Then, the coefficients 𝑐 , . . . , 𝑐 𝑛 are known to satisfy (cid:169)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:171) · · · 𝑆 · · · 𝑆 𝑆 · · · ... ... ... ... ... 𝑆 𝑛 − 𝑆 𝑛 − 𝑆 𝑛 − · · · 𝑆 (cid:170)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:172)(cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) 𝑆 · (cid:169)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:171) 𝑐 𝑐 𝑐 ...𝑐 𝑛 (cid:170)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:172)(cid:124)(cid:123)(cid:122)(cid:125) ¯ 𝑐 = (cid:169)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:171) 𝑆 𝑆 𝑆 ...𝑆 𝑛 (cid:170)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:172)(cid:124)(cid:123)(cid:122)(cid:125) ¯ 𝑏 and furthermore, 𝑐 𝑛 = (− ) 𝑛 det ( 𝐴 ) and if 𝑐 𝑛 ≠ , then 𝐴 − = 𝑐 𝑛 𝑛 − ∑︁ 𝑖 = 𝑐 𝑖 𝐴 𝑛 − − 𝑖 , with 𝑐 = . It is now easy to see that we can compute the 𝑆 ′ 𝑖 𝑠 in for - MATLANG . Indeed, for 𝑖 = , . . . , 𝑛 we can consider 𝑒 powTr ( 𝑉 , 𝑣 ) : = Σ 𝑤 .𝑤 𝑇 · (cid:0) 𝑒 pow ( 𝑉 , 𝑣 ) · 𝑉 (cid:1) · 𝑤 with 𝑒 pow ( 𝑉 , 𝑣 ) : = Π 𝑤 . ( succ ( 𝑤, 𝑣 ) × 𝑉 + ( − succ ( 𝑤, 𝑣 )) × 𝑒 Id ) . We have that 𝑒 pow ( 𝐴, 𝑏 𝑗 ) = 𝐴 𝑗 and thus 𝑒 powTr ( 𝐴, 𝑏 𝑗 ) = tr ( 𝐴 𝑗 ) . Define: 𝑒 𝑆 ( 𝑉 , 𝑣 ) : = 𝑓 / ( , + Σ 𝑤 . succ ( 𝑤, 𝑣 )) × 𝑒 powTr ( 𝑉 , 𝑣 ) . Here 𝑒 𝑆 ( 𝐴, 𝑏 𝑖 ) = 𝑆 𝑖 . Note that 𝑖 + is computed summing up to the dimension indicated by 𝑣 , and adding 1. We can now easily construct thevector ¯ 𝑏 used in the system of equations by means of the expression: 𝑒 ¯ 𝑏 ( 𝑉 ) : = Σ 𝑤 .𝑒 𝑆 ( 𝑉 , 𝑤 ) × 𝑤 .
We next construct the matrix 𝑆 . We need to be able to shift a vector 𝑎 in 𝑘 positions, i.e., such that ( 𝑎 , . . . , 𝑎 𝑛 ) ↦→ ( , . . . , 𝑎 , . . . , 𝑎 𝑛 − 𝑘 ) . Weuse 𝑒 getNextMatrix defined in section B.1, i.e., we define: 𝑒 shift ( 𝑎, 𝑣 ) : = Σ 𝑤 . ( 𝑤 𝑇 · 𝑎 ) × ( 𝑒 getNextMatrix ( 𝑣 ) · 𝑤 ) performs the desired shift when 𝑢 is assigned a vector 𝑎 and 𝑣 is 𝑏 𝑘 . The matrix 𝑆 is now obtained as follows: 𝑆 ( 𝑉 ) : = 𝑒 Id + Σ 𝑣.𝑒 shift ( 𝑒 ¯ 𝑏 ( 𝑉 ) , 𝑣 ) · 𝑣 𝑇 We now observe that 𝑆 is lower triangular with nonzero diagonal entries. So, Lemma C.1 tells us that we can invert it, i.e., 𝑒 lowerDiagInv ( 𝑆 ) = 𝑆 − .As a consequence, 𝑒 ¯ 𝑐 ( 𝑉 ) : = 𝑒 lowerDiagInv ( 𝑆 ( 𝑉 )) · 𝑒 ¯ 𝑏 ( 𝑉 ) . outputs ¯ 𝑐 when 𝑉 is interpreted as matrix 𝐴 . Observe that we only use the division operator. We now have all coefficients of the characteristicpolynomial of 𝐴 .We can now define 𝑒 det ( 𝑉 ) : = (cid:16)(cid:16) ( Π 𝑤 . (− ) × 𝑒 ( 𝑉 )) 𝑇 · 𝑒 max (cid:17) × 𝑒 ¯ 𝑐 ( 𝑉 ) (cid:17) 𝑇 · 𝑒 max , an expression that, when 𝑉 is interpreted as any matrix 𝐴 , outputs det ( 𝐴 ) . Here, ( Π 𝑤 . (− ) × 𝑒 ( 𝑉 )) is the 𝑛 dimensional vector with (− ) 𝑛 in all of its entries. Since 𝑐 𝑛 = (− ) 𝑛 det ( 𝐴 ) , we extract (− ) 𝑛 (− ) 𝑛 det ( 𝐴 ) = det ( 𝐴 ) with 𝑒 max . We use a slightly different, but equivalent, system of equations than the one mentioned in the paper. onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč
For the inverse, we have that 𝐴 − = 𝑐 𝑛 𝑛 − ∑︁ 𝑖 = 𝑐 𝑖 𝐴 𝑛 − − 𝑖 = 𝑐 𝑛 𝐴 𝑛 − + 𝑛 − ∑︁ 𝑖 = 𝑐 𝑖 𝑐 𝑛 𝐴 𝑛 − − 𝑖 . We compute 𝑐 𝑛 𝐴 𝑛 − as 𝑓 / ( , 𝑒 ¯ 𝑐 ( 𝐴 ) 𝑇 · 𝑒 max ) × 𝑒 pow ( 𝐴, 𝑒 max ) and (cid:205) 𝑛 − 𝑖 = 𝑐 𝑖 𝑐 𝑛 𝐴 𝑛 − − 𝑖 as Σ 𝑣.𝑓 / (cid:16) 𝑒 ¯ 𝑐 ( 𝐴 ) 𝑇 · 𝑣, 𝑒 ¯ 𝑐 ( 𝐴 ) 𝑇 · 𝑒 max (cid:17) × 𝑒 invPow ( 𝐴, 𝑣 ) , where 𝑒 invPow ( 𝑉 , 𝑣 ) : = Π 𝑤 . ( − max ( 𝑤 )) × [( − succ ( 𝑤, 𝑣 )) × 𝑉 + succ ( 𝑤, 𝑣 ) × 𝑒 Id ] + max ( 𝑤 ) × 𝑒 Id . Here, 𝑒 invPow ( 𝐴, 𝑏 𝑖 ) = 𝐴 𝑛 − − 𝑖 and 𝑒 invPow ( 𝐴, 𝑏 𝑛 ) = 𝐼 . Note that we always multiply by 𝑒 Id in the last step. To conclude, we define: 𝑒 inv ( 𝑉 ) : = 𝑓 / ( , 𝑒 ¯ 𝑐 ( 𝑉 ) 𝑇 · 𝑒 max ) × 𝑒 pow ( 𝑉 , 𝑒 max ) + (cid:104) Σ 𝑣.𝑓 / (cid:16) 𝑒 ¯ 𝑐 ( 𝑉 ) 𝑇 · 𝑣, 𝑒 ¯ 𝑐 ( 𝑉 ) 𝑇 · 𝑒 max (cid:17) × 𝑒 invPow ( 𝑉 , 𝑣 ) (cid:105) , an expression that, when 𝑉 is interpreted as any invertible matrix 𝐴 , computes 𝐴 − . □ As an observation, here we only use operators Σ and Π defined in section 6. We also assume access to order. D PROOFS OF SECTION 5D.1 Linear space functions
We start by showing a crucial ingredient for making the correspondence between for - MATLANG and arithmetic circuits. More specifically,we show that any polynomial time Turing machine, working within linear space and producing linear space output, can be simulated in for - MATLANG . For this proof and section only, we will denote the canonical vectors as e , . . . , e 𝑛 , since 𝑏 will be used to represent a value ona position of a tape.We consider deterministic Turing Machines (TM) 𝑇 consisting of ℓ read-only input tapes, denoted by 𝑅 , . . . , 𝑅 ℓ , a work tape, denotedby 𝑊 , and a write-only output tape, denoted by 𝑂 . The TM 𝑇 has a set 𝑄 of 𝑚 states, denoted by 𝑞 , . . . , 𝑞 𝑚 . We assume that 𝑞 is theinitial state and 𝑞 𝑚 is the accepting state. The input and tape alphabet are Σ = { , } and Γ = Σ ∪ { ▷ , ◁ } , respectively. The special symbol ▷ denotes the beginning of each of the tapes, the symbol ◁ denotes the end of the ℓ input tapes. The transition function Δ is defined as usual,i.e., Δ : 𝑄 × Γ ℓ + → 𝑄 × Γ × {← , ⊔ , →} ℓ + such that Δ ( 𝑞, ( 𝑎 , . . . , 𝑎 ℓ , 𝑏, 𝑐 )) = (cid:0) 𝑞 ′ , ( 𝑏 ′ , 𝑐 ′ ) , ( d , . . . , d ℓ + ) (cid:1) with d 𝑖 ∈ {← , ⊔ , →} , means thatwhen 𝑇 is in state 𝑞 and the ℓ + heads on the tapes read symbols 𝑎 , . . . , 𝑎 ℓ , 𝑏, 𝑐 , respectively, then 𝑇 transitions to state 𝑞 ′ , writes 𝑏 ′ , 𝑐 ′ onthe work and output tapes, respectively, at the position to which the work and output tapes’ heads points at, and finally moves the heads onthe tapes according d , . . . , d ℓ + . More specifically, ← indicates a move to the left, → a move to the right, and finally, ⊔ indicates that thehead does not move.We assume that Δ is defined such that it ensures that on none of the tapes, heads can move beyond the leftmost marker ▷ . Furthermore,the tapes 𝑅 , . . . , 𝑅 ℓ are treated as read-only and the heads on these tapes cannot move beyond the end markers ◁ . Similarly, Δ ensures thatthe output tape 𝑂 is write only, i.e., its head cannot move to the left. We also assume that Δ does not change the occurrences of ▷ or writes ◁ on the work and output tape.A configuration of 𝑇 is defined in the usual way. That is, a configuration of the input tapes is of the form ▷ 𝑤 𝑞𝑤 ◁ with 𝑤 , 𝑤 ∈ Σ ∗ and represents that the current tape content is ▷ 𝑤 𝑤 ◁ , 𝑇 is in state 𝑞 and the head is positioned on the first symbol of 𝑤 . Similarly,configurations of the work and output tape are represented by ▷ 𝑤 𝑞𝑤 . A configuration of 𝑇 is consists of configurations for all tapes. Giventwo configurations 𝑐 and 𝑐 , we say that 𝑐 yields 𝑐 if 𝑐 is the result of applying the transition function Δ of 𝑇 based on the informationin 𝑐 . As usual, we close this “yields” relation transitively.Given ℓ input words 𝑤 , . . . , 𝑤 ℓ ∈ Σ ∗ , we assume that the initial configuration of 𝑇 is given by (cid:0) 𝑞 ▷ 𝑤 ◁ , 𝑞 ▷ 𝑤 ◁ , . . . , 𝑞 ▷ 𝑤 ℓ ◁ , 𝑞 ▷ , 𝑞 ▷ (cid:1) and an accepting configuration is assumed to be of the form (cid:0) ▷ 𝑞 𝑚 𝑤 ◁ , ▷ 𝑞 𝑚 𝑤 ◁ , . . . , ▷ 𝑞 𝑚 𝑤 ℓ ◁ , ▷ 𝑞 𝑚 , ▷ 𝑞 𝑚 𝑤 (cid:1) for some 𝑤 ∈ Σ ∗ . We say that 𝑇 computes the function 𝑓 : ( Σ ∗ ) ℓ → Σ ∗ if for every 𝑤 , . . . , 𝑤 ℓ ∈ Σ ∗ , the initial configuration yields (transitively) an accepting configurationsuch that the configuration on the output tape is given by ▷ 𝑞 𝑚 𝑓 ( 𝑤 , . . . , 𝑤 ℓ ) .We assume that once 𝑇 reaches an accepting configuration it stays indefinitely in that configuration (i.e., it loops). We further assume that 𝑇 only reaches an accepting configuration when all its input words have the same size. Furthermore, when all inputs have the same size, 𝑇 will reach an accepting configuration.We say that 𝑇 is a linear space machine when it reaches an accepting configuration on inputs of size 𝑛 by using O( 𝑛 ) space on its worktape and additionally needs O( 𝑛 𝑘 ) steps to do so. A linear input-output function is a function of the form 𝑓 = (cid:208) 𝑛 ≥ 𝑓 𝑛 : ( Σ 𝑛 ) ℓ → Σ 𝑛 . Inother words, for every ℓ words of the same size 𝑛 , 𝑓 returns a word of size 𝑛 . We say that a linear input-output function is a linear spaceinput-output function if there exists a linear space machine 𝑇 that for every 𝑛 ≥ , on input 𝑤 , . . . , 𝑤 ℓ ∈ Σ 𝑛 the TM 𝑇 has 𝑓 𝑛 ( 𝑤 , . . . , 𝑤 ℓ ) onits the output tape when (necessarily) reaching an accepting configuration. xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA Proposition D.1.
Let 𝑓 = (cid:208) 𝑛 ≥ 𝑓 𝑛 : ( Σ 𝑛 ) ℓ → Σ 𝑛 be a linear space input-ouput function computed by a linear space machine 𝑇 with 𝑚 states, ℓ input tapes, which consumes O( 𝑛 ) space and runs in O( 𝑛 𝑘 − ) time on inputs of size 𝑛 . There exists (i) a MATLANG schema S = (M , size ) where M consists matrix variables 𝑄 , . . . , 𝑄 𝑚 , 𝑅 , . . . , 𝑅 ℓ , 𝐻 , . . . , 𝐻 ℓ ,𝑊 , . . . ,𝑊 𝑠 , 𝐻 𝑊 , . . . , 𝐻 𝑊 𝑠 , 𝑂, 𝐻 𝑂 , 𝑣 , . . . , 𝑣 𝑘 with size ( 𝑉 ) = 𝛼 × forall 𝑉 ∈ M ; and (ii) a MATLANG expression 𝑒 𝑓 over S such that for the instance I = (D , mat ) over S with D( 𝛼 ) = 𝑛 and mat ( 𝑅 𝑖 ) = vec ( 𝑤 𝑖 ) ∈ R 𝑛 , for 𝑖 ∈ [ ℓ ] and all other matrix variables instantiated with the zero vector in R 𝑛 for words 𝑤 , . . . , 𝑤 ℓ ∈ Σ 𝑛 and such that vec ( 𝑤 𝑖 ) is the 𝑛 × -vector encoding the word 𝑤 𝑖 , we have that mat ( 𝑂 ) = vec ( 𝑓 𝑛 ( 𝑤 , . . . , 𝑤 𝑛 )) ∈ R 𝑛 after evaluating 𝑒 𝑓 on I . Proof. The expression 𝑒 𝑓 we construct will simulate the TM 𝑇 . To have some more control on the space and time consumption of 𝑇 , letus first assume that 𝑛 is large enough, say larger than 𝑛 ≥ 𝑁 , such that 𝑇 runs in 𝑠𝑛 space and 𝑐𝑛 𝑘 − ≤ 𝑛 𝑘 time for constants 𝑠 and 𝑐 . Wedeal with 𝑛 < 𝑁 later on.To simulate 𝑇 we need to encode states, tapes and head positions. The matrix variables in M mentioned in the proposition will take theseroles. More specifically, the variables 𝑅 , . . . , 𝑅 ℓ will hold the input vectors, 𝑊 , . . . ,𝑊 𝑠 will hold the contents of the work tape, where 𝑠 is theconstant mentioned earlier, and 𝑂 will hold the contents of the output tape. The vectors corresponding to the work and output tape areinitially set to the zero vector. The vector for the input tape 𝑅 𝑖 is set to vec ( 𝑤 𝑖 ) , for 𝑖 ∈ [ ℓ ] .With each tape we associate a matrix variable encoding the position of the head. More specifically, 𝐻 , . . . , 𝐻 ℓ correspond to the inputtape heads, 𝐻 𝑊 , . . . , 𝐻 𝑊 𝑠 are the heads for the work tape, and 𝐻 𝑂 is the head of the output tape. All these vectors are initialised with thezero vector. Later on, these vectors will be zero except for a single position, indicating the positions in the corresponding tapes the headspoint to. For those positions 𝑗 , < 𝑗 < 𝑛 , the head vectors will carry value . When 𝑗 = or 𝑛 and when it concerns positions for the inputtape, the head vectors can carry value or . We need to treat these border cases separately because we only have 𝑛 positions available tostore the input words, whereas the actual input tapes consist of 𝑛 + symbols because of ▷ and ◁ . So when, for example, 𝐻 has a in itsfirst entry, we interpret it as the head is pointing to the first symbol of the input word 𝑤 . When 𝐻 has a in its first position, we interpretit as the head pointing to ▷ . The end marker ◁ is dealt with in the same way, by using value or in the last position of 𝐻 . We use thisencoding for all input tapes, and also for the work tape 𝑊 and output tape 𝑂 with the exception that no end marker ◁ is present.To encode the states, we use the variables 𝑄 , . . . , 𝑄 𝑚 . We will ensure that when 𝑇 is in state 𝑞 𝑖 when mat ( 𝑄 𝑖 ) = [ , , . . . , ] 𝑇 ∈ R 𝑛 ,otherwise mat ( 𝑄 𝑖 ) is the zero vector in R 𝑛 .Finally, the variables 𝑣 , . . . , 𝑣 𝑘 represent 𝑘 canonical vectors which are used to iterate in for-loops. By iterating over then, we can perform 𝑛 𝑘 iterations, which suffices for simulating the O( 𝑛 𝑘 − ) steps used by 𝑇 to reach an accepting configuration.With these matrix variables in place, we start by defining 𝑒 𝑓 . It will consists of two subexpressions 𝑒 ≥ 𝑁𝑓 , for dealing with 𝑛 ≥ 𝑁 , and 𝑒 < 𝑁𝑓 ,for dealing with 𝑛 < 𝑁 . We explain the expression 𝑒 ≥ 𝑁𝑓 first.In our expressions we use subexpressions which we defined before in section B.1. These subexpression require some auxiliary variables,as detailed below. As a consequence, 𝑒 𝑓 will be an expressions defined over an extended schema S ′ . Hence, the instance I in the statementof the Proposition is an instance I ′ of S ′ which coincides with I on S and in which the auxiliary matrix variables are all instantiated withzero vectors or matrices, depending on their size.Now, we specify the finite auxiliary variables involved in the for - MATLANG expression. These arise when computing the following for - MATLANG expressions defined • 𝑒 Prev ( 𝑧, 𝑍, 𝑧 ′ , 𝑍 ′ ) , and expression over auxiliary variables 𝑧 , 𝑧 ′ , 𝑍 and 𝑍 ′ with size ( 𝑧 ) = size ( 𝑧 ′ ) = size ( 𝑍 ) = 𝛼 × and size ( 𝑍 ′ ) = 𝛼 × 𝛼 .On input I ′ with mat ( 𝑧 ) = mat ( 𝑧 ′ ) = mat ( 𝑍 ) the zero column vector of dimension 𝑛 , and mat ( 𝑍 ′ ) the zero 𝑛 × 𝑛 matrix, ⟦ 𝑒 Prev ⟧(I ′ ) returns the 𝑛 × 𝑛 matrix Prev such that
Prev · e 𝑖 : = (cid:40) e 𝑖 − if 𝑖 > if 𝑖 = . • 𝑒 Next ( 𝑧, 𝑍, 𝑧 ′ , 𝑍 ′ ) , and expression over auxiliary variables 𝑧 , 𝑧 ′ , 𝑍 and 𝑍 ′ with size ( 𝑧 ) = size ( 𝑧 ′ ) = size ( 𝑍 ) = 𝛼 × and size ( 𝑍 ′ ) = 𝛼 × 𝛼 .On input I ′ with mat ( 𝑧 ) = mat ( 𝑧 ′ ) = mat ( 𝑍 ) the zero column vector of dimension 𝑛 , and mat ( 𝑍 ′ ) the zero 𝑛 × 𝑛 matrix, ⟦ 𝑒 Next ⟧(I ′ ) returns the 𝑛 × 𝑛 matrix Next such that
Next · e 𝑖 : = (cid:40) e 𝑖 + if 𝑖 < 𝑛 if 𝑖 = 𝑛. • min ( 𝑣, 𝑧, 𝑍, 𝑧 ′ , 𝑍 ) with auxiliary variables 𝑧 , 𝑧 ′ , 𝑍 and 𝑍 ′ as before, and 𝑣 is one of the (vector) variables in M . For an 𝑛 × vector v ,on input I ′ [ 𝑣 ← v ] ⟦ min ⟧(I ′ [ 𝑣 ← v ]) : = (cid:40) if v = e otherwise . We also need a finite number of auxiliary variables, these will be specified in the proof. onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč • max ( 𝑣, 𝑧, 𝑍, 𝑧 ′ , 𝑍 ) with auxiliary variables 𝑧 , 𝑧 ′ , 𝑍 and 𝑍 ′ as before, and and 𝑣 is one of the (vector) variables in M . For an 𝑛 × vector v , on input I ′ [ 𝑣 ← v ] ⟦ max ⟧(I ′ [ 𝑣 ← v ]) : = (cid:40) if v = e 𝑛 otherwise . • 𝑒 min ( 𝑧, 𝑍, 𝑧 ′ , 𝑍 ′ , 𝑧 ′′ , 𝑍 ′′ ) , an expressions with auxiliary variables 𝑧 , 𝑧 ′ , 𝑧 ′′ , 𝑍 , 𝑍 ′ and 𝑍 ′′ with size ( 𝑧 ) = size ( 𝑧 ′ ) = size ( 𝑧 ′′ ) = size ( 𝑍 ) = size ( 𝑍 ′′ ) = 𝛼 × and size ( 𝑍 ′ ) = 𝛼 × 𝛼 . On input I ′ with matrix variables instantiated with zero vectors (or matrix for 𝑍 ′ ), ⟦ 𝑒 min ⟧(I ′ ) = e . • 𝑒 max ( 𝑧, 𝑍, 𝑧 ′ , 𝑍 ′ , 𝑧 ′′ , 𝑍 ′′ ) , an expressions with auxiliary variables 𝑧 , 𝑧 ′ , 𝑧 ′′ , 𝑍 , 𝑍 ′ and 𝑍 ′′ with size ( 𝑧 ) = size ( 𝑧 ′ ) = size ( 𝑧 ′′ ) = size ( 𝑍 ) = size ( 𝑍 ′′ ) = 𝛼 × and size ( 𝑍 ′ ) = 𝛼 × 𝛼 . On input I ′ with matrix variables instantiated with zero vectors (or matrix for 𝑍 ′ ), ⟦ 𝑒 max ⟧(I ′ ) = e 𝑛 .We thus see that we only need 𝑧, 𝑧 ′ , 𝑧 ′′ , 𝑍, 𝑍 ′ , 𝑍 ′′ as auxiliary variables and these can be re-used whenever 𝑒 𝑓 calls these functions. Fromnow one, we omit the auxiliary variables from the description of 𝑒 𝑓 .Let us first define 𝑒 ≥ 𝑁𝑓 . Since we want to simulate 𝑇 we need to be able to check which transitions of 𝑇 can be applied based on a currentconfiguration. More precisely, suppose that we want to check whether 𝛿 ( 𝑞 𝑖 , ( 𝑎 , . . . , 𝑎 ℓ , 𝑏, 𝑐 )) is applicable, then we need to check whether 𝑇 is in state 𝑞 𝑖 , we can do this by checking min ( 𝑄 𝑖 ) , and whether the heads on the tapes read symbols 𝑎 , . . . , 𝑎 ℓ , 𝑏, 𝑐 . We check the latter bythe following expressions. For the input tape 𝑅 𝑖 we define test _ inp 𝑖𝑏 : = ( − min ( / · 𝐻 𝑖 )) · ( − max ( / · 𝐻 𝑖 )) · ( − 𝑅 𝑇𝑖 · 𝐻 𝑖 ) if 𝑏 = ( − min ( / · 𝐻 𝑖 )) · ( − max ( / · 𝐻 𝑖 )) · ( 𝑅 𝑇𝑖 · 𝐻 𝑖 ) if 𝑏 = min ( / · 𝐻 𝑖 ) if 𝑏 = ▷ max ( / · 𝐻 𝑖 ) if 𝑏 = ◁ , which returns if and only if either 𝑏 ∈ { , } is the value in mat ( 𝑅 𝑖 ) at the position encoded by mat ( 𝐻 𝑖 ) , or when 𝑏 = ▷ and mat ( 𝐻 𝑖 ) isthe vector ( , , . . . , ) ∈ R 𝑛 , or when 𝑏 = ◁ and mat ( 𝐻 𝑖 ) is the vector ( , , . . . , ) ∈ R 𝑛 . Similarly, for the output tape we define test _ out 𝑏 : = ( − min ( / · 𝐻 𝑂 )) · ( − 𝑂 𝑇 · 𝐻 𝑂 ) if 𝑏 = ( − min ( / · 𝐻 𝑂 )) · ( 𝑂 𝑇 · 𝐻 𝑂 ) if 𝑏 = min ( / · 𝐻 𝑂 ) if 𝑏 = ▷ , and for the work tapes 𝑊 , . . . ,𝑊 𝑠 we define test _ work 𝑖𝑏 : = ( − min ( / · 𝐻 𝑊 𝑖 )) · ( − 𝑊 𝑇𝑖 · 𝐻 𝑊 𝑖 )) if 𝑏 = ( − min ( / · 𝐻 𝑊 𝑖 )) · ( 𝑊 𝑇𝑖 · 𝐻 𝑊 𝑖 ) if 𝑏 = min ( / · 𝐻 𝑊 𝑖 ) if 𝑏 = ▷ and 𝑖 = . We then combine all these expressions into a single expression for 𝑞 𝑖 ∈ 𝑄 , 𝑎 , . . . , 𝑎 ℓ , 𝑏, 𝑐 ∈ Γ : isconf 𝑞 𝑖 ,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 : = min ( 𝑄 𝑖 ) · (cid:169)(cid:173)(cid:171) ℓ (cid:214) 𝑗 = test _ inp 𝑗𝑎 𝑗 (cid:170)(cid:174)(cid:172) · (cid:169)(cid:173)(cid:171) 𝑠 ∑︁ 𝑗 = test _ work 𝑗𝑏 (cid:170)(cid:174)(cid:172) · test _ out 𝑐 . This expression will return if and only if the vectors representing the tapes, head positions and states are such that Q i is the first canonicalvector (and thus 𝑇 is in state 𝑞 𝑖 ), the heads point to entries in the tape vectors storing the symbols 𝑎 , . . . , 𝑎 ℓ , 𝑏, 𝑐 or they point to the first (orlast for input tapes) positions but have value (when the symbols are ▷ or ◁ ).To ensure that at the beginning of the simulation of 𝑇 by 𝑒 ≥ 𝑁𝑓 we correctly encode that we are in the initial configuration, we thus needto initialise all vectors mat ( 𝐻 ) , mat ( 𝐻 ) , . . . , mat ( 𝐻 ℓ ) , mat ( 𝐻 𝑊 ) , mat ( 𝐻 𝑂 ) with the vector ( , , , . . . , ) ∈ R since all heads read thesymbol ▷ . Similarly, we have to initialise Q with the first canonical vector since 𝑇 is in state 𝑞 .We furthermore need to be able to correctly adjust head positions. We do this by means of the predecessor and successor expressionsdescribed above. A consequence of our encoding is that we need to treat the border cases (corresponding to ▷ and ◁ ) differently. Morespecifically, for the input tapes 𝑅 𝑖 and heads 𝐻 𝑖 we define move _ inp 𝑖 d : = × min ( 𝐻 𝑖 ) × 𝐻 𝑖 + / × max ( / × 𝐻 𝑖 ) × 𝐻 𝑖 + ( − min ( 𝐻 𝑖 ))( − max ( / × 𝐻 𝑖 )) × 𝑒 Prev · 𝐻 𝑖 if d = ← × max ( 𝐻 𝑖 ) × 𝐻 𝑖 + / × min ( / × 𝐻 𝑖 ) × 𝐻 𝑖 + ( − min ( / × 𝐻 𝑖 ))( − max ( 𝐻 𝑖 )) × 𝑒 Next · 𝐻 𝑖 if d = → 𝐻 𝑖 if d = ⊔ . In other words, we shift to the previous (or next) canonical vector when d is ← or → , respectively, unless we need to move to or from theposition that will hold ▷ or ◁ . In those case we readjust mat ( 𝐻 𝑖 ) (which will either ( , , . . . , ) , ( , , . . . , ) , ( , . . . , , ) or ( , . . . , , ) ) by xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA either dividing or multiplying with . In this way we can correctly infer whether or not the head points to the begin and end markers. Forthe output tape we proceed in a similar way, but only taking into account the begin marker and recall that we do not have moves to the left: move _ outp d : = (cid:40) / × min ( / × 𝐻 𝑂 ) × 𝐻 𝑂 + ( − min ( / × 𝐻 𝑂 )) × 𝑒 Next × 𝐻 𝑂 if d = → 𝐻 𝑂 if d = ⊔ . Since we represent the work tape by 𝑠 vectors 𝑊 , . . . ,𝑊 𝑠 we need to ensure that only one of the head vectors 𝐻 𝑊 𝑖 has a non-zero value andthat by moving left or right, we need to appropriately update the right head vector. We do this as follows. We first consider the work tapes 𝑊 𝑖 for 𝑖 ≠ , 𝑠 and define move _ work 𝑖 d : = − min ( 𝐻 𝑊 𝑖 ) × 𝐻 𝑊 𝑖 + ( − min ( 𝐻 𝑊 𝑖 )) × 𝑒 Prev · 𝐻 𝑊 𝑖 + min ( 𝐻 𝑊 𝑖 + ) × 𝑒 max if d = ←− max ( 𝐻 𝑊 𝑖 ) × 𝐻 𝑊 𝑖 + ( − max ( 𝐻 𝑊 𝑖 )) × 𝑒 Next · 𝐻 𝑊 𝑖 + max ( 𝐻 𝑊 𝑖 − ) × 𝑒 min if d = → 𝐻 𝑊 𝑖 if d = ⊔ . In other words, we set the 𝐻 𝑊 𝑖 to zero when a move brings us to either 𝑊 𝑖 − or 𝑊 𝑖 + , we move the successor or predecessor when stayingwithin 𝑊 𝑖 , or initialise 𝐻 𝑊 𝑖 with the first or last canonical vector when moving from 𝑊 𝑖 − to 𝑊 𝑖 (right move) or from 𝑊 𝑖 + to 𝑊 𝑖 (left move).For 𝑖 = 𝑠 we can ignore the parts in the previous expression that involve 𝑊 𝑠 + (which does not exist): move _ work 𝑠 d : = − min ( 𝐻 𝑊 𝑠 ) × 𝐻 𝑊 𝑖 + ( − min ( 𝐻 𝑊 𝑠 )) × 𝑒 Prev · 𝐻 𝑊 𝑠 if d = ←− max ( 𝐻 𝑊 𝑠 ) × 𝐻 𝑊 𝑠 + ( − max ( 𝐻 𝑊 𝑠 )) × 𝑒 Next · 𝐻 𝑊 𝑠 + max ( 𝐻 𝑊 𝑠 − ) × 𝑒 min if d = → 𝐻 𝑊 𝑠 if d = ⊔ . For 𝑖 = , we can ignore the part involving 𝑊 (which does not exist) but have to take ▷ into account: move _ work d : = × min ( 𝐻 𝑊 ) × 𝐻 𝑊 𝑖 + ( − min ( 𝐻 𝑊 )) × 𝑒 Prev · 𝐻 𝑊 + min ( 𝐻 𝑊 ) × 𝑒 max if d = ← / × min ( / × 𝐻 𝑊 ) × 𝐻 𝑊 + ( − max ( / × 𝐻 𝑊 )) × 𝑒 Next · 𝐻 𝑊 if d = → 𝐻 𝑊 if d = ⊔ . A final ingredient for defining 𝑒 ≥ 𝑁𝑓 are expressions which update the work and output tape. To define these expression, we need the positionand symbol to put on the tape. For the output tape we define write _ outp 𝑏 : = min ( / × 𝐻 𝑂 ) × 𝑂 if 𝑏 = ▷ ( − min ( / × 𝐻 𝑂 )) × (cid:16) ( − 𝑂 𝑇 · 𝐻 𝑂 ) × 𝑂 + ( 𝑂 𝑇 · 𝐻 𝑂 ) × ( 𝑂 − 𝐻 𝑂 ) (cid:17) if 𝑏 = ( − min ( / × 𝐻 𝑂 )) × (cid:16) ( − 𝑂 𝑇 · 𝐻 𝑂 ) × ( 𝑂 + 𝐻 𝑂 ) + ( 𝑂 𝑇 · 𝐻 𝑂 ) × 𝑂 (cid:17) if 𝑏 = and similarly for the work tapes 𝑖 ≠ : write _ work 𝑖𝑏 : = 𝑊 𝑖 if 𝑏 = ▷ ( − 𝑊 𝑇𝑖 · 𝐻 𝑊 𝑖 ) × 𝑊 𝑖 + ( 𝑊 𝑇𝑖 · 𝐻 𝑊 𝑖 ) × ( 𝑊 𝑖 − 𝐻 𝑊 𝑖 ) if 𝑏 = ( − 𝑊 𝑇𝑖 · 𝐻 𝑊 𝑖 ) × ( 𝑊 𝑖 + 𝐻 𝑊 𝑖 ) + ( 𝑊 𝑇𝑖 · 𝐻 𝑊 𝑖 ) × 𝑊 𝑖 if 𝑏 = , and for 𝑊 we have to take care again of the begin marker: write _ work 𝑏 : = min ( / × 𝐻 𝑊 ) × 𝑊 if 𝑏 = ▷ ( − min ( / × 𝐻 𝑊 ) × (cid:16) ( − 𝑊 𝑇 · 𝐻 𝑊 ) × 𝑊 + ( 𝑊 𝑇 · 𝐻 𝑊 ) × ( 𝑊 − 𝐻 𝑊 ) (cid:17) if 𝑏 = ( − min ( / × 𝐻 𝑊 ) × (cid:16) ( − 𝑊 𝑇 · 𝐻 𝑊 ) × ( 𝑊 + 𝐻 𝑊 ) + ( 𝑊 𝑇 · 𝐻 𝑊 ) × 𝑊 (cid:17) if 𝑏 = . We are now finally ready to define 𝑒 ≥ 𝑁𝑓 : 𝑒 ≥ 𝑁𝑓 : = for 𝑣 , . . . , 𝑣 𝑘 , 𝑄 , . . . , 𝑄 𝑚 , 𝐻 , . . . , 𝐻 ℓ ,𝑊 , . . . ,𝑊 𝑠 , 𝐻 𝑊 , . . . , 𝐻 𝑊 𝑠 , 𝑂, 𝐻 𝑂 . ( 𝑒 𝑄 , . . . , 𝑒 𝑄 𝑚 , 𝑒 𝐻 , . . . , 𝑒 𝐻 ℓ , 𝑒 𝑊 , . . . , 𝑒 𝑊 𝑠 , 𝑒 𝐻 𝑊 , . . . , 𝑒 𝐻 𝑊𝑠 , 𝑒 𝑂 , 𝑒 𝐻 𝑂 ) . onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč We refer to section A.1 for the definition of this form of the for-loop. The expressions used are (we use ★ below to mark irrelevant informationin the transitions): 𝑒 𝑄 : = (cid:169)(cid:173)(cid:171) 𝑘 (cid:214) 𝑗 = min ( 𝑣 𝑖 ) (cid:170)(cid:174)(cid:172) × 𝑒 min + ∑︁ ( 𝑞 𝑖 ,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 ) Δ ( 𝑞 𝑖 ,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 ) = ( 𝑞 , ★ ) isconf 𝑞 𝑖 ,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 × 𝑒 min 𝑒 𝑄 𝑗 : = ∑︁ ( 𝑞 𝑖 ,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 ) Δ ( 𝑞 𝑖 ,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 ) = ( 𝑞 𝑗 , ★ ) isconf 𝑞 𝑖 ,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 × 𝑒 min for 𝑗 ≠ 𝑒 𝐻 𝑖 : = (cid:169)(cid:173)(cid:171) 𝑘 (cid:214) 𝑗 = min ( 𝑣 𝑖 ) (cid:170)(cid:174)(cid:172) × 𝑒 min + ∑︁ ( 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑑 ) Δ ( 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 ) = ( ★ , d i , ★ ) isconf 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 × move _ inp 𝑖 d 𝑖 𝑒 𝐻 𝑊𝑖 : = (cid:169)(cid:173)(cid:171) 𝑘 (cid:214) 𝑗 = min ( 𝑣 𝑖 ) (cid:170)(cid:174)(cid:172) × 𝑒 min + ∑︁ ( 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑑 ) Δ ( 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 ) = ( ★ , d ℓ + , ★ ) isconf 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 × move _ work 𝑖 d ℓ + 𝑒 𝐻 𝑂 : = (cid:169)(cid:173)(cid:171) 𝑘 (cid:214) 𝑗 = min ( 𝑣 𝑖 ) (cid:170)(cid:174)(cid:172) × 𝑒 min + ∑︁ ( 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑑 ) Δ ( 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 ) = ( ★ , d ℓ + ) isconf 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 × move _ outp d ℓ + 𝑒 𝑊 𝑖 : = ∑︁ ( 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑑 ) Δ ( 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 ) = ( ★ ,𝑏 ′ ,𝑐 ′ , ★ ) isconf 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 × write _ work 𝑖𝑏 ′ 𝑒 𝑂 : = ∑︁ ( 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑑 ) Δ ( 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 ) = ( ★ ,𝑏 ′ ,𝑐 ′ , ★ ) isconf 𝑞,𝑎 ,...,𝑎 ℓ ,𝑏,𝑐 × write _ outp 𝑐 ′ . The correctness of 𝑒 ≥ 𝑁𝑓 should be clear from the construction (one can formally verify this by induction on the number of iterations). Wenext explain how the border cases 𝑛 < 𝑁 can be dealt with. For each 𝑛 < 𝑁 and every possible input words 𝑤 , . . . , 𝑤 ℓ of size 𝑛 , we define a for - MATLANG expression which checks whether mat ( 𝑅 𝑖 ) = vec ( 𝑤 𝑖 ) for each 𝑖 ∈ [ ℓ ] . This can be easily done since 𝑛 can be regarded as aconstant. For example, to check whether mat ( 𝑅 𝑖 ) = [ , , ] 𝑇 we simply write ( − 𝑅 𝑇𝑖 · 𝑒 min ) × ( 𝑅 𝑇𝑖 · 𝑒 Next · 𝑒 min ) × ( − 𝑅 𝑇𝑖 · 𝑒 Next · 𝑒 Next · 𝑒 min ) × ( − 𝑒 ( 𝑅 𝑖 ) 𝑇 · 𝑒 Next · 𝑒 Next · 𝑒 Next · 𝑒 min ) which will evaluate to if and only if mat ( 𝑅 𝑖 ) = [ , , ] 𝑇 . We note that the final factor is in place to check that the dimension of mat ( 𝑅 𝑖 ) isthree. We denote by 𝑒 𝑖𝑛,𝑤 the expression which evaluates to if and only if mat ( 𝑅 𝑖 ) = vec ( 𝑤 ) for | 𝑤 | = 𝑛 . We can similarly write any word 𝑤 of fixed size in the matrix variable 𝑂 . For example, suppose that 𝑤 = then we write 𝑂 + 𝑒 min + 𝑒 Next · 𝑒 Next · 𝑒 min . We write 𝑒 𝑛,𝑤 be the expression which write 𝑤 of size | 𝑤 | = 𝑛 in matrix variable 𝑂 . Then, the expressions 𝑒 𝑛,𝑤 ,...,𝑤 𝑛 ,𝑤 : = 𝑒 𝑛,𝑤 · · · · · 𝑒 ℓ𝑛𝑤 ℓ · 𝑒 𝑛,𝑤 will write 𝑤 in 𝑂 if and only if mat ( 𝑅 𝑖 ) = vec ( 𝑤 𝑖 ) for 𝑖 ∈ [ ℓ ] . We now simply take the disjunction over all words 𝑤 , . . . , 𝑤 ℓ ∈ Σ 𝑛 and 𝑤 = 𝑓 𝑛 ( 𝑤 , . . . , 𝑤 ℓ ) ∈ Σ 𝑛 : 𝑒 𝑛 : = ∑︁ 𝑤 ,...,𝑤 ℓ ∈ Σ 𝑛 𝑒 𝑛,𝑤 ,...,𝑤 ℓ ,𝑓 𝑛 ( 𝑤 ,...,𝑤 ℓ ) , which correctly evaluates 𝑓 𝑛 . We next take a further disjunction by letting ranging from 𝑛 = , . . . , 𝑁 − : 𝑒 < 𝑁𝑓 : = 𝑁 − ∑︁ 𝑛 = 𝑒 𝑛 Since every possible input is covered and only a unique expression 𝑒 𝑛,𝑤 ,...,𝑤 ℓ ,𝑓 𝑛 ( 𝑤 ,...,𝑤 ℓ ) will be triggered 𝑒 < 𝑁𝑓 will correctly evaluate 𝑓 oninputs smaller than 𝑁 .Our final expression 𝑒 𝑓 is now given by 𝑒 𝑓 : = 𝑒 < 𝑁𝑓 + dim _ is _ greater _ than N × 𝑒 ≥ 𝑁𝑓 xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA where dim _ is _ greater _ than N is the expression 𝑒 ( 𝑅 𝑖 ) 𝑇 · 𝑒 Next · · · · · 𝑒 Next (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) 𝑁 times which will evaluate to if an only if the input dimension is largeror equal than 𝑁 . □ D.2 Circuit evaluation
We prove theorem 5.1:Theorem 5.1.
For any uniform arithmetic circuit family { Φ 𝑛 | 𝑛 = , , . . . } of logarithmic depth there is a for - MATLANG schema S and anexpression 𝑒 Φ using a matrix variable 𝑣 , with type S ( 𝑣 ) = ( 𝛼, ) and type S ( 𝑒 ) = ( , ) , such that for any input 𝑎 , . . . , 𝑎 𝑛 to the circuit Φ 𝑛 : • If I = (D , mat ) is a MATLANG instance such that D( 𝛼 ) = 𝑛 and mat ( 𝑣 ) = [ 𝑎 . . . 𝑎 𝑛 ] 𝑇 • Then ⟦ 𝑒 ⟧(I) = Φ 𝑛 ( 𝑎 , . . . , 𝑎 𝑛 ) . Proof. Let Φ 𝑛 be a circuit with 𝑛 input gates and such that it can be computed by a 𝐿 − uniform arithmetic circuit of log-depth. Each gateof the circuit that encodes 𝑓 has an id ∈ { , } 𝑛 . From now on, when we write 𝑔 for a gate of the circuit, we mean the id encoding 𝑔 . Let 𝑛 𝑘 be a polynomial such that the number of wires 𝑊 ( 𝑛 ) ≤ 𝑛 𝑘 for 𝑛 big enough. Further, we assume that 𝑊 ( 𝑛 ) ≤ 𝑛 𝑘 . We need this because thefor-matlang simulation of the circuit is in a depth first search way, so 𝑊 ( 𝑛 ) wires will be traversed. Then we have that: • the number of gates is bounded by 𝑛 𝑘 . • we need at most 𝑘 log ( 𝑛 ) bits to store the 𝑖𝑑 of a gate. • the depth of the circuit is at most 𝑘 ′ log ( 𝑛 ) for some 𝑘 ′ .So, let 𝑛 and 𝑘 such that ∀ 𝑛 ≥ 𝑛 : 2 𝑊 ( 𝑛 ) ≤ 𝑛 𝑘 𝑘 ⌈ log ( 𝑛 )⌉ ≤ 𝑛 − 𝑘 ′ ⌈ log ( 𝑛 )⌉ ≤ 𝑛 ( ★ ) We know 𝑛 and 𝑘 exist. Let 𝑛 ≥ 𝑛 . Towards the end, we will deal with the case when 𝑛 < 𝑛 .Let 𝑔 be a gate. The children of 𝑔 are denoted by 𝑔 , . . . , 𝑔 𝑙 . 𝑔𝑔 · · · 𝑔 𝑙 For example, a circuit that encodes the function 𝑓 ( 𝑎 , 𝑎 , 𝑎 , 𝑎 ) = 𝑎 𝑎 + 𝑎 𝑎 is +× 𝑎 𝑎 × 𝑎 𝑎 We can simulate the polynomial 𝑥 + 𝑥𝑦 by doing 𝑓 ( 𝐴 ) where 𝐴 = [ 𝑥 𝑥 𝑥 𝑦 ] 𝑇 . The main idea is to traverse the circuit top down in adepth first search way and store visited gates in a stack and its corresponding current values in another stack, and aggregate in the iterationsaccording to the gate type.For a stack 𝑆 , the operations are standard: • 𝑆 .push ( 𝑠 ) : pushes 𝑠 into 𝑆 . • 𝑆 .pop : pops the top element. • 𝑆 .size : the length of the stack. • 𝑆 .top : the top element in the stack. onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč For the pseudo-code, G and V denote stacks of gates and values, respectively. The property that holds during the simulation is that thevalue in V [ 𝑖 ] is the value that G[ 𝑖 ] currently outputs. The algorithm ends with G = [ 𝑔 root ] and V = [ 𝑣 root ] after traversing the circuit,and returns 𝑣 root During the evaluation algorithm there will be two possible configurations of G and V .(1) G .size = V .size + : this means that G .top is a gate that we visit for the first time and we need to initialize its value.(2) G .size = V .size : here V .top is the value of evaluating the circuit in gate G .top . Therefore, we need to aggregate the value V .top to the parent gate of 𝑔 .We assume the circuit has input gates, + , × -gates and allow constant -gate.The idea is to traverse the circuit top down in a depth first search way. For example, in the circuit 𝑓 ( 𝑎 , 𝑎 , 𝑎 , 𝑎 ) = 𝑎 𝑎 + 𝑎 𝑎 above,we would initialize the output gate value as because it is a + gate, so G = {+} , V = { } . Then stack the left × gate to G , stack its initialvalue (i.e. ) to V . Now stack 𝑎 to G and its value (i.e. 𝑎 ) to V . Since we are on an input gate we pop the gate and value pair off of G and V respectively, aggregate 𝑎 to V .top and continue by stacking the 𝑎 gate to G . We pop 𝑎 off of V (and its gate off of G ) and aggregateits value to V .top . We pop and aggregate the value of the left × gate to V .top (the root value). Then continue with the right × gate branchsimilarly.For the pseudo-code, we supply ourselves with the following functions:– isplus ( 𝑔 ) : true if and only if 𝑔 is a + -gate.– isprod ( 𝑔 ) : true if and only if 𝑔 is a × -gate.– isone ( 𝑔 ) : true if and only if 𝑔 is a -gate.– isinput ( 𝑔 ) : true if and only if 𝑔 is an input gate.– getfirst ( 𝑔 ) : outputs the first child of 𝑔 .– getinput ( 𝑔 ) : outputs 𝐴 [ 𝑖 ] when 𝑔 is the 𝑖 -th input.– not_last ( 𝑔 , 𝑔 ) : true if and only if 𝑔 is not the last child gate of 𝑔 .– next_gate ( 𝑔 , 𝑔 ) : outputs the next child gate of 𝑔 after 𝑔 .– getroot () : outputs the root gate of the circuit.The corresponding { , } 𝑛 → { , } 𝑛 functions are:– isplus ( 𝑔 ) : if and only if 𝑔 is a + -gate.– isprod ( 𝑔 ) : if and only if 𝑔 is a × -gate.– isone ( 𝑔 ) : if and only if 𝑔 is a -gate.– isinput ( 𝑔 ) : if and only if 𝑔 is an input gate.– getfirst ( 𝑔 ) : outputs the id of the first child of 𝑔 .– getinput ( 𝑔 ) : outputs canonical vector 𝑏 𝑖 , where the 𝑖 -th input gate of Φ 𝑛 is encoded by 𝑔 .– not_last ( 𝑔 , 𝑔 ) : if and only if 𝑔 is not the last child gate of 𝑔 .– next_gate ( 𝑔 , 𝑔 ) : outputs the id of the next child gate of 𝑔 after 𝑔 .– getroot () : outputs the id of the root gate of the circuit.The previous functions are all definable by an 𝐿 -transducer and can be defined from the 𝐿 -transducer of 𝑓 . Then, by proposition D.1, foreach of these functions there is a for - MATLANG expression that simulates them.Now, we give the pseudo-code of the top-down evaluation. We define the functions
𝐼𝑛𝑖𝑡𝑖𝑎𝑙𝑖𝑧𝑒 (algorithm 1),
𝐴𝑔𝑔𝑟𝑒𝑔𝑎𝑡𝑒 (algorithm 2) and
𝐸𝑣𝑎𝑙𝑢𝑎𝑡𝑒 (algorithm 3). The main algorithm is
𝐸𝑣𝑎𝑙𝑢𝑎𝑡𝑒 . Algorithm 1
Initialize (pseudo-code) function Initialize( G , V , 𝐴 ) ⊲ The stacks and input. Here, G .size = V .size + if isplus (G .top ) then V .push ( ) G .push ( getfirst (G .top )) else if isprod (G .top ) then V .push ( ) G .push ( getfirst (G .top )) else if isone (G .top ) then V .push ( ) else if isinput (G .top ) then V .push ( 𝐴 [ getinput (G .top )]) return G , V xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA Algorithm 2
Aggregate (pseudo-code) function Aggregate( G , V ) ⊲ Here, G .size = V .size 𝑔 = G .pop 𝑣 = V .pop if isplus (G .top ) then V .top = V .top + 𝑣 else if isprod (G .top ) then V .top = V .top · 𝑣 if not_last (G .top , 𝑔 ) then G .push ( next_gate (G .top , 𝑔 )) return G , V Algorithm 3
Evaluate (pseudo-code) function Evaluate( 𝐴 ) ⊲ Input 𝑛 × vector 𝐴 . Here, G and V are empty G .push ( getroot ()) while G .size ≠ or V .size ≠ do if G .size ≠ 𝑉 .size then (G , V) : = Initialize (G , V , 𝐴 ) else (G , V) : = Aggregate (G , V) return V .top The
𝐸𝑣𝑎𝑙𝑢𝑎𝑡𝑒 algorithm gives us the output of the circuit. Note that after each iteration it either holds that G .size = V .size + or G .size = V .size . Furthermore, when we start we have G .size = and V .size = . The condition G .size = and V .size = holdsonly when we have traversed all the circuit, and the value in V .top is the value that the root of the circuit outputs after its computation.Next, we show how to encode this algorithm in for - MATLANG .Let 𝑛 ∈ N be big enough for ( ★ ) to hold and let 𝑛 ≥ 𝑘 . Hence, the number of gates (values) is bounded by 𝑛 𝑘 and we need 𝑘 log ( 𝑛 ) bits toencode the id of each gate.To simulate the two stacks G and V we keep a matrix 𝑋 of dimensions 𝑛 × 𝑛 . • Column 𝑛 will store a canonical vector that marks the top of stack 𝑉 (values). • Column 𝑛 − will store a canonical vector that marks the top of stack 𝐺 (gates). • Column 𝑛 − is the stack of values where 𝑋 [ , 𝑛 − ] is the bottom of the stack. • Columns to 𝑛 − are the stack of gates.If we have 𝑗 gates in the stack and currently G .size = V .size then 𝑋 would look like: 𝑋 = id 𝑣 id 𝑣 ... ... ... ... id 𝑗 𝑣 𝑗 ... ... ... ... . Since 𝑛 ≥ 𝑛 , ( ★ ) holds and thus we never use more than 𝑛 − bits to encode an id . Also, 𝑗 ≤ 𝑛 given that we never keep more gatesthan the depth of the tree. As a consequence, we never keep more than 𝑛 values either.An important detail is that the ids of the gates are encoded as id 𝑟 for it to have dimension 𝑛 , where id 𝑟 is the corresponding binarynumber in reverse.We make a series of definitions to make the notation more clear. Refer to section B.1 for more information about these expressions.Let 𝑏 𝑖 be the 𝑖 -th canonical vector. Next and
Prev denote the successor and predecessor matrices respectively, such that
Next · 𝑏 𝑖 = (cid:40) 𝑏 𝑖 + if 𝑖 ≤ 𝑛 otherwise onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč Prev · 𝑏 𝑖 = (cid:40) 𝑏 𝑖 − if 𝑖 ≥ 𝑛 otherwiseWe write expressions 𝑒 min for the first canonical vector and 𝑒 max for the last canonical vector. For any 𝑖 we write 𝑒 min + i = Next 𝑖 · 𝑒 min 𝑒 max + i = Prev 𝑖 · 𝑒 max We use the extra { , } 𝑛 → { , } 𝑛 functions that have a for - MATLANG translation: min ( 𝑒 ) = (cid:40) if 𝑒 = 𝑒 min otherwise max ( 𝑒 ) = (cid:40) if 𝑒 = 𝑒 max otherwise succ ( 𝑏 𝑖 , 𝑏 𝑗 ) = (cid:40) if 𝑖 ≤ 𝑗 otherwiseWhen used in for - MATLANG these functions output [ ] and [ ] .Now 𝑒 𝑉 : = 𝑒 max − 𝑒 𝐺 𝑡𝑜𝑝 : = 𝑒 max − 𝑒 𝑉 𝑡𝑜𝑝 : = 𝑒 max For a canonical vector, let
Iden ( 𝑏 𝑖 ) : = Σ 𝑣. succ ( 𝑣, 𝑏 𝑖 ) · ( 𝑣 · 𝑣 𝑇 ) . This matrix has ones in the diagonal up to position 𝑖 marked by 𝑒 𝑖 . We define the following sub-matrices of 𝑋 : 𝑉 𝑡𝑜𝑝 : = 𝑋 · 𝑒 𝑉 𝑡𝑜𝑝 𝑉 : = Iden ( 𝑉 𝑡𝑜𝑝 ) · 𝑋 · 𝑒 𝑉 𝐺 𝑡𝑜𝑝 : = 𝑋 · 𝑒 𝐺 𝑡𝑜𝑝 𝐺 : = Iden ( 𝐺 𝑡𝑜𝑝 ) · 𝑋 · Iden ( 𝑒 max − ) For example, if we are in a step where G .size = V .size + then 𝑋 = id 𝑣 id 𝑣 ... ... ... ... id 𝑗 − 𝑣 𝑗 − id 𝑗 ... ... ... ... , 𝐺 = id id ... ... ... ... id 𝑗 − id 𝑗 ... ... ... ... , 𝑉 = 𝑣 𝑣 ...𝑣 𝑗 − ... , 𝐺 𝑡𝑜𝑝 = ... ... , 𝑉 𝑡𝑜𝑝 = ... ... Here, 𝑉 is a vector encoding the stack of values in 𝑋 and 𝐺 is a matrix encoding the stack of gates in 𝑋 . Note that what is over the top ofthe stacks is always set to zero due to Iden ( 𝐺 𝑡𝑜𝑝 ) and Iden ( 𝑉 𝑡𝑜𝑝 ) . Also, note that 𝐺 is of the same size as 𝑋 . We sometimes omit the zeroesdue to simplicity.To set the initial state (algorithm 3 line 2) we define the for - MATLANG expression:START : = 𝑒 min · getroot () 𝑇 + 𝑒 min · 𝑒 𝑇𝐺 𝑡𝑜𝑝 . For the initialize step, we define the for - MATLANG expressions: INIT_PLUS (algorithm 1, lines 2, 3, 4), INIT_PROD (algorithm 1, lines 5, 6, 7),CONST (algorithm 1, lines 8, 9) and INPUT (algorithm 1, lines 10, 11): xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA
INIT_PLUS : = isplus (cid:16) 𝐺 𝑇 · 𝐺 𝑡𝑜𝑝 (cid:17) × (cid:20) 𝐺 + ( Next · 𝐺 𝑡𝑜𝑝 ) · getfirst (cid:16) 𝐺 𝑇 · 𝐺 𝑡𝑜𝑝 (cid:17) 𝑇 + Next · 𝐺 𝑡𝑜𝑝 · 𝑒 𝑇𝐺 𝑡𝑜𝑝 + 𝑉 · 𝑒 𝑇𝑉 + Next · 𝑉 𝑡𝑜𝑝 · 𝑒 𝑇𝑉 𝑡𝑜𝑝 (cid:21) INIT_PROD : = isprod (cid:16) 𝐺 𝑇 · 𝐺 𝑡𝑜𝑝 (cid:17) × (cid:20) 𝐺 + ( Next · 𝐺 𝑡𝑜𝑝 ) · getfirst (cid:16) 𝐺 𝑇 · 𝐺 𝑡𝑜𝑝 (cid:17) 𝑇 + Next · 𝐺 𝑡𝑜𝑝 · 𝑒 𝑇𝐺 𝑡𝑜𝑝 + ( 𝑉 + Next · 𝑣 𝑡𝑜𝑝 ) · 𝑒 𝑇𝑉 + Next · 𝑉 𝑡𝑜𝑝 · 𝑒 𝑇𝑉 𝑡𝑜𝑝 (cid:21) CONST : = isone (cid:16) 𝐺 𝑇 · 𝐺 𝑡𝑜𝑝 (cid:17) × (cid:104) 𝐺 + ( 𝑉 + Next · 𝑉 𝑡𝑜𝑝 ) · 𝑒 𝑇𝑉 + Next · 𝑉 𝑡𝑜𝑝 · 𝑒 𝑇𝑉 𝑡𝑜𝑝 (cid:105) INPUT : = isinput (cid:16) 𝐺 𝑇 · 𝐺 𝑡𝑜𝑝 (cid:17) × (cid:20) 𝐺 + (cid:18) 𝑉 + (cid:18) 𝑣 𝑇 · Next · 𝑉 𝑡𝑜𝑝 · getinput (cid:16) 𝐺 𝑇 · 𝐺 𝑡𝑜𝑝 (cid:17) 𝑇 (cid:19)(cid:19) · 𝑒 𝑇𝑉 + Next · 𝑉 𝑡𝑜𝑝 · 𝑒 𝑇𝑉 𝑡𝑜𝑝 (cid:21) Where 𝑣 is the matrix variable stated in the theorem, the one associated with the input 𝐴 of the circuit. Here, 𝐺 𝑇 · 𝐺 𝑡𝑜𝑝 is to get the currentid in the top of the stack. In INIT_PLUS we get the current stack 𝐺 , we add Next · 𝐺 𝑡𝑜𝑝 · getfirst (cid:16) 𝐺 𝑇 · 𝐺 𝑡𝑜𝑝 (cid:17) 𝑇 which is an 𝑛 × 𝑛 matrixwith the first child of 𝐺 𝑇 · 𝐺 𝑡𝑜𝑝 in the next row. Then Next · 𝐺 𝑡𝑜𝑝 · 𝑒 𝑇𝐺 𝑡𝑜𝑝 adds Next · 𝐺 𝑡𝑜𝑝 to the 𝑛 − column to mark the gate we added asthe top. Next, we do the same with the values by adding 𝑉 · 𝑒 𝑉 + Next · 𝑉 𝑡𝑜𝑝 · 𝑒 𝑇𝑉 𝑡𝑜𝑝 .The for - MATLANG expression equivalent to algorithm 1 isINIT : = INIT_PLUS + INIT_PROD + CONST + INPUT . The idea is to return the matrix for the next iteration. Recall that here G .size = V .size + . So, when the operation is INPUT or CONST,if we start with id 𝑣 id 𝑣 ... ... ... ... id 𝑗 − 𝑣 𝑗 − id 𝑗 ... ... ... ... , then we return id 𝑣 id 𝑣 ... ... ... ... id 𝑗 − 𝑣 𝑗 − id 𝑗 𝑣 𝑗 ... ... ... ... . When the operation is INIT_PLUS or INIT_PROD, if we start with id 𝑣 id 𝑣 ... ... ... ... id 𝑗 − 𝑣 𝑗 − id 𝑗 ... ... ... ... , then we return id 𝑣 id 𝑣 ... ... ... ... id 𝑗 − 𝑣 𝑗 − id 𝑗 𝑣 𝑗 id 𝑗 + ... ... ... ... . For the aggregate expression (algorithm 2) we do the following. Let E [ 𝑏 𝑖 , 𝑐 ] = Σ 𝑣. ( 𝑣 𝑇 · 𝑏 𝑖 ) · 𝑐 · 𝑣 · 𝑣 𝑇 + ( − 𝑣 𝑇 · 𝑏 𝑖 ) · 𝑣 · 𝑣 𝑇 , namely, it is the identity with 𝑐 in position ( 𝑖, 𝑖 ) .We define the expressions: AGG_PLUS (algorithm 2, lines 4, 5), AGG_PROD (algorithm 2, lines 6, 7), IS_NOT_LAST (algorithm 2, lines 8,9), IS_LAST and POP: onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč POP : = Iden ( Prev · 𝐺 𝑡𝑜𝑝 ) · 𝐺 + Prev · 𝑉 𝑡𝑜𝑝 · 𝑒 𝑇𝑉 𝑡𝑜𝑝 AGG_PLUS : = isplus (cid:16) 𝐺 𝑇 · (cid:0) 𝑃 · 𝐺 𝑡𝑜𝑝 (cid:1)(cid:17) × (cid:104)(cid:16) Iden ( Prev · 𝑉 𝑡𝑜𝑝 ) · 𝑉 + (cid:16) 𝑉 𝑇 · 𝑉 𝑡𝑜𝑝 (cid:17) (cid:0) Prev · 𝑉 𝑡𝑜𝑝 (cid:1)(cid:17) · 𝑒 𝑇𝑉 (cid:105) AGG_PROD : = isprod (cid:16) 𝐺 𝑇 · (cid:0) 𝑃 · 𝐺 𝑡𝑜𝑝 (cid:1)(cid:17) × (cid:104)(cid:16) E (cid:104) Prev · 𝑉 𝑡𝑜𝑝 , 𝑉 𝑇 · 𝑉 𝑡𝑜𝑝 (cid:105) · Iden ( Prev · 𝑉 𝑡𝑜𝑝 ) · 𝑉 (cid:17) · 𝑒 𝑇𝑉 (cid:105) IS_NOT_LAST : = not_last (cid:16) 𝐺 𝑇 · (cid:0) 𝑃 · 𝐺 𝑡𝑜𝑝 (cid:1) , 𝐺 𝑇 · 𝐺 𝑡𝑜𝑝 (cid:17) × (cid:20) 𝐺 𝑡𝑜𝑝 · next_gate (cid:16) 𝐺 𝑇 · (cid:0) Prev · 𝐺 𝑡𝑜𝑝 (cid:1) , 𝐺 𝑇 · 𝐺 𝑡𝑜𝑝 (cid:17) 𝑇 + 𝐺 𝑡𝑜𝑝 · 𝑒 𝑇𝐺 𝑡𝑜𝑝 (cid:21) IS_LAST : = (cid:16) − not_last (cid:16) 𝐺 𝑇 · (cid:0) 𝑃 · 𝐺 𝑡𝑜𝑝 (cid:1) , 𝐺 𝑇 · 𝐺 𝑡𝑜𝑝 (cid:17)(cid:17) × (cid:104)(cid:0) Prev · 𝐺 𝑡𝑜𝑝 (cid:1) · 𝑒 𝑇𝐺 𝑡𝑜𝑝 (cid:105) The for - MATLANG expression equivalent to algorithm 2 isAGG : = POP + AGG_PLUS + AGG_PROD + IS_NOT_LAST + IS_LAST . The
𝐸𝑣𝑎𝑙𝑢𝑎𝑡𝑒 method (algorithm 3) is defined as follows:EVAL [ 𝑣 ] = 𝑒 𝑇 min · (cid:8) for 𝑋, 𝑣 , . . . , 𝑣 𝑘 . : (cid:16) Π 𝑘𝑖 = min ( 𝑣 𝑖 ) (cid:17) × 𝑆𝑇𝐴𝑅𝑇 + (cid:16) − Π 𝑘𝑖 = min ( 𝑣 𝑖 ) (cid:17) × (cid:16)(cid:0) − min ( 𝐺 𝑡𝑜𝑝 ) · min ( 𝑉 𝑡𝑜𝑝 ) (cid:1) × (cid:104)(cid:16) − 𝐺 𝑇𝑡𝑜𝑝 · 𝑉 𝑡𝑜𝑝 (cid:17) × INIT + (cid:16) 𝐺 𝑇𝑡𝑜𝑝 · 𝑉 𝑡𝑜𝑝 (cid:17) × AGG (cid:105) + min ( 𝐺 𝑡𝑜𝑝 ) × min ( 𝑉 𝑡𝑜𝑝 ) × 𝑋 (cid:17)(cid:9) · 𝑒 𝑉 Note that the for -expression does the evaluation. The final output is in 𝑋 [ , 𝑚𝑎𝑥 − ] , we extract this value by multiplying the finalresult as 𝑒 𝑇 min · [ for ( . . . )] · 𝑒 𝑉 .Finally, we need to take care of all 𝑛 < 𝑛 , where ( ★ ) does not necessarily hold. For any 𝑖 , let:Eval [ 𝑖, 𝐴 ] : = the × matrix with the value of the polynomial Φ 𝑛 ( 𝐴 ) when 𝑛 = 𝑖. Then we define: Φ 𝑛 ( 𝑎 , . . . , 𝑎 𝑖 ) = Σ 𝑛 − 𝑖 = (cid:16) 𝑒 𝑇 min + i · ( 𝑒 diag ( 𝑒 ( 𝑣 )) · 𝑒 max ) (cid:17) × EVAL [ 𝑖, 𝑣 ] + (cid:16) ( Next 𝑛 · 𝑒 min ) 𝑇 · 𝑒 ( 𝑣 ) (cid:17) × EVAL [ 𝑣 ] . Above, ( 𝑒 𝑇 min + i · 𝑒 max ) checks if the dimension is equal to 𝑖 (we multiply by the 𝑛 × 𝑛 identity 𝑒 diag ( 𝑒 ( 𝑣 )) to ensure typing), and ( Next 𝑛 · 𝑒 min ) 𝑇 · 𝑒 ( 𝑒 min ) checks if the dimension is greater or equal than 𝑛 . □ D.3 From MATLANG to uniform ACs
We consider circuits over matrices (multiple output gates). We will write Φ ( 𝐴 , . . . , 𝐴 𝑘 ) , where Φ is an arithmetic circuit with multipleoutput gates, and each 𝐴 𝑖 is a matrix of dimensions 𝛼 𝑖 × 𝛽 𝑖 , with 𝛼 𝑖 , 𝛽 𝑖 ∈ { 𝑛, } to denote the input matrices for a circuit Φ . We will alsowrite type S ( Φ ) = ( 𝛼, 𝛽 ) , with 𝛼, 𝛽 ∈ { 𝑛, } , to denote the size of the output matrix for Φ . When { Φ 𝑛 | 𝑛 = , , . . . } is a uniform family ofarithmetic circuits over matrices, we will assume that the Turing machine for generating Φ 𝑛 also gives us the information about how toaccess a position of each input matrix, and how to access the positions of the output matrix, as is usually done when handling matrices witharithmetic circuits [29]. The notion of degree is extended to be the sum of the degrees of all the output gates. The former will be denotedas Φ 𝑛 [ 𝑖, 𝑗 ] when type S ( Φ ) = ( 𝑛, 𝑛 ) , Φ 𝑛 [ 𝑖, ] when type S ( Φ ) = ( 𝑛, ) , Φ 𝑛 [ , 𝑗 ] when type S ( Φ ) = ( , 𝑛 ) and Φ 𝑛 when type S ( Φ ) = ( , ) .Also, when we write 𝑎 ⊕ 𝑏 we mean + 𝑎 𝑏 When we write (cid:201) 𝑛𝑙 = 𝑎 𝑙 we mean xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA + 𝑎 · · · 𝑎 𝑛 Same with ⊗ . Now we prove the statement.Theorem 5.3. Let 𝑒 be a for - MATLANG expression over a schema S , and let 𝑉 , . . . ,𝑉 𝑘 be the variables of 𝑒 such that type S ( 𝑉 𝑖 ) ∈{( 𝛼, 𝛼 ) , ( 𝛼, ) , ( , 𝛼 ) , ( , )} . Then there exists a uniform arithmetic circuit family over matrices Φ 𝑛 ( 𝐴 , . . . , 𝐴 𝑘 ) such that: • For any instance I = (D , mat ) such that D( 𝛼 ) = 𝑛 and mat ( 𝑉 𝑖 ) = 𝐴 𝑖 it holds that: • ⟦ 𝑒 ⟧(I) = Φ 𝑛 ( 𝐴 , . . . , 𝐴 𝑘 ) . Proof. Let 𝑒 be a for - MATLANG expression.If 𝑒 = 𝑉 then Φ 𝑒𝑛 : = Φ ( 𝐴 ) , and we have that • If type S ( 𝑉 ) = ( , ) then type S ( Φ 𝑒𝑛 ) = ( , ) and Φ 𝑒𝑛 has the one input/output gate. • If type S ( 𝑉 ) = ( , 𝛼 ) then type S ( Φ 𝑒𝑛 ) = ( , 𝑛 ) and Φ 𝑒𝑛 has 𝑛 input/output gates. • If type S ( 𝑉 ) = ( 𝛼, ) then type S ( Φ 𝑒𝑛 ) = ( 𝑛, ) and Φ 𝑒𝑛 has 𝑛 input/output gates. • If type S ( 𝑉 ) = ( 𝛼, 𝛼 ) then type S ( Φ 𝑒𝑛 ) = ( 𝑛, 𝑛 ) and Φ 𝑒𝑛 has 𝑛 input/output gates.If 𝑒 = 𝑒 ′ 𝑇 then Φ 𝑒𝑛 = Φ 𝑒 ′ 𝑛 . • If type S ( Φ 𝑒 ′ 𝑛 ) = ( , ) then Φ 𝑒𝑛 = Φ 𝑒 ′ 𝑛 and type ( Φ 𝑒𝑛 ) = ( , ) . • If type S ( Φ 𝑒 ′ 𝑛 ) = ( , 𝑛 ) then type ( Φ 𝑒𝑛 ) = ( 𝑛, ) and Φ 𝑒𝑛 [ 𝑖, ] : = Φ 𝑒 ′ 𝑛 [ , 𝑖 ] . • If type S ( Φ 𝑒 ′ 𝑛 ) = ( 𝑛, ) then type ( Φ 𝑒𝑛 ) = ( , 𝑛 ) and Φ 𝑒𝑛 [ , 𝑗 ] : = Φ 𝑒 ′ 𝑛 [ 𝑗, ] . • If type S ( Φ 𝑒 ′ 𝑛 ) = ( 𝑛, 𝑛 ) then type ( Φ 𝑒𝑛 ) = ( 𝑛, 𝑛 ) and Φ 𝑒𝑛 [ 𝑖, 𝑗 ] : = Φ 𝑒 ′ 𝑛 [ 𝑗, 𝑖 ] .If 𝑒 = ( 𝑒 ′ ) where type S ( Φ 𝑒 ′ 𝑛 ) = ( 𝛼, 𝛽 ) then type S ( Φ 𝑒𝑛 ) = ( 𝛼, ) and Φ 𝑒𝑛 [ 𝑖, ] : = .If 𝑒 = 𝑒 + 𝑒 we have • When type S ( Φ 𝑒 𝑛 ) = type S ( Φ 𝑒 𝑛 ) = ( , ) then type S ( Φ 𝑒𝑛 ) = ( , ) and Φ 𝑒𝑛 : = Φ 𝑒 𝑛 ⊕ Φ 𝑒 𝑛 . • When type S ( Φ 𝑒 𝑛 ) = type S ( Φ 𝑒 𝑛 ) = ( , 𝑛 ) then type S ( Φ 𝑒𝑛 ) = ( , 𝑛 ) and Φ 𝑒𝑛 [ , 𝑗 ] : = Φ 𝑒 𝑛 [ , 𝑗 ] ⊕ Φ 𝑒 𝑛 [ , 𝑗 ] . • When type S ( Φ 𝑒 𝑛 ) = type S ( Φ 𝑒 𝑛 ) = ( 𝑛, ) then type S ( Φ 𝑒𝑛 ) = ( 𝑛, ) and Φ 𝑒𝑛 [ 𝑖, ] : = Φ 𝑒 𝑛 [ 𝑖, ] ⊕ Φ 𝑒 𝑛 [ 𝑖, ] . • When type S ( Φ 𝑒 𝑛 ) = type S ( Φ 𝑒 𝑛 ) = ( 𝑛, 𝑛 ) then type S ( Φ 𝑒𝑛 ) = ( 𝑛, 𝑛 ) and Φ 𝑒𝑛 [ 𝑖, 𝑗 ] : = Φ 𝑒 𝑛 [ 𝑖, 𝑗 ] ⊕ Φ 𝑒 𝑛 [ 𝑖, 𝑗 ] .If 𝑒 = 𝑓 ( 𝑒 , . . . , 𝑒 𝑘 ) we have two cases • When 𝑓 is the function 𝑓 ⊙ (recall that this function is definable in MATLANG [∅] by Lemma A.1) then – If type S ( Φ 𝑒 𝑛 ) = . . . = type S ( Φ 𝑒 𝑘 𝑛 ) = ( , ) then Φ 𝑒𝑛 : = (cid:203) 𝑘𝑙 = Φ 𝑒 𝑙 𝑛 . – If type S ( Φ 𝑒 𝑛 ) = . . . = type S ( Φ 𝑒 𝑘 𝑛 ) = ( , 𝑛 ) then Φ 𝑒𝑛 [ , 𝑗 ] : = (cid:203) 𝑘𝑙 = Φ 𝑒 𝑙 𝑛 [ , 𝑗 ] . – If type S ( Φ 𝑒 𝑛 ) = . . . = type S ( Φ 𝑒 𝑘 𝑛 ) = ( 𝑛, ) then Φ 𝑒𝑛 [ 𝑖, ] : = (cid:203) 𝑘𝑙 = Φ 𝑒 𝑙 𝑛 [ 𝑖, ] . – If type S ( Φ 𝑒 𝑛 ) = . . . = type S ( Φ 𝑒 𝑘 𝑛 ) = ( 𝑛, 𝑛 ) then Φ 𝑒𝑛 [ 𝑖, 𝑗 ] : = (cid:203) 𝑘𝑙 = Φ 𝑒 𝑙 𝑛 [ 𝑖, 𝑗 ] . • When 𝑓 is any function, we prove the case when type S ( Φ 𝑒 𝑛 ) = . . . = type S ( Φ 𝑒 𝑘 𝑛 ) = ( , ) (only case necessary, as discussed inAppendix A.2). Here Φ 𝑒𝑛 is 𝑓 Φ 𝑒 𝑛 · · · Φ 𝑒 𝑘 𝑛 Note that since for the context of this result we only consider for - MATLANG = MATLANG [∅] , this case is not strictly necessary, modulofor 𝑓 ⊙ , 𝑓 ⊕ due to Lemma A.1. However, if we extend the circuits with the same functions allowed in for - MATLANG , then our inductiveconstruction still goes through, as just illustrated.If 𝑒 = 𝑒 · 𝑒 we have • When type S ( Φ 𝑒 𝑛 ) = ( , ) and type S ( Φ 𝑒 𝑛 ) = ( , ) then type S ( Φ 𝑒𝑛 ) = ( , ) and Φ 𝑒𝑛 : = Φ 𝑒 𝑛 ⊗ Φ 𝑒 𝑛 . • When type S ( Φ 𝑒 𝑛 ) = ( , ) and type S ( Φ 𝑒 𝑛 ) = ( , 𝑛 ) then type S ( Φ 𝑒𝑛 ) = ( , 𝑛 ) and Φ 𝑒𝑛 [ , 𝑗 ] : = Φ 𝑒 𝑛 ⊗ Φ 𝑒 𝑛 [ , 𝑗 ] . • When type S ( Φ 𝑒 𝑛 ) = ( 𝑛, ) and type S ( Φ 𝑒 𝑛 ) = ( , ) then type S ( Φ 𝑒𝑛 ) = ( 𝑛, ) and Φ 𝑒𝑛 [ 𝑖, ] : = Φ 𝑒 𝑛 [ 𝑖, ] ⊗ Φ 𝑒 𝑛 . • When type S ( Φ 𝑒 𝑛 ) = ( 𝑛, ) and type S ( Φ 𝑒 𝑛 ) = ( , 𝑛 ) then type S ( Φ 𝑒𝑛 ) = ( 𝑛, 𝑛 ) and Φ 𝑒𝑛 [ 𝑖, 𝑗 ] : = Φ 𝑒 𝑛 [ 𝑖, ] ⊗ Φ 𝑒 𝑛 [ , 𝑗 ] . onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč • When type S ( Φ 𝑒 𝑛 ) = ( , 𝑛 ) and type S ( Φ 𝑒 𝑛 ) = ( 𝑛, ) then type S ( Φ 𝑒𝑛 ) = ( , ) and Φ 𝑒𝑛 : = 𝑛 (cid:202) 𝑘 = (cid:0) Φ 𝑒 𝑛 [ , 𝑘 ] ⊗ Φ 𝑒 𝑛 [ 𝑘, ] (cid:1) . • When type S ( Φ 𝑒 𝑛 ) = ( , 𝑛 ) and type S ( Φ 𝑒 𝑛 ) = ( 𝑛, 𝑛 ) then type S ( Φ 𝑒𝑛 ) = ( , 𝑛 ) and Φ 𝑒𝑛 [ , 𝑗 ] : = 𝑛 (cid:202) 𝑘 = (cid:0) Φ 𝑒 𝑛 [ , 𝑘 ] ⊗ Φ 𝑒 𝑛 [ 𝑘, 𝑗 ] (cid:1) . • When type S ( Φ 𝑒 𝑛 ) = ( 𝑛, 𝑛 ) and type S ( Φ 𝑒 𝑛 ) = ( 𝑛, ) then type S ( Φ 𝑒𝑛 ) = ( 𝑛, ) and Φ 𝑒𝑛 [ 𝑖, ] : = 𝑛 (cid:202) 𝑘 = (cid:0) Φ 𝑒 𝑛 [ 𝑖, 𝑘 ] ⊗ Φ 𝑒 𝑛 [ 𝑘, ] (cid:1) . • When type S ( Φ 𝑒 𝑛 ) = ( 𝑛, 𝑛 ) and type S ( Φ 𝑒 𝑛 ) = ( 𝑛, 𝑛 ) then type S ( Φ 𝑒𝑛 ) = ( 𝑛, 𝑛 ) and Φ 𝑒𝑛 [ 𝑖, 𝑗 ] : = 𝑛 (cid:202) 𝑘 = (cid:0) Φ 𝑒 𝑛 [ 𝑖, 𝑘 ] ⊗ Φ 𝑒 𝑛 [ 𝑘, 𝑗 ] (cid:1) . If 𝑒 = for 𝑋, 𝑣 . 𝑒 ′ ( 𝑋, 𝑣 ) , then define Φ as the zero matrix circuit type S ( Φ ) = ( , ) if type S ( Φ 𝑒 ′ 𝑛 ) = ( , ) and type S ( Φ ) = ( 𝑛, 𝑛 ) if type S ( Φ 𝑒 ′ 𝑛 ) = ( 𝑛, 𝑛 ) . Also, Φ = and Φ [ 𝑖, 𝑗 ] = ∀ 𝑖, 𝑗 for each case respectively. Now for 𝑖 = , . . . , 𝑛 , define Φ 𝑣 𝑖 as the circuit such that type S ( Φ 𝑣 𝑖 ) = ( 𝑛, ) and Φ 𝑣 𝑖 [ 𝑖, ] : = and zero otherwise. Finally, define Φ 𝑒𝑛 = Φ 𝑒 ′ 𝑛 (cid:16) Φ 𝑒 ′ 𝑛 (cid:16) · · · (cid:16) Φ 𝑒 ′ 𝑛 (cid:16) Φ , Φ 𝑣 (cid:17) , Φ 𝑣 (cid:17) · · · , Φ 𝑣 𝑛 − (cid:17) , Φ 𝑣 𝑛 (cid:17) . Note that every circuit adds a constant number of layers except when 𝑒 = for 𝑋, 𝑣 . 𝑒 ′ ( 𝑋, 𝑣 ) . This means that the depth still is polynomial.When 𝑒 = for 𝑋, 𝑣 . 𝑒 ′ ( 𝑋, 𝑣 ) we have that the depth of the circuit is 𝑛 · 𝑝 ( 𝑛 ) , where the depth of 𝑒 ′ ( 𝑋, 𝑣 ) is 𝑝 ( 𝑛 ) , so it also remains polynomial.Here, we do not need to translate scalar multiplication because it can be simulated using the ones operator and 𝑓 ⊙ (see section A.2).Finally, we remark that when composing the circuits the fact that uniformity is preserved (i.e. the resulting circuit can be generatedby a LOGSPACE machine) is proved analogously as when composing two LOGSPACE transducers [3]. The only more involved case istreating for-loop construction, however, notice here that we only need to keep track of where we are in the evaluation (i.e. which 𝑣 𝑖 we areprocessing), and not of all the previous results, given that they update the resulting matrix in a fixed order. □ D.4 Undecidability
Let 𝑒 be a for - MATLANG expression over a matrix schema S = (M , size ) and let 𝑉 , . . . ,𝑉 𝑘 be the variables of 𝑒 , each of type ( 𝛼, 𝛼 ) , ( , 𝛼 ) , ( 𝛼, ) or ( , ) . We know from Theorem 5.3 that there exists a uniform arithmetic circuit family { Φ 𝑛 | 𝑛 = , , . . . } such that ⟦ 𝑒 ⟧(I) = Φ 𝑛 ( 𝐴 , . . . , 𝐴 𝑘 ) for any instance I such that D( 𝛼 ) = 𝑛 and mat ( 𝑉 𝑖 ) = 𝐴 𝑖 for 𝑖 = , . . . , 𝑘 . We are interested in deciding whetherthere exists such a uniform arithmetic circuit family { Φ 𝑛 | 𝑛 = , , . . . } of polynomial degree, i.e., such that degree ( Φ 𝑛 ) = O( 𝑝 ( 𝑛 )) for somepolynomial 𝑝 ( 𝑥 ) . If such a circuit family exists, we call 𝑒 of polynomial degree.Proposition 5.5. Given a for - MATLANG expression 𝑒 over a schema S , it is undecidable to check whether 𝑒 is of polynomial degree. Proof. We show undecidability based on the following undecidable language: {⟨ 𝑀 ⟩ | 𝑀 is a deterministic TM which halts on the empty input } , where ⟨ 𝑀 ⟩ is some string encoding of 𝑀 . Consider a TM 𝑀 described by ( 𝑄, Γ = { , } , 𝑞 , 𝑞 𝑚 , Δ ) with 𝑄 = { 𝑞 , . . . , 𝑞 𝑚 } its states, 𝑞 beingthe initial state and 𝑞 𝑚 being the halting state, Γ is the tape alphabet, and Δ is a transition function from 𝑄 × Γ → 𝑄 × Γ × {← , ⊔ , →} . Thesimulation of linear space TM, as given in the proof of Proposition D.1 can be easily modified to any TM 𝑀 provided that we limit theexecution of 𝑀 to exactly 𝑛 steps. Let 𝑒 𝑀 denote this expression. Similarly as in the linear space TM simulation, we have vector variables 𝑄 , . . . , 𝑄 𝑚 encoding the states, a single relation 𝑇 encoding the tape and relation 𝐻 𝑇 encoding the position of the tape. When an instance I assigns 𝑛 to 𝛼 , we have a tape of length 𝑛 at our disposal. This suffices if we let 𝑀 run for 𝑛 steps. We further observe that all vector variablescan be assumed to be zero, initially. This is because we do not have input. So, let I 𝑛 denote the instance which assigns vector variablesto the 𝑛 -dimensional zero vector. Furthermore, by contrast to the linear space TM simulation, we use a single vector 𝑣 (instead of 𝑘 suchvectors) to simulate 𝑛 steps of 𝑀 . Finally, we modify the expression given in the proof of Proposition D.1 such ⟦ 𝑒 𝑀 ⟧(I 𝑛 ) returns if 𝑀 halts in at most 𝑛 steps, and if 𝑀 did not halt yet after 𝑛 steps.As a consequence, when 𝑀 does not halt, ⟦ 𝑒 𝑀 ⟧(I 𝑛 ) = for all 𝑛 ≥ . When 𝑀 halts, there will be an 𝑛 such that ⟦ 𝑒 𝑀 ⟧(I 𝑛 ) = It nowsuffices to consider the for - MATLANG expression 𝑑 𝑀 : = 𝑒 𝑀 · 𝑒 exp xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA where 𝑒 exp = for 𝑣, 𝑋 = ( 𝑋 ) 𝑇 · ( 𝑋 ) . 𝑋 · 𝑋 such that 𝑒 exp (I 𝑛 ) = 𝑛 𝑛 . Then, when 𝑀 does not halt we can clearly compute 𝑑 𝑀 witha constant degree circuit “0” for any 𝑛 , otherwise, the circuit needed will be of exponential degree for at least one 𝑛 , simply because nopolynomial degree uniform circuit family can compute 𝑛 𝑛 . In other words, deciding whether 𝑑 𝑀 has polynomial degree coincides withdeciding whether 𝑀 halts. □ E PROOFS OF SECTION 6E.1 From sum - MATLANG to RA + 𝐾 We prove proposition 6.3.Proposition 6.3.
For each sum - MATLANG expression 𝑒 over schema S such that S( 𝑒 ) = ( 𝛼, 𝛽 ) with 𝛼 ≠ ≠ 𝛽 , there exists a RA + 𝐾 expression 𝑄 ( 𝑒 ) over relational schema Rel (S) such that Rel (S)( 𝑄 ( 𝑒 )) = { row 𝛼 , row 𝛽 } and such that for any instance I over S , ⟦ 𝑒 ⟧(I) 𝑖,𝑗 = ⟦ 𝑄 ( 𝑒 )⟧ Rel (I) ( 𝑡 ) for tuple 𝑡 ( row 𝛼 ) = 𝑖 and 𝑡 ( col 𝛽 ) = 𝑗 . Similarly for when 𝑒 has schema S( 𝑒 ) = ( 𝛼, ) , S( 𝑒 ) = ( , 𝛽 ) or S( 𝑒 ) = ( , ) , then 𝑄 ( 𝑒 ) has schemaRel (S)( 𝑄 ( 𝑒 )) = { row 𝛼 } , Rel (S)( 𝑄 ( 𝑒 )) = { col 𝛼 } , or Rel (S)( 𝑄 ( 𝑒 )) = {} , respectively. Proof. We start from a matrix schema S = (M , size ) , where M ⊂ V is a finite set of matrix variables, and size : V ↦→
Symb × Symb isa function that maps each matrix variable to a pair of size symbols. On the relational side we have for each size symbol 𝛼 ∈ Symb \ { } ,attributes 𝛼 , row 𝛼 , and col 𝛼 in A . We also reserve some special attributes 𝛾 , 𝛾 , . . . whose role will become clear shortly. For each 𝑉 ∈ M and 𝛼 ∈ Symb we denote by 𝑅 𝑉 and 𝑅 𝛼 its corresponding relation name, respectively.Then, given S we define the relational schema Rel (S) such that dom ( Rel (S)) = { 𝑅 𝛼 | 𝛼 ∈ Symb } ∪ { 𝑅 𝑉 | 𝑉 ∈ M} where Rel (S)( 𝑅 𝛼 ) = { 𝛼 } and for all 𝑉 ∈ M : Rel (S)( 𝑅 𝑉 ) = { row 𝛼 , col 𝛽 } if size ( 𝑉 ) = ( 𝛼, 𝛽 ){ row 𝛼 } if size ( 𝑉 ) = ( 𝛼, ){ col 𝛽 } if size ( 𝑉 ) = ( , 𝛽 ){} if size ( 𝑉 ) = ( , ) . Next, for a matrix instance I = (D , mat ) over S , let 𝑉 ∈ M with size ( 𝑉 ) = ( 𝛼, 𝛽 ) and let mat ( 𝑉 ) be its corresponding 𝐾 -matrix ofdimension D( 𝛼 ) × D( 𝛽 ) . The 𝐾 -instance in RA + 𝐾 according to I is Rel (I) with data domain D = N \ { } . For each 𝑉 ∈ M we define 𝑅 Rel (I) 𝑉 ( 𝑡 ) : = mat ( 𝑉 ) 𝑖 𝑗 whenever 𝑡 ( row 𝛼 ) = 𝑖 ≤ D( 𝛼 ) and 𝑡 ( col 𝛽 ) = 𝑗 ≤ D( 𝛽 ) , and otherwise. Also, for each 𝛼 ∈ Symb we define 𝑅 Rel (I) 𝛼 ( 𝑡 ) : = whenever 𝑡 ( 𝛼 ) ≤ D( 𝛼 ) , and otherwise. If size ( 𝑉 ) = ( 𝛼, ) then 𝑅 Rel (I) 𝑉 ( 𝑡 ) : = mat ( 𝑉 ) 𝑖 whenever 𝑡 ( row 𝛼 ) = 𝑖 ≤ D( 𝛼 ) and otherwise. Similarly, if size ( 𝑉 ) = ( , 𝛽 ) then 𝑅 Rel (I) 𝑉 ( 𝑡 ) : = mat ( 𝑉 ) 𝑗 whenever 𝑡 ( col 𝛽 ) = 𝑗 ≤ D( 𝛽 ) and otherwise. If size ( 𝑉 ) = ( , ) then 𝑅 Rel (I) 𝑉 (()) : = mat ( 𝑉 ) .Let 𝑒 be a sum - MATLANG expression. In the following we need to distinguish between matrix variables 𝑣 that occur in 𝑒 as part of asub-expression Σ 𝑣. (·) , i.e., those variables that will be used to iterate over by means of canonical vectors, and those that are not. To makethis distinction clear, we use 𝑣 , 𝑣 , . . . for those “iterator” variables, and capital 𝑉 for the other variables occurring in 𝑒 . For simplicity, weassume that each occurrence of Σ has its own iterator variable associated with it.We define free (iterator) variables, as follows. free ( 𝑉 ) : = ∅ , free ( 𝑣 ) : = { 𝑣 } , free ( 𝑒 𝑇 ) : = free ( 𝑒 ) , free ( 𝑒 + 𝑒 ) : = free ( 𝑒 ) ∪ free ( 𝑒 ) , free ( 𝑒 · 𝑒 ) : = free ( 𝑒 ) ∪ free ( 𝑒 ) , free ( 𝑓 ⊙ ( 𝑒 , . . . , 𝑒 𝑘 )) : = free ( 𝑒 ) ∪ · · · ∪ free ( 𝑒 𝑘 ) , and free ( 𝑒 = Σ 𝑉 .𝑒 ) = free ( 𝑒 ) \ { 𝑣 } . We will explicitlydenote the free variables in an expression 𝑒 by writing 𝑒 ( 𝑣 , . . . , 𝑣 𝑘 ) .We now use the following induction hypotheses: • If 𝑒 ( 𝑣 , . . . , 𝑣 𝑘 ) is of type ( 𝛼, 𝛽 ) then there exists a RA + 𝐾 expression 𝑄 such that Rel (S)( 𝑄 ( 𝑒 )) = { row 𝛼 , col 𝛽 , 𝛾 , . . . ,𝛾 𝑘 } and such that ⟦ 𝑄 ( 𝑒 )⟧ Rel (I) ( 𝑡 ) = ⟦ 𝑒 ⟧(I [ 𝑣 ← 𝑏 𝑖 , . . . , 𝑣 𝑘 ← 𝑏 𝑖 𝑘 ]) 𝑖,𝑗 for tuple 𝑡 ( row 𝛼 ) = 𝑖 , 𝑡 ( col 𝛽 ) = 𝑗 and 𝑡 ( 𝛾 𝑠 ) = 𝑖 𝑠 for 𝑠 = , . . . , 𝑘 . • If 𝑒 ( 𝑣 , . . . , 𝑣 𝑘 ) is of type ( 𝛼, ) then there exists a RA + 𝐾 expression 𝑄 such that Rel (S)( 𝑄 ( 𝑒 )) = { row 𝛼 , 𝛾 , . . . ,𝛾 𝑘 } and such that ⟦ 𝑄 ( 𝑒 )⟧ Rel (I) ( 𝑡 ) = ⟦ 𝑒 ⟧(I [ 𝑣 ← 𝑏 𝑖 , . . . , 𝑣 𝑘 ← 𝑏 𝑖 𝑘 ]) 𝑖, for tuple 𝑡 ( row 𝛼 ) = 𝑖 , and 𝑡 ( 𝛾 𝑠 ) = 𝑖 𝑠 for 𝑠 = , . . . , 𝑘 . And similarly for when 𝑒 is type ( , 𝛽 ) . • If 𝑒 ( 𝑣 , . . . , 𝑣 𝑘 ) is of type ( , ) then there exists a RA + 𝐾 expression 𝑄 such that Rel (S)( 𝑄 ( 𝑒 )) = { 𝛾 , . . . ,𝛾 𝑘 } and such that ⟦ 𝑄 ( 𝑒 )⟧ Rel (I) ( 𝑡 ) = ⟦ 𝑒 ⟧(I [ 𝑣 ← 𝑏 𝑖 , . . . , 𝑣 𝑘 ← 𝑏 𝑖 𝑘 ]) , for tuple 𝑡 ( 𝛾 𝑠 ) = 𝑖 𝑠 for 𝑠 = , . . . , 𝑘 .Clearly, this suffices to show the proposition since we there consider expressions 𝑒 for which free ( 𝑒 ) = ∅ , in which case the above statementsreduce to the one given in the proposition. onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč The proof is by induction on the structure of sum - MATLANG expressions. In line with the simplifications in Section A.2, it suffices toconsider pointwise function application with 𝑓 ⊙ instead of scalar multiplication. (We also note that we can express the one-vector operatorin sum - MATLANG , so scalar multiplication can be expressed using 𝑓 ⊙ in sum - MATLANG ).Let 𝑒 be a sum - MATLANG expression. • If 𝑒 = 𝑉 then 𝑄 ( 𝑒 ) : = 𝑅 𝑉 . • If 𝑒 = 𝑣 𝑝 then 𝑄 ( 𝑒 ) : = 𝜎 { row 𝛼 ,𝛾 𝑝 } (cid:0) 𝜌 row 𝛼 → 𝛼 ( 𝑅 𝛼 ) ⊲⊳ 𝜌 𝛾 𝑝 → 𝛼 ( 𝑅 𝛼 ) (cid:1) when 𝑣 𝑝 is of type ( 𝛼, ) . It is here that we introduce the attribute 𝛾 𝑝 associated with iterator variable 𝑣 𝑝 . We note that ⟦ 𝑄 ( 𝑣 𝑝 )⟧ Rel (I) ( 𝑡 ) = ⟦ 𝑣 𝑝 ⟧(I [ 𝑣 𝑝 ← 𝑏 𝑗 ]) 𝑖, = ( 𝑏 𝑗 ) 𝑖, for 𝑡 ( row 𝛼 ) = 𝑖 and 𝑡 [ 𝛾 𝑝 ] = 𝑗 . Indeed, ( 𝑏 𝑗 ) 𝑖, = if 𝑗 = 𝑖 and this holds when 𝑡 ( row 𝛼 ) = 𝑡 [ 𝛾 𝑝 ] = 𝑗 , and ( 𝑏 𝑗 ) 𝑖, = if 𝑗 ≠ 𝑖 and thisalso holds when 𝑡 ( row 𝛼 ) ≠ 𝑡 [ 𝛾 𝑝 ] = 𝑗 . • If 𝑒 ( 𝑣 , . . . , 𝑣 𝑘 ) = ( 𝑒 ( 𝑣 , . . . , 𝑣 𝑘 )) 𝑇 with S( 𝑒 ) = ( 𝛼, 𝛽 ) then 𝑄 ( 𝑒 ) : = 𝜌 row 𝛼 → col 𝛼 , col 𝛽 → row 𝛽 (cid:0) 𝑄 ( 𝑒 ) (cid:1) if 𝛼 ≠ ≠ 𝛽 ; 𝜌 row 𝛼 → col 𝛼 (cid:0) 𝑄 ( 𝑒 ) (cid:1) if 𝛼 ≠ = 𝛽 ; 𝜌 col 𝛽 → row 𝛽 (cid:0) 𝑄 ( 𝑒 ) (cid:1) if 𝛼 = ≠ 𝛽 ; 𝑄 ( 𝑒 ) if 𝛼 = = 𝛽. • If 𝑒 = 𝑒 ( 𝑣 , . . . , 𝑣 𝑘 ) + 𝑒 ( 𝑣 , . . . , 𝑣 𝑘 ) with S( 𝑒 ) = S( 𝑒 ) = ( 𝛼, 𝛽 ) then 𝑄 ( 𝑒 ) : = 𝑄 ( 𝑒 ) ∪ 𝑄 ( 𝑒 ) . We assume here that 𝑒 and 𝑒 havethe same free variables. This is without loss of generality. Indeed, as an example, suppose that we have 𝑒 ( 𝑣 , 𝑣 ) and 𝑒 ( 𝑣 , 𝑣 ) . Then,we can replace 𝑒 by 𝑒 ( 𝑣 , 𝑣 , 𝑣 ) = ( 𝑣 𝑇 · 𝑣 ) × 𝑒 ( 𝑣 , 𝑣 ) and similarly, 𝑒 by 𝑒 ( 𝑣 , 𝑣 , 𝑣 ) = ( 𝑣 𝑇 · 𝑣 ) × 𝑒 ( 𝑣 , 𝑣 ) , where in additionwe replace scalar multiplication with its simulation using 𝑓 ⊙ and the ones vector, as mentioned earlier. • If 𝑒 = 𝑓 ⊙ ( 𝑒 , . . . , 𝑒 𝑘 ) with S( 𝑒 𝑖 ) = S( 𝑒 𝑗 ) for all 𝑖, 𝑗 ∈ [ , 𝑘 ] , then 𝑄 ( 𝑒 ) : = 𝑄 ( 𝑒 ) (cid:90) · · · (cid:90) 𝑄 ( 𝑒 𝑘 ) . • If 𝑒 = 𝑒 · 𝑒 with S( 𝑒 ) = ( 𝛼,𝛾 ) and S( 𝑒 ) = ( 𝛾, 𝛽 ) , we have two cases. If 𝛾 = then 𝑄 ( 𝑒 ) : = 𝑄 ( 𝑒 ) (cid:90) 𝑄 ( 𝑒 ) . If 𝛾 ≠ then 𝑄 ( 𝑒 ) : = 𝜋 { row 𝛼 , col 𝛽 ,𝛾 ,...,𝛾 𝑘 } (cid:16) 𝜌 𝐶 → col 𝛾 ( 𝑄 ( 𝑒 )) (cid:90) 𝜌 𝐶 → row 𝛾 ( 𝑄 ( 𝑒 )) (cid:17) , when Rel (S)( 𝑄 ( 𝑒 )) = { row 𝛼 , col 𝛾 , 𝛾 ′ , . . . ,𝛾 ′ ℓ } , Rel (S)( 𝑄 ( 𝑒 )) = { row 𝛾 , col 𝛽 , 𝛾 ′′ , . . . ,𝛾 ′′ ℓ } and { 𝛾 , . . . ,𝛾 𝑘 } = { 𝛾 ′ , . . . ,𝛾 ′ 𝑘 , 𝛾 ′′ , . . . ,𝛾 ′′ 𝑚 } . • If 𝑒 ( 𝑣 , . . . , 𝑣 𝑝 − , 𝑣 𝑝 + , . . . , 𝑣 𝑘 ) = Σ 𝑣 𝑝 .𝑒 ( 𝑣 , . . . , 𝑣 𝑘 ) where S( 𝑒 ) = ( 𝛼, 𝛽 ) and S( 𝑉 ) = ( 𝛾, ) . Then we do 𝑄 ( 𝑒 ) : = 𝜋 Rel (S) ( 𝑄 ( 𝑒 ))\{ 𝛾 𝑝 } 𝑄 ( 𝑒 ) . Indeed, by induction we know that ⟦ 𝑄 ( 𝑒 )⟧ Rel (I) ( 𝑡 ) = ⟦ 𝑒 ⟧(I [ 𝑣 ← 𝑏 𝑖 , . . . , 𝑣 𝑘 ← 𝑏 𝑖 𝑘 ]) 𝑖,𝑗 for tuple 𝑡 ( row 𝛼 ) = 𝑖 , 𝑡 ( col 𝛽 ) = 𝑗 and 𝑡 ( 𝛾 𝑠 ) = 𝑖 𝑠 for 𝑠 = , . . . , 𝑘 . Hence, for 𝑡 ( row 𝛼 ) = 𝑖 , 𝑡 ( col 𝛽 ) = 𝑗 and 𝑡 ( 𝛾 𝑠 ) = 𝑖 𝑠 for 𝑠 = , . . . , 𝑘 and 𝑠 ≠ 𝑝 , ⟦ 𝑄 ( 𝑒 )⟧ Rel (I) ( 𝑡 ) : = (cid:202) 𝑖 𝑝 = ,..., D ( 𝛾 ) ⟦ 𝑒 ⟧(I [ 𝑣 ← 𝑏 𝑖 , . . . , 𝑣 𝑘 ← 𝑏 𝑖 𝑘 ]) 𝑖,𝑗 , which precisely corresponds to ⟦ Σ 𝑣 𝑝 .𝑒 ( 𝑣 , . . . , 𝑣 𝑘 )⟧(I [ 𝑣 ← 𝑏 𝑖 , . . . , 𝑣 𝑝 − ← 𝑏 𝑝 − , 𝑣 𝑝 + ← 𝑏 𝑝 + , . . . , 𝑣 𝑘 ← 𝑏 𝑘 ]) 𝑖,𝑗 . All other cases, when expressions have type ( 𝛼, ) , ( , 𝛽 ) or ( , ) can be dealt with in a similar way. □ E.2 From RA + 𝐾 to sum - MATLANG
Let R be binary relational schema. For each 𝑅 ∈ R we associate a matrix variable 𝑉 𝑅 such that, if 𝑅 is a binary relational signature, then 𝑉 𝑅 represents a (square) matrix, if 𝑅 is unary, then 𝑉 𝑅 represents a vector and if | 𝑅 | = then 𝑉 𝑅 represents a constant. Formally, fix a symbol 𝛼 ∈ Symb \ { } . Let Mat (R) denote the MATLANG schema (M R , size R ) such that M R = { 𝑉 𝑅 | 𝑅 ∈ R} and size R ( 𝑉 𝑅 ) = ( 𝛼, 𝛼 ) whenever | 𝑅 | = , size R ( 𝑉 𝑅 ) = ( 𝛼, ) whenever | 𝑅 | = and size R ( 𝑉 𝑅 ) = ( , ) whenever | 𝑅 | = . Let J be the 𝐾 -instance of R and suppose that adom (J ) = { 𝑑 , . . . , 𝑑 𝑛 } is the active domain (with arbitrary order) of J . Define the matrix instance Mat (J ) = (D J , mat J ) such that D J ( 𝛼 ) = 𝑛 , mat J ( 𝑉 𝑅 ) 𝑖,𝑗 = 𝑅 J (( 𝑑 𝑖 , 𝑑 𝑗 )) whenever | 𝑅 | = , mat J ( 𝑉 𝑅 ) 𝑖 = 𝑅 J (( 𝑑 𝑖 )) whenever | 𝑅 | = , and mat J ( 𝑉 𝑅 ) , = 𝑅 J whenever | 𝑅 | = . Note that we consider the active domain of the whole 𝐾 -instance.We next translate RA + 𝐾 expressions in to sum - MATLANG expressions over an extended schema. More specifically, for each attribute 𝐴 ∈ A we define a vector variable 𝑣 𝐴 of type ( 𝛼, ) . Then for each RA + 𝐾 expression 𝑄 with attributes 𝐴 , . . . , 𝐴 𝑘 we define a sum - MATLANG expression 𝑒 𝑄 ( 𝑣 𝐴 , . . . , 𝑣 𝐴 𝑘 ) of type ( , ) such that the following inductive hypothesis holds: ⟦ 𝑒 𝑄 ⟧( Mat (J )[ 𝑣 𝐴 ← 𝑏 𝑖 , . . . , 𝑣 𝐴 𝑘 ← 𝑏 𝑖 𝑘 ]) = ⟦ 𝑄 ⟧ J ( 𝑡 ) (∗) where 𝑡 ( 𝐴 𝑠 ) = 𝑖 𝑠 for 𝑠 = , . . . , 𝑘 . The proof of this claim follows by induction on the structure of expressions: • If 𝑄 = 𝑅 , then 𝑒 𝑄 : = 𝑣 𝑇𝐴 · 𝑉 𝑅 · 𝑣 𝐴 if R( 𝑅 ) = { 𝐴 , 𝐴 } with 𝐴 < 𝐴 ; 𝑒 𝑄 : = 𝑉 𝑇𝑅 · 𝑣 𝐴 if R( 𝑅 ) = { 𝐴 } ; and 𝑒 𝑄 : = 𝑉 𝑅 if R( 𝑅 ) = {} . xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA • If 𝑄 = 𝑄 ∪ 𝑄 then 𝑒 𝑄 : = 𝑒 𝑄 + 𝑒 𝑄 . • If 𝑄 = 𝜋 𝑌 ( 𝑄 ) for 𝑌 ⊆ R( 𝑄 ) and { 𝐵 , . . . , 𝐵 𝑙 } = R( 𝑄 ) \ 𝑌 then 𝑒 𝑄 : = Σ 𝑣 𝐵 . Σ 𝑣 𝐵 . . . . Σ 𝑣 𝐵 𝑙 . 𝑒 𝑄 • If 𝑄 = 𝜎 𝑌 ( 𝑄 ) with 𝑌 ⊆ R( 𝑄 ) then 𝑒 𝑄 : = 𝑒 𝑄 · (cid:214) 𝐴,𝐵 ∈ 𝑌 ( 𝑣 𝑇𝐴 · 𝑣 𝐵 ) . Here Π is the matrix multiplication of expressions of type ( , ) . • If 𝑄 = 𝜌 𝑋 ↦→ 𝑌 ( 𝑄 ) then 𝑒 𝑄 : = 𝑒 𝑄 [ 𝑣 𝐵 ← 𝑣 𝐴 | 𝐴 ∈ 𝑋, 𝐵 ∈ 𝑌, 𝐴 ↦→ 𝐵 ] . In other words, we rename variable 𝑣 𝐵 with variable 𝑣 𝐵 in all the expression 𝑒 𝑄 . • If 𝑄 = 𝑄 ⊲⊳ 𝑄 then 𝑒 𝑄 : = 𝑒 𝑄 · 𝑒 𝑄 where the product is over expression of type ( , ) .One can check, by induction over the construction, that the inductive hypothesis (∗) holds in each case. Now we can obtain proposition 6.4.Proposition 6.4. Let R be a binary relational schema. For each RA + 𝐾 expression 𝑄 over R such that |R( 𝑄 )| = , there exists a sum - MATLANG expression Ψ ( 𝑄 ) over MATLANG schema Mat (R) such that for any 𝐾 -instance J with adom (J ) = { 𝑑 , . . . , 𝑑 𝑛 } over R , ⟦ 𝑄 ⟧ J (( 𝑑 𝑖 , 𝑑 𝑗 )) = ⟦ Ψ ( 𝑄 )⟧( Mat (J )) 𝑖,𝑗 . Similarly for when |R( 𝑄 )| = , or |R( 𝑄 )| = respectively. Proof. As a consequence of the previous discussion above, when 𝑄 is a RA + 𝐾 expression such that R( 𝑄 ) = { 𝐴 , 𝐴 } with 𝐴 < 𝐴 thenwe define Ψ ( 𝑄 ) = Σ 𝑣 𝐴 . Σ 𝑣 𝐴 . 𝑒 𝑄 · ( 𝑣 𝐴 · 𝑣 𝑇𝐴 ) . Instead, when R( 𝑄 ) = { 𝐴 } we have Ψ ( 𝑄 ) = Σ 𝑣 𝐴 . ( 𝑣 𝐴 · 𝑒 𝑄 ) . And when R( 𝑄 ) = {} we have Ψ ( 𝑄 ) = 𝑒 𝑄 . By using the inductive hypothesis (∗) one can check that Ψ ( 𝑄 ) works in each case as expected. □ E.3 Weighted logics and FO - MATLANG
We prove proposition 6.7:Proposition 6.7.
Weighted logics over Γ and FO - MATLANG over S have the same expressive power. More specifically, • for each FO - MATLANG expression 𝑒 over S such that S( 𝑒 ) = ( , ) , there exists a WL-formula Φ ( 𝑒 ) over WL (S) such that for every instance I of S , ⟦ 𝑒 ⟧(I) = ⟦ Φ ( 𝑒 )⟧ WL (I) . • for each WL-formula 𝜑 over Γ without free variables, there exists a FO - MATLANG expression Ψ ( 𝜑 ) such that for any structure A over Mat ( Γ ) , ⟦ 𝜑 ⟧ A = ⟦ Ψ ( 𝜑 )⟧( Mat (A)) . Proof. Both directions are proved by induction on the structure of expressions. ( FO - MATLANG to WL)
First, let S = (M , size ) be a schema of square matrices, that is, there exists an 𝛼 such that size ( 𝑉 ) ∈ { , 𝛼 } × { , 𝛼 } for every 𝑉 ∈ M . Define the relational vocabulary WL (S) = { 𝑅 𝑉 | 𝑉 ∈ M} such that arity ( 𝑅 𝑉 ) = if size ( 𝑉 ) = ( 𝛼, 𝛼 ) , arity ( 𝑅 𝑉 ) = if size ( 𝑉 ) ∈ {( 𝛼, ) , ( , 𝛼 )} , and arity ( 𝑅 𝑉 ) = otherwise. Then given a matrix instance I = (D , mat ) over S with D( 𝛼 ) = 𝑛 define thestructure WL (I) = ({ , . . . , 𝑛 } , { 𝑅 I 𝑉 }) such that 𝑅 I 𝑉 ( 𝑖, 𝑗 ) = mat ( 𝑉 ) 𝑖,𝑗 if size ( 𝑉 ) = ( 𝛼, 𝛼 ) , 𝑅 I 𝑉 ( 𝑖 ) = mat ( 𝑉 ) 𝑖 if size ( 𝑉 ) ∈ {( 𝛼, ) , ( , 𝛼 )} , and 𝑅 I 𝑉 = mat ( 𝑉 ) if size ( 𝑉 ) = ( , ) .Similar to the proof of Proposition 6.3, for each expression 𝑒 ( 𝑣 , . . . , 𝑣 𝑘 ) of type ( 𝛼, 𝛼 ) we must encode in WL the 𝛼 and the vector variables 𝑣 , . . . , 𝑣 𝑘 . For this, we use variables 𝑥 row 𝛼 , 𝑥 col 𝛼 , and 𝑥 𝑣 𝑖 for each variable 𝑣 , . . . , 𝑣 𝑘 . Then we use the following inductive hypothesis (similarto Proposition 6.3): • If 𝑒 ( 𝑣 , . . . , 𝑣 𝑘 ) is of type ( 𝛼, 𝛼 ) then there exists a WL formula 𝜑 𝑒 ( 𝑥 row 𝛼 , 𝑥 col 𝛼 , 𝑥 𝑣 , . . . , 𝑥 𝑣 𝑘 ) such that ⟦ 𝜑 𝑒 ⟧ WL (I) ( 𝜎 ) = ⟦ 𝑒 ⟧(I [ 𝑣 ← 𝑏 𝑖 , . . . , 𝑣 𝑘 ← 𝑏 𝑖 𝑘 ]) 𝑖,𝑗 for assignment 𝜎 with 𝜎 ( 𝑥 row 𝛼 ) = 𝑖 , 𝜎 ( 𝑥 col 𝛼 ) = 𝑗 and 𝜎 ( 𝑥 𝑣 𝑠 ) = 𝑖 𝑠 for 𝑠 = , . . . , 𝑘 . • If 𝑒 ( 𝑣 , . . . , 𝑣 𝑘 ) is of type ( 𝛼, ) then there exists a WL formula 𝜑 𝑒 ( 𝑥 row 𝛼 , 𝑥 𝑣 , . . . , 𝑥 𝑣 𝑘 ) such that ⟦ 𝜑 𝑒 ⟧ WL (I) ( 𝜎 ) = ⟦ 𝑒 ⟧(I [ 𝑣 ← 𝑏 𝑖 , . . . , 𝑣 𝑘 ← 𝑏 𝑖 𝑘 ]) 𝑖 for assignment 𝜎 with 𝜎 ( 𝑥 row 𝛼 ) = 𝑖 and 𝜎 ( 𝑥 𝑣 𝑠 ) = 𝑖 𝑠 for 𝑠 = , . . . , 𝑘 . And similarly for when 𝑒 is type ( , 𝛼 ) . onference’17, July 2017, Washington, DC, USA Floris Geerts, Thomas Muñoz, Cristian Riveros, and Domagoj Vrgoč • If 𝑒 ( 𝑣 , . . . , 𝑣 𝑘 ) is of type ( , ) then there exists a WL formula 𝜑 𝑒 ( 𝑥 𝑣 , . . . , 𝑥 𝑣 𝑘 ) such that ⟦ 𝜑 𝑒 ⟧ WL (I) ( 𝜎 ) = ⟦ 𝑒 ⟧(I [ 𝑣 ← 𝑏 𝑖 , . . . , 𝑣 𝑘 ← 𝑏 𝑖 𝑘 ]) for assignment 𝜎 with 𝜎 ( 𝑥 𝑣 𝑠 ) = 𝑖 𝑠 for 𝑠 = , . . . , 𝑘 .If we prove the previous statement we are done, because the last bullet is what we want to show when 𝑒 has no free vector variables. Thenrest of the proof is to go by induction on the structure of FO - MATLANG expressions. For a WL-formula 𝜑 and FO-variables 𝑥, 𝑦 , we will write 𝜑 [ 𝑥 ↦→ 𝑦 ] the formula 𝜑 when 𝑥 is replaced with 𝑦 all over the formula (syntactically). Let 𝑒 be a FO - MATLANG expression. • If 𝑒 : = 𝑉 and S( 𝑒 ) = ( 𝛼, 𝛼 ) then 𝜑 𝑒 : = 𝑅 𝑉 ( 𝑥 row 𝛼 , 𝑥 col 𝛼 ) . Similarly, if S( 𝑒 ) is of type ( 𝛼, ) , ( , 𝛼 ) , or ( , ) , then 𝜑 𝑒 : = 𝑅 𝑉 ( 𝑥 row 𝛼 ) , 𝜑 𝑒 : = 𝑅 𝑉 ( 𝑥 col 𝛼 ) , and 𝜑 𝑒 : = 𝑅 𝑉 , respectively. • If 𝑒 : = 𝑣 , for 𝑣 ∈ { 𝑣 , . . . , 𝑣 𝑘 } , and S( 𝑣 ) = ( 𝛼, ) then 𝜑 𝑒 : = 𝑥 row 𝛼 = 𝑥 𝑣 . Similarly, if S( 𝑣 ) = ( , 𝛼 ) then 𝜑 𝑒 : = 𝑥 col 𝛼 = 𝑥 𝑣 . • if 𝑒 : = 𝑒 𝑇 and S( 𝑒 ) = ( 𝛼, 𝛼 ) then 𝜑 𝑒 : = 𝜑 𝑒 [ 𝑥 row 𝛼 ↦→ 𝑥 col 𝛼 , 𝑥 col 𝛼 ↦→ 𝑥 row 𝛼 ] . Similarly, if S( 𝑒 ) is equal to ( 𝛼, ) or ( , 𝛼 ) then 𝜑 𝑒 : = 𝜑 𝑒 [ 𝑥 row 𝛼 ↦→ 𝑥 col 𝛼 ] and 𝜑 𝑒 : = 𝜑 𝑒 [ 𝑥 col 𝛼 ↦→ 𝑥 row 𝛼 ] , respectively. • If 𝑒 = 𝑒 + 𝑒 with S( 𝑒 ) = S( 𝑒 ) , then 𝜑 𝑒 : = 𝜑 𝑒 ⊕ 𝜑 𝑒 . • If 𝑒 = 𝑓 ⊙ ( 𝑒 , . . . , 𝑒 𝑘 ) with S( 𝑒 𝑖 ) = S( 𝑒 𝑗 ) for all 𝑖, 𝑗 ∈ [ , 𝑘 ] , then 𝜑 𝑒 : = 𝜑 𝑒 ⊙ 𝜑 𝑒 · · · ⊙ 𝜑 𝑒 𝑘 . • If 𝑒 = 𝑒 · 𝑒 with S( 𝑒 ) = S( 𝑒 ) = ( 𝛼, 𝛼 ) , then 𝜑 𝑒 : = Σ 𝑦. 𝜑 𝑒 [ 𝑥 col 𝛼 ↦→ 𝑦 ] ⊙ 𝜑 𝑒 [ 𝑥 row 𝛼 ↦→ 𝑦 ] where 𝑦 is a fresh variable not mentionedin 𝜑 𝑒 or 𝜑 𝑒 . Instead, if S( 𝑒 ) = ( 𝛼 ′ , ) and S( 𝑒 ) = ( , 𝛼 ′′ ) with 𝛼 ′ , 𝛼 ′′ ∈ { 𝛼, } , then 𝜑 𝑒 : = 𝜑 𝑒 ⊙ 𝜑 𝑒 . • If 𝑒 = Σ 𝑣.𝑒 ( 𝑣 ) , then we define 𝜑 𝑒 : = Σ 𝑥 𝑣 . 𝜑 𝑒 ( 𝑥 𝑣 ) . • If 𝑒 = Π ◦ 𝑣.𝑒 ( 𝑣 ) , then 𝜑 𝑒 : = Π 𝑥 𝑣 . 𝜑 𝑒 ( 𝑥 𝑣 ) .From the construction it is now straightforward to check that the inductive hypothesis holds for all cases. To conclude this direction, wehave to define Φ ( 𝑒 ) : = 𝜑 𝑒 for every expression 𝑒 and we are done. (WL to FO - MATLANG ) We now encode weighted structures into matrices and vectors. Let Γ be a relational vocabulary where arity ( 𝑅 ) ≤ .Define Mat ( Γ ) = (M Γ , size Γ ) such that M Γ = { 𝑉 𝑅 | 𝑅 ∈ Γ } and size Γ ( 𝑉 𝑅 ) is equal to ( 𝛼, 𝛼 ) , ( 𝛼, ) , or ( , ) if arity ( 𝑅 ) = , arity ( 𝑅 ) = ,or arity ( 𝑅 ) = , respectively, for some 𝛼 ∈ Symb . Similarly, let A = ( 𝐴, { 𝑅 A } 𝑅 ∈ Γ ) be a structure with 𝐴 = { 𝑎 , . . . , 𝑎 𝑛 } , ordered arbitrarily.Then we define the matrix instance Mat (A) = (D , mat ) such that D( 𝛼 ) = 𝑛 , mat ( 𝑉 𝑅 ) 𝑖,𝑗 = 𝑅 A ( 𝑎 𝑖 , 𝑎 𝑗 ) if arity ( 𝑅 ) = , mat ( 𝑉 𝑅 ) 𝑖, = 𝑅 A ( 𝑎 𝑖 ) if arity ( 𝑅 ) = , and mat ( 𝑉 𝑅 ) , = 𝑅 A otherwise.Similar to the above direction, we have to encode the FO variables of a formula 𝜑 with vector variables in the equivalent FO - MATLANG expression 𝑒 𝜑 . For this, for each FO variable 𝑥 we define a vector variable 𝑣 𝑥 of type ( 𝛼, ) . Then for each formula 𝜑 ( 𝑥 , . . . , 𝑥 𝑘 ) we define anexpression 𝑒 𝜑 ( 𝑣 𝑥 , . . . , 𝑣 𝑥 𝑘 ) of type ( , ) such that for every assignment 𝜎 of 𝑥 , . . . , 𝑥 𝑘 we have: ⟦ 𝑒 𝜑 ⟧( Mat (A)[ 𝑣 𝑥 ← 𝑏 𝑖 , . . . , 𝑣 𝑥 ← 𝑏 𝑖 𝑘 ]) = ⟦ 𝜑 ⟧ A ( 𝜎 ) such that 𝜎 ( 𝑥 𝑠 ) = 𝑖 𝑠 for every 𝑠 ≤ 𝑘 . Note that when the formula has no free variables, the proof of the proposition is shown. Finally, weproceed by induction over the formula 𝜑 over Γ . • If 𝜑 : = 𝑥 = 𝑦 , then 𝑒 𝜑 : = 𝑣 𝑇𝑥 · 𝑣 𝑦 . • If 𝜑 : = 𝑅 ( 𝑥, 𝑦 ) , then 𝑒 𝜑 : = 𝑣 𝑇𝑥 · 𝑉 𝑅 · 𝑣 𝑦 . Similarly, if 𝜑 : = 𝑅 ( 𝑥 ) or 𝜑 : = 𝑅 , then 𝑒 𝜑 : = 𝑉 𝑇𝑅 · 𝑣 𝑥 and 𝑒 𝜑 : = 𝑉 𝑅 , respectively. • If 𝜑 = 𝜑 ⊕ 𝜑 , then 𝑒 𝜑 : = 𝑒 𝜑 + 𝑒 𝜑 . • If 𝜑 = 𝜑 ⊙ 𝜑 , then 𝑒 𝜑 : = 𝑓 ⊙ ( 𝑒 𝜑 , 𝑒 𝜑 ) . • If 𝜑 = Σ 𝑥. 𝜑 , then 𝑒 𝜑 : = Σ 𝑣 𝑥 . 𝑒 𝜑 . • If 𝜑 = Π ◦ 𝑥.𝜑 , then 𝑒 𝜑 : = Π 𝑣 𝑥 . 𝑒 𝜑 .The inductive hypothesis can be proved following the above construction. To finish the proof, we define Ψ ( 𝜑 ) : = 𝑒 𝜑 and the proposition isshown. □ xpressive power of linear algebra query languages Conference’17, July 2017, Washington, DC, USA E.4 Matrix inversion in prod - MATLANG extended with order
We conclude by verifying that the fragment defined in Section 6.3, i.e, prod - MATLANG extended with order and 𝑓 > , can perform matrixinversion and compute the determinant. To this aim, we verify that all order predicates in Section B.1 can be derived using Σ , Π , 𝑓 > and 𝑒 𝑆 < . Given this, it suffices to observe that Csanky’s algorithm, as shown in Section C.3, only relies on expressions using Σ and Π and orderinformation on canonical vectors and 𝑓 / . As consequence, our fragment can perform matrix inversion and compute the determinant.It remains to show that if we have 𝑒 𝑆 < , using Σ and Π and 𝑓 > we can can define all order predicates from Section B.1. We note that due tothe restricted for-loops in Σ and Π , we do not have access to the intermediate result in the iterations and as such, it is unclear whether orderinformation can be computed. This is why we assume access to 𝑒 𝑆 < .We first remark that if we have 𝑒 𝑆 < , we can also obtain 𝑒 𝑆 ≤ by adding 𝑒 Id . Hence, we can compute succ and succ + as well. Furthermore, 𝑒 min : = Σ 𝑣. [ Π 𝑤 . succ ( 𝑤, 𝑣 )] × 𝑣.𝑒 max : = Σ 𝑣. [ Π 𝑤 . ( − succ ( 𝑤, 𝑣 ))] × 𝑣. Both expressions are only using Σ and Π and succ , so are in our fragment. Furthermore, if we have 𝑓 > then we can define 𝑒 Pred : = 𝑒 𝑆 < − 𝑓 > ( 𝑒 𝑆 < ) Also, recall that 𝑒 Next : = 𝑒 𝑇 Pred . As a consequence, we can now define prev ( 𝑣 ) and next ( 𝑣 ) as in B.1. Similarly, it is readily verified that also 𝑒 getPrevMatrix ( 𝑉 ) , 𝑒 getNextMatrix ( 𝑉 ) , 𝑒 min + 𝑖 and 𝑒 max + 𝑖𝑖