The tensor rank of 5x5 matrices multiplication is bounded by 98 and its border rank by 89
TThe tensor rank of × matrices multiplicationis bounded by and its border rank by Alexandre Sedoglavic umr cnrs 9189 cristalUniversité de LilleF-59000 Lille, France
Alexey V. Smirnov
Russian Federal Center of Forensic ScienceDepartment of JusticeMoscow, Russia
ABSTRACT
We present a non-commutative algorithm for the product of × by × matrices using multiplications. This algorithm allowsto construct a non-commutative algorithm for multiplying × (resp. × , × ) matrices using (resp. , ) multi-plications. Furthermore, we describe an approximate algorithmthat requires multiplications and computes this product with anarbitrary small error. CCS CONCEPTS • Computing methodologies → Exact arithmetic algorithms ; Linear algebra algorithms . KEYWORDS algebraic complexity, fast matrix multiplication
Even if matrix multiplication is one of the most fundamental tool inscientific computing, it is still not completely understood. Strassen’salgorithm [28], with recursive multiplications and additions,was the first sub-cubic time algorithm for matrix product (with acost of 𝑂 ( 𝑛 log ) ) and finding explicit algorithms for small sizematrix product remains today a challenge. In order to describe thecontributions presented in this paper, let us recall some well-knownterminologies for the sake of clarity.Definitions 1.1. We denote by M 𝑚,𝑛,𝑝 the bilinear map repre-senting the matrix product of a 𝑚 × 𝑛 -matrix by a 𝑛 × 𝑝 matrix. Inparticular, given any field F , there exist 𝑟 𝑚 × 𝑝 matrices ( 𝐶 𝑖 ) ≤ 𝑖 ≤ 𝑟 , 𝑟 linear forms ( ℓ 𝑖 ) ≤ 𝑖 ≤ 𝑟 from F 𝑚 × 𝑛 into F and 𝑟 linear forms ( ℓ ′ 𝑖 ) ≤ 𝑖 ≤ 𝑟 from F 𝑛 × 𝑝 into F such that the product of the 𝑚 × 𝑛 matrix 𝐴 bythe 𝑛 × 𝑝 matrix 𝐵 is computed by the following computational scheme: M 𝑚,𝑛,𝑝 ( 𝐴, 𝐵 ) = 𝐴 · 𝐵 = 𝑟 ∑︁ 𝑖 = ℓ 𝑖 ( 𝐴 ) ℓ ′ 𝑖 ( 𝐵 ) 𝐶 𝑖 . (1) This non-commutative scheme is classically interpreted as a tensor (see precise encoding in Section 2) and we recall that the number 𝑟 of its summands is the rank of that tensor. In this work, the no-tations ⟨ 𝑚 × 𝑛 × 𝑝 : 𝑟 ⟩ stands for a tensor of rank 𝑟 encoding theproduct M 𝑚,𝑛,𝑝 . We denotes by ⟨ 𝑚 × 𝑛 × 𝑝 ⟩ the whole family of suchschemes independently of their rank. The tensor rank R ⟨ 𝑚 × 𝑛 × 𝑝 ⟩ ofthe considered matrix product is the smallest integer 𝑟 such that thereis a tensor ⟨ 𝑚 × 𝑛 × 𝑝 : 𝑟 ⟩ in ⟨ 𝑚 × 𝑛 × 𝑝 ⟩ . Similarly, { 𝑚 × 𝑛 × 𝑝 : 𝑟 } denotes a computational scheme of rank 𝑟 involving a parameter 𝜖 whose limit computes the matrix product M 𝑚,𝑛,𝑝 exactly as 𝜖 tendsto zero. The border rank of M 𝑚,𝑛,𝑝 is the smallest integer 𝑟 such thatthere exits an approximate scheme { 𝑚 × 𝑛 × 𝑝 : 𝑟 } . The tensor rank R ⟨ 𝑛 × 𝑛 × 𝑛 ⟩ is related to the number of multi-plications needed to compute the product of two 𝑛 × 𝑛 and gives ameasures of its complexity such as the exponent 𝜔 of matrix multi-plication equal to lim 𝑛 →∞ R ⟨ 𝑛 × 𝑛 × 𝑛 ⟩ . There is no result relativeto asymptotic complexity in the present work and we refer to [1]for a presentation of these researches, bibliographic references andthe last best asymptotic bound 𝑂 ( 𝑛 . ) known to date.Alongside these galactic algorithms, a substantial amount ofwork was devoted to the practical design of computational schemesfor small matrix products (see [8] for bibliographic referencesand [23] for last known results). Our contribution follows this pathand improves the complexity of the product of × matrices andseveral other algorithms.The structure of ⟨ × × ⟩ is broadly understood: there is basi-cally just one rank tensor ⟨ × × ⟩ introduced in [28] up tosymmetries as shown in [10, Thm 0.1]; furthermore, in that casethe tensor rank is known to be (see [29, Thm 3.1]). But alreadythe structure of ⟨ × × ⟩ remains unresolved and very little isknown about the actual complexity of small matrix products.Pursuing the works done the last years in [18–20, 22], wepropose a new non-commutative algorithm for multiplying × matrices using multiplications. More precisely, by presentingexplicit algorithms, we establish the following result:Theorem 1.1. The tensor rank of matrix multiplication tensorencoding the product of a × by a × matrix is bounded by .This implies that the tensor rank of matrix multiplication tensorencoding the product of × matrices is bounded by .Furthermore, the border rank of matrix multiplication tensor en-coding the product of × matrices is bounded by . This theorem results mainly from the work initiated by the sec-ond author in [26]. This work induces many algorithms (e.g. see [23]for a complete list) and especially an algorithm ⟨ × × ⟩ thatleads to tensors ⟨ × × ⟩ and ⟨ × ×
12 : 1040 ⟩ (thislast one defines a generic algorithm with cost 𝑂 ( 𝑛 . ) for suitablesquare matrices). In the forthcoming sections, we present several matrix product algo-rithms as trilinear forms. To do so, let us review the needed notionsthrough an well-known example. The matrix product 𝐶 = 𝐴 · 𝐵 could be computed using Strassen algorithm by performing the a r X i v : . [ c s . CC ] F e b lexandre Sedoglavic and Alexey V. Smirnov following computations (see [28]): 𝜌 ← 𝑎 ( 𝑏 − 𝑏 ) ,𝜌 ← ( 𝑎 + 𝑎 ) 𝑏 , 𝜌 ← ( 𝑎 − 𝑎 )( 𝑏 + 𝑏 ) ,𝜌 ← ( 𝑎 + 𝑎 ) 𝑏 , 𝜌 ← ( 𝑎 + 𝑎 )( 𝑏 + 𝑏 ) ,𝜌 ← 𝑎 ( 𝑏 − 𝑏 ) , 𝜌 ← ( 𝑎 − 𝑎 )( 𝑏 + 𝑏 ) , (cid:0) 𝑐 𝑐 𝑐 𝑐 (cid:1) = (cid:16) 𝜌 + 𝜌 − 𝜌 + 𝜌 𝜌 + 𝜌 𝜌 + 𝜌 𝜌 + 𝜌 + 𝜌 − 𝜌 (cid:17) . (2)In order to consider this algorithm under a geometric standpoint,we present it as a tensor. Matrix multiplication is a bilinear map: F 𝑚 × 𝑛 × F 𝑛 × 𝑝 → F 𝑚 × 𝑝 , ( 𝐴, 𝐵 ) → 𝐴 · 𝐵, (3)where the spaces F ·×· are finite vector spaces over a field F thatcan be endowed with the Frobenius inner product: ⟨ 𝑀, 𝑁 ⟩ = Trace ( 𝑀 ⊺ · 𝑁 ) . (4)Hence, this inner product establishes an isomorphism between F ·×· and its dual space (cid:0) F ·×· (cid:1) ★ allowing for example to associate matrixmultiplication and the trilinear form Trace ( 𝐶 ⊺ · 𝐴 · 𝐵 ) : F 𝑚 × 𝑛 × F 𝑛 × 𝑝 × ( F 𝑚 × 𝑝 ) ★ → F , ( 𝐴, 𝐵, 𝐶 ⊺ ) → ⟨ 𝐶, 𝐴 · 𝐵 ⟩ . (5)As by construction, the space of trilinear forms is the canonicaldual space of order three tensor product, we could associate theStrassen multiplication algorithm (2) with the tensor S defined by: (cid:205) 𝑖 = 𝑃 𝑖 ⊗ 𝑄 𝑖 ⊗ 𝑆 𝑖 = (cid:0) (cid:1) ⊗ (cid:0) − (cid:1) ⊗ (cid:0) (cid:1) + (cid:0) (cid:1) ⊗ (cid:0) (cid:1) ⊗ (cid:0) − (cid:1) + (cid:0) (cid:1) ⊗ (cid:0) (cid:1) ⊗ (cid:0) − (cid:1) + (cid:0) − (cid:1) ⊗ (cid:0) (cid:1) ⊗ (cid:0) (cid:1) + (cid:0) (cid:1) ⊗ (cid:0) (cid:1) ⊗ (cid:0) (cid:1) + (cid:0) (cid:1) ⊗ (cid:0) − (cid:1) ⊗ (cid:0) (cid:1) + (cid:0) − (cid:1) ⊗ (cid:0) (cid:1) ⊗ (cid:0) (cid:1) (6)in ( F 𝑚 × 𝑛 ) ★ ⊗ ( F 𝑛 × 𝑝 ) ★ ⊗ F 𝑚 × 𝑝 with 𝑚 = 𝑛 = 𝑝 = . As stated inDefinitions 1.1, this tensor is a particular description of the matrixproduct bilinear map. Indeed, there is an infinite set of equivalent tensors shown by the following theorem:Theorem 2.1 ([9, § 2.8]). The isotropy group of any matrix multi-plication tensor in ⟨ 𝑚 × 𝑛 × 𝑝 ⟩ is ( psl ( 𝑚 ) × psl ( 𝑛 ) × psl ( 𝑝 )) ⋊ 𝔖 ,where psl stands for the group of matrices of determinant ± and 𝔖 for the symmetric group on elements. The following proposition gives another description from aninvariant perspective.Proposition 2.2 ([16, § 2.5.1]).
The tensor defining the prod-uct of a 𝑚 × 𝑛 -matrix by a 𝑛 × 𝑝 -matrix is isomorphic to the ten-sor Id 𝑚 × 𝑚 ⊗ Id 𝑛 × 𝑛 ⊗ Id 𝑝 × 𝑝 . This isomorphism gives the well-knownexpression of the classical matrix multiplication tensor: 𝑚 ∑︁ 𝑖 = 𝑛 ∑︁ 𝑗 = 𝑝 ∑︁ 𝑘 = 𝐸 𝑗𝑖 ⊗ 𝐸 𝑘𝑗 ⊗ 𝐸 𝑖𝑘 , (7) where 𝐸 𝑗𝑖 denotes the matrix with its coefficient at the intersection ofline 𝑖 and column 𝑗 equal to and all its other coefficients equal to . Let us introduce now an invariant for the action described inTheorem 2.1 that will be useful in our presentation. Definition 2.1.
Given a tensor P decomposable as sum of rank-one elementary tensors: P = 𝑞 ∑︁ 𝑖 = 𝑃 𝑖 ⊗ 𝑄 𝑖 ⊗ 𝑆 𝑖 (8) where 𝑃 𝑖 , 𝑄 𝑖 and 𝑆 𝑖 are matrices of suitable sizes for 𝑖 in { , . . . , 𝑞 } .The type of a tensor P is the list [( rank 𝑃 𝑖 , rank 𝑄 𝑖 , rank 𝑆 𝑖 )] 𝑖 = ...𝑞 .Inspired by [13, § 4], we encode the type of a tensor P as thefollowing polynomial: 𝑇 P ( 𝑋, 𝑌, 𝑍 ) = 𝑞 ∑︁ 𝑖 = 𝑋 rank 𝑃 𝑖 𝑌 rank 𝑄 𝑖 𝑍 rank 𝑆 𝑖 . (9)Hence, the type of Strassen’s tensor is 𝑋 𝑌 𝑍 + 𝑋𝑌𝑍 .Now we precise the trilinear form of tensor that is used in thesequel of this work. Given any triple ( 𝐴, 𝐵, 𝐶 ) of suitable size ma-trices, one can explicitly express from tensor S the Strassen matrixmultiplication algorithm computing 𝐴 · 𝐵 by the complete contrac-tion {S , 𝐴 ⊗ 𝐵 ⊗ 𝐶 } : (cid:16) ( F 𝑚 × 𝑛 ) ★ ⊗ ( F 𝑛 × 𝑝 ) ★ ⊗ F 𝑚 × 𝑝 (cid:17) ⊗ (cid:16) F 𝑚 × 𝑛 ⊗ F 𝑛 × 𝑝 ⊗ ( F 𝑚 × 𝑝 ) ★ (cid:17) → F , S ⊗ ( 𝐴 ⊗ 𝐵 ⊗ 𝐶 ) → (cid:205) 𝑖 = ⟨ 𝑃 𝑖 , 𝐴 ⟩⟨ 𝑄 𝑖 , 𝐵 ⟩⟨ 𝑆 𝑖 , 𝐶 ⟩ (10)that is equal to Trace ( 𝐴 · 𝐵 · 𝐶 ) . Hence, for inputs 𝐴 = ( 𝑎 𝑖 𝑗 ) , 𝐵 = ( 𝑏 𝑖 𝑗 ) and 𝐶 = ( 𝑐 𝑖 𝑗 ) of suitable sizes ( 𝑖 = , and 𝑗 = , ), the represen-tation of Strassen Algorithm (2) as a trilinear form is: 𝑎 ( 𝑏 − 𝑏 ) ( 𝑐 + 𝑐 )+ ( 𝑎 + 𝑎 ) 𝑏 (− 𝑐 + 𝑐 )+ ( 𝑎 + 𝑎 ) 𝑏 ( 𝑐 − 𝑐 )+ ( 𝑎 − 𝑎 ) ( 𝑏 + 𝑏 ) 𝑐 + ( 𝑎 + 𝑎 ) ( 𝑏 + 𝑏 ) ( 𝑐 + 𝑐 )+ 𝑎 ( 𝑏 − 𝑏 ) ( 𝑐 + 𝑐 )+ ( 𝑎 − 𝑎 ) ( 𝑏 + 𝑏 ) 𝑐 . (11)Before ending this section let us recall that, as stated in introduction,the ⟨ × × ⟩ is pretty well-understood even in its rectangularcounterpart as shown by the following proposition:Proposition 2.3 ([14, Thm 1]). The 𝑚 × by × 𝑛 matrix prod-uct can be encoded by a ⟨ 𝑚 × × 𝑛 : ⌈( 𝑚𝑛 + max ( 𝑚, 𝑛 ))/ ⌉⟩ ten-sor. The next section is devoted to gather classical results used in thesequel of this paper to construct new algorithm for small matrixproduct.
Introducing tensor to represents matrix product and their relation-ship with the trace operator induces naturally several interestingresults on matrix product algorithms. First, let us remark that—given three matrices
𝐴, 𝐵 and 𝐶 of suitable sizes—the followingproperties of the trace operator:Trace ( 𝐴 · 𝐵 · 𝐶 ) = Trace ( 𝐶 · 𝐴 · 𝐵 ) , = Trace ( 𝐵 · 𝐶 · 𝐴 ) , = Trace (cid:0) ( 𝐴 · 𝐵 · 𝐶 ) ⊺ (cid:1) , = Trace (cid:0) 𝐶 ⊺ · 𝐵 ⊺ · 𝐴 ⊺ (cid:1) , (12) he tensor rank of × matrices multiplicationis bounded by and its border rank by show that the following relations hold: ⟨ 𝑚 × 𝑛 × 𝑝 : 𝑟 ⟩ = ⟨ 𝑝 × 𝑚 × 𝑛 : 𝑟 ⟩ = ⟨ 𝑛 × 𝑝 × 𝑚 : 𝑟 ⟩ , = ⟨ 𝑝 × 𝑛 × 𝑚 : 𝑟 ⟩ = ⟨ 𝑚 × 𝑝 × 𝑛 : 𝑟 ⟩ , = ⟨ 𝑛 × 𝑚 × 𝑝 : 𝑟 ⟩ . (13)Furthermore, for all ℓ such that ≤ ℓ ≤ 𝑚 − there is a naturalisomorphism between F 𝑚 × 𝑛 and F ℓ × 𝑛 ⊕ F ( 𝑚 − ℓ )× 𝑛 ; The same re-mark shows that Id 𝑚 × 𝑚 is equal to Id ℓ × ℓ ⊕ Id ( 𝑚 − ℓ )×( 𝑚 − ℓ ) . Thisrelation and the tensor product’s properties imply that the ten-sor Id 𝑚 × 𝑚 ⊗ Id 𝑛 × 𝑛 ⊗ Id 𝑝 × 𝑝 is equal to Id ℓ × ℓ ⊗ Id 𝑛 × 𝑛 ⊗ Id 𝑝 × 𝑝 + Id ( 𝑚 − ℓ )×( 𝑚 − ℓ ) ⊗ Id 𝑛 × 𝑛 ⊗ Id 𝑝 × 𝑝 . (14)These simple remarks and Proposition 2.2 recall that small sizesmatrix product algorithms allow by their direct sum to constructan algorithm for the product of matrices of bigger sizes as shownby the following well-known lemma:Lemma 2.1. Given ⟨ ℓ × 𝑛 × 𝑝 : 𝑟 ⟩ and ⟨( 𝑚 − ℓ ) × 𝑛 × 𝑝 : 𝑠 ⟩ , onecan construct ⟨ 𝑚 × 𝑛 × 𝑝 : 𝑟 + 𝑠 ⟩ as follow: ⟨ 𝑚 × 𝑛 × 𝑝 : 𝑟 + 𝑠 ⟩ = ⟨ ℓ × 𝑛 × 𝑝 : 𝑟 ⟩ ⊕ ⟨( 𝑚 − ℓ ) × 𝑛 × 𝑝 : 𝑠 ⟩ . (15) There is a similar construction using the tensor product: ⟨ 𝑚𝑢 × 𝑛𝑣 × 𝑝𝑤 : 𝑟𝑠 ⟩ = ⟨ 𝑚 × 𝑛 × 𝑝 : 𝑟 ⟩ ⊗ ⟨ 𝑢 × 𝑣 × 𝑤 : 𝑠 ⟩ . (16)We say that a tensor is “atomic” if it is not constructed using theabove constructions, Proposition 2.3 or if it is induced by Strassen’salgorithm (see [8, § 2] and [24] for such constructions).The previous sections were devoted to the notions and notationsnecessary to describe concisely the new results on which the nextsections focus. We could now present the main result of this paper. The rank of the classical tensor in ⟨ × × ⟩ is . Proposition 2.3and Lemma 2.1 improves the resulting bound to as shown bythe following relations: ⟨ × × ⟩ = ⟨ × × ⟩ ⊕ ⟨ × × ⟩ , (17) ⟨ × × ⟩ = ⟨ × × ⟩ ⊕ ⟨ × × ⟩ . (18)This bound was superseded using ⟨ × × ⟩ introduced in [26]and above standard constructions.Our results are also rooted in the same kind of experimentalmathematics combining computer power and human efforts. Thus,let us give a short account of a method allowing to find new atomictensors in the next section. As any tensor in ⟨ 𝑚 × 𝑛 × 𝑝 ⟩ encodes the same bilinear map, Propo-sition 7 implies that the following relation always holds: ⟨ 𝑚 × 𝑛 × 𝑝 : 𝑟 ⟩ − 𝑚 ∑︁ 𝑖 = 𝑛 ∑︁ 𝑗 = 𝑝 ∑︁ 𝑘 = 𝐸 𝑗𝑖 ⊗ 𝐸 𝑘𝑗 ⊗ 𝐸 𝑖𝑘 = . (19)Using an ansatz with undetermined coefficients for ⟨ 𝑚 × 𝑛 × 𝑝 : 𝑟 ⟩ ,this relation defines the Brent over-determined system of ( 𝑚𝑛𝑝 ) cubic polynomial equations in ( 𝑚𝑛 + 𝑛𝑝 + 𝑝𝑚 ) 𝑟 unknowns (see [5,§ 5, eq 5.03]). Theoretically, a Gröbner basis computation allowsto describe all solutions of this system and thus close the topic. But such a resolution is not possible in practice for any matrixsize of interest. Nevertheless several original matrix multiplicationalgorithms where found by hand (e.g. ⟨ × × ⟩ in [15] andprobably Strassen’s algorithm [28]) and almost every method forsolving were tried (e.g. sat solver in [13, § 2]) with—up to ourknowledge—few complexity improvements.For now, the almost only productive approach remains numericaloptimization method using least-squares methods and heuristics.In fact, while objective functions derived from Equation (19) arenon-convex and nonlinear, they can be splitted in three linear sub-systems (by helding two components of the unknown tensor fixedfor example) and their resolution boils down to linear algebra. Nev-ertheless, in order to obtain new results, a regularization term needsto be added and the following is chosen in this paper:Arg min 𝑃 𝑖 , 𝑄 𝑖 , 𝑆 𝑖 (cid:13)(cid:13)(cid:13)(cid:205) 𝑞𝑖 = 𝑃 𝑖 ⊗ 𝑄 𝑖 ⊗ 𝑆 𝑖 − (cid:205) 𝑚𝑖 = (cid:205) 𝑛𝑗 = (cid:205) 𝑝𝑘 = 𝐸 𝑗𝑖 ⊗ 𝐸 𝑘𝑗 ⊗ 𝐸 𝑖𝑘 (cid:13)(cid:13)(cid:13) + 𝜆 (cid:16)(cid:205) 𝑞𝑖 = (cid:13)(cid:13)(cid:13) 𝑃 𝑖 − (cid:101) 𝑃 𝑖 (cid:13)(cid:13)(cid:13) + (cid:13)(cid:13)(cid:13) 𝑄 𝑖 − (cid:102) 𝑄 𝑖 (cid:13)(cid:13)(cid:13) + (cid:13)(cid:13)(cid:13) 𝑆 𝑖 − (cid:101) 𝑆 𝑖 (cid:13)(cid:13)(cid:13)(cid:17) . (20)with model matrices (cid:101) 𝑃 𝑖 , (cid:102) 𝑄 𝑖 , (cid:101) 𝑆 𝑖 defining the regularization and ascalar parameter 𝜆 that determines the weight of the regularizationterm. The models are designed to drive the solution to match a de-sired structure and are choosed carefully for each iteration (see [26]for a detailed presentation). This approach gives a uniform methodfor deriving exact algorithm but also approximate one when theyare a order polynomial approximation w.r.t their parameter 𝜖 .A better precision—an approximation order greater then —requires several other heuristics. If all dimensions of the problemare greater than , it is not yet possible to obtain acceptably shortexact algorithms. Hence, the success of the resolution presentedhere relies on heuristical expertise and tyazhelaya rabota.However,the works ot that topic done since [26] show that the objective func-tion of the found approximate algorithms allows to presumablyestimate the exact rank of large problems. This point will appear ina future work.Let us now describe the latest exact matrix multiplication tensorfound with this method. ⟨ × × ⟩ description Before the detailed description done in the next section, let us firstpresent the type introduced in Definition 2.1 of this tensor: 𝑋 𝑌 𝑍 + 𝑋𝑌 𝑍 + 𝑋 𝑌 𝑍 + 𝑋𝑌 𝑍 + 𝑋 𝑌𝑍 + 𝑋𝑌𝑍 + 𝑋 𝑌 𝑍 + 𝑋𝑌 𝑍 + 𝑋𝑌 𝑍 + 𝑋 𝑌𝑍 + 𝑋𝑌𝑍 + 𝑋𝑌 𝑍 + 𝑋𝑌𝑍 = 𝑇 ( 𝑋, 𝑌, 𝑍 ) . (21)Remark 3.1. Considering that the indeterminates commute, therelation 𝑇 ( 𝑋, 𝑌, 𝑍 ) = 𝑇 ( 𝑍, 𝑌, 𝑋 ) holds. Unfortunately, even if thisproperty suggests the existence of a symmetry (see Equation (12)and [9, 𝜋 in Thm 3.4] for a detailed description), this tensor does nothave a non-trivial stabilizer. Such stabilizer are not uncommon fortensor found using the method sketched in Section 3.1: for example thestabilizer (( 𝐶 × 𝐶 ) ⋊ 𝑆 ) ⋊ 𝐶 of ⟨ × × ⟩ is of order (see [23]). So, even if Comon’s conjecture was disproved in full gen-erality (see [25]), there might be a tensor in ⟨ × × ⟩ of rank with stabilizer 𝐶 × 𝐶 × 𝑆 . lexandre Sedoglavic and Alexey V. Smirnov As two matrix multiplication tensor of same rank could sharethe same type, this invariant is not a faithful descriptions and wegive its explicit expression in the next section.
Trilinear form of ⟨ × × ⟩ . We split the forthcoming de-scription in expressions whose components have the type corre-sponding to their indices as follow: ⟨ × × ⟩ = 𝜏 + 𝜏 + 𝜏 + 𝜏 + 𝜏 + 𝜏 + 𝜏 + 𝜏 + 𝜏 + 𝜏 + 𝜏 + 𝜏 + 𝜏 . (22)Let us start with 𝜏 : 𝜏 = 𝑎 (cid:18) 𝑏 + 𝑏 + 𝑏 − 𝑏 + 𝑏 − 𝑏 − 𝑏 + 𝑏 + 𝑏 (cid:19) ( 𝑐 + 𝑐 )− ( 𝑎 − 𝑎 ) (cid:18) 𝑏 − 𝑏 + 𝑏 + 𝑏 + 𝑏 − 𝑏 + 𝑏 − 𝑏 − 𝑏 (cid:19) 𝑐 . (23) 𝜏 = ( 𝑎 − 𝑎 − 𝑎 ) ( 𝑏 − 𝑏 ) ( 𝑐 + 𝑐 + 𝑐 ) (24) 𝜏 = ( 𝑎 + 𝑎 − 𝑎 ) 𝑏 ( 𝑐 + 𝑐 + 𝑐 ) . (25) 𝜏 = ( 𝑎 − 𝑎 + 𝑎 ) ( 𝑏 + 𝑏 ) ( 𝑐 + 𝑐 + 𝑐 + 𝑐 )+ ( 𝑎 − 𝑎 + 𝑎 ) ( 𝑏 − 𝑏 ) ( 𝑐 + 𝑐 − 𝑐 − 𝑐 )+ ( 𝑎 + 𝑎 + 𝑎 ) ( 𝑏 − 𝑏 ) ( 𝑐 + 𝑐 + 𝑐 − 𝑐 )+ ( 𝑎 + 𝑎 + 𝑎 ) ( 𝑏 + 𝑏 + 𝑏 ) (cid:18) 𝑐 + 𝑐 − 𝑐 + 𝑐 (cid:19) + ( 𝑎 − 𝑎 − 𝑎 ) ( 𝑏 + 𝑏 ) ( 𝑐 − 𝑐 − 𝑐 )+ ( 𝑎 − 𝑎 − 𝑎 ) ( 𝑏 − 𝑏 + 𝑏 ) ( 𝑐 − 𝑐 − 𝑐 )+ ( 𝑎 + 𝑎 − 𝑎 ) ( 𝑏 + 𝑏 ) ( 𝑐 + 𝑐 + 𝑐 )+ ( 𝑎 − 𝑎 + 𝑎 ) ( 𝑏 − 𝑏 ) ( 𝑐 + 𝑐 − 𝑐 )+ ( 𝑎 + 𝑎 + 𝑎 ) ( 𝑏 − 𝑏 ) ( 𝑐 + 𝑐 − 𝑐 )+ ( 𝑎 + 𝑎 ) (cid:18) 𝑏 + 𝑏 − 𝑏 − 𝑏 − 𝑏 − 𝑏 (cid:19) ( 𝑐 + 𝑐 + 𝑐 )+ ( 𝑎 + 𝑎 + 𝑎 ) ( 𝑏 − 𝑏 + 𝑏 ) ( 𝑐 − 𝑐 + 𝑐 )+ (cid:18) 𝑎 + 𝑎 − 𝑎 − 𝑎 − 𝑎 (cid:19) ( 𝑏 + 𝑏 ) ( 𝑐 − 𝑐 − 𝑐 )+ ( 𝑎 − 𝑎 + 𝑎 ) (− 𝑏 − 𝑏 + 𝑏 + 𝑏 ) ( 𝑐 − 𝑐 )+ ( 𝑎 + 𝑎 ) ( 𝑏 + 𝑏 ) ( 𝑐 + 𝑐 )+ ( 𝑎 + 𝑎 ) ( 𝑏 − 𝑏 + 𝑏 ) ( 𝑐 − 𝑐 )+ ( 𝑎 − 𝑎 + 𝑎 ) ( 𝑏 − 𝑏 + 𝑏 + 𝑏 ) ( 𝑐 − 𝑐 )+ ( 𝑎 + 𝑎 − 𝑎 ) ( 𝑏 + 𝑏 − 𝑏 ) ( 𝑐 − 𝑐 ) . (26) 𝜏 = (cid:18) 𝑎 − 𝑎 + 𝑎 + 𝑎 − 𝑎 − 𝑎 + 𝑎 (cid:19) ( 𝑏 − 𝑏 + 𝑏 ) 𝑐 . (27) 𝜏 = − ( 𝑎 + 𝑎 ) ( 𝑏 + 𝑏 ) (cid:18) 𝑐 − 𝑐 − 𝑐 + 𝑐 − 𝑐 + 𝑐 (cid:19) . (28) 𝜏 = (cid:18) 𝑎 + 𝑎 + 𝑎 + 𝑎 + 𝑎 − 𝑎 − 𝑎 (cid:19) 𝑏 ( 𝑐 + 𝑐 + 𝑐 )+ (cid:18) 𝑎 + 𝑎 − 𝑎 − 𝑎 − 𝑎 − 𝑎 − 𝑎 (cid:19) 𝑏 ( 𝑐 − 𝑐 + 𝑐 )+ (cid:18) 𝑎 − 𝑎 + 𝑎 + 𝑎 − 𝑎 − 𝑎 + 𝑎 (cid:19) 𝑏 ( 𝑐 − 𝑐 )+ (cid:18) 𝑎 − 𝑎 + 𝑎 − 𝑎 + 𝑎 + 𝑎 + 𝑎 (cid:19) 𝑏 ( 𝑐 + 𝑐 )+ (cid:18) 𝑎 + 𝑎 − 𝑎 − 𝑎 − 𝑎 − 𝑎 − 𝑎 (cid:19) 𝑏 ( 𝑐 − 𝑐 ) . (29) 𝜏 = ( 𝑎 − 𝑎 ) 𝑏 (cid:18) 𝑐 + 𝑐 + 𝑐 + 𝑐 + 𝑐 − 𝑐 − 𝑐 + 𝑐 + 𝑐 + 𝑐 (cid:19) − ( 𝑎 + 𝑎 ) 𝑏 (cid:18) 𝑐 − 𝑐 + 𝑐 + 𝑐 + 𝑐 − 𝑐 − 𝑐 (cid:19) + ( 𝑎 + 𝑎 ) ( 𝑏 − 𝑏 ) (cid:18) 𝑐 − 𝑐 + 𝑐 + 𝑐 − 𝑐 − 𝑐 + 𝑐 (cid:19) + ( 𝑎 − 𝑎 ) 𝑏 (cid:18) 𝑐 + 𝑐 − 𝑐 − 𝑐 + 𝑐 + 𝑐 + 𝑐 (cid:19) − ( 𝑎 + 𝑎 ) 𝑏 ( 𝑐 + 𝑐 + 𝑐 − 𝑐 − 𝑐 − 𝑐 ) . (30) 𝜏 = ( 𝑎 − 𝑎 + 𝑎 ) ( 𝑏 + 𝑏 + 𝑏 + 𝑏 − 𝑏 ) 𝑐 − ( 𝑎 + 𝑎 + 𝑎 ) ( 𝑏 + 𝑏 + 𝑏 − 𝑏 ) 𝑐 . (31) 𝜏 = 𝑎 ( 𝑏 + 𝑏 − 𝑏 − 𝑏 ) ( 𝑐 + 𝑐 − 𝑐 )+ 𝑎 ( 𝑏 − 𝑏 + 𝑏 + 𝑏 ) ( 𝑐 + 𝑐 + 𝑐 ) . (32) 𝜏 = ( 𝑎 − 𝑎 ) (cid:18) 𝑏 + 𝑏 − 𝑏 + 𝑏 + 𝑏 + 𝑏 − 𝑏 (cid:19) 𝑐 . (33) 𝜏 = 𝑎 ( 𝑏 − 𝑏 ) 𝑐 − 𝑎 (− 𝑏 + 𝑏 + 𝑏 + 𝑏 ) ( 𝑐 + 𝑐 )+ 𝑎 ( 𝑏 + 𝑏 − 𝑏 + 𝑏 ) ( 𝑐 + 𝑐 )+ 𝑎 ( 𝑏 + 𝑏 + 𝑏 + 𝑏 ) ( 𝑐 − 𝑐 )+ 𝑎 ( 𝑏 − 𝑏 − 𝑏 ) ( 𝑐 + 𝑐 )+ 𝑎 ( 𝑏 + 𝑏 + 𝑏 − 𝑏 − 𝑏 ) ( 𝑐 + 𝑐 )+ 𝑎 ( 𝑏 + 𝑏 ) ( 𝑐 + 𝑐 )+ 𝑎 ( 𝑏 + 𝑏 − 𝑏 ) 𝑐 + 𝑎 ( 𝑏 + 𝑏 + 𝑏 − 𝑏 ) 𝑐 + 𝑎 ( 𝑏 + 𝑏 + 𝑏 − 𝑏 − 𝑏 ) 𝑐 + 𝑎 ( 𝑏 + 𝑏 − 𝑏 ) 𝑐 − ( 𝑎 + 𝑎 ) ( 𝑏 + 𝑏 − 𝑏 + 𝑏 ) 𝑐 + ( 𝑎 − 𝑎 ) ( 𝑏 + 𝑏 + 𝑏 + 𝑏 − 𝑏 ) 𝑐 . (34) 𝜏 = ( 𝑎 + 𝑎 ) 𝑏 𝑐 + ( 𝑎 + 𝑎 + 𝑎 − 𝑎 − 𝑎 − 𝑎 ) 𝑏 ( 𝑐 − 𝑐 )+ ( 𝑎 + 𝑎 + 𝑎 ) 𝑏 ( 𝑐 + 𝑐 − 𝑐 − 𝑐 )+ ( 𝑎 + 𝑎 ) 𝑏 ( 𝑐 − 𝑐 − 𝑐 )+ 𝑎 ( 𝑏 + 𝑏 + 𝑏 − 𝑏 ) ( 𝑐 + 𝑐 + 𝑐 + 𝑐 )+ ( 𝑎 + 𝑎 − 𝑎 − 𝑎 ) ( 𝑏 + 𝑏 − 𝑏 ) 𝑐 + ( 𝑎 − 𝑎 + 𝑎 ) 𝑏 ( 𝑐 + 𝑐 + 𝑐 ) . (35) he tensor rank of × matrices multiplicationis bounded by and its border rank by Induced geometry.
As shown by Theorem 2.1, the action of thegroup ( psl ( ) × psl ( ) × psl ( )) ⋊ 𝐶 on the tensor introduced inthe previous section defines classically a manifold of dimension of tensors ⟨ × × ⟩ . Furthermore, this tensor have serendip-itous equalities that is couple of summands that shares the samefactor (e.g. the two first summands of Equation 35 share the fac-tor 𝑏 ). Up to our knowledge, this property was first introducedin [27, § 9.3] but does not seem to receive the attention it deserves.For example, this property allows to define new transformationof a matrix multiplication tensor into another as shown by thefollowing lemma:Lemma 3.1. Given any invertible 𝑞 × 𝑞 -matrix 𝑀 , the tensor withserendipitous equalities (cid:205) 𝑞𝑖 = 𝑈 𝑖 ⊗ 𝑉 𝑖 ⊗ 𝑊 involving the component 𝑊 is equal to the tensor (cid:205) 𝑞𝑖 = 𝛼 𝑖 ⊗ 𝛽 𝑖 ⊗ 𝑊 defined by: (cid:32) 𝛼 ... 𝛼 𝑞 (cid:33) = Transpose ( 𝑀 ) (cid:169)(cid:173)(cid:171) 𝑈 ... 𝑈 𝑞 (cid:170)(cid:174)(cid:172) , (cid:169)(cid:173)(cid:171) 𝛽 ... 𝛽 𝑞 (cid:170)(cid:174)(cid:172) = 𝑀 − (cid:169)(cid:173)(cid:171) 𝑉 ... 𝑉 𝑞 (cid:170)(cid:174)(cid:172) . (36)The proof of this lemma reduces to the trivial computation ofthe expression (cid:205) 𝑞𝑖 = 𝑈 𝑖 ⊗ 𝑉 𝑖 ⊗ 𝑊 − 𝛼 𝑖 ⊗ 𝛽 𝑖 ⊗ 𝑊 .This lemma shows that the dimension of the manifold inducedby the tensor introduced in this section is greater then that couldbe expected.The following sections are devoted to present other interestingconsequences. on tensor rankof ⟨ × × ⟩ Lemma 2.1 and the tensor presented in Section 3.2 allows to con-struct the following tensor: ⟨ × × ⟩ = ⟨ × × ⟩ ⊕ ⟨ × × ⟩ , (37)with the construction of tensor ⟨ × × ⟩ taken from [14] (itsexplicit expression is given in [23]). Remark that the best theoret-ical lower bound on the corresponding tensor rank is (see [3,Theorem 2]).Furthermore, this new atomic tensor also improves the construc-tion of the following algorithms: ⟨ × ×
10 : 686 ⟩ = ⟨ × × ⟩ ⊗ ⟨ × × ⟩ , (38) ⟨ × ×
15 : 2088 ⟩ = ⟨ × × ⟩ ⊗ ⟨ × × ⟩ . (39)All these tensors are explicitly presented via [23]. Consequence of this new upper bound.
A group-theoretic ap-proach to the conception of matrix multiplication algorithm relatedto Fourier transform on finite groups was introduced by Cohn andUmans in [6]. The new tensor constructed in Equation (37) allowsto exhibit a limitation of this approach as shown by the followingremark.Remark 3.2.
It is shown in [11] that no group can realize × ma-trix multiplication better then Makarov’s algorithm ⟨ × × ⟩ using the group-theoretic approach of Cohn and Umans [6]. Hence,the tensor presented in this note shows that this approach does not pro-duce better algorithms then ⟨ × × ⟩ . The same assertion holdsfor ⟨ × × ⟩ and ⟨ × × ⟩ (see [12, Theorem 7.3]). The next section is devoted to describe a new approximate algo-rithm { × × } . Approximate matrix multiplication tensors were first introducedby Bini et ali in [2] in order to improve asymptotic bounds. Froma practical point of view, these approximate algorithms could beused efficiently when the coefficients are in Z / 𝑝 Z (see [4]). Fur-thermore, from a theoretical point of view, these tensors allow towork with Euclidean closure of the Brent algebraic variety definedby Equation (19) and not the Zariski closure induced by dealingwith exact tensors. As the Zariski closure is often much largerthen the Euclidean closure, this shift of standpoint brings usuallylower bounds. Hence, the exact tensor ⟨ × × ⟩ presentedin [14] is optimal; Smirnov describes { × × } in [26] andthat bound on the corresponding border rank was proved to beoptimal in [7, Theorem 1.4]. Similarly, while the best upper boundfor tensor rank of ⟨ × × ⟩ is ([15]), { × × } could befound in [26] ([7, Theorem 1.1] reports that the lower bound is for the corresponding border rank).Lemma 2.1 shows that these atomic approximate matrix multi-plication tensors lead to the following tensor used in the sequel: { × × } = { × × } ⊕ { × × } . (40)The results in [2] are based on a partial matrix multiplication algo-rithm that computes approximately the product of a × -matrix 𝐴 with one element vanishing (e.g. 𝑎 = ) and a × -matrix 𝐵 afull matrix. This kind of tensor were used to improve the bound onthe exponent of matrix multiplication (see [21, § 3]).Let us now show how to complete a tensor constructed usingEquation 40 in order to define { × × } . This section presents a partial approximate tensor T 𝜖 that definesan algorithm computing the product of a × -matrix 𝐴 with vanishing elements: 𝑎 𝑖 𝑗 = , ∀( 𝑖, 𝑗 ) such that ≤ 𝑖 ≤ , ≤ 𝑗 ≤ . (41)and a full × -matrix 𝐵 . The type of T 𝜖 . Remarks that the isotropy introduced in Theo-rem 2.1 and the associated the invariant presented in Definition 2.1for exact matrix multiplication tensors remain obviously valid forapproximates ones. The type of T 𝜖 is: 𝑋 𝑌 𝑍 + 𝑋 𝑌 𝑍 + 𝑋 𝑌𝑍 + 𝑋𝑌 𝑍 + 𝑋 𝑌𝑍 + 𝑋𝑌 𝑍 + 𝑋𝑌𝑍 + 𝑋𝑌𝑍 . (42) lexandre Sedoglavic and Alexey V. Smirnov
Again, we are going to split its description by trilinear form intoseveral summands whose subscript indicate the type of their com-ponents as follow: T 𝜖 = 𝜌 + 𝜌 + 𝜌 + 𝜌 + 𝜌 + 𝜌 + 𝜌 + 𝜌 . (43) 𝜌 = (cid:16) 𝑎 − 𝑎 𝜖 (cid:17) (cid:16) 𝑏 − 𝑏 − 𝑏 𝜖 (cid:17) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) + (cid:16) 𝑎 + 𝑎 𝜖 (cid:17) (cid:169)(cid:173)(cid:171) 𝑏 + 𝑏 − 𝑏 + 𝑏 𝜖 + 𝑏 − 𝑏 𝜖 (cid:170)(cid:174)(cid:172) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) + (cid:16) 𝑎 + 𝑎 + 𝑎 𝜖 (cid:17) (cid:16) 𝑏 + 𝑏 𝜖 (cid:17) (cid:16) 𝑐 + 𝑐 𝜖 + 𝑐 𝜖 (cid:17) + (cid:16) 𝑎 − 𝑎 + 𝑎 𝜖 (cid:17) (cid:16) 𝑏 + 𝑏 𝜖 (cid:17) (cid:16) 𝑐 − 𝑐 − 𝑐 𝜖 (cid:17) + (cid:32) 𝑎 − 𝑎 + 𝑎 𝜖 + 𝑎 𝜖 (cid:33) (cid:16) 𝑏 − 𝑏 𝜖 (cid:17) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) + ( 𝑎 + 𝑎 ) (cid:18) 𝑏 − 𝑏 𝜖 + 𝑏 𝜖 (cid:19) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) + ( 𝑎 − 𝑎 ) (cid:16) 𝑏 + 𝑏 𝜖 − 𝑏 𝜖 (cid:17) (cid:16) 𝑐 + 𝑐 𝜖 + 𝑐 𝜖 (cid:17) + ( 𝑎 + 𝑎 ) (cid:18) 𝑏 + 𝑏 𝜖 (cid:19) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) + ( 𝑎 + 𝑎 ) (cid:18) 𝑏 + 𝑏 𝜖 (cid:19) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) + ( 𝑎 + 𝑎 ) (cid:18) 𝑏 + 𝑏 𝜖 (cid:19) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) + ( 𝑎 + 𝑎 ) (cid:18) 𝑏 + 𝑏 𝜖 (cid:19) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) + ( 𝑎 − 𝑎 ) (cid:18) 𝑏 + 𝑏 𝜖 (cid:19) (cid:16) 𝑐 − 𝑐 𝜖 (cid:17) + ( 𝑎 − 𝑎 ) (cid:18) 𝑏 + 𝑏 𝜖 (cid:19) (cid:16) 𝑐 − 𝑐 𝜖 (cid:17) + ( 𝑎 − 𝑎 ) (cid:18) 𝑏 + 𝑏 𝜖 (cid:19) (cid:16) 𝑐 − 𝑐 𝜖 (cid:17) + ( 𝑎 + 𝑎 ) (cid:18) 𝑏 − 𝑏 𝜖 (cid:19) (cid:16) 𝑐 − 𝑐 𝜖 (cid:17) + ( 𝑎 + 𝑎 ) (cid:18) 𝑏 − 𝑏 𝜖 (cid:19) (cid:16) 𝑐 − 𝑐 𝜖 (cid:17) + ( 𝑎 + 𝑎 ) (cid:18) 𝑏 − 𝑏 𝜖 (cid:19) (cid:16) 𝑐 − 𝑐 𝜖 (cid:17) + ( 𝑎 + 𝑎 ) (cid:18) 𝑏 − 𝑏 𝜖 (cid:19) (cid:16) 𝑐 − 𝑐 𝜖 (cid:17) + ( 𝑎 − 𝑎 ) (cid:18) 𝑏 − 𝑏 𝜖 (cid:19) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) + ( 𝑎 − 𝑎 ) (cid:18) 𝑏 − 𝑏 𝜖 (cid:19) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) . (44) 𝜌 = (cid:16) 𝑎 𝜖 − 𝑎 + 𝑎 𝜖 (cid:17) 𝑏 (cid:16) 𝑐 + 𝑐 𝜖 − 𝑐 𝜖 − 𝑐 𝜖 (cid:17) + ( 𝑎 − 𝑎 ) 𝑏 (cid:16) 𝑐 − 𝑐 𝜖 − 𝑐 𝜖 (cid:17) . (45) 𝜌 = (cid:16) 𝑎 − 𝑎 𝜖 − 𝑎 𝜖 − 𝑎 𝜖 (cid:17) (cid:16) 𝑏 + 𝑏 𝜖 + 𝑏 𝜖 (cid:17) 𝑐 + (cid:16) 𝑎 + 𝑎 𝜖 (cid:17) (cid:16) 𝑏 + 𝑏 − 𝑏 + 𝑏 𝜖 + 𝑏 𝜖 (cid:17) 𝑐 − (cid:16) 𝑎 + 𝑎 𝜖 (cid:17) (cid:16) 𝑏 + 𝑏 𝜖 + 𝑏 𝜖 − 𝑏 𝜖 (cid:17) 𝑐 . (46) 𝜌 = 𝑎 (cid:18) 𝑏 + 𝑏 + 𝑏 − 𝑏 + 𝑏 𝜖 − 𝑏 (cid:19) 𝑐 + (cid:16) 𝑎 + 𝑎 𝜖 (cid:17) (cid:16) 𝑏 − 𝑏 𝜖 + 𝑏 𝜖 (cid:17) 𝑐 + (cid:16) 𝑎 + 𝑎 𝜖 (cid:17) (cid:16) 𝑏 + 𝑏 + 𝑏 𝜖 + 𝑏 𝜖 (cid:17) 𝑐 + 𝑎 (cid:18) 𝑏 + 𝑏 − 𝑏 − 𝑏 + 𝑏 + 𝑏 − 𝑏 + 𝑏 𝜖 (cid:19) 𝑐 − 𝑎 (cid:18) 𝑏 + 𝑏 − 𝑏 − 𝑏 + 𝑏 + 𝑏 − 𝑏 + 𝑏 𝜖 (cid:19) 𝑐 − 𝑎 (cid:18) 𝑏 + 𝑏 + 𝑏 − 𝑏 + 𝑏 + 𝑏 − 𝑏 + 𝑏 𝜖 (cid:19) 𝑐 . (47) 𝜌 = 𝑎 (cid:16) 𝑏 + 𝑏 𝜖 (cid:17) (cid:169)(cid:173)(cid:173)(cid:171) 𝑐 − 𝑐 − 𝑐 + 𝑐 − 𝑐 − 𝑐 𝜖 + 𝑐 𝜖 + 𝑐 − 𝑐 𝜖 (cid:170)(cid:174)(cid:174)(cid:172) + 𝑎 𝜖 (cid:169)(cid:173)(cid:173)(cid:173)(cid:171) 𝑏 − 𝑏 − 𝑏 + 𝑏 𝜖 + 𝑏 + 𝑏 − 𝑏 𝜖 + 𝑏 − 𝑏 𝜖 (cid:170)(cid:174)(cid:174)(cid:174)(cid:172) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) + 𝑎 𝜖 (cid:18) 𝑏 − 𝑏 𝜖 (cid:19) (cid:169)(cid:173)(cid:171) 𝑐 − 𝑐 − 𝑐 + 𝑐 𝜖 + 𝑐 + 𝑐 𝜖 (cid:170)(cid:174)(cid:172) + 𝑎 𝜖 (cid:18) 𝑏 𝜖 + 𝑏 + 𝑏 𝜖 (cid:19) (cid:16) 𝑐 − 𝑐 𝜖 + 𝑐 (cid:17) . (48) 𝜌 = (cid:16) 𝑎 + 𝑎 − 𝑎 𝜖 (cid:17) (cid:16) 𝑏 − 𝑏 𝜖 (cid:17) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) + (cid:16) 𝑎 + 𝑎 − 𝑎 𝜖 (cid:17) (cid:16) 𝑏 − 𝑏 𝜖 (cid:17) (cid:16) 𝑐 − 𝑐 𝜖 (cid:17) + (cid:16) 𝑎 − 𝑎 − 𝑎 − 𝑎 + 𝑎 𝜖 (cid:17) 𝑏 (cid:16) 𝑐 − 𝑐 𝜖 (cid:17) + (cid:16) 𝑎 + 𝑎 + 𝑎 𝜖 (cid:17) (cid:16) 𝑏 + 𝑏 𝜖 (cid:17) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) + (cid:16) 𝑎 + 𝑎 − 𝑎 𝜖 (cid:17) (cid:16) 𝑏 − 𝑏 𝜖 (cid:17) (cid:16) 𝑐 + 𝑐 𝜖 (cid:17) + (cid:16) 𝑎 − 𝑎 𝜖 (cid:17) (cid:18) 𝑏 𝜖 + 𝑏 (cid:19) (cid:16) 𝑐 𝜖 + 𝑐 (cid:17) + (cid:16) 𝑎 + 𝑎 − 𝑎 + 𝑎 𝜖 (cid:17) (cid:16) 𝑏 − 𝑏 𝜖 (cid:17) 𝑐 . (49) he tensor rank of × matrices multiplicationis bounded by and its border rank by 𝜌 = 𝑎 𝑏 (cid:16) 𝑐 + 𝑐 − 𝑐 − 𝑐 + 𝑐 + 𝑐 − 𝑐 − 𝑐 𝜖 (cid:17) − 𝑎 𝑏 (cid:16) 𝑐 + 𝑐 − 𝑐 − 𝑐 + 𝑐 + 𝑐 − 𝑐 − 𝑐 𝜖 (cid:17) − 𝑎 𝑏 (cid:16) 𝑐 + 𝑐 − 𝑐 − 𝑐 + 𝑐 + 𝑐 − 𝑐 − 𝑐 𝜖 (cid:17) − 𝑎 𝑏 (cid:16) 𝑐 + 𝑐 − 𝑐 − 𝑐 − 𝑐 − 𝑐 𝜖 (cid:17) − 𝑎 𝑏 (cid:16) 𝑐 𝜖 + 𝑐 + 𝑐 − 𝑐 𝜖 (cid:17) + 𝑎 𝑏 (cid:16) 𝑐 + 𝑐 − 𝑐 𝜖 + 𝑐 𝜖 (cid:17) + (cid:16) 𝑎 − 𝑎 𝜖 (cid:17) 𝑏 (cid:16) 𝑐 − 𝑐 𝜖 − 𝑐 − 𝑐 𝜖 (cid:17) + (cid:16) 𝑎 − 𝑎 𝜖 (cid:17) (cid:18) 𝑏 + 𝑏 𝜖 − 𝑏 𝜖 (cid:19) (cid:16) 𝑐 − 𝑐 𝜖 + 𝑐 𝜖 (cid:17) . (50) 𝜌 = (cid:16) 𝑎 − 𝑎 𝜖 (cid:17) (cid:16) 𝑏 − 𝑏 𝜖 (cid:17) (cid:16) ( 𝑐 + 𝑐 − 𝑐 ) 𝜖 + 𝑐 (cid:17) + (cid:16) 𝑎 𝜖 + 𝑎 (cid:17) (cid:18) 𝑏 𝜖 + 𝑏 𝜖 (cid:19) (cid:16) 𝑐 𝜖 + 𝑐 (cid:17) + (cid:16) 𝑎 − 𝑎 𝜖 (cid:17) (cid:18) 𝑏 − 𝑏 + 𝑏 − 𝑏 𝜖 (cid:19) (cid:16) 𝑐 − 𝑐 𝜖 (cid:17) + 𝑎 𝑏 𝑐 + 𝑎 𝑏 𝑐 . (51)Remark that there is serendipitous equalities in this tensor.We conclude the construction of an approximate tensor { × × } in the next section. on border rankof ⟨ × × ⟩ Using the approximate matrix multiplication tensor defined in theprevious section and the construction made in Equation (40), onecan construct easily: { × × } = T 𝜖 + { × × } + { × × } . (52)Remark that the best theoretical lower bound on the correspondingborder rank is (see [17, Corollary 1.2])). We have presented there the upper bound (resp. ) for thetensor (resp. border) rank of the × -matrix product while thebest theoretical lower bound is (resp. ) (see [3, Theorem 2](resp. [17, Corollary 1.2])).Furthermore, the new ⟨ × × ⟩ improves also • ⟨ × ×
10 : 686 ⟩ = ⟨ × × ⟩ ⊗ ⟨ × × ⟩• ⟨ × ×
15 : 2088 ⟩ = ⟨ × × ⟩ ⊗ ⟨ × × ⟩ and we have a quite clear idea of all exact matrix multiplicationtensors up to size × × induced by the best known-to-date“atomic” such tensors. But as ⟨ × ×
15 : 2088 ⟩ ’s tensor rank islesser then the first decomposition ⟨ × × ⟩ ⊗ ⟨ × × ⟩ that comes to mind, the most obvious decomposition does not al-ways lead to the current most efficient one. Hence, there is a—simplebut requiring expensive calculation—computer based search to doin order to construct a database for approximate tensor similarto [23]. REFERENCES [1] J. Alman and V. V. Williams. A refined laser method and faster matrix multiplica-tion. Technical Report 2010.05846, arXiv, Oct. 2020. arXiv:2010.05846 .[2] D. Bini, M. Capovani, F. Romani, and G. Lotti. 𝑂 ( 𝑛 . ) complexity for 𝑛 × 𝑛 approximate matrix multiplication. Information Processing Letters , 8(5):234–235,June 1979. doi:10.1016/0020-0190(79)90113-3 .[3] M. Bläser. A 𝑛 -lower bound for the rank of 𝑛 × 𝑛 -matrix multiplication overarbitrary fields. In Proceedings of the 40th Annual Symposium on Foundations ofComputer Science , pages 45–50, New York City, New York, USA, Oct. 17-18 1999.IEEE. doi:10.1109/SFFCS.1999.814576 .[4] B. Boyer and J.-G. Dumas. Matrix multiplication over word-size modular ringsusing approximate formulae. Technical Report 00987812, hal, May 2014.[5] R. P. Brent. Algorithms for matrix multiplication. Technical Report STAN-CS-70-157, Computer Science Departement. Standford university, Mar. 1970. URL:http://i.stanford.edu/pub/cstr/reports/cs/tr/70/157/CS-TR-70-157.pdf.[6] H. Cohn and C. Umans. A group-theoretic approach to fast matrix multiplication.In
Proceedings of the 44th annual Symposium on Foundation of Computer Science ,pages 438–449. IEEE Computer Society, Oct. 11-14 2003. arXiv:arXiv:math.GR/0307321 , doi:10.1109/SFCS.2003.1238217 .[7] A. Conner, A. Harper, and J. M. Landsberg. New lower bounds for matrix mul-tiplication and the 3x3 determinant. Technical Report 1911.07981, arXiv, Nov.2019.[8] C.-É. Drevet, M. Nazrul Islam, and É. Schost. Optimization techniques for smallmatrix multiplication. Theoretical Computer Science , 412(22):2219–2236, May2011. doi:10.1016/j.tcs.2010.12.012 .[9] H. F. Groote, de. On varieties of optimal algorithms for the computation of bilinearmappings I. The isotropy group of a bilinear mapping.
Theoretical ComputerScience , 7(2):1–24, 1978. doi:10.1016/0304-3975(78)90038-5 .[10] H. F. Groote, de. On varieties of optimal algorithms for the computation of bilinearmappings II. Optimal algorithms for × -matrix multiplication. TheoreticalComputer Science , 7(2):127–148, 1978. doi:10.1016/0304-3975(78)90045-2 .[11] S. Hart, I. Hedtke, M. Müller-Hannemann, and S. Murthy. A fast search algorithmfor (m,m,m) triple product property triples and application for × matrixmultiplication. Technical Report 1305.0448, arXiv, May 2013. arXiv:arXiv:1305.0448 .[12] I. Hedtke and S. Murthy. Search and test algorithms for triple product propertytriples. Groups Complexity Cryptology , 4(1):111–133, Jan. 2012. arXiv:arXiv:1104.5097 , doi:10.1515/gcc-2012-0006 .[13] M. J. Heule, M. Kauers, and M. Seidl. New ways to multiply 3 x 3-matrices.Technical Report 1905.10192, arXiv, May 2019. arXiv:arxiv.org:1905.10192 .[14] J. E. Hopcroft and L. R. Kerr. On minimizing the number of multiplicationnecessary for matrix multiplication. SIAM Journal on Applied Mathematics , 20(1),Jan. 1971. doi:10.1137/0120004 .[15] J. D. Laderman. A noncommutative algorithm for multiplying × matricesusing multiplications. Bulletin of the American Mathematical Society , 82(1):126–128, Jan. 1976. doi:10.1090/S0002-9904-1976-13988-2 .[16] J. M. Landsberg.
Geometry and complexity theory , volume 169 of
CambridgeStudies in Advanced Mathematics . Cambrigde University Press, Dec. 2016. doi:10.1017/9781108183192 .[17] J. M. Landsberg and G. Ottaviani. New lower bounds for the border rank ofmatrix multiplication.
Theory of computing , 11:285–298, Aug. 2015. arXiv:arXiv:1206.1530 , doi:10.4086/toc.2015.v011a011 .[18] O. M. Makarov. A noncommutative algorithm for multiplying × matricesusing multiplications. Information Processing Letters , 23(3):115–117, Oct.1986. doi:10.1016/0020-0190(86)90108-0 .[19] O. M. Makarov. A non-commutative algorithm for multiplying × matricesusing one hundred multiplications. USSR Computational Mathematics and Mathe-matical Physics , 27(1):205–207, May 1987. doi:10.1016/0041-5553(87)90145-5 .[20] G. Schachtel. A noncommutative algorithm for multiplying × matricesusing multiplications. Information Processing Letters , 7(4):180–182, June 1978. doi:10.1016/0020-0190(78)90063-7 .[21] A. Schönhage. Partial and total matrix multiplication.
SIAM Journal on Computing ,10(3), Aug. 1981. doi:10.1137/0210032 .[22] A. Sedoglavic. A non-commutative algorithm for multiplying × matricesusing multiplications. Technical Report 1707.06860, arXiv, July 2017. arXiv:arXiv:1707.06860 .[23] A. Sedoglavic. Fast matrix multiplication database [online]. Dec. 2017. URL:http://fmm.univ-lille.fr/.[24] A. Sedoglavic. A non-commutative algorithm for multiplying × matricesusing multiplications. Technical Report hal-01572046, hal, Aug. 2017. URL:https://hal.archives-ouvertes.fr/hal-01572046v2.[25] Y. Shitov. A counterexample to comon’s conjecture. Technical Report 1705.08740,arXiv, May 2017. arXiv:1705.08740 .[26] A. V. Smirnov. The bilinear complexity and practical algorithms for matrixmultiplication. Computational Mathematics and Mathematical Physics , 53(2):1781–1795, Dec. 2013. doi:10.1134/S0965542513120129 . lexandre Sedoglavic and Alexey V. Smirnov [27] W. D. Smith. Fast matrix multiplication formulae. Technical report, 2002.[28] V. Strassen. Gaussian elimination is not optimal. Numerische Mathematik ,13(4):354–356, Aug. 1969. doi:10.1007/BF02165411 . [29] S. Winograd. On multiplication of × matrices. Linear algebra and its applica-tion , 4(4):381 – 388, Oct. 1971. doi:10.1016/0024-3795(71)90009-7doi:10.1016/0024-3795(71)90009-7