On the Complexity of Solving Generic Over-determined Bilinear Systems
aa r X i v : . [ c s . S C ] J un On the Complexity of Solving Generic Over-determined BilinearSystems
John B. Baena [email protected]
Daniel Cabarcas [email protected]
Javier Verbel [email protected]
Abstract
In this paper, we study the complexity of solving generic over-determined bilinear systems overa finite field F . Given a generic bilinear sequence B ∈ F [ x , y ], with respect to a partition ofvariables x , y , we show that, the solutions of the system B = can be e ffi ciently found on the F [ y ]-module generated by B . Following this observation, we propose three variations of Gr¨obnerbasis algorithms, that only involve multiplication by monomials in the y -variables, namely, y -XL, based on the XL algorithm, y -MLX, based on the mutant XL algorithm, and y -HXL, basedon a hybrid approach. We define notions of regularity for over-determined bilinear systems,that capture the idea of genericity, and we develop the necessary theoretical tools to estimatethe complexity of the algorithms for such sequences. We also present extensive experimentalresults, testing our conjecture, verifying our results, and comparing the complexity of the variousmethods. Keywords:
Bilinear systems, y -Degree of regularity, Complexity.
1. Introduction
The problem of solving systems of polynomial equations has many important applicationsall over science and engineering. The main abstraction to tackle the problem is the Gr¨obner basis(Buchberger, 2006). Due to its importance, the past three decades have seen qualitative improve-ments in the algorithms to solve the problem and in the understanding of its complexity (Faugere,1999, 2002; Courtois et al., 2000a; Bardet, 2004; Mohamed et al., 2008; Buchmann et al., 2010).The complexity of the problem is very sensitive to the structure of the system. For genericsystems (characterized as regular or semi-regular sequences) the complexity of the problem is
Email addresses: [email protected] (John B. Baena), [email protected] (Daniel Cabarcas), [email protected] (Javier Verbel)
Preprint submitted to Journal of Symbolic Computation June 18, 2020 ell understood both classical (Bardet, 2004), and quantum complexity (Faug`ere et al., 2017;Bernstein and Yang, 2018). There are also several works that adapt to very particular types ofsystems and analyze their complexity (Kipnis and Shamir, 1999; Faug`ere et al., 2014; Verbel et al.,2019; Bardet et al., 2020).In this paper, we study the complexity of solving determined and over-determined bilinearsystems over a finite field. More precisely, we are interested on systems of the form B = ,where B is a sequence of m bilinear polynomials in n variables, with coe ffi cients on a finitefield F , and such that n ≤ m . By bilinear we mean that, there is a partition of the variables x = ( x , x , . . . , x n x ), y = ( y , y , . . . , y n y ) such that the quadratic part of every equation is somepolynomial f ( x , y ) such that for all λ, µ , f ( λ x , µ y ) = λµ f ( x , y ). The key observation that drives this work is that, for a generic bilinear system B = , itssolutions can be e ffi ciently found on the F [ y ]-module generated by B , denoted by I y ( B ). This isin contrast to the typical approach of looking for a solution on the ideal generated by B .Based on this observation, we propose variations of Gr¨obner basis algorithms that only in-volve multiplication by mononials in the y -variables. We propose three such algorithms, y -XL,based on the XL algorithm (Courtois et al., 2000b), y -MLX, based on the mutant XL algorithm(Cabarcas, 2011), and y -HXL, based on the so called hybrid approach (Bardet et al., 2011).In order to analyze the complexity of these algorithms, we study the structure of I y ( B ).By looking at the Jacobian of B with respect to x , we show that in the determined and over-determined cases there are non-trivial syzygies of degree strictly less than n x in F [ y ] m .We define a notion of d -regularity for homogeneous bilinear sequences by focusing on the F [ y ]-module I y ( B ). In the same line, we define a notion of degree of regularity that capturesthe idea of the minimum degree at which the Hilbert polynomial equals the Hilbert function, butlooking only at I y ( B ) instead of at the ideal. Supported on this notion of degree of regularity, wethen define a notion of y -semiregularity, applicable to determined or over-determiend bilinearsequences, as being d -regular for as high a degree d as possible. We compute this degree ofsuch y -semiregular sequences. And we conjecture, based on extensive experimental evidence,that, for fixed parameters n x , n y , and m , there exists an open Zariski set O , contained in the setof all homogeneous bilinear sequences, such that every sequence in O has this y -semiregularityproperty.The degree of regularity is an important value for measuring the complexity of Gr¨obner basisalgorithms, but it is not the only one. Assuming a sequence B is y -semiregular, we also calculatethe analogous of first fall degree and witness degree in I y ( B ). A subtle yet important contributionof this paper is a careful and clear explanation of each of these three degrees and the role theyplay in the complexity of di ff erent algorithms.We then compute the complexity of the three proposed algorithms for y -semiregular se-quences. We estimate that y -XL solves the system in O m n y + ˜ d − d − ! n x n y + ˜ d − d − ! ω − multiplications over F q , where 2 ≤ ω ≤ d = & n y ( n x + m − n x − ' + . y -MXL is O m n y + d y − f f ( B ) − d y − f f ( B ) − ! " n x n y + d y − f f ( B ) − d y − f f ( B ) − ! ω − multiplications over F q , where d y − f f ( B ) = min ( d ∈ Z + | d > n x ( n y − m − n x + ) . For a x , a y fixed, the complexity of y -HLX a x , a y using Weidemann’s algorithm is given by O q a x + a y ( n y − a y + n x − a x + n y − a y + ˜ d − d − ! and using Gaussian elimination is O q a x + a y m n y − a y + ˜ d − d − ! ( n x − a x + n y − a y + ˜ d − d − ! ω − where ˜ d is is given by ˜ d = & ( n y − a y )( n x − a x + m − n x + a x − ' + . We finally show extensive experimental evidence, testing our conjecture, verifying our re-sults, and comparing the complexity to that of out-of-the-box algorithms.
There are several works that have studied the solvability and complexity of the problem offinding solutions for bilinear systems of equations over any field E . To the best of our knowledge,the first specialized methods for solving bilinear systems date from late 90’s, with a work byCohen and Tomasi (1997). They studied the solvability of bilinear systems when E is the fieldof real numbers and proposed an algorithm to solve themVinh (2009) studied the solvability of bilinear systems over finite fields and provided esti-mates for the number of solutions. The same year, Johnson and Link proposed an algorithm forsolving bilinear systems over any field E (Johnson and Link, 2009). This is a deterministic andvery e ffi cient algorithm when m = n x n y . However, the algorithm is probabilistic and, accordingto our judgment, not e ffi cient for m < n x n y . Based on the ideas of Johnson and Link, Yang(2011) proposed an algorithm for solving bilinear systems over any field E , with m < n x n y . Forthis goal, a generic MinRank problem with n x n y − m + E with target rank 1 needsto be solved (Yang, 2011, Sec. 2.6). They do not provide complexity estimates.The complexity of solving a bilinear system over a finite field F via Gr¨obner basis algorithmsis analyzed in (Faug`ere et al., 2011). They use the F5 algorithm, extending the F5 Criterion toavoid reductions to zero during the Gr¨obner basis computation for bilinear ideals. The extendedcriterion is named BILINF5CRITERION, and it works for the under-determined and determinedcases, i.e, m ≤ n x + n y (Faug`ere et al., 2011, Prop. 1). They also provided an upper bound for the3egree of regularity, which is used to estimate the complexity of computing a Gr¨obner basis forthe zero-dimensional and determined bilinear systems. Their estimate is O n x + n y + min { n x + , n y + } min { n x + , n y + } ! ω ! . There are two main di ff erences between the approach followed by Faugre et al. and the onepresented in our work.The first one is that they analyze the behavior of computing a Gr¨obner basis algorithm forthe ideal I ⊂ F [ x , y ] generated by bilinear equations, while in this paper we analyze the samebehavior but for the F [ y ]-module, I y , generated by bilinear equations. The second di ff erence isthat in (Faug`ere et al., 2011) the complexity estimates are only meaningful when m = n x + n y butnot for the case n x + n y ≤ m as in this paper.Faug`ere et al. (2014) considered a Gr¨obner basis algorithm that does not use all monomialsfrom a polynomial ring. This method is applicable when the initially given polynomials aresparse and with the same support. They use only monomials that appear in the support of theinitial polynomials. For bilinear systems in F [ x , y ] that means multiplying by monomials ofdegree 2 d formed by a monomial of degree d in F [ x ] and a monomial of degree d in F [ y ]. Thisleads to a completely di ff erent approach to the one discussed in the present paper.In cryptography, the security of several schemes can be broken via solving a system of bi-linear equations (Kipnis and Shamir, 1999; Cabarcas et al., 2017; Vates and Smith-Tone, 2017;Bardet et al., 2020). The complexity of solving such systems has been studied in (Faug`ere et al.,2011; Verbel et al., 2019; Bardet et al., 2020). Verbel et al. (2019) and Bardet et al. (2020) pro-posed two di ff erent modified Gr¨obner basis algorithms for solving these particular bilinear sys-tems, which only involve one group of variables during the Gr¨obner basis computation. Thecomplexity of these algorithms relies on the structure of the F [ y ]-module I y , which in these casesis generated by very particular bilinear equations. In this paper, we consider generic bilinearequations. We do not expect that the results presented here provided a tight upper bound for thebilinear systems considered in (Verbel et al., 2019; Bardet et al., 2020). Instead, the present workgeneralizes the ideas of these papers.
2. Preliminaries
Throughout this paper, we adopt the following notation. • F denotes the field with q elements. • F a × b denotes the ring of matrices of size a × b with entries in F . We use bold uppercaseletters to denote matrices. Similarly, F c denotes the space of all vectors of length c withentries in F , and we use bold lowercase letters to denote vectors. The entry of a matrix A indexed by ( i , j ) is denoted by A [ i , j ]. • We distinguish two sets of variables, the x -variables and the y -variables, represented re-spectively by the tuples x = ( x . . . . , x n x ) and y = ( y , . . . , y n y ), with n x ≤ n y . • The polynomial ring in x -variables and y -variables over F is denoted by F [ x , y ]. It is dotedwith graded lexicographic monomial order, where x > · · · > x n x > y > · · · > y n y . F [ x ]( resp. F [ y ]) denote the subring of polynomials over F in the x -variables ( resp. y -variables).4 The degree of a sequence of polynomials S ∈ F [ x ] m is the maximum degree of the poly-nomials in S . • F [ x , y ] α,β denotes the set of homogeneous polynomials in F [ x , y ] of degree α + β , such thateach of their monomials has degree α in the x -variables and degree β in y -variables. • Let S be a sequence of polynomials in F [ x , y ] m . We say S is1. An under-determined sequence: if m < n x + n y .2. A determined sequence: if m = n x + n y .3. An over-determined sequence: if m > n x + n y . Definition 1 (Jacobian) . Let S = ( f , . . . , f m ) be a sequence of polynomials in F [ x , y ] . TheJacobian of S with respect to x is the matrix jac x ( S ) ∈ F [ x , y ] m × n x defined by jac x ( S )[ i , j ] = ∂ f i ∂ x j . The Jacobian jac y ( S ) is defined analogously. Definition 2 (Zariski Topology) . The Zariski topology on F k is the topology whose closed setsare the algebraic subsets of F k , i.e., all sets of the form n a ∈ F k | g ( a ) = , ∀ g ∈ S o for someS ⊂ F [ x , . . . , x k ] . An open Zariski set is the complement of an algebraic set. Each open set is expected to bevery large compared to F k because it is dense in F k (Hartshorne, 1977). The problem of solving a system of polynomial equations over a size q field F is closelyrelated to the problem of finding a Groebner Basis for an ideal in F [ x ]. For instance, when asequence of polynomials S ∈ F [ x ] m has a unique solution over F , this solution can be found byfinding a Groebner basis for the ideal generated by S ∪ n x qi − x i o n x i = .The most e ffi cient Groebner basis algorithms (eg. XL (Courtois et al., 2000b), Mutant XL(Cabarcas, 2011) and F4 (Faug`ere, 1999) follow an idea first explored by Lazard (1983). Theyamass most of the computation on finding a staggered basis for the row space of the Macaulaymatrix of S ∪ n x qi − x i o n x i = . Definition 3 (Macaulay matrix) . The Macaulay matrix of degree d of a sequence of polynomialsS ∈ F [ x ] m , denoted M ≤ d ( S ) , is the matrix formed by the coe ffi cients of all polynomials of the form m f , where f ∈ S and m ∈ F [ x ] is a monomial of degree at most d − deg( f ) . The columns of thematrix correspond to the monomials produced in all the products m f , and sorted in decreasingorder with respect to the grevlex ordering. For ≤ j ≤ d, we use M j ( S ) to denote the rowsubmatrix of M ≤ d ( S ) formed by taking only monomials m ∈ F [ x ] with degree exactly j − . Werefer to M j ( S ) as the degree j part of the Macaulay matrix M ≤ d ( S ) . The complexity of this kind of Gr¨obner basis algorithms depend on the degree up to whichthe Macaulay matrix must be constructed, often called the solving degree . In certain cases, thesolving degree can be approximated by values that do not depend on the algorithm used, but onlyon the ideal h S i itself, for instance, the degree of regularity , and the first fall degree .5 efinition 4 (Degree of regularity) . Let S be a sequence of homogeneous polynomials over F [ x ] m . The degree of regularity of S is defined asd reg ( S ) = min (cid:8) d ∈ Z + | dim ( I d ) = dim ( F [ x ] d ) (cid:9) , where I d and F [ x ] d are the F -vector spaces consisting of degree d polynomials in h S i and F [ x ] ,respectively. If S is not homogeneous, and ˜ S is the homogeneous part of S of highest degree,then d reg ( S ) : = d reg ( ˜ S ) . An equivalent way of defining d reg ( S ) is as the minimum integer d such that the dimension of I d is (cid:16) n x + d − d (cid:17) . Syzygies and trivial syzygies are crucial concepts in understanding the complexityof Groebner basis algorithms. Definition 5 (Syzygy) . A syzygy of a sequence of polynomials ( f , . . . , f m ) ∈ F [ x ] m is anothersequence of polynomials ( g , . . . , g m ) ∈ F [ x ] m satisfying P mi = g i f i = . Definition 6 (Trivial Syzygy) . Let S = ( f , . . . , f m ) ∈ F [ x ] m . For any ≤ i < j ≤ m, letT i , j = f i e j − f j e i , where e i is the i-th canonical basis vector. Any element in the F [ x ] -modulegenerated by the vectors n T i , j | ≤ i < j ≤ m o is called a trivial syzygy. The degree of regularity can be determined for a large family of sequences called semiregular.These are sequences with no relations among their polynomials except the trivial ones.
Definition 7 (semiregular) . A sequence S = ( f , . . . , f m ) ∈ F [ x ] m is called d-regular if for eachg ∈ F [ x ] and for all ≤ i ≤ m, the facts g f i ∈ h f , . . . , f i − i and deg( g f i ) < d imply f i ∈h f , . . . , f i − i . A semiregular sequence S is a sequence that is d reg ( S ) -regular. When S is a sequence of homogeneous quadratic polynomials, S is semiregular, if and onlyif, all the syzygies of S of degree less than d reg ( S ) − Definition 8 (First Fall Degree) . Let S = ( f , . . . , f m ) be a sequence of quadratic polynomials in F [ x ] . Let ˜ S be the sequence of polynomials formed by the homogeneous part of largest degree ofeach polynomial in S . The first fall degree of S , denote by d f f ( S ) , is the minimum integer d suchthat ˜ S has a non-trivial syzygy of degree d − . For most quadratic sequences S ∈ F [ x ] m the complexity of computing a Groebner basis forthe ideal I = h S i is given by O n x + dd ! ω ! , with d = d reg ( S ) = d f f ( S ). This is because most quadratic sequences S have semiregularquadratic part ˜ S . In this case, there are no non-trivial degree falls below d = d reg ( S ). Further-more, M d ( ˜ S ) spans F [ x ] d , hence d reg ( S ) = d f f ( S ), and also, for k > d , the leading term of anypolynomial in I k is divisible by some polynomial in the row space of M d ( S ). Definition 9 (Bilinear Polynomial) . A quadratic polynomial f ∈ F [ x , y ] is called bilinear withrespect to x and y if it can be written asf = xAy ⊤ + bx ⊤ + cy ⊤ + d , where A ∈ F n x × n y , b ∈ F n x , c ∈ F n y and d ∈ F . We use B ( n x , n y , m ) to denote the set of all lengthm bilinear sequences in F [ x , y ] , where there are n x x -variables and n y y -variables. The subset of B ( n x , n y , m ) consisting only of homogeneous sequences is denoted by B ( h ) ( n x , n y , m ) .
6e now define the notion of generic bilinear sequences, which captures the properties ofa sequence that only depend on the sequence being bilinear, without considering the particularcoe ffi cients appearing in the sequence. For instance, a property over F [ x ] that does not dependon the particular coe ffi cients used is: any two polynomials f , f ∈ F [ x ] satisfy f f − f f = generic bilinear properties , meaning that they are satisfiedby generic bilinear sequences. Definition 10.
Let a denote the set of parameters { a ki , j | ≤ i ≤ n x , ≤ j ≤ n y , ≤ k ≤ m } ∪ { a k ℓ | ≤ ℓ ≤ n x + n y , ≤ k ≤ m } , and let F ( a ) denote the ring of fractions of the polynomial ring F [ a ] . A generic bilinear sequence B ( a ) ∈ F ( a )[ x , y ] m is a sequence of polynomials ( f , . . . , f m ) , where for each ≤ k ≤ mf k = n x , n y X i , j = a ki , j x i y j + n x X i = a ki x i + n y X j = a kn x + j y j + a k . We say B ( a ) = ( f , . . . f m ) is a generic homogeneous bilinear sequence iff k = n x , n y X i , j = a ki , j x i y j . The following propositions, introduced by Verbel et al. (2019), highlight two generic bilinearproperties.
Proposition 11.
Let B ( a ) = ( f , f , . . . , f m ) be a generic homogeneous bilinear sequence in F ( a )[ x , y ] m . Suppose G = ( g , g , . . . , g m ) is a sequence in F [ y ] m , then, P mi = g i f i = , if and onlyif, G ⊤ belongs to the left-kernel of jac x ( B ( a )) . It is important to notice that, if a syzygy contains variables of only one set ( x or y ), then thatsyzygy cannot be trivial. Proposition 12.
Let B ( a ) be a generic homogeneous bilinear sequence in F ( a )[ x , y ] m . If a se-quence G ∈ F [ y ] m is a syzygy of B ( a ) , then G is nontrivial.
3. Algebraic Structure
Here we analyse the algebraic structure of the F [ y ]-module generated by a bilinear sequence B ∈ B ( n x , n y , m ) over a finite field F . This F [ y ]-module is denoted by I y ( B ), and is defined asthe set of the linear combinations of polynomials in B and coe ffi cients in F [ y ]. Our approach isbased on the particular structure of bilinear sequences, so we analyse the module of syzygies ofa generic homogeneous sequence. Notice that the field equations related to F [ x , y ] do not havethis structure. Thus, the natural procedure of concatenating the field equations to the originalsequence and then apply the well-known theory of sequences over algebraic closed field –as in(Bardet et al., 2011)– is not considered in this case.7 .1. Jacobian Syzygies of Generic Bilinear Sequences Faug`ere et al. (2011) combined Proposition 11 and Cramer’s rule to find nontrivial syzygiesfor a generic homogeneous sequence. When m > n x this method provides (cid:16) mn x + (cid:17) syzygies ofdegree n x in F [ y ] m . In the under-determined case, m < n x + n y , it was conjectured by Faug`ere et al.(2011) that, when applied to sequences in F [ x , y ] m , those syzygies form a basis for the left kernelof jac x ( B ), for each sequence B in an open Zariski set O ⊂ B ( h ) ( n x , n y , m ). This conjecture doesnot hold for the determined ( m = n x + n y ) and over-determined ( m > n x + n y ) cases.In the determined and over-determined cases, the degree n x syzygies described in (Faug`ere et al.,2011) do exist, but they do not form a basis for the left kernel of jac x ( B ). In those cases, the left-kernel of the jac x ( B ) is expected to have elements of degree less than n x .Let B ( a ) be a generic homogeneous sequence in F ( a )[ x , y ] m . Then, the Jacobian jac x ( B ( a )) isa matrix of size m × n x , where each entry is a generic homogeneous linear form in F ( a )[ y ]. Let A be the set resulting of multiplying each row of jac x ( B ( a )) by each degree d − F [ y ]. So, A is a set of m n y + d − d − ! elements living in the F ( a )-vector space of sequences of size n x containing degree d − F ( a )[ y ]. The coe ffi cients of a vanishing F ( a )-linear combination of the elements of A can be used to build a degree d − jac x ( B ( a )), and consequently, adegree d − B ( a ) via propositions 11 and 12.Since F ( a ) is a fraction field, every nonzero algebraic expression in F ( a ) has an inverse. Thereis a vanishing F ( a )-liner combination of the elements of A whenever m n y + d − d − ! > n x n y + d − d − ! . (1)Notice, Inequality (1) holds if and only if d > n x ( n y − m − n x + . Therefore, the first fall degree of a generic bilinear sequence ˜ B ( a ) ∈ F ( a )[ x , y ] m is upper boundedby min ( d ∈ Z + | d > n x ( n y − m − n x + ) . In addition, it is easy to see that n x + n y ≤ m ⇐⇒ n x ( n y − m − n x + < n x + . (2)Indeed, n x + n y ≤ mn x + n y − < mn y − < m − n x n y − m − n x < n x ( n y − m − n x + < n x + . d ≤ n x + n x in F ( a )[ y ] m . y -Degree of regularity The concepts of y -degree of regularity and y -semiregularity for a homogeneous bilinear se-quence are introduced in this section. Based on theoretical arguments and a wide experimentalverification, we conjecture that most of the sequences in a space B ( h ) ( n x , n y , m ) are y -semiregular.We deduce the y -degree of regularity for y -semiregular sequences, see Proposition 20. Definition 13.
The y -Macaulay matrix of degree d of a sequence of bilinear polynomials B ∈B ( n x , n y , m ) is defined as the matrix M y , ≤ d ( B ) containing the coe ffi cients of the polynomials of theform m f , where f ∈ B and m ∈ F [ y ] is a monomial of degree at most d − . The columns of thematrix correspond to the monomials produced in all the products m f , and sorted in decreasingorder with respect to the grevlex ordering. For ≤ j ≤ d, we use M y , j ( B ) to denote the rowsubmatrix of M y , ≤ d ( B ) formed by taking only monomials m ∈ F [ y ] with degree exactly j − . Werefer to M y , j ( B ) as the degree j part of the y -Macaulay matrix M y , ≤ d ( B ) . Example 14.
Consider the sequence B = ( x y + x y + x y , x y + x y ) ∈ B (2 , , . Thenthe degree y -Macaulay matrix M y , ≤ ( B ) is given byy f y f y f y f f f | {z } M y , ≤ ( B ) x y x y y x y x y x y y x y x y x y x y x y , where f = x y + x y + x y and f = x y + x y . Notice that M y , ( B ) is the submatrix consistingof the first four rows of M y , ≤ ( B ) , which were constructed multiplying f and f by monomials ofdegree = − . Since every row of the y -Macaulay matrix M y , ≤ d ( B ) represents one polynomial, we can definethe F -vector space J y , j ( B ) generated by the polynomials represented by the rows of M y , j ( B ), forany 2 ≤ j ≤ d . This is introduced in the following definition. Definition 15.
Given a bilinear sequence B = ( f , . . . , f m ) ∈ B ( n x , n y , m ) , we use the symbolsI y , ≤ d ( B ) and J y , ≤ d ( B ) to denote the following F -vector spacesI y , ≤ d ( B ) = h | h = m X i = g i f i , g i ∈ F [ y ] and deg ( h ) ≤ d J y , ≤ d ( B ) = h | h = m X i = g i f i , g i ∈ F [ y ] and deg ( g i ) ≤ d − . We use I y , j ( B ) (resp. J y , j ( B ) ) to denote the elements in I y , ≤ d ( B ) (resp. J y , ≤ d ( B ) ) of degree j (resp.for g i ’s having exactly degree j − ). We use I y ( B ) to denote ∪ ∞ j = I y , j . emark 16. In general, for any bilinear sequence B we have J y , ≤ d ( B ) ⊆ I y , ≤ d ( B ) , but equalitydoes not always hold. A homogeneous sequence is called y - d -regular if it has no syzygyes over F [ y ] of degree atmost d . More precisely, Definition 17.
A homogeneous bilinear sequence B ∈ B ( h ) ( n x , n y , m ) is said to be y -d-regular iffor each j = , . . . , d, Rank (cid:16) M y , j ( B ) (cid:17) = m n y + j − j − ! . The y -degree of regularity of a homogeneous bilinear sequence B ∈ B ( h ) ( n x , n y , m ) is the min-imum integer d such that every degree d monomial in F [ x , y ], which is linear in the x variables,belongs to J y , d ( B ). Definition 18.
The y -degree of regularity d y , reg ( B ) of a homogeneous bilinear sequence B ∈B ( h ) ( n x , n y , m ) is defined to be the minimum integer d satisfying that Rank (cid:16) M y , d ( B ) (cid:17) = n x n y + d − d − ! . Alternatively, it can be defined asd y , reg ( B ) = min n d | dim (cid:16) J y , d ( B ) (cid:17) = dim (cid:0) F [ x , y ] , d − (cid:1)o . Definition 19.
A homogeneous bilinear sequence B ∈ B ( h ) ( n x , n y , m ) is y -semiregular if it is y -d-regular for every d less than d y , reg ( B ) . A sequence B ∈ B ( n x , n y , m ) is said y -semiregular if itsquadratic homogeneous part is y -semiregular. In this section, we show that the syzygies in F [ y ] m of a y -semiregular bilinear sequenceare only the ones we know there exist for a generic bilinear sequence F ( a )[ x , y ] m , see Section3.1. Hence, y -semiregular sequences can be thought as generic sequences in B ( n x , n y , m ). As aconsequence, if a sequence B ∈ B ( n x , n y , m ) is y -semiregular, then its y -degree of regularity mustbe less than or equal to n x + Proposition 20.
Let n x , n y , m be positive integers with n x + n y ≤ m. If B ∈ B ( h ) ( n x , n y , m ) is y -semiregular, then d y , reg ( B ) = & n x ( n y − m − n x ' + . Proof.
Let n x , n y and m be positive integers with n x + n y ≤ m . Suppose B ∈ B ( h ) ( n x , n y , m ) is a y -semiregular sequence and, for simplicity, let us set ˜ d = d y , reg ( B ). If˜ d < n x ( n y − m − n x + m n y + ˜ d − d − ! < n x n y + ˜ d − d − ! = Rank (cid:16) M y , ˜ d ( B ) (cid:17) . M y , ˜ d ( B ) would be less than its rank, which is a contradic-tion. Thus n x ( n y − / ( m − n x ) + ≤ ˜ d .Now, assume there is an integer d satisfying n x ( n y − m − n x + ≤ d < ˜ d . Hence
Rank (cid:16) M y , d ( B ) (cid:17) < n x n y + d − d − ! ≤ m n y + d − d − ! , where the strict inequality is provided by the definition of the y -degree of regularity (see Defini-tion 18). Thus, B is not y - d -regular, for d < ˜ d . That contradicts the fact that B is y -semiregular.Therefore, we must have d y , reg ( B ) = & n x ( n y − m − n x ' + . Remark 21.
Notice that Equation (2) and Proposition 20 imply that the y -degree of regularityof a y -semiregular sequence B is less than or equal to n x + . Based on extensive experimental results, and following an analogous approach to the oneused in (Faug`ere et al., 2011), we now conjecture that being y -semiregular is a generic propertyin the set of over-determined homogeneous bilinear sequences. Conjecture 22.
Suppose n x , n y , m are positive integers with n x + n y ≤ m. There exists an openZariski set O ⊂ B ( h ) ( n x , n y , m ) such that every homogeneous sequence B ∈ O is y -semiregular. See more details in Section 5 and Table 1.
In this section we introduce the y -first fall degree for bilinear sequences and we estimate itfor y -semiregular sequences. Definition 23 ( y -First Fall Degree) . Let B be a bilinear sequence and ˜ B = ( ˜ f , . . . , ˜ f m ) be thehomogeneous sequence formed by the quadratic part of every polynomial in B. We say B hasa y -degree fall at degree d, if there is a sequence of degree d − homogeneous polynomialsG ∈ F [ y ] m that is a non-trivial syzygy of ˜ B. The y -first fall degree of B, denoted d y − f f ( B ) , is thesmallest d such that B has a y -degree fall at degree d. In general, a degree fall for a sequence of polynomials B over F [ x , y ] is obtained from a non-trivial syzygy over F [ x , y ] of the sequence ˜ B , which is the one formed by the homogeneous partof highest degree of each polynomial in B . For semiregular sequences we can precisely predict atwhat degree non-trivial syzygies first appear (Bardet et al., 2005; Ding and Schmidt, 2013). Ananalogous prediction can be done if we only consider coe ffi cients in F [ y ] instead of the wholepolynomial ring F [ x , y ]. Proposition 24.
Let n x , n y , m be positive integers with n x + n y ≤ m. Let B ∈ B ( n x , n y , m ) be asequence such that its quadratic homogeneous part ˜ B ∈ B ( h ) ( n x , n y , m ) is y -semiregular. Thus,d y − f f ( B ) = min ( d ∈ Z + | d > n x ( n y − m − n x + ) . roof. Let n x , n y , m be positive integers with n x + n y ≤ m . Let B = ( f , . . . , f m ) ∈ B ( n x , n y , m )be a y -semiregular sequence with quadratic homogeneous part denoted by ˜ B = ( ˜ f , . . . , ˜ f m ) ∈B ( h ) ( n x , n y , m ). Set ˜ d = min ( d ∈ Z + | d > n x ( n y − m − n x + ) . By Proposition 20 d y , reg ( ˜ B ) = l n x ( n y − m − n x m +
1. So d y , reg ( ˜ B ) = ˜ d or d y , reg ( ˜ B ) = ˜ d −
1. In any case, if j < ˜ d we have Rank (cid:16) M y , j ( ˜ B ) (cid:17) = m n y + j − j − ! ≤ n x n y + j − j − ! . The case d y , reg ( ˜ B ) = ˜ d − n x ( n y − m − n x is an integer, and this is equal to what occurswhen the matrix is square and invertible at degree d y , reg ( ˜ B ). Then, for each j < ˜ d , the rows of M y , j ( B ) are linearly independent, hence there are not degree j − g , . . . , g m ∈ F [ y ]such that P mi = g i ˜ f i = y -degree falls of B up to degree ˜ d −
1. Atdegree ˜ d m n y + ˜ d − d − ! > n x n y + ˜ d − d − ! , then the rows of M y , ˜ d ( B ) are linearly dependent. Hence there exist degree ˜ d − G = ( g , . . . , g m ) ∈ F [ y ] m such that P mi = g i ˜ f i =
0, and by Proposition 12, G is a non-trivialsyzygy of ˜ B , thus ˜ d is the y -first fall degree of B . Remark 25.
Note that if B is a y -semiregular sequence, then d y − f f ( B ) = d y , reg ( ˜ B ) + alwayswhen m − n x divides n x ( n y − . Otherwise, d y − f f ( B ) = d y , reg ( ˜ B ) .3.4. The Witness Degree In this section we define and estimate the y -witness degree for a bilinear sequences. Anal-ogously to the witness degree definition in (Bardet et al., 2011), the y -witness degree for anbilinear sequence B is defined as the minimum integer d such that all the polynomials in I y , ≤ d ( B )can be written as an F -linear combination of the polynomials represented by the rows of the y -Macaulay matrix M y , ≤ d ( B ). More precisely: Definition 26 ( y -Witness Degree) . Suppose F is a field with q elements and B = ( f , . . . , f m ) isa bilinear sequence in B ( n x , n y , m ) . The y -witness degree d y , wit ( B ) of B is defined asd y , wit ( B ) = min n d ∈ Z + | I y , ≤ d ( B ) = J y , ≤ d ( B ) o . The y -witness degree of a bilinear sequence B can be upper-bounded in most cases by the˜ y -degree of regularity of its homogenization B ( h ) (see Theorem 28), which is simply the homoge-neous bilinear sequence containing the ˜ y -homogenization of the polynomials in B , as explainedin the following definition. Definition 27 (˜ y -homogenization) . Let f be a polynomial in the F -span of ∪ ∞ j = F [ x , y ] , j ∪ F [ y ] .We define the ˜ y -homogenization of f as the homogeneous polynomial f ( h ) ∈ F [˜ x , ˜ y ] given byf ( h ) (˜ x , ˜ y ) = x y deg( f ) − f x x , . . . , x n x x , y y , . . . , y n y y ! , here ˜ x = ( x , x ) and ˜ y = ( y , y ) are sets of variables with sizes n x + and n y + , respec-tively. Conversely, if ˜ f is a homogeneous polynomial in F [˜ x , ˜ y ] , with ˜ x = ( x , x ) and ˜ y = ( y , y ) ,we define its y -dehomogenization as the polynomial ˜ f (1 , x , , y ) in F [ x , y ] . For a sequenceB = ( f , . . . , f m ) , where f , . . . , f m ∈ ∪ ∞ j = F [ x , y ] , j ∪ F [ y ] , we define its ˜ y -homogenization B ( h ) as the sequence B ( h ) = (cid:16) f ( h )1 , . . . , f ( h ) m (cid:17) . Finally, for a sequence ˜ B of homogeneous polynomi-als, we define its y -dehomogenization as the sequence B in which the i-th component is the y -dehomogenization of the i-th component of ˜ B. In particular, if f = P n x i = P n y j = a i , j x i y j + P n x i = b i x i + P n y j = c j y j + d ∈ F [ x , y ] is a bilinearpolynomial, the ˜ y -homogenization f ( h ) of f is the homogeneous bilinear polynomial in the setsof variables ˜ x = ( x , x ), ˜ y = ( y , y ), given by f ( h ) (˜ x , ˜ y ) = n x X i = n y X j = a i , j x i y j , where a i , = b i for i > a , j = c j for j > a , = d . Theorem 28.
Let n x , n y , m be positive integers with n x + n y ≤ m − . If Conjecture 22 is true,then there is an open Zariski set O ⊂ B ( n x , n y , m ) such that each B ∈ O satisfies the followingproperty: the system B = has no solution and belongs to the F -vector space generated by thepolynomials representing the rows of the y -Macaulay matrix M y , ≤ ˜ d ( B ) , where ˜ d = & n y ( n x + m − n x − ' + . Moreover, d wit ( B ) ≤ ˜ d.Proof. Let n x , n y , m be positive integers with n x + n y ≤ m −
2. Assuming veracity of Conjecture22, there is an open set ˜ O ⊂ B ( h ) ( n x + , n y + , m ) such that any homogeneous sequence ˜ B ∈ ˜ O is ˜ y -semiregular. Define O as the set { ˜ B (1 , x , , y ) | ˜ B ∈ ˜ O } ⊂ B ( n x , n y , m ) . We claim that theset O is an open Zariski set. Indeed, each sequence ˜ B ( x , x , y , y ) ∈ ˜ O can be uniquely identifiedwith a vector in F m [( n x + n y + , and the same vector identifies the sequence ˜ B (1 , x , , y ). So, ˜ O is a Zariski open subset of B ( h ) ( n x + , n y + , m ) if and only if O is an open Zariski subset of B ( n x , n y , m ).We will now show that each sequence in O satisfies the property stated in the theorem. Recallthat ˜ x = ( x , x , . . . , x n x ), ˜ y = ( y , y , . . . , y n y ) are the sets of variables and let B be a bilinearsequence in O . Clearly, the ˜ y -homogenization B ( h ) of B is an element in ˜ O . Since B ( h ) is ˜ y -semiregular, any monomial of the form f (˜ x , ˜ y ) = x i ˜ m , where i = , , , . . . , n x and ˜ m ∈ F [˜ y ] is amonomial of degree d ˜ y , reg ( B ( h ) ) −
1, can be written as an F [˜ y ]-linear combination of polynomialsin B ( h ) . That is, assuming B ( h ) (˜ x , ˜ y ) = ( f (˜ x , ˜ y ) , . . . , f m (˜ x , ˜ y )), we have f (˜ x , ˜ y ) = m X i = g i (˜ y ) f i (˜ x , ˜ y ) , for some g i (˜ y ) ∈ F [˜ y ] having degree d ˜ y , reg ( B ( h ) ) −
2. Consequently, the polynomial f (1 , x , , y ),which is the y -dehomogenization of f (˜ x , ˜ y ), can be written as f (1 , x , , y ) = m X i = g i (1 , y ) f i (1 , x , , y ) , (3)13here each g i (1 , y ) is in F [ y ] and has degree at most d ˜ y , reg ( B ( h ) ) −
2. This means f (1 , x , , y )belongs to J y , ≤ ˜ d ( B ), where ˜ d = d ˜ y , reg ( B ( h ) ).Notice that for every monomial m ∈ F [ y ] of degree at most d ˜ y , reg ( B ( h ) ) − x i in x , there are monomials h (˜ x , ˜ y ) = x m and h (˜ x , ˜ y ) = x i m in F [˜ x , ˜ y ] of degree at most d ˜ y , reg ( B ( h ) ), such that m = h (1 , x , , y ) and x i m = h (1 , x , , y ). Therefore, Equation (3) impliesthat every monomial like x i m or m belong to J y , ≤ ˜ d ( B ), where B ( x , y ) = ( f (1 , x , , y ) , . . . , f m (1 , x , , y )).This implies d wit ( B ) ≤ d ˜ y , reg ( B ( h ) ). Notice that for the particular case f (˜ x , ˜ y ) = y d ˜ y , reg ( B )0 , we getthat f (1 , x , , y ) = J y , ≤ ˜ d ( B ). Hence the system B = has no solution.Finally, Proposition 20 implies d wit ( B ) ≤ & n y ( n x + m − n x − ' + . Corollary 29.
Let n x , n y , m be positive integers such that n x + n y ≤ m − . Then, there is a set S ⊆ B ( n x , n y , m ) containing an open Zariski set such that every B ∈ S satisfies the followingcondition: The system B = has a solution if and only if < J y , ≤ ˜ d ( B ) , with ˜ d = & n y ( n x + m − n x − ' + . Proof.
The set S is the union of the set O from Theorem 28 and the set of sequences B such that B = has a solution.A computational way know whether 1 ∈ J y , ˜ d ( B ) for a a given B ∈ B ( n x , n y , m ), where n x + n y ≤ m −
2, is by testing the solvability of the linear system z · M y , ≤ ˜ d ( B ) = e , where e = (cid:16) · · · (cid:17) ∈ F n x ( ny + ˜ d − d − ) is a row vector. Such a system has a solution for z if and only if1 ∈ J y , ˜ d ( B ).
4. Complexity Analysis
Here we estimate the complexity of solving over-defined generic bilinear systems over finitefields. We propose three slight variants of XL-like algorithms, specifically designed for solvingbilinear systems, namely, y -XL, y -MutantXL, and y -Hybrid. We analyse the complexity of thesealgorithms and compare them with the e ffi ciency of the F4 algorithm. All over this section n x , n y and m are positive integers with n x + n y ≤ m − n x ≤ n y . y -XL y -XL is an algorithm for solving a bilinear system B = . For B ∈ B ( n x , n y , m ), y -XL looksfor a linear polynomial in J y , ≤ d ( B ), for some integer d . Provided the existence of a solution, wesay y -XL solves the system B = at degree d , if it finds at least one linear equation in J y , ≤ d ( B ).Otherwise, we say it does not solve the system at degree d . The minimum degree d at which y -XL solves a bilinear sequence B is denoted by y -XL sol ( B ).Finding linear polynomials is not the only criterion for deciding whether XL succeeds ornot in solving a system, c.f. (Cox et al., 2007). However, studying other termination criteria for y -XL is outside of the scope of this paper. The complexity of y -XL, provided that B = has14 lgorithm 1 y -XL function y -XL( B , d ) ⊲ Where B ∈ B ( n x , n y , m ) - d ∈ Z + L = ∅ E = EchelonForm (cid:16) M y , ≤ d ( B ) (cid:17) L = Linear polynomials representing rows in E . return L end function a solution, is upper bounded by the complexity of computing the echelon form of the matrix M y , ≤ d ( B ), where d = d y , wit ( B ) , which is a matrix of size m n y + d y , wit ( B ) − d y , wit ( B ) − ! × ( n x + n y + d y , wit ( B ) − d y , wit ( B ) − ! . In most cases, and regardless of whether the system has a solution or not, we can preciselyestimate its witness degree. By Corollary 29, for most sequences B ∈ B ( n x , n y , m ) with n x + n y ≤ m −
2, the complexity of deciding whether or not the system B = has a solution (and findingone if it exists) is upper bounded by O m n y + ˜ d − d − ! n x n y + ˜ d − d − ! ω − operations over F . Here ˜ d is as defined in Corollary 29, and ω is the exponent of the complexityof multiplying two square matrices of size n .We experimentally verified this result for B ∈ B ( n x , n y , m ) chosen uniformly at random andforced to have a uniform random solution a ∈ F ( n x + n y + . In most cases, linear polynomials canbe found by applying y -XL at degree ˜ d . The results of our experiments are summarized in Tables2 and 3. y -MutantXL This section introduces the mutant variant of y -XL, which we will refer to as y -MXL. Theidea here is to apply the same strategy of MutantXL (Mohamed et al., 2008; Cabarcas, 2011), butonly using monomials involving y variables. Generally speaking, this strategy consists of takingthe degree falls that appear in y -XL, multiplying them by the y -variables and appending them tothe set of polynomials. This process is repeated again and again until degree one polynomialsappear.As in the y -XL algorithm, an integer d and a bilinear sequence B are provided as the input of y -MXL. Similarly, we say that y -MXL solves a system B = at degree d (provided a solutionexists), if it finds linear polynomials. We denote by y -MXL sol ( B ) the minimum integer d at which y -MXL solves the system B = .The advantage of y -MXL over y -XL is that the mutant version might finish at degree d y − f f ( B )(or not far from it), which is in general smaller than d y , wit ( B ). Moreover, following Section 3.3,we can precisely estimate d y − f f ( B ) for most bilinear sequences as T f f ( n x , n y , m ) = min ( d ∈ Z + | d > n x ( n y − m − n x + ) . y -MXL sol ( B ) is an open question. From the experimental data showedin Tables 2 and 3, we conjecture that if B ∈ B ( n x , n y , m ) is y -semiregular and T f f ( n x , n y , m ) = T f f ( n x , n y , m − y -MXL sol ( B ) = d y − f f ( B ). This conjecture is reasonable because when T f f is equal for ( n x , n y , m ) and ( n x , n y , m − B = using y -MXL, where B ∈ B ( n x , n y , m ) and T f f ( n x , n y , m ) = T f f ( n x , n y , m −
1) is given by O m n y + d y − f f ( B ) − d y − f f ( B ) − ! " n x n y + d y − f f ( B ) − d y − f f ( B ) − ! ω − multiplications over F , where ω is the linear algebra constant and d y − f f ( B ) = T f f ( n x , n y , m ). y -Hybrid Approach In this section we describe and analyze the complexity of a hybrid algorithm for solvingthe system B = for a bilinear sequence B . Hybrid approaches for solving generic systems ofbilinear equations have been studied by di ff erent researchers (Bardet et al., 2011; Bettale et al.,2009, 2012). The general idea is to try all possible values for some variables and check theconsistency of the resulting partial evaluation.Throughout this section x = ( x , . . . , x n x ) and y = ( y , . . . , y n y ) are enumerated sets of vari-ables. For integers a x , a y , with 0 ≤ a x < n x and 0 ≤ a y < n y , we use x a x and y a y to denote thevectors of variables ( x a x + , . . . , x n x ) and ( y a y + , . . . , y n y ), respectively. Definition 30 (Partial Evaluation) . Let B ( x , y ) ∈ B ( n x , n y , m ) , u = ( u , . . . , u a x ) ∈ F a x and v = ( v , . . . , v a y ) ∈ F a y , where ≤ a x < n x and ≤ a y < n y . We use B ( u , v ) ( x a x , y a y ) to denote thebilinear sequence in B ( n x − a x , n y − a y , m ) given byB ( u , . . . , u a x , x a x + , . . . x n x , v , . . . , v a y , y a y + , . . . y n y ) . The sequence B ( u , v ) ( x a x , y a y ) is called the partial evaluation of B in ( u , v ) . For short we use B ( u , v ) ,when the involved variables x , y , x a x and y a y are clear in the context. Given B ∈ B ( n x , n y , m ), the y -HXL a x , a y algorithm goes through all pairs of vectors ( u , v ) ∈ F a x × F a y , and checks the consistency of the partially evaluated bilinear system B ( u , v ) ( x a x , y a y ) = . It stops when it finds a system B ( u , v ) = being consistent. This procedure is described inAlgorithm 2. This computes a partial solution of the system B = . It can then be appliedrecursively until a whole vector u ∈ F n x or v ∈ F n y is found, such that the system B ( u , v ) = has asolution. After this, a complete solution can be found by solving a linear system in the remainingunknown variables.For ( u , v ) ∈ F a x × F a y define a random variable X ( u , v ) taking values in the set of bilinearequations B ( n x − a x , n y − a y , m ). In each realization of X ( u , v ) a sequence B ( x , y ) ∈ B ( n x , n y , m ) ischosen and the output of X ( u , v ) is B ( u , v ) . If B ∈ B ( n x , n y , m ) is chosen uniformly at random, thenthe random variable X ( u , v ) is uniform in B ( n x − a x , n y − a y , m ).By Corollary 29, when n x + n y ≤ m −
2, there is a subset
S ⊂ B ( n x − a x , n y − a y , m ) containingan open Zariski set, such that for every B ( u , v ) ∈ S , the consistency (or inconsistency) of thesystem B ( u , v ) = can be verified by checking the inconsistency (or consistency) of the linearsystem z · M y , ≤ ˜ d ( B ( u , v ) ) = e , where˜ d = & ( n y − a y )( n x − a x + m − n x + a x − ' + . (4)16 lgorithm 2 y-HXL a x , a y function y -HXL a x , a y ( B ) ⊲ Where B ∈ B ( n x , n y , m ) d = l ( n y − a y )( n x − a x + m − n x + a x − m + e = (cid:16) · · · (cid:17) ∈ F n x ( ny + d − d − ) for ( u , v ) ∈ F a x × F a y do if z · M y , ≤ d (cid:0) B ( u , v ) (cid:1) = e is inconsistent then return ( u , v ) end if end for end function Moreover, by the same result and since X ( u , v ) is a uniform random variable, the y -witness degreeof the partially evaluated sequence is expected to be upper bounded by ˜ d . That is why this is thevalue chosen for d in step 2 of Algorithm 2.It is possible and advantageous to use Wiedemann’s Algorithm to check the consistency of thelinear system in step 5 of Algorithm 2. Given a matrix A with coordinates in F , and provided thatthe system z · A = b has at least a solution for z , Wiedemann’s Algorithm returns a solution by per-forming an expected number of operations over F upper bounded by O (cid:0) n ( η + n log n ) log n (cid:1) .Here η is the number of non-zero entries in A , and n , n are the minimum and maximum betweenthe number of rows and columns of A , respectively (Wiedemann, 1986).For a x , a y fixed, the complexity of y -HLX a x , a y (using Wiedemann’s algorithm) is given by O q a x + a y ( n y − a y + n x − a x + n y − a y + ˜ d − d − ! . For a x , a y fixed, the complexity of y -HXL a x , a y (using Gaussian elimination) is given by O q a x + a y m n y − a y + ˜ d − d − ! ( n x − a x + n y − a y + ˜ d − d − ! ω − , where ˜ d is the integer defined in Equation (4).Computing asymptotic formulas for the values a x and a y , which lead to an optimal complexityof y -HXL, is out of the scope of this paper. Instead, we use the complexity formulas to find,numerically, the optimal values of a x and a y for a given set of parameters q , m , n x , n y . The resultsare shown in Table 4. We also compare this optimal complexity of y -HXL with the complexityof y -MXL, see Section 4.2. We can see that in most of the cases y -HXL with the optimal a x , a y outperforms y -MXL. In order to solve a bilinear systems, it is also possible to use an out-of-the-box algorithm,for example applying the F4 algorithm or trying all possible values of the x -variables. Here, weestimate the complexity of this approach, as a point of reference for comparison.As mentioned in Section 2.2, the complexity of solving a system of polynomial equations B = , where B is semiregular, can be estimated by using the first degree fall. Bilinear sequences17n F [ x , y ] are not s, they form a relative small set of polynomials in the set of all quadraticpolynomials in F [ x , y ]. Thus we would not expect that the first fall degree d f f ( B ) of a bilinearsequence B is also its solving degree. However, in all the experiments we were able to conduct,this was the case (see Tables 2 and 3). Therefore, we estimate that the complexity of solving abilinear system B = using F4, where B ∈ B ( n x , n y , m ) is y -semiregular, is given by O n x + n y + d f f ( B ) d f f ( B ) ! ω ! . Another way to solve a bilinear system is by trying all possible values of the variables fromone set ( x or y ) and then check the consistency of the remaining linear system. Since n x ≤ n y ,it is better to test all x variables. Notice this simple method can be seen as a special case of thealgorithm y -HXL a x , a y when a x = n x and a y =
0. The complexity of this method is O (cid:16) q n x mn ω − y (cid:17) .
5. Experimental Results
In this section we show some experimental results that confirm the theoretical findings of thepaper, illustrate some of the results, and fill in some gaps.In order to evaluate the validity of Conjecture 22, we performed the experiment describedin Algorithm 3, whose results are presented in Table 1. They show that with high probability, arandomly chosen bilinear sequence is y -semiregular, supporting the validity of the conjecture. Algorithm 3
Randomly Testing y -Semiregulary Input:
Positive integers n x , n y , m such that n x + n y ≤ m Output:
True , if a randomly chosen bilinear sequence is y -semiregular. False , otherwise. B ← B ( h ) ( n x , n y , m ) ⊲ Uniformly at random ˜ d ← l n x ( n y − m − n x m + r ← (cid:16) n y + ˜ d − d − (cid:17) + n x (cid:16) n y + ˜ d − d − (cid:17) M ← M y , ≤ ˜ d ( B ) if Rank ( M ) = r then return True else return False end if Algorithm 3 indeed checks whether a sequence B ∈ B ( h ) ( n x , n y , m ) is y -semiregular. In gen-eral, for every 2 ≤ j < ˜ d , we have that Rank (cid:16) M y , j ( B ) (cid:17) ≤ m (cid:16) n y + j − j − (cid:17) and Rank (cid:16) M y , ˜ d ( B ) (cid:17) ≤ n x (cid:16) n y + ˜ d − d − (cid:17) (see Section 3.1). In particular, when B is homogeneous, the rank of the whole y -Macaulay matrix is given by Rank (cid:16) M y , ≤ ˜ d ( B ) (cid:17) = P ˜ dj = Rank (cid:16) M y , j ( B ) (cid:17) . Thus, when the conditionin step 5 is satisfied, we guarantee that for each j = , . . . , ˜ d − Rank (cid:16) M y , j ( B ) (cid:17) = m n y + j − j − ! , Rank (cid:16) M y , ˜ d ( B ) (cid:17) = n x n y + ˜ d − d − ! , which means B is y -semiregular.Table 1 shows notable variations in the percentage of y -semiregular sequences across dif-ferent parameters. This phenomenon deserves some explanation. For every choice of param-eters such that ( m − n x ) does not divide n x ( n y − B ∈ B ( n x , n y , m ) happens to be y -semiregular is overwhelming. In the other cases, when( m − n x ) does divide n x ( n y − ff erence is because, when n x ( n y − / ( m − n x ) is an integer, we have that for ˜ d = n x ( n y − / ( m − n x ) + n x n y + ˜ d − d − ! = m n y + ˜ d − d − ! , and thus, if B ∈ B ( h ) ( n x , n y , m ) the submatrix M y , ˜ d ( B ) is a square matrix. Since the coe ffi cients ofthis matrix are on a finite field, we expect it to be invertible with near-one but non-overwhelmingprobability in the size of the field F . 19 able 1: Experimental results to verify Conjecture 22. Here we use n x =
4, the column % shows the percentage of timesthe chosen sequence B ∈ B ( h ) ( n x , n y , m ) was y -semiregular, and ˜ d , which is equal to ⌈ n x ( n y − / ( m − n x ) ⌉ +
1, indicatesthe y -degree of regularity according to Proposition 20. The random experiment described in Algorithm 3 was executed ahundred of times for every choice of parameters. . n y m ˜ d % n y m ˜ d % n y m ˜ d %4 8 4 88 5 18 3 100 7 17 3 1009 4 100 6 10 5 99 18 3 10010 3 93 11 4 100 19 3 10011 3 100 12 4 100 20 3 10012 3 100 13 4 100 21 3 10013 3 100 14 3 92 22 3 10014 3 100 15 3 100 8 12 5 9815 3 99 16 3 100 13 5 10016 2 93 17 3 100 14 4 1009 5 99 18 3 100 15 4 1005 10 4 100 19 3 100 16 4 10011 4 100 20 3 100 17 4 10012 3 90 7 11 5 100 18 3 9213 3 100 12 4 86 19 3 10014 3 100 13 4 100 20 3 10015 3 100 14 4 100 21 3 10016 3 100 15 4 100 22 3 10017 3 100 16 3 88 23 3 100Tables 2 and 3 serve to compare our theoretical estimates with experimental results for thevarious algorithms on randomly chosen bilinear sequences. It is worth comparing the solvingdegree of y -XL and y -MXL, which is the minimum degree where the algorithms find linearpolynomials in I y (see sections 4.1 and 4.2). It is known that, for every B ∈ B ( n x , n y , m ), with n x + n y ≤ m −
2, we have y -MXL sol ( B ) ≤ y -XL sol ( B ). However, the inequality might be strictin some cases. The tables show that this is indeed the case for some parameters. Notice thatfor certain parameters y -MXL sol ( B ) = y -XL sol ( B ) −
1. For such parameters y -MXL shows anexponential speed up over y − XL.It is also known that d f f is upper bounded by d y − f f . Yet, it might be the case that for someparameters, d f f is strictly less than d y − f f . However, for every single instance we ran, with n x + n y ≤ m , we observed d f f = d y − f f and this was also the solving degree of F4. Based on this,we conjecture that d y − f f is a tight upper bound for the solving degree of F4 for y -semiregular20equences. Thus, the complexity of solving a system B = , where B ∈ B ( n x , n y , m ), is given by O n x + n y + d y − f f ( B ) d y − f f ( B ) ! ω ! . This is less e ffi cient than y -MXL, provided that T f f ( n x , n y , m ) = T f f ( n x , n y , m − able 2: Comparison between the solving degrees for y -XL, y -MXL and F4, the y -first fall degree and the first fall degreeof a sequence B ∈ B (4 , n y , m ) chosen uniformly at random over GF (13). The column T wit shows the theoretical upperbound for the solving degree of y -XL given in Theorem 28 and T f f is the theoretical upper bound for d y − f f given byConjecture 22, as explained in Section 4.2. For each sequence B , d y − f f and F4 f f are, respectively, the y -first fall degreeand the first fall degree of B . y -XL sol and y -MXL sol are the solving degree in y -XL and in y -MXL of B , respectively, asdefined in Sections 4.1 and 4.2. F4 sol is the maximum degree reached during the Gr¨obner basis computation of the idealgenerated by B . The five rightmost columns show the most common value obtained for each set of parameters out of ahundred random instances, and the value in parenthesis indicates the corresponding relative frequency. n y m T f f T wit d y − f f y -XL sol y -MXL sol F4 f f F4 sol
10 4 5 4 (0.93) 5 (0.89) 4 (1.0) 4 (0.87) 4 (0.99)4 11 3 5 3 (1.0) 5 (1.0) 4 (1.0) 3 (1.0) 3 (1.0)12 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)13 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)14 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)15 3 3 3 (1.0) 3 (0.89) 3 (1.0) 3 (1.0) 3 (1.0)16 3 3 3 (1.0) 3 (1.0) 3 (1.0) 3 (0.89) 3 (1.0)11 4 6 4 (1.0) 6 (0.98) 4 (0.98) 4 (1.0) 4 (1.0)5 12 4 5 4 (0.95) 5 (1.0) 4 (1.0) 4 (0.96) 4 (1.0)13 3 5 3 (1.0) 5 (1.0) 4 (1.0) 3 (1.0) 3 (1.0)14 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)15 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)16 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)17 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)18 3 3 3 (1.0) 3 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)12 4 6 4 (1.0) 6 (0.99) 4 (0.99) 4 (1.0) 4 (1.0)6 13 4 5 4 (1.0) 5 (1.0) 4 (1.0) 4 (1.0) 4 (1.0)14 4 5 4 (0.94) 5 (1.0) 4 (1.0) 4 (0.95) 4 (1.0)15 3 4 3 (1.0) 4 (0.95) 4 (1.0) 3 (1.0) 3 (1.0)16 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)17 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)18 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)19 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)20 3 3 3 (1.0) 3 (0.95) 3 (1.0) 3 (1.0) 3 (1.0)22 able 3: Comparison between the solving degrees for y -XL, y -MXL and F4, the y -first fall degree and the first falldegree of a sequence B ∈ B (4 , n y , m ) chosen uniformly at random over GF (13). The column T wit shows the theoreticalupper bound for solving degree of y -XL given in Theorem 28 and T f f is the theoretical upper bound for d y − f f given byConjecture 22, as explained in Section 4.2. For each sequence B , d y − f f and F4 f f are, respectively, the y -first fall degreeand the first fall degree of B . y -XL sol and y -MXL sol are the solving degree in y -XL and in y -MXL of B , respectively, asdefined in the sections 4.1 and 4.2. F4 sol is the maximum degree reached during the Gr¨obner basis computation of theideal generated by B . The five rightmost columns shows the most common value obtained for each set of parameters outof hundred of realizations, the value in parenthesis indicates the corresponding the relative frequency. n y m T f f T wit d y − f f ( B ) y -XL sol y -MXL sol F4 f f F4 sol
13 4 6 4 (1.0) 6 (1.0) 5 (1.0) 4 (1.0) 4 (1.0)7 14 4 5 4 (1.0) 5 (1.0) 4 (1.0) 4 (1.0) 4 (1.0)15 4 5 4 (1.0) 5 (1.0) 4 (1.0) 4 (1.0) 4 (1.0)16 4 5 4 (0.94) 5 (1.0) 4 (1.0) 4 (0.95) 4 (1.0)17 3 4 3 (1.0) 4 (1.0) 4 (1.0) 3 (1.0) 3 (1.0)18 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)19 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)20 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)21 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)22 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)14 4 6 4 (1.0) 6 (1.0) 5 (1.0) 4 (1.0) 4 (1.0)15 4 5 4 (1.0) 5 (0.9) 4 (1.0) 4 (1.0) 4 (1.0)8 16 4 5 4 (1.0) 5 (1.0) 4 (1.0) 4 (1.0) 4 (1.0)17 4 5 4 (1.0) 5 (1.0) 4 (1.0) 4 (1.0) 4 (1.0)18 4 5 4 (0.91) 5 (1.0) 4 (1.0) 4 (0.92) 4 (1.0)19 3 4 3 (1.0) 4 (1.0) 4 (1.0) 3 (1.0) 3 (1.0)20 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)21 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)22 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)23 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)24 3 4 3 (1.0) 4 (1.0) 3 (1.0) 3 (1.0) 3 (1.0)Based on the complexity estimates in Section 4, we compare the complexity of y -MXLand y -HXL for di ff erent parameters. Table 4 illustrates some of the trends. In the case of y -HXL, for each set of parameters, we report the optimal number of variables to guess andthe optimal linear algebra algorithm between Strassen’s and Wiedemann’s. In the case of y -MXL, we accept the conjecture that y -MXL sol ( B ) = d y , f f ( B ) for B ∈ B ( n x , n y , m ) provided T f f ( n x , n y , m ) = T f f ( n x , n y , m − y -HXL outperforms y -MXL. The advantage of y -HXL is specially acute for smaller23alues of n y , q , and m , but still significant for larger values.It is worth noting the behavior of the optimal number of variables to guess in y -MXL. Forsmall fields, it is better to guess most x -variables. As the size of the field grows, guessing obvi-ously becomes more expensive. Also, as m grows, guessing becomes less attractive, because thewitness degree becomes smaller, thus checking consistency becomes less expensive. However,this tendency is less pronounced for larger values of n y , because the witness degree is propor-tional to n y . Table 4: Complexity estimates comparison between y -MXL and y -HXL a x , a y for n x = n y = ,
30 and di ff erent valuesof q and m . The columns MXL and
HXL indicate the complexity of y -MXL algorithm and y -HXL a x , a y , respectively. Theyare computed as log ( ∗ ), where ∗ are the complexity estimates in Section 4 for given values q , n y , a x , a y .The values a x and a y are the ones providing an optimal complexity in y -HXL a x , a y . The column Alg indicates the linear algebra algorithmgiven better complexity in y -HXL a x , a y , ’S’ means Strassen’s Algorithm while ’W’ means Widemann’s Algorithm, weuse ω = . n y
20 30 q m MXL a x a y HXL Alg m MXL a x a y HXL Alg
42 110 19 0 59 S 52 136 20 0 61 S5 46 101 19 0 59 S 56 128 20 0 61 S50 94 19 0 60 S 60 119 20 0 61 S54 90 20 0 60 W 64 115 19 0 61 S58 86 20 0 60 W 68 110 19 0 61 S62 82 20 0 60 W 72 106 19 0 61 S42 110 19 0 85 S 52 136 20 0 89 S13 46 101 19 0 86 S 56 128 20 0 89 S50 94 2 1 86 W 60 119 20 0 89 S54 90 3 0 80 W 64 115 19 0 87 S58 86 3 0 77 W 68 110 19 0 87 S62 82 2 0 74 W 72 106 19 0 87 S42 110 3 0 98 W 52 136 20 0 114 S31 46 101 1 0 92 W 56 128 1 0 110 W50 94 1 0 87 W 60 119 1 0 104 W54 90 1 0 82 W 64 115 1 0 101 W58 86 1 0 79 W 68 110 0 1 97 W62 82 1 0 76 W 72 106 0 1 94 W
References
1. Buchberger, B.. Bruno buchbergers phd thesis 1965: An algorithm for finding the basis elements of theresidue class ring of a zero dimensional polynomial ideal. Journal of Symbolic Computation 2006;41(3):475– 511. URL: . oi: https://doi.org/10.1016/j.jsc.2005.09.007 ; logic, Mathematics and Computer Science: Interac-tions in honor of Bruno Buchberger (60th birthday).2. Faugere, J.C.. A new e ffi cient algorithm for computing grobner bases (f4). Journal of Pure and Applied Algebra1999;139:61–88.3. Faugere, J.C.. A new e ffi cient algorithm for computing grobner bases without reduction to zero (f5).ISSAC 2002, ACM Press 2002;:75–83.4. Courtois, N., Klimov, A., Patarin, J., Shamir, A.. E ffi cient algorithms for solving overdefined systems of multivariatepolynomial equations. In: Preneel, B., ed. Advances in Cryptology — EUROCRYPT 2000. Berlin, Heidelberg:Springer Berlin Heidelberg; 2000a:392–407.5. Bardet, M.. ´Etude des syst`emes alg´ebriques surd´etermin´es. Applications aux codes correcteurset `a la cryptographie. Theses; Universit´e Pierre et Marie Curie - Paris VI; 2004. URL: https://tel.archives-ouvertes.fr/tel-00449609 .6. Mohamed, M.S.E., Mohamed, W.S.A.E., Ding, J., Buchmann, J.. Mxl2: Solving polynomial equations over gf(2)using an improved mutant strategy. In: Buchmann, J., Ding, J., eds. Post-Quantum Cryptography. Berlin, Heidelberg:Springer Berlin Heidelberg. ISBN 978-3-540-88403-3; 2008:203–215.7. Buchmann, J., Cabarcas, D., Ding, J., Mohamed, M.S.E.. Flexible partial enlargement to accelerate gr¨obner basiscomputation over F . In: Bernstein, D.J., Lange, T., eds. Progress in Cryptology – AFRICACRYPT 2010. Berlin,Heidelberg: Springer Berlin Heidelberg. ISBN 978-3-642-12678-9; 2010:69–81.8. Faug`ere, J., Horan, K., Kahrobaei, D., Kaplan, M., Kashefi, E., Perret, L.. Quantum algorithm for solving multivariatequadratic equations. CoRR 2017;abs / http://doi.acm.org/10.1145/2608628.2608663 . doi: .12. Verbel, J., Baena, J., Cabarcas, D., Perlner, R., Smith-Tone, D.. On the complexity of “superdetermined” minrankinstances. In: Ding, J., Steinwandt, R., eds. Post-Quantum Cryptography. Cham: Springer International Publishing.ISBN 978-3-030-25510-7; 2019:167–186.13. Bardet, M., Bros, M., Cabarcas, D., Gaborit, P., Perlner, R., Smith-Tone, D., Tillich, J.P., Verbel, J.. Algebraicattacks for solving the rank decoding and minrank problems without grbner basis. 2020. arXiv:2002.08322 .14. Courtois, N., Klimov, A., Patarin, J., A.Shamir, . E ffi cient algorithms for solving overdefined systems of multivariatepolynomial equations. EUROCRYPT 2000, LNCS 2000b;1807:392–407.15. Cabarcas, D.. Groebner bases computation and mutant polynomials. Ph.D. thesis; University of Cincinnati; 2011.16. Bardet, M., Faug`ere, J., Salvy, B., Spaenlehauer, P.. On the complexity of solving quadratic boolean systems. CoRR2011;abs / arXiv:0903.1156 .19. Johnson, C.R., Link, J.A.. Solution theory for complete bilinear systems of equa-tions. Numerical Linear Algebra with Applications 2009;16(1112):929–934. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/nla.676 . doi: . arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/nla.676 .20. Yang, D.. Solution theory for systems of bilinear equations. Ph.D. thesis; College of William and Mary; 2011.21. Faug`ere, J.C., Din, M.S.E., Spaenlehauer, P.J.. Groebner bases of bihomogeneous ideals generated by poly-nomials of bidegree (1,1): Algorithms and complexity. Journal of Symbolic Computation 2011;46(4):406– 437. URL: .doi: https://doi.org/10.1016/j.jsc.2010.10.014 .22. Cabarcas, D., Smith-Tone, D., Verbel, J.A.. Key recovery attack for zhfe. In: Lange, T., Takagi, T., eds.Post-Quantum Cryptography. Cham: Springer International Publishing; 2017:289–308.23. Vates, J., Smith-Tone, D.. Key recovery attack for all parameters of hfe-. In: Lange, T., Takagi, T., eds.Post-Quantum Cryptography. Cham: Springer International Publishing; 2017:272–288.24. Hartshorne, R.. Varieties; chap. 1. New York, NY: Springer New York. ISBN 978-1-4757-3849-0; 1977:1–59.25. Faug`ere, J.C.. A new e ffi cient algorithm for computing Gr¨obner bases ( F ). J Pure Appl Algebra 1999;139(1-3):61–88.E ff ective methods in algebraic geometry (Saint-Malo, 1998).26. Lazard, D.. Gr¨obner-bases, gaussian elimination and resolution of systems of algebraic equations. In:Computer Algebra, EUROCAL’83, European Computer Algebra Conference, London, England, March 28-30, 1983, Proceedings. / e (Undergraduate Texts in Mathematics). Secaucus, NJ, USA: Springer-Verlag New York, Inc.; 2007. ISBN 0387356509.30. Bettale, L., Faug`ere, J., Perret, L.. Hybrid approach for solving multivariate systems over finite fields.J Mathematical Cryptology 2009;3(3):177–197.31. Bettale, L., Faug`ere, J., Perret, L.. Solving polynomial systems over finite fields: improved analysis of the hybrid ap-proach. In: International Symposium on Symbolic and Algebraic Computation, ISSAC’12, Grenoble, France - July 22 - 25, 2012.2012:67–74.32. Wiedemann, D.. Solving sparse linear equations over finite fields. IEEE Transactions on Information Theory1986;32(1):54–62. doi: ..