An Algebraic Attack on Rank Metric Code-Based Cryptosystems
Magali Bardet, Pierre Briaud, Maxime Bros, Philippe Gaborit, Vincent Neiger, Olivier Ruatta, Jean-Pierre Tillich
AAn Algebraic Attack on Rank MetricCode-Based Cryptosystems
Magali Bardet , , Pierre Briaud , Maxime Bros , Philippe Gaborit , VincentNeiger , Olivier Ruatta , and Jean-Pierre Tillich LITIS, University of Rouen Normandie, France Inria, 2 rue Simone Iff, 75012 Paris, France Univ. Limoges, CNRS, XLIM, UMR 7252, F-87000 Limoges, France
Abstract.
The Rank metric decoding problem is the main problem con-sidered in cryptography based on codes in the rank metric. Very efficientschemes based on this problem or quasi-cyclic versions of it have beenproposed recently, such as those in the submissions ROLLO and RQCcurrently at the second round of the NIST Post-Quantum CryptographyStandardization Process. While combinatorial attacks on this problemhave been extensively studied and seem now well understood, the sit-uation is not as satisfactory for algebraic attacks, for which previouswork essentially suggested that they were ineffective for cryptographicparameters. In this paper, starting from Ourivski and Johansson’s alge-braic modelling of the problem into a system of polynomial equations,we show how to augment this system with easily computed equationsso that the augmented system is solved much faster via Gr¨obner bases.This happens because the augmented system has solving degree r , r + 1or r + 2 depending on the parameters, where r is the rank weight, whichwe show by extending results from Verbel et al. (PQCrypto 2019) onsystems arising from the MinRank problem; with target rank r , Verbel et al. lower the solving degree to r + 2, and even less for some favorableinstances that they call “superdetermined”. We give complexity boundsfor this approach as well as practical timings of an implementation using magma . This improves upon the previously known complexity estimatesfor both Gr¨obner basis and (non-quantum) combinatorial approaches,and for example leads to an attack in 200 bits on ROLLO-I-256 whoseclaimed security was 256 bits. Keywords:
Post-quantum cryptography · NIST-PQC candidates · rankmetric code-based cryptography · Gr¨obner basis.
Rank metric code-based cryptography.
In the last decade, rank metriccode-based cryptography has proved to be a powerful alternative to more tradi-tional code-based cryptography based on the Hamming metric. This thread ofresearch started with the GPT cryptosystem [37] based on Gabidulin codes [36],which are rank metric analogues of Reed-Solomon codes. However, the strong a r X i v : . [ c s . CR ] F e b lgebraic structure of those codes was successfully exploited for attacking theoriginal GPT cryptosystem and its variants with the Overbeck attack [53] (seefor example [51] for one of the latest related developments). This has to be tracedback to the algebraic structure of Gabidulin codes that makes masking extremelydifficult; one can draw a parallel with the situation in the Hamming metric whereessentially all McEliece cryptosystems based on Reed-Solomon codes or variantsof them have been broken. However, recently a rank metric analogue of theNTRU cryptosystem from [44] has been designed and studied, starting with thepioneering paper [38]. Roughly speaking, the NTRU cryptosystem relies on alattice that has vectors of rather small Euclidean norm. It is precisely thosevectors that allow an efficient decoding/deciphering process. The decryption ofthe cryptosystem proposed in [38] relies on LRPC codes that have rather shortvectors in the dual code, but this time for the rank metric. These vectors areused for decoding in the rank metric. This cryptosystem can also be viewed asthe rank metric analogue of the MDPC cryptosystem [50] that relies on shortvectors in the dual code for the Hamming metric.This new way of building rank metric code-based cryptosystems has led to asequence of proposals [38,40,5,6], culminating in submissions to the NIST post-quantum competition [1,2], whose security relies solely on the decoding problemin rank metric codes with a ring structure similar to the ones encountered rightnow in lattice-based cryptography. Interestingly enough, one can also build sig-nature schemes using the rank metric; even though early attempts which reliedon masking the structure of a code [41,9] have been broken [24], a promisingrecent approach [8] only considers random matrices without structural masking. Decoding in rank metric.
In other words, in rank metric code-based cryptog-raphy we are now only left with assessing the difficulty of the decoding problemfor the rank metric. The rank metric over F Nq , where F q is the finite field ofcardinality q and N = mn is a composite integer, consists in viewing elementsin this ambient space as m × n matrices over F q and considering the distance d ( X , Y ) between two such matrices X and Y as d ( X , Y ) = Rank ( Y − X ) . A (linear matrix) code C in F m × nq is simply a F q -linear subspace in F m × nq , gen-erated by K matrices M , . . . , M K . The decoding problem for the rank metricat distance r is as follows: given a matrix Y in F m × nq at distance ≤ r from C , re-cover an element M in C at distance ≤ r from Y . This is precisely the MinRankproblem given as input Y and M , . . . , M K : Problem 1 (MinRank).Input : an integer r ∈ N and K + 1 matrices Y , M , . . . , M K ∈ F m × nq . Output : field elements x , x , . . . , x K ∈ F q such thatRank (cid:32) Y − K (cid:88) i =1 x i M i (cid:33) ≤ r.
2s observed in [20], the MinRank problem is NP-complete and the best knownalgorithms solving it have exponential complexity bounds.
Matrix codes specified as F q m -linear codes. However, the trend in rankmetric code-based cryptography has been to consider a particular form of linearmatrix codes: they are linear codes of length n over an extension F q m of degree m of F q , that is, F q m -linear subspaces of F nq m . In the rest of this section, we fix a basis( β , . . . , β m ) of F q m as a F q -vector space. Then such codes can be interpreted asmatrix codes over F m × nq by viewing a vector x = ( x , . . . , x n ) ∈ F nq m as a matrixMat( x ) = ( X ij ) i,j in F m × nq , where ( X ij ) ≤ i ≤ m is the column vector formed bythe coordinates of x j in the basis ( β , . . . , β m ), that is, x j = X j β + · · · + X mj β m .Then the “rank” metric d on F nq m is the rank metric on the associated matrixspace, namely d ( x , y ) := | y − x | , where we define | x | := Rank (Mat( x )) . An F q m -linear code C of length n and dimension k over F q m specifies a matrixcode Mat( C ) := { Mat( c ) : c ∈ C} in F m × nq of dimension K := mk over F q : it isreadily verified that a basis of this F q -subspace is given by (Mat( β i c j )) ≤ i ≤ m, ≤ j ≤ k where ( c , . . . , c k ) is a basis of C over F q m .There are several reasons for this trend. On the one hand, the families ofmatrix codes for which an efficient decoding algorithm is known are families of F q m -linear codes. On the other hand, F q m -linear codes have a much shorter de-scription than general matrix codes. Indeed, a matrix code in F m × nq of dimension K = km can be specified by a basis of it, which uses Kmn log( q ) = km n log( q )bits, whereas a matrix code obtained from an F q m -linear code of dimension k over F q m can be specified by a basis ( c , . . . , c k ) of it, which uses kmn log( q ) bitsand thus saves a factor m .Progress in the design of efficient algorithms for decoding F q m -linear codessuggests that their additional structure may not have a significant impact onthe difficulty of solving the decoding problem. For instance, a generic matrixcode over F m × nq of dimension K = mk can be decoded using the informationset decoder of [39] within a complexity of the order of q kr when the errorshave rank at most r and m ≥ n , compared to q kr − m for the decoding of alinear code over F nq m in the same regime, using a similar decoder [10]. Moreover,even if the decoding problem is not known to be NP-complete for these F q m -linear codes, there is a randomised reduction to an NP-complete problem [42](namely to decoding in the Hamming metric). Hereafter, we will use the followingterminology. Problem 2 ( ( m, n, k, r ) -decoding problem).Input : an F q m -basis ( c , . . . , c k ) of a subspace C of F nq m , an integer r ∈ N , avector y ∈ F nq m at distance at most r of C (i.e. | y − c | ≤ r for some c ∈ C ). Output : c ∈ C and e ∈ F nq m such that y = c + e and | e | ≤ r .The region of parameters which is of interest for the NIST submissions corre-sponds to m = Θ ( n ), k = Θ ( n ) and r = Θ ( √ n ).3 r¨obner basis techniques for decoding in the rank metric. The afore-mentioned algorithm from [10] for solving the decoding problem follows a combi-natorial approach pioneered in [52], which is related to decoding techniques forthe Hamming metric. Another approach consists in viewing the decoding prob-lem as a particular case of MinRank and using the algebraic techniques designedfor this problem; namely these techniques use a suitable algebraic modelling of aMinRank instance into a system of multivariate polynomial equations, and thensolve this system with Gr¨obner basis techniques. Several modellings have beenconsidered, such as the Kipnis-Shamir modelling [45] and the minors modelling(described for example in [34]); the complexity of solving MinRank using thesemodellings has been investigated in [33,34]. The Kipnis-Shamir modelling boilsdown to a polynomial system which is affine bilinear . This means that each equa-tion has degree at most 2 and the set of variables can be partitioned into twosets { x , . . . , x s } ∪ { y , . . . , y t } such that all monomials of degree 2 involved inthe equations are of the form x i y j ; in other words, the equations are formed bya quadratic part which is bilinear plus an affine part. Although the complexityof solving this system can be bounded by that of solving bilinear systems, whichis studied in [35], the complexity estimates thus obtained are very pessimistic,as observed experimentally in [21]. A theoretical explanation of why Gr¨obnerbasis techniques perform much better on the Kipnis-Shamir modelling than ongeneric bilinear systems was later given in [56]. It was also demonstrated therethat the Kipnis-Shamir approach is more efficient than the minors approach onseveral multivariable encryption or signature schemes relying on the MinRankproblem. However, the speed-up obtained for the Kipnis-Shamir modelling inthe latter reference mostly comes from the “superdetermined” case consideredtherein. When applied to the ( m, n, k, r )-decoding problem, this corresponds tothe case where m = n and km < nr ; this condition is not met in the decodingproblem instances we are interested in.Another algebraic approach to solve the ( m, n, k, r )-decoding problem wassuggested in [39, § V.]. It is based on a new modelling specific to F q m -linearcodes which fundamentally relies on the underlying F q m -linear structure andon q -polynomials. Also, it results in a system of polynomial equations that aresparse and have large degree. This approach seems to be efficient only if rk isnot much larger than n . Our contribution.
If one compares the best known complexity estimates, thealgebraic techniques appear to be less efficient than the combinatorial ones, suchas [52], [39], and [10] for the parameters of the rank metric schemes proposedto the NIST [7,3] or of other rank metric code-based cryptosystems [49]. In[55], Levy-dit-Vehel and Perret pioneered the use of Gr¨obner basis techniques tosolve the polynomial system arising in the Ourivski-Johansson algebraic mod-elling [52], with promising practical timings. In this paper, we follow on fromthis approach and show how this polynomial system can be augmented with ad-ditional equations that are easy to compute and bring on a substantial speed-upin the Gr¨obner basis computation for solving the system. This new algebraic4lgorithm results in the best practical efficiency and complexity bounds that arecurrently known for the decoding problem; in particular, it significantly improvesupon the above-mentioned combinatorial approaches.There are several reasons why the Ourivski-Johansson algebraic modellingimproves upon the Kipnis-Shamir one. First, it has the same affine bilinearstructure and a similar number of equations, but it involves much fewer vari-ables. Indeed, for the case of interest to us where m and k are in Θ ( n ) and r is in Θ ( n / ), the Kipnis-Shamir modelling involves Θ ( n ) equations and variables,while the Ourivski-Johansson one involves Θ ( n ) equations and Θ ( n / ) vari-ables. Second, this modelling naturally leads to what corresponds to reducingby one the value of r , as explained in Section 3. Third, and most importantly,the main properties that ensure that the Kipnis-Shamir modelling behaves muchbetter with respect to Gr¨obner basis techniques than generic bilinear systemsalso hold for the Ourivski-Johansson modelling. In essence, this is due to a solv-ing degree which is remarkably low: at most r + 2 for the former modelling andat most r + 1 for the latter. Recall that the solving degree indicates the max-imum degree reached during a Gr¨obner basis computation; it is known to be astrong predictor of the complexity of the most expensive step in a Gr¨obner basiscomputation and has been widely used for this purpose with confirmations vianumerical experiments, see for instance [43,29,26,27,28,56].To prove the third point, we start from the result about degree falls at thecore of [56], which is based on work from [35], and we extend it to a moregeneral setting which includes the Ourivski-Johansson modelling. In our case,these degree falls mean that from the initial system of quadratic equations f i = 0of the Ourivski-Johansson modelling, we are able to build many new equationsof degree r that are combinations (cid:80) i f i g ij = 0 where the g ij ’s are polynomialsof degree r − j -th new equation. We also prove that, when theparameters satisfy the condition m (cid:18) n − k − r (cid:19) ≥ (cid:18) nr (cid:19) − , (1)by using that these polynomials (cid:80) i f i g ij can be expressed as linear combinationsof only a few other polynomials, we can perform suitable linear combinations ofthe equations (cid:80) i f i g ij = 0’s giving (cid:0) n − r − (cid:1) − r −
1. Allthese polynomial combinations are easily computed from the initial quadraticequations. By adding these equations and then performing Gr¨obner basis com-putations on the augmented system, we observe that experimentally the Gr¨obnerbasis algorithm behaves as expected from the degree fall heuristic: – if (1) holds, this degree is r and the overall complexity is O (cid:16)(cid:16) (( m + n ) r ) r r ! (cid:17) ω (cid:17) operations in F q . – if (1) does not hold, the maximum degree reached in the Gr¨obner basis com-putation is r + 1 (in some intermediate cases), or r + 2, leading to an overallcomplexity of at most O (cid:16)(cid:16) (( m + n ) r ) r +1 ( r +1)! (cid:17) ω (cid:17) (resp. O (cid:16)(cid:16) (( m + n ) r ) r +2 ( r +2)! (cid:17) ω (cid:17) ) op-erations in F q , where ω is the exponent of matrix multiplication;5ote that for a majority of parameters proposed in [7,3], the condition (1) holds.Taking for ω the smallest value currently achievable in practice, which is ω ≈ . In the whole paper, we use the following notation and definitions: – Matrices and vectors are written in boldface font M . – For a matrix M its entry in row i and column j is denoted by M [ i, j ]. – The transpose of a matrix M is denoted by M T . – For a given ring R , the space of matrices with m rows and n columns andcoefficients in R is denoted by R m × n . – For M ∈ R m × n , we denote by vec row ( M ) the column vector formed byconcatenating the rows of M , i.e. vec row ( M ) = (cid:0) M { } , ∗ . . . M { n } , ∗ (cid:1) T . – For M ∈ R m × n , we denote by vec col ( M ) the column vector formed byconcatenating the columns of M , i.e. vec col ( M ) = vec row ( M T ). – { ..n } stands for the set of integers from 1 to n , and for any subset J ⊂{ k + 1 ..n } , we denote by J − k the set J − k = { j − k : j ∈ J } ⊂ { ..n − k } . – For two subsets I ⊂ { ..m } and J ⊂ { ..n } , we write M I,J for the submatrixof M formed by its rows (resp. columns) with index in I (resp. J ). – We use the shorthand notation M ∗ ,J = M { ..m } ,J and M I, ∗ = M I, { ..n } ,where M has m rows and n columns. – F q is a finite field of size q , and α ∈ F q m is a primitive element, so that(1 , α, . . . , α m − ) is a basis of F q m as an F q -vector space. For β ∈ F q m , wedenote by [ α i − ] β its i th coordinate in this basis. – For v = ( v , . . . , v n ) ∈ F nq m . The support of v is the F q -vector subspaceof F q m spanned by the vectors v , . . . , v n . Thus this support is the columnspace of the matrix Mat( v ) associated to v (for any choice of basis), and itsdimension is precisely Rank(Mat( v )). – An [ n, k ] F q m -linear code is an F q m -linear subspace of F nq m of dimension k endowed with the rank metric. In what follows, parameters are chosen in the cryptographically relevant regionmentionned in the introduction, say m = Θ ( n ), k = Θ ( n ) and r = Θ ( √ n ).Decoding instances will then have a single solution e . For simplicity, we assumethat the rank of e is exactly r ; in general one can run the algorithm for increas-ing values of the target rank up to r , until a solution is found, and the mostexpensive step will correspond to the largest considered rank. We consider herethe ( m, n, k, r )-decoding problem for the code C and assume we have received y ∈ F nq m at distance r from C and look for c ∈ C and e such that y = c + e and | e | = r . 6 .1 Solving the MinRank instance using Kipnis-Shamir’s modelling As explained in Section 1, a possible approach to perform the decoding is to solvethe underlying MinRank instance with km + 1 matrices in F m × nq ; this is done byintroducing M := Mat( y ) and M , . . . , M km which is an F q -basis of Mat( C ).Several methods have been developed, and so far the Kipnis-Shamir modelling[45] seems to be the most efficient to solve this MinRank instance. We want tofind ( z , . . . , z km ) in F mk +1 q such that Rank( (cid:80) kmi =0 z i M i ) = r . ( z , z , . . . , z km ) isa solution to the MinRank problem if and only if the right kernel of (cid:80) kmi =0 z i M i contains a subspace of dimension n − r of F nq . With high probability, a basisof such a space can be written in systematic form, that is, in the form (cid:0) I n − r K (cid:1) .Thus we have to solve the system (cid:32) km (cid:88) i =0 z i M i (cid:33) (cid:18) I n − r K (cid:19) = 0 , (2)over F q , where K is an r × ( n − r ) matrix of indeterminates. This system is affinebilinear and has m ( n − r ) equations and km + 1 + r ( n − r ) variables, which are z , z , . . . , z km and the r ( n − r ) entries of K ; each equation has a bilinear partas well as a linear part which only involves the variables z i . We recall here the modelling considered in [7,2]. Let H be a parity-check matrixof C , i.e. C = { c ∈ F nq m : cH T = } . The ( m, n, k, r )-decoding problem can be algebraically described by the system eH T = s where e ∈ F nq m has rank r and s ∈ F ( n − k ) q m is given by s := yH T . Let( S , . . . , S r ) ∈ F rq m be a basis of the support of e ; then, e = ( S · · · S r ) C ,where C ∈ F r × nq is the matrix of the coordinates of e in the basis ( S , . . . , S r ).Then expressing the elements S i in the basis (1 , α, . . . , α m − ) of F q m over F q yields ( S · · · S r ) = (1 α · · · α m − ) S for some matrix S ∈ F m × rq . Thus, thesystem is rewritten as (cid:0) α · · · α m − (cid:1) SCH T = s , over F q m with solutions in F q . (3)This polynomial system, that we refer to as the syndrome modelling , has m ( n − k )equations and mr + nr variables when it is written over F q . It is affine bilinear(without terms of degree 1) with respect to the two sets of variables comingfrom the support and from the coordinates of the error. Besides, this systemadmits ( q r − q r − q ) · · · ( q r − q r − ) solutions since this is the number of basesof the support. These solutions to the system all correspond to the same uniquesolution e of the initial decoding problem. We can easily impose a unique solutionby fixing some of the unknowns as in the Kipnis-Shamir modelling, or as hasbeen done in the Ourivski-Johansson modelling that we will present next. It7s worthwhile to note that this kind of modelling has, as the Kipnis-Shamirmodelling, Θ (cid:0) n (cid:1) equations for our choice of parameters but significantly fewervariables since we now have only Θ (cid:0) n / (cid:1) unknowns. The Ourivski-Johansson’smodelling will be a related modelling that gives a further improvement. We now describe the algebraic modelling considered in the rest of this paper,which is basically Ourivski and Johansson’s one [52]. It can be viewed as anhomogenising trick. Instead of looking for c ∈ C and e of rank r that satisfy y = c + e , or what is the same for c ∈ C such that | c + y | = r , we look for c ∈ C and λ ∈ F q m such that | c + λ y | = r. (4)It is precisely here that the F q m -linearity of C is used in a crucial way. Once wehave found such a c and λ , we have found a c + λ y such that c + λ y = µ e forsome non-zero µ ∈ F q m from which we deduce easily e . The point of proceedingthis way is that there are q m − C := C + (cid:104) y (cid:105) and that we look for a rank r wordin ˜ C , since all such words are precisely the multiples λ e for nonzero λ ∈ F q m ofthe error e we are looking for. Let ˜ G = (cid:0) I k +1 R (cid:1) (resp. ˜ H = (cid:0) − R T I n − k − (cid:1) )be the generator matrix in systematic form (resp. a parity-check matrix) ofthe extended code ˜ C ; note that for a vector v , we have v ∈ ˜ C if and only if v ˜ H T = 0. Using the notation e = (1 α · · · α m − ) SC as above, and writing C = ( C C ) with C ∈ F r × ( k +1) q and C ∈ F r × ( n − k − q , the fact that e ∈ ˜ C yields the system (cid:0) α · · · α m − (cid:1) S ( C − C R ) = 0 , over F q m with solutions in F q . (5)Since all multiples λ e are solutions of this system, we can specify the first columnof C to (1 0 · · · T . In this way, there is a single λ e satisfying these constraints:the one where λ is the inverse of the first coordinate of e (assuming it is nonzero,see below). The system still admits several solutions which correspond to dif-ferent bases of the support of λ e . To fix one basis of this support, similarly towhat is done in [52, Sec. 3], we can specify S = 1, or equivalently, set the firstcolumn of S to be (1 0 · · · T , and take an r × r invertible submatrix of S andspecify it to be the identity matrix; thus the system has a single solution.Doing so, the resulting system is affine bilinear (without constant term), with( n − k − m equations and ( m − r + nr variables, and has a unique solution.For the sake of presentation, in Section 5 we present our results assuming thatthe first coordinate of e is nonzero and that the top r × r block of S is invertible;these results are easily extended to the general case. Under these assumptions,our system can be rewritten as follows: F = (cid:26)(cid:0) α · · · α m − (cid:1) (cid:18) I r S (cid:48) (cid:19) (cid:18) C − (cid:18) C (cid:48) (cid:19) R (cid:19)(cid:27) , (6)8here S (cid:48) is the ( m − r ) × ( r −
1) submatrix S { r +1 ..m } , { ..r } and C (cid:48) is the r × k submatrix C ∗ , { ..k +1 } . We call the entries of S (cid:48) the support variables whereasthe entries of C (cid:48) and C are called the coefficient variables . In Section 6.2 wegive a procedure to handle the general case, by making several attempts to findthe invertible block of S and a nonzero component of e . We refer to [23] for basic definitions and properties of monomial orderings andGr¨obner bases.
Field equations and monomial ordering
Since we are looking for solutionsin F q , we augment the polynomial system we want to solve with the field equa-tions, that is, the equation x qi − x i = 0 for each variable x i arising in the system.In our case, as the system we consider in practice has mainly only one solution in F q (see Section 6), the ideal of the system with the field equations is radical, andfor any monomial ordering the reduced Gr¨obner basis is the set of linear polyno-mials { x i − a i } i , where { x i } i are the variables and a i ∈ F q is the i -th coordinateof the solution. The classical approach consists in computing the Gr¨obner ba-sis with respect to a degree-reverse lexicographic order (grevlex), that will keepthe degree of the polynomials as small as possible during the computation, andbehaves usually better than other monomial orderings in terms of complexity. Generic Gr¨obner bases algorithms and their link with linear algebra
Since the first descriptions of algorithms to compute Gr¨obner bases [18], far moreefficient algorithms have been developed. On the one hand, substantial practicalspeed-ups were achieved by incorporating and accelerating fast linear algebraoperations such as Gaussian elimination on the Macaulay matrices, which aresparse and structured (see Faug`ere’s F4 algorithm [31], variants of the XL algo-rithm [22], and for instance GBLA [17]). We recall that the Macaulay matrix indegree d of a homogeneous system ( f i ) i is the matrix whose columns correspondto the monomials of degree d sorted in descending order w.r.t. a chosen mono-mial ordering, whose rows correspond to the polynomials tf i for all i where t is a monomial of degree d − deg( f i ), and whose entry in row tf i and column u is the coefficient of the monomial u in the polynomial tf i . In the case of a sys-tem containing field equations, we consider compact Macaulay matrices, whereall monomials are reduced w.r.t. the field equations. For an affine system, theMacaulay matrix in degree d contains all polynomials { tf i } for deg( tf i ) ≤ d andthe columns are the monomials of degree less than or equal to d .The approaches from F4 or XL are similar in that they both compute rowechelon forms of some submatrices of Macaulay matrices for some given degree;in fact, it was proven in [11] that the XL algorithm computes a so-called d -Gr¨obner basis, which is a basis of the initial system where all computations indegree larger than d are ignored, and that one can rephrase the original XLalgorithm in terms of the original F4 algorithm.9ow, many variants of these algorithms have been designed to tackle specificfamilies of polynomial systems, and it seems that none of them performs alwaysbetter than the others. In our experimental considerations, we rely on the im-plementation of the F4 algorithm which is available in magma V2.22-2 and isrecognised for its efficiency.On the other hand, improvements have been obtained by refining criteriawhich allow one to avoid useless computations (avoiding to consider monomi-als that cannot appear, a priori detection of reductions to zero as in the F5algorithm [32] and other signature-based algorithms that followed, see [30] for asurvey). Complexity analysis for homogeneous systems
For homogeneous systems,and for a graded monomial ordering, the complexity of these algorithms in termsof arithmetic operations is dominated by the cost of the row echelon forms on allMacaulay matrices up to degree d , where d is the largest degree of a polynomialin the reduced Gr¨obner basis . This degree d is called the index of regularity , or degree of regularity , and it only depends on the ideal generated by the system,not on the specific generators forming the system. Some algorithms may needto go beyond degree d to check that no new polynomials will be produced,like the XL Algorithm or the F4 Algorithm without the F5 criteria, but thosecomputations may be avoided if one knows in advance the degree of regularityof the system. This parameter can be precisely estimated for different familiesof generic systems, using the notions of regularity, of semi-regularity in the over-determined case, and of bi-regularity in the bilinear case [12,15,14,35]. However,those bounds may be very pessimistic for other specific (sub-)families of systems,and deriving estimations in this situation is difficult a priori, in particular foraffine systems. Definition 1.
Let ( f i ) i be (non necessarily homogeneous) polynomials in a poly-nomial ring R . A syzygy is a vector ( s i ) i , s i ∈ R such that (cid:80) i s i f i = 0 . Thedegree of the syzygy is defined as max i (deg( f i ) + deg( s i )) . The set of all syzygiesof ( f i ) i is an R -module called the syzygy module of ( f i ) i . For a given family of systems, there are syzygies that occur for any systemin the family. For instance, for any system ( f i ) i , the syzygy module contains the R -module spanned by the so-called trivial syzygies ( e j f i − e i f j ) i,j , where e i isthe coordinate vector with 1 at index i . A system is called regular if its syzygymodule is generated by these trivial syzygies.Let us consider the particular case of a zero-dimensional system ( f i ) i ofhomogeneous polynomials, generating an ideal I . As the system is homogenousand has a finite number of solution, then it must have only 0 as a solution(with maybe some multiplicities). In this case, the degree of regularity of thesystem is the lowest integer d reg such that all monomials of degree d reg are in If the system contains redundant polynomials of degree larger than d , additionaloperations are needed to check that those polynomials reduce to zero w.r.t. theGr¨obner basis, but this has usually a negligible cost. I (see [12,15]). Such a system is called semi-regular if the set of its syzygies of degree less than d reg ( I ) is exactly the set of trivialsyzygies of degree less than d reg ( I ). Note that there may be non-trivial syzygiesin degree d reg ( I ), which may be different for each system. As a consequence, allpolynomials occurring in the computation of a Gr¨obner basis have degree ≤ d reg and the arithmetic complexity is bounded by the cost of the row echelon formon the Macaulay matrices in degree ≤ d reg . Complexity analysis for affine systems
For affine systems, things are differ-ent. The degree of regularity can be defined in the same way w.r.t. the Gr¨obnerbasis for a grevlex ordering. But is not any more related to the complexity of thecomputation: for instance, a system with only one solution will have a degree ofregularity equal to 1. We need another parameter to control the complexity ofthe computation.Let ( f i ) i be a system of affine polynomials, and f hi the homogeneous part ofhighest degree of f i . Let I = (cid:104){ f i } i (cid:105) and I h = (cid:104){ f hi } i (cid:105) , and let d h reg be the degreeof regularity of I h . What may happen is that, during the computation of thebasis in some degree d , some polynomials of degree less than d may be added tothe basis. This will happen any time a syzygy ( s hi ) i for ( f hi ) i of degree d is suchthat there exists no syzygy ( s i ) i for ( f i ) i where s hi is the homogeneous part ofhighest degree of s i . In that case, (cid:80) i s hi f i is a polynomial of degree less than d (the homogeneous part of highest degree cancels), that will not be reduced tozero during the Gr¨obner basis computation since this would give a syzygy ( s i ) i for ( f i ) i with homogeneous part ( s hi ) i . This phenomenon is called a degree fall in degree d , and we will call such syzygies ( s hi ) that cannot be extended to syzy-gies for ( f i ) i in the same degree partial syzygies ; the corresponding polynomial (cid:80) i s hi f i is called the residue .In cryptographic applications, the first degree fall d ff has been widely used asa parameter controlling the complexity in algebraic cryptanalysis, for instancein the study of some HFE-type systems [29,43,25] and Kipnis-Shamir systems[56]. This first degree fall is simply the smallest d such that there exists a degreefall in degree d on ( f i ) i , and this quantity does depend on ( f i ) i : it might bedifferent for another set of generators of the same ideal. Still, this notion takeson its full meaning while computing a Gr¨obner basis for a graded ordering, if weadmit that the algorithm terminates shortly after reaching the first degree falland without considering polynomials of higher degree. This can happen for somefamilies of systems, as explained in the next paragraph, but there are examplesof systems where the first degree fall d ff is not the maximal degree reachedduring the computation, in which case it is not related to the complexity of thecomputation.If the system ( f hi ) i is semi-regular, then the computation in degree less than d h reg will act as if the polynomials where homogeneous: there cannot be degreefalls, as they would correspond to syzygies for the system ( f hi ) i that is assumedto be semi-regular. In degree d h reg , degree falls will occur for the first time, but atthis point the remainder of the computation is negligible compared to the previ-ous ones: by definition of d h reg , all monomials of degree d h reg are leading terms of11olynomials in the basis, and the remaining steps in the computation will nec-essarily deal with polynomials of degree at most d h reg . Hence, the computationsare almost the same as the ones for ( f hi ) i , and the complexity is controlled by d h reg , which is here the first degree fall for the system ( f i ) i .The behavior of the computation may be very different if degree falls occur ina much smaller degree. A good example of what may happen for particular fami-lies of systems is the affine bilinear case. It is proven in [35, Prop. 5] that a genericaffine bilinear system of m equations ( f , . . . , f m ) ∈ K [ x , . . . , x n x , y , . . . , y n y ]in n x + n y ≥ m variables is regular. In particular, the Macaulay bound d reg ≤ n x + n y + 1 applies [46]. However, it was also proven in [35, Thm. 6] that fora zero-dimensional affine bilinear system ( m = n x + n y ), d reg satisfies a muchsharper inequality d reg ≤ min( n x + 1 , n y + 1). The reason is that (homogeneous)bilinear systems are not regular, but the syzygy module of those systems is wellunderstood [35]. In particular, there are syzygies for ( f hi ) i coming from Jacobianmatrices, that are partial syzygies for ( f i ) i and produce degree falls.For affine systems, that are mainly encountered in cryptographic applica-tions, and in particular for systems coming from a product of matrices whosecoefficients are the variables of the system, the Jacobian matrices have a veryparticular shape that is easily described, and leads to a series of degree falls thatreduces the degree of regularity of those systems. This is explained in detail inSection 5. It has been realized in [56] that the firstdegree fall in the Kipnis and Shamir modelling can be traced back to partialsyzygies obtained from low degree vectors in the kernel of the Jacobian of thebilinear part of a system either with respect to the kernel variables or the linearvariables. This argument can also be adapted to our case and Jacobians withrespect to the support variables are relevant here. To understand the relevanceof the Jacobians for bilinear affine systems over some field K in general, con-sider a bilinear affine system F = { f , . . . , f M } ⊂ K [ s , . . . , s t s , c , . . . , c t c ] of M equations in t s variables s and t c variables c . We denote by F h := { f h , . . . , f hM } the bilinear part of these equations. In other words each f i can be written as f i = f hi + r i , where each r i is affine and f ih is bilinear with respect to { s , . . . , s t s }∪{ c , . . . , c t c } .We define the Jacobian matrices associated to F h asJac S ( F h ) = ∂f h ∂s . . . ∂f h ∂s ts ... ... ... ∂f hM ∂s . . . ∂f hM ∂s ts and Jac C ( F h ) = ∂f h ∂c . . . ∂f h ∂c tc ... ... ... ∂f hM ∂c . . . ∂f hM ∂c tc . S ( F h ) is a matrix with linear entries in K [ c , . . . , c t c ] whereasJac C ( F h ) is a matrix with linear entries in K [ s , . . . , s t s ]. As shown in [56][Prop.1 & 2] vectors in the left kernel of these Jacobians yield partial syzygies. This isessentially a consequence of the following identities that are easily verified:Jac S ( F h ) s ... s t s = f h ... f hM and Jac C ( F h ) c ... c t c = f h ... f hM . For instance, a vector ( g , . . . , g M ) in the left kernel of Jac C ( F h ) is a syzygy for F h , as it satisfies M (cid:88) i =1 g i f hi = ( g · · · g M ) f h ... f hM = ( g · · · g M ) Jac C ( F h ) c ... c t c = 0 . This gives typically a degree fall for F at degree 2 + max(deg g i ), with thecorresponding residue given by M (cid:88) i =1 g i f i = M (cid:88) i =1 g i f hi + M (cid:88) i =1 g i r i = M (cid:88) i =1 g i r i . These Jacobians are matrices with entries that are linear forms. The kernel ofsuch matrices is well understood as shown by the next result.
Theorem 1 ([35]).
Let M be an M × t matrix of linear forms in K [ s , . . . , s t s ] .If t < M , then generically the left kernel of M is generated by vectors whosecoefficients are maximal minors of M , specifically vectors of the form V J = ( . . . , (cid:124)(cid:123)(cid:122)(cid:125) j / ∈ J , . . . , ( − l +1 det( M J \{ j } , ∗ ) (cid:124) (cid:123)(cid:122) (cid:125) j ∈ J,j = j l , . . . ) ≤ j ≤ M where J = { j < j < · · · < j t +1 } ⊂ { , . . . , M } , J = t + 1 . A direct use of this result however yields degree falls that occur for verylarge degrees, namely at degrees t s + 2 or t c + 2. In the case of the Kipnis-Shamirmodelling, the syndrome modelling or the Ourivski-Johansson modelling, due tothe particular form of the systems, degree falls occur at much smaller degreesthan for generic bilinear affine systems. Roughly speaking, the reason is that theJacobian of a system coming from a matrix product splits as a tensor product, aswe now explain. This has been realized in [56] for the Kipnis-Shamir modelling,and here we slightly generalize this result in order to use it for more generalmodellings, and in particular for the Ourivski-Johansson modelling.13 acobian matrices of systems coming from matrix products. Considera system
AXY = where A = ( a i,s ) ≤ i ≤ m, ≤ s ≤ p , X = ( x s,t ) ≤ s ≤ p, ≤ t ≤ r and Y = ( y t,j ) ≤ t ≤ r, ≤ j ≤ n . The variables considered for this Jacobian matrix arethe x s,t . The matrices A and Y may have polynomial coefficients, but theydo not involve the x s,t variables. Below, we use the Kronecker product of twomatrices, for example A ⊗ Y T = (cid:0) a i,s Y T (cid:1) ≤ i ≤ m, ≤ s ≤ p . We use the notations vec row ( A ) = (cid:0) A { } , ∗ . . . A { n } , ∗ (cid:1) T and vec col ( A ) = vec row ( A T ). Lemma 1.
The Jacobian matrix of the system
AXY = m × n with respect tothe variables X can be written, depending on the order of the equations andvariables: Jac vec col ( X ) ( vec col ( AXY )) = Y T ⊗ A ∈ K [ A , Y ] nm × rp Jac vec row ( X ) ( vec row ( AXY )) = A ⊗ Y T ∈ K [ A , Y ] nm × rp . Proof.
For 1 ≤ i ≤ m , 1 ≤ j ≤ n , the equation in row i and column j of AXY is f i,j = p (cid:88) s =1 r (cid:88) t =1 a i,s x s,t y t,j . We then have, for 1 ≤ s ≤ p and 1 ≤ t ≤ r , ∂f i,j ∂x s,t = a i,s y t,j so that in row order,Jac x s, ,...,x s,r ( { f i, , . . . , f i,n } ) = (cid:18) ∂f i,j ∂x s,t (cid:19) ≤ j ≤ n ≤ t ≤ r = a i,s ( y t,j ) ≤ j ≤ n ≤ t ≤ r = a i,s Y T . The result follows from the definition of the Kronecker product of matrices. Theproof when the equations and variables are in column order is similar. (cid:117)(cid:116)
Application to the Kipnis-Shamir modelling.
Recall the system: (cid:32) km (cid:88) i =1 x i M i (cid:33) (cid:18) I n − r K (cid:19) = m,n − r , (7)where M i ∈ F m × nq and K is an r × ( n − r ) matrix of indeterminates. If we writeeach M i = ( M (cid:48) i M (cid:48)(cid:48) i ) with M (cid:48) i ∈ F m × ( n − r ) q and M (cid:48)(cid:48) i ∈ F m × rq , then we have km (cid:88) i =1 x i (cid:0) M (cid:48) i + M (cid:48)(cid:48) i K (cid:1) = m,n − r (KS)The bilinear and linear parts of the system are respectively (cid:80) kmi =1 x i M (cid:48)(cid:48) i K and (cid:80) kmi =1 x i M (cid:48) i . Using Lemma 1 (with equations in column order), when we computethe Jacobian with respect to the entries of K (the so-called kernel variables in[56]), we obtainJac vec col ( K ) ( vec col ( km (cid:88) i =1 x i M (cid:48)(cid:48) i K )) = km (cid:88) i =1 x i ( I n − r ⊗ M (cid:48)(cid:48) i ) = I n − r ⊗ (cid:32) km (cid:88) i =1 x i M (cid:48)(cid:48) i (cid:33) . vec col ( K ) is generated by the vectors ( v , . . . , v n − r ) with v l inthe left kernel of M = (cid:80) kmi =1 x i M (cid:48)(cid:48) i , that should be generated by (cid:0) mr +1 (cid:1) vectorsof minors, according to Theorem 1. Hence the kernel of Jac vec col ( K ) is generatedby (cid:0) mr +1 (cid:1) ( n − r ) vectors. It is here that we see the point of having this tensorproduct form. These kernel vectors have entries that are polynomials of degree r by using Theorem 1. This gives degree falls at degree r + 2 and yields partialsyzygies that have degree r + 1. These considerations are a slightly different wayof understanding the results given in [56, § r + 2 for the very same reason as can bereadily verified. Let us apply now Lemma 1 to the Ourivski-Johansson modelling. Application to the Ourivski-Johansson modelling.
The system here is F = (cid:26)(cid:0) α · · · α m − (cid:1) (cid:18) I r S (cid:48) (cid:19) (cid:18) C − (cid:18) C (cid:48) (cid:19) R (cid:19)(cid:27) , (8)where S (cid:48) is the ( m − r ) × ( r −
1) matrix S { r +1 ..m } , { ..r } and C (cid:48) is the r × k matrix C ∗ , { ..k +1 } . We add to F the field equations F q = { s qi,j − s i,j , r + 1 ≤ i ≤ m, ≤ j ≤ r, c qi,j − c i,j , ≤ i ≤ r, ≤ j ≤ n } .With high probability, this system has a unique solution. As we used thefield equations, the ideal (cid:104)F , F q (cid:105) is radical. The system has n S = ( m − r )( r − S , n C = ( n − r variables C , and n − k − F q m , hence n eq = ( n − k − m equations over F q , plus the field equations.Consider the system F h formed by the bilinear parts of the equations in F .A simple computation shows that F h = (cid:8) α r (cid:0) α · · · α m − r − (cid:1) S (cid:48) ( C (cid:48)(cid:48) − C (cid:48)(cid:48) R (cid:48) ) (cid:9) , where C (cid:48)(cid:48) = C { ..r } , { k +2 ..n } , C (cid:48)(cid:48) = C { ..r } , { ..k +1 } and R (cid:48) = R { ..k +1 } , ∗ .If we take the equations and variables in row order, and use Lemma 1, thenJac vec row ( S ) ( vec row ( F h )) = α r (cid:0) α · · · α m − r − (cid:1) ⊗ (cid:0) C (cid:48)(cid:48) − C (cid:48)(cid:48) R (cid:48) (cid:1) T (9)The elements in the left kernel of Jac vec row ( S ) ( vec row ( F h )) are those in the rightkernel of C (cid:48)(cid:48) − C (cid:48)(cid:48) R (cid:48) , and applying Theorem 1, they belong to the vector spacegenerated by the vectors V J for any J = { j < j < · · · < j r } ⊂ { , . . . , n − k − } of size r defined by V J = ( . . . , (cid:124)(cid:123)(cid:122)(cid:125) j / ∈ J , . . . , ( − l +1 det( C (cid:48)(cid:48) − C (cid:48)(cid:48) R (cid:48)∗ ,J \{ j } ) (cid:124) (cid:123)(cid:122) (cid:125) j = j l ∈ J , . . . ) ≤ j ≤ n − k − . Each V J gives a syzygy for F h and when applying it to F it yields a degreefall in degree r + 1 because the entries of V J are homogeneous polynomials ofdegree r −
1. The inner product of V J with the vector of the equations gives anequation of degree ≤ r since the homogeneous part of highest degree cancels, ashas been observed at the beginning of this section. Now the affine part of theequations F is (cid:0) α · · · α r − (cid:1) ( C − C R ).15ritting ˜ H = (cid:0) − R T I n − k − (cid:1) , thendet( C (cid:48)(cid:48) − C (cid:48)(cid:48) R (cid:48)∗ ,J \{ j } ) = det(( C ˜ H T ) { ..r } ,J \{ j } ) . Using the reverse of Laplace’s formula expressing a determinant in terms ofminors, we can compute the inner product of the vector V J with the i th row of C − C R = C ˜ H T , that is 0 for 2 ≤ i and det(( C ˜ H T ) ∗ ,J ) for i = 1.The product gives V J (cid:0)(cid:0) α · · · α r − (cid:1) ( C − C R ) (cid:1) T = V J ( C − C R ) T (cid:0) α · · · α r − (cid:1) T = det( C − C R ) ∗ ,J . (10)This yields a corresponding equation that will be reduced to zero by a degree-( r + 1) Gr¨obner basis of F . Hence the partial syzygies of degree r coming fromthe degree fall in the ( r + 1)-Macaulay matrix are exactly the maximal minorsof C − C R . We have thus proven that Theorem 2.
The equations
MaxMinors( C − C R ) = 0 , that are the maximalminors of the matrix C − C R , belong to the ideal (cid:104)F , F q (cid:105) . Moreover, they arereduced to zero by a degree ( r + 1) -Gr¨obner basis of {F , F q } .Remark 1. If we are only interested in the first part of the theorem about themaximal minors, then there is a simple and direct proof which is another illus-tration of the role of the matrix form of the system. Indeed, let ( S ∗ , C ∗ ) be asolution of {F , F q } , then the nonzero vector (cid:0) S ∗ · · · S ∗ m (cid:1) = (cid:0) α · · · α m − (cid:1) S ∗ belongs to the left kernel of the matrix C ∗ − C ∗ R . Hence this matrix has rankless than r , and the equations MaxMinors( C − C R ) = 0 are satisfied for anysolution of the system {F , F q } , which means that the equations belong to theideal (cid:104)F , F q (cid:105) as this ideal is radical. C − C R ) The previous theorem allows us to obtain directly degree r equations withouthaving to compute first the Macaulay matrix of degree r + 1. This is a signifi-cant saving when performing the Gr¨obner basis computation. A nice feature ofthese equations is that they only involve one part of the unknowns, namely thecoefficient variables.Moreover all these equations can be expressed by using a limited number ofpolynomials as we now show. Some of them will be of degree r , some of themwill be of degree r −
1. If we perform Gaussian elimination on these equations bytreating these polynomials as variables and trying to eliminate the ones corre-sponding to the polynomials of degree r first, then if the number of equations wehad was greater than the number of polynomials of degree r , we expect to findequations of degree r −
1. Roughly speaking, when this phenomenon happenswe just have to add all the equations of degree r − r . 16et us analyse precisely the behavior we just sketched. The shape of theequations MaxMinors( C − C R ) = 0 is given by the following proposition,where by convention det( M ∅ , ∅ ) = 1 and the columns of R are indexed by { k +2 ..n } : Proposition 1.
MaxMinors( C − C R ) is a set of (cid:0) n − k − r (cid:1) polynomials P J ,indexed by J ⊂ { k + 2 ..n } of size r : P J = (cid:88) T ⊂{ ..k +1 } ,T ⊂ J such that T = T ∪ T has size T = r ( − σ J ( T ) det( R T ,J \ T ) det( C ∗ ,T ) . where σ J ( T ) is an integer depending on T and J .If / ∈ T , the polynomial det( C ∗ ,T ) is homogeneous of degree r and contains r ! monomials; if ∈ T , det( C ∗ ,T ) is homogeneous of degree r − and contains ( r − monomials.Proof. The matrix C − C R has size r × ( n − k − (cid:0) n − k − r (cid:1) different minors P J = det( C ( − RI n − k − ) ∗ ,J ). To compute them, we use the Cauchy-Binet formula for the determinant of a product of non-square matrices:det( AB ) = (cid:88) T ⊂{ ..p } , T = r det( A ∗ ,T ) det( B T, ∗ )where A ∈ K r × p , B ∈ K p × r , and p ≥ r . We apply this formula to P J , and usethe fact that, for T = T ∪ T with T ⊂ { ..k + 1 } and T ⊂ { k + 2 ..n } ,det (cid:32)(cid:18) − RI n − k − (cid:19) T ∪ T ,J (cid:33) = 0 if T (cid:54)⊂ J = ( − σ J ( T ) det( R T ,J \ T ) if T ⊂ J, using the Laplace expansion of this determinant along the last rows, with σ J ( T ) = d ( k + r ) + ( d − d/ (cid:80) t ∈ T P os ( t, J ) where P os ( t, J ) is the position of t in J , and d = J − T . (cid:117)(cid:116) Each polynomial P J can be expanded into m equations over F q , the polynomial P J [ i ] being the coefficient of P J in α i − . When computing a grevlex Gr¨obnerbasis of the system of the P J [ i ]’s over F q , with an algorithm like F4 using linearalgebra, the first step consists in computing a basis of the P J [ i ]’s over F q .It appears that there may be a fall of degree in this first step, in degree r ,that produces equations of degree r −
1. The following heuristic explains whenthis fall of degree occurs.
Heuristic 1 –
Overdetermined case: when m (cid:0) n − k − r (cid:1) ≥ (cid:0) nr (cid:1) − , generically,a degree- r Gr¨obner basis of the projected system
MaxMinors( C − C R ) = 0 of m (cid:0) n − k − r (cid:1) equations over F q contains (cid:0) n − r − (cid:1) − equations of degree r − ,that are obtained by linear combinations of the initial equations. Intermediate case: when (cid:0) nr (cid:1) − > m (cid:0) n − k − r (cid:1) > (cid:0) n − r (cid:1) , generically a degree- r Gr¨obner basis of the projected system
M axM inors ( C − C R ) = 0 contains m (cid:0) n − k − r (cid:1) − (cid:0) n − r (cid:1) equations of degree r − , that are obtained by linearcombinations of the initial equations. – Underdetermined case:
When m (cid:0) n − k − r (cid:1) ≤ (cid:0) n − r (cid:1) , then generically a degree- r Gr¨obner basis of the system contains m (cid:0) n − k − r (cid:1) polynomials that are all ofdegree r .Remark 2. Here overdetermined/underdetermined refers to the system of max-imal minors given by the set of equations MaxMinors( C − C R ) = 0 Remark 3.
The degree- r Gr¨obner bases also contain polynomials of degree r inthe overdetermined and intermediate cases, but we will not compute them, asexperimentally they bring no speed-up to the computation, see Section 6.1. Proposition 2.
Computing the polynomials in a degree- r Gr¨obner basis of theprojected equations
MaxMinors amounts to solving a linear system with ν = m (cid:0) n − k − r (cid:1) equations in µ = (cid:0) nr (cid:1) variables, which costs O (min( µ, ν ) ω − µν ) op-erations in the base field, where ω is the exponent of matrix multiplication (seeSection 6.2).Proof. It is possible to view the system MaxMinors( C − C R ) projected over F q as a linear system of µ = m (cid:0) n − k − r (cid:1) equations, whose variables are the ν = (cid:0) nr (cid:1) unknowns x T = det( C ∗ ,T ) for all T ⊂ { ..n } of size r . The matrix associated tothis linear system is a matrix M of size µ × ν whose coefficient in row ( i, J ) : i ∈ { ..m } , J ⊂ { k + 2 ..n } , J = r , and column x T is, with T = T ∩ { k + 2 ..n } : M [( i, J ) , x T ] = (cid:40) [ α i − ]( − σ J ( T ) det( R T ∩{ ..k +1 } ,J \ T ) if T ⊂ J, α i − ] β is the i st component of β ∈ F q m viewed in the vector space F mq with generator basis (cid:0) α . . . α m − (cid:1) .A basis of the vector space generated by the equations MaxMinors( C − C R ) = 0 is given by ˜ M · T where ˜ M is the row echelon form of M and T isthe column vector formed by the polynomials det( C ∗ ,T ) : T = r . As we aresearching for equations of degree r −
1, we order the variables x T such that theones with 1 ∈ T that correspond to polynomials det( C ∗ ,T ) of degree r − (cid:117)(cid:116) Heuristic 1 can be stated in terms of the matrix M . In the overdeterminedcase, that is when m (cid:0) n − k − r (cid:1) ≥ (cid:0) nr (cid:1) −
1, we expect matrix M to have rank (cid:0) nr (cid:1) − M · T produces (cid:0) n − r (cid:1) equations of degree r , and (cid:0) n − r − (cid:1) − r −
1, that have all the shape det( C ∗ ,T ) or det( C ∗ ,T ) − det( C ∗ ,T ) where T corresponds to the free variable x T of the linear system,1 ∈ T . In the intermediate and underdetermined cases, we also expect matrix M to be full rank in general, and to be also full rank on the columns correspondingto the c T ’s of degree r . 18 Experimental results, complexity bounds, and security
We did various computations for different values of the parameters ( m, n, k, r ).We got our best complexity results by doing the following steps:1. compute the set of equations F which comes from (cid:0) α · · · α m − (cid:1) S ( C − C R )specialised as in (6),2. compute the system MaxMinors( C − C R ),3. compute the matrix M from (11) and its echelon form ˜ M , let J be the setof the resulting equations of degree r − C variables,4. if J is empty, then let J be the set of equations coming from ˜ M of degree r in the C variables,5. compute G a reduced degree- d Gr¨obner basis of the system {F , J , F q } ,where d = r in the overdetermined case ,r or r + 1 in the intermediate case ,r + 2 in the underdetermined case . The computations are done using magma v2.22-2 on a machine with an Intel R (cid:13) Xeon R (cid:13) • n S = ( r − m − r ): the number of variables in S • n C = r ( n − C • n eq = m ( n − k − F• d : n syz : the number of equations in J , where d denotes the degree of theequations and n syz the number of them: • r − (cid:0) n − r − (cid:1) − • r − m (cid:0) n − k − r (cid:1) − (cid:0) n − r (cid:1) in the intermediate case • r : m (cid:0) n − k − r (cid:1) in the underdetermined case • T syz. : time of computing the n syz equations of degree r − r in J• T Gbsyz : time of the Gr¨obner basis computation of {J , F q }• T Gb : time of the Gr¨obner basis computation of {F , J , F q }• d ff : the degree where we observe the first fall of degree • d max : the maximal degree where some new polynomial is produced by theF4 algorithm • “Max Matrix size”: the size of the largest matrix reduced during the F4computation, given by magma . We did not take into account the useless steps(the matrices giving no new polynomials)Table 1 page 20 gives our timings on the parameters proposed in [55]. Foreach set of parameters, the first row of the table gives the timing for the directcomputation of a Gr¨obner basis of {F , F q } whereas the second row gives thetimings for the Gr¨obner basis of {F , J , F q } . We can see that, apart from verysmall parameters, the computation of the equations MaxMinors( C − C R ) isnegligible compared to the time of the Gr¨obner basis computation.19mong the proposed parameters, only the (15 , , ,
3) was in the case wherethe system MaxMinors is underdetermined. In that case, the most consumingpart of the computation is the Gr¨obner basis of the system MaxMinors, thatdepends only on the C variables. Once this computation is done, the remainingGr¨obner basis of {F , J , F q } has a negligible cost.Table 2 page 21 gives timing for different values of k and r , with m =14 and n = 18 fixed. For r = 2, the values k ∈ { .. } correspond to theoverdetermined case, the value k = 12 to the intermediate one, and k = 13 tothe underdetermined case. The values k ∈ { .. } behave all like k = 11. Asfor the parameters from [55], the hardest cases are the ones when the systemMaxMinors is underdetermined, where the maximal degree reached during thecomputation is r + 2. For the overdetermined cases, the maximal degree is r , andfor the intermediate cases, it may be r or r + 1.For r = 3, the overdetermined cases are k ∈ { .. } , k = 9 is intermediateand k ∈ { .. } are underdetermined. Values of k ≥
12 do not allow a uniquedecoding for r = 3, the Gilbert-Varshamov bound being 2 for those values.For r = 4 the tradeoffs are 1 ≤ k ≤ k = 7 and 8 ≤ k ≤ r = 5, 1 ≤ k ≤ k = 6 and 7 ≤ k ≤
8. We could not performthe computations for the intermediate and underdetermined cases, due to a lackof memory. We also observe that the first fall of degree ( d ff ) does not alwayspredict the complexity of the computation.Table 3 page 21 gives the timings for a fixed r = 3, a ratio n = 2 k andvarious values of k . Again, we can observe that for defavorable cases ( k = 6 , r + 2 or r + 1 rather than r , making the computationharder for small values of k than for larger. Table 1.
We compare the behavior of the Gr¨obner basis computation for the param-eters considered in [48], with and without adding to the system the equations J . m n k r n S n C n eq n syz T syz T Gbsyz T Gb d ff d max Max Mat Size25 30 15 2 23 58 350 0.4 s 3 3 18550 × × × × × × × × × × ≥ × × × able 2. m = 14 and n = 18. k r n syz n S n C n eq T Syz. T Gbsyz T Gb d ff d max Max Matrix size Mem.11 2 1:16 12 34 84 < . < .
1s 2 2 322 ×
251 34 Mo12 2 1:4 12 34 70 < . < .
1s 3 3 1820 × < .
1s 32 s 0 s 3 4 231187 × × × × × × × × Table 3.
The parameters are r = 3, m = n , k = n . k n syz n S n C n eq T syz T Gbsyz T Gb d max Memory6 3:120 18 33 60 0.2s 117 s 0.02s 5 4.9 Go7 3:280 22 39 84 0.1s 9.7 s 0.1s 4 0.3 Go8 2:104 26 45 112 0.2s 0.1s 3 .04 Go17 2:527 62 99 544 34.3s 4.7s 3 0.3 Go27 2:1377 102 159 1404 650.2s 161.3s 3 2.7 Go37 2:2627 142 219 2664 5603.6s 3709.4s 3 15.0 Go47 2:4277 182 279 4324 26503.9s 26022.6s 3 83.0 Go F Now, we give an upper bound on the complexity of our algebraic approach tosolve the ( m, n, k, r )-decoding problem using the modelling of Section 3.3. Thecomplexity is estimated in terms of the number of operations in F that thealgorithm uses. This allows us to update the number of bits of security forseveral cryptosystems, as showed in Table 4: Loidreau’s one [49], ROLLO [7],and RQC [3]. Note that the restriction to F is only there because we want toderive security values. If one works over a larger field F q , a similar analysis canbe derived. The only change in this case is to consider the relevant number ofmonomials. Note also that even if Algorithm 1 works over any field, its successprobability given in Proposition 3 depends on q .Remark that, in Table 4, for the sets of parameters which do not satisfyEq. (1), which correspond to underdetermined instances, we assume that thesystem can be solved at d = r + 1. It is a conservative choice: in the experimentsof Section 6.1, the maximal degree is often r for the underdetermined cases.The complexity bound follows from the fact that the Gr¨obner basis algorithmworks with Macaulay matrices of degree δ for increasing values of δ up to d , thedegree for which the Gr¨obner basis is found (see Section 4 for a more detaileddescription). At each of these steps, the algorithm performs a Gaussian elimi-nation algorithm on a Macaulay matrix which has at most (cid:0) ( m − r )( r − n − rδ (cid:1) ryptosystem Parameters ( m, n, k, r ) d = r d = r + 1 Previous
Loidreau (128 , , , , , , , , , , , ,
7) 174.0 , , , , , , , , ,
7) 191.6 , , , , , , , , ,
7) 175.9 , , , , , , , , ,
7) 188.4
Table 4.
Security in bits for several cryptosystems with respect to our attack, com-puted using Eq. (12) with ω = 2 . d = r or d = r + 1. The values in bold correspondthe most likely maximal degree, i.e. r if Eq. (1) holds and r + 1 otherwise. The lastcolumn gives the previous best known security values, based on the attack in [10]. columns and fewer rows than columns. The number of columns is the number ofsquarefree monomials of degree δ in ( m − r )( r −
1) + ( n − r variables.In general, Gaussian elimination of a µ × ν matrix of rank ρ over a field hasa complexity of O ( ρ ω − µν ) ⊆ O (max( µ, ν ) ω ) operations in that field [19,54].Here, ω is the exponent of matrix multiplication, with naive bounds 2 ≤ ω ≤ ω is ω ≈ .
37 [47], by an improvement ofCoppersmith-Winograd’s algorithm. In terms of practical performances, the bestknown method is based on Strassen’s algorithm, which allows one to take ω ≈ . F [4],also for sparse matrices [16], and even to take advantage of the specific structureof Macaulay matrices (see [17]; we expect Magma ’s closed-source implementationof F to use similar techniques). However, none of these optimized algorithmshas been proven to reach a complexity which is asymptotically better than theone mentioned above, apart from speed-ups by constant factors.As a result, we bound the complexity of the step of degree δ in the Gr¨obnerbasis computation by that of performing Gaussian elimination on a µ × ν matrixover F , with µ ≤ ν = (cid:0) ( m − r )( r − n − rδ (cid:1) ; the overall computation then costs O (cid:32)(cid:32) d (cid:88) δ =0 (cid:18) ( m − r )( r −
1) + ( n − rδ (cid:19)(cid:33) ω (cid:33) (12)22perations in F . Let us now focus on the case m = n = 2 k and r ≈ √ n . Thenthe complexity of our approach is as in Eq. (12) with d = r . Using a similaranalysis, the approach based on Kipnis-Shamir’s modelling has a complexity of O (cid:32)(cid:32) r +2 (cid:88) δ =0 (cid:18) km + r ( n − r ) δ (cid:19)(cid:33) ω (cid:33) operations. Asymptotically, the dominant term in the former bound is of theorder of 2 ωr log ( n ) , to be compared to 2 ωr log ( n ) in the Kipnis-Shamir bound.Also, the aforementioned combinatorial attacks ([10]) would have a complexityof the order of 2 rn when m = n = 2 k .Finally, note that the complexity bound stated above was derived underassumptions: in Section 3.3, we presented the modelling along with some as-sumptions which allowed us to specialize variables a priori and still ensure thatthe algorithm of Section 5 yields the solution λ e . In general, the assumptionmight not hold, that is, the specific specialization made in Section 3.3 could bewrong. We use Algorithm 1 in order to specialize more variables: it first usesthe specialization detailed in Section 3.3, and if that one fails, follows on withother similar specializations. This algorithm uses the subroutine Solve ( S , C , R ),which augments the system as explained in Section 5 and returns a solution toEq. (5) if one is found and ∅ otherwise. Input:
Matrix R Output:
A solution to the system in Eq. (5) or ∅ S = m × r matrix of variables C = r × n matrix of variablesSet the first column and the first row of S to [1 0 · · · C to [1 0 · · · T Choose at random (cid:98) m − r − (cid:99) disjoint subsets T i ⊆ { , . . . , m } of cardinality r − for i ← to (cid:98) m − r − (cid:99) do Set the ( r − × ( r −
1) submatrix S T i , { ,...,r } to I r − sol = Solve ( S , C , R ) if sol (cid:54) = ∅ then return sol return ∅ Algorithm 1: ( m, n, k, r )-DecodingFor positive integers a and b with a ≤ b , we denote by p q,a,b := (cid:81) a − i =0 (cid:0) − q i − b (cid:1) the probability that a uniformly random matrix in F a × bq has full rank. Proposition 3.
Fix integers m, n, k, r , and let c ∈ { , . . . , (cid:98) m − r − (cid:99)} . Suppose thata ( m, n, k, r ) -rank decoding instance is chosen uniformly at random, and that theinput matrix R is built from this instance. Then, the probability that Algorithm 1makes at most c calls to Solve ( S , C , R ) before finding a solution is greater than − q − r − q − n (cid:18) − (1 − p q,r − ,r − ) c p q,r − ,m − (cid:19) . Solve ( S , C , R ), Algorithm 1 will return a solution witha probability always greater than 0 .
8; note that for these instances the quantity (cid:98) m − r − (cid:99) is greater than 15, and around 20 for most of them.In the event where Algorithm 1 returns ∅ after (cid:98) m − r − (cid:99) calls to Solve ( S , C , R ),one can run it again until a solution is found. The probabilities mentioned inthe previous paragraph show that for parameters of interest a second run of thealgorithm is very rarely needed. In this paper we introduce a new approach for solving the Rank Metric Decodingproblem with Gr¨obner basis techniques. Our approach is based on adding partialsyzygies to a newer version of a modelling due to Ourivski and Johansson.Overall our analysis shows that our attack, for which we give a general es-timation, clearly outperforms all previous attacks in rank metric for a classical(non quantum) attacker. In particular we obtain an attack below the claimedsecurity level for all rank-based schemes proposed to the NIST Post-QuantumCryptography Standardization Process. Note that there has been some very re-cent progress [13] on the modelling and the attack proposed here. This results ineven less complex attacks and in the removal of the Gr¨obner basis computationstep: it is replaced by solving a linear system. Although our attack and its recentimprovement really improve on previous attacks for rank metric, they meanwhilesuffer from two limitations.First these attacks do not benefit from a direct Grover quantum speed-up,unlike combinatorial attacks. For the NIST parameters (with the exception ofRollo-I-192 for the latest attack [13]) the best quantum attacks still remain quan-tum attacks based on combinatorial attacks, because of the Grover speed-up.Second, these attacks need an important amount of memory for large parame-ters.
Acknowledgements
This work has been supported by the French ANR projects CBCRYPT (ANR-17-CE39-0007) and the MOUSTIC project with the support from the EuropeanRegional Development Fund (ERDF) and the Regional Council of Normandie.The authors would like to thank the anonymous reviewers for their valuablecomments and suggestions, as well as Ray Perlner and Daniel Smith for usefuldiscussions.
References
1. Aguilar Melchor, C., Aragon, N., Bettaieb, S., Bidoux, L., Blazy, O., Deneuville,J.C., Gaborit, P., Hauteville, A., Z´emor, G.: Ouroboros-R. First round submission o the NIST post-quantum cryptography call (Nov 2017), https://pqc-ouroborosr.org2. Aguilar Melchor, C., Aragon, N., Bettaieb, S., Bidoux, L., Blazy, O., Deneuville,J.C., Gaborit, P., Z´emor, G.: Rank quasi cyclic (RQC). First round submission tothe NIST post-quantum cryptography call (Nov 2017), https://pqc-rqc.org3. Aguilar Melchor, C., Aragon, N., Bettaieb, S., Bidoux, L., Blazy, O., Deneuville,J.C., Gaborit, P., Z´emor, G., Couvreur, A., Hauteville, A.: Rank quasi cyclic(RQC). Second round submission to the NIST post-quantum cryptography call(Apr 2019), https://pqc-rqc.org4. Albrecht, M., Bard, G.: The M4RI Library – Version 20140914. The M4RI Team(2014), http://m4ri.sagemath.org5. Aragon, N., Blazy, O., Deneuville, J.C., Gaborit, P., Hauteville, A., Ru-atta, O., Tillich, J.P., Z´emor, G.: LAKE – Low rAnk parity checkcodes Key Exchange. First round submission to the NIST post-quantumcryptography call (Nov 2017), https://csrc.nist.gov/CSRC/media/Projects/Post-Quantum-Cryptography/documents/round-1/submissions/LAKE.zip6. Aragon, N., Blazy, O., Deneuville, J.C., Gaborit, P., Hauteville, A., Ru-atta, O., Tillich, J.P., Z´emor, G.: LOCKER – LOw rank parity ChecKcodes EncRyption. First round submission to the NIST post-quantumcryptography call (Nov 2017), https://csrc.nist.gov/CSRC/media/Projects/Post-Quantum-Cryptography/documents/round-1/submissions/LOCKER.zip7. Aragon, N., Blazy, O., Deneuville, J.C., Gaborit, P., Hauteville, A., Ruatta, O.,Tillich, J.P., Z´emor, G., Aguilar Melchor, C., Bettaieb, S., Bidoux, L., Magali, B.,Otmani, A.: ROLLO (merger of Rank-Ouroboros, LAKE and LOCKER). Secondround submission to the NIST post-quantum cryptography call (Mar 2019), https://pqc-rollo.org8. Aragon, N., Blazy, O., Gaborit, P., Hauteville, A., Z´emor, G.: Durandal: a rankmetric based signature scheme. In: Advances in Cryptology - EUROCRYPT 2019- 38th Annual International Conference on the Theory and Applications of Cryp-tographic Techniques, Darmstadt, Germany, May 19-23, 2019, Proceedings, PartIII. LNCS, vol. 11478, pp. 728–758. Springer (2019). https://doi.org/10.1007/978-3-030-17659-4 25, https://doi.org/10.1007/978-3-030-17659-4 259. Aragon, N., Gaborit, P., Hauteville, A., Ruatta, O., Z´emor, G.: Ranksign –a signature proposal for the NIST’s call. First round submission to the NISTpost-quantum cryptography call (Nov 2017), https://csrc.nist.gov/CSRC/media/Projects/Post-Quantum-Cryptography/documents/round-1/submissions/RankSign.zip10. Aragon, N., Gaborit, P., Hauteville, A., Tillich, J.P.: A new algorithm for solvingthe rank syndrome decoding problem. In: 2018 IEEE International Symposium onInformation Theory, ISIT 2018, Vail, CO, USA, June 17-22, 2018. pp. 2421–2425.IEEE (2018). https://doi.org/10.1109/ISIT.2018.843746411. Ars, G., Faug`ere, J.C., Imai, H., Kawazoe, M., Sugita, M.: Comparison betweenXL and Gr¨obner basis algorithms. In: ASIACRYPT (2004)12. Bardet, M.: ´Etude des syst`emes alg´ebriques surd´etermin´es. Applications aux codescorrecteurs et `a la cryptographie. Ph.D. thesis, Universit´e Paris VI (Dec 2004),http://tel.archives-ouvertes.fr/tel-00449609/en/13. Bardet, M., Bros, M., Cabarcas, D., Gaborit, P., Perlner, R., Smith-Tone, D.,Tillich, J.P., Verbel, J.: Algebraic attacks for solving the Rank Decoding and Min-Rank problems without Gr¨obner basis. arXiv e-prints arXiv:2002.08322 (Feb 2020)14. Bardet, M., Faug`ere, J.C., Salvy, B.: On the complexity of the F Gr¨obner basisalgorithm. J. Symbolic Comput. , 49–70 (2015)
5. Bardet, M., Faug`ere, J.C., Salvy, B., Yang, B.Y.: Asymptotic expansion of thedegree of regularity for semi-regular systems of equations. In: MEGA’05 – EffectiveMethods in Algebraic Geometry. pp. 1–14 (2005)16. Bouillaguet, C., Delaplace, C.: Sparse Gaussian elimination modulo p: An update.In: Proceedings CASC 2016 – Computer Algebra in Scientific Computing. pp. 101–116. Springer International Publishing (2016)17. Boyer, B., Eder, C., Faug`ere, J., Lachartre, S., Martani, F.: GBLA: Gr¨obner basislinear algebra package. In: Proceedings of the ACM on International Symposiumon Symbolic and Algebraic Computation, ISSAC 2016, Waterloo, ON, Canada,July 19-22, 2016. pp. 135–142 (2016). https://doi.org/10.1145/2930889.293091418. Buchberger, B.: Ein Algorithmus zum Auffinden der Basiselemente des Restk-lassenringes nach einem nulldimensionalen Polynomideal. Ph.D. thesis, UniversitatInnsbruck (1965)19. Bunch, J.R., Hopcroft, J.E.: Triangular factorization and inversion by fast matrixmultiplication. Mathematics of Computation (125), 231–236 (1974)20. Buss, J.F., Frandsen, G.S., Shallit, J.O.: The computational complexity of someproblems of linear algebra. J. Comput. System Sci. (3), 572–596 (Jun 1999)21. Cabarcas, D., Smith-Tone, D., Verbel, J.: Key recovery attack for ZHFE. In:Post-Quantum Cryptography 2017. LNCS, vol. 10346, pp. 289–308. Utrecht,The Netherlands (Jun 2017). https://doi.org/10.1007/978-3-319-59879-6 17, https://doi.org/10.1007/978-3-319-59879-6 1722. Courtois, N., Klimov, A., Patarin, J., Shamir, A.: Efficient algorithms for solv-ing overdefined systems of multivariate polynomial equations. In: Preneel, B. (ed.)Advances in Cryptology - EUROCRYPT 2000. pp. 392–407. Springer Berlin Hei-delberg, Berlin, Heidelberg (2000)23. Cox, D., Little, J., O’Shea, D.: Ideals, Varieties, and algorithms: an Introductionto Computational Algebraic Geometry and Commutative Algebra. UndergraduateTexts in Mathematics, Springer-Verlag, New York. (2001)24. Debris-Alazard, T., Tillich, J.P.: Two attacks on rank metric code-based schemes:Ranksign and an identity-based-encryption scheme. In: Advances in Cryptology- ASIACRYPT 2018. LNCS, vol. 11272, pp. 62–92. Springer, Brisbane, Aus-tralia (Dec 2018). https://doi.org/10.1007/978-3-030-03326-2 3, https://doi.org/10.1007/978-3-030-03326-2 325. Ding, J., Hodges, T.J.: Inverting HFE systems is quasi-polynomial for all fields.In: Advances in Cryptology - CRYPTO (2011)26. Ding, J., Kleinjung, T.: Degree of regularity for HFE-. Cryptology ePrint Archive,Report 2011/570 (2011), http://eprint.iacr.org/2011/570, https://eprint.iacr.org/2011/57027. Ding, J., Schmidt, D.: Solving degree and degree of regularity for polynomialsystems over a finite fields. In: Number Theory and Cryptography - Papers inHonor of Johannes Buchmann on the Occasion of His 60th Birthday. LNCS,vol. 8260, pp. 34–49. Springer (2013). https://doi.org/10.1007/978-3-642-42001-6 4, https://doi.org/10.1007/978-3-642-42001-6 428. Ding, J., Yang, B.Y.: Degree of regularity for HFEv and HFEv-. In:Post-Quantum Cryptography 2013. pp. 52–66. Limoges, France (Jun2013). https://doi.org/10.1007/978-3-642-38616-9 4, https://doi.org/10.1007/978-3-642-38616-9 429. Dubois, V., Gama, N.: The degree of regularity of HFE systems. In: Advances inCryptology - ASIACRYPT 2010. LNCS, vol. 6477, pp. 557–576. Springer, Singa-pore (Dec 2010). https://doi.org/10.1007/978-3-642-17373-8 32, https://doi.org/10.1007/978-3-642-17373-8 32
0. Eder, C., Faug`ere, J.C.: A survey on signature-based algorithms for com-puting Gr¨obner bases. Journal of Symbolic Computation (1-3), 61–88 (1999)32. Faug`ere, J.C.: A new efficient algorithm for computing gr¨obner bases without re-duction to zero: F5. In: Proceedings ISSAC’02. pp. 75–83. ACM press (2002)33. Faug`ere, J.C., Levy-dit-Vehel, F., Perret, L.: Cryptanalysis of Minrank. In: Wagner,D. (ed.) Advances in Cryptology - CRYPTO 2008. LNCS, vol. 5157, pp. 280–296(2008)34. Faug`ere, J., Safey El Din, M., Spaenlehauer, P.: Computing loci of rankdefects of linear matrices using Gr¨obner bases and applications to cryp-tology. In: International Symposium on Symbolic and Algebraic Computa-tion, ISSAC 2010, Munich, Germany, July 25-28, 2010. pp. 257–264 (2010).https://doi.org/10.1145/1837934.183798435. Faug`ere, J.C., Safey El Din, M., Spaenlehauer, P.J.: Gr¨obner bases of bihomoge-neous ideals generated by polynomials of bidegree (1,1): Algorithms and complex-ity. J. Symbolic Comput. (4), 406–437 (2011)36. Gabidulin, E.M.: Theory of codes with maximum rank distance. ProblemyPeredachi Informatsii (2), 1006–1019 (2016)40. Gaborit, P., Ruatta, O., Schrek, J., Z´emor, G.: New results for rank-based cryp-tography. In: Progress in Cryptology - AFRICACRYPT 2014. LNCS, vol. 8469,pp. 1–12 (2014)41. Gaborit, P., Ruatta, O., Schrek, J., Z´emor, G.: Ranksign: An efficient signaturealgorithm based on the rank metric (extended version on arxiv). In: Post-QuantumCryptography 2014. LNCS, vol. 8772, pp. 88–107. Springer (2014), https://arxiv.org/pdf/1606.00629.pdf42. Gaborit, P., Z´emor, G.: On the hardness of the decoding and the minimum distanceproblems for rank codes. IEEE Trans. Information Theory , 7245–7252(2016)43. Granboulan, L., Joux, A., Stern, J.: Inverting HFE is quasipolynomial. In: Ad-vances in Cryptology - CRYPTO 2006. LNCS, vol. 4117, pp. 345–356. Springer,Santa Barbara, California, USA (Aug 2006). https://doi.org/10.1007/11818175 20,https://doi.org/10.1007/11818175 2044. Hoffstein, J., Pipher, J., Silverman, J.H.: NTRU: A ring-based public key cryp-tosystem. In: Buhler, J. (ed.) Algorithmic Number Theory, Third InternationalSymposium, ANTS-III, Portland, Oregon, USA, June 21-25, 1998, Proceedings.LNCS, vol. 1423, pp. 267–288. Springer (1998)45. Kipnis, A., Shamir, A.: Cryptanalysis of the HFE public key cryptosystem by relin-earization. In: Advances in Cryptology - CRYPTO’99. LNCS, vol. 1666, pp. 19–30. (9), 1983–1996 (2018). https://doi.org/10.1007/s10623-017-0434-5, https://doi.org/10.1007/s10623-017-0434-552. Ourivski, A.V., Johansson, T.: New technique for decoding codes in the rank metricand its cryptography applications. Problems of Information Transmission (3),237–246 (2002). https://doi.org/10.1023/A:102036932007853. Overbeck, R.: A new structural attack for GPT and variants. In: Mycrypt. LNCS,vol. 3715, pp. 50–63 (2005)54. Storjohann, A.: Algorithms for Matrix Canonical Forms. Ph.D. thesis, Swiss Fed-eral Institute of Technology – ETH (2000)55. L´evy-dit Vehel, F., Perret, L.: Algebraic decoding of codes in rank metric (Jun2006), communication at YACC06, Porquerolles, France56. Verbel, J., Baena, J., Cabarcas, D., Perlner, R., Smith-Tone, D.: On thecomplexity of “superdetermined” Minrank instances. In: Post-Quantum Cryp-tography 2019. LNCS, vol. 11505, pp. 167–186. Springer, Chongqing, China(May 2019). https://doi.org/10.1007/978-3-030-25510-7 10, https://doi.org/10.1007/978-3-030-25510-7 10 Appendix: Proof of Proposition 3
Let n, m, k, r be positive integers such that n and m are both greater than r .Let E be a F q -vector space of F q m of dimension r spanned by { E , E , . . . , E r } and let e ∈ F nq m whose components generate E . By definition, there exists anon-zero coordinate e j of e , and hereafter one focuses on the vector space λE = (cid:104) λE , λE , . . . , λE r (cid:105) where λ = e − j .Given a basis (1 , α, . . . , α m − ) of F q m over F q , one can write a basis of λE asa matrix S ∈ F m × rq . By construction, 1 ∈ λE , so that we can set the first columnand the first row of S to the vectors [1 0 · · · T and [1 0 · · · (cid:98) S for the remaining ( m − × ( r −
1) block of S . One can also express the coordi-nates of the components of λ e (with respect to the basis { λE , λE , . . . , λE r } )28s a matrix C ∈ F r × nq . By construction, the j -th column of C is the vector[1 0 · · · T .Lemma 2 estimates the probability to come across an index j such that e j isnon-zero. Once such an index is found, Lemma 3 computes the probability thatAlgorithm 1 succeeds in finding a non-singular block in (cid:98) S . Lemma 2.
With the same notation and hypotheses as above, if an index j ischosen uniformly at random in { , . . . , n } , then e j will be non-zero with proba-bility (1 − q − r ) / (1 − q − n ) .Proof. A component e j of e will be non-zero if and only if its correspondingcolumn of coordinates in the matrix C is non-zero. If the components of e werechosen uniformly at random in the vector space E of dimension r , the probabilityfor a random component to be equal to zero would be exactly q − r . This is notthe case since there is a constraint on C , more precisely it has to be of rank r .Taking this into account, we can count the number of full rank matrices in F r × nq that have a zero column. The ratio between those matrices and all the fullrank matrices in F r × nq is exactly the probability for a column chosen at randomin C to be zero: r − (cid:89) i =0 q n − − q i q n − q i = q n − r − q n − . One concludes the proof by taking the complementary event. (cid:117)(cid:116)
Lemma 3.
Let c ∈ { , . . . , (cid:98) m − r − (cid:99)} ; with the same notation and hypotheses asabove, if E and e are chosen uniformly at random, and if the inverse of a non-zero coordinate of e , λ , is given, then at least one of the c disjoint blocks B i in (cid:98) S is not singular with probability greater than − (1 − p q,r − ,r − ) c p q,r − ,m − Proof.
Since λ is a fixed nonzero element in F q m and since E is uniformly random,the vector space λE is also uniformly random. Therefore (cid:98) S is a matrix chosenuniformly at random among all the full rank matrices in F ( m − × ( r − q . Theprobability that all the c blocks B i in (cid:98) S are singular is then bounded fromabove by (cid:16) q ( r − − q ( r − p q,r − ,r − (cid:17) c q ( r − m − − c ( r − q ( m − r − p q,r − ,m − , (13)which is the ratio between the number of matrices in F ( m − × ( r − q with c singulardisjoint blocks and the total amount of full rank matrices in F ( m − × ( r − q . It isan upper bound since the number of matrices with c singular blocks includesmatrices that are not of full rank.The reader can check that the term (13) is equal to(1 − p q,r − ,r − ) c p q,r − ,m − . B i ’s is non-singular is obtained usingthe complementary probability. (cid:117)(cid:116) In Algorithm 1, the first requirement not to return fail is to find an index j suchthat e j is non-zero; Lemma 3 gives the probability of this event, that is to say(1 − q − r ) / (1 − q − n ). Once this index is found, the associated vector space λE isdistributed uniformly among all the vector spaces of F q m of dimension r since E is chosen at random. Using Lemma 2, one has a lower bound on the probabilitythat at least one of the c block B i ’s is non singular. Thus the probability ofProposition 3 is 1 − q − r − q − n (cid:18) − (1 − p q,r − ,r − ) c p q,r − ,m − (cid:19) . (cid:117)(cid:116)(cid:117)(cid:116)