Smooth determinantal varieties and critical loci in Multiview Geometry
aa r X i v : . [ m a t h . AG ] F e b SMOOTH DETERMINANTAL VARIETIES AND CRITICAL LOCI INMULTIVIEW GEOMETRY
MARINA BERTOLINI, ROBERTO NOTARI, AND CRISTINA TURRINI
Abstract.
Linear projections from P k to P h appear in computer vision as models ofimages of dynamic or segmented scenes. Given multiple projections of the same scene, theidentification of many enough correspondences between the images allows, in principle, toreconstruct the position of the projected objects. A critical locus for the reconstructionproblem is a variety in P k containing the set of points for which the reconstruction fails.Critical loci turn out to be determinantal varieties. In this paper we determine andclassify all the smooth critical loci, showing that they are classical projective varieties. Introduction
In this paper we classify the smooth determinantal varieties arising in the multiviewgeometry and computer vision settings as critical loci for reconstruction problems. Sincecritical loci and determinantal varieties belong to different research fields, it is mandatoryto explain the relation between them.Photos of static three-dimensional scenes taken from pinhole cameras are usually mod-elled by linear projections from P to P . Similarly, in computer vision, linear projectionsfrom P k to P h , are used to describe videos or images of particular dynamic and segmentedscenes ([30, 17, 18]). For this reason a camera can be identified with a linear projection π : P k P h .The reconstruction problem is the following: given a set of points in P k with unknowncoordinates, called scene , and n images of it in n target spaces P h i , i = 1 ...n , taken fromunknown cameras, the goal is to recover the positions of cameras and scene points in theambient space P k .Sufficiently many images and sufficiently many corresponding points in the given imagesshould in principle allow for a successful projective reconstruction, where correspondingpoints in the targets are images of the same point in the scene. Anyway, there existsets of points, in the ambient space P k , for which the projective reconstruction fails.These configurations of points are called critical , which means that there exist other nonprojectively equivalent sets of points and cameras that give the same images in the targetspaces.Critical loci turn out to be algebraic varieties and have been studied by many authors,indeed there is a wide literature on the subject. With analysis ad hoc, in the classicalcase of projections from P to P [11, 23, 24, 15, 22, 16, 29], the critical loci can betwisted cubic curves [11], or quadric surfaces [23, 24]. In the case of projections onto P from higher dimensional spaces, [10, 5, 4], critical loci have been proven to be minimaldegree varieties [10] for one projection, or, in a more general setting and under suitablegenericity assumptions, either hypersurfaces, if the ambient space is odd dimensional, ordeterminantal varieties of codimension two if the ambient space is even dimensional [5].Later, in [2, 9, 7, 8] the study of the ideal of critical loci has been formalized makinguse of the so-called Grassmann tensor introduced in [17]. A seminal case of this approach Date : February 22, 2021.
Key words and phrases.
Determinantal varieties, Minimal degree varieties, Multiview Geometry, Crit-ical loci.The authors are members of GNSAGA of INdAM. has been considered in [2], where the authors computed the equations of the critical locusfor triples of projections from P to P . In [9, 7] the case of three projections from P to P is studied in detail. When the projections are general enough, critical loci are shownto be Bordiga surfaces, and conversely, every Bordiga surface is shown to be critical forsuitable triples of projections. When the genericity assumptions are not fulfilled, criticalloci are shown to be not irreducible, with components of different dimensions. Finally, in[8], critical loci which are hypersurfaces in the ambient space are investigated.On the other hand, the classification of embedded smooth projective varieties is aclassical problem in algebraic geometry. For low degree or dimension and codimension,the classical approach to a classification problem consists in applying suitable techniquesto get a finite list of possible cases and further to construct examples for the survivingcases. Determinantal varieties are quite classical varieties, whose study takes advantageof homological techniques. The seminal result in the subject is Hilbert-Burch Theorem,but it is worth mentioning the structure theorem of codimension 3 Gorenstein ideals byD. Buchsbaum and D. Eisenbud, or Buchsbaum-Rim and Eagon-Northcott complexes.Under this view point, since critical loci are in the class of determinantal varieties,in this paper we approach in full generality the problem of determining which smoothdeterminantal varieties appear as critical loci and of classifying them. More precisely, wedetermine under what assumptions the critical locus for a reconstruction problem for n projections from P k to P h i for i = 1 , . . . n is a smooth variety, and provide a complete andeffective classification of smooth varieties with codimension at least 2 that can be critical.The classification results are summarized as follows, where n is the number of projec-tions. • n = 2All smooth critical loci are minimal degree varieties. Conversely, with the onlyexception of Veronese surface in P , every minimal degree variety embedded in P k , with codimension c , c ≤ k ≤ c + 1, is the critical locus for suitable pairs ofprojections. • n = 3 X in P k is a smooth critical locus if and only if X is – a cubic plane curve, in the case of triples of projections from P to P , P ,and P ; – a cubic surface in P , in the case of triples of projections from P to P , P ,and P ; – a Bordiga surface in P in the case of triples of projections from P to P , P ,and P . • n = 4Smooth critical loci are quartic determinantal surfaces in P , in the case of4 − uples of projections from P to P , P , P , and P , containing four pairwiseskew lines. Conversely, given four pairwise skew lines, it is possible to construct asmooth determinantal surface of degree 4 through them that is the critical locusfor a suitable reconstruction problem as above.The plan of the paper is as follows. In section 2, we introduce the setting of multipleview geometry and we recall the construction of the Grassmann tensor. In section 3, weintroduce critical loci and determine the generators of their ideals, showing in particularthat critical loci are determinantal varieties. In section 4, we give some numerical boundsfor critical loci to be smooth, and in particular we show that a critical locus can besmooth only if the number n of projections is at most 4. The remaining sections 5, 6, 7are devoted to the study of critical loci in the cases n = 2 , n = 3 , n = 4, respectively. MOOTH DETERMINANTAL VARIETIES AND CRITICAL LOCI 3 On multiview Geometry and Grassmann tensors
In this section we fix notation and terminology and give a short overview of classicalfacts in Computer Vision related to the problem of projective reconstruction of scenesand cameras from multiple views.In this context, a camera P is a linear projection from P k onto P h , from a linear subspace C of dimension k − h −
1, called center of projection . The target space P h is called view .A scene is a set of points X i ∈ P k , i = 1 , . . . , N .Using homogeneous coordinates in P k and P h , we identify P with a ( h + 1) × ( k + 1)matrix of maximal rank, defined up to a multiplicative constant. Hence C comes out tobe the right annihilator of P. Let us consider a set of n cameras P j : P k \ C j → P h j projecting the same scene in P k and the corresponding set of images in the different target spaces. In this setting, properlinear subspaces L i ⊆ P h i , i = 1 . . . n , are said to be corresponding if there exists at leasta point X ∈ P k such that P i ( X ) ∈ L i for all i = 1 . . . n .In the context of multiple view geometry, the problem of projective reconstruction of ascene, given multiple images of it, is the following: given many enough scene points in P k and identified a suitable number of corresponding subspaces on each image, one wants toget the projection matrices (up to projective transformations), i.e. the cameras, and thecoordinates in P k of the scene points.2.1. The Grassmann tensors.
Hartley and Schaffalitzky, [17], have constructed a setof multiview tensors, called
Grassmann tensors , encoding the relations between sets ofcorresponding subspaces. We recall here the basic elements of their construction.We consider n projections P j : P k \ C j → P h j , j = 1 , . . . , n, with centers C , . . . , C n . First generality assumption : we assume that the intersection C ∩ · · · ∩ C n is empty.Let L j ⊆ P h j be a general linear subspaces of codimension α j , j = 1 , . . . , n . We saythat ( L , . . . , L n ) is a n –tuple of corresponding subspaces if and only if ( P ) − ( L ) ∩ · · · ∩ ( P n ) − ( L n ) is not empty, where Y is the Zariski closure of Y . We allow α j to be equal to0 for some j . If this happens, the associated view does not impose any constrain to thereconstruction problem, and so the effect of setting α j = 0 is to decrease the number ofviews.We remark that the Computer Vision community uses a slightly different definitionof corresponding spaces: the spaces are said to be corresponding if ( P ) − ( L ) ∩ · · · ∩ ( P n ) − ( L n ) is not empty. The difference is that the centers C j are not considered whenthe inverse images are intersected in this setting, while we prefer to include them, so toget projective varieties, and not only open subsets of them.From the Grassmann formula, if P j α j = k + 1 , the existence of points in the previousintersection gives a constrain which allows us to construct the Grassmann tensor. Hartleyand Schaffalitzky call the n –tuple ( α , . . . , α n ) a profile for the reconstruction problem.We remark that we allow α j = 0, too, while, in [17], α j ≥ j = 1 , . . . , n .Let { L , . . . , L n } be n general linear subspaces as above and let S j be the maximal rankmatrix of type ( h j + 1) × ( h j − α j + 1) whose columns are a basis for L j . By definition, ifthe L j ’s are corresponding subspaces, there exists a point Y ∈ P k such that P j ( Y ) ∈ L j for every j . In other words there exist n vectors v j ∈ C h j − α j +1 j = 1 , . . . , n such that:(1) P S . . . P S . . . P n . . . S n Yv v ... v n = . M.BERTOLINI, R.NOTARI, AND C.TURRINI
The coefficient matrix T P ,...,P n S ,...,S n is square of order n + P h j = k + 1 + P ( h j − α j + 1),where the left side is the number of rows, the right side is the number of columns, and theycoincide due to our assumptions on the profile. The existence of a non–trivial solution( Y , v , . . . , v n ) of system (1) implies that the determinant of T P ,...,P n S ,...,S n is zero.The determinant T P ,...,P n ( L , . . . , L n ) = det( T P ,...,P n S ,...,S n ) can be thought of as a n –linearform (tensor) in the Pl¨ucker coordinates of the spaces L j ’s, in the corresponding Grass-mann variety. This tensor is called Grassmann tensor . From the above discussion, itfollows that this tensor vanishes if and only if the linear spaces L , . . . , L n are correspond-ing. In [17], the authors show that the Grassmann tensor allows the reconstruction of theprojection matrices, up to the only case when all target spaces are P . For this reason, theComputer Vision community does not consider the case above in reconstruction problems.3. Critical loci and their ideals
Roughly speaking, one guesses that the reconstruction problem can be successfullysolved if sufficiently many views and sufficiently many sets of corresponding points in thegiven views are known. This is generally true, but even in the classical set–up of twoprojections from P to P one can have non projectively equivalent pairs of scenes andcameras that produce the same images in the view planes, thus preventing reconstruction.Such configurations and the loci they describe are referred to as critical . In [5], criticalloci for projective reconstruction of camera centers and scene points from multiple viewsfor projections from P k to P have been introduced and studied.Now we recall the basic definition. Definition 3.1.
Given n projections Q j : P k P h j , a set of points { X , . . . , X N } in P k issaid to be a critical configuration for projective reconstruction for Q , . . . , Q n if there existsanother set of n projections P i : P k P h i and another set { Y , . . . , Y N } ⊂ P k , non-projectively equivalent to { X , . . . , X N } , such that, for all i = 1 , . . . , n and j = 1 , . . . , N ,we have P i ( Y j ) = Q i ( X j ), up to homography in the targets. The two sets { X j } and { Y j } are called conjugate critical configurations , with associated conjugate projections { Q i } and { P i } .In next Proposition 3.1, we prove that points in critical configurations fill an alge-braic variety, called critical locus X , whose ideal can be obtained by making use of theGrassmann tensor introduced above.Indeed, the Grassmann tensor T P ,...,P n ( L , . . . , L n ) encodes the algebraic relations be-tween corresponding subspaces in the different views of the projections P , . . . , P n . Henceby definition of critical set, if { X j , Y j } are conjugate critical configurations, then, foreach j , the projections Q ( X j ) , . . . , Q n ( X j ) are corresponding points not only for theprojections Q , . . . , Q n , but for the projections P , . . . , P n , too.Following the construction above, we first choose a profile ( α , . . . , α n ), and a point X in the critical locus. If Q i ( X ) ∈ L i , for every i = 1 , . . . , n , then T P ,...,P n ( L , . . . , L n ) =0. The previous condition is fulfilled if L i is spanned by Q i ( X ) and any other h i − α i independent points in P h i . So, we can suppose S i = (cid:0) Q i ( X ) x i . . . x i,h i − α i (cid:1) = (cid:0) Q i ( X ) S ′ i (cid:1) of maximal rank h i − α i + 1, that is to say, S ′ i is a general ( h i + 1) × ( h i − α i ) matrix ofrank h i − α i . Due to this choice, the matrix T P ,...,P n S ,...,S n becomes T P ,...,P n S ,...,S n = P Q ( X ) S ′ . . . P Q ( X ) S ′ . . . P n . . . Q n ( X ) S ′ n . MOOTH DETERMINANTAL VARIETIES AND CRITICAL LOCI 5
The determinant det( T P ,...,P n S ,...,S n ) is a sum of products of maximal minors of S ′ , . . . , S ′ n , andmaximal minors of the matrix(2) M P ,...,P n Q ,...,Q n = P Q ( X ) 0 0 . . . P Q ( X ) 0 . . . P n . . . Q n ( X ) . Such a matrix is a ( n + n X i =1 h i ) × ( n + k + 1) matrix, the last n columns of which are oflinear forms, while the first k + 1 columns are of constants.More explicitly, if we consider M P ,...,P n Q ,...,Q n as a block matrix, the coefficients are the minorsobtained by delating h i − α i rows from the i –th block (cid:0) P i . . . Q i ( X ) 0 . . . (cid:1) , for every block.If we allow the profile to change, because X is in the critical locus independently fromthe profile, we get all the possible maximal minors of M P ,...,P n Q ,...,Q n . The discussion above ispart of the proof of the following result. Proposition 3.1.
The ideal I ( X ) of the critical locus X is generated by the maximalminors of M P ,...,P n Q ,...,Q n , and so X is a determinantal variety. Moreover, X contains thecenters of the projections Q j ’s.Proof. We only have to prove that the center C ′ j of Q j is contained in X for every j =1 , . . . , n . C ′ j is the zero locus of Q j ( X ), and so M P ,...,P n Q ,...,Q n drops rank at every point in C ′ j ,for each j . (cid:3) We remark that, if α i ≥
1, then some maximal minors of M P ,...,P n Q ,...,Q n do not appear indet( T P ,...,P n S ,...,S n ) for whatever profile, and so they should not be among the generators of I ( X ). This fact supports our choice to allow α i = 0.From the first generality assumption, it follows both that the first k + 1 columns of M P ,...,P n Q ,...,Q n are linearly independent, and that the linear forms in the last n columns of theabove matrix span a linear space of dimension k + 1 in R = ( R = K [ x , . . . , x k ]) , where P k = Proj( R ). In fact, no point is common to either the centers of the P i ’s or of the Q j ’s.As in [9], we write the matrix M P ,...,P n Q ,...,Q n as the following block matrix M P ,...,P n Q ,...,Q n = (cid:18) A BC D (cid:19) where A is a ( n − k − n X i =1 h i ) × ( k + 1) matrix, B is a ( n − k − n X i =1 h i ) × n matrix, C is an order ( k + 1) square matrix, and, finally, D is a ( k + 1) × n matrix. We assumethat C is invertible. By performing elementary operations on columns and rows, we canreduce M P ,...,P n Q ,...,Q n to the following easier form (cid:18) N X I k +1 (cid:19) where N X = B − AC − D is a ( n − k − n X i =1 h i ) × n matrix of linear forms. Furthermore,the maximal minors of M P ,...,P n Q ,...,Q n span the same ideal as the maximal minors of N X . Hence,we have the following result. M.BERTOLINI, R.NOTARI, AND C.TURRINI
Corollary 3.1. I ( X ) is generated by the maximal minors of N X = B − AC − D , with thesame notations as above. Since the critical locus X is a determinantal variety whose ideal is generated by themaximal minors of a matrix of linear forms, the expected dimension of X is(3) ed X = k − n − k − n X i =1 h i ) − n ! = 2 k − n X i =1 h i . From Porteous’s formula ([1], formula 4 .
2, p. 86),we get, if dim( X ) = ed X ,(4) deg( X ) = (cid:18) n − k − P ni =1 h i n − (cid:19) . Second generality assumption: we assume projections P , . . . , P n , and Q , . . . , Q n aregeneral enough to guarantee that the critical locus X has the expected dimension 2 k − P ni =1 h i .From now on, every time we assert we are in the general case, we assume that both thegenerality assumptions hold. 4. Numerical bounds
In this section, we deduce both a lower bound for P h i , and an upper bound on thenumber n of views to get smooth critical loci. Proposition 4.1.
In the same notations as above, we have (5) k + 1 ≤ n X i =1 h i . Proof.
As 0 ≤ α i ≤ h i for every i = 1 , . . . , n , and P ni =1 α i = k + 1, then we have k + 1 ≤ n X i =1 h i . (cid:3) We are interested in studying the case X is irreducible and non–singular. To begin, werelate the projection centers to singular critical loci. Lemma 4.1.
If two centers of the projections Q , . . . , Q n intersect, the critical locus issingular.Proof. If the centers of Q and Q , for example, have a common point, two columns ofmatrix (2) vanish, and so its rank is at most k + n −
1. From generalities on determinantalvarieties, it follows that the critical locus is singular. (cid:3)
Now, we can compute an upper bound on the number n of views to get an associatedsmooth critical locus. Theorem 4.1.
Let X be the codimension c ≥ critical locus for a couple of n ≥ projections P , . . . , P n and Q , . . . , Q n from P k to P h i , i = 1 , . . . , n . Then, either X is notirreducible, or is singular.Proof. The center C i of Q i has dimension k − h i −
1. Since C i ⊆ X , we have thatdim C i ≤ dim X , and so k − h i − ≤ k − c , or equivalently, h i ≥ c − . Let us assume that c − ≤ h ≤ · · · ≤ h n .If h = c −
1, then the center C of Q has codimension c and is contained in X . Then, X is not irreducible. MOOTH DETERMINANTAL VARIETIES AND CRITICAL LOCI 7
Assume now that h ≥ c . Since dim X = 2 k − P ni =1 h i = k − c , we get k = n X i =1 h i − c. On the other hand, the center C j of projection Q j has dimensiondim C j = k − h j − X i = j h i − c − . We know that X is singular if two centers meet. The two centers with smaller dimensionare C n − and C n . The condition that guarantees they do not meet is dim C n + dim C n − − k <
0, that is to say, n − X i =1 h i < c + 2 . Since we are in the case h ≥ c , the left side becomes c ( n − < c + 2, i.e. n < c .Hence, if h ≥ c and n ≥ X is singular. (cid:3) The same proof allows us to state also the following result.
Theorem 4.2.
Let X be the codimension critical locus for a couple of n ≥ projections P , . . . , P n and Q , . . . , Q n from P k to P h i , i = 1 , . . . , n . Then, either X is not irreducible,or is singular. Hence, we have to study the cases n = 2 and n = 3, for every codimension c and n = 4for c = 1, only. 5. The n = 2 view case In this section, we want to prove that, under some mild assumptions, the critical locusfor n = 2 is a smooth and irreducible variety of minimal degree, and, conversely, thatevery smooth irreducible variety of minimal degree, but the Veronese surface, is criticalfor a suitable couple of projections. In this way, we classify all smooth critical loci inComputer Vision for 2 views. E.g., when the codimension is 1, the critical locus for twoprojections from P to P is a quadric surface, and this is well–known in the ComputerVision community; when c = 2, the critical locus for two projections from P to P is arational normal scroll; when c = 3 the critical locus for two projections from P to P iseither P ( O P (2) ⊕ O P (2)), or P ( O P (1) ⊕ O P (3)).When n = 2, we have h + h = k + c . Moreover, k > h ≥ h ≥ c + 1. We remark thatthe last inequality on the right is a consequence of the previous equality.Now, we prove that the critical locus is a variety of minimal degree. Proposition 5.1.
In the general case, the codimension c critical loci for two views areminimal degree varieties.Proof. Corollary 3.1 implies that the ideal I ( X ) is generated by the maximal minors of N X whose type is ( c + 1) ×
2, and so it is generated by quadrics. Furthermore, fromequation (4), we get that deg( X ) = 1 + c . This description proves that X ⊆ P k is aminimal degree variety (see [12]). (cid:3) The generality assumption in Proposition 5.1 implies that the minors of N X definevariety of the expected codimension c .From the classification of minimal degree varieties in [12], we get that X is singular assoon as k ≥ c + 1). Hence, smooth irreducible varieties of minimal degree that can becritical loci are embedded in P k for c + 2 ≤ k ≤ c + 1.Now, we consider the converse of Proposition 5.1. M.BERTOLINI, R.NOTARI, AND C.TURRINI
Proposition 5.2.
With the only exception of Veronese surfaces in P , every codimension c minimal degree variety embedded in P k with c + 2 ≤ k ≤ c + 1 , is the critical locus fora suitable pair of projections.Proof. Let us consider matrix M in the case of 2 projections, and the matrix N X weobtain from it, as discussed in Section 2. In the 2 view case, we have M = (cid:18) P Q ( X ) 0 P Q ( X ) (cid:19) = (cid:18) A BC D (cid:19) , where P ( P , respectively) is a ( h + 1) × ( k + 1) (( h + 1) × ( k + 1), respectively) fullrank matrix, and, up to transposition, Q ( X ) = ( Q ( X ) , . . . , Q ,h +1 ( X )) and Q ( X ) =( Q ( X ) , . . . , Q ,h +1 ( X )) and the linear forms in each one of them are linearly indepen-dent. Moreover, A is of type ( c + 1) × ( k + 1), C is of type ( k + 1) × ( k + 1), and weassume it is invertible, B is of type ( c + 1) ×
2, and finally, D is of type ( k + 1) ×
2. Theassumption on the rank of C is always fulfilled up to collecting rows of M in a differentway.We recall that h i ≥ c + 1 for i = 1 , N X , we get N X = Q ( X )... Q ,c +1 ( X ) − E Q ,c +2 ( X )... Q ,h +1 ( X ) (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) − F Q ( X )... Q ,h +1 ( X ) where AC − = ( E | F ). It is evident that the first column of N X depends on the first view,and the second from the second view.Let us consider now a codimension c variety V of minimal degree, and let N be the( c + 1) × V (see [12]). To fix notation, let n ij be theelements of N . By comparing N and N X , we can choose E and Q ,c +2 ( X ) , . . . , Q ,h +1 ( X )as general as possible, and we get Q ( X )... Q ,c +1 ( X ) = n ... n c +1 , + E Q ,c +2 ( X )... Q ,h +1 ( X ) . We choose F as ( − I | F ′ ), F ′ being general, and Q ,c +2 ( X ) , . . . , Q ,h +1 ( X ) arbitrary. Sim-ilarly to the previous case, we get Q ( X )... Q ,c +1 ( X ) = n ... n c +1 , + F ′ Q ,c +2 ( X )... Q ,h +1 ( X ) . Once we choose a general invertible matrix C of order k + 1, we compute A = ( E | F ) C and so we get the matrix M as required. We remark that the assumptions on the ranks of P , P , Q ( X ) and Q ( X ) are satisfied by the generality of the choices in the construction. (cid:3) The n = 3 view case In this section, we classify all smooth varieties that can be obtained as critical loci fortwo triples of projections.Before approaching the problem, we briefly recall the list of codimension c smoothdeterminantal varieties associated to matrices of type ( c + 2) ×
3. In the case underconsideration, since there are three views, we have h + h + h = k + c . From the proofof Theorem 4.1, we know that, if a view verifies h i ≤ c −
1, then either the critical locusdoes not have codimension c , or is not irreducible. Then, we can assume h i ≥ c for i = 1 , ,
3, from which we get that k ≥ c . On the other hand, a determinantal variety ofcodimension c as the ones we consider, is singular when embedded in P k with k ≥ c + 2. MOOTH DETERMINANTAL VARIETIES AND CRITICAL LOCI 9
Hence, smooth determinantal varieties can be critical loci for two triples of projectiononly if embedded in a projective space P k with k = 2 c or k = 2 c + 1. The degree of suchvarieties is the expected one, namely deg( X ) = (cid:0) c +22 (cid:1) , as it follows from equation (4).Thanks to classification results on smooth varieties with small invariants, in the case weare dealing with, the list of smooth varieties is complete for c ≤
3. This is not a limitationfor us because, as we will see in Theorem 6.1, smooth critical loci appear only when c ≤ (cid:0) c +22 (cid:1) for c ≤ c = 1 plane cubic curves: they can be critical for two triples of projections from P to P . Even if this case is not of interest for the Computer Vision community, weinclude it for completeness from a geometric perspective; c = 1 cubic surfaces: they can be critical for two triples of projections from P to P , P , P ; c = 2 Bordiga surfaces: they can be critical for two triples of projections from P to P ; c = 2 Bordiga scrolls: they can be critical for two triples of projections from P to P , P , P ; c = 3 3-fold scrolls on P : they can be critical for two triples of projections from P to P ; c = 3 4-fold scrolls on P : they can be critical for two triples of projections from P to P , P , P .The codimension 1 cases are part of classical results on the classification of smoothhypersurfaces, the codimension 2 ones are in [20], while the codimension 3 ones are in[13]. We briefly describe the codimension 2 and 3 varieties.The Bordiga surface is the embedding in P of the blow–up of P at 10 general pointsvia the linear system of plane quartics through the points. Let Z ⊆ P be a set of 10general points, and B ⊆ P the associated Bordiga surface. The ideal sheaves I Z and I B are described from the following exact sequences:0 → O P ( − N Z −→ O P ( − → I Z → → O P ( − N B −→ O P ( − → I B → . The matrices N Z and N B are not independent since it holds(6) ( x , . . . , x ) N Z = ( z , z , z ) N TB where x , . . . , x are coordinates in P and z , z , z are coordinates in P .The Bordiga scroll X is P ( E ) embedded in P via the tautological bundle ξ , where E isany rank 2 vector bundle defined by the extension0 → O P → E → I Z (4) → Z ⊆ P is a set of 10 general points, as for the Bordiga surface (see [25]). Bycomparing the resolution of I Z and the defining extension for E , we get the followingexact sequence 0 → O P ( − N E −→ O P → E → . The minimal free resolution of X is0 → O P ( − N X −→ O P ( − → I X → , and, as for the Bordiga surface, the matrices N X and N E are related in the equation( x , . . . , x ) N E = ( z , z , z ) N TX . In the codimension 3 case, the resolutions of the two scrolls are obtained by means ofthe Eagon–Northcott complex, and it follows that they both have sectional genus 6. Insuch a case, we can construct them both similarly to the case of the Bordiga scroll. Weremark that, in principle, the 3–fold scroll could be constructed as blow–up of a scroll at four double points, but it is not known whether it exists. The starting point is now a set Z of 15 general points in P , and a rank 2 vector bundle E defined by the extension0 → O P → E → I Z (5) → . Since the minimal free resolution of I Z is0 → O P ( − → O P ( − → I Z → , we get the following presentation of E :0 → O P ( − N E −→ O P → E → . Let N X be the 5 × x , . . . , x ) N E = ( z , z , z ) N TX . Then, the defining ideal I ( X ) of X is generated by the 3 × N X .Since the construction of the 4–fold scroll is analogous to the one of the 3–fold scroll,we only stress the differences. This time, the vector bundle to consider is the rank 3 onedefined by the extension 0 → O P → E → I Z (5) → , where Z is a set of 15 general points as in the previous case. Then, the matrix N X isobtained as in the previous case, but N E is now a matrix with type 8 × c ,determinantal variety X ⊆ P k , with k = 2 c or k = 2 c + 1, whose defining ideal isgenerated by the 3 × c + 2) × N of linear forms. The result iscontained in the following classification Theorem. Theorem 6.1. X is the critical locus for two suitable triples of projections from P k if andonly if either X ⊆ P is a cubic curve, or X ⊆ P is a cubic surface, or, finally, X ⊆ P is a Bordiga surface. In particular, c ≤ .Proof. From the previous discussion, it follows that h = h = c , and h = c + ε for k = 2 c + ε , ε = 0 or 1. Let us consider general projections P i , Q i : P k → P h i for i = 1 , , N X as in Corollary 3.1, and so we get N X = Q ( X ) 0 0... Q ,c +1 ( X ) 0 00 Q ( X ) 0 − AC − Q ( X ) 0...0 Q ,c +1 ( X ) 00 0 Q ( X )...0 0 Q ,h +1 ( X ) . If we multiply the matrices above, and perform elementary operations on the rows, we getthe matrix N = ( n ij ) such that: ( i ) its j -th column depend on Q j only, for j = 1 , ,
3; ( ii ) n c +2 , = n c +1 , = 0. In the case h = c , it is possible to perform elementary operations ofthe rows of N so that ( iii ) n c, = 0.It follows that the critical locus is actually a codimension c scheme whose defining idealis generated by the 3 × c + 2) × X ⊆ P can bewritten as L L L + M M M = 0 for suitable linear forms L i , M i , i = 1 , ,
3. An equationof this kind for the cubic surface is called Cayley–Salmon. We remark that, for a givensurface, there are 120 different Cayley–Salmon equations that define it (see [14]). Thelines defined by L i = M j = 0 are contained in the cubic surface X . The Cayley–Salmon MOOTH DETERMINANTAL VARIETIES AND CRITICAL LOCI 11 equation is the locus where N X = L M M L L M drops rank. If we add scalar multiples of the first two columns to the third one, we get amatrix N X that verifies constrains ( i ) , ( ii ) above, and the linear forms on the third columnare linearly independent. If we compare matrices N X and N X under the simplifyingassumption that ( E | F ) = − − e , we get Q = L , Q = M , Q = e M + L , Q = M , and Q j = n j for j = 1 , , Q , Q , Q . If we choose a general invertible matrix C of order 4, we get A = ( E | F ) C , and so we obtain also the projections P , P , P .We have then proven that every smooth cubic surface is the critical locus for two triplesof projections from P to P , P , P , as claimed.A plane cubic curve is obtained as a section of a smooth cubic surface with a generalplane. Hence, the argument above shows also that every plane cubic curve is critical fortwo triples of projections from P to P , as claimed. We remark that, for plane cubiccurves, one has to start from the Cayley–Salmon equation and does not have to performfurther elementary operations on the columns of the matrix.The Bordiga surface has been considered from the point of view of critical loci in [9]. Inthat paper, the authors proved that the critical locus for two triples of projections from P to P is in the irreducible component of the Hilbert scheme containing the Bordiga surfaceas general element ([9], Proposition 5.1), and conversely, that every Bordiga surface X isactually critical for two suitable triples of projections ([9], Theorem 5.1). The key pointof the proof of Theorem 5.1 is that, if the unit points in P are in Z , then the matrix N X fulfils the constraints ( i ) , ( ii ) , ( iii ) above, since it is related to N Z in equation (6).To complete the proof, we’ll prove that the critical locus X is never smooth in theremaining cases.To this end, we consider a general critical locus X , its associated matrix N X as obtainedat the beginning of the proof, and we take the codimension c +3 linear space L ⊆ X definedby n = · · · = n c +1 , = n c +2 , = n c +2 , = 0. As X is at least 3–dimensional in the caseswe are considering, L is not empty. To prove that the points in L are singular for X , weevaluate the Jacobian matrix at them. Without loss of generality, we can make a changeof coordinates, so that n i, = x i for i = 1 , . . . , c + 1, n c +2 , = x c +2 and n c +2 , = x c +3 .Furthermore, we assume that, at a point in L , the rank of the ( c + 1) × N X is 2. In the case this does nothold, the point is singular by general properties of determinantal varieties. To simplifynotation, we denote ( i i ) the determinant of the minor of the above matrix obtained bytaking rows i and i . Let f ijh be the determinant of the submatrix of N X obtained bytaking rows i, j, h with 1 ≤ i < j < h ≤ c + 2. The derivative of f ijh with respect to anyvariable is the sum of 3 determinants, two columns of which are from N X and the thirdcolumn is the derivative of the corresponding column in N X . We have to evaluate thederivatives at P ∈ L . If h = c + 2, then the gradient of f ij,c +2 at P is the null matrix, asit is easy to check. If h ≤ c + 1, we get ∇ f ijh ( P ) = ( jh ) ~e i − ( ih ) ~e j + ( ij ) ~e h where ~e k is the k –th element of the canonical basis. Without loss of generality, we canassume (12) = 0, equivalent to the rank two assumption. The matrices ∇ f ( P ) , . . . , ∇ f ,c +1 ( P ) are linearly independent, since (12) I c − is a submatrix of the Jacobian matrix corresponding to the above generators. Let us consider now f ijh with 2 < j < h ≤ c + 1and i = 1 or i = 2. We have(12) ∇ f ijh ( P ) − ( ij ) ∇ f h ( P ) + ( ih ) ∇ f j ( P ) = [(12)( jh ) − (1 j )(2 h ) + (1 h )(2 j )] ~e i = 0because the equation in square brackets is a Pl¨ucker relation that holds for rank 2 matricesof type ( c + 1) × c ≥
3. Finally, we consider f ijh with 2 < i < j < h ≤ c + 1.We have (12) ∇ f ijh ( P ) − ( ij ) ∇ f h + ( ih ) ∇ f j ( P ) − ( jh ) ∇ f i ( P ) == − [(2 i )( jh ) − (2 j )( ih ) + (2 h )( ij )] ~e + [(1 i )( jh ) − (1 j )( ih ) + (1 h )( ij )] ~e = 0because we get Pl¨ucker relations once more. Hence, ∇ f ijh ( P ) is in the span of ∇ f ( P ) ,. . . , ∇ f ,c +1 ( P ) for every 1 ≤ i < j < h ≤ c + 2, and so the Jacobian matrix has rank c − P ∈ L . This proves that every P ∈ L is singular for the criticallocus X , and so the proof is complete. (cid:3) The n = 4 view case According to Theorem 2.2, when we have 4 views, the codimension of the critical locusis 1, otherwise the critical locus is either not irreducible or singular. In such a case, h + h + h + h = k + 1. On the other hand, a degree 4, determinantal hypersurface issingular if embedded in P k with k ≥
4. Hence, the only possible case is k = 3, and h i = 1for every i = 1 , . . . ,
4. As previously said, this case is not of interest for the ComputerVision community, and we insert it for sake of completeness from a geometrical point ofview.The study of quartic determinantal surfaces in P is a classical topic, and we brieflyrecall the main results (see [21] for more results on the subject).Quartic surfaces in P are parameterized by points in P . It is known that the generalquartic surface in P is not determinantal, and that the locus of determinantal ones is adivisor in P . Determinantal quartic surfaces are characterized as the ones that containa non–hyperelliptic curve C of degree 6 and genus 3 (see, e.g., [3]). Such a curve is alsocalled Schur’s sextic.Moreover, a general quartic surface does not contain any line. It is known that not ruledquartic surfaces can contain any number of lines in the range 1 to 52, or 54 , , ,
64 lines.In [26, 27, 28], the author studied quartic determinantal surfaces containing one or twolines. In particular, if N is an order 4 matrix of linear forms in P whose determinant isthe defining equation of the quartic surface S , and ℓ ⊂ S is a line, then, up to elementaryoperations on rows and columns of N , the linear forms defining ℓ are either in a row orcolumn of N , or in a 3 × × N .Now we discuss the connections between quartic determinantal surfaces containing linesand the reconstruction problem in Computer Vision. Proposition 7.1.
Let P i , Q i : P → P , i = 1 , . . . , , be two –tuples of projections. Then,in the general case, the associated critical locus is a smooth quartic determinantal surface.Proof. The matrix M associated to the two 4–tuples of projections is described in equation(2), and is a square matrix of order 8. From M , we get matrix N X = B − AC − D oforder 4 whose determinant defines the critical locus X . In the considered case, matrices B, D are B = Q ( X ) 0 0 0 Q ( X ) 0 0 00 Q ( X ) 0 00 Q ( X ) 0 0 , D = Q ( X ) 00 0 Q ( X ) 00 0 0 Q ( X )0 0 0 Q ( X ) . Then, the first two columns of N X are the first two columns of B , while the last twocolumns of N X depend of the two non–zero columns of D . Then, the critical locus is a MOOTH DETERMINANTAL VARIETIES AND CRITICAL LOCI 13 quartic determinantal surface. When computing X in a random case, we get a smoothsurface, and so the general critical locus is smooth. (cid:3) Remark . In the notation of [26], the four lines, centers of projections Q , . . . , Q , areof type 4 ′′ .Now we highlight a geometrical property of such critical loci. Proposition 7.2.
In the same hypotheses as above, the critical locus contains twistedcubic curves meeting three of the four lines at two points.Proof.
Let N X be the matrix constructed in the proof of Proposition 7.1, and let N ′ itssumatrix consisting of the first three columns. The maximal minors of N ′ define a Schurcurve containing the centers of projections Q , Q , Q as components. To fix notation, weset N ′ = Q ( X ) 0 n ′ Q ( X ) 0 n ′ Q ( X ) n ′ Q ( X ) n ′ where the center of Q is the line Q ( X ) = Q ( X ) = 0, the center of Q is the line Q ( X ) = Q ( X ) = 0 and the center of Q is n ′ = n ′ = 0, also defined by n ′ = n ′ = 0.The two couples of generators of the third line are related by the equation (cid:18) n ′ n ′ (cid:19) = (cid:18) a a a a (cid:19) (cid:18) n ′ n ′ (cid:19) where A = ( a ij ) is invertible. The residual curve is the twisted cubic curve defined by the2 × (cid:18) Q ( X ) Q ( X ) (cid:12)(cid:12)(cid:12)(cid:12) Adj( A ) (cid:18) Q ( X ) Q ( X ) (cid:19)(cid:12)(cid:12)(cid:12)(cid:12) n ′ n ′ (cid:19) , as it can be checked. Each center meets the twisted cubic curve above at two points,because the generators of each line vanish two quadrics of the three that generate thetwisted cubic. (cid:3) Now, we consider a partial converse of the above Proposition.
Proposition 7.3.
Let ℓ , . . . , ℓ ⊂ P be lines, pairwise skew. Then there is a quarticdeterminantal surface S containing the lines that is critical for two –tuples of projec-tions from P to P . The given lines are centers for the four projections Q , . . . , Q .Proof. Let ℓ i be the line defined by I ( ℓ i ) = h q i , q i i , where q i , q i are linearly independentlinear forms in C [ x , . . . , x ]. Let us consider the matrix N = q e q + e q e q + e q q e q + e q e q + e q q e q + e q e q + e q q e q + e q e q + e q , and let S be the surface defined by det( N ) = 0. We assume E = ( e ij ) to be invertible.Hence, we can reconstruct M from N by choosing a general invertible matrix C and bysetting A = − EC . Hence, S is critical for suitable projections as claimed. (cid:3) To complete the section, we make some final remarks on quartic surfaces that containfour skew lines that explain why it is not possible to give a stronger converse of Proposition5.1.
Remark . Given 4 pairwise skew general lines in P , the linear subspace V of C containing quartic surfaces through the lines has dimension 15. Here, four lines are generalif they are not contained in the same quadric. Since the conditions of being determinantal and of containing lines are independent, we expect that the quartic determinantal surfacescontaining the four lines are a locus of dimension 14 in V .From a parameter count, quartic determinantal surfaces that are critical loci dependon the elements of E . Since we can get the same quartic surface for different choices ofmatrix E (see the proof of Proposition 5.2), we have that the parameters are actuallyless than 16. Algebraically, from every rows of E , different from the first one, an elementcan be disregarded. So, the locus in V of critical surfaces has dimension 13. From ageometrical point of view, a quartic determinantal surface containing 4 lines as above iscritical if, and only if, it contains a twisted cubic curve meeting three of the four lines at 2points, and not meeting the last line, for every choice of three among the four lines. Sincea twisted cubic curve is uniquely determined by 6 points, we get such a curve by choosingtwo points of the first three lines. If we choose a second twisted cubic curve meeting alllines but the third one, we need 6 more points, two for each of the three selected lines.Once those two twisted cubics are selected, the quartic surface through the four lines isgiven, and it is possible to get its defining equation as determinant of a matrix as in theproof of Proposition 5.2. The parameters from which this construction depends are 12(the points on the lines), and one more because we can multiply the matrix by a scalarso that the determinant defines the same surface. Hence, we get once more a locus ofdimension 13.In conclusion, we expect the critical quartic surfaces to fill a codimension 2 subset in V . References [1] E. Arbarello, M. Cornalba, P. Griffiths, J.D. Harris. Geometry of Algebraic Curves.
GMW 267 ,Springer Verlag, New York, 1985.[2] K. ˚Astr¨om and F. Kahl. Ambiguous Configurations for the 1 D Structure and Motion Problem.
Journal of Mathematical Imaging and Vision , 18:191–203, 2003.[3] A. Beauville. Determinantal Hypersurfaces.
Michigan Math. J. (2000), 39–64.[4] M. Bertolini, G. Besana, and C. Turrini. Applications Of Multiview Tensors In Higher Dimensions.In Tensors in Image Processing and Computer Vision , Editors: Aja-Fernandez, S., de Luis Garcia,R., Tao, D., Li, X.,
Advances in Pattern Recognition . 2009.[5] M. Bertolini, G. Besana, and C. Turrini. Critical loci for projective reconstruction from multiple viewsin higher dimension: A comprehensive theoretical approach.
Linear Algebra and its Applications ,469(2015), 335–363.[6] M. Bertolini, G. Besana, and C. Turrini. Generalized Fundamental Matrices as Grassmann Tensors
Annali di Matematica Pura ed Applicata , n.2, 196(2017), 539–553.[7] M. Bertolini, GM. Besana, R. Notari and C. Turrini. Critical loci in computer vision and matricesdropping rank in codimension one.
Journal of Pure and Applied Algebra (12) , (2020), 1–27.[8] M. Bertolini and L. Magri. Critical Hypersurfaces and Instability for Reconstruction of Scenes inHigh Dimensional Projective Spaces.
Machine Graphics & Vision , (1) , (2020), 3–20.[9] M. Bertolini, R. Notari and C. Turrini. The Bordiga surface as critical locus for 3-view reconstruc-tions. MEGA 2017, Effective Methods in Algebraic Geometry, Nice (France), June 12-16, 2017. J.Symb. Comp. , (2019), 74–97.[10] M. Bertolini and C. Turrini. Critical configurations for 1-view in projections from P k → P . Journalof Mathematical Imaging and Vision , 2007.[11] T. Buchanan. The twisted cubic and camera calibration.
Comput. Vision Graphics Image Process. ,42(1):130–132, 1988.[12] D. Eisenbud, J. Harris. On Varieties of Minimal Degree (A Centennial Account).
Proceedings ofSymposia in Pure Mathematics , vol. (1987), pp. 3–13.[13] M.L. Fania, E.L. Livorni. Degree Ten Manifolds of Dimension n Greater than or Equal to 3.
Math.Nachr. (1997), 79–108.[14] M.A. Hahn, S. Lamboglia, A. Vargas. A short note on Cayley–Salmon equations.
Le Matematiche ,vol. LXXV, Issue II (2020), 559–574.[15] R.I. Hartley. Ambiguous configurations for 3-view projective reconstruction. In
European Conferenceon Computer Vision , pages I: 922–935, 2000.[16] R.I. Hartley and F. Kahl. Critical configurations for projective reconstruction from multiple views.
International Journal of Computer Vision , 71(1):5–47, 2007.
MOOTH DETERMINANTAL VARIETIES AND CRITICAL LOCI 15 [17] R.I. Hartley and F. Schaffalitzky. Reconstruction from projections using Grassmann tensors. In
Proceedings of the 8th European Conference on Computer Vision, Prague, Czech Republic , LNCS.Springer, 2004.[18] R. Hartley and R. Vidal. The multibody trifocal tensor: Motion segmentation from 3 perspectiveviews. In
Conference on Computer Vision and Pattern Recognition , 2004.[19] R.I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge UniversityPress, 2004.[20] P. Ionescu. Embedded projective varieties of small invariants. Proceeding of 1982 Week of AlgebraicGeometry, Bucharest. LNM (1984), 142–187.[21] C.M. Jessop. Quartic surface. Cambridge University Press, 1916.[22] F. Kahl, R. Hartley, and K. ˚Astr¨om. Critical configurations for n-view projective reconstruction. In
IEEE Computer Society Conference on Computer Vision and Pattern Recognition , pages II:158–163,2001.[23] J. Krames. Zur ermittlung eines objectes aus zwei perspectiven (ein beitrag zur theorie der“gefh¨ahlichen ¨Orter”).
Monatsh. Math. Phys. , 49:327–354, 1940.[24] S.J. Maybank. Theory of Reconstruction from Image Motion. Springer-Verlag New York, Inc., Se-caucus, NJ, USA, 1992.[25] G. Ottaviani. On 3–folds in P which are scrolls. Annali Scuola Normale Pisa , Vol. 19, n.3 (1992),451–471.[26] T.G. Room. Self-transformations of determinantal quartic surfaces. II.
Proc. London Math. Soc. (2)51 (1950), 362–382.[27] T.G. Room. Self-transformations of determinantal quartic surfaces. III.
Proc. London Math. Soc. (2)51 (1950), 383–387.[28] T.G. Room. Self-transformations of determinantal quartic surfaces. IV.
Proc. London Math. Soc. (2)51 (1950), 388–400.[29] A. Shashua and S.J. Maybank. Degenerate n point configurations of three views: Do critical surfacesexist? TR 96-19, Hebrew University, 1996.[30] L. Wolf and A. Shashua. On projection matrices P k → P , k = 3 , ..., International Journal of Computer Vision , 2002.
Email address : [email protected] Email address : [email protected] Email address : [email protected]@unimi.it