Reconstruction of~3-D Rigid Smooth Curves Moving Free when Two Traceable Points Only are Available
aa r X i v : . [ c s . C V ] A p r Reconstruction of 3-D Rigid Smooth CurvesMoving Free when Two Traceable Points Only areAvailable
Mieczys law A. K lopotek
Institute of Computer Science Polish Academy of Sciences
Reconstruction of shape of surrounding objects is a vital task to cope with by future(and partially by present) intelligent systems interacting with real world. In general,recovering of shapes of 3D space objects from 2D images [1, 2, 3, 4, 8, 16, 17, 20, 21,28] has not been very successful as this task is too under-constrained (unless shape[5, 23], shading etc [7, 22] clues are available). Hence practical applications arerather based on sensing (via laser beams, ultrasonic methods etc. [18, 19]). Thoughsuccessful in recovering shapes of surfaces, sensing fails to reconstruct curve-shapedobjects as well as curved surface edges.So this remains still a competition area for 2D projection based recognitionmethods. Some promising results were in fact achieved in recovering objects frommultiframes (a time sequence of projections of the moving object) [6, 13, 24, 25, 26]as this task is over-constrained. Also in cases where features of interest cannot beall traced from frame to frame — e.g. smooth-curve shaped objects [9, 10, 11, 12,14, 15, 27]. In fact, only several points (usually end points) are traceable, and theremaining ones are not. The strategy consists usually of two stages: reconstructionof space parameters of traceable points, thereafter reconstruction of non-traceablepoints.This paper extends (in sections 3 & 4) previous results in that sense that fororthogonal projections of rigid smooth (true-3D) curves moving totally free it reducesthe number of required traceable points to two only (the best results known sofar to the author are 3 points from free motion and 2 for motion restricted torotation around a fixed direction and and 2 for motion restricted to influence ofa homogeneous force field). The method used is exploitation of information ontangential projections. Section 5 contains a remark on possibility of simplificationof reconstruction of flat curves moving free for prospective projections.1
Previous Work
The following table summarizes previous work in the area of reconstruction of rigidcurves from multiframes under various shape and motion restrictions for orthogo-nal and prospective projections. Shift/rotation motion is “uniform” if in the sameelapsed time the same amount of shift/rotation occurs. The motion is free if it doesnot fit this requirement of uniformity. A homogeneous force field causes the masscenter point to have a constant acceleration vector.
Motion Type Number of Number
Refe- traceable points of Frames renceFLAT (2D) CURVES IN 3D
Orthogonal Projection free motion 2 2 [12]Prospective Projection free motion 3 3 [11]REAL 3 D CURVESOrthogonal Projection: uniform rotational motion 2 4 [27, 12] free rotation around afixed direction 2 4 [9] free motion 3 3 [9, 10] free motion, bounded by homogeneousforce field 2 n [10]Prospective Projection: rotation-free motion 2 2 [9] uniform rotational motion 2 5 [9] free rotation around afixed direction 3 3 [9] free motion 4 3 [9] free motion, bounded by homogeneousforce field 2 n [11]Stereoscopic Vision: free motion 2 1 [9, 28]
Let us characterize the traceables of the smooth curve. We assume that we cantrace two points (usually endpoints) of it. Let the traceable points be A and B .2 BA’ B’Figure 1: A smooth 3-D curve and its orthogonal projectionTheir projections be called A ′ i and B ′ i ( i – frame index). Both A ′ i and B ′ i are ob-servables. Furthermore we can observe the angles between A ′ i B ′ i and the projectionsof tangentials at A and B being projections of angles between AB and tangentialsthemselves (Fig.1). Let us call α the angle between AB and the tangential at B ,and β the angle between AB and tangential at A . φ be the angle between the planecontaining AB and tangential at A and the plane containing AB and the tangentialat B .Length of AB be called c. c, α , β , φ are fixed through all frames.Let us consider the relation between the i th frame and the curve – especially theline AB and the tangential at B . We can always imagine that the current positionof the curve was achieved as follows:1. At the beginning A , B and tangential at B lay in the frame plane in such away that A ′ i = A . Let us draw a straight line l AB . Let p l
1. (Fig.2).2. First the curve is rotated by an angle δ i around the by now line AB (Fig. 3.).Let us fix on the tangential at B the point S at which by now the tangentialcrosses the plane p
1. Let S ′ be the orthogonal projection of S in the plane p l
1. Then we have: AS ′ S = 90 o , SAS ′ = δ i , ABS = α ,hence: 31A B Framep1Figure 2: The 3-D curve ’lying’ on the projection plane1) ASAB = tg α AS ′ AS = cos δ i SS ′ AB = sin δ i
3. Thereafter we rotate the whole curve together with the point S (not S ′ ) aroundthe line l τ i (Fig.4.). Let S ′′ be orthogonal projection of thenewly positioned S onto the frame plane. Then obviously BAB ′ = τ i , S ′′ S ′′ A = 90 o , SS ′ S ′′ = 90 o − τ i . Hence:4) AB ′ AB = cos τ i S ′ S ′′ SS ′ = cos(90 o − τ i )Let us denote by D ′ the crossing point of the lines l B ′ S ′′ . As we knowthe line l B ′ S ′′ (being the orthogonal projection of thetangential BS at B ), we know also the position of D ′ . We obtain:6) AS ′ = AD ′ + D ′ S ′ As AB ′ is parallel to S ′ S ′′ (both in frame plane and both perpendicular to l AD ′ D ′ S ′ = AB ′ S ′ S ′′
41A B SS’Figure 3: The 3-D curve rotated around A’B’4. The shift of the whole curve from the projection frame in perpendicular di-rection (Fig.5.) has no effect on the shape of projection and hence may beomitted from consideration.Remark: we have dropped index i on primed and double primed points and onS to increase the legibility of formulas.Summarizing, we obtained 7 equations in unknowns: c = AB, α — global for all frames τ i , δ i , AS ′ , SS ′ , D ′ S ′ , S ′ S ′′ — local for a frame(as A ′ , B ′ and D ′ are visible, so AB ′ = c ′ i and AD ′ = d ′ i are known).We derive eliminating AS by (1):2 ′ ) AS ′ = cos δ i ∗ c ∗ tg α and3 ′ ) SS ′ = sin δ i ∗ c ∗ tg α Eliminating AS ′ and SS ′ by (2’) and (3’) we derive:5 ′′ ) S ′ S ′′ = sin τ i ∗ sin δ i ∗ c ∗ tg α and6 ′′ ) cos δ i ∗ c ∗ tg α = AD ′ + D ′ S ′ Eliminating S ′ S ′′ and D ′ S ′ by (5”) and (6”) we obtain:51A B SS’D’ S”Figure 4: The 3-D curve rotated twice61A B SD’ S”Figure 5: The 3-D curve in space7 ′′′ ) AD ′ cos δ i c tg α − AD ′ = AB ′ sin τ i sin δ i tg α Substituting (4) into (7 ′′′ ) we get:8) d ′ i c tg α sin arccos( c ′ /c ) sin δ c ′ i c tg α cos δ i − d ′ i c ′ i . – one equation with one local (frame dependent) unknown δ i .However, by analogy, we can derive the second equation for the same frame consid-ering the opposite side of the frame plane and the point B and the tangential at A instead of the point A and the tangential at the point B . So we have the line l l B , observable point E ′ (and edge BE ′ = e ′ i ) instead of D ′ (and d ′ i ). The rotation around l l δ i ), but the rotationaround AB must be τ i + φ , φ being the angle between the plane containing AB andtangential at A and the plane containing AB and the tangential at B (fixed for allframes). So we obtain:9) e ′ i c tg β sin arccos( c ′ /c ) sin( δ i + φ ) = c ′ i c tg β cos( δ i + φ ) − e ′ i c ′ i .Let us introduce auxiliary (frame) terms, containing only frame knowns and globalunknowns: q i = c ′ i c tg α, p i = d ′ i c tg α sin arccos( c ′ i /c ) q i = c ′ i c tg β, p i = e ′ i c tg α sin arccos( c ′ i /c )So we have the equation system: 70) p i sin δ i = q i cos δ i − d ′ i c ′ i . p i sin( δ i + φ ) = q i cos( δ i + φ ) − e ′ i c ′ i .Let us transform (10):10 ′ ) d ′ i c ′ i / q p i + q i = cos δ i q i / q p i + q i − sin δ i p i / q p i + q i If we introduce a new auxiliary variable ω i (with global unknowns only) ω i = arc tg ( p i /q i )then we have:10 ′′ ) d ′ i c ′ i / q p i + q i = cos( δ i + ω i ) and by analogy:11 ′′ ) e ′ i c ′ i / q p i + q i = cos( δ i + φ + ω i )and hence:10 ′′′ ) arccos( d ′ i c ′ i / q p i + q i ) = δ i + ω i and11 ′′′ ) arccos( e ′ i c ′ i / q p i + q i ) = δ i + φ + ω i And thus we come to our final formula:12) arccos( e ′ i c ′ i / q p i + q i ) − arccos( d ′ i c ′ i / q p i + q i ) = φ + ω i − ω i – one equation for each frame in unknowns: c , α , β and φ , which does not containany frame dependent unknown.For determining all these four unknowns characterizing the reconstructed curve weneed at least four frames.Degenerated cases (parallelism of lines) are treated easily and will not be consideredhere.The formula (12) is, regrettably, not a practical one, though the equation systemis solvable. Therefore the result is more of theoretical importance than of practicalone. However, it is possible to transform this formula into a (high degree) polyno-mial in c , tg α , tg β and tg φ , which can be a basis of a linear equation systemconstructed from a superfluous number of additionally observed frames, where thesolution is based on conjecture of linear coefficient independence formulated in [10]and successfully applied therein to free motion under orthogonal projection withthree traceable points and 3+1 frames.Let us briefly outline the transformation of (12) into a lopynomial in the above-mentioned variables. After ”tangentializing” and squaring twice we obtain a quasi-polynomial of the form:12’) s + s + y + y s s − s s − ys − ys s − ys − ys s − y s s − ys s = 08ith y standing for y = tg ( φ + ω i − ω i )and s , s being proper polynomials: s = ( d ′ i + c ′ i ) c tgα − d ′ i c ′ i ( tg α − d ′ i c ′ i s = ( e ′ i + c ′ i ) c tgβ − e ′ i c ′ i ( tg β − e ′ i c ′ i So the only non-polynomial factor is y . However: y = ( d ′ i c ′ i − d ′ i c )( e ′ i c ′ i − e ′ i c )( tg φ + 1)(1 + d i e i c ′ i − d i e i c + tgφ ( e i c i − e i c i ) r − c ′ i c ) ((1 + d i e i /c ′ i − d i e i /c ) − tg φ ( e i /c i − d i /c i ) (1 − c ′ i /c )) − polynomial polynomial ∗ q (1 − c ′ i /c )which is easily squared to obtain a proper polynomial in the above-mentioned vari-ables.To solve a system of equations being polynomials of high degree when superfluousobservations from real world are available we proceed the following way: we trans-form the equations in the following form:0 = X expression − in − observables ∗ product − of − variables − and − their − natural − powers We insist on each product − of − variables − and − their − natural − powers be different in each summand. For each product − ... we introduce a new variable a k (something like the procedure when seeking a model for polynomial regressionby means of linear regression method). In this way we obtain a linear equationsystem which we solve using Gaussian method (if the number of equations is equalto the number of new variables a i or by the least squares methods if the number ofequations (that is, observed frames) is higher.Solving such an equation system results in obtaining another one with equationsof the form: productof variables = constant , which after application of logarithmresults in a new linear equation system, this time in variables of primary interest..Why should this method ( conjecture of linear coefficient independence) work ? Ofcourse, degenerate cases are possible. It works however the very same way the linearregression does: we usually observe much less variables than there are degrees offreedom in the real world. 9 BA’B’ XX’ A”B” X”Figure 6: Recovering non-traceable points The formulas of previous section allow to identify the the relative (length c) aswell as the absolute position of the feature points
A.B as well as the (absolute andrelative) direction of tangentials at A and B in space for each frame.To recover the shape of the whole smooth curve, it is necessary to recover non-feature points also. It will be possible only if points
A, B and tangentials are notco-planar. Then each point in space is defined by means of parameters ( p , p , p A + p ∗ AB + p ∗ AC + p ∗ AB × AC (the point C be such that AC is the tangential at A and AC is of unit length, x –cross product indicator), and each straight line as:14) f ( u ) = p ∗ AB + p ∗ AC + p ∗ AB × AC + u ∗ ( q ∗ AB + q ∗ AC + q ∗ AB × AC )(O be the coordinate system origin) It is obvious that in case of rigidly connectedpoints A, B, C every point and every straight line rigidly connected with them willretain the p , p , p , q , q , q
3) parameter set while the motion continues.So let us select a point X o in frame 0 lying in the projection plane on the projectedcurve (Fig.6.), this point being projection of a point X of the curve. We will succeed10ith the reconstruction task for the point X if we find out the distance XX o for theframe 0 . x be the name of the straight line connecting X with X o (vertical to theprojection plane). Let us obtain the parameters p , p , p f ( u ) for the frame 1 positionof A, B, C . Then this projected line will cross the projected curve at some pointsone of them being the point X – the projection of our point X of the curve. (Onambiguity we can recall the continuity of the curve). As we know the equation ofthe straight line XX in frame 1 as well as that of the straight line x we can easilyrecover the distance XX , and later XX of the first frame. Proceeding in this waywe recover the whole curve.(Ambiguities are resolved by continuity requirement). This work profited from analysis of Lee’s [12] method of reconstructing correspon-dence between two orthogonal projections of a flat curve in 3D. The basic idea therewas that having two traceable (end)points of the curve we have in fact three of them:the third being the crossing point of tangentials at curve endpoints (as the curveis assumed “flat” that is planar, the tangentials – unless parallel – in fact have acommon point). Then Lee simply exploited the Tales theorem in a straight forwardway.We would like to point out here that there is also a similar simple method forreconstruction of correspondence between two PROSPECTIVE projections of a flatcurve in 3D, but three traceable points of the curve are required then. Though nobetter bound is achieved for the number of traceable points required than that in[11], however the computational effort is drastically reduced: Let the three traceablepoints be called
A, B, C (Fig.7.). Clearly usually the tangentials at A and B sharea point, say D . Let us call E the common point of straight lines AB and DC .Let A ′ , B ′ , C ′ D ′ , E be projections of A, B, C, D, E respectively in the first frame,and A ′′ , B ′′ , C ′′ , D ′′ , E ′′ be respective projections in the second frame (Projectionsof A, B and C are visible, and projections of D and E are easily obtainable bydrawing). Now let us consider X ′ , a projection of the non-traceable point X of thecurve in the first frame (let us select X ′ freely on the curve projection image of thefirst frame.). We want to find X ′′ being the projection of X in the second frame.Let us call Y the common point of lines AB and DX — its projection Y ′ can beobtained by drawing as crossing point of A ′ B ′ and D ′ X ′ . Let us look for Y ′′ - theprojection of Y in the second frame. The well known elementary geometry theoremon prospective projection double quotient states that:15) AEAY : BEBY = A ′ E ′ A ′ Y ′ : B ′ E ′ B ′ Y ′ and16) AEAY : BEBY = A ′′ E ′′ A ′′ Y ′′ : B ′′ E ′′ B ′′ Y ′′ hence11’ B’C’D’E’ X’Y’A” B”C”D” E”X”Y”Figure 7: Prospective projections of 2D-curves in 3D17) A ′ E ′ A ′ Y ′ : B ′ E ′ B ′ Y ′ = A ′′ E ′′ A ′′ Y ′′ : B ′′ E ′′ B ′′ Y ′′ As the positions of the remaining points A ′ , B ′ , E ′ , Y ′ , A ′′ , B ′′ , E ′′ are known, sobased on (17) Y ′′ is easily found on the line A ′′ B ′′ . But X ′′ is the crossing point ofthe straight line D ′′ Y ′′ and the image of curve projection in the second frame, so easyto find Q.E.D. (ambiguities are resolved by continuity requirement). Degeneratedcases (parallelism of lines) are treated easily and will not be considered here. This paper makes two basic contributions to solution of the problem of reconstruc-tion of rigid smooth curves from multiframes:1. decreases to 2 the theoretical lower bound on the number of traceable pointsrequired to reconstruct the shape of a true 3-D curve from multiframes underorthogonal projection with totally unpredictable motion assumed (the previousbound was either 3 points or 2 points with geometrical or physical restrictionon freedom of motion)2. introduces a new algorithm (based on double quotient) for reconstruction offlat curves in 3 D from multiframes under prospective projection using 3 trace-able points, which is drastically simpler than that given in [11].12t this point the basic statement holding for all reconstruction algorithms basedon multiframes should be repeated: unless the motion is a degenerate one (e.g. nomotion at all, or no rotation at all, or rotation around an axis perpendicular to theframe plane etc.). If we compare the table given in Section 2 with the results ofsections 3/4, we see easily that there is some ranking on the complexity of recoveringalgorithms depending on the amount and type of information available. E.g. from[9] we know that with 3 traceable points and three frames available we obtain anequation system with 3 mixed-quadratic equations in three variables. From [10]we know that adding one frame more leads us to an equation system with 3 linearequations in three variables. We can also observe that three point mean a specialcase of two points and two lines. From the complexity of equation (12) and the factthat 4 frames are required at least, however, we see that availability of two lines is amuch weaker information that that stemming from a third point. Further researchis necessary to simplify eventually the solution given in (12). Also we hope thatexploiting some insights from consideration of flat curves in 3D under prospectiveprojection also the bound of 4 traceable points necessary by now for true 3 D curvesmay be broken in future research work.