LLENGTH LEARNING FOR PLANAR EUCLIDEAN CURVES 1
Length Learning for Planar Euclidean Curves
Barak Or and Liam Hazan
Abstract —In this work, we used deep neural networks (DNNs)to solve a fundamental problem in differential geometry. Onecan find many closed-form expressions for calculating curvature,length, and other geometric properties in the literature. As weknow these concepts, we are highly motivated to reconstruct themby using deep neural networks. In this framework, our goal is tolearn geometric properties from examples. The simplest geomet-ric object is a curve. Therefore, this work focuses on learningthe length of planar sampled curves created by a sine wavesdataset. For this reason, the fundamental length axioms werereconstructed using a supervised learning approach. Followingthese axioms a simplified DNN model, we call
ArcLengthNet ,was established. The robustness to additive noise and discretiza-tion errors were tested.
Index Terms —Differential geometry, length learning, planarEuclidean curves, supervised learning.
I. I
NTRODUCTION
The calculation of curve length is the major componentin many classical and modern problems involving numericaldifferential geometry [1], [2]. For example, a handwrittensignature involves the computation of the length along thecurve [3]. Several numerical constraints affect the qualityof the length calculation; additive noise, discretization error,and partial information. A robust approach to handle it isrequired.Lastly, Machine Learning (ML) has become highly popular.It has achieved great success in solving many classification,regression and anomaly detection tasks [4]. A sub-field ofML is the Deep Neural Networks (DNN), which outperformsmany classic methods by design deep architectures [4].An efficient DNN architecture finds intrinsic properties byusing a convolutional operator (and some more sophisticatedoperators) and generalize them. Their success is related tothe enormous amount of data and their capability to optimizeit by high computational available resources. In this work,we address a fundamental question in the field of differentialgeometry [5] and we aim to reconstruct a basic propertyusing DNN. The simplest geometric object is a curve. Tocharacterize a curve, one can define metrics such as lengthand curvature, and distinguish one curve from another.There are many close form expressions for calculation ofthe length, curvature, and other geometric properties in theclassical literature [6]. However, since we know the powerfulfunctions of DNN, we are highly motivated to reconstructthe fundamental length property for curves, the arclength,by designing a DNN. We focused on the two-dimensionalEuclidean domain. The formulation of this task was done in a
B.Or is with the Department of Marine Technologies, Charney School ofMarine Sciences, University of Haifa, Israel (e-mail: [email protected]).L.Hazan is with Industrial Engineering School at Technion, Israel(e-mail:[email protected]). supervised learning method where a data-dependent learning-based approach was applied by feeding each example at atime through our DNN model and by minimizing a uniqueloss function that satisfies the length axioms. For simplicity,we focused on sine wave curves and created a dataset bytuning the wave amplitude, phase, translation and rotation tocover a wide-range of geometric representation. The resultedtrained DNN was called
ArcLengthN et . It obtains a 2Dvector as an input, represents the planar Euclidean sampledcurve, and outputs their respective length.Related papers in the literature mainly address a higher levelof geometric information by deep learning approach [7], [8].Saying that, a fundamental property was reconstructed byDNN in [9], where a curvature-based invariant signature waslearned by using a Siamese network configuration [10]. Theypresented the advantages of using DNN to reconstructingthe curvature signature, among which it mainly results inrobustness to noise and sampling errors.The main contributions of this work is to reconstruct thelength property. For that, two architectures were designed.One is based on Convolutional Neural Networks (CNNs), andthe other is based on Recurrent Neural Networks (RNNs).We showed that the CNN-based architecture overcomesthe RNN-based architecture, and by that establishing the
ArcLengthN et as a CNN based architecture. The advantageof the
ArcLengthN et is presented.The remainder of the paper is organized as follows: Section2 summarizes the geometric background of the lengthproperties. Section 3 provides a detailed description of thelearning approach were the two architectures are presented.Section 4 presents the results followed by the discussion.Section 5 gives the conclusions.II. G
EOMETRIC B ACKGROUND OF A RCLENGTH
In this section, the length properties are presented and thediscretization error is reviewed.
A. Length properties
Consider a planar parametric differential curve in the Eu-clidean space, C ( p ) = { x ( p ) , y ( p ) } ∈ R , where x and y arethe curve coordinates parameterized by parameter p ∈ [0 , N ] ,where N is a partition parameter. The Euclidean length of thecurve, is given by l ( p ) = (cid:90) p | C ˜ p (˜ p ) | d ˜ p = (cid:90) p (cid:113) x p + y p d ˜ p, (1)where x p = dxdp , y p = dydp . Summing all the increments resultsin the total length of C , given by L = (cid:90) N | C ˜ p (˜ p ) | d ˜ p. (2) a r X i v : . [ c s . G R ] F e b ENGTH LEARNING FOR PLANAR EUCLIDEAN CURVES 2
Fig. 1. Discretization
Following the length definition, the main length axioms areprovided.
Additivity : The length additives with respect to concatenation,where for any C and C the following holds L ( C ) + L ( C ) = L ( C ∪ C ) . (3) Invariant : length is invariant with respect to rotation ( R ) andtranslation ( T ), L (( T + R ) C ) = L ( C ) . (4) Monotonic : length is monotone, where for any C and C thefollowing holds L ( C ) ≤ L ( C ) C ⊂ C . (5) Non-negativity : The length of any curve is non-negative, L ( C ) ≥ . (6)In order to reconstruct the length property by DNN, a dis-cretization of the curve should be applied. As a consequence,it prone to errors. B. Discretization error
The curve C lies on a close interval [ α, β ] . In order to findthe length by a discretized process, a partition of the intervalis done, where P = { α = p < p < p < · · · < p N = β } . (7)For every partition P , the curve length can be represented bythe sum s ( P ) = N (cid:88) n =1 | C ( p n ) − C ( p n − ) | . (8)The discretization error is given by, e d = L − s ( P )= (cid:82) N | C p ( p ) | dp − N (cid:80) n =1 | C ( p n ) − C ( p n − ) | . (9)where obviously, e d → when N → ∞ (for further reading,the reader refers to [11]). Fig. 1 illustrates a general curvewith their discretized representation. III. L EARNING APPROACH
A. Motivation
The motivation for using DNN for this task lies in thecore challenge of implementing equations (1) and (2) inreal-life scenarios. These equations involve nonlinearity andderivatives. Poor sampling and additive noise might lead tonumerical errors [12]. The differential and integral operatorscan be obtained by using convolution filters [9]. The differ-ential invariants can be interpreted as a high pass filter andthe integration as a low pass filter. Hence, it is convenientto use the Convolutional Neural Network (CNN) for our task.Another approach to deal with this task involves the RecurrentNeural Network (RNN), where the curve is considered a time-series [13], [14]. A modified version of RNN is the Long-ShortTerm Memory (LSTM), where a weighting for the past timesteps is employed [15]. We implemented two architectures.The first is based on a simplified CNN, and the second isbased on a simplified LSTM. We found that the CNN-basedarchitecture overcomes the LSTM-based architecture. The restof this section presents the details of our process to obtainedboth.
B. Data generation
The reconstruction of the length properties was done in asupervised learning approach, where many curve exampleswith their lengths as labels were considered. Each curve isrepresented by × N vector for the x and y coordinates anda fixed number of points N .We created a dataset with , to fully enable DNNtraining. This large amount of examples aimed to cover curvetransformations and to satisfy different patterns. These curveswere created by taking a sinus wave with a random samplingof amplitude a , phase φ , translation T , and rotation, R . Thegeneral curve in our data is given by C ( p ) = R ˜ C ( p ) + T , (10)where ˜ C ( p ) = a sin ( p + φ ) . (11)A random cutting point was randomly selected to dividethe curve into two new curves to reconstruct the additivityproperty. Fig. 2 shows some curves we created. The data setwas split into train and test sets, where additional holdout setwas created. During the data creation phase, we demandednon-negativity of the labels and created many curves ofdifferent length with rotation and translation examples tocover the various axioms (3)-(6). C. Loss function
In order to show the additive property, we designed a uniqueloss function for each example, where it designed as follows J k = ( L ( s ) − O ( s ) − O ( s )) k + λ (cid:107) Θ ij (cid:107) , (12)where s , s , and s are the input curves that hold the equality L ( s ) = L ( s ) + L ( s ) , O is the DNN output, k is the ENGTH LEARNING FOR PLANAR EUCLIDEAN CURVES 3
Fig. 2. Curve examplesFig. 3. ArcLengthNet architecture example index, λ is a regularization parameter, Θ ij are thevarious DNN weight, and (cid:107)·(cid:107) is the two dimension norm. Byminimizing J k by passing the examples through a model, theweights were tuned. The optimized model is characterized bythe optimal weights that provided by Θ ∗ ij = arg min Θ ij (cid:88) k J k . (13) D. ArcLengthNet Architecture
In this subsection the
ArcLengthN et is presented (theCNN-based architecture). The model architecture is verysimplified, including a convolutional layer and two fully-connected layers with only one activation function. Each curveis represented by N = 200 points. This representation isinserted into a convolutional layer with a small kernel of size . It is processed into a fully connected layer that outputs only weights through a Rectified Linear Unit (ReLU) activationfunction to another fully connected layer which finally outputsthe length. The architecture is shown in Fig.3. E. ArcLengthN et
Training
The DNN was trained by passing many examples in smallbatches with a back-propagation method. The training processwas carried out in batches of examples for epochs.The optimizer we used is Stochastic Gradient Descent (SGD)with momentum and weight decay [16]. Various parametersare provided in Table 1. Fig.4 shows a graph of the train andtest losses as a function of the number of epochs.
Fig. 4. ArcLengthNet training
F. LSTM-based Architecture
The task of length learning can be interpreted as a time-series based learning. Each point of the curve can be inter-preted as a time step with x and y coordinates. The DNN aimsto generalize the local properties into a global length. The typ-ical architecture to deal with time-series data is the RecurrentNeural Network (RNN). One modification of RNN is the LongShort-Term Memory (LSTM) architecture, where has feedbackconnections (in opposite to the classical RNN). The LSTM isused in handwritten recognition tasks with a great success [17].Motivated by this success, we designed an LSTM architecture,presented in Fig.5. Similar to ArcLengthN et , the LSTM-based architecture is simplified, including blocks with vector outputs. They concatenated and inserted to two fullyconnected layers ( each) that outputs connected with a ReLUactivation function. Fig.6 shows a graph of train and test lossesas a function of the number of epochs. TABLE IL
EARNING P ARAMETERS
Description Symbol ValueNunber of examples K , Train/test ratio - / Regularization parameter λ . Partition parameter N Batch size -
Learning rate η . Momentum − . Weight decay - . Epochs -
IV. R
ESULTS AND DISCUSSION
Both architectures were trained in the same approach withthe same database. As shown in Fig.3 and Fig.6 these ar-chitectures were well trained after epochs. A holdoutset was defined to test the performance of the architectureson unseen data. This set contains , examples that havenot been used in train set or test test. The ArcLengthN et obtained a minimum MSE of . , where the LSTM based ENGTH LEARNING FOR PLANAR EUCLIDEAN CURVES 4
Fig. 5. LSTM-based architectureFig. 6. LSTM-based architecture training model obtained a minimum MSE of . . Hence, we con-clude that the ArclengthN et architecture overcomes LSTM-based architecture. The monotonic property was tested forthe
ArcLengthN et on this holdout set, where a linearrelation was established between the true length and the
ArcLengthN et (Fig.4).
A. Performance measure
In order to verify our results, we used the Mean SquaredError (MSE) criterion, Root-MSE (RMSE), as also a unique
Fig. 7. Monotonic property on holdout set (
ArcLengthNet ) criterion: RMSE-over-Mean-Length-Ratio (RMLR), given by RMLR = RMSE E [ L ] , (14)where E is the expected value operator. This measure providesa normalized error with respect to the curve length.variouscurves of different lengths, we must appropriately weigh theerrors. V. C ONCLUSION AND FURTHER WORK
A learning-based approach to reconstruct the length ofcurves was presented. The powerful of deep neural networksto reconstruct the fundamental axioms was demonstrated. Theresults can be further used to improve handwritten signatures,and reconstruct some more differential geometry propertiesand theorems. R
EFERENCES[1] B. Guenter and R. Parent, “Computing the arc length of parametriccurves,”
IEEE Computer Graphics and Applications , vol. 10, no. 3, pp.72–78, 1990.[2] H.-B. Hellweg and M. Crisfield, “A new arc-length method for handlingsharp snap-backs,”
Computers & Structures , vol. 66, no. 5, pp. 704–709,1998.[3] S. Y. Ooi, A. B. J. Teoh, Y. H. Pang, and B. Y. Hiew, “Image-basedhandwritten signature verification using hybrid methods of discreteradon transform, principal component analysis and probabilistic neuralnetwork,”
Applied Soft Computing , vol. 40, pp. 274–282, 2016.[4] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature , vol. 521,no. 7553, pp. 436–444, 2015.[5] S. Sternberg,
Lectures on differential geometry . American MathematicalSoc., 1999, vol. 316.[6] R. Kimmel,
Numerical geometry of images: Theory, algorithms, andapplications . Springer Science & Business Media, 2003.[7] M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst,“Geometric deep learning: going beyond euclidean data,”
IEEE SignalProcessing Magazine , vol. 34, no. 4, pp. 18–42, 2017.[8] J. Berg and K. Nystr¨om, “A unified deep artificial neural networkapproach to partial differential equations in complex geometries,”
Neu-rocomputing , vol. 317, pp. 28–41, 2018.[9] G. Pai, A. Wetzler, and R. Kimmel, “Learning invariant representationsof planar curves,” arXiv preprint arXiv:1611.07807 , 2016.[10] D. Chicco, “Siamese neural networks: An overview,”
Artificial NeuralNetworks , pp. 73–94, 2020.[11] M. P. Do Carmo,
Differential geometry of curves and surfaces: revisedand updated second edition . Courier Dover Publications, 2016.
ENGTH LEARNING FOR PLANAR EUCLIDEAN CURVES 5 [12] S. Qian and J. Weiss, “Wavelets and the numerical solution of partialdifferential equations,”
Journal of Computational Physics , vol. 106,no. 1, pp. 155–175, 1993.[13] A. C. Tsoi and A. Back, “Discrete time recurrent neural networkarchitectures: A unifying review,”
Neurocomputing , vol. 15, no. 3-4,pp. 183–223, 1997.[14] K. Smagulova and A. P. James, “A survey on lstm memristive neuralnetwork architectures and applications,”
The European Physical JournalSpecial Topics , vol. 228, no. 10, pp. 2313–2324, 2019.[15] S. Hochreiter and J. Schmidhuber, “Long short-term memory,”
Neuralcomputation , vol. 9, no. 8, pp. 1735–1780, 1997.[16] S. Ruder, “An overview of gradient descent optimization algorithms,” arXiv preprint arXiv:1609.04747 , 2016.[17] A. Graves, M. Liwicki, S. Fern´andez, R. Bertolami, H. Bunke, andJ. Schmidhuber, “A novel connectionist system for unconstrained hand-writing recognition,”