A Deep Collocation Method for the Bending Analysis of Kirchhoff Plate
AA Deep Collocation Method for the BendingAnalysis of Kirchhoff Plate
Hongwei Guo a , Timon Rabczuk b , and Xiaoying Zhuang ∗a,c,da Institute of Continuum Mechanics, Leibniz Universität Hannover,Appelstraße 11, 30157 Hannover, Germany b Institute of Structural Mechanics, Bauhaus-UniversitätWeimar, Marienstr.15 D-99423 Weimar, Germany c Department of Geotechnical Engineering, Tongji University, Siping Road1239, 200092 Shanghai, P.R.China d Key Laboratory of Geotechnical and Underground Engineering ofMinistry of Education, Tongji University, 200092 Shanghai, P.R.China
Abstract
In this paper, a deep collocation method (DCM) for thin plate bending problems isproposed. This method takes advantage of computational graphs and backpropaga-tion algorithms involved in deep learning. Besides, the proposed DCM is based on afeedforward deep neural network (DNN) and differs from most previous applicationsof deep learning for mechanical problems. First, batches of randomly distributed col-location points are initially generated inside the domain and along the boundaries.A loss function is built with the aim that the governing partial differential equations(PDEs) of Kirchhoff plate bending problems, and the boundary/initial conditions areminimised at those collocation points. A combination of optimizers is adopted to inthe backpropagation process to minimize the loss function so as to obtain the optimalhyperparameters. In Kirchhoff plate bending problems, the C1 continuity requirementposes significant difficulties in traditional mesh-based methods. This can be solved by ∗ Corresponding authors:Xiaoying Zhuang, +49 511 762-19589, [email protected] a r X i v : . [ m a t h . NA ] F e b he proposed DCM, which uses a deep neural network to approximate the continuoustransversal deflection, and is proved to be suitable to the bending analysis of Kirchhoffplate of various geometries. Keywords:
Deep learning, Collocation method, Kirchhoff plate, Higher-order PDEs.
Thin plates are widely employed as basic structural components in engineering fields[1], which combines light weight, efficient load-carrying capacity, economy with tech-nological effectiveness. Their mechanical behaviours have long been studied by var-ious methods such as finite element method [2, 3], boundary element method [4, 5],meshfree method [6], isogeometric analysis [7], and numerical manifold method [8–10]. The Kirchhoff bending problem is a classical fourth-order problem, its mechan-ical behaviour is described by fourth-order partial differential equation as it is prettydifficult to construct a shape function to be globally C continuous but piecewise C continuous, namely, H regular, for those mesh-based numerical method. However,according to the universal approximation theorem, see Cybenko [11] and Hornic [12],any continuous function can be approximated arbitrarily well by a feedforward neuralnetwork, even with a single hidden layer, which offers a new possibility of analysingKirchhoff plate bending problems. We will first give a brief introduction of deep learn-ing.Deep learning was first brought up as a new branch of machine learning in therealm of artificial intelligence in 2006 [13], which uses deep neural networks to learnfeatures of data with high-level of abstractions [14]. The deep neural networks adoptartificial neural network architectures with various hidden layers, which exponentiallyreduce the computational cost and amount of training data in some applications [15].The major two desirable traits of deep learning lie in the nonlinear processing in mul-tiple hidden layers in supervised or unsupervised learning [13]. Several types of deepneural networks such as convolutional neural networks (CNN) and recurrent/recursiveneural networks (RNN) [16] have been created, which further boost the application ofdeep learning in image processing [17], object detection [18], speech recognition [19]and many other domains including genomics [20] and even finance [21].As a matter of fact, artificial neural networks (ANN) which are main tools in deeplearning have been around since the 1940’s [22] but have not performed well until re-cently. They only become a major part of machine learning in the last several decadesdue to strides in computing techniques and explosive growth in date collection andavailability, especially the arrival of backpropagation technique and advance in deepneural networks. However, based on the function approximation capabilities of feedforward neural networks, ANN were adopted to solving partial differential equations(PDEs) [23–25], which results in a solution that can be described by a closed analyt- cal form. Basically, ANN methods can be suitable for solving PDEs in that they aresmooth enough, solutions in analytical forms can be evaluated at arbitrary points in oroutside the problem domain. Yadav et al. elaborately introduced the network methodsfor differential equations [26]. In the past, when neural networks with many hiddenlayers were tried to solve nonlinear PDEs in order to get a better results, it usually tooka long time for training, which is due to a vanishing gradient problem. However, theproposal of pretraining, which sets the initial values of connection weights and biases,with the back propagation algorithm are now proposed to solve this problem efficiently.More recently, with improved theory incorporating unsupervised pre-training, stacksof auto-encoder variants, and deep belief nets, deep learning has become a central andpopular hotspot in research and applications.Also, some researchers studied the application of deep learning in solving PDEs.Mills et al. deployed a deep conventional neural network to solve Schr ¨o dinger equa-tion, which directly learned the mapping between potential and energy [27]. E etal. applied deep learning-based numerical methods for high-dimensional parabolicPDEs and back-forward stochastic differential equations, which was proven to be effi-cient and accurate even for 100-dimensional nonlinear PDEs [28, 29]. Also, E and Yuproposed a Deep Ritz method for solving variational problems arising from partial dif-ferential equations [30]. Raissi et al. however solves PDEs in a different way and hasmade a series of contribution to this field. They first applied the probabilistic machinelearning in solving linear and nonlinear differential equations using Gaussian Pro-cesses and later introduced a data-driven Numerical Gaussian Processes to solve time-dependent and nonlinear PDEs, which circumvented the need for spatial discretiza-tion [31–33]. Later, Raissi et al. [34–36] introduced a physical informed neural net-works for supervised learning of nonlinear partial differential equations from Burger’sequations to Navier-Stokes equations. Two distinct models were tailored for spatio-temporal datasets: continuous time and discrete time models. Raissi later employeda deep learning approach for discovering nonlinear PDEs from noisy observations inspace and time with two deep neural networks, one for the representation of nonlinear-dynamic PDEs and one for a prior on the unknown solution [37]. Raissi applied a deepneural networks in solving coupled forward-backward stochastic differential equationsand their corresponding high-dimensional PDEs [38]. Beck et al. [39, 40] studied thedeep learning in solving stochastic differential equations and Kolmogorov equations,and validated the accuracy and speed proposed method, especially in high dimensions.Nabian and Meidani studied the presentation of high-dimensional random partial dif-ferential equations with a feed-forward fully-connected deep neural networks [41,42].Based on the physics informed deep neural networks, Tartakovsky et al. studied the es-timation of parameters and unknown physics in PDE models [43]. Qin et al. appliedthe deep residual network and observation data to approximate unknown governingdifferential equations [44]. Sirignano and Spiliopoulos [45] gave a theoretic motiva-tion of using deep neural networks as PDE approximators, which converges as the umber of hidden layers tend to infinity. Based on this, a deep Galerkin method wastested to solve PDEs including high-dimensional ones. Berg and Nystr ¨o m [46] pro-posed a unified deep neural network approach to approximate solutions to PDEs andthen used deep learning to discover PDEs hidden in complex data sets from measure-ment data [47]. In general, a deep feed-forward neural networks can well-sever as asuitable solution approximators, especially for high-dimensional PDEs with complexdomains.Meanwhile, some researchers study the surrogate of FEM by deep learning, whichmainly trains the deep neural networks from datasets obtained from FEM. From workdone by Liang et al. [48,49], a machine learning approach was first used to investigatethe relationship between geometric features of aorta and FEM-predicted ascendingaortic aneurysm rupture risk and then a deep learning was used to estimate the stressdistribution of the aorta, which will be beneficial to real-time patient-specific com-putational simulations. Lee et al. introduced the background information involved inusing deep learning for structural engineering [50]. Later, Wang et al. [51] applieddeep learning in calculating U* index for the high efficient load paths analysis, withtraining data obtained from ANSYS results.However, in this research, we will not confine deep learning application withinFEM datasets. Rather, the deflection of Kirchhoff plate is first approximated withdeep physical informed feedforward neural networks with hyperbolic tangent activa-tion functions and trained by minimizing loss function related to governing equationof Kirchhoff bending problems and related boundary conditions. The training data fordeep neural networks are obtained by randomly distributed collocation points fromthe physical domain of the plate. And clearly, this deep collocation method is a trulymesh-free method without the need of background grids. In this study, the methodis established and applied to enrich deep learning with longstanding developments inengineering mechanics.The paper is organised as follows: First a brief introduction of Kirchhoff platebending strong form with typical boundary conditions is given. Then we introduce abasic knowledge of the deep learning technique and algorithms, which be helpful forlater application. For numerical analysis, the deep collocation method with varyinghidden layers and neurons are adopted for plates with various shapes, boundary andload conditions, hoping to manifest the favourable numerical features such as highaccuracy and robustness of the proposed method. Based on Kirchhoff plate bending theory [1], the relation between lateral deflection w ( x, y ) of the middle surface ( z = 0) and rotations about the x , y -axis can be given ig. 1. Kirchhoff plate in the coordinate system. Fig. 2. Cover systems of an irregular polygon plate. Ω m4 Ω m3 Ω m2 Ω m1 Ω m8 Ω m7 Ω m6 Ω m5 Ω p1-1 Ω p1-2 Ω p1-3 Ω p1-4 Ω p1-8 Ω p1-7 Ω p1-6 Ω p1-5 E E Figure 1: Kirchhoff plate in the coordinate system. by θ x = ∂w∂x , θ y = ∂w∂y . (1)Under the coordinate system shown in Figure 1, the displacement field in a thin platecan be expressed as: u ( x, y, z ) = − z ∂w∂x ,v ( x, y, z ) = − z ∂w∂y ,w ( x, y, z ) = w ( x, y ) . (2)It is obviously that the transversal deflection of the middle plane of the thin plate can beregard as the field variables of the bending problem of thin plates. The correspondingbending and twist curvatures are the generalized strains: k x = − ∂ w∂x , k y = − ∂ w∂y , k xy = − ∂ w∂x∂y . (3)Therefore, the geometric equations of Kirchhoff bending can be expressed as: k = k xx k yy k xy = − ∂ w∂x ∂ w∂y ∂ w∂x∂y = L w, (4) ith L being the differential operator defined as L = − (cid:16) ∂ ∂x ∂ ∂y ∂ ∂x∂y (cid:17) T . Ac-cordingly, the bending and twisting moments, shown in Figure 1 can be obtained as: M x = − D (cid:18) ∂ w∂x + ν ∂ w∂y (cid:19) ,M y = − D (cid:18) ∂ w∂y + ν ∂ w∂x (cid:19) ,M xy = M yx = − D (1 − ν ) ∂ w∂xy . (5)Here D = Eh − ν ) is the bending rigidity, where E and ν are the Young’s modulusand Poisson ratio, and h is the thickness of the thin plate. For isotropic thin plate, theconstitutive equation can be expressed in Matrix form M=Dk (6)with D = D ν ν − ν ) / . The shear forces can be obtained in terms of thegeneralizsed stress components Q x = ∂M x ∂x + ∂M xy ∂y , Q y = ∂M xy ∂x + ∂M y ∂y (7)The differential equation for the deflections for thin plate based on Kirchhoff’sassumptions can be expressed by transversal deflection as (cid:53) (cid:0) (cid:53) w (cid:1) = (cid:53) w = pD (8)where (cid:53) () = ∂ ∂x + 2 ∂ ∂x ∂y + ∂ ∂y is commonly called biharmonic operator.Consequently, the Kirchhoff plate bending problems can be boiled down to a fourthorder PDE problem, which pose difficulty for tradition mesh-based method in con-structing a shape function to be H regular. Moreover, the boundary conditions ofKirchhoff plate taken into consideration in this paper can be generally classified intothree parts, namely, ∂ Ω = Γ + Γ + Γ . (9)For clamped edge boundary, Γ : w = ˜ w, ∂w∂n = ˜ θ n , w = ˜ w, ˜ θ n are functions ofarc length along this boundary.For simply supported edge boundary, Γ : w = ˜ w, M n = ˜ M n , ˜ M n is also afunction of arc length along this boundary.For free boundary conditions, Γ : M n = ˜ M n , ∂M ns ∂s + Q n = ˜ q , where ˜ q is theload exerted along this boundary.It should be noted that n , s here refer to the normal and tangent directions alongthe boundaries. Deep Collocation Method for solving Kirchhoffplate bending
In this section, we will begin with introducing some preliminaries on deep learning,including the feed forward neural network architectures, some useful algorithms in-volved in deep learning. Then based on those basis, the formulation of deep colloca-tion method is elucidated.
The basic architecture of a fully connected feedforward neural network is shown inFigure 2, which comprises of multiple layers: input layer, one or more hidden layersand output layer. Each layer consists of one or more nodes called neurons, shown inthe Figure 2 by small coloured circles, which is the basic unit of computation. For aninterconnected structure, every two neurons in neighbouring layers have a connection,which is represented by a connection weight. Depicted in Figure 2, the weight betweenneuron k in hidden layer l − and neuron j in hidden layer l is denoted by w ljk . Noconnection exists among neurons in the same layer as well as in the non-neighbouringlayers. Input data, defined from x to x N , flow through this neural network via con-nections between neurons, starting from input layer, through hidden layer l − , l , tooutput layer, which eventually output data from y to y M . The feedforward neuralwork defines a mapping F N N : R N → R M .However, it should be noted that the number of neurons on each hidden layers andnumber of hidden layers can be any number and are invariably determined througha trial and error procedure. It has also been concluded that any continuous functioncan be approximated with any desired precision by a feed forward with even a singlehidden layer [52, 53].On each neuron in the feed forward neural network, a bias is supplied includingneurons in the output layer except the neurons in the input layer, which is definedby b lj for bias of neuron j in layer l . Besides, the activation function is defined foroutput of each neuron in order to introduce a non-linearity into the neural networkand make the back-propagation possible where gradients are supplied along with anerror to update weights and biases. The activation function in layer l will be denotedby σ here. There are many activation functions can be used such as sigmoids function,hyperbolic tangent function ( T anh ) , Rectified linear units ( Relu ) , and so on. Somesuggestions upon the choice of activation function can be referred in [54]. Hence,for the value on each neuron in the hidden layers and output layer adds the weightedsum of values of output values from the previous layer with corresponding connectionweights to basis on the neuron. A intermediate quantity for neuron j on hidden layer orward propagation of activation values ... ... b l − k ... ... b lj ... ... x x N b b M y y M w ljk Input Layer Hidden Layer l − Hidden Layer l Output Layer
Back propagation of errors
Figure 2: Architecture of a fully connected feedforward back-propagation neural network. l is defined as a lj = (cid:88) k w ljk y l − k + b lj , (10)and its output is given by the activation of the above weighted input y lj = σ (cid:16) a lj (cid:17) = σ (cid:32)(cid:88) k w ljk y l − k + b lj (cid:33) , (11)where y l − k is the output from previous layer.So, basically, when Equation 11 is applied to compute y lj , the intermediate quantity a lj was calculated along the way. This quantity turns out to be useful and named hereas weighted input to neuron j on hidden layer l . Equation 10 can be written in acompact matrix form, which calculate weighted inputs for all neurons on certain layerefficiently, obtaining: a = W l y l − + b l , (12)and accordingly, from Equation 12, y l = σ ( a ) , where activation function is appliedelementwise. A feedforward network thus defines a function f ( x ; θ ) depending oninput data x and parametrised by θ consisting of weights and biases in each layer. Thedefined function provides an efficient way to approximate unknown field variables. .2 Backpropagation Backpropagation ( backward propagation ) is an important and computationally effi-cient mathematical tool to compute gradients in deep learning [55]. Essentially, back-propagation is based on recursively applying the chain rule and decides which compu-tations can be run in parallel from computational graphs. In our problem, the govern-ing equation is the fourth order partial derivatives of field variable w ( x ) approximatedby the deep neural networks f ( x ; θ ) , so this makes backpropagation a critical role. Forthe approximation defined by f ( x ; θ ) , in order to find the weights and biases, a lossfunction L ( f , w ) is defined to be minimised [56]. The backpropagation algorithm forcomputing the gradient of loss function L ( f , w ) can be defined as follows [55]:• Input : Input dataset x , ..., x n , prepare activation y for input layer;• Feedforward : For each layer xl = 2 , , ..., L , compute a l = (cid:80) k W l y l − + b l ,and σ (cid:0) a l (cid:1) ;• Output error : Compute the error δ L = ∇ y L L (cid:12) σ (cid:48) L ( a L ) • Backpropagation error : For each l = L − , L − , ..., , compute δ l = (cid:0) ( W l +1 ) T δ l +1 (cid:1) (cid:12) σ (cid:48) l ( a l ) ;• Output : The gradient of the loss function is given by ∂ L ∂w ljk = y l − k δ lj and ∂ L ∂b lj = δ lj .Here, (cid:12) denotes the Hadamard product.Now, there are a lists of deep learning frameworks for us to choose to setup a train-ing. The main two approaches, Pytorch and Tensorflow, however computing deriva-tives in the computational graphs distinctly. The former inputs a numerical value andthen compute the derivatives at this node, while the latter computers the derivativesof a symbolic variable, then store the derivative operations into new nodes addedto the graph for later use. Obviously, the latter is more advantageous in computinghigher-order derivatives, which can be computed from its extended graph by runningbackpropagation repeatedly. In this paper, since the fourth-order derivatives of fieldvariables is needed to be computed, the Tensorflow framework is thus adopted forcalculation [57]. The formulation of a deep collocation in solving Kirchhoff plate bending problemsis introduction in this section. Collocation method is a widely used method seekingnumerical solutions for ordinary, partial differential and integral equations [58]. Itis a popular method for trajectory optimization in control theory. A set of randomlydistributed points (also known as collocation points) is often deployed to represent a esired trajectory that minimizes the loss function while satisfying a set of constraints.The collocation methods tend to be relatively insensitive to instability of system (suchas blowing/vanishing gradients with neural networks), then it can be a viable way totrain the deep neural networks in this paper [59].Recalled form Section 2, Equation 8,9, the solving of Kirchhoff plate bendingproblems can be boiled down to the solving of a fourth order biharmonic equationswith the type of boundary constraints. Thus we first discretize the physical domainwith collocation points denoted by x Ω = ( x , ..., x N Ω ) T . Another set of collocationpoints are deployed to discretize boundary conditions denoted by x Γ ( x , ..., x N Γ ) T .Then the transversal deflection w is approximated with the aforementioned deep feed-forward neural network w h ( x ; θ ) . A loss function can thus be constructed to findthe approximate solution by considering the minimizing of governing equation withboundary conditions approximated by w h ( x ; θ ) . The mean squared error loss form isadopted here.Substituting w h ( x Ω ; θ ) into Equation 8, we can get: G ( x Ω ; θ ) = (cid:53) w h ( x Ω ; θ ) − pD , (13)which results in a physical informed deep neural network G ( x Ω ; θ ) .For boundary conditions illustrated in Section 2, considering all three boundaries,they can also be expressed by the neural network approximation w h ( x Γ ; θ ) as:On Γ , we have w h ( x Γ ; θ ) = ˜ w, ∂ w h ( x Γ ; θ ) ∂n = ˜ θ n . (14)On Γ , w h ( x Γ ; θ ) = ˜ w, ˜ M n ( x Γ ; θ ) = ˜ M n , (15)where ˜ M n ( x Γ ; θ ) can be obtained from Equation 5 by combing w h ( x Γ ; θ ) .On Γ , M n ( x Γ ; θ ) = ˜ M n , ∂M ns ( x Γ ; θ ) ∂s + Q n ( x Γ ; θ ) = ˜ q, (16)where M ns ( x Γ ; θ ) can be obtained from Equation 5 and Q n ( x Γ ; θ ) can be obtainedfrom Equation 7 by combing w h ( x Γ ; θ ) .It should be noted that n , s here refer to the normal and tangent directions alongthe boundaries. As induced physical informed neural network G ( x ; θ ) , M n ( x ; θ ) , M ns ( x ; θ ) , Q n ( x ; θ ) share the same parameters as w h ( x ; θ ) . Considering the gen-erated collocation points in domain and on boundaries, they can all be learned byminimizing the mean square error loss function: L ( θ ) = M SE = M SE G + M SE Γ + M SE Γ + M SE Γ , (17) ith M SE G = 1 N d N d (cid:88) i =1 (cid:13)(cid:13) G ( x Ω ; θ ) (cid:13)(cid:13) = 1 N Ω N Ω (cid:88) i =1 (cid:13)(cid:13) (cid:53) w h ( x Ω ; θ ) − pD (cid:13)(cid:13) ,M SE Γ = 1 N Γ N Γ1 (cid:88) i =1 (cid:13)(cid:13) w h ( x Γ ; θ ) − ˜ w (cid:13)(cid:13) + 1 N Γ N Γ1 (cid:88) i =1 (cid:13)(cid:13)(cid:13) ∂ w h ( x Γ1 ; θ ) ∂n − ˜ θ n (cid:13)(cid:13)(cid:13) ,M SE Γ = 1 N Γ N Γ2 (cid:88) i =1 (cid:13)(cid:13) w h ( x Γ ; θ ) − ˜ w (cid:13)(cid:13) + 1 N Γ N Γ2 (cid:88) i =1 (cid:13)(cid:13) ˜ M n ( x Γ ; θ ) − ˜ M n (cid:13)(cid:13) ,M SE Γ = 1 N Γ N Γ3 (cid:88) i =1 (cid:13)(cid:13) ˜ M n ( x Γ ; θ ) − ˜ M n (cid:13)(cid:13) + 1 N Γ N Γ3 (cid:88) i =1 (cid:13)(cid:13)(cid:13) ∂M ns ( x Γ3 ; θ ) ∂s + Q n ( x Γ ; θ ) − ˜ q (cid:13)(cid:13)(cid:13) , (18)where x Ω ∈ R N , θ ∈ R K is the neural network parameters. If L ( θ ) = 0 , w h ( x ; θ ) is then a solution to transversal deflection. Our goal becomes to find the a set of pa-rameters θ that the approximated deflection w h ( x ; θ ) minimize the loss L ( θ ) . And if L ( θ ) is a very small value, then the approximation w h ( x ; θ ) is very closely satisfyinggoverning equations and boundary conditions, namely w h = arg min θ ∈ R K L ( θ ) (19)Then, the solving of thin plate bending problems by deep collocation method canbe reduced to an optimization problem. In deep learning Tensorflow/Pytorch frame-work, there are a variety available optimizers. One of the most widely used optimiza-tion method can be gradient descent based method is the Adam optimization algo-rithm [60], which is also adopted in the numerical study in this paper. Take a descentstep at collocation point x i with Adam-based learning rates α i , θ i +1 = θ i + α i (cid:53) θ L ( x i ; θ i ) (20)And then the process in Equation 20 is repeated until convergence criterion is satisfied. In this section, several numerical examples on plate bending problems with variousshapes and boundary conditions is studied. And for implementation, a combined op-timizer suggested by Berg et al. in [46] is adopted using L-BFGS optimizer [61] firstand in linear search where BFGS may fail, a Adam optimizer is then applied with avery small learning rate. For all numerical examples, predicted maximum transversewith increasing layers are studied in order to show a convergence of deep collocationmethod in solving the plate bending problem. .1 Simply-supported square plate A simply-supported square plate under a sinusoidal distribution of transverse loadingis studied. The distributed load is given by p = p D sin (cid:0) πxa (cid:1) sin (cid:0) πyb (cid:1) . (21)Here, a , b the length of the plate. D denotes the flexural stiffness of the plate and de-pends on the plate thickness and material properties.The exact solution for this prob-lem is given by w = p π D (cid:16) a + b (cid:17) sin (cid:0) πxa (cid:1) sin (cid:0) πyb (cid:1) . (22)Here, w represents the transverse plate deflection. For this numerical example, we firstgenerate 1000 randomly distributed collocation points in the physical domain depictedin Figure 3. And we thoroughly studied the influence of deep neural network with avarying number of hidden layer and neurons on the maximum deflection at the centreof the plate, which is then shown in Table 1. The numerical results are compared withthe exact solution. It is clear that the results predicted by more hidden layers are moredesirable, especially for neural networks with three hidden layers. To better reflectthe deflection vector in the whole physical domain, the contour plot, contour errorplot of deflection for increasing hidden layers with 50 neurons are shown in Figure 5,Figure 6, Figure 7 Figure 3: Collocation points discretize the square domain.12able 1: Maximum deflection predicted by deep collocation method.
Simply-supported Square Plate Predicted MaximumDeflection Exact Maximumdeflection 1 H X U R Q V S H U K L G G H Q O D \ H U 5 H O D W L Y H H U U R U R I W U D Q V Y H U V D O G H I O H F W L R Q ( x ) 2 Q H K L G G H Q O D \ H U 7 Z R K L G G H Q O D \ H U V 7 K U H H K L G G H Q O D \ H U V Figure 4: The relative error of deflection with varying hidden layers and neurons.13 n Table 1, we employed a varying number of hidden layers from 1 to 4 and in eachlayer and the number of neurons varies from 20 to 60. We calculated the correspondingmaximum transversal deflection at the centre of the square plate. From the L relativeerror of deflection vector at all predicted points is shown in Figure 4 for each case.And it is very clear for even the neural network with only one single hidden layerwith 20 neurons, the results is already very accurate and favourable. For most cases,with increasing neurons and hidden layers, the results converge to the exact solutionand the results are very accurate even with a few neurons and a single hidden layer. InFigure 4, all three hidden layer types get very accurate results. Though the single layerwith 20 neurons is the most accurate in all three types with 20 neurons, the magnitudeof all is × − and the other two results are also very accurate. And as the numberof hidden layer and neurons increases, the relative error curves become flat and obtainresults around exact solutions.From Figure 5, Figure 6, Figure 7, we can observe that the deflection is accuratelypredicted by the deep collocation method, which agree well with the exact solutions.And as the hidden layer number increases, the numerical results converge to the exactsolutions in the whole square plate. The predicted plate deformation agrees well withthe exact deformation. All these lend some credence to the suitable application ofthis deep learning based method. The advantageous of neural networks with hiddenlayers is not conspicuously reflected in this numerical example, as the next numericalexample shows more clearly. w pred (a) Predicted deflection contour w ex w pred (b) Deflection error contour [ \ w p r e d (c) Predicted deflection x y w e x a c t (d) Exact deflection Figure 5: ( a ) Predicted deflection contour ( b ) Deflection error contour ( c ) Predicted de-flection ( d ) Exact deflection of the simply-supported square plate with 1 hidden layers and50 neurons. 15 w pred (a) Predicted deflection contour w ex w pred (b) Deflection error contour [ \ w p r e d (c) Predicted deflection x y w e x a c t (d) Exact deflection Figure 6: ( a ) Predicted deflection contour ( b ) Deflection error contour ( c ) Predicted de-flection ( d ) Exact deflection of the simply-supported square plate with 2 hidden layers and50 neurons. 16 w pred (a) Predicted deflection contour w ex w pred (b) Deflection error contour [ \ w p r e d (c) Predicted deflection x y w e x a c t (d) Exact deflection Figure 7: ( a ) Predicted deflection contour ( b ) Deflection error contour ( c ) Predicted de-flection ( d ) Exact deflection of the simply-supported square plate with 3 hidden layers and50 neurons.
A clamped square plate under a uniformly distributed transverse loading is also an-alyzed with deep collocation method in this section. There is no available explicitform exact solution for deflection of among the whole plate. And to better illustrationthe accuracy of this method, the analytical solution obtained by the Galerkin method eferred in [62] is adopted as a comparison: a a a a = b pD . . . . , (23) w = b qD (cid:26) a (cid:0) − xa (cid:1) (cid:0) − yb (cid:1) (cid:0) xa (cid:1) (cid:0) yb (cid:1) + a (cid:0) − xa (cid:1) (cid:16) yb − y b (cid:17) (cid:0) xa (cid:1) (cid:0) yb (cid:1) (cid:27) + b qD (cid:26) a (cid:16) xa − x a (cid:17) (cid:0) − yb (cid:1) (cid:0) xa (cid:1) (cid:0) yb (cid:1) + a (cid:16) xa − x a (cid:17) (cid:16) yb − y b (cid:17) (cid:0) xa (cid:1) (cid:0) yb (cid:1) (cid:27) . (24)For the maximum transversal deflection at the centre of an isotropic square plate,Ritz method gives the maximum deflection at the centre as w max = 0 . qa D [62],and Timoshenko and Krieger [63] gave a exact solution w max = 0 . qa D .Here, D denotes the flexural stiffness of the plate and depends on the plate thicknessand material properties. a , b the length dimension of the plate. 1000 randomly gen-erated collocation points as in Figure 3 are also used to discritize the clamped squareplate here.For this clamped case, a deep feedforward neural network with increasing layersand neurons is also studied in order to validate the convergence of this scheme. First,the maximum central deflection shown in Table 2 is also calculated for changing layersand neurons and are compared with aforementioned Ritz method, Galerkin methodand exact solution by Timoshenko. It is demonstrated that our deep collocation methodgive most agreeable results with the exact solution. However, for neural networkswith single hidden layer, the results are not that accurate even with 60 neurons. Butas the neuron number increases, the results are indeed more accurate for the neuralnetwork with single hidden layer. This can be observed for the other two hidden layertypes. Additionally, as the number of hidden layer increases, the results are much moreaccurate than the single hidden layer neural network results.The relative error with the analytical solution with different hidden layers and dif-ferent neurons is shown in Figure 8. Although the magnitude of relative error of de-flection for this numerical example is × − , this dose not mean that our deepcollocation method is not that accurate for this problem. For it is mentioned that thedeflection vector as a comparison to calculate the relative error is gained from Galerkinmethod, and we have gotten from Table 2, that our method gives more accurate maxi-mum deflection than Galerkin method. As hidden layers increase, the two flat relativeerror curves nearly coincide and converge to the exact solution. Clamped Square Plate Predicted MaximumDeflection Galerkin method Ritz method Exact solution 1 H X U R Q V S H U K L G G H Q O D \ H U 5 H O D W L Y H H U U R U R I W U D Q V Y H U V D O G H I O H F W L R Q ( x ) 2 Q H K L G G H Q O D \ H U 7 Z R K L G G H Q O D \ H U V 7 K U H H K L G G H Q O D \ H U V Figure 8: The relative error of deflection with varying hidden layers and neurons.19 u comp (a) Predicted deflection contour u ex u comp (b) Deflection error contour [ \ w p r e d (c) Predicted deflection x y w e x a c t (d) Exact deflection Figure 9: ( a ) Predicted deflection contour ( b ) Deflection error contour ( c ) Predicted deflec-tion ( d ) Exact deflection of the clamped square plate with 3 hidden layers and 50 neurons.
Finally, to better depict the favourable of our method, the deflection contour, rela-tive error contour and deformed deflection of the middle surface are also listed for thedeep neural network with three layers and 50 neurons in Figure 9. It is clear that thedeep collocation method yields results agrees well with the analytical solution. .3 Clamped circular plate A clamped circular plate with radius R under a uniform load p is studied here. 1000collocation points shown in Figure 10 are deployed among the circular plate first.Then, we applied deep collocation method to study the deformation of this circularplate. This problem has a exact solution, which can be referred in [63]: w = p (cid:0) R − (cid:0) x + y (cid:1)(cid:1) D , (25)with D denotes the flexural stiffness of the plate and depends on the plate thicknessand material properties.The maximum deflection at the central of the circular plate with varying hiddenlayers and neurons in Table 3 and compared with exact solution. It is obvious that thepredicted maximum deflection is very accurate, and as the neuron and hidden numberincrease, the maximum deflection are more and more close to the exact solution.The relative error for deflection of clamped circular plate with increasing hid-den layers and neurons is shown in Figure 11 in order to show the convergent of thismethod. From this figure, we can get that the as hidden layer number increases, the rel-ative error curves become flat and converge very well to the exact solution. However,all neural networks perform well with a relative error magnitude of × − . Figure 10: Collocation points discretize the circular domain.21able 3: Maximum deflection predicted by deep collocation method.
Clamped
Circular
Plate
Predicted MaximumDeflection Exact solution1 hidden layer, 30 neurons 15.59581 hidden layer, 40 neurons 15.56851 hidden layer, 50 neurons 15.62012 hidden layers, 30 neurons 15.62512 hidden layers, 40 neurons 15.62642 hidden layers, 50 neurons 15.62243 hidden layers, 30 neurons 15.62693 hidden layers, 40 neurons 15.62473 hidden layers, 50 neurons 15.6229 15.6250
Finally, the deformation contour, deflection error contour, predicted and exact de-formation figure are displayed in Figure 12. The deflection of this circular plate agreeswell with the exact solution. The accuracy of this collocation method is again shownhere, which also illustrates that this deep collocation method can be easily and agree-ably applied to simulate deformation of plates of various shapes. 1 H X U R Q V S H U K L G G H Q O D \ H U 5 H O D W L Y H H U U R U R I W U D Q V Y H U V D O G H I O H F W L R Q ( x ) 2 Q H K L G G H Q O D \ H U 7 Z R K L G G H Q O D \ H U V 7 K U H H K L G G H Q O D \ H U V Figure 11: The relative error of deflection with varying hidden layers and neurons.22 w pred (a) Predicted deflection contour w exact w pred (b) Deflection error contour [ \ w p r e d (c) Predicted deflection x y w e x a c t (d) Exact deflection Figure 12: ( a ) Predicted deflection contour ( b ) Deflection error contour ( c ) Predicted de-flection ( d ) Exact deflection of the clamped circular plate with 3 hidden layers and 50neurons.
The simply-supported square plate resting on Winkler foundation is studied in thissection, which assumes that the foundation’s reaction p ( x, y ) can be described by p ( x, y ) = k w , with k a constant called f oundation modulus . Considering a plate n a continuous Winkler foundation, the governing Equation 8 can be written as (cid:53) (cid:0) (cid:53) w (cid:1) = (cid:53) w = ( p − q ) D = ( p − k w ) D (26)The analytical solution for this numerical example is given as [63]: w = 16 pab ∞ (cid:88) m =1 , , , ··· ∞ (cid:88) n =1 , , , ··· sin mπxa sin nπyb mn (cid:20) π D (cid:16) m a + n b (cid:17) + k (cid:21) (27)For this numerical example, the arrangement of collocation points are the same asthat in Figure 3. For the detail implementation, neural networks with different neuronsand deepth are applied in the calculation. Also, maximum deflections shown in Table4 at the central point in all those cases are first studied in order to unveil the accuracyof the deep collocation method.Good agreement can be obsevered in this numerical example as well. From Table4, we can obsevered as hidden layer and neuron number grows, the maximum deflec-tion becomes more accurate and close to the analytical serial solution for even twohidden layers. The relative error shown in Figure 13 better depicts the advantages ofdeep neural network than shallow wide neural network. And with more hidden lay-ers, with neurons increase, the relative error cure becomes flat and very close to zero,which shows that the deep collocation method with only two hidden layers can wellapproximate the deflection.To better illustrate the deflection distribution around the whole plate, deflectioncontour, deflection error contour, deformation contour on deformed figure are shownin Figure 14 and compared with the analytical solution. It is demonstrated that theproposed method agrees well with the analytical solution. Table 4: Maximum deflection predicted by deep collocation method.
Square Plate on Winklerfoundation Predicted MaximumDeflection Exact solution1 hidden layer, 30 neurons 0.339991 hidden layer, 40 neurons 0.356891 hidden layer, 50 neurons 0.321682 hidden layers, 30 neurons 0.322482 hidden layers, 40 neurons 0.321762 hidden layers, 50 neurons 0.321683 hidden layers, 30 neurons 0.322163 hidden layers, 40 neurons 0.321723 hidden layers, 50 neurons 0.32181 0.32137 1 H X U R Q V S H U K L G G H Q O D \ H U 5 H O D W L Y H H U U R U R I W U D Q V Y H U V D O G H I O H F W L R Q ( x ) 2 Q H K L G G H Q O D \ H U 7 Z R K L G G H Q O D \ H U V 7 K U H H K L G G H Q O D \ H U V Figure 13: The relative error of deflection with varying hidden layers and neurons.25 w pred (a) Predicted deflection contour w ex w pred (b) Deflection error contour [ \ w p r e d (c) Predicted deflection x y w e x a c t (d) Exact deflection Figure 14: ( a ) Predicted deflection contour ( b ) Deflection error contour ( c ) Predicted de-flection ( d ) Exact deflection of the simply-supported plate on Winkler foundation with 3hidden layers and 50 neurons.
In this study,study the bending analysis of Kirchhoff plates of various shapes, loadsand boundary conditions. The governing equation of this problem is a fourth order par-tial differential equation (biharmonic equation), which is an important kind of PDEsin engineering mechanics. The proposed deep collocation method is a truly "mesh- ree" method, and can be used to approximate any continuous function, which is verysuitable for the analysis of thin plate bending problems. The deep collocation methodis very simple in implementation, which can be further applied in a wide variety ofengineering problems.Moreover, the deep collocation method with randomly distributed collocations anddeep neural networks perform very well with a MSE loss function minimized by thecombined L-BFGS and Adam optimizer. An accurate result can even be gotten forthe single layer and 20 neurons case. However, as the increase of hidden layers andneurons on each layer, most results become more accurate and converge to the exactand analytical solution. For circular plates, this method become extremely efficientand accurate, and accurate results can be obtained with only a few layers and neurons.More importantly, once those deep neural networks are trained, they can be used toevaluate the solution at any desired points with minimal additional computation time.However, there are still some intriguing issues remained to be studied for the deepneural network based method such as the influence of choosing other neural networktypes, activation functions, loss function forms, weight/bias initialization, and opti-mizers on the accuracy and efficiency of this deep collocation method, which will bestudied in our future research. Acknoledgement : References [1] Eduard Ventsel and Theodor Krauthammer.
Thin plates and shells: theory: anal-ysis, and applications . CRC press, 2001.[2] Klaus-Jürgen Bathe.
Finite element procedures . Klaus-Jurgen Bathe, 2006.[3] Thomas JR Hughes.
The finite element method: linear static and dynamic finiteelement analysis . Courier Corporation, 2012.[4] John T Katsikadelis.
The boundary element method for engineers and scientists:theory and applications . Academic Press, 2016.[5] Carlos Alberto Brebbia and Stephen Walker.
Boundary element techniques inengineering . Elsevier, 2016.[6] Gui-Rong Liu.
Meshfree methods: moving beyond the finite element method .CRC press, 2009.[7] Vinh Phu Nguyen, Cosmin Anitescu, Stéphane PA Bordas, and Timon Rabczuk.Isogeometric analysis: an overview and computer implementation aspects.
Mathematics and Computers in Simulation , 117:89–116, 2015.
8] Hong Zheng, Zhijun Liu, and Xiurun Ge. Numerical manifold space of hermitianform and application to kirchhoff’s thin plate problems.
International Journalfor Numerical Methods in Engineering , 95(9):721–739, 2013.[9] Hongwei Guo and Hong Zheng. The linear analysis of thin shell problems usingthe numerical manifold method.
Thin-Walled Structures , 124:366–383, 2018.[10] Hongwei Guo, Hong Zheng, and Xiaoying Zhuang. Numerical manifold methodfor vibration analysis of kirchhoff’s plates of arbitrary geometry.
Applied Math-ematical Modelling , 66:695–727, 2019.[11] George Cybenko. Approximation by superpositions of a sigmoidal function.
Mathematics of control, signals and systems , 2(4):303–314, 1989.[12] Kurt Hornik. Approximation capabilities of multilayer feedforward networks.
Neural networks , 4(2):251–257, 1991.[13] Rocio Vargas, Amir Mosavi, and Ramon Ruiz. Deep learning: A review. Oct2018.[14] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature ,521(7553):436, 2015.[15] Ali Al-Aradi, Adolfo Correia, Danilo Naiff, Gabriel Jardim, and Yuri Saporito.Solving nonlinear and high-dimensional partial differential equations via deeplearning. arXiv preprint arXiv:1811.08782 , 2018.[16] Josh Patterson and Adam Gibson.
Deep learning: A practitioner’s approach . "O’Reilly Media, Inc.", 2017.[17] Liping Yang, Alan MacEachren, Prasenjit Mitra, and Teresa Onorati. Visually-enabled active deep learning for (geo) text and image classification: a review.
ISPRS International Journal of Geo-Information , 7(2):65, 2018.[18] Zhong-Qiu Zhao, Peng Zheng, Shoutao Xu, and Xindong Wu. Object detec-tion with deep learning: A review.
IEEE transactions on neural networks andlearning systems , 2019.[19] Ali Bou Nassif, Ismail Shahin, Imtinan Attili, Mohammad Azzeh, and KhaledShaalan. Speech recognition using deep neural networks: a systematic review.
IEEE Access , 2019.[20] Tianwei Yue and Haohan Wang. Deep learning for genomics: A conciseoverview. arXiv preprint arXiv:1802.00810 , 2018.[21] Thomas Fischer and Christopher Krauss. Deep learning with long short-termmemory networks for financial market predictions.
European Journal of Oper-ational Research , 270(2):654–669, 2018.
22] Warren S McCulloch and Walter Pitts. A logical calculus of the ideas imma-nent in nervous activity.
The bulletin of mathematical biophysics , 5(4):115–133,1943.[23] Isaac E Lagaris, Aristidis Likas, and Dimitrios I Fotiadis. Artificial neural net-works for solving ordinary and partial differential equations.
IEEE transactionson neural networks , 9(5):987–1000, 1998.[24] Isaac E Lagaris, Aristidis C Likas, and Dimitris G Papageorgiou. Neural-network methods for boundary value problems with irregular boundaries.
IEEETransactions on Neural Networks , 11(5):1041–1049, 2000.[25] Kevin Stanley McFall and James Robert Mahan. Artificial neural networkmethod for solution of boundary value problems with exact satisfaction of arbi-trary boundary conditions.
IEEE Transactions on Neural Networks , 20(8):1221–1233, 2009.[26] Neha Yadav, Anupam Yadav, Manoj Kumar, et al.
An introduction to neuralnetwork methods for differential equations . Springer, 2015.[27] Kyle Mills. Deep learning and the schrödinger equation.
Physical Review A ,96(4), 2017.[28] Weinan E, Jiequn Han, and Arnulf Jentzen. Deep learning-based numericalmethods for high-dimensional parabolic partial differential equations and back-ward stochastic differential equations.
Communications in Mathematics andStatistics , 5(4):349–380, Nov 2017.[29] +Jiequn Han, +Arnulf Jentzen, and +Weinan E. Solving high-dimensional par-tial differential equations using deep learning.
PNAS; Proceedings of the Na-tional Academy of Sciences , 115(34):8505–8510, 2018.[30] Weinan E and Bing Yu. The deep ritz method: A deep learning-based numericalalgorithm for solving variational problems.
Communications in Mathematicsand Statistics , 6(1):1–12, Feb 2018.[31] Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Machine learningof linear differential equations using gaussian processes.
Journal of Computa-tional Physics , 348:683 – 693, 2017.[32] Maziar Raissi and George Em Karniadakis. Hidden physics models: Machinelearning of nonlinear partial differential equations.
Journal of ComputationalPhysics , 357:125 – 141, 2018.[33] M. Raissi, P. Perdikaris, and G. Karniadakis. Numerical gaussian processes fortime-dependent and nonlinear partial differential equations.
SIAM Journal onScientific Computing , 40(1):A172–A198, 2018.
34] Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics informeddeep learning (part i): Data-driven solutions of nonlinear partial differentialequations. 11 2017.[35] Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics informeddeep learning (part ii): Data-driven discovery of nonlinear partial differentialequations. 11 2017.[36] M. Raissi, P. Perdikaris, and G.E. Karniadakis. Physics-informed neural net-works: A deep learning framework for solving forward and inverse problemsinvolving nonlinear partial differential equations.
Journal of ComputationalPhysics , 378:686 – 707, 2019.[37] Maziar Raissi. Deep hidden physics models: Deep learning of nonlinear partialdifferential equations.
J. Mach. Learn. Res. , 19(1):932–955, January 2018.[38] Maziar Raissi. Forward-backward stochastic neural networks: Deep learning ofhigh-dimensional partial differential equations. 04 2018.[39] Christian Beck, Sebastian Becker, Philipp Grohs, Nor Jaafari, and ArnulfJentzen. Solving stochastic differential equations and kolmogorov equations bymeans of deep learning. 06 2018.[40] Christian Beck, Weinan E, and Arnulf Jentzen. Machine learning approximationalgorithms for high-dimensional fully nonlinear partial differential equations andsecond-order backward stochastic differential equations.
Journal of NonlinearScience , Jan 2019.[41] Mohammad Amin Nabian and Hadi Meidani. A deep neural network surrogatefor high-dimensional random partial differential equations. 06 2018.[42] Mohammad Amin Nabian and Hadi Meidani. Physics-informed regularizationof deep neural networks. 10 2018.[43] Alexandre M. Tartakovsky, Carlos Ortiz Marrero, Paris Perdikaris, Guzel D. Tar-takovsky, and David Barajas-Solano. Learning parameters and constitutive rela-tionships with physics informed deep neural networks. 08 2018.[44] Tong Qin, Kailiang Wu, and Dongbin Xiu. Data driven governing equationsapproximation using deep neural networks. 11 2018.[45] Justin Sirignano and Konstantinos Spiliopoulos. Dgm: A deep learning al-gorithm for solving partial differential equations.
Journal of ComputationalPhysics , 375:1339 – 1364, 2018.[46] Jens Berg and Kaj Nyström. A unified deep artificial neural network approach topartial differential equations in complex geometries.
Neurocomputing , 317:28 –41, 2018.
47] Jens Berg and Kaj Nyström. Data-driven discovery of pdes in complex datasets.
Journal of Computational Physics , 384:239 – 252, 2019.[48] Liang Liang, Minliang Liu, Caitlin Martin, John A. Elefteriades, and Wei Sun. Amachine learning approach to investigate the relationship between shape featuresand numerically predicted risk of ascending aortic aneurysm.
Biomechanics andModeling in Mechanobiology , 16(5):1519–1533, Apr 2017.[49] Liang Liang, Minliang Liu, Caitlin Martin, and Wei Sun. A deep learning ap-proach to estimate stress distribution: a fast and accurate surrogate of finite-element analysis.
Journal of The Royal Society Interface , 15(138):20170844,Jan 2018.[50] Seunghye Lee, Jingwan Ha, Mehriniso Zokhirova, Hyeonjoon Moon, and Jae-hong Lee. Background information of deep learning for structural engineering.
Archives of Computational Methods in Engineering , 25(1):121–129, Jul 2017.[51] Qingguo Wang, Geng Zhang, Chenchen Sun, and Nan Wu. High efficient loadpaths analysis with u* index generated by deep learning.
Computer Methods inApplied Mechanics and Engineering , 344:499 – 511, 2019.[52] Ken-Ichi Funahashi. On the approximate realization of continuous mappings byneural networks.
Neural Networks , 2(3):183 – 192, 1989.[53] Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforwardnetworks are universal approximators.
Neural Networks , 2(5):359 – 366, 1989.[54] Soufiane Hayou, Arnaud Doucet, and Judith Rousseau. On the selection of ini-tialization and activation function for deep neural networks. 05 2018.[55] Michael A. Nielsen. Neural networks and deep learning, 2018.[56] Katarzyna Janocha and Wojciech Marian Czarnecki. On loss functions for deepneural networks in classification.
Schedae Informaticae , 1/2016, 2017.[57] Ali Al-Aradi, Adolfo Correia, Danilo Naiff, Gabriel Jardim, and Yuri Saporito.Solving nonlinear and high-dimensional partial differential equations via deeplearning. 11 2018.[58] Satya N Atluri.
Methods of computer modeling in engineering & the sciences ,volume 1. Tech Science Press Palmdale, 2005.[59] Pulkit Agrawal. Collocation based approach for training recurrent neural net-works.[60] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization.
CoRR , abs/1412.6980, 2015.[61] Dong C. Liu and Jorge Nocedal. On the limited memory bfgs method for largescale optimization.
Mathematical Programming , 45(1-3):503–528, Aug 1989.
62] Y. Khan, P. Tiwari, and R. Ali. Application of variational methods to a rect-angular clamped plate problem.
Computers & Mathematics with Applications ,63(4):862 – 869, 2012.[63] Stephen P Timoshenko and Sergius Woinowsky-Krieger.
Theory of plates andshells . McGraw-hill, 1959.. McGraw-hill, 1959.