Efficient Cloth Simulation using Miniature Cloth and Upscaling Deep Neural Networks
EEfficient Cloth Simulation using Miniature Cloth and Upscaling DeepNeural Networks
TAE MIN LEE, YOUNG JIN OH, and IN-KWON LEE,
Yonsei University
Cloth simulation requires a fast and stable method for interactively andrealistically visualizing fabric materials using computer graphics. We proposean efficient cloth simulation method using miniature cloth simulation andupscaling Deep Neural Networks (DNN). The upscaling DNNs generatethe target cloth simulation from the results of physically-based simulationsof a miniature cloth that has similar physical properties to those of thetarget cloth. We have verified the utility of the proposed method throughexperiments, and the results demonstrate that it is possible to generate fastand stable cloth simulations under various conditions.CCS Concepts: •
Computing methodologies → Physical simulation .Additional Key Words and Phrases: Cloth simulation, physically-based sim-ulation, physically-based animation, cloth animation, neural network, deeplearning
ACM Reference Format:
Tae Min Lee, Young Jin Oh, and In-Kwon Lee. 2019. Efficient Cloth Simulationusing Miniature Cloth and Upscaling Deep Neural Networks.
ACM Trans.Graph.
1, 1 (July 2019), 10 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn
Cloth simulation is an essential technology that has been used tomodel various fabric materials in the field of computer graphics,such as movies and games. Therefore, many methods for simulat-ing cloth naturally and realistically have been studied [Baraff andWitkin 1998; Choi and Ko 2005b; Kaldor et al. 2008; Terzopoulos et al.1987; Weidner et al. 2018]. In addition, because real-time interactivegraphics content is required in research areas such as augmentedand virtual reality applications, studies of efficient cloth simulationmethods have been continuously carried out [Bender et al. 2013; Ohet al. 2008; Vassilev et al. 2001].Recently, in the field of physically-based simulations aside fromcloth simulation, studies using Deep Neural Networks (DNN) havebeen actively carried out in an effort to reduce the long computationtimes or improve the simulation quality. For example, the DNNinference is used to replace some of the complex computations inthe physically-based simulation with the inference process of a neu-ral network [Tompson et al. 2016; Yang et al. 2016] or to generatefine-grained simulations based on coarse-grained simulations [Chuand Thuerey 2017; Xie et al. 2018]. However, most studies usingDNN for physically-based simulation have focused on fluid simu-lation. Because it is difficult to apply previous methods directly tocloth simulation, further investigation of the DNN model for clothsimulation is necessary.
Authors’ address: Tae Min Lee; Young Jin Oh; In-Kwon Lee, Yonsei University, ComputerScience, Seoul, {dnflxoals,skrcjstk}@gmail.com, [email protected].© 2019 Association for Computing Machinery.This is the author’s version of the work. It is posted here for your personal use. Not forredistribution. The definitive Version of Record was published in
ACM Transactions onGraphics , https://doi.org/10.1145/nnnnnnn.nnnnnnn.
To perform efficient cloth simulation using DNN, we must use anetwork model with low computational cost. We could use Convolu-tional Neural Networks (CNN), which are frequently used with highaccuracy and quality in the field of computer vision [Krizhevskyet al. 2012; Radford et al. 2015; Simonyan and Zisserman 2014; Zhuet al. 2017], for cloth simulation. However, because these neural net-work models have high computational costs arising from the largenumber of iterative convolution calculations, it is challenging tosimulate cloth in a more efficient manner than that of conventionalphysically-based simulation.Even if the cloth simulation is computed efficiently, the resultshould be of a similar quality to that generated from existing physically-based simulation methods. The recently proposed H-DNN (hierarchi-cal cloth simulation using DNN) method [Oh et al. 2018] constructsa hierarchy of cloth particles and performs a physical simulation atthe coarsest level. It is then possible to compute a finer-level clothsimulation through DNN inference (see Figure 1(a)). Although thismethod results in efficient computation times, the quality of thesimulations is much lower than that of existing physically-basedsimulations, mainly because the physical characteristics of the coars-est cloth model (e.g., the rest length between the cloth particles) arenot the same as those of the close in the finest model. [Feng et al.2010; Kavan et al. 2011].The cloth simulation method proposed in this paper efficientlygenerates results using a miniature cloth and upscaling DNN. Theminiature cloth is a down-sampled and down-scaled version ofthe target cloth that we want to simulate. The miniature cloth issimulated by the conventional physically-based simulation method.Based on the results of the miniature cloth simulation, the upscalingDNN yields the target cloth simulation (see Figure 1(b)). This issimilar to cinematographic methods, where miniature environmentsare created for filming by reducing the target-scale objects when itis difficult or expensive to film a full-scale environment [Fielding2013]. Just as the results from filming a miniature environment areexpressed in the target-scale via post-processing, we use upscalingDNN inference to generate a simulation of the target cloth basedon the simulation of the miniature cloth.To obtain results from the proposed method that are similar tothose from the conventional physically-based simulation, we usea miniature cloth that maintains the same rest length as the targetcloth. Because the miniature cloth simulation shows similar move-ments to the target cloth simulation, high-frequency details that aresimilar to the target cloth simulation are sufficiently preserved. Inaddition, we can simulate a miniature cloth using fewer particlesthan in the target cloth, and the upscaling DNN can be a lightweightneural network with low computational cost, allowing for moreefficient simulation than conventional methods.This paper is organized as follows. Section 2 discusses worksrelated to cloth simulation and physically-based simulation withDNN. In Section 3, we present the training method and architecture
ACM Trans. Graph., Vol. 1, No. 1, Article . Publication date: July 2019. a r X i v : . [ c s . G R ] J u l • Lee, T. et al. (a) Hierarchical cloth model (b) Miniature cloth modelFig. 1. Structure difference between hierarchical cloth in H-DNN [Oh et al. 2018] and our miniature cloth model: (a) hierarchical cloth model and (b) miniaturecloth model. of the upscaling DNN. In Section 4, we present the process for targetcloth simulation. In Section 5, we discuss the performance of theproposed method based on the obtained results. Finally, Section 6presents our conclusions and discusses possible future work. Physically-based simulation methods for realistically expressing themovement of elastic materials have also been extensively studiedfor cloth simulation [Choi and Ko 2005a; Terzopoulos et al. 1987].Breen et al. [Breen et al. 1994] proposed a mass-spring system forcloth simulation, and Provoet et al. [Provot et al. 1995] proposed acloth simulation method using explicit Euler integration. Cloth sim-ulation using explicit Euler integration is able to generate real-timeinteractive results because rapid calculation is possible, althoughthis method did not produce stable results when simulated with alarge time step. Therefore, a method using implicit Euler integrationwas proposed that generates stable results even when simulatedwith a large time step [Baraff and Witkin 1998]. To accelerate theimplicit Euler integration, which has a high computational cost, Liuet al. [Liu et al. 2013] proposed a method using a solver based on theblock coordinate descent scheme. Projective dynamics proposed byBouaziz et al. [Bouaziz et al. 2014] extended the previous techniques,which were limited to the mass-spring system, allowing their ap-plication to various constrained dynamics problems. To solve theoptimization problems in projective dynamics quickly, a methodusing the quasi-Newton optimization algorithm [Liu et al. 2017]and another method applying Anderson acceleration [Peng et al.2018] have been proposed. Overby et al. [Overby et al. 2017] appliedthe Alternating Direction Method of Multipliers (ADMM) to solvethe optimization problems in projective dynamics quickly and gen-erate stable results, even when various constraints were changeddynamically.
In recent years, studies applying DNN, which has excellent per-formance in approximating complex computation processes, tophysically-based simulation have been actively carried out. Amongthem, methods using DNN to train the physical movements of sim-ple rigid or elastic bodies and to predict future movements directlyhave been proposed. A neural physics engine [Chang et al. 2016]was introduced to train intuitive physics based on a real scene andthe past states of a rigid body, then generating a physical motionsequence. Visual interaction networks [Watters et al. 2017] wereproposed to predict the next positions of an elastic body basedon a training process using a physically-based simulation video.These studies are limited to simple structures and predict futuremovements, but it is difficult to apply these methods to predict thechanges in rigid or elastic bodies with complex structures.Unlike methods in which DNN directly predicts future move-ments, methods using DNN to facilitate faster computation forexisting physically-based simulation methods have also been pro-posed. Tompson et al. [Tompson et al. 2016] proposed a methodfor replacing the pressure projection step, which has the highestcomputational cost, with DNN inference for fast fluid simulationcalculations. Wiewel et al. [Wiewel et al. 2018] proposed a longshort-term memory-based method for predicting changes in fluidpressure fields. Luo et al. [Luo et al. 2018] suggested a method forwarping linear elastic simulations into nonlinear elastic simulationsvia a DNN inference.Additionally, methods for using DNN inference combined withlow-resolution results from physically-based simulations have beenproposed to reduce the computational cost of generating high-resolution simulations. In the field of fluid simulation, Chu andThuerey [Chu and Thuerey 2017] proposed a CNN-based methodto obtain low-resolution simulations for the generation of high-resolution results. Xie et al. [Xie et al. 2018] proposed a techniquefor converting low-resolution grid simulations into high-resolution
ACM Trans. Graph., Vol. 1, No. 1, Article . Publication date: July 2019. fficient Cloth Simulation using Miniature Cloth and Upscaling Deep Neural Networks • 3
Fig. 2. Process of creating a miniature cloth system. results by adding a temporal coherence discriminator to a super-resolution generative adversarial networks [Ledig et al. 2017]. Inthe field of cloth simulation, Oh et al. [Oh et al. 2018] proposed amethod for calculating the positions of particles at the coarsest-levelusing physically-based simulation and then predicting the positionsof finer-level particles via DNN inference. This method (we call thismethod H-DNN) uses the coarsest-level simulation with a smallnumber of particles and a lightweight neural network for greatersimulation efficiency compared to that of conventional physically-based simulation methods. However, because the physical propertiesof the coarsest-level and target cloth are different, even if a finer-level is predicted, the results show that the high-frequency detailsare not sufficiently preserved.
The upscaling DNN generates the target cloth simulation based onthe simulation of a miniature cloth that is obtained via the down-sampling and down-scaling (DSDS) of the target cloth. To obtaintraining data for the upscaling DNN, target cloth simulations arecarried out via conventional physically-based simulation method,and the results of miniature cloth simulation are generated via down-sampling and down-scaling. We designed and used lightweightneural networks that have a low computational cost to simulate thetarget cloth efficiently.
Figure 2 shows the process of creating a miniature cloth system,which is obtained by applying a down-sampling and down-scaling(DSDS) step with a factor of two to the target cloth system. In thedown-sampling process, a 3 × × Table 1. The number of particles in the miniature cloth and target cloth ineach example. The number of particles in the miniature cloth is approxi-mately / of that in the target cloth. Example
The input and output feature vectors of the upscaling DNN are de-fined as the positions of the particles in the patches of the miniatureand target cloths, respectively. H-DNN [Oh et al. 2018] uses the posi-tions of the particles in a single frame as feature values. This makesit difficult for DNN to preserve the time coherence between consecu-tive frames naturally. In contrast, our method uses the concatenatedpositions of three consecutive frames, including the current and theprevious two frames, as input feature vectors. This allows DNN togenerate more natural results by taking into account the smoothtransition between the neighboring frames. In addition, using theparticle positions of consecutive frames in an input vector has theeffect of reflecting the moving particles’ velocity on the input. Wehave tested several combinations of various features, for example,including the velocity of particles as features. Finally, we found thatthis simple method of combining the particle positions of severalframes was the most effective (see Section 5.1).Figure 3 illustrates the input and output feature vectors of theproposed DNN model. Consider the positions of four particles x p , x q , x r , and x s in a 2 × s is defined as follows: s = [ x p , x q , x r , x s ] . (1)The input feature vector δδδ t at the simulation time t is defined as δδδ t = [ s t − , s t − , s t ] , (2)which concatenates the state vectors of three consecutive simulationresults at time t − t −
1, and t . The output feature vector o t at thesimulation time t is defined as o t = (cid:2) ˆx o , ˆx p , ˆx q , ˆx r , ˆx s , ˆx t , ˆx u , ˆx v , ˆx w (cid:3) , (3)which represents the positions of the nine particles in a 3 × × Figure 4 shows the structure of the upscaling DNN model used in theproposed method. We designed a lightweight network consisting offour Fully Connected (FC) layers with a Rectified Linear Unit (ReLU)function as the activation function for fast inference. The first threeFC layers calculate intermediate vectors of size 64, and the final FClayer calculates the output feature vectors. The size of the input
ACM Trans. Graph., Vol. 1, No. 1, Article . Publication date: July 2019. • Lee, T. et al.
Fig. 3. Input and output feature vectors of the upscaling DNN.Fig. 4. Upscaling Deep Neural Networks model.Table 2. The number of input feature and ground-truth output pairs col-lected in each example and the corresponding number of frames.
Example x , y , z position of thethree frames of the particle patch in the miniature cloth. The sizeof the output feature vectors is 27 because it represents the x , y , z position for one frame of the corresponding particle patch in thetarget cloth.We define the loss function as the Mean Squared Error (MSE) ofthe corresponding ground-truth output vector g and output vector o computed by inference as follows: Loss = N N (cid:213) n = (cid:213) i = ( o i − g i ) , (4)where i represents the indices of the 3 × N is the total number of inferences made by theDNN model. The DNN model was trained by using the adaptivemoment estimation optimization method [Kingma and Ba 2014]until the loss plateaued. We set the learning rate to 0.0001 and batchsize to 50,000.Table 2 shows the amount of training data that comprised theinput features and ground-truth output pairs collected to train theupscaling DNN in each example and the corresponding numberof frames. As shown in the table, training data was collected formore than four million pairs in all three examples. To collect thetraining data, we carried out target cloth simulations with P-ADMM(projective dynamics applying the ADMM) [Overby et al. 2017]. Thetraining data contain results from various simulations where themass value, the spring constant, and the time step were varied. Theupscaling DNN was trained separately for each example, and thetraining time was approximately eight hours, respectively. Our cloth simulation system uses the results of miniature cloth sim-ulation and an upscaling DNN to simulate the target cloth efficiently.An overview of our system is presented in Figure 5. First, the minia-ture cloth system is created by down-sampling and down-scalingoperations on the target cloth system (as in the data generationprocess). Then, physically-based simulations are performed on theminiature cloth system. Finally, the results of target cloth simulation
ACM Trans. Graph., Vol. 1, No. 1, Article . Publication date: July 2019. fficient Cloth Simulation using Miniature Cloth and Upscaling Deep Neural Networks • 5
Fig. 5. Target cloth simulation generation. are generated by the upscaling DNN inference, which takes the re-sults of miniature cloth simulations as input feature vectors. In thisprocess, because the miniature cloth system is a down-scaled ver-sion of the target cloth system, the external forces in the miniaturecloth system should also be down-scaled.
Consider a mass-spring system with positions x , velocities v , a mass-matrix M , and time step h . The next positions of the target cloth x t + are calculated by using the following implicit integration equations: x t + − ( x t + h v t ) = M − h ( f I ( x t + ) + f E ) , (5)where f I represents the internal forces of the cloth model and f E represents the external forces [Liu et al. 2013]. Because the miniaturecloth system is a version of the target cloth system down-scaled bya factor of two, x t = x ′ t and h v t = x t − x t − = h v ′ t , where x ′ t and v ′ t are the current positions and the velocities in the miniaturecloth system, respectively. Based on (5), we can express the updaterule for the miniature cloth system as follows:2 x ′ t + − ( x ′ t + h v ′ t ) = M − h (cid:0) f I ( x ′ t + ) + f E (cid:1) . (6)Assuming that each spring follows Hooke’s law [Halliday et al.2013], the internal force can be expressed as f I ( x ′ t + ) = k ∆ L = f I ( x ′ t + ) , (7)where k is the spring constant and ∆ L is the spring displacement.By substituting (7) into (6), we can derive the final updated rule forminiature cloth simulation as follows: x ′ t + − ( x ′ t + h v ′ t ) = M − h (cid:18) f I ( x ′ t + ) + f E (cid:19) , (8) which indicates that the external force of the miniature cloth systemshould be set to 1 / Upscaling DNN infers the positions of the particles at frame t of thetarget cloth using the positions of the particles at frames t , t − t − Our DNN model was configured and trained using Tensorflow [Abadiet al. 2015] on a PC with a quad-core Intel Core i7-3770 CPU witheight threads, 32 GB of RAM, and an NVIDIA GTX 970 GPU. Weused GPU acceleration for the training process. However, becausethe P-ADMM and H-DNN methods used to compare performancedo not use GPU acceleration, we did not use GPU acceleration inthe testing of our approach; thus, we made the comparison un-der the same conditions. In all three methods, parallel processesbased on OpenMP [Dagum and Menon 1998] were used for the testprocesses. In addition, we constructed the inference machine bymoving the weights and bias values of the neural network modeltrained in Python-based Tensorflow to the C++-based deep learn-ing framework tiny-dnn [Tin 2017]. This shortened the inferencetime and allowed simulations in conjunction with the P-ADMMlibrary [Overby et al. 2017], which is written in C++.
ACM Trans. Graph., Vol. 1, No. 1, Article . Publication date: July 2019. • Lee, T. et al. (a) Curtain (b) Flag(c) CollisionFig. 6. Results of target cloth simulation achieved by upscaling DNN inference based on miniature cloth simulation: (a) curtain, (b) flag, and (c) collision. (Left)Miniature cloth. (Right) Target cloth.Fig. 7. Miniature cloth created with different DSDS factors. From left toright: target cloth and DSDS factors 2, 3, and 4.Fig. 8. Target cloth generated with different DSDS factors. From left toright: DSDS factors 2, 3, and 4.
Figure 6 shows the results of miniature cloth simulation calculatedusing P-ADMM and the results of target cloth simulation generatedby the upscaling DNN inference. The red ones are the miniaturecloth, and the blue ones are the target cloth. The proposed methodgenerated stable target cloth simulations under various conditionsin all three examples (curtain, flag, and collision).
We compared the performance of the three different cases by chang-ing only the input features of the proposed DNN structure to identify
Table 3. Training and test MSE according to the input features of the upscal-ing DNN: the positions of the particles in the current frame, the positionsand velocities of the particles in the current frame, and the positions of theparticles in three consecutive frames (from top to bottom).
Features Training error Test errorPositions 9.094e-05 2.85e-04Positions + Velocities 8.876e-05 2.778e-04Positions in 3 frames 8.856e-05 2.691e-04the appropriate input feature combination for the upscaling DNN.Three DNN models were trained with three types of feature com-binations: the location of the particles in the current frame of aminiature patch, the location and velocity of the particles in thecurrent frame, and the locations of the particles in three consecutiveframes. The training and test data for the three cases were preparedwith 834,861 input data and 122,388 output data, respectively, inidentical simulations of the flag example. Table 3 shows the trainingand test errors for three different input combinations. The error isthe value of the loss function in Equation (4). Training is stable inall three cases, but, when using three consecutive frames of data asan input feature, the test error is lower than the other two cases.
We investigated how the performance of the proposed method wasaffected by increasing DSDS factor (2, 3, and 4) used to construct theminiature model from the original model (see Figure 7). Consideringthe relationship between the miniature and target cloth systemsshown in Section 4.1, the external forces in the miniature cloth
ACM Trans. Graph., Vol. 1, No. 1, Article . Publication date: July 2019. fficient Cloth Simulation using Miniature Cloth and Upscaling Deep Neural Networks • 7 (a) Bilinear (b) Biquadratic (c) Bicubic (d) Upscaling DNNFig. 9. Generated results according to upscaling methods: (a) bilinear interpolation, (b) biquadratic interpolation, (c) bicubic interpolation, and (d) upscalingDNN.Table 4. Euclidean distance error per particle and computation time perframe (msec) according to different DSDS factors.
DSDS factor Euclidean error Time per frame (ms)2 1.816e-03 37.53 2.916e-03 19.14 3.901e-03 16.2system should also be down-scaled. That is, with DSDS factors 3and 4, external forces were set to 1/3 and 1/4 of the external forcesgiven in the target system, respectively. For DSDS 3 and 4, morethan four million pairs of data were collected, and 80% and 20% ofthe data were used in training and testing, respectively, in the case ofDSDS factor 2. As shown in Table 4, the larger the factor, the betterthe time performance because of the decreasing inference time,but the per particle Euclidean distance error between the inferredposition and the ground-truth output position increased. The resultsgenerated by the inference of the trained upscaling DNN under eachDSDS factor are shown in Figure 8. As shown in the figure, as theDSDS factor increased, the quality of the results decreased.
We compared the performance of various DNN models to determinethe most suitable structure of the upscaling DNN in the proposedmethod. Using the dataset constructed in Section 3, we used 80%as training data and 20% as test data in each example, respectively.Table 5 shows the test error of various DNN models, where U*FC-Vmeans the DNN has U hidden layers with V nodes at each layer; forexample, the 4*FC-64 model is the DNN structure with four hiddenlayers and 64 nodes at each layer that is proposed in Section 3.3.Compared to the 5*FC-64 model with an increased number of layersand the 3*FC-64 model with a decreased number of layers, 4*FC-64showed the lowest test error when trained with the same training
Table 5. Test error of DNN models with different structures. U*FC-V meansthe DNN has U hidden layers with V nodes at each layer.
DNN model Curtain Flag Collision4*FC-32 9.594e-06 1.674e-04 1.66e-053*FC-64 6.405e-06 1.276e-04 1.357e-054*FC-64 6.214e-06 1.217e-04 1.17e-055*FC-64 7.158e-06 1.677e-04 1.454e-054*FC-128 6.772e-06 1.486e-04 1.414e-05data. In addition, when compared with the 4*FC-128 and 4*FC-32 models with increased and decreased, respectively, numbers ofnodes per layer, 4*FC-64 also showed the lowest test error.
We also compared the performance when using conventional in-terpolation methods and DNN for the upscaling process. Bilinear,biquadratic, and bicubic methods [Press et al. 2007] were used forthe interpolation. As shown in Table 6, the use of interpolation meth-ods improved the computation time performance but increased theEuclidean distance error slightly. Furthermore, the interpolationmethods generated undesirable effects when expressing detailedwrinkles, as can be seen in the large wrinkles starting from the upperleft part of the cloth (see Figure 9). When using interpolation-basedmethods, it is difficult to maintain a topology that considers therelationship between particles in the wrinkle portion where self-intersection occurs. However, the DNN model can produce similarresults to P-ADMM because training maintains a relatively goodtopology between particles in the wrinkle portion.
We compared the results obtained using our proposed method withthose from the existing method for verification. The results of thesimulation of each example using P-ADMM, H-DNN [Oh et al. 2018],and the proposed method are shown in Figure 10, 11, 12. All three
ACM Trans. Graph., Vol. 1, No. 1, Article . Publication date: July 2019. • Lee, T. et al.
Table 6. Euclidean distance error (per particle) and computation time perframe (msec) according to the upscaling method.
Upscaling Euclidean error Time per frame (ms)Bilinear 1.826e-03 17.7Biquadratic 1.825e-03 17.7Bicubic 1.825e-03 17.8DNN 1.816e-03 37.5
Fig. 10. Results of simulation of the curtain example. From left to right:P-ADMM, H-DNN, and the proposed method.Fig. 11. Results of simulation of the flag example. From left to right: P-ADMM, H-DNN, and the proposed method. methods produced stable simulations. However, the results of theproposed method are more similar to those of P-ADMM than thoseobtained using H-DNN. As shown in Figure 10, the proposed methodis more similar to P-ADMM than H-DNN in the upper left part of thegenerated cloth. In particular, the proposed method preserves thehigh-frequency details of P-ADMM better than H-DNN in the colli-sion parts of the cloth (see also Figure 12). In particular, in the flagexample (see Figure 11) with applied wind force, the use of H-DNNresulted in a significantly different movement from that obtainedusing P-ADMM. This is because the wind force is modeled as afunction using the triangle area, the normal, and the tangent vectorsof the cloth model [Wejchert and Haumann 1991]. H-DNN doesnot preserve the triangle area of the target cloth, so the wind forceis computed incorrectly. However, because the proposed methodpreserves the size of the target cloth’s triangles, the results aremore similar to those generated by the P-ADMM, and the detailedhigh-frequency movement is better represented.The proposed method even produces stable results when thephysical properties of the cloth are altered, such as the mass of aparticle and the spring constant. Figure 13 and Figure 14 show theresults of simulation using P-ADMM, H-DNN, and the proposedmethod when the mass and spring constant are altered, respectively.The proposed method always generates stable results, even whenthe mass and spring constant are changed. The proposed method
Fig. 12. Results of simulation of the collision example. From left to right:P-ADMM, H-DNN, and the proposed method.(a) P-ADMM(b) H-DNN(c) Proposed MethodFig. 13. Results of simulation when the mass value is changed: (a) P-ADMM,(b) H-DNN, and (c) the proposed method. Mass values from left to right:1.5, 3.0, 4.5.Table 7. Per particle Euclidean distance error with P-ADMM result.
Example H-DNN Proposed methodCurtain 1.841e-03 1.816e-03Flag 2.492e-02 7.767e-03Collision 5.982e-03 5.973e-03also preserves more high-frequency details compared to H-DNN(see Figure 13 and Figure 14).We compared the per particle Euclidean distance error to verifythat the results of the proposed method are more similar to theresults of P-ADMM compared to the results of H-DNN. Table 7shows the per particle Euclidean distance error with P-ADMM resultin each example. Because the proposed method generates resultswith much better preservation of the high-frequency details, in allexamples, it has a lower error than H-DNN. In particular, in the flagexample, the proposed method shows much less error than H-DNNbecause the results of the proposed method are stretched to a similarextent as the results of the P-ADMM.
ACM Trans. Graph., Vol. 1, No. 1, Article . Publication date: July 2019. fficient Cloth Simulation using Miniature Cloth and Upscaling Deep Neural Networks • 9
Table 8. Comparison of computation time per frame (msec) for P-ADMM, H-DNN, and the proposed method. The curtain and flag examples were renderedwith a time step of 40 msec and the collision example was rendered with a time step of 20 msec.
Example (a) P-ADMM(b) H-DNN(c) Proposed MethodFig. 14. Results of simulation when the spring constant is changed: (a) P-ADMM, (b) H-DNN, and (c) the proposed method. Spring constants fromleft to right: 40, 70, 150.
We compared the time performance to verify that the proposedmethod generates results efficiently. Table 8 lists the mean timesrequired to generate one frame of simulation for P-ADMM, H-DNN,and the proposed method. For the simulation of a cloth containingthe same number of particles, the proposed method generates resultsmore efficiently than the other two methods. This is because theproposed method uses a small number of particles and a lightweightnetwork model. Additionally, the inference process of the proposedmethod is performed in patch units, as compared to H-DNN’s trian-gle units. Therefore, the number of inferences is smaller, resultingin more efficient computation time.
We have proposed an efficient cloth simulation method using minia-ture cloth simulation and upscaling DNN. The proposed methodperforms physically-based simulation in a miniature cloth systemcreated by applying down-sampling and down-scaling operations tothe target cloth. Using the results, the upscaling DNN generates thetarget cloth simulation. Because this method uses a miniature cloth
Fig. 15. Failure case for the proposed method. These are the results of asimulation with a wind direction opposite to the wind direction in thetraining data. with a small number of particles and a lightweight network model,efficient target cloth simulation is possible. In addition, the minia-ture cloth retains the physical properties of the target cloth. In otherwords, high-frequency details are preserved better than when usingH-DNN because the results are generated using miniature cloth sim-ulations that move in a similar way to the target cloth simulation.We have demonstrated experimentally that the proposed methodcan generate fast and stable simulations under various conditions.Figure 15 presents the results generated by the proposed methodwhen simulating a wind direction that is opposite to the wind direc-tion in the training data. As shown in the figure, unwanted wrinklesare formed on the cloth as a result of incorrect inference by theDNN. The proposed method has a limitation in that it is difficultto make accurate inferences about situations that do not exist inthe training data. This is a common limitation of learning-basedmethods that is typically solved by using more training data. There-fore, it is essential to construct training data that is suitable fora wide variety of situations. We plan to generalize the proposedmethod to other physically-based simulation fields, such as fluidor soft-body simulation. Recently, novel network structures withhigher accuracy compared to plain networks have been introducedin the image classification field [He et al. 2016; Huang et al. 2017].We are considering using these novel network structures to con-struct a better model that reduces the differences from the resultsof ground-truth simulation.
REFERENCES
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro,Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat,Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, RafalJozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, RajatMonga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens,Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, VijayVasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, MartinWicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale MachineLearning on Heterogeneous Systems. Software available from tensorflow.org.David Baraff and Andrew Witkin. 1998. Large Steps in Cloth Simulation. In
Proceedingsof the 25th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’98) . ACM, New York, NY, USA, 43–54. https://doi.org/10.1145/280814.280821Jan Bender, Daniel Weber, and Raphael Diziol. 2013. Fast and stable cloth simulationbased on multi-resolution shape matching.
Computers & Graphics
37, 8 (2013), 945 –954. https://doi.org/10.1016/j.cag.2013.08.003Sofien Bouaziz, Sebastian Martin, Tiantian Liu, Ladislav Kavan, and Mark Pauly. 2014.Projective Dynamics: Fusing Constraint Projections for Fast Simulation.
ACM Trans.Graph.
33, 4, Article 154 (July 2014), 11 pages. https://doi.org/10.1145/2601097.2601116David E. Breen, Donald H. House, and Michael J. Wozny. 1994. Predicting the Drape ofWoven Cloth Using Interacting Particles. In
Proceedings of the 21st Annual Conferenceon Computer Graphics and Interactive Techniques (SIGGRAPH ’94) . ACM, New York,NY, USA, 365–372. https://doi.org/10.1145/192161.192259Michael B Chang, Tomer Ullman, Antonio Torralba, and Joshua B Tenenbaum. 2016. Acompositional object-based approach to learning physical dynamics. arXiv preprintarXiv:1612.00341 (2016).Kwang-Jin Choi and Hyeong-Seok Ko. 2005a. Research problems in clothing simulation.
Computer-aided design
37, 6 (2005), 585–592.Kwang-Jin Choi and Hyeong-Seok Ko. 2005b. Stable but Responsive Cloth. In
ACMSIGGRAPH 2005 Courses (SIGGRAPH ’05) . ACM, New York, NY, USA, Article 1.https://doi.org/10.1145/1198555.1198571Mengyu Chu and Nils Thuerey. 2017. Data-driven Synthesis of Smoke Flows withCNN-based Feature Descriptors.
ACM Trans. Graph.
36, 4, Article 69 (July 2017),14 pages. https://doi.org/10.1145/3072959.3073643Leonardo Dagum and Ramesh Menon. 1998. OpenMP: An Industry-Standard API forShared-Memory Programming.
IEEE Comput. Sci. Eng.
5, 1 (Jan. 1998), 46–55.Wei-Wen Feng, Yizhou Yu, and Byung-Uck Kim. 2010. A Deformation Transformerfor Real-time Cloth Animation.
ACM Trans. Graph.
29, 4, Article 108 (July 2010),9 pages. https://doi.org/10.1145/1778765.1778845Raymond Fielding. 2013.
Techniques of Special Effects of Cinematography . Focal Press.David Halliday, Robert Resnick, and Jearl Walker. 2013.
Principles of Physics (10th ed.).Wiley.K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep Residual Learning for Image Recognition.In . 770–778.https://doi.org/10.1109/CVPR.2016.90G. Huang, Z. Liu, L. v. d. Maaten, and K. Q. Weinberger. 2017. Densely ConnectedConvolutional Networks. In . 2261–2269. https://doi.org/10.1109/CVPR.2017.243Jonathan M. Kaldor, Doug L. James, and Steve Marschner. 2008. Simulating KnittedCloth at the Yarn Level.
ACM Trans. Graph.
27, 3, Article 65 (Aug. 2008), 9 pages.https://doi.org/10.1145/1360612.1360664Ladislav Kavan, Dan Gerszewski, Adam W. Bargteil, and Peter-Pike Sloan. 2011. Physics-inspired Upsampling for Cloth Simulation in Games.
ACM Trans. Graph.
30, 4, Article93 (July 2011), 10 pages. https://doi.org/10.1145/2010324.1964988Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. ImageNet Classifica-tion with Deep Convolutional Neural Networks. In
Advances in Neural Informa-tion Processing Systems 25 , F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Wein-berger (Eds.). Curran Associates, Inc., 1097–1105. http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdfC. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani,J. Totz, Z. Wang, and W. Shi. 2017. Photo-Realistic Single Image Super-ResolutionUsing a Generative Adversarial Network. In . 105–114. https://doi.org/10.1109/CVPR.2017.19Tiantian Liu, Adam W. Bargteil, James F. O’Brien, and Ladislav Kavan. 2013. FastSimulation of Mass-spring Systems.
ACM Trans. Graph.
32, 6, Article 214 (Nov.2013), 7 pages. https://doi.org/10.1145/2508363.2508406Tiantian Liu, Sofien Bouaziz, and Ladislav Kavan. 2017. Quasi-Newton Methods forReal-Time Simulation of Hyperelastic Materials.
ACM Trans. Graph.
36, 3, Article116a (May 2017). https://doi.org/10.1145/2990496Ran Luo, Tianjia Shao, Huamin Wang, Weiwei Xu, Kun Zhou, and Yin Yang. 2018.DeepWarp: DNN-based Nonlinear Deformation. arXiv preprint arXiv:1803.09109 (2018). SeungWoo Oh, Junyong Noh, and Kwangyun Wohn. 2008. A physically faithful multi-grid method for fast cloth simulation.
Computer Animation and Virtual Worlds
Proceedings of Computer Graphics International 2018 (CGI2018) . ACM, New York, NY, USA, 139–146. https://doi.org/10.1145/3208159.3208162M. Overby, G. E. Brown, J. Li, and R. Narain. 2017. ADMM ⊇ Projective Dynamics: FastSimulation of Hyperelastic Models with Dynamic Constraints.
IEEE Transactions onVisualization and Computer Graphics
23, 10 (Oct 2017), 2222–2234. https://doi.org/10.1109/TVCG.2017.2730875Yue Peng, Bailin Deng, Juyong Zhang, Fanyu Geng, Wenjie Qin, and Ligang Liu. 2018.Anderson Acceleration for Geometry Optimization and Physics Simulation.
ACMTrans. Graph.
37, 4, Article 42 (July 2018), 14 pages. https://doi.org/10.1145/3197517.3201290William H Press, Saul A Teukolsky, William T Vetterling, and Brian P Flannery. 2007.
Numerical recipes 3rd edition: The art of scientific computing . Cambridge universitypress.Xavier Provot et al. 1995. Deformation constraints in a mass-spring model to describerigid cloth behaviour. In
Graphics interface . Canadian Information Processing Society,147–147.Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised representationlearning with deep convolutional generative adversarial networks. arXiv preprintarXiv:1511.06434 (2015).Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks forlarge-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Demetri Terzopoulos, John Platt, Alan Barr, and Kurt Fleischer. 1987. Elastically De-formable Models. In
Proceedings of the 14th Annual Conference on Computer Graphicsand Interactive Techniques (SIGGRAPH ’87) . ACM, New York, NY, USA, 205–214.https://doi.org/10.1145/37401.37427Jonathan Tompson, Kristofer Schlachter, Pablo Sprechmann, and Ken Perlin. 2016.Accelerating eulerian fluid simulation with convolutional networks. arXiv preprintarXiv:1607.03597 (2016).Tzvetomir Vassilev, Bernhard Spanlang, and Yiorgos Chrysanthou. 2001. Fast clothanimation on walking avatars. In
Computer Graphics Forum , Vol. 20. Wiley OnlineLibrary, 260–267.Nicholas Watters, Daniel Zoran, Theophane Weber, Peter Battaglia, Razvan Pascanu,and Andrea Tacchetti. 2017. Visual Interaction Networks: Learning a PhysicsSimulator from Video. In
Advances in Neural Information Processing Systems 30 ,I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, andR. Garnett (Eds.). Curran Associates, Inc., 4539–4547.Nicholas J. Weidner, Kyle Piddington, David I. W. Levin, and Shinjiro Sueda. 2018.Eulerian-on-lagrangian Cloth Simulation.
ACM Trans. Graph.
37, 4, Article 50 (July2018), 11 pages. https://doi.org/10.1145/3197517.3201281Jakub Wejchert and David Haumann. 1991. Animation Aerodynamics. In
Proceedingsof the 18th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’91) . ACM, New York, NY, USA, 19–22. https://doi.org/10.1145/122718.122719Steffen Wiewel, Moritz Becher, and Nils Thuerey. 2018. Latent-space Physics: TowardsLearning the Temporal Evolution of Fluid Flow. arXiv preprint arXiv:1802.10123 (2018).You Xie, Erik Franz, Mengyu Chu, and Nils Thuerey. 2018. tempoGAN: A TemporallyCoherent, Volumetric GAN for Super-resolution Fluid Flow.
ACM Trans. Graph.
Computer Animation and Virtual Worlds
27, 3-4 (2016), 415–424.J. Zhu, T. Park, P. Isola, and A. A. Efros. 2017. Unpaired Image-to-Image TranslationUsing Cycle-Consistent Adversarial Networks. In2017 IEEE International Conferenceon Computer Vision (ICCV)