Hybrid physics-based deep learning methodology for moving interface and fluid-structure interaction
HHybrid physics-based deep learning methodology for moving interfaceand fluid-structure interaction
Rachit Gupta a , Rajeev Jaiman a, ∗ a Department of Mechanical Engineering, University of British Columbia, Vancouver, BC Canada V6T 1Z4
Abstract
In this work, we present a hybrid physics-based deep learning (DL) framework for handling movinginterfaces and predicting fluid-structure interaction (FSI). Using the discretized Navier-Stokes (NS) inthe Arbitrary Lagrangian-Eulerian (ALE) reference frame, we generate full-order flow snapshots andpoint-cloud displacements as target physical data for the learning and inference of coupled fluid-structuredynamics. This integrated operation of the physics-based modeling with the DL-based reduced-ordermodel (DL-ROM) makes our framework hybrid. The proposed multi-level framework is composed of twodata-driven physics-DL drivers that predict unsteady flow and track the moving point cloud displacementsrespectively, while synchronously exchange the force information at the physical interface. The firstcomponent of our proposed framework relies on the proper orthogonal decomposition-based recurrentneural network (POD-RNN) as a semi-supervised procedure to infer the point cloud ALE description.This model essentially relies on the POD basis modes to reduce dimensionality and evolving them inthe time domain. The second component utilizes the convolution-based recurrent autoencoder network(CRAN) as a self-supervised DL procedure to predict the nonlinear flow dynamics at static Eulerianprobes. We introduce these probes as spatially structured query nodes in the moving point cloud to resolvethe field Lagrangian to Eulerian conflict as well as conveniently train the CRAN driver. We design a novelsnapshot-field transfer and load recovery (FTLR) algorithm to optimally select these Eulerian probes.They are selected in such a way that the two components (i.e., POD-RNN and CRAN) are constrained atthe interface to recover bulk force quantities. These hybrid physics-DL drivers ultimately rely on recurrentneural networks to evolve the low-dimensional state. A popular prototypical fluid-structure interactionproblem of flow past a freely oscillating cylinder is selected to test the efficacy of the proposed hybridDL-ROM methodology. The framework tracks the interface description accurately and predicts highlynon-linear wake dynamics for nearly 500 time-steps with limited input demonstrator within excellent to areasonable accuracy. These results motivate us to further explore the application of this hybrid frameworkfor the concept of digital twinning of engineering systems, especially those involving moving boundariesand fluid-structure interactions.
Keywords:
Fluid-structure interaction, Hybrid physics-DL modeling, Proper orthogonal decomposition,Convolution and recurrent neural networks, Digital twin
1. Introduction
Fluid-structure interaction (FSI) is a coupled physical phenomenon that involves a mutual interplayand bidirectional interaction of fluid flow with structural dynamics. This phenomenon is ubiquitous innature and engineering systems such as fluttering flags [51, 16], flying bats [10, 23], offshore platforms ∗ Corresponding author
Email addresses: [email protected] (Rachit Gupta), [email protected] (Rajeev Jaiman)
Preprint submitted to Journal of Computers & Fluids February 19, 2021 a r X i v : . [ phy s i c s . f l u - dyn ] F e b nd pipelines [21, 22] and ship maneuvering [53]. For example, the two-way coupling between fluid andsolid exhibits rich flow dynamics such as wake-body interaction and vortex-induced vibrations (VIVs)[28], which are important to understand from an engineering design or a decision-making standpoint.Due to the complex characteristics of the fluid-structure coupling, frequent but reliable practice is tomodel these complex interplay and underlying dynamics by solving numerically a coupled set of partialdifferential equations describing the physical laws. Of particular interest in these numerical techniquesfor unsteady fluid-structure interaction is to accurately simulate the wake-body interaction in terms ofvortex-induced loads and structural displacements, which represent the prominent coupled dynamicaleffects.Advanced numerical techniques such as arbitrary Lagrangian-Eulerian [12], level set [50], immersedboundary [40], fictitious domain [56] and phase field [34] can provide high-fidelity PDE-based numeri-cal solutions and physical insights of underlying FSI phenomena. Using standard finite difference, finitevolume or finite element method, accurate solutions have been possible by solving millions of fluid andstructural variables using full-order models (FOMs) and incorporating proper treatment of fluid-structureinterface. As the complexity of an FSI system elevates, the accurate treatment of the interface is directlylinked with increasing fidelity near the interface, which implies solving more and more unknown variablesnumerically. These equation-based forward simulations become difficult to model at higher resolutionsof space and time. Moreover, these multiscale simulations come at the expense of prohibitively largecomputing time and high-dimensionality, rendering them almost ineffective in real-time predictions orcontrol. The applicability of DL-based reduced-order modeling has emerged as a promising alternativefor constructing efficient data-driven models of such coupled physical problems. The primary reason isthe ability and flexibility of deep neural networks to automatically extract complex functional relationsbetween data. However, these black-box DL techniques do not account for prior domain knowledge andphysical laws that are crucial for interpretability, data efficiency and reliable decision making. Hence,there is a growing interest to integrate physical-based or mechanistic models with modern machine learn-ing algorithms for efficient predictions, control and optimization.In the last few years, there are many promising approaches established in the research communityfor a synergistic coupling of DL-based models and physical knowledge i.e. physics-based DL models .These models are often trained to represent a full or partial parametrization of any forward physicalprocess while emulating the governing equations. These coarse-grained inference models and surrogaterepresentations aim to reduce high computational cost in forecasting dynamics and modeling quantities ofinterests (QOIs). One end has been in the development of these trained parameters to fulfill state-to-statetime advancements [17, 4, 49] and inverse modeling [9, 30, 39, 31]. Here, we refer to state-to-state timeadvancement as inferring dependent physical variables from previous states while inverse modeling asidentifying physical system parameters from output state information. The other end in the physics-basedDL modeling is about the numerical approximation underlying differential equations using deep neuralcalculus [18, 26, 5, 47]. The approximate solution of PDEs via deep neural calculus has been found tobe slow in training, reduced accuracy and a lack of generality to multiphysics and multiscale systemsinvolving complex geometries and boundary conditions.For increasing generality in these approximations, we can classify the physics-based DL-ROMs intothree types: (a) modification of the DL objective or loss function, (b) novel architecture design, and (c)synchronous (hybrid) operation of physics and ML drivers. With regard to the loss function modification,Raissi et al. [43] developed physics-informed neural networks by constraining residuals of a partialdifferential equation (PDE) as the objective in training a multi-layered perceptron (MLP). The authorshave extended this framework to solve parameters in the basic PDEs [42] and inversely infer QOIs forcanonical VIV system [44]. Many researchers have similarly enforced constitutive, energy and stabilityconservation in DL losses or intermediate parameters to boost generalizability; for instance Karpatne et al. [24], Zhu et al. [57], Erichson et al. [13], Geneva et al. [14] to cite a few. For physically-constrained ML architecture, Daw et al. [11] and Chang et al. [8] modified the RNN cell structure2y intermediate constraints while Muralidhar et al. [35] and Ruthotto and Haber [48] took a similarroute for CNNs. Recently, Li et al. [27] developed Fourier Neural Operators (FNOs) by applying aFourier transform and linear weight operator on the input before the activation function in a neural net.The authors demonstrated the performance of FNO for nonlinear operators in the Burger’s equation,Darcy flow and the 2D Navier-Stokes equations. The third dimension involves the development of hybridphysics-ML frameworks that are designed to operate synchronously. They can also be articulated in a wayto learn low-dimensional temporal dynamics such as projection variables and operate in synchronizationwith FOMs to boost predictive abilities. In this paper, we are interested in the development of such thirdtype of frameworks to address the curse of spatial dimensionality while utilizing the data sets from thecoupled PDE system of fluid-structure interaction. In particular, we are interested in constructing thestate-to-state time-advancements of the flow field while handling moving interface description.We address the issue of high-dimensionality by relying on reduced-order models (ROMs). Thesemodels can project high-dimensional data on low-dimensional spaces linearly or non-linearly for datacompression [3] and feature extraction [31]. The projected spaces are formed such that the loss of infor-mation is minimum or recoverable. More recently, these spaces have been combined with deep learningto predict the temporal dynamics without any knowledge of governing laws. This combination of projec-tion and deep learning helps in reducing dimensionality and boosting real-time predictive abilities. Thereexist many variants of such projection-based physics-ML models in literature such as the POD-CNN byMiyanawala et al. [32], the POD-RNN from Reddy et al. [45], the CNN-RNN by Gonzalez et al. [15]and residual-based data-assisted ROMs by Wan et al. [55]. These models have been successfully testedfor a variety of canonical problems such as a lid-driven cavity, flow past bluff bodies and Kolmogorovflow.However, the applicability of these models in the literature is limited by making simplifying assump-tions of the interface effects such as small or no solid displacements, to the best of the authors’ knowl-edge. There is hence a need to propose a systematic data-driven framework that addresses a realistic andcoupled fluid-structure interaction system with efficient predictive abilities. The present work revolvesaround extending the frameworks discussed in our previous work by Reddy et al. in [6], wherein theauthors presented an assessment of two hybrid DL-ROMs the so-called the POD-RNN and the CRAN forthe flow past a single and side-by-side stationary cylinder configurations. The present work reflects thatthe CRAN achieves longer-time series prediction due to its non-linear projection space compared to thePOD-RNN but the former relies on a reference deep learning grid. This deep learning grid can suffer theloss of interface description when projecting and interpolating the fluid variables from the full-order gridto the deep learning grid. Although the POD-RNN has lesser flow predictive abilities, this hybrid modelrecovers a high-dimensional interface in the prediction.So the question arises is there a way to learn and predict structural and fluid dynamics in a data-drivensetting? How can we develop an efficient data-driven model that predicts flow field and moving interfacedescription? To answer these questions, we present a novel multi-level framework that complementstwo drivers called the POD-RNN and the CRAN in a unifying hybrid physics-based DL framework. Weassume that the full-order FSI simulation is achieved accurately using the coupling of ALE type movingmesh with the Navier-Stokes equations [20]. In this work, we project the fluid variables on a fixed deeplearning grid and learn it in the non-linear space of the CRAN in synchronization with the POD-RNN thatlearns and predicts ALE displacements. We detail the snapshot-field transfer and load recovery (FTLR)scheme that selects the best deep learning grid for the CRAN in a way that interface force recovery ispossible. The novelty of the work, hence, lies in the integrated effect that comes from superimposing thetwo-hybrid physics-DL models in one framework to predict the fluid-structure coupling: vortex-inducedloads by recovering physical data and motion effects. This data-driven model can serve as a prototypicalframework in digitizing next-generation marine vessels by incorporating intelligence through learningfrom online and offline data.The article is organized as follows: Section 2 describes the full-order governing physical model and3 reduced-order representation of fluid-structure interaction. Section 3 discusses the hybrid physics-based DL frameworks and two-level modeling for solid and fluid dynamics. Section 4 describes thesnapshot-FTLR technique and best deep learning grid search. The article ends with application to afreely oscillating VIV of a cylinder in section 5 and conclusions in section 6.
2. Full-order vs reduced-order modeling
In this section, we start by describing our full-order (high-dimensional) data generation process forcoupled fluid-structure interaction, which follows by a brief description of the reduced system dynamicsof fluid-structure interaction.
A coupled fluid-solid system in an arbitrary Lagrangian Eulerian (ALE) reference frame is modeledby the PDE system ρ f (cid:18) ∂ u f ∂ t + ( u f − w ) · ∇ u f (cid:19) = ∇ · σ f + b f on Ω f ( t ) , (1) ∇ · u f = Ω f ( t ) , (2) m s · ∂ u s ∂ t + c s · u s + k s · ( ϕ s ( z , t ) − z ) = F s + b s on Ω s ( t ) , (3)where u f and w represent the fluid and mesh velocities respectively and b f is the body force applied on thefluid Ω f . σ f is the Cauchy stress tensor for a Newtonian fluid written as σ f = − p f I + µ f (cid:0) ∇ u f + ( ∇ u f ) T (cid:1) . Here, p f is pressure in the fluid, I is the identity tensor and µ f is the fluid viscosity. Any arbitrary sub-merged rigid body Ω s experiences transient vortex-induced loads and as a result may undergo large struc-tural motion if mounted elastically. The rigid-body motion along the two Cartesian axes is modeled byEq. (3), where m s , c s and k s denote the mass, damping and stiffness constants per unit length respectively. u s represents the rigid-body motion at time t with F s and b s as the fluid traction and body forces actingon it respectively. Here, ϕ s denotes the position vector that transforms the initial position z of the rigidbody to time t . This coupled system must satisfy the no-slip and traction continuity conditions at thefluid-body interface Γ fs ( t ) as follows: u f ( ϕ s ( z , t ) , t ) = u s ( z , t ) , (4) (cid:90) ϕ f ( γ , t ) σ f · n d Γ + F s = ∀ γ ∈ Γ fs ( t ) . (5)Here, n , γ and d Γ are the outward normal, an element and differential surface area of the fluid-solid in-terface. ϕ f ( γ , t ) is the corresponding fluid part at time t . While Eq. (4) enforces the velocity continuityon the moving interface, the balances the net force exerted by the fluid part ϕ f ( γ , t ) on the rigid body isgiven by Eq. (5) . The coupled differential equations in Eqs. (1)-(3) are numerically solved using thePetrov–Galerkin finite-element and semi-discrete time stepping [21]. The weak form of the incompress-ible Navier-Stokes equations is solved in space using equal-order isoparametric finite elements for thefluid velocity and pressure. We employ the nonlinear partitioned staggered procedure for the stable androbust coupling of fluid-structure interaction. This completes the description of a coupled full-order FSIsystem.With respect to model order reduction, this coupled non-linear system can be expressed as a dynamicalsystem in the following form: d z dt = F ( z ) , (6)4hereby F ( . ) denotes a vector-valued differential operator describing the spatially discretized PDEs inEqs. (1)-(3). The state vector z can be written as z = { p f , u f , u s } ∈ R m × n for a coupled FSI domain with m degrees of freedom and n time-steps. d z / dt is the system dynamics which determines the instantaneousphysics of fluid-structure system and creating a low-order representation that preserves the essence of theoriginal system. From the perspective of a data-driven approach, the idea is to build the differential operator by pro-jecting onto a set of low-dimensional modes. In that sense, we decompose F ( z ) to contain constant C ,linear Bz and nonlinear F (cid:48) ( z ) dynamical components as F P ( z ) = C + Bz + F (cid:48) ( z ) . (7)Relying on the projection-based ROM we represent the state vector z using a subspace spanned by thecolumn vectors of low-dimensional modes V ∈ R m × k , with k << m . This approximates the state vector z as V ˆ z with ˆ z ∈ R k × n and thereby reducing the system dynamics as d ˆ z dt = V T C + V T B V ˆ z + V T F (cid:48) ( V ˆ z ) . (8)which can be re-written in the reduced form as d ˆ z dt = f ( ˆ z ) . (9)for the purpose of reduced-order modeling of a coupled nonlinear fluid-structure system.In general, when the non-linear term is ignored in Eq. (8), one can construct the suitable subspaceprojection f ( · ) = V T B ( · ) using singular value decomposition (SVD) of the state vector z = V Σ W T = ∑ kj = σ j v j w Tj where v j are the POD modes of state z , Σ ∈ R k × k is a diagonal matrix with diagonal entries σ (cid:62) σ (cid:62) . . . (cid:62) σ k (cid:62) W ∈ R n × k is an orthonormal matrix. The total energy contained in eachPOD mode v j is given by using σ j . Note that V and W are respectively orthonormal vectors of zz T and z T z . However, in practice, the estimation of the non-linear term in Eq. (8) is not straightforward.For that purpose, we rely on constructing the non-linear subspace projection f ( · ) = V T B ( · ) + V T F P (cid:48) ( · ) as a self-supervised DL parametrization without any SVD decomposition of the state z . Instead, thedecomposition be achieved by composing trainable layers of encoder θ enc and decoder space θ dec (usingcombination of CNNs and ANNs) such that the observable loss is minimized as θ ∗ = arg min { θ enc θ dec } (cid:107) z − f − ( f ( z ; θ enc ) ; θ dec ) (cid:107) . (10)After constructing a proper subspace projection f , the state-to-state time advancements is essentiallyachieved by progressing the reduced-order state using an RNN evolver θ evolver such that θ ∗ = arg min θ evolver (cid:13)(cid:13)(cid:13)(cid:13) d ˆ z dt ( θ evolver ) − f ( z ) (cid:13)(cid:13)(cid:13)(cid:13) . (11)These projection based-models help in reducing the dimensionality of incoming data z by encodingdominant patterns that are propagated using deep learning methods. This hybridization is hence designedto exploit the advantage of compressing input features as well as maintain higher predictive abilitiesof neural networks. Thus, these frameworks can potentially represent a reduced-order approximation of F ( z ) . In the upcoming section, we discuss the formulation of these frameworks in the context of predictiveFSI. 5 CRAN 𝛀 𝐬 :𝛀 𝐟 : POD-RNN 𝛀 𝐟𝐬 Flow field, rigid body motion & forcepredictions 𝐒 𝑛 INTERFACE LOAD RECOVERY 𝛟ത𝐅 𝑏 𝐒 𝑛−1 𝐒 𝐒 𝑛 𝐒 𝑛−1 𝐲 𝐲 𝑛 𝐲 𝑛−1 (𝐲 𝑛+𝑝 , 𝐒 𝑛+𝑝 )(𝐲 𝑛+𝑝−1 , 𝐒 𝑛+𝑝−1 ) (𝐲 𝑛+1 , 𝐒 𝑛+1 ) Figure 1: Illustration of a two-level hybrid physics-DL framework. The field variables Ω f ( t ) are leaned and predicted on the DLgrid using the CRAN (blue box), while the exact interface information Ω s ( t ) (moving point cloud) is predicted via the POD-RNN(red box). These boxes exchange interface information (grey box) that couples the system via level-set Φ and force signals ¯ F b . Theyellow box demonstrates the synchronized predictions
3. Hybrid DL-ROM methodology for fluid-structure dynamics
In this work, we propose a two-level hybrid physics-DL framework that is directed to model fluid andsolid variables during fluid-structure coupling. In the context of the nature of these data, the first levelis the POD-RNN driver for learning ALE/solid displacements Ω s ( t ) , while the second is the CRAN forlearning non-linear flow field Ω f ( t ) . The illustration of this approach is provided in Fig. 1. On a giventraining data, the POD-RNN model (red box) is employed to learn the ALE point cloud displacements.This tracking assists in identification of a level-set function Φ for discovering fluid and solid points inthe DL grid for the CRAN. As the CRAN process learns the flow variables on this uniform DL grid(blue box), the level Φ accommodates in calculating pixelated force signals ¯ F b on the moving interface.These signals, which are practically corrupted due to grid coarsening, are corrected using interface loadrecovery (grey box). The justification for the driver framework in the context of a general FSI system ishighlighted in sub-sections 3.1 and 3.2. The snapshot-FTLR as an interface load recovery is delineatedsubsequently in section 4. This hybrid technique proposed by Reddy et al. [45] utilizes high-fidelity time-series of snapshotdata obtained from full-order simulations or experimental measurements. These high-dimensional dataare decomposed into spatial modes (dominant POD basis), the mean-field and the temporal coefficients6s reduced-order dynamics using the Galerkin projection of POD. The spatial modes and mean-field formthe offline data-base, while the time-coefficients are propagated and learned using variants of recurrentneural networks: closed-loop or encoder-decoder type. This projection and propagation are hence calledthe POD-RNN. In this work, we learn and infer the full-order ALE mesh displacements (instead of flowfield) using the POD-RNN hybridization. The key idea is that (a) the complete position and shape of theexact fluid-solid interface are inherent in the ALE (point cloud) displacements, and (b) the POD energyspectrum for these mesh displacements are fast decaying, which altogether means that the moving pointcloud can be effectively compressed using finite POD modes and then propagated with RNNs.Let Y = { y y ... y n } ∈ R m × n be the ALE displacement data set in a given cartesian direction fromthe initial position y . Here, y i ∈ R m is the displacement snapshot at time t i and n represents the numberof such snapshots. m (cid:29) n are the number of data probes, for instance, here the number mesh nodes in thefull-order simulation. The target is to predict the future mesh position: y n + , y n + , ... using the trainingdata set Y . The hybrid POD-RNN framework can be constructed by the following three-step process: Step 1: Construct the POD basis for dataset Y Given the n time-snapshots of ALE displacement along any co-ordinate axis, determine the offlinedata base: the mean vector ( y ∈ R m ) and the POD basis modes ( ν ∈ R m × k ). Here k < n (cid:28) m and k isselected based on the eigenvalues of the co-variance matrix. This projection reduces the order of high-dimensional moving point cloud from O ( m ) to O ( k ) . The low-dimensional POD time-coefficients areobtained linearly using POD reconstruction technique y i ≈ y + ν A i , i = , , ..., n , (12)where A i = [ a i a i ... a ik ] ∈ R k are the time-coefficients of the k most energetic modes for the time t i .POD time-coefficients A = { A , A , ..., A n } are hence generated for the data set Y . Once generated, theproblem simply boils down to predict A n + , A n + , ... using the reduced-order dynamical coefficient A n inan iterative manner. For a detailed analysis on linear or non-linear POD decomposition, readers can referto [33, 7]. Step 2: Supervised learning and prediction of time-coefficients using RNN
Training consists of learning a dynamical operator g that allows to recurrently predict finite time-series of POD coefficients. We employ the long short-term memory (LSTM) type RNN as the machinelearning method, which exploits both the long-term dependencies in data and prevents vanishing gradientproblem in training [19]. The RNN is employed as a one-to-one dynamical mapping between the PODtime coefficients A i − and A i . A i = f ( y i ) , A i = g ( A i − ) , (13)where f is the known nonlinear function relating the time coefficients and the ALE data. From Eq. (12), f is the inner product (cid:104) y i − y , ν (cid:105) and g is the trainable operator between the adjacent POD time coefficients.Set the recurrent neural network such that the time-set { A , A , ..., A n − } is mapped to the time-advancedPOD coefficients { A , A , ..., A n } in a way that g mapping is one-to-one A i = g ( A i − ; θ evolver ) , i = , ..., n , (14)where θ evolver are the weight operators of the LSTM cell. Once trained, θ evolver forms an offline parameterrecurrent space which generates a finite amount of predicted time coefficients. This closed-loop networkpredicts the output iteratively with the same trained weights meaning that the generated output is fed asan input for the next prediction. Step 3: Reconstruction to point cloud y n + 𝒱 T 𝐑𝐍𝐍 𝒱 + ത𝐲 … … … … 𝐲 𝑛 𝐲 𝑛+1 𝐲 𝑛+2 𝐲 𝑛+1 𝐲 𝑛+2 𝐲 𝑛+3 𝐲 𝑛+𝑝 𝐀 𝑛+1 𝐀 𝑛+2 𝐀 𝑛+3 𝐀 𝑛+𝑝 𝐀 𝑛 𝐀 𝑛+1 𝐀 𝑛+2 𝐀 𝑛+𝑝−1 Figure 2: Schematic of the predictive POD-RNN framework. With one input driver y n , the predictions y n + , y n + ,... y n + p areachieved autonomously In Step 2, the reduced-order dynamics is predicted using the RNN parameter space. Once A n + isgenerated, it is straightforward to reconstruct the predicted ALE displacements at t n + ( y n + ) as follows: y n + ≈ y + ν A n + . (15)Herein the trained LSTM-RNN is utilized iteratively to predict the desired future time coefficients startingfrom A n . An illustration of the entire process is shown in Fig. 2. To track the moving point cloud, thepredicted time-coefficients are linearly combined with the mean value and the spatially invariant POD.This technique of POD-RNN on the ALE displacements is capable of predicting the FSI motion forfinite time-steps y n + , y n + , ... y n + p autonomously provided that the error is within the acceptable rangeof accuracy. The hybrid POD-RNN method can provide an optimal low-dimensional space (encoding) in whichthe modes are not only computationally inexpensive to obtain but can be physically interpretable as well[33]. However, there are several problems with the POD encoding for highly non-linear flow physics:(a) the encoding is obtained linearly, which may cause significant loss of flow physics characterized bylarge Kolmogorov n-width, (b) the POD decomposition of flow problems dominated turbulence [52] of-ten results in the POD energy spectrum to be slow decaying, implying that the number of POD modescan increase significantly and hard to learn with RNN, and (c) the POD basis ν have additional orthog-onality constraint on the low-dimensional space which can limit its flexibility in general. So a naturalevolution is to replace POD with an alternate and much more flexible encoding that addresses the abovechallenges. One approach is to use CNNs instead of POD, which leads to a non-intrusive type hybridiza-tion framework called the CNN-RNN or simply the CRAN. Furthermore, as depicted in our previouswork [6], CRAN outperforms POD-RNN in complex flow predictions by nearly 25 times. Hence, due tothe aforementioned advantages, we rely on the CRAN for learning non-linear flow variables.While operating synchronously in a highly non-linear neural space, this projection and propagationtechnique extracts low-dimensional embedding whereby the flow variables are extracted via CNNs andthe encoding evolved via RNN. Since there is no knowledge of the mean or basis vectors here, we con-struct a decoding space of transpose convolution that up-samples the low dimensional encoding back tothe high-dimensional space. This gives the architecture a shape of an autoencoder [38, 15] as illustrated8n Fig.3. For the sake of completeness, the CRAN architecture is subsequently discussed. For furtherdetails about the CRAN architecture, readers can refer to our previous work [6].Let S = { S S ... S n } ∈ R N x × N y × n denote the 2D time snapshots of field data set (such as pressure orvelocity data from a flow solver), where S i ∈ R N x × N y is the field snapshot at time t i and n represents thenumber of such snapshots. N x and N y are the number of data probes in the respective Cartesian axes. Forinstance, here, the number of fixed (Eulerian) query points introduced in the moving point cloud. Theseprobes are structured as a spatially uniform space for the CRAN architecture as it relies on field unifor-mity. These probes are optimally selected based on the field convergence and the interface load recoveryusing the snapshot-FTLR method as discussed in section 4. The target of this end-to-end learning is toencode-propagate-decode the future values at the field probes: S n + , S n + , ... using the training data set S . The hybrid CRAN framework is constructed using the following process: Step 1: Find the non-linear encoding feature space for dataset S Given the n snapshots of any field data, construct a trainable neural encoder space θ enc using layers oflinear convolution kernels with non-linear activation (Conv2d) as shown in Fig. 3. The dimensionalityof the field 2D probes is conveniently and gradually down-sampled using convolution filters and plainfeed-forward networks until a finite size of feature basis A is reached, similar to POD encoding as wepreviously discussed. A i = f ( S i ; θ enc ) , i = , , ..., n , (16)where A i = [ a i a i ... a ik ] ∈ R k are the time-coefficients of the k encoded feature space at time t i . Thecoefficients A = { A , A , ..., A n } are determined in a self-supervised fashion with no energy or orthogo-nality constraint unlike POD. Here k < n (cid:28) ( N x × N y ) and k is the unknown hyperparamter for the optimalfeature extraction based on the input data S . This projection reduces the order of high-dimensional fieldfrom O ( N x × N y ) to O ( k ) . Once decomposed, the problem boils down to predict A n + , A n + , ... using thefeature coefficient A n and spatially up-sample to S n + , S n + , ... as generative output. Step 2: Supervised learning and prediction of time-coefficients using RNN
Similar to the POD-RNN, the training of the low-dimensional space consists of learning a one-to-one dynamical operator g that allows to predict a certain series of time-coefficients. Without a lossof generality, we employ the LSTM-RNN in a closed-loop fashion. The RNN is employed as a statetransformation between the feature time-coefficients A i − and A i . Instantiate the recurrent neural networksuch that the time-set { A , A , ..., A n − } is mapped to the time-advanced coefficients { A , A , ..., A n } sothat dynamic operator g is a one-to-one transformation A i = g ( A i − ; θ evolver ) , i = , ..., n , (17)where θ evolver are the weight operators of the LSTM cell. Likewise, once trained, θ evolver forms an offlinerecurrent parameter space which predicts finite time-steps of self-supervised features A obtained fromCNN. Step 3: Generating the high-dimensional time-advanced state S n + Once A n + is generated, there isa need to prolongate the time-advanced features at t n + ( S n + ) using gradual reverse convolution (De-Conv2d) as these low-dimensional features are often hard to interpret. As there is no knowledge of themean or POD basis vector, we rely on decoding the features gradually using transpose convolution space θ dec , which is a mirror of the encoder space. S n + = f − ( A n + ; θ dec ) . (18)The trained LSTM-RNN to employed iteratively to generate desired future state S n + , S n + ... S n + p starting9 … … Spatial down-samplingSpatial prolongation ReshapeTransform
Conv2d 𝑛 𝑐𝑜𝑛𝑣 Temporalencoding Temporaldecoding
LSTM-RNN
FC 𝑛 𝑓𝑢𝑙𝑙 DeConv2d 𝑛 𝑐𝑜𝑛𝑣 Input : DeConv2d
DeConv2d (𝑛 𝑐𝑜𝑛𝑣 −1) Generative output:
FC 𝑛 𝑓𝑢𝑙𝑙 𝐀 𝑖 𝐀 𝑖+1 𝐒 𝑛 𝐒 𝑛+1 , 𝐒 𝑛+2 ,… 𝐒 𝑛+𝑝 𝐀 𝑛 𝐀 𝑛+1 ,𝐀 𝑛+2 , 𝐀 𝑛+𝑝 Figure 3: Schematic of the predictive CRAN framework. The encoding is achieved by gradually reducing input dimension from( N x × N y ) to N A using CNNs (Conv2d) and fully connected (FC) networks. The decoding is similarly realized using FC layers andtranspose CNNs (DeConv2d). Between encoding and decoding the LSTM-RNN evolves the low-dimensional state A from S n . This technique of CNN-RNN on flow variables is self-supervised and is capable of predictingthe flow variables at the fixed probes provided that the prediction error is within the acceptable range ofaccuracy. This process is illustrated in Fig. 3.
4. Snapshot-field transfer and load recovery
A convolution neural network relies on a spatially uniform input data stream for identifying general2D image features. This is accounted for the uniform dimension and operation of adaptive CNN kernels.Given that most practical computational fluid dynamics (CFD) and FSI problems are modeled in a highlyunstructured and body conformal mesh, there is a natural need to introduce a field data-processing step tointerpolate the scattered information as snapshot images before learning them in the CRAN framework(see a uniform DL grid in Fig. 4). Consequently, in this section, we introduce a general data processingstep that can be utilized for mapping flow field variables from an unstructured grid to a uniform DL grid( N x × N y ). We achieve this via interpolation and projection of field information in an iterative processthat allows a recoverable interface force data loss. Once this loss is observed in the training forces, wecorrect them by reconstructing to a higher-order CFD force. We select the interface force as the primarycriteria because the boundary layer forces are highly grid sensitive. This iterative process is cyclicallyperformed as shown in Fig. 4. We refer to the entire cyclic process as a snapshot-field transfer and loadrecovery or (snapshot-FTLR) method. Analogous to multigrid methods in CFD [41, 36], here, we areinterested in finding the best DL grid that can recover the full-order interface load correctly and capturethe Lagrangian-Eulerian interface. The loss of data information is observed in the training data andassumed unchanged to be corrected in the hybrid neural predictions. This procedure is described in thefollowing steps:1. Field transfer: The first step involves the field transfer from a highly unstructured moving pointcloud (x ( t ) , y ( t ) ) to a reference DL grid (X , Y). The size of the reference grid can be chosen N x × N y . We rely on Scipy’s griddata function [1] for interpolating scattered CFD data and fittinga surface. This function generates the interpolant using Clough-Tocher scheme [2] by triangulating10 𝚽 (1) Field transfer (2) Level-set (3) Pixelated force: (4) Damp noises:(5) Interface loadrecovery: 𝐒(X, Y, 𝑡)𝐬(x 𝑡 , y 𝑡 , 𝑡)ത𝐅 𝑏 (𝑡) ζ ത𝐅 𝑏 (𝑡) Ψ ത𝐅 𝑏 (𝑡) DL gridCFD grid
Figure 4: Illustration of mesh-to-mesh field transfer and load recovery for iteratively finding the best CRAN snaphot DL grid the scattered data s ( x ( t ) , y ( t ) , t ) with Qhull, and forming a piecewise cubic Bezier interpolatingpolynomial on each triangle. The gradients of the interpolant are chosen so that the curvature of theinterpolating surface is approximately minimized [37, 46]. Once this is achieved, the field valuesat the static Eulerian probes N x × N y are generated as S ( X , Y , t ) .2. Level-set: With the known learned interface description, a level Φ is set for all the cells on theDL grid. The exact interface description is known for a stationary solid boundary and predictedsynchronously from the ALE displacements via the POD-RNN framework. The process is furtherdemonstrated in section 3.1. This step provides the identification of the solid, fluid and interfacecells on the DL grid.3. Pixelated force : The next step is the calculation of forces exerted by the interface cells thatcontain the rigid body description. We refer to these forces as the pixelated forces. The methodof measuring the pixelated force incorporates construction of Cauchy Stress tensor in the interfacepixels.For an interface cell k (from Fig. 5) at time instant n , the pixelated force f n k for a Newtonian fluidcan be written as f n k = ( σ n a;k − σ n b;k ) . n x ∆ y + ( σ n c;k − σ n d;k ) . n y ∆ x , (19)where σ n ∗ ;k is the Cauchy Stress Tensor for any point inside the cell k. We calculate this tensorat the mid-points of cell faces a − d − b − c as depicted in Fig. 5 by relying on finite differenceapproximation to calculate field values at the face mid-points. n x and n y are normals in the x - and y -direction, while ∆ x = ∆ y refers to cell size. For the present case, we only consider the pressurecomponent while calculating f n k .These individual pixelated forces are summed over all the interface cells (say N F in number) to get This section has been presented in our previous work from Bukka et al. in [6]. It is outlined here for completeness. luid cellsInterface cellsSolid cells 𝐒 ≠ 0 (𝑖, 𝑗)(𝑖, 𝑗 + 1)(𝑖, 𝑗 − 1) (𝑖 + 1, 𝑗)(𝑖 − 1, 𝑗) ab dc Δx Δy Face mid-point
𝐒 = 0
Figure 5: Identification of interface cells in the DL grid and finite difference interpolation of field at the cell faces (green cross).The blue nodes contain flow field values and the red dots are marked as zero the total pixelated force signal as F nb = N F ∑ k = f n k . (20)These total pixelated signals (lift or drag) F b = { F b , F b , ..., F nb } , however, can be corrupted withmissing or noisy information compared to its higher-order counterpart. This is accounted due tothe loss of sub-grid information that leads to a coarser calculation of forces on the sharp interface.Thus, these output signals need to be corrected.4. Damp noises: To correct the pixelated signals, we first apply a moving average Gaussian filter ζ tosmoothen the output force. This is achieved by damping the high-frequency noises in these signalsthat are encountered due to a full-order interface movement on a low-resolution DL grid. Thisdamping smoothens the force propagation for easier reconstruction. To smooth a pixelated forcesignal F nb at a time-step n , we perform the following operations:(a) collect the time-series high-frequency noisy signals F b (lift or drag) of window length 2 k + F b = { F n − kb , ..., F n − b , F nb , F n + b , ..., F n + kb } , (21)(b) select Gaussian-like weights of the specified window length with mean weight w n for F nb w = { w n − k , ..., w n − , w n , w n + , ..., w n + k } , such that n + k ∑ i = n − k w i = , (22)(c) apply the weighted average filter to damp the noise at F nb F nb : = n + k ∑ i = n − k w i F ib = ζ F nb . (23)12. Interface force recovery: Finally, the smooth pixelated force propagation ζ F b can still lack themean and derivative effects compared to its full-order counterpart F Γ fs . This is where we introducethe functional mapping Ψ as described in Algorithm 1. If this mapping can recover the bulk forcescorrectly, then we can utilize the selected grid. If not, then refine the grid and repeat the above stepsfrom (1)-(5), until we find the best snapshot grid that recovers these bulk quantities reasonably.The mesh-to-mesh field transfer and load recovery for snapshot DL grid search are expressed in asimilar procedure as Algorithm 2. Once selected, the chosen DL grid is utilized for learning the flowvariables in the CRAN framework with the POD-RNN operating on the moving interface. Algorithm 3essentially employs the aforementioned interface load recovery to extract higher-order forces from thesynchronized field and interface predictions. Algorithm 1: Functional reconstruction mapping Ψ for higher-order force recovery on low-resolution DL grid Algorithm 1Input : De-noised pixelated force signals F b , full-order FSI forces F Γ fs Output : Reconstructed pixelated force signal Ψ F b To reconstruct smooth pixelated force signals F b :1. Collect the n time-steps of denoised pixelated force and full-order FSI forces (lift or drag): F b = { F b F b ... F n − b F nb } ; F Γ fs = { F Γ fs F Γ fs ... F n − Γ fs F n Γ fs } ;2. Get the mean and fluctuating force components: F (cid:48) b = F b − mean ( F b ) ; F (cid:48) Γ fs = F Γ fs − mean ( F Γ fs ) ;3. Define the time-dependent derivative error using E c : E c = ( F (cid:48) Γ fs − F (cid:48) b ) ./ ( F (cid:48) b ) with F (cid:48) b (cid:54) = F b : = F (cid:48) b + mean ( F Γ fs ) + mean ( E c ) F (cid:48) b = Ψ F b ;13 lgorithm 2: Iterative snapshot DL grid search via interface force recovery Algorithm 2Input : Full-order field data s = { s s ... s n } and FSI forces F Γ fs = { F Γ fs F Γ fs ... F n Γ fs } Output : Grid selection N x × N y Initialise any non-conformal uniform grid ( N x × N y ) of cell size ∆ x = ∆ y: while :1. Project point cloud CFD data s ( x ( t ) , y ( t )) ∈ R m × n on snapshot grid S ( X , Y ) : S ← s s.t. S ∈ R N x × N y × n ;2. Apply learned level-set function Φ ∈ R N x × N y × n on S ( X , Y ) element wise: S : = Φ ∗ S ;3. Calculate pixelated force signals F b from interface cells Γ fs in S using Eqs. (19) - (20);4. Damp the high-frequency noises in pixelated signals F b using moving average Gaussian filter ζ : F b : = ζ F b using Eqs. (21)-(23);5. Reconstruct the smooth pixelated force F b to full-order by functional mapping Ψ : F b : ≈ Ψ F b using Algorithm 1;6. if ( ( (cid:107) F b − F Γ fs (cid:107) / (cid:107) F Γ fs (cid:107) ) ≤ ε ): break ; else :Refine the grid N x : = N x , N y : = N y ; Algorithm 3: Extracting FSI forces using low-resolution predicted fields and ALE displacements
Algorithm 3Input : Field predictions from CRAN ˆ S = { S n + S n + ... S n + p } , displacement predictions fromPOD-RNN ˆ y = { y n + y n + ... y n + p } , calculated interface mapping Ψ from training data Output : Full-order force predictions: ˆ F Γ fs = { F n + Γ fs F n + Γ fs ... F n + p Γ fs } To extract full-order FSI forces (lift or drag) :1. Apply predicted level-set function ˆ Φ ∈ R N x × N y × p from POD-RNN on ˆ S ∈ R N x × N y × p elementwise:ˆ S : = Φ ∗ ˆ S ;2. Calculate pixelated force signals ˆ F b from interface cells Γ fs in ˆ S using Eqs. (19) - (20);3. Damp high-frequency noises in extracted pixelated signals ˆ F b using moving average gaussianfilter ζ :ˆ F b : = ζ ˆ F b using Eqs. (21)-(23);4. Reconstruct the extracted smooth pixelated force ˆ F b to full-order by known functional map-ping Ψ :ˆ F Γ fs ≈ Ψ ˆ F b able 1: An elastically-mounted circular cylinder undergoing vortex-induced vibration: Comparison of full-order forces and dis-placements values in the present study and benchmark data Mesh N ∗ cyl N ∗ f luid C D ( C L ) rms ( Ax ) rms (cid:0) Ay (cid:1) max Present 124 25,916 2.0292 0.1124 0.0055 0.5452Ref. [28] 168 25,988 2.0645 0.0901 0.0088 0.5548 N cyl are the number cylinder surface elements N f luid are the number Eulerian fluid elements
5. Application: Vortex-induced vibration (VIV)
In this section, we test our proposed hybrid physics-DL framework and the snapshot-FTLR methodfor a benchmark fluid-structure problem. For this purpose, we consider an unsteady elastically-mountedcylinder problem in an external flow. The wake flow behind the bluff body generates switching vortexloads on the solid, which causes its vibration. This solid vibration, in turn, affects the near-wake flowdynamics resulting in a two-way synchronized oscillating system. The flow physics is highly sensitiveand directly related to the interaction dynamics of the downstream vortex patterns with the motion ofthe cylinder. Of particular interest is to learn and predict such synchronized wake-body interaction in aphysics-DL data-driven sense.A rigid cylinder is mounted in transverse and streamwise directions (see Fig. 6 (a)) with linear andhomogeneous springs and no damping, resulting in alike natural frequencies ( f nx = f ny ) in both directions.Computation is carried at a fixed cylinder mass ratio m ∗ = m πρ f D =
10, reduced velocity U r = U ∞ f n D = U ∞ π (cid:113) km D =
5, and Reynolds number Re = ρ f U ∞ D µ f = m is the solid mass, f n is the naturalfrequency of the cylinder and U ∞ is the uniform inlet x -velocity. Rest all variables imply their usualmeaning. At these non-dimensional parameters, a high and sensitive nature of amplitude response isobtained [28].The freely oscillating structural system is installed in a 2D computational domain with the centerat sufficient distances from the boundaries to capture the downstream wake dynamics. A no-slip andtraction continuity is enforced on the cylinder surface (Eqs. (4)-(5)), while a uniform velocity profile U ∞ is maintained at the inlet Γ in . Traction-free boundary is implemented on the outlet Γ out and slip conditionis specified at the top Γ top and bottom Γ bottom . We discretize the fluid domain via unstructured finiteelement meshes. The final mesh, which is obtained after following standard rules of mesh convergence,is presented in Fig. 6 (b) which contains a total of 25916 quadrilateral elements with 26114 nodes. Thefull-order simulation is carried out via finite element Navier-Stokes solver in the ALE framework asdescribed in section 2.1 and the output details are tabulated in Table 1 for validation. By incorporatingthe surface traction from the Cauchy stress tensor over the first boundary layer elements on the fluid-solidsurface, the fluid force along the fluid-solid boundary is computed. The drag and lift force coefficientsare calculated as C D = ρ f U ∞ D (cid:90) Γ fs ( σ f . n ) . n x d Γ , C L = ρ f U ∞ D (cid:90) Γ fs ( σ f . n ) . n y d Γ . (24)This full-order simulation is carried out for a total of 250 tU ∞ / D with a time-step of 0 . tU ∞ / D . Wechoose a sufficiently small time-step to account for varying amplitudes and frequency of structural forcesand test the robustness of the proposed physics-DL framework. A total of 10 ,
000 time-snapshots aregenerated at every 0 . tU ∞ / D for the pressure field, x -velocity and the ALE displacements. From these15 Γ 𝑖𝑛 Γ 𝑜𝑢𝑡 Γ 𝑡𝑜𝑝 Γ 𝑏𝑜𝑡𝑡𝑜𝑚
15𝐷 𝐷10𝐷 (a) (b)Figure 6: The VIV of a cylinder: (a) Schematic of the unsteady elastically mounted cylinder problem, (b) representative full-orderdomain and near cylinder unstructured mesh a) (b)Figure 7: The VIV of a cylinder: Cumulative and percentage of modal energies. (a)-(b) for ALE x-displacements Y x , (c)-(d)for ALE y-displacements Y y . The fast decaying energy spectrum reveals that the linear decomposition of ALE can be effectivelyrepresented within first few dominant modes full-order data, n tr = tU ∞ / D ) are reserved for training and n ts = tU ∞ / D ) are kept for testing. Thus the total time-steps are N = The application of the POD-RNN driver on the 2D ALE displacements are first discussed. The pri-mary objective here is to accurately infer full-order structural displacements via moving point cloud.These predictions are then superimposed with the background fluid snapshot DL grid synchronously us-ing the snapshot-FTLR method. The POD-RNN driver on the ALE grid system can be summarized asfollows:1. Time instances of ALE x -displacements or y -displacements Y = (cid:8) y y . . . y N (cid:9) ∈ R m × N (here, m = N = y ∈ R m × N and the offline database of meanfield ¯ y ∈ R m and POD modes ν ∈ R m × N are obtained.2. Eigenvalues Λ N × N of the covariance matrix ˜ y T ˜ y ∈ R N × N are extracted as their magnitude measuresthe energy of respective POD modes. Cumulative and percentage of modal energies are plotted inFig. 7 and 8 with respect to the number of modes. It is evident that nearly entire system energy( > .
99% of the total energy) is concentrated in the first 1-2 POD modes. This means that theALE field can optimally be reduced with just k = m = y ) is concentrated at the movinginterface and propagates linearly to the far-field boundaries with a poisson operator. With these17 a) (b)Figure 8: The VIV of a cylinder: Cumulative and percentage of modal energies. (a)-(b) for ALE x -displacements Y x , (c)-(d)for ALE y -displacements Y y . The fast decaying energy spectrum reveals that the linear decomposition of ALE can be effectivelyrepresented within first few dominant modes < . × − .3. The dynamical evolution of the dominant POD modes ν ∈ R m × k is obtained using A = ν T ˜ y ∈ R k × N and is depicted in Fig. 9. The N temporal coefficients are divided into training ( n tr = n ts = k (here, k = k modes are learned and predicted in one time instant. The nearly periodic trend inthe modes allows an easier hyperparameter construction (see parameter details in Table 2) of theLSTM-RNN albeit different phases and amplitudes of modes exist. The training took around 5minutes on a single graphics processing unit (GPU) and the prediction of modal coefficients ˆ A is depicted in Fig. 10 for testing the time-series output. We keep the multi-step prediction cycleto p =
100 while testing the output, implying that one input time-step is used to predict the next100 time-coefficients. It is worth mentioning that that an encoder-decoder type (seq2seq) LSTMarchitecture would have been preferred to extract the temporal coefficients in the case of highlychaotic dynamics. In the present case of the periodic oscillations, a simple closed-loop LSTM-RNN is sufficient for point cloud tracking.
FSI displacement prediction : The predicted modal coefficients at any time-step i , ˆ A i ∈ R k , can simplybe reconstructed back to the point cloud ˆ y i ∈ R m using the mean field ¯ y ∈ R m and k spatial POD modes ν ∈ R m × k as ˆ y i ≈ ¯ y + ν ˆ A i (see Eq. 15). Fig. 11 depicts such comparison of predicted and true values ofthe VIV motion in x and y directions of the cylinder. The results indicate that the POD-RNN on ALE isable to accurately predict the position of moving FSI interface. These results are inferred in a closed-loop18 a)(b)Figure 9: The VIV of a cylinder: Time history of k = tU ∞ / D ) for (a) ALE x -displacements Y x and (b) ALE y -displacements Y y Table 2: The VIV of a cylinder: Network and hyper-parameters details of the closed-loop RNN
ALE k Cell h ∗ Optimizer α ∗ Epochsx or y 2 LSTM 256
Adam ∗ h : LSTM hidden cell dimension α : initial learning rate Adam : adaptive moment optimization [25] a)(b)Figure 10: The VIV of a cylinder: Dynamical evolution (prediction) of modal coefficients on test data (from 225 till 250 tU ∞ / D )for (a) ALE x-displacements Y x and (b) ALE y-displacements Y y a) (b)Figure 11: The VIV of a cylinder: Predicted and actual (the POD-RNN driver). (a) x-position and (b) y-position of the interface,normalised by diameter of cylinder D . The root mean squared error (RMSE) between true and predicted is 2 . × − and1 . × − for Ax / D and Ay / D respectively fashion by predicting p =
100 steps from one demonstrator at a time. This incorporation of ground datademonstrators in the prediction combats the compounding effect of errors thereby boosting long termpredictive abilities [54]. This sub-section, hence, completes the moving CFD point cloud prediction.
The POD-RNN driver assists in retrieving the definition of the moving interface and get the Φ level-set feature. However, the best snapshot DL grid ( N x × N y ) must be sought for the field prediction tobe carried out on the CRAN framework, as highlighted in section 4. The main concept is to estimatefull-order point cloud data on a snapshot grid where a higher-order load recovery of Ψ is possible at thepractical level. This is consequently discussed:1. We start by projecting the point cloud CFD training data (for instance pressure field) s = { s s ... s N } ∈ R m × N as spatially uniform snapshots S = { S S ... S N } ∈ R N x × N y × N . This uniformity is achivedvia SciPy’s griddata function [1] by mapping the m dimensional unstructured data on a 2-d refer-ence grid (here, N x = N y = , , , , , D × D with ≈ D length for downstream cylinder wake. With respect to the number of pixels, wecompare the surface-fitted interpolant construction provided by griddata : nearest , linear and cubic form. This is attained by sampling the field’s maximum and minimum values with pixel numberson the fitted surface process. As seen from Fig. 12, for any pressure time-step tU ∞ / D = nearest levels-off to the true p max and p min on grid refinement. This behavior is expected as thismethod assigns the value of the nearest neighbor in distributed triangulation-based information.The linear and cubic , however, are higher-order triangulation-based linear and cubic interpolationtechniques with C and C continuity. The linear method linearly converges to the true values ongrid refinement. Cubic approach, however, works fairly well on a coarser grid N x = N y = , N x = N y = , , tU ∞ / D =
125 at 512 × a) (b)(c) (d)Figure 12: The VIV of a cylinder: The convergence of pressure field with number of pixels and interpolation schemes for time-step tU ∞ / D =
22L pixelated grid. The qualitative estimate supports the convergence behavior. It can be observedthat nearest contain oscillations compared to full-order description largely because of a discontin-uous assignment of fields at the selected probes. A linear achieves nearly perfect fit in terms ofthe near wake snapshot and interface dynamics with least bias. Notably, the cubic tends to overfit.Hence, we rely on the linear technique for coarse-grain field assignment.2. The primary part of grid selection boils down to a pixelated force propagation (lift and drag) F b = { F b , F b , ..., F nb } (see Eqs. (19)-(20)) within a possible functional correction Ψ . Fig. 14 (a)-(b)demonstrate the DL grid dependence of normalized pressure pixelated F b / . ρ f U ∞ D vs full-orderforces F Γ fs / . ρ f U ∞ D . It is observed that grid coarsening leads to the pixelated force propagationto contain some noises, especially in the direction where the solid displacements are large (whichis the lift here). These noisy signals become dominant in reducing the time-step and increasinggrid size. The reason is attributed to the loss of DL sub-grid information attributed to a lowerfidelity in the vicinity of the moving boundary and a linear way of pixelated force calculation usingfinite difference approximation. Interestingly, these noises are unique to sharp fluid-solid interfacepropagation on a low-resolution DL grid. The force signals are devoid of any high-frequency noisesfor static FSI boundaries as discussed in our previous work [6].3. To correct these forces, while still maintaining a lower DL grid resolution, we first apply Gaussianfiltering ζ (using Eqs. (21)-(23)) on the pixelated force propagation to damp the high-frequencynoises. Fig. 15 (a)-(b) depict the smooth trend ζ F b in the normalized forces on various DL gridsfor drag and lift coefficients via a Gaussian filter length 2 k + =
20. It is noted that the pixelationprocess results in mean and derivative variation in bulk quantities from the full-order counterpart.These errors indeed are reduced with super-resolution. However, we are interested in refining thesnapshot DL grid to an extent where bulk quantities can be linearly reconstructed to the full-orderwith mean and derivative correction (using Algorithm 1). The pixelated force and reconstructionprocess Ψ on different snapshot DL grids is tabulated in Table 3. Columns 3 and 4 tabulate themean and rms of the total pixelated drag and lift coefficients for various grid sizes. Column 5in Table 3 similarly denote the mean of time-dependent derivative correction E c observed fromAlgorithm 1. Columns 6 and 7 depict the reconstruction error ε in corrected forces Ψ F b andfull-order F Γ fs as calculated from step 6 of Algorithm 2. The reconstruction accuracy is nearly99 .
8% and 96 .
5% for drag and lift respectively onward grid 512 × Ψ mapping and correcting ontraining forces. We select snapshot 512 ×
512 as the DL grid for flow field CRAN predictions as itaccounts for a reasonable force recovery while avoiding the necessity of super-resolution.
With the chosen DL grid, the end-to-end nonlinear learning based on the CRAN is used to check forflow field predictions (pressure and x -velocity) until the exact solid/ALE is estimated. The applicabilityof Algorithm 3 to derive bulk quantities from predictive fields is also addressed. The overall procedure isdiscussed as:1. Point cloud flow data (pressure or x-velocity) s = (cid:8) s s . . . s N (cid:9) ∈ R m × N from a NS solver (here, m = , N = S = (cid:8) S S . . . S N (cid:9) ∈ R N x × N y × N (here, N x = N y = n tr = tU ∞ / D ) and n ts = tU ∞ / D ). 23 Full-order grid (a) (b)(c) (d)Figure 13: The VIV of a cylinder: Qualitative behavior of different interpolation methods for NS pressure field in ALE reference attime-step tU ∞ / D = ×
512 DL grid withrespect to (a) full-order CFD gridTable 3: The VIV of a cylinder: Comparison of pixelated forces and reconstruction accuracy on various snapshot DL grids withrespect to full-order
Grid Cell-size ζ C D , p ζ ( C L , p ) rms mean ( E c ) ε ( C D , p ) ε ( C L , p )
128 0.0625 1.8295 0.1491 0.8534 7.6 × − × −
256 0.0313 1.7470 0.1212 0.9124 3.3 × − × −
512 0.0157 1.7130 0.1180 0.9307 2.9 × − × − × − × − FOM 2.2 × − a) (b)Figure 14: The VIV of a cylinder: Total pixelated force propagation (from 100-125 tU ∞ ) on various snapshot DL grids vs full-orderfor (a) pressure drag coefficient C D , p and (b) pressure lift coefficient C L , p on training field
2. The training time-steps n tr are further sparsed for every 0 . tU ∞ / D , thereby reducing the totaltrainable steps to n tr /
2. This is essentially carried out to speed up the CRAN I/O data processing,while still maintaining the same time horizon of training (100-225 tU ∞ / D ). Following this, thestandard principles of normalization and batch wise arrangement N s =
100 is followed to generatethe scaled featured input S (cid:48) = { S (cid:48) S (cid:48) . . . S (cid:48) N s } ∈ [ , ] N x × N y × N t × N s . Note that the closed loop LSTM-RNN’s time-step dependency of size N t =
25 is selected such that n tr / = N s N t . While training, amini-batch of size n s = A of sizes: N A = , ,
128 (see Fig. 3 ). These modes are obtained from a non-linear space of the CNNs, thus representing a stronger ROM projection. Since these projectionsare self-supervised, we experiment the different sizes of the modes based on the convergence intraining. We first instantiate the training of all the three CRAN models for N train = ,
000 itera-tions starting from the pressure fields. We select N A =
128 based on a faster decay in the overallloss (Eqs. (10)-(11)) compared to N A = ,
64 evolver state. N A =
128 pressure model is furtheroptimized until total iterations reach to N train = with overall mean squared loss ≈ . × − .Once the pressure training is complete, we transfer the learning to x-velocity field and optimise theCRAN network subsequently for N train = ,
000 iterations.Each CRAN model is trained on a single GPU Quadro GP100/PCIe/SSE2 with Intel Xeon(R) Gold6136 CPU @ 3.00GHz × 24 processor for nearly 2.7 days. We took a long training time so that theCRAN based hybrid-framework self-supervises with maximum optimisation and the least error. Westop the training only if the loss is steady to ≈ . × − . The test time-steps depict the predictiveperformance of these networks in terms of accuracy and overfitting check. Field and force prediction : Herein, N A =
128 trained CRAN model is employed to demonstrate thefield predictions for pressure (and likewise, x-velocity) for a multi-step predictive cycle of p = N t = a) (b)(c) (d)Figure 15: The VIV of a cylinder: Interface load behavior vs time (from 100-125 tU ∞ ) on the DL snapshot grids. (a)-(b) denote thesmoothen force propagation and (c)-(d) depicts recovered interface force information for drag and lift coefficients able 4: The VIV of a cylinder: Deep CRAN layer details: convolutional kernel sizes, numbers and stride. Layers 6,7,8 representfully connected feed-forward encoding, while 9,10,11 depict similar decoding. The low-dimensional state A is evolved betweenlayers 8 and 9 Conv2d encoder DeConv2d decoderLayer kernel-size kernels stride Layer kernel-size kernels stride1 10 ×
10 2 4 12 5 × ×
10 4 4 13 5 × × × × ×
10 2 45 5 × ×
10 1 4
Figs. 16 and 17 depict the comparison of predicted and true values of pressure and x-velocity fields re-spectively at time-steps 9300 (232 . tU ∞ / D ), 9600 (240 tU ∞ / D ) and 9960 (249 tU ∞ / D ) on the test data.The normalized reconstruction error E n is, similarly, constructed by taking the absolute value of differ-ences between the true and predicted field and normalizing it with L norm of the truth. It is evident fromthese predictions that the major portions of the errors are concentrated in the nonlinear flow separationregion of the vibrating cylinder. The reconstruction error is in the order of 10 − for pressure and 10 − for x -velocity. The field prediction is accurately predicted in terms of the expected cylinder motion. Similarto the POD-RNN, the feed-back demonstrators combat the compounding errors and thus enforcing theCRAN predictive trajectory devoid of divergence.The synchronous point cloud structural and coarse-grained flow field predictions are employed tocalculate the integrated pressure loads C D , p , C L , p using Algorithm 3 and is demonstrated in Fig. 18. Theinterface load algorithm extracts the full-order pressure loads with consistent phases using projectedfields on the DL grid (shown in green line) largely accounted for accurate flow separation and near-wakepredictions. The bursts/sinks in the force amplitudes, however, reflect upon the sensitivity of a coarse-grained field to full-order structural response. As we note that the pixelated force signals are sensitiveto spurious oscillations, the neural predictions can lead to residuals albeit higher-order force recoveryon training data. To correct these residuals, the propagation pattern is learned with a one-to-one LSTM-RNN (green to black Fig. 18). The principal idea is to seek a system identification operator [29] thatcorrects the amplitude residuals in bulk quantities which the CRAN architecture develops. We achievethis by learning the initial 300 performance steps and forecasting all 500 steps. The filtered green signals(shown in the red line) demonstrate good precision for the drag and reasonable in the lift compared to thefull-order. 27 a)(b)(c)Figure 16: The VIV of a cylinder: Comparison of predicted and true fields along with normalized reconstruction error E n at tU ∞ / D = (a) 232.5, (b) 240 (c) 249 for pressure field ( P ) a)(b)(c)Figure 17: The VIV of a cylinder: Comparison of predicted and true fields along with normalized reconstruction error E n at tU ∞ / D = (a) 232.5, (b) 240 (c) 249 for x-velocity field ( U ) a) (b)Figure 18: The VIV of a cylinder: Interface force prediction on test time-steps from the predictive framework (from 225-250 tU ∞ :(a) pressure drag coefficient C D , p and (b) pressure lift coefficient C L , p
6. Conclusions
We have presented a hybrid physics-DL framework based on coarse to fine-grained learning for fluid-structure interaction with special emphasis on moving boundaries. A low-resolution inference of flowfield and full-order mesh description with a new definition of load recovery has been discussed. Byanalyzing a prototype VIV model, we have first shown that the POD-RNN component infers the pointcloud dynamics by projecting the ALE displacements on ∼ Acknowledgement
The authors would like to acknowledge the Natural Sciences and Engineering Research Council ofCanada (NSERC) for the funding. This research was supported in part through computational resourcesand services provided by Advanced Research Computing at the University of British Columbia.
References [1] (2019). Scipy reference guide, release 1.4.1. https: // docs. scipy. org/ doc/ scipy-1. 4.1/ scipy-ref-1. 4. 1. pdf , . 302] Alfeld, P. (1984). A trivariate clough—tocher scheme for tetrahedral data.
Computer Aided Geo-metric Design , , 169–181.[3] Berkooz, G., Holmes, P., & Lumley, J. L. (1993). The proper orthogonal decomposition in theanalysis of turbulent flows. Annual review of fluid mechanics , , 539–575.[4] Brown, M. E., Lary, D. J., Vrieling, A., Stathakis, D., & Mussa, H. (2008). Neural networks as atool for constructing continuous ndvi time series from avhrr and modis. International Journal ofRemote Sensing , , 7141–7158.[5] Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2016). Discovering governing equations from databy sparse identification of nonlinear dynamical systems. Proceedings of the National Academy ofSciences , , 3932–3937.[6] Bukka, S. R., Gupta, R., Magee, A. R., & Jaiman, R. K. (2021). Assessment of unsteady flowpredictions using hybrid deep learning based reduced-order models. Physics of Fluids , , 013601.[7] Burkardt, J., Gunzburger, M., & Lee, H.-C. (2006). Pod and cvt-based reduced-order modeling ofnavier–stokes flows. Computer Methods in Applied Mechanics and Engineering , , 337–355.[8] Chang, B., Chen, M., Haber, E., & Chi, E. H. (2019). Antisymmetricrnn: A dynamical system viewon recurrent neural networks. arXiv preprint arXiv:1902.09689 , .[9] Chen, H., Zhang, Y., Kalra, M. K., Lin, F., Chen, Y., Liao, P., Zhou, J., & Wang, G. (2017). Low-dose ct with a residual encoder-decoder convolutional neural network. IEEE transactions on medicalimaging , , 2524–2535.[10] Chen, P.-T., Joshi, S. P., Swartz, S., Breuer, K., & Reich, G. W. (2014). Bat-inspired flapping flight.In (p. 1120).[11] Daw, A., Thomas, R. Q., Carey, C. C., Read, J. S., Appling, A. P., & Karpatne, A. (2020). Physics-guided architecture (pga) of neural networks for quantifying uncertainty in lake temperature mod-eling. In Proceedings of the 2020 SIAM International Conference on Data Mining (pp. 532–540).SIAM.[12] Donea, J. (1983). Arbitrary Lagrangian–Eulerian finite element methods.
Computational Methodsfor Transient Analysis , (pp. 474–516).[13] Erichson, N. B., Muehlebach, M., & Mahoney, M. W. (2019). Physics-informed autoencoders forlyapunov-stable fluid flow prediction. arXiv preprint arXiv:1905.10866 , .[14] Geneva, N., & Zabaras, N. (2020). Modeling the dynamics of pde systems with physics-constraineddeep auto-regressive networks.
Journal of Computational Physics , , 109056.[15] Gonzalez, F. J., & Balajewicz, M. (2018). Deep convolutional recurrent autoencoders for learninglow-dimensional feature dynamics of fluid systems. arXiv preprint arXiv:1808.01346 , .[16] Gurugubelli, P. S., & Jaiman, R. (2015). Self-induced flapping dynamics of a flexible inverted foilin a uniform flow, . , 657–694.[17] Ham, Y.-G., Kim, J.-H., & Luo, J.-J. (2019). Deep learning for multi-year enso forecasts. Nature , , 568–572.[18] Han, J., Jentzen, A., & Weinan, E. (2018). Solving high-dimensional partial differential equationsusing deep learning. Proceedings of the National Academy of Sciences , , 8505–8510.3119] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation , , 1735–1780.[20] Jaiman, R., Geubelle, P., Loth, E., & Jiao, X. (2011). Transient fluid–structure interaction withnon-matching spatial and temporal discretizations. Computers & Fluids , , 120–135.[21] Jaiman, R., Guan, M., & Miyanawala, T. (2016). Partitioned iterative and dynamic subgrid-scalemethods for freely vibrating square-section structures at subcritical reynolds number. Computers &Fluids , , 68–89.[22] Joshi, V., Gurugubelli, P., Law, Y., Jaiman, R., & Adaikalaraj, P. (2018). A 3d coupled fluid-flexible multibody solver for offshore vessel-riser system. In International Conference on OffshoreMechanics and Arctic Engineering (p. V002T08A009). American Society of Mechanical Engineersvolume 51210.[23] Joshi, V., Jaiman, R. K., & Ollivier-Gooch, C. (2020). A variational flexible multibody formulationfor partitioned fluid-structure interaction: Application to bat-inspired drones and unmanned air-vehicles. arXiv preprint arXiv:2005.01971 , .[24] Karpatne, A., Watkins, W., Read, J., & Kumar, V. (2017). Physics-guided neural networks (pgnn):An application in lake temperature modeling. arXiv preprint arXiv:1710.11431 , .[25] Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprintarXiv:1412.6980 , .[26] Lagaris, I. E., Likas, A., & Fotiadis, D. I. (1998). Artificial neural networks for solving ordinaryand partial differential equations.
IEEE transactions on neural networks , , 987–1000.[27] Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., & Anandkumar,A. (2020). Fourier neural operator for parametric partial differential equations. arXiv preprintarXiv:2010.08895 , .[28] Li, Z., Yao, W., Yang, K., Jaiman, R. K., & Khoo, B. C. (2016). On the vortex-induced oscillations ofa freely vibrating cylinder in the vicinity of a stationary plane wall. Journal of Fluids and Structures , , 495–526.[29] Ljung, L., Andersson, C., Tiels, K., & Schön, T. B. (2020). Deep learning and system identification.In Proc IFAC Congress, Berlin .[30] Lunz, S., Öktem, O., & Schönlieb, C.-B. (2018). Adversarial regularizers in inverse problems. In
Advances in Neural Information Processing Systems (pp. 8507–8516).[31] Miyanawala, T., & Jaiman, R. (2017). An efficient deep learning technique for the navier-stokesequations: Application to unsteady wake flow dynamics. arXiv preprint , .[32] Miyanawala, T., & Jaiman, R. K. (2019). A hybrid data-driven deep learning technique for fluid-structure interaction. In
International Conference on Offshore Mechanics and Arctic Engineering (p. V002T08A004). American Society of Mechanical Engineers volume 58776.[33] Miyanawala, T. P., & Jaiman, R. K. (2018). Decomposition of wake dynamics in fluid-structureinteraction via low-dimensional models. arXiv preprint arXiv:1806.00396 , .[34] Mokbel, D., Abels, H., & Alanda, S. (2018). A phase-field model for fluid–structure interaction.
Journal of computational physics , , 823–840.3235] Muralidhar, N., Islam, M. R., Marwah, M., Karpatne, A., & Ramakrishnan, N. (2018). Incorporatingprior domain knowledge into deep neural networks. In (pp. 36–45). IEEE.[36] Nastase, C. R., & Mavriplis, D. J. (2006). High-order discontinuous galerkin methods using anhp-multigrid approach. Journal of Computational Physics , , 330–357.[37] Nielson, G. M. (1983). A method for interpolating scattered data based upon a minimum normnetwork. Mathematics of Computation , , 253–271.[38] Otto, S. E., & Rowley, C. W. (2019). Linearly recurrent autoencoder networks for learning dynam-ics. SIAM Journal on Applied Dynamical Systems , , 558–593.[39] Parish, E. J., & Duraisamy, K. (2016). A paradigm for data-driven predictive modeling using fieldinversion and machine learning. Journal of Computational Physics , , 758–774.[40] Peskin, C. S. (2002). The immersed boundary method. Acta numerica , , 479–517.[41] Pierce, N. A., & Giles, M. B. (1997). Preconditioned multigrid methods for compressible flowcalculations on stretched meshes. Journal of Computational Physics , , 425–445.[42] Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2017). Physics informed deep learning (part i):Data-driven solutions of nonlinear partial differential equations. arXiv preprint arXiv:1711.10561 , .[43] Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deeplearning framework for solving forward and inverse problems involving nonlinear partial differentialequations. Journal of Computational Physics , , 686–707.[44] Raissi, M., Wang, Z., Triantafyllou, M. S., & Karniadakis, G. E. (2019). Deep learning of vortex-induced vibrations. Journal of Fluid Mechanics , , 119–137.[45] Reddy, S. B., Magee, A. R., & Jaiman, R. K. (2019). Reduced order model for unsteady fluid flowsvia recurrent neural networks. In ASME 2019 38th International Conference on Ocean, Offshoreand Arctic Engineering . American Society of Mechanical Engineers.[46] Renka, R. J., Renka, R., & Cline, A. (1984). A triangle-based c interpolation method. The RockyMountain journal of mathematics , (pp. 223–237).[47] Rudy, S. H., Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2017). Data-driven discovery of partialdifferential equations.
Science Advances , , e1602614.[48] Ruthotto, L., & Haber, E. (2019). Deep neural networks motivated by partial differential equations. Journal of Mathematical Imaging and Vision , (pp. 1–13).[49] San, O., & Maulik, R. (2018). Machine learning closures for model order reduction of thermalfluids.
Applied Mathematical Modelling , , 681–710.[50] Sethian, J. A. (1999). Level Set Methods and Fast Marching Methods: Evolving interfaces in com-putational geometry . computer vision and materials science, Cambridge University Press.[51] Shelley, M. J., & Zhang, J. (2011). Flapping and bending bodies interacting with fluid flows.
Annu.Rev. Fluid Mech. , , 449–465.[52] Sirovich, L. (1987). Turbulence and the dynamics of coherent structures. i. coherent structures. Quarterly of applied mathematics , , 561–571.3353] Takashina, J. (1986). Ship maneuvering motion due to tugboats and its mathematical model. Journalof the Society of Naval Architects of Japan , , 93–102.[54] Venkatraman, A., Hebert, M., & Bagnell, J. A. (2015). Improving multi-step prediction of learnedtime series models. In Twenty-Ninth AAAI Conference on Artificial Intelligence .[55] Wan, Z. Y., Vlachas, P., Koumoutsakos, P., & Sapsis, T. (2018). Data-assisted reduced-order mod-eling of extreme events in complex dynamical systems.
PloS one , , e0197704.[56] Yu, Z. (2005). A DLM/FD method for fluid/fexible-body interactions, . , 1–27.[57] Zhu, Y., Zabaras, N., Koutsourelakis, P.-S., & Perdikaris, P. (2019). Physics-constrained deep learn-ing for high-dimensional surrogate modeling and uncertainty quantification without labeled data. Journal of Computational Physics ,394