Xiaoguang Ren

Coupling Strategies Investigation of Hybrid Atomistic-Continuum Method Based on State Variable Coupling

Different configurations of coupling strategies influence greatly the accuracy and convergence of the simulation results in the hybrid atomistic-continuum method. This study aims to quantitatively investigate this effect and offer the guidance on how to choose the proper configuration of coupling strategies in the hybrid atomistic-continuum method. We first propose a hybrid molecular dynamics- (MD-) continuum solver in LAMMPS and OpenFOAM that exchanges state variables between the atomistic region and the continuum region and evaluate different configurations of coupling strategies using the sudden start Couette flow, aiming to find the preferable configuration that delivers better accuracy and efficiency. The major findings are as follows: the region plays the most important role in the overlap region and the “4-layer-1” combination achieves the best precision with a fixed width of the overlap region; the data exchanging operation only needs a few sampling points closer to the occasions of interactions and decreasing the coupling exchange operations can reduce the computational load with acceptable errors; the nonperiodic boundary force model with a smoothing parameter of 0.1 and a finer parameter of 20 can not only achieve the minimum disturbance near the MD-continuum interface but also keep the simulation precision.

A New Method to Simulate Free Surface Flows for Viscoelastic Fluid

Free surface flows arise in a variety of engineering applications. To predict the dynamic characteristics of such problems, specific numerical methods are required to accurately capture the shape of free surface. This paper proposed a new method which combined the Arbitrary Lagrangian-Eulerian (ALE) technique with the Finite Volume Method (FVM) to simulate the time-dependent viscoelastic free surface flows. Based on an open source CFD toolbox called OpenFOAM, we designed an ALE-FVM free surface simulation platform. In the meantime, the die-swell flow had been investigated with our proposed platform to make a further analysis of free surface phenomenon. The results validated the correctness and effectiveness of the proposed method for free surface simulation in both Newtonian fluid and viscoelastic fluid.

WI-USHER: A grid-based parallel algorithm for particle insertion in hybrid atomistic-continuum method

The hybrid atomistic-continuum coupling method based on domain decomposition serves as an important tool for the micro-fluid simulation. There exists a certain degree of parallelism load imbalance when directly using the USHER algorithm in the domain decomposition–based hybrid atomistic-continuum coupling method. In this article, we propose a grid-based parallel algorithm for particle insertion, named WI-USHER, to improve the efficiency of the particle insertion operation when restricting the size of the region to be inserted or with higher number density. The WI-USHER algorithm slices the region to be inserted into finer grids with proper spacing scale, marks parts of finer grids in black according to three exclusive rules, that is, Single Particle Occupation (SPO), Single Particle Coverage (SPC), and Multi-Particles Coverage (MPC), and finds the target insertion point in the remained white grids. We use two test cases to show the superiority of our WI-USHER algorithm over the USHER algorithm. The WI-USHER algorithm performs lower averaged force evaluation times, which decreases from O ( 10 4 ) to O ( 10 3 ) compared to the USHER algorithm when the number density of slightly high to high value. The percentage of the total parallel simulation time processed by the particle insertion operation decreases from 23.5% to 3% compared to the USHER algorithm.

DMRPar: A Dynamic Mesh Repartitioning Scheme for Dam Break Simulations in OpenFOAM

For parallel dam break simulations in OpenFOAM (Open Source Field Operation and Manipulation), the core procedure is solving linear systems using iterative methods and the iterative convergence rate is significant to the overall efficiency. A dynamic mesh repartitioning scheme DMRPar (Dynamic Mesh Re-Partitioning) considering the iterative convergence feature is implemented in OpenFOAM. Given that the numerical characteristics of linear systems change a lot along with the complex flow field, DMRPar takes linear system information from the previous timestep into account for the repartitioning at the current timestep. The implementation reuses current mesh topology in OpenFOAM and calculates distributed adjacency graph structure for the mesh. The repartitioning heuristic is based on a general multi-level parallel graph partitioning package called ParMetis. Numerical results on two typical dam break simulations show that DMRPar outperforms the traditional static partitioning method significantly in the total simulation time.

A Hybrid Decomposition Parallel Algorithm for Multi-scale Simulation of Viscoelastic Fluids

The method of Brownian configuration fields (BCF) is a promising multi-scale approach for the simulationof viscoelastic fluids, however, it is a computationally expensive method, which restricts its application in complex scenarios. Therefore, it is of great importance to optimize the parallel implementation in order to improve computational efficiency. In this paper we propose a hybrid decomposition parallel algorithm named: MCDPar, whichenables the simulation problem to bedecomposed simultaneously over mesh cells andthe Brownian configuration fields. Compution processes are split into multiple groups and Brownian configuration fields are equally associated with these groups. Meanwhile, within each group the processes are concurrently executed based on the traditional mesh decomposition approach. Finally we implemented the MCDPar algorithm in a micro-macro numerical solver based on OpenFOAM. Experimental results show that the micro-macro simulation time of viscoelastic fluids issignificantly reduced with improved scalability and parallel efficiency. In the test case with Nf = 2000 and Ncell = 262144, the speedup of the MCDParis up to 9.23x with a 7.5x increase in number of cores compared to the original parallel algorithm.

Using multigrid methods in CFD simulations

This paper studies two typical ways to use multigrid methods in CFD simulations. One is implementing multigrid methods in CFD softwares start from scratch, and the other is calling multigrid methods from existing libraries. The experimental results on two benchmark cases, the lid-driven cavity flow and the flow around a cylinder, show that the solving performance and parallel scalability of calling multigrid methods from third-party libraries are much better than that of using the multigrid methods implemented in CFD softwares. The result not only provides some guidances for users who try to simulate with multigrid methods, but also gives a possible recommendation for the development and application of softwares for exascale systems. It is a more clever and efficient way to develop softwares on massively parallel systems based on some existing packages and libraries, which reduces the degree of coupling between different functional modules and improves the development efficiency.

The way to develop software towards exascale computing

This paper preliminarily discusses how to develop software towards exascale computing. Two typical development models are studied, one is “all-round contract” which means all the functional modules are implemented in a single framework, and the other is “co-design” which using the existing packages to realize functional modules. As a widely used CFD software whose numerical solving module is implemented in a typical “all-round contract” manner, OpenFOAM is concerned in this work. After redesigning and reimplementing the solving module of OpenFOAM in a “co-design” way by inserting PETSc, the source lines of the code decrease dramatically, which improves the development efficiency. Tests of two CFD benchmark cases and a practical large-scale case on a 128-node cluster show that the newly implemented numerical module also has a higher solving efficiency, compared with the original numerical module in OpenFOAM. Therefore, it is recommended to develop software in a “co-design” manner towards exascale computing, and softwares already implemented in “all-round contract” way should also be reconsidered.

A multi-user performance analysis framework for CFD simulations

This paper proposes a multi-user performance analysis framework for computational fluid dynamics (CFD) simulations. Independent performance datasets are given according to different requirements of computer programmers, algorithm developers and application developers. The performance analysis framework is designed based on the general procedure of CFD processing: pre-processing, problem solving and post-processing. Optimisation of the data acquisition is carried out based on data dependence to realise lightweight. A profiler is implemented in OpenFOAM based on the framework, and experiments are conducted to verify the performance information extraction and analysis. The results on a subsystem of Tianhe-1A indicate that the multiuser performance tool can probe the performance information successfully during the parallel execution. And from the performance results, we found that the direction of mesh partition turns out to be a critical factor of simulation performance for CFD problems.

A Parallel RBF Mesh Deformation Method with Multi-greedy Algorithm in OpenFOAM

Radial Basis Function(RBF) mesh deformation method has been widely used in CFD simulations with moving boundaries due to its high robustness and accuracy. The original implementation of the RBF mesh deformation method in OpenFOAM(a widely used CFD software) is purely serial with relatively low computational performance. To reduce the time cost of the mesh motion in large-scale simulations, this paper proposes a parallel RBF mesh deformation method with multi-greedy algorithm in OpenFOAM. The proposed multi- greedy method could reduce the control points used by the RBF interpolation on both the moving boundary and the static boundary, which is more applicable than the previous typical greedy algorithm. Based on a master-worker algorithm, the computation of the mesh deformation is highly parallelized. Tests on the benchmark of a three-dimensional moving fish show that with an error tolerance of 1e-4, the interpolation time of the internal mesh motion using our multi-greedy method is about 10.2 times faster than the original one, and with a parallelism of 132, the time cost of the whole mesh motion is greatly reduced with a speedup of 37.

A Hybrid Parallel Algorithm for Solving Eeuler Equation Using Explicit RKDG Method Based on OpenFOAM

OpenFOAM is a framework of the open source C CFD toolbox for flexible engineering simulation, which uses finite volume method (FVM) in the discretization of partial differential equations (PDEs). The problem solving procedure in OpenFOAM consists in equations dicretization stage, equations solving stage and field limiting stage. The best parallelism is limited by the equation solving stage, which contains communications. Compared to FVM, discontinuous Galerkin (DG) method is a high-order discretization method, which can accelerate the convergence of the residuals over same mesh scale and has higher resolution of the flow. Based on OpenFOAM with DG method, the ratio of overhead in equations discretization stage increases, especially when solving Euler equations using an explicit method. The equations discretization stage has a better potential parallelism than the other two stages due to no existence of communication. In this paper, we will analysis the difference of time cost in these three stages between original OpenFOAM and OpenFOAM with DG method. By decoupling these three stages, a hybrid parallel algorithm for solving PDEs is proposed and implemented based on OpenFOAM with DG method. The experimental results show that the simulation time is reduced by 16%, and the relative speedup of the hybrid parallel algorithm is up to 2.88 compared to the original parallel algorithm with the same degree of parallelism.