Jérôme Frisch

Communication Schemes of a Parallel Fluid Solver for Multi-scale Environmental Simulations

A lot of different environmental simulations use computational fluid dynamics for detailed airflow computation and pollution transportation. Unfortunately, a multi-scale computational fluid dynamics simulation is very time consuming and computational intensive, as a high geometric discretisation has to be chosen in order to capture all required physical phenomena, so that without any parallelisation strategies, these computations tend to be impossible to perform. In this paper, we will discuss communication schemes using the message passing paradigm implemented in a previously validated fluid simulation code. Advantages and disadvantages of the current implementation will be discussed and improvements will be proposed.

A sliding window technique for interactive high-performance computing scenarios

Interactive high-performance computing is doubtlessly beneficial for many computational science and engineering applications whenever simulation results should be visually processed in real time, i.e. during the computation process. Nevertheless, interactive HPC entails a lot of new challenges that have to be solved - one of them addressing the fast and efficient data transfer between a simulation back end and visualisation front end, as several gigabytes of data per second are nothing unusual for a simulation running on some (hundred) thousand cores. Here, a new approach based on a sliding window technique is introduced that copes with any bandwidth limitations and allows users to study both large and small scale effects of the simulation results in an interactive fashion.

Resolving Neighbourhood Relations in a Parallel Fluid Dynamic Solver

Computational Fluid Dynamics simulations require an enormous computational effort if a physically reasonable accuracy should be reached. Therefore, a parallel implementation is inevitable. This paper describes the basics of our implemented fluid solver with a special aspect on the hierarchical data structure, unique cell and grid identification, and the neighbourhood relations in-between grids on different processes. A special server concept keeps track of every grid over all processes while minimising data transfer between the nodes.

Measuring and Comparing the Scaling Behaviour of a High-Performance CFD Code on Different Supercomputing Infrastructures

Parallel code design is a challenging task especially when addressing petascale systems for massive parallel processing (MPP), i.e. parallel computations on several hundreds of thousands of cores. An in-house computational fluid dynamics code, developed by our group, was designed for such high-fidelity runs in order to exhibit excellent scalability values. Basis for this code is an adaptive hierarchical data structure together with an efficient communication and (numerical) computation scheme that supports MPP. For a detailled scalability analysis, we performed several experiments on two of Germanys national supercomputers up to 140,000 processes. In this paper, we will show the results of those experiments and discuss any bottlenecks that could be observed while solving engineering-based problems such as porous media flows or thermal comfort assessments for problem sizes up to several hundred billion degrees of freedom.

Engineering-Based Thermal CFD Simulations on Massive Parallel Systems

The development of parallel Computational Fluid Dynamics (CFD) codes is a challenging task that entails efficient parallelization concepts and strategies in order to achieve good scalability values when running those codes on modern supercomputers with several thousands to millions of cores. In this paper, we present a hierarchical data structure for massive parallel computations that supports the coupling of a Navier–Stokes-based fluid flow code with the Boussinesq approximation in order to address complex thermal scenarios for energy-related assessments. The newly designed data structure is specifically designed with the idea of interactive data exploration and visualization during runtime of the simulation code; a major shortcoming of traditional high-performance computing (HPC) simulation codes. We further show and discuss speed-up values obtained on one of Germany’s top-ranked supercomputers with up to 140,000 processes and present simulation results for different engineering-based thermal problems.

Adaptive Distributed Data Structure Management for Parallel CFD Applications

Computational fluid dynamics (CFD) simulations require a lot of computing resources in terms of CPU time and memory in order to compute with a reasonable physical accuracy. If only uniformly refined domains are applied, the amount of computing cells is growing rather fast if a certain small resolution is physically required. This can be remedied by applying adaptively refined grids. Unfortunately, due to the adaptive refinement procedures, errors are introduced which have to be taken into account. This paper is focussing on implementation details of the applied adaptive data structure management and a qualitative analysis of the introduced errors by analysing a Poisson problem on the given data structure, which has to be solved in every time step of a CFD analysis. Furthermore an adaptive CFD benchmark example is computed, showing the benefits of an adaptive refinement as well as measurements of parallel data distribution and performance.

Design and optimisation of an efficient HDF5 I/O Kernel for massive parallel fluid flow simulations

More and more massive parallel codes running on several hundreds of thousands of cores are entering the computational science and engineering domain, allowing high‐fidelity computations on up to trillions of unknowns for very detailed analyses of the underlying problems. Such runs typically produce gigabytes of data, hindering both efficient storage and (interactive) data exploration. Advanced approaches based on inherently distributed data formats such as hierarchical data format version 5 become necessary here to avoid long latencies when storing the data and to support fast (random) access when retrieving the data for visual processing. This paper shows considerations and implementation aspects of an I/O kernel based on hierarchical data format version 5 that supports fast checkpointing, restarting, and selective visualisation using a single shared output file for an existing computational fluid dynamics framework. This functionality is achieved by including the frameworks hierarchical data structure in the file, which also opens the door for additional steering functionality. Finally, the performance of the kernels write routines are presented. Bandwidths close to the theoretical peak on modern supercomputing clusters were achieved by avoiding file‐locking and using collective buffering.

Multi-scale high-performance fluid flow

Generation of randomly distributed geometrically complex porous media samples.Hierarchical data structure and fast multigrid-like solver.Computation on high-performance computing clusters (more than 100.000 processes).Multi-scale approach based on coupling of Darcy and Navier-Stokes equations.Interactive data exploration during the runtime of the simulation. Computational fluid dynamic (CFD) calculations on geometrically complex domains such as porous media require high geometric discretisation for accurately capturing the tested physical phenomena. Moreover, when considering a large area and analysing local effects, it is necessary to deploy a multi-scale approach that is both memory-intensive and time-consuming. Hence, this type of analysis must be conducted on a high-performance parallel computing infrastructure. In this paper, the coupling of two different scales based on the Navier-Stokes equations and Darcys law is described followed by the generation of complex geometries, and their discretisation and numerical treatment. Subsequently, the necessary parallelisation techniques and a rather specific tool, which is capable of retrieving data from the supercomputing servers and visualising them during the computation runtime (i.e. in situ) are described. All advantages and possible drawbacks of this approach, together with the preliminary results and sensitivity analyses are discussed in detail.

Simulation-Based Monitoring Analysis of Air-Source Domestic Hot Water Heat Pumps

The following research takes particular merit from long-term monitoring data of eight identical built-in compact air-to-water heat pump systems. Occupantrelated domestic hot water consumption data were therefore analyzed in different German households within the same building envelope and boundary conditions. Research progress is made based on realistic domestic hot water profiles. Measurement uncertainties as well as influencing system parameters are quantified and a reliable and validated simulation model is provided. The obtained results depict significant deviations from standardized and deterministic calculation methods. Efficient operation was found to be case-sensitive and limited by storage tank volumes and thermal storage behavior. This approach documents these findings with respect to analyzed and simulated occupant consumption data and thus aims to enhance further building performance predictions.

Massive Parallel Fluid Flow Simulations Using Hierarchical Data Format Version 5 (HDF5)

More and more massive parallel codes running on several hundreds of thousands of cores enter the computational science and engineering domain, allowing high-fidelity computations up to trillions of unknowns for very detailed analyses of the underlying problems. During such runs, typically gigabytes of data are being produced, hindering both efficient storage and (interactive) data exploration. Here, advanced approaches based on inherently distributed data formats such as HDF5 become necessary in order to avoid long latencies when storing the data and to support fast (random) access when retrieving the data for visual processing. Avoiding file locking and using collective buffering, we achieved write bandwiths to a single file close to the theoretical peak on a modern supercomputing cluster. The structure of our output file supports a very fast interactive visualisation and introduces additional steering functionality.