Markus Flatken

Enabling In Situ Pre- and Post-processing for Exascale Hemodynamic Simulations - A Co-design Study with the Sparse Geometry Lattice-Boltzmann Code HemeLB

Todays fluid simulations deal with complex geometries and numerical data on an extreme scale. As computation approaches the exascale, it will no longer be possible to write and store the full-sized data set. In situ data analysis and scientific visualisation provide feasible solutions to the analysis of complex large scaled CFD simulations. To bring pre- and postprocessing to the exascale we must consider modifications to data structure and memory layout, and address latency and error resiliency. In this respect, a particular challenge is the exascale data processing for the sparse geometry lattice-Boltzmann code HemeLB, intended for hemodynamic simulations. In this paper, we assess the needs and challenges of HemeLB users and sketch a co-design infrastructure and system architecture for pre- and post-processing the simulation data. To enable in situ data visualisation and analysis during a running simulation, post-processing needs to work on a reduced subset of the original data. Particular choices of data structure and visualisation techniques need to be co-designed with the application scientists in order to achieve efficient and interactive data processing and analysis. In this work, we focus on the hierarchical data structure and suitable visualisation techniques which provide possible solutions to interactive in situ data processing at exascale. Architectural challenges and road-maps will be presented as the major focus of this paper. We sketch a software architecture which integrates pre- and post-processing techniques that can provide in situ analysis and ultimately computational steering to HemeLB.

Topology-based Analysis for Multimodal Atmospheric Data of Volcano Eruptions

Many scientific applications deal with data from a multitude of different sources, e.g., measurements, imaging and simulations. Each source provides an additional perspective on the phenomenon of interest, but also comes with specific limitations, e.g. regarding accuracy, spatial and temporal availability. Effectively combining and analyzing such multimodal and partially incomplete data of limited accuracy in an integrated way is challenging. In this work, we outline an approach for an integrated analysis and visualization of the atmospheric impact of volcano eruptions. The data sets comprise observation and imaging data from satellites as well as results from numerical particle simulations. To analyze the clouds from the volcano eruption in the spatiotemporal domain we apply topological methods. We show that topology-related extremal structures of the data support clustering and comparison. We further discuss the robustness of those methods with respect to different properties of the data and different parameter setups. Finally we outline open challenges for the effective integrated visualization using topological methods.

Dynamic Scheduling for Progressive Large-Scale Visualisation

The ever-increasing compute capacity of high-performance systems enables scientists to simulate physical phenomena with a high spatial and temporal accuracy. Thus, the simulation output can yield dataset sizes of many terabytes. An efficient analysis and visualization process becomes very difficult especially for explorative scenarios where users continuously change input parameters. Using a distributed rendering pipeline may relieve the visualization frontend considerably but is often not sufficient. Therefore, we additionally propose a progressive data streaming and rendering approach. The main contribution of our method is the importance-guided order of data processing for block structured datasets. This requires a dynamic scheduling of data chunks on the parallel post-processing system which has been implemented by using an R-Tree. In this paper, we demonstrate the efficiency of our implementation for view-dependent feature extraction with varying viewpoints.

In-situ processing and interactive visualization for large-scaled numerical simulations

With the increasing power of the HPC hardware systems, numerical simulations are heading towards exa-scale computing. Early inspection and analysis of on-going large simulations enables domain experts to obtain first insight into their running simulation process and intermediate results. Compared to conventional post-processing, such in-situ processing has the advantage of keeping data in memory, avoiding to store the large amount of raw data to disk, providing on-the-fly analysis, and preventing early failures in the simulation process. In this poster we present a distributed and scalable software infrastructure, which provides distributed in-situ data processing, feature extraction and interactive exploration at users front-end. We have integrated and extended our system to multiple simulation applications, ranging from Lattice-Boltzmann blood flow simulation to grid based simulation for propulsion systems. A user-interactive front-end is integrated to our system, allowing to directly interact with the visualization of running simulations, gain insight, and make decisions.

Distributed Post-processing and Rendering for Large-Scale Scientific Simulations

With the ever-increasing capacity of high performance computing (HPC) systems, the computational simulation models become still finer and more accurate. However, the size and complexity of the data produced poses tremendous challenges for the visualization and analysis task. Especially when explorative approaches are demanded, distributed and parallel post-processing architectures have to be developed in order to allow interactive human-computer interfaces. Such infrastructures can also be exploited for the evaluation of ongoing simulation runs. The application here ranges from online monitoring to computational steering. But also remote and parallel rendering can be integrated into the overall setup. This chapter gives an overview of current solutions and ongoing research activities in this domain.