Andrew C. Bauer

The ParaView Coprocessing Library: A scalable, general purpose in situ visualization library

As high performance computing approaches exascale, CPU capability far outpaces disk write speed, and in situ visualization becomes an essential part of an analysts workflow. In this paper, we describe the ParaView Coprocessing Library, a framework for in situ visualization and analysis coprocessing. We describe how coprocessing algorithms (building on many from VTK) can be linked and executed directly from within a scientific simulation or other applications that need visualization and analysis. We also describe how the ParaView Coprocessing Library can write out partially processed, compressed, or extracted data readable by a traditional visualization application for interactive post-processing. Finally, we will demonstrate the librarys scalability in a number of real-world scenarios.

Ultrascale Visualization of Climate Data

Collaboration across research, government, academic, and private sectors is integrating more than 70 scientific computing libraries and applications through a tailorable provenance framework, empowering scientists to exchange and examine data in novel ways.

In situ methods, infrastructures, and applications on high performance computing platforms

The considerable interest in the high performance computing (HPC) community regarding analyzing and visualization data without first writing to disk, i. e., in situ processing, is due to several factors. First is an I/O cost savings, where data is analyzed/visualized while being generated, without first storing to a filesystem. Second is the potential for increased accuracy, where fine temporal sampling of transient analysis might expose some complex behavior missed in coarse temporal sampling. Third is the ability to use all available resources, CPUs and accelerators, in the computation of analysis products. This STAR paper brings together researchers, developers and practitioners using in situ methods in extreme‐scale HPC with the goal to present existing methods, infrastructures, and a range of computational science and engineering applications using in situ analysis and visualization.

ParaView Catalyst: Enabling In Situ Data Analysis and Visualization

Computer simulations are growing in sophistication and producing results of ever greater fidelity. This trend has been enabled by advances in numerical methods and increasing computing power. Yet these advances come with several costs including massive increases in data size, difficulties examining output data, challenges in configuring simulation runs, and difficulty debugging running codes. Interactive visualization tools, like ParaView, have been used for post-processing of simulation results. However, the increasing data sizes, and limited storage and bandwidth make high fidelity post-processing impractical. In situ analysis is recognized as one of the ways to address these challenges. In situ analysis moves some of the post-processing tasks in line with the simulation code thus short circuiting the need to communicate the data between the simulation and analysis via storage. ParaView Catalyst is a data processing and visualization library that enables in situ analysis and visualization. Built on and designed to interoperate with the standard visualization toolkit VTK and the ParaView application, Catalyst enables simulations to intelligently perform analysis, generate relevant output data, and visualize results concurrent with a running simulation. In this paper, we provide an overview of the Catalyst framework and some of the success stories.

Beam Optics Analysis - An Advanced 3D Trajectory Code

Calabazas Creek Research, Inc. has completed initial development of an advanced, 3D program for modeling electron trajectories in electromagnetic fields. The code is being used to design complex guns and collectors. Beam Optics Analysis (BOA) is a fully relativistic, charged particle code using adaptive, finite element meshing. Geometrical input is imported from CAD programs generating ACIS-formatted files. Parametric data is inputted using an intuitive, graphical user interface (GUI), which also provides control of convergence, accuracy, and post processing. The program includes a magnetic field solver, and magnetic information can be imported from Maxwell 2D/3D and other programs. The program supports thermionic emission and injected beams. Secondary electron emission is also supported, including multiple generations. Work on field emission is in progress as well as implementation of computer optimization of both the geometry and operating parameters. The principle features of the program and its capabilities are presented.

Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures

A key trend facing extreme-scale computational science is the widening gap between computational and I/O rates, and the challenge that follows is how to best gain insight from simulation data when it is increasingly impractical to save it to persistent storage for subsequent visual exploration and analysis. One approach to this challenge is centered around the idea of in situ processing, where visualization and analysis processing is performed while data is still resident in memory. This paper examines several key design and performance issues related to the idea of in situ processing at extreme scale on modern platforms: scalability, overhead, performance measurement and analysis, comparison and contrast with a traditional post hoc approach, and interfacing with simulation codes. We illustrate these principles in practice with studies, conducted on large-scale HPC platforms, that include a miniapplication and multiple science application codes, one of which demonstrates in situ methods in use at greater than 1M-way concurrency.

HPCCloud: A Cloud/Web-Based Simulation Environment

Advanced modeling and simulation has enabled the design of a variety of innovative products and the analysis of numerous complex phenomenon. However, significant barriers exist to widespread adoption of these tools. In particular, advanced modeling and simulation: (1) is considered complex to use, (2) needs in-house expertise, and (3) requires high capital costs. In this paper, we describe the development of an end-to-end, advanced modeling and simulation cloud platform that encapsulates best practices for scientific computing in the cloud, and demonstrate using Hydra-TH as a prototypical application. As an alternative to traditional advanced modeling and simulation workflows, our Web-based approach simplifies the processes, decreases the need for in-house computational science and engineering experts, and lowers the capital investments. In addition to providing significantly improved, intuitive software, the environment offers reproducible workflows where the full lifecycle of data from input to final analyzed results can be saved, shared, and even published.

In Situ Analysis as a Parallel I/O Problem

This paper presents a combination of state-of-the-practice for extraction-based in situ analysis, the treatment of in situ analysis as a form of I/O, and a proposal to the community regarding the handling of in situ and I/O libraries. Extraction in situ analysis extracts subsets of data using certain techniques (e.g., isosurfacing) and saves only the extracted information. It was found that this approach, while reducing output, still poses parallel I/O issues. An initial solution was to rework ParaView Catalyst output to use ADIOS, though that only brought about minor improvements. Further aggregation, even for large-scale extraction I/O, was the primary key to solving that issue. Finally, the paper presents the position that a single library could provide I/O, in situ analysis, code coupling, and resiliency capabilities because of the overlap in steps necessary to bootstrap each.

3D finite element trajectory code with adaptive meshing

A new, finite element, adaptive mesh trajectory code is available for designing electron devices, including electron guns and collectors. The finite element technique provides for more efficient simulation of particles in electromagnetic fields, and the adaptive meshing insures optimal use of computational resources. The adaptive meshing removes responsibility from the user for mesh generation and allows for dramatic simplification of code operation. The code imports geometry from commercial CAD programs and includes an intuitive, user-friendly, graphical user interface and post processor.

In Situ Summarization with VTK-m

Summarization and compression at current and future scales requires a framework for developing and benchmarking algorithms. We present a framework created by integrating existing, production-ready projects and provide timings of two particular algorithms that serve as exemplars for summarization: a wavelet-based data reduction filter and a generator for creating image-like databases of extracted features (isocontours in this case). Both support browser-based, post-hoc, interactive visualization of the summary for decision-making. A study of their weak-scaling on a distributed multi-GPU system is included.