Terry J. Ligocki

Extraction of crack-free isosurfaces from adaptive mesh refinement data

Adaptive mesh refinement (AMR) is a numerical simulation technique used in computational fluid dynamics (CFD). It permits the efficient simulation of phenomena characterized by substantially varying scales in complexity of local behavior of certain variables. By using a set of nested grids at different resolutions, AMR combines the simplicity of structured rectilinear grids with the possibility to adapt to local changes in complexity and spatial resolution. Hierarchical representations of scientific data pose challenges when isosurfaces are extracted. Cracks can arise at the boundaries between regions represented at different resolutions. We present a method for the extraction of isosurfaces from AMR data that avoids cracks at the boundaries between levels of different resolution.

Performance and scaling of locally-structured grid methods for partial differential equations

In this paper, we discuss some of the issues in obtaining high performance for block-structured adaptive mesh refinement software for partial differential equations. We show examples in which AMR scales to thousands of processors. We also discuss a number of metrics for performance and scalability that can provide a basis for understanding the advantages and disadvantages of this approach.

An adaptive mesh semi-implicit conservative unsplit method for resistive MHD

We present a cell-centered semi-implicit algorithm for solving the equations of single fluid resistive MHD for block structured adaptive meshes. The unsplit method [1] is extended for the ideal MHD part, and the diffusive terms are solved implicitly. The resulting second-order accurate scheme is conservative while preserving the ∇ · B = 0 constraint. Numerical results from a variety of verification tests are presented.

Scalability challenges for massively parallel AMR applications

PDE solvers using Adaptive Mesh Refinement on block structured grids are some of the most challenging applications to adapt to massively parallel computing environments. We describe optimizations to the Chombo AMR framework that enable it to scale efficiently to thousands of processors on the Cray XT4. The optimization process also uncovered OS-related performance variations that were not explained by conventional OS interference benchmarks. Ultimately the variability was traced back to complex interactions between the application, system software, and the memory hierarchy. Once identified, software modifications to control the variability improved performance by 20% and decreased the variation in computation time across processors by a factor of 3. These newly identified sources of variation will impact many applications and suggest new benchmarks for OS-services be developed.

Embedded boundary grid generation using the divergence theorem, implicit functions, and constructive solid geometry

To construct finite-volume methods for PDEs in arbitrary dimension to arbitrary accuracy in the presence of irregular boundaries, we show that estimates of moments, integrals of monomials, over various regions are all that are needed. If implicit functions are used to represent the irregular boundary, the needed moments can be computed straightforwardly and robustly by using the divergence theorem, Taylor expansions, least squares, recursion, and 1D root finding. Neither a geometric representation of the irregular boundary nor its interior is ever needed or computed. The implicit function representation is general and robust. Implicit functions can be combined via constructive solid geometry to form complex boundaries from a rich set of primitives including interpolants of sampled data, for example, 2D/3D image data and digital elevation maps.

Virtual-Reality Based Interactive Exploration of Multiresolution Data

We describe a system supporting the interactive exploration of threedimensional scientific data sets in a virtual reality (VR) environment. This system aids a scientist in understanding a data set by interactively placing and manipulating visualization primitives, e. g., isosurfaces or streamlines, and thereby finding features in the data and understanding its overall structure.

Parallelization of Radiance For Real Time Interactive Lighting Visualization Walkthroughs

Radiance is a software package developed at Lawrence Berkeley National Laboratory for lighting visualization. Lighting visualization predicts how lighting would appear if a modelled scene were to be physically realized. Unlike most lighting systems, Radiance physically models the effects of lighting, providing an image that is closer to physical reality. This is of obvious benefit to architects and lighting designers. Such visualizations are computationally expensive: rendering a single image can take hours on a standard workstation environment. Ideally, an architect would like to be able to interactively navigate through a scene to get a full impression of the true appearance of a particular model. With this goal in mind, we have (1) developed a geometric-based method (point cloud) to reuse pixels from a previous frame and (2) developed a parallel, distributed memory implementation of Radiance and the point cloud using MPI for inter-processor communication.

A normalized-cut algorithm for hierarchical vector field data segmentation

In the context of vector field data visualization, it is often desirable to construct a hierarchical data representation. One possibility to construct a hierarchy is based on clustering vectors using certain similarity criteria. We combine two fundamental approaches to cluster vectors and construct hierarchical vector field representations. For clustering, a locally constructed linear least-squares approximation is incorporated into a similarity measure that considers both Euclidean distance between point pairs (for which dependent vector data are given) and difference in vector values. A modified normalized cut (NC) method is used to obtain a near-optimal clustering of a given discrete vector field data set. To obtain a hierarchical representation, the NC method is applied recursively after the construction of coarse-level clusters. We have applied our NC-based segmentation method to simple, analytically defined vector fields as well as discrete vector field data generated by turbulent flow simulation. Our test results indicate that our proposed adaptation of the original NC method is a promising method as it leads to segmentation results that capture the qualitative and topological nature of vector field data.

Embedded boundary algorithms and software for partial differential equations

In this paper, we give an overview of a set of methods being developed for solving classical PDEs in irregular geometries, or in the presence of free boundaries. In this approach, the irregular geometry is represented on a rectangular grid by specifying the intersection of each grid cell with the region on one or the other side of the boundary. This leads to a natural conservative discretization of the solution to the PDE on either side of the boundary. Stable and robust hyperbolic and linear elliptic/parabolic solvers have been designed and implemented. Example applications of this approach are shown for compressible and incompressible gas dynamics problems in complex geometries, and for surface diffusion in a cell membrane.

Parallel In Situ Detection of Connected Components in Adaptive Mesh Refinement Data

Adaptive Mesh Refinement (AMR) represents a significant advance for scientific simulation codes, greatly reducing memory and compute requirements by dynamically varying simulation resolution over space and time. As simulation codes transition to AMR, existing analysis algorithms must also make this transition. One such algorithm, connected component detection, is of vital importance in many simulation and analysis contexts, with some simulation codes even relying on parallel, in situ connected component detection for correctness. Yet, current detection algorithms designed for uniform meshes are not applicable to hierarchical, non-uniform AMR, and to the best of our knowledge, AMR connected component detection has not been explored in the literature. Therefore, in this paper, we formally define the general problem of connected component detection for AMR, and present a general solution. Beyond solving the general detection problem, achieving viable in situ detection performance is even more challenging. The core issue is the conflict between the communication-intensive nature of connected component detection (in general, and especially for AMR data) and the requirement that in situ processes incur minimal performance impact on the co-located simulation. We address this challenge by presenting the first connected component detection methodology for structured AMR that is applicable in a parallel, in situ context. Our key strategy is the incorporation of an multi-phase AMR-aware communication pattern that synchronizes connectivity information across the AMR hierarchy. In addition, we distil our methodology to a generic framework within the Combo AMR infrastructure, making connected component detection services available for many existing applications. We demonstrate our methods efficacy by showing its ability to detect ice calving events in real time within the real-world BISICLES ice sheet modelling code. Results show up to a 6.8x speedup of our algorithm over the existing specialized BISICLES algorithm. We also show scalability results for our method up to 4,096 cores using a parallel Combo-based benchmark.