Cameron W. Smith

Scalable Implicit Flow Solver for Realistic Wing Simulations with Flow Control

Massively parallel computation provides an enormous capacity to perform simulations on a timescale that can change the paradigm of how scientists, engineers, and other practitioners use simulations to address discovery and design. This work considers an active flow control application on a realistic and complex wing design that could be leveraged by a scalable, fully implicit, unstructured flow solver and access to high-performance computing resources. The article describes the active flow control application; then summarizes the main features in the implementation of a massively parallel turbulent flow solver, PHASTA; and finally demonstrates the methods strong scalability at extreme scale. Scaling studies performed with unstructured meshes of 11 and 92 billion elements on the Argonne Leadership Computing Facilitys Blue Gene/Q Mira machine with up to 786,432 cores and 3,145,728 MPI processes.

PUMI: Parallel Unstructured Mesh Infrastructure

The Parallel Unstructured Mesh Infrastructure (PUMI) is designed to support the representation of, and operations on, unstructured meshes as needed for the execution of mesh-based simulations on massively parallel computers. In PUMI, the mesh representation is complete in the sense of being able to provide any adjacency of mesh entities of multiple topologies in O(1) time, and fully distributed to support relationships of mesh entities across multiple memory spaces in a manner consistent with supporting massively parallel simulation workflows. PUMIs mesh maintains links to the high-level model definition in terms of a model topology as produced by CAD systems, and is specifically designed to efficiently support evolving meshes as required for mesh generation and adaptation. To support the needs of parallel unstructured mesh simulations, PUMI also supports a specific set of services such as the migration of mesh entities between parts while maintaining the mesh adjacencies, maintaining read-only mesh entity copies from neighboring parts (ghosting), repartitioning parts as the mesh evolves, and dynamic mesh load balancing. Here we present the overall design, software structures, example programs, and performance results. The effectiveness of PUMI is demonstrated by its applications to massively parallel adaptive simulation workflows.

Bringing HPC to Engineering Innovation

Although theres a widespread belief that the effective application of high-performance computing will dramatically increase industrial innovation, progress in this area has been slow and limited because of a combination of technical and economic impediments. Here, such impediments are outlined, along with efforts to address them.

Improving Unstructured Mesh Partitions for Multiple Criteria Using Mesh Adjacencies

The scalability of unstructured mesh-based applications depends on partitioning methods that quickly balance the computational work while reducing communication costs. Zhou et al. [SIAM J. Sci. Com...

A Parallel Unstructured Mesh Infrastructure

Two Department of Energy (DOE) office of Sciences Scientific Discovery through Advanced Computing (SciDAC) Frameworks, Algorithms, and Scalable Technologies for Mathematics (FASTMath) software packages, Parallel Unstructured Mesh Infrastructure (PUMI) and Partitioning using Mesh Adjacencies (ParMA), are presented.

In-memory Integration of Existing Software Components for Parallel Adaptive Unstructured Mesh Workflows

Reliable mesh-based simulations are needed to solve complex engineering problems. Mesh adaptivity can increase reliability by reducing discretization errors, but requires multiple software components to exchange information. Often, components exchange information by reading and writing a common file format. This file-based approach becomes a problem on massively parallel computers where filesystem bandwidth is a critical performance bottleneck. Our data stream and component interface approaches avoid the filesystem bottleneck. In this paper we present our approaches and their use within the PHASTA computational fluid dynamics solver and Albany multiphysics framework. Information exchange performance results are reported on up to 2048 cores of a BlueGene/Q system.

Enabling HPC simulation workflows for complex industrial flow problems

The use of simulation based engineering taking advantage of massively parallel computing methods by industry is limited due to the costs associated with developing and using high performance computing software and systems. To address industries ability to effectively include large-scale parallel simulations in daily production use, two key areas need to be addressed. The first is access to large-scale parallel computing systems that are cost effective to use. The second is support for complete simulation workflow execution on these systems by industrial users. This paper presents an approach, and set of associated software components, that can support industrial users on large-scale parallel computing systems available at various national laboratories, universities, or on clouds.

HPC Simulation Workflows for Engineering Innovation

Efforts to develop component-based simulation workflows for industrial applications using XSEDE parallel computing systems are presented.

PHASTA Science Gateway for High Performance Computational Fluid Dynamics

The Parallel Hierarchic Adaptive Stabilized Transient Analysis (PHASTA) software supports modeling compressible or incompressible, laminar or turbulent, steady or unsteady flows in 3D using unstructured grids. PHASTA has been applied to industrial and academic flows on complex, as-designed geometric models with over one billion mesh elements using upwards of one million compute cores. The PHASTA Science Gateway (phasta.scigap.org) brings these increasingly critical technologies to a larger user base by providing a central hub for simulation execution, simulation data management, and documentation. Researchers and engineers using the gateway can easily define and execute simulations on the TACC Stampede2 Skylake and Knights Landing nodes without being burdened by the details of remote access, the job scheduler, and filesystem configuration. In addition to simplifying the simulation execution process, the gateway creates a searchable archive of past jobs that can be shared with other users to support reproducibility and increase productivity. Our poster presents the construction of the gateway with Apache Airavata, the simulation definition process, applications it currently supports, and our ongoing efforts to expand functionality, the user base, and the community.

The PHASTA Science Gateway: Web-based Execution of Adaptive Computational Fluid Dynamics Simulations

The Parallel Hierarchic Adaptive Stabilized Transient Analysis (PHASTA) software supports modeling compressible or incompressible, laminar or turbulent, steady or unsteady flows in 3D using unstructured grids. PHASTA, coupled with the Parallel Unstructured Mesh Infrastructure (PUMI), supports parallel, automated, adaptive simulation workflows. Researchers can easily execute these workflows on the TACC Stampede Xeon and Knights Landing nodes without being burdened by the details of each system using the PHASTA science gateway (created with Apache Airavata). In addition to abstracting away job execution and filesystem details, the gateway creates a searchable archive of past jobs to support reproducibility. Our poster presents the construction of the PHASTA gateway, the workflows it currently supports, and our ongoing efforts to expand functionality and the user base.