Steven R. Brandt

A survey of high level frameworks in block-structured adaptive mesh refinement packages

Over the last decade block-structured adaptive mesh refinement (SAMR) has found increasing use in large, publicly available codes and frameworks. SAMR frameworks have evolved along different paths. Some have stayed focused on specific domain areas, others have pursued a more general functionality, providing the building blocks for a larger variety of applications. In this survey paper we examine a representative set of SAMR packages and SAMR-based codes that have been in existence for half a decade or more, have a reasonably sized and active user base outside of their home institutions, and are publicly available. The set consists of a mix of SAMR packages and application codes that cover a broad range of scientific domains. We look at their high-level frameworks, their design trade-offs and their approach to dealing with the advent of radical changes in hardware architecture. The codes included in this survey are BoxLib, Cactus, Chombo, Enzo, FLASH, and Uintah. A survey of mature openly available state-of-the-art structured AMR libraries and codes.Discussion of their frameworks, challenges and design trade-offs.Directions being pursued by the codes to prepare for the future many-core and heterogeneous platforms.

From physics model to results: An optimizing framework for cross-architecture code generation

Starting from a high-level problem description in terms of partial differential equations using abstract tensor notation, the Chemora framework discretizes, optimizes, and generates complete high performance codes for a wide range of compute architectures. Chemora extends the capabilities of Cactus, facilitating the usage of large-scale CPU/GPU systems in an efficient manner for complex applications, without low-level code tuning. Chemora achieves parallelism through MPI and multi-threading, combining OpenMP and CUDA. Optimizations include high-level code transformations, efficient loop traversal strategies, dynamically selected data and instruction cache usage strategies, and JIT compilation of GPU code tailored to the problem characteristics. The discretization is based on higher-order finite differences on multi-block domains. Chemoras capabilities are demonstrated by simulations of black hole collisions. This problem provides an acid test of the framework, as the Einstein equations contain hundreds of variables and thousands of terms.

Report on the Third Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE3).

This report records and discusses the Third Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE3). The report includes a description of the keynote presentation of the workshop, which served as an overview of sustainable scientific software. It also summarizes a set of lightning talks in which speakers highlighted to-the-point lessons and challenges pertaining to sustaining scientific software. The final and main contribution of the report is a summary of the discussions, future steps, and future organization for a set of self-organized working groups on topics including developing pathways to funding scientific software; constructing useful common metrics for crediting software stakeholders; identifying principles for sustainable software engineering design; reaching out to research software organizations around the world; and building communities for software sustainability. For each group, we include a point of contact and a landing page that can be used by those who want to join that group’s future activities. The main challenge left by the workshop is to see if the groups will execute these activities that they have scheduled, and how the WSSSPE community can encourage this to happen.

CaKernel --A parallel application programming framework for heterogenous computing architectures

With the recent advent of new heterogeneous computing architectures there is still a lack of parallel problem solving environments that can help scientists to use easily and efficiently hybrid supercomputers. Many scientific simulations that use structured grids to solve partial differential equations in fact rely on stencil computations. Stencil computations have become crucial in solving many challenging problems in various domains, e.g., engineering or physics. Although many parallel stencil computing approaches have been proposed, in most cases they solve only particular problems. As a result, scientists are struggling when it comes to the subject of implementing a new stencil-based simulation, especially on high performance hybrid supercomputers. In response to the presented need we extend our previous work on a parallel programming framework for CUDA --CaCUDA that now supports OpenCL. We present CaKernel --a tool that simplifies the development of parallel scientific applications on hybrid systems. CaKernel is built on the highly scalable and portable Cactus framework. In the CaKernel framework, Cactus manages the inter-process communication via MPI while CaKernel manages the code running on Graphics Processing Units GPUs and interactions between them. As a non-trivial test case we have developed a 3D CFD code to demonstrate the performance and scalability of the automatically generated code.

Simplifying complex software assembly: the component retrieval language and implementation

Assembling simulation software along with the associated tools and utilities is a challenging endeavor, particularly when the components are distributed across multiple source code versioning systems. It is problematic for researchers compiling and running the software across many different supercomputers, as well as for novices in a field who are often presented with a bewildering list of software to collect and install. In this paper, we describe a language (CRL) for specifying software components with the details needed to obtain them from source code repositories. The language supports public and private access. We describe a tool called Get Components which implements CRL and can be used to assemble software. We demonstrate the tool for application scenarios with the Cactus Framework on the NSF TeraGrid resources. The tool itself is distributed with an open source license and freely available from our web page.

Using GPU's to accelerate stencil-based computation kernels for the development of large scale scientific applications on heterogeneous systems

We present CaCUDA - a GPGPU kernel abstraction and a parallel programming framework for developing highly efficient large scale scientific applications using stencil computations on hybrid CPU/GPU architectures. CaCUDA is built upon the Cactus computational toolkit, an open source problem solving environment designed for scientists and engineers. Due to the flexibility and extensibility of the Cactus toolkit, the addition of a GPGPU programming framework required no changes to the Cactus infrastructure, guaranteeing that existing features and modules will continue to work without modification. CaCUDA was tested and benchmarked using a 3D CFD code based on a finite difference discretization of Navier-Stokes equations.

Piraha: A simplified grammar parser for component little languages

Software codes in scientific computing often implement their own little languages for expressing configuration data, interface definitions, and runtime parameters. Such languages are of particular importance for component-based frameworks. These languages can initially be somewhat ad-hoc and then expand organically.

Dynamic deployment of a component framework with the Ubiqis system

Component frameworks provide one strategy for the development and deployment of complex multiphysics applications which are increasingly developed by collaborative, distributed, and diverse teams, and are deployed across multiple supercomputer architectures located at numerous sites. One longstanding issue is the management of components. The complexity involved in choosing, locating, compiling, verifying, and deploying codes consisting of hundreds of independent modules can be overwhelming for beginners, and is a huge burden on developers and users. In the case of Cactus, framework-specific tools have been developed to address some of these issues, but it still suffers from being somewhat ad-hoc, difficult to maintain, and lacking in capabilities. This paper describes a dynamic framework for deploying applications that we are developing, called Ubiqis, which works automatically and on-demand. It is decentralized, requires no per-application configuration by administrative staff, and can work recursively to resolve dependencies.

Beowulf bootcamp: teaching local high schools about HPC

The Beowulf Bootcamp is an initiative designed to raise the awareness of and interest in high performance computing in the high schools the area of Baton Rouge, Louisiana. The goal is to familiarize students with all aspects of a supercomputer, giving them hands-on experience with touching or assembling hardware components. No less significant to the High Performance Computing adventure was an understanding of the software. Students not only installed the operating system and ran benchmarks, but they learned how to program. Our goal for the programming section was ambitious but focused: we sought to give the students a basic understanding of MPI.

Runtime analysis tools for parallel scientific applications

This paper describes the Alpaca runtime tools. These tools leverage the component infrastructure of the Cactus Framework in a novel way to enable runtime steering, monitoring, and interactive control of a simulation. Simulation data can be observed graphically, or by inspecting values of variables. When GPUs are available, images can be generated using volume ray casting on the live data. In response to observed error conditions or automatic triggers, users can pause the simulation to modify or repair data, or change runtime parameters. In this paper we describe the design of our implementation of these features and illustrate their value with three use cases.