Douglass E. Post

Software Development Environments for Scientific and Engineering Software: A Series of Case Studies

The need for high performance computing applications for computational science and engineering projects is growing rapidly, yet there have been few detailed studies of the software engineering process used for these applications. The DARPA High Productivity Computing Systems Program has sponsored a series of case studies of representative computational science and engineering projects to identify the steps involved in developing such applications (i.e. the life cycle, the workflows, technical challenges, and organizational challenges). Secondary goals were to characterize tool usage and identify enhancements that would increase the programmers productivity. Finally, these studies were designed to develop a set of lessons learned that can be transferred to the general computational science and engineering community to improve the software engineering process used for their applications. Nine lessons learned from five representative projects are presented, along with their software engineering implications, to provide insight into the software development environments in this domain.

Computational Science Demands a New Paradigm

The field has reached a threshold at which better organization becomes crucial. New methods of verifying and validating complex codes are mandatory if computational science is to fulfill its promise for science and society.

Software Project Management and Quality Engineering Practices for Complex, Coupled Multiphysics, Massively Parallel Computational Simulations: Lessons Learned From ASCI

Many institutions are now developing large-scale, complex, coupled multiphysics computational simulations for massively parallel platforms for the simulation of the performance of nuclear weapons and certification of the stockpile, and for research in climate and weather prediction, magnetic and inertial fusion energy, environmental systems, astrophysics, aerodynamic design, combustion, biological and biochemical systems, and other areas. The successful development of these simulations is aided by attention to sound software project management and software engineering. We have developed “lessons learned” from a set of code projects that the Department of Energy National Nuclear Security Agency has sponsored to develop nuclear weapons simulations over the last 50 years. We find that some, but not all, of the software project management and development practices (rather than processes) commonly employed for non-technical software add value to the development of scientific software and we identify those that we judge add value. Another key finding, consistent with general software industry experience, is that the optimal project schedule and resource level are solely determined by the requirements once the requirements are fixed.

Development of a Weather Forecasting Code: A Case Study

Computational science is increasingly supporting advances in scientific and engineering knowledge. The unique constraints of these types of projects result in a development process that differs from the process more traditional information technology projects use. This article reports the results of the sixth case study conducted under the support of the Darpa High Productivity Computing Systems Program. The case study aimed to investigate the technical challenges of code development in this environment, understand the use of development tools, and document the findings as concrete lessons learned for other developers benefit. The project studied here is a major component of a weather forecasting system of systems. It includes complex behavior and interaction of several individual physical systems (such as the atmosphere and the ocean). This article describes the development of the code and presents important lessons learned.

HPC needs a tool strategy

The High Productivity Computing Systems (HPCS) program seeks a tenfold productivity increase in High Performance Computing (HPC). A change of this magnitude in software development and maintenance demands a transformation similar to other great leaps in industrial productivity. By analogy, this requires a dramatic change to the infrastructure and to the way software developers use it. Software tools such as compilers, libraries, debuggers and analyzers constitute an essential part of the HPC infrastructure, without which codes cannot be efficiently developed nor production runs accomplished.The underappreciated HPC software infrastructure is not up to the task and is becoming less so in the face of increasing scale, complexity, and mission importance. Infrastructure dependencies are seen as significant risks to success, and significant productivity gains remain unrealized. Support models for this infrastructure are not aligned with its strategic value.To achieve the potential of the software infrastructure, both for stability and for productivity breakthroughs, a dedicated, long-term, client-focused support structure must be established. Goals for tools in the infrastructure would include ubiquity, portability, and longevity commensurate with the projects they support, typically decades. The strategic value of such an infrastructure necessarily transcends individual projects, laboratories, and organizations.

Case study of the Falcon code project

The field of computational science is growing rapidly. Yet there have been few detailed studies of the development processes for high performance computing applications. As part of the High Productivity Computing Systems (HPCS) program we are conducting a series of case studies of representative computational science projects to identify the steps involved in developing such applications, including the life cycle, workflows and tasks, and technical and organizational challenges. We are seeking to identify how software development tools are used and the enhancements that would increase the productivity of code developers. The studies are also designed to develop a set of lessons learned that can be transferred to the general computational science community to improve the code development process. We have carried a detailed study of the Falcon (Fig.1) code project. That project is located at a large institution under contract to a national sponsor. The project team consisted of about 15 scientists charged with developing a multi-physics simulation that would utilize large-scale supercomputers with 1000s of processors. The expected life time of the code project is about 30 years. The case study findings reinforced the importance of sound software project management and the challenges associated with verification and validation.

Highlights of the CREATE Program

The Computational Research and Engineering Acquisition Tools and Environments (CREATE) Program was established as a new 12-year program in FY 2008 by the Department of Defense (DoD). The CREATE goal is to enable major improvements in DoDs acquisition engineering design and analysis processes by developing and deploying scalable, multi-disciplinary, physics-based computational engineering software products for the design and analysis of DoD Ships, Air Vehicles, and Radio Frequency Antennas. Meshing and Geometry (MG) generation is being provided by a fourth project, MG. CREATE is a multi-institutional, multi-service, multi-agency and multi-disciplinary program with participation by the Navy, Air Force, Army, the Office of the Secretary of Defense, industry, and academia. The CREATE products are being developed and released on an annual cycle. In 2010, the program released five new products: SENTRI 1.0 -- RF antenna design; NESM 0.1 -- Ship Shock Analysis; IHDE 1.0 -- Ship Hydrodynamic Design and Analysis; Kestrel 1.0 -- Fixed-wing air vehicle analysis; and Helios 1.0 -- Rotorcraft analysis. Enhanced versions of these products will be released every year starting in 2011. In 2011, five additional products will begin annual releases: DaVinci -- a tool for the rapid physics-based design of air vehicles; RDI -- an integrated suite of tools to enable rapid physics-based design of naval ships; Firebolt -- components to provide models for gas turbine propulsion systems for Kestrel and Helios; NavyFoam -- a high- fidelity hydrodynamics analysis tool for predicting drag and resistance, sea-keeping and seaway loads; and Capstone -- components to enable the generation of geometries and meshes for all of the other products. The CREATE products are designed to be modular, maintainable, extensible, and scalable. To accomplish this, the CREATE team1 has developed a set of software engineering and software project management practices and processes that strike the appropriate balance between the agility and flexibility, and organizational structures and planning that are appropriate for developing complex physics-based, scalable and sustainable engineering software.

Guest Editor's Introduction: Computational Science and Engineering for the US Department of Defense

The US Department of Defense is reducing its dependence on the traditional design, build, break, and fix paradigm for designing and testing weapons systems by supplementing empirical testing with computational science and engineering. This introduction to the theme issue explores how the DoDs computational systems have contributed to various scientific works.

Guest Editors' Introduction: Verification and Validation in Computational Science and Engineering

An encompassing goal of contemporary scientific computing is to provide quantitatively accurate predictions that can help society make important decisions. The span of this intended influence includes such widely different fields as astrophysics, weather and climate forecasting, quantitative economic policy, environmental regulation, and performance certification of complex engineered systems such as nuclear power plants. To the degree that we believe accurate computational science and engineering (CSE) will have an increasingly greater impact on problems of societal importance, we must also be concerned about the consequences of inaccurate or wrong CSE. Human life need not necessarily be at risk, but it is highly likely that money, time, environmental quality, and other factors will be.

The Future of Computing Performance

A new study explains why the days of obtaining performance increases due to higher processor speed are mostly over, and where we go from here.