Jean Utke

OpenAD/F: A Modular Open-Source Tool for Automatic Differentiation of Fortran Codes

The Open/ADF tool allows the evaluation of derivatives of functions defined by a Fortran program. The derivative evaluation is performed by a Fortran code resulting from the analysis and transformation of the original program that defines the function of interest. Open/ADF has been designed with a particular emphasis on modularity, flexibility, and the use of open source components. While the code transformation follows the basic principles of automatic differentiation, the tool implements new algorithmic approaches at various levels, for example, for basic block preaccumulation and call graph reversal. Unlike most other automatic differentiation tools, Open/ADF uses components provided by the Open/AD framework, which supports a comparatively easy extension of the code transformations in a language-independent fashion. It uses code analysis results implemented in the OpenAnalysis component. The interface to the language-independent transformation engine is an XML-based format, specified through an XML schema. The implemented transformation algorithms allow efficient derivative computations using locally optimized cross-country sequences of vertex, edge, and face elimination steps. Specifically, for the generation of adjoint codes, Open/ADF supports various code reversal schemes with hierarchical checkpointing at the subroutine level. As an example from geophysical fluid dynamics, a nonlinear time-dependent scalable, yet simple, barotropic ocean model is considered. OpenAD/Fs reverse mode is applied to compute sensitivities of some of the models transport properties with respect to gridded fields such as bottom topography as independent (control) variables.

Evaluating higher derivative tensors by forward propagation of univariate Taylor series

This article considers the problem of evaluating all pure and mixed partial derivatives of some vector function defined by an evaluation procedure. The natural approach to evaluating derivative tensors might appear to be their recursive calculation in the usual forward mode of computational differentiation. However, with the approach presented in this article, much simpler data access patterns and similar or lower computational counts can be achieved through propagating a family of univariate Taylor series of a suitable degree. It is applicable for arbitrary orders of derivatives. Also it is possible to calculate derivatives only in some directions instead of the full derivative tensor. Explicit formulas for all tensor entries as well as estimates for the corresponding computational complexities are given.

Advances in Automatic Differentiation

This collection covers advances in automatic differentiation theory and practice. Computer scientists and mathematicians will learn about recent developments in automatic differentiation theory as well as mechanisms for the construction of robust and powerful automatic differentiation tools. Computational scientists and engineers will benefit from the discussion of various applications, which provide insight into effective strategies for using automatic differentiation for inverse problems and design optimization.

Recent Advances in Algorithmic Differentiation

The proceedings represent the state of knowledge in the area of algorithmic differentiation (AD). The 31 contributed papers presented at the AD2012 conference cover the application of AD to many areas in science and engineering as well as aspects of AD theory and its implementation in tools. For all papers the referees, selected from the program committee and the greater community, as well as the editors have emphasized accessibility of the presented ideas also to non-AD experts. In the AD tools arena new implementations are introduced covering, for example, Java and graphical modeling environments or join the set of existing tools for Fortran. New developments in AD algorithms target the efficiency of matrix-operation derivatives, detection and exploitation of sparsity, partial separability, the treatment of nonsmooth functions, and other high-level mathematical aspects of the numerical computations to be differentiated. Applications stem from the Earth sciences, nuclear engineering, fluid dynamics, and chemistry, to name just a few. In many cases the applications in a given area of science or engineering share characteristics that require specific approaches to enable AD capabilities or provide an opportunity for efficiency gains in the derivative computation. The description of these characteristics and of the techniques for successfully using AD should make the proceedings a valuable source of information for users of AD tools.

Toward adjoinable MPI

Automatic differentiation is the primary means of obtaining analytic derivatives from a numerical model given as a computer program. Therefore, it is an essential productivity tool in numerous computational science and engineering domains. Computing gradients with the adjoint (also called reverse) mode via source transformation is a particularly beneficial but also challenging use of automatic differentiation. To date only ad hoc solutions for adjoint differentiation of MPI programs have been available, forcing automatic differentiation tool users to reason about parallel communication dataflow and dependencies and manually develop adjoint communication code. Using the communication graph as a model we characterize the principal problems of adjoining the most frequently used communication idioms. We propose solutions to cover these idioms and consider the consequences for the MPI implementation, the MPI user and MPI-aware program analysis. The MIT general circulation model serves as a use case to illustrate the viability of our approach.

Fast higher-order derivative tensors with Rapsodia

A number of practical problems in physics can be solved by using accurate higher-order derivatives. Such derivatives can be obtained with automatic differentiation. However, one has to be concerned with the complexity of computing higher-order derivative tensors even for a modest order and number of independents. Initial experiments using univariate Taylor polynomials with interpolation and operator overloading with unrolled loops showed better runtimes than using other automatic differentiation tools. Motivated by these results, we developed the Rapsodia code generator that produces Fortran and C++libraries for the most common intrinsics. Here we explain the algorithmic approach, implementation, and present test results on a select set of applications. Further details on the Rapsodia tool, and an example for user extensions are given in the Appendix.

Control flow reversal for adjoint code generation

We describe an approach to the reversal of the control flow of structured programs. It is used to automatically generate adjoint code for numerical programs by semantic source transformation. After a short introduction to applications and the implementation tool set, we describe the building blocks using a simple example. We then illustrate the code reversal within basic blocks. The main part of the paper covers the reversal of structured control flow graphs. We show the algorithmic steps for simple branches and loops and give a detailed algorithm for the reversal of arbitrary combinations of loops and branches in a general control flow graph

Interpretative adjoints for numerical simulation codes using MPI

Abstract An essential performance and correctness factor in numerical simulation and optimization is access to exact derivative information. Adjoint derivative models are particularly useful if a function’s number of inputs far exceeds the number of outputs. The propagation of adjoints requires the data flow to be reversed, implying the reversal of all communication in programs that use message-passing. This paper presents recent advances made in developing the adjoint MPI library AMPI. The described proof of concept aims to serve as the basis for coupling other overloading AD tools with AMPI. We illustrate its use in the context of a specific overloading tool for algorithmic differentiation (AD) for C++ programs. A simplified but representative application problem is discussed as a case study.

Flattening Basic Blocks

The application of cross country elimination strategies requires access to the computational graph or at least subgraphs for certain scopes, e.g. a basic block. Under the presence of aliased variables the construction of these (sub)graphs encounters ambiguities. We propose an algorithm to construct ambiguity free subgraphs.

A Framework for Proving Correctness of Adjoint Message-Passing Programs

We propose a technique for proving correctness of adjoint message passing programs that relies on data dependences in partitioned global address space. As an example we discuss asynchronous unbuffered send/receive using MPI.