Kshitij Kulshreshtha

On the numerical stability of algorithmic differentiation

In contrast to integration, the differentiation of a function is an ill-conditioned process, if only an oracle is available for its pointwise evaluation. That is, unrelated small variations in the value of the composite function are allowed at nearly identical arguments. In contrast, we show here that, if the function is defined by an evaluation procedure as a composition of arithmetic operations and elementary functions, then automatic, or algorithmic differentiation is backward stable in the sense of Wilkinson. More specifically, the derivative values obtained are exact for a perturbation of the elementary components at the level of the machine precision. We also provide a forward error analysis for both the forward and reverse mode. The theoretical analysis is confirmed by numerical experiments.

Algorithmic differentiation for piecewise smooth functions: a case study for robust optimization

This paper presents a minimization method for Lipschitz continuous, piecewise smooth objective functions based on algorithmic differentiation (AD). We assume that all nondifferentiabilities are caused by abs(), min(), and max(). The optimization method generates successively piecewise linearizations in abs-normal form and solves these local subproblems by exploiting the resulting kink structure. Both the generation of the abs-normal form and the exploitation of the kink structure are possible due to extensions of standard AD tools. This work presents corresponding drivers for the AD tool ADOL-C which are embedded in the nonsmooth solver LiPsMin. Finally, minimax problems from robust optimization are considered. Numerical results and a comparison of LiPsMin with other nonsmooth optimization methods are discussed.

A Mixed Approach to Adjoint Computation with Algorithmic Differentiation

Various algorithmic differentiation tools have been developed and applied to large-scale simulation software for physical phenomena. Until now, two strictly disconnected approaches have been used to implement algorithmic differentiation (AD), namely, source transformation and operator overloading. This separation was motivated by different features of the programming languages such as Fortran and C++. In this work we have for the first time combined the two approaches to implement AD for C++ codes. Source transformation is used for core routines that are repetitive, where the transformed source can be optimized much better by modern compilers, and operator overloading is used to interconnect at the upper level, where source transformation is not possible because of complex language constructs of C++. We have also devised a method to apply the mixed approach in the same application semi-automatically. We demonstrate the benefit of this approach using some real-world applications.

Computing Derivatives in a Meshless Simulation Using Permutations in ADOL-C

In order to compute derivatives in a meshless simulation one needs to take into account the ever changing neighborhood relationships in a point-cloud that describes the domain. This may be implemented using permutations of the independent and dependent variables during the assembly of the discretized system. Such branchings are difficult to handle for operator overloading AD tools using traces. In this paper, we propose a new approach that allows the derivative computations for an even larger class of specific branches without retracing.

Efficient computation of derivatives for solving optimization problems in R and Python using SWIG-generated interfaces to ADOL-C†

Scripting languages are gaining acceptance because of their ease of use and value for rapid prototyping in many fields, including machine learning and statistics. In the context of algorithmic differentiation, however, the main development effort continues to be concentrated on traditional compiled languages such as Fortran and C/C++, whether source transformation tools or operator overloading tools. There is therefore a need for AD tools for computing derivatives efficiently within scripting languages. ADOL-C is an operator overloading-based C++ library that provides accurate first- and higher order derivatives for applications in C++. SWIG is a preprocessor that uses the C/C++ header files to wrap the API of a library to be callable from scripting languages such as R and Python and several other high-level programming languages. Although every language has its caveats, the overall process of making the C/C++ API available via SWIG is the same for all scripting languages. After an initial effort required per language, because SWIG is an automated interface generator based on the librarys actual header files, only minimal effort is required to maintain the scripting interface in sync with upstream developments in the original C/C++ library. In addition to achieving our original goal of creating an interface for R, we were able to generate an interface for Python that proved an order of magnitude faster than the previously implemented interface. This paper gives an overview of the interface generation process, the challenges we encountered with both scripting languages and some numerical results to demonstrate both usefulness and efficiency.

Exploiting Sparsity in Automatic Differentiation on Multicore Architectures

We discuss the design, implementation and performance of algorithms suitable for the efficient computation of sparse Jacobian and Hessian matrices using Automatic Differentiation via operator overloading on multicore architectures. The procedure for exploiting sparsity (for runtime and memory efficiency) in serial computation involves a number of steps. Using nonlinear optimization problems as test cases, we show that the algorithms involved in the various steps can be adapted to multithreaded computations.

Modelling and optimisation of the design and topology of flexible frames with rigid joints

Structural optimisation currently relies heavily on methods based on discretisation. In simpler cases like the simulation of frames and trusses, where discretisation is not necessary, only the elongation or compression is considered and the joints are free, like ball and socket joints, in order to avoid bending the trusses. In this dissertation a discretisation free method for the modelling and optimisation of frames is developed which considers bending of the beams along with compression or elongation with joints between the beams being rigid. Rigid joints are commonly the result of welding two beams together or connecting them using mutiple rivets. The optimisation problems, both state and design optimisation, are formulated via the total elastic energy and the work done by external forces. Moreover, for the optimal sizing problem a topological sensitivity for introduction of new beams between any two arbitrary positions in the frame is discussed.