On the implementation of automatic differentiation tools

  • Authors:
  • Christian H. Bischof;Paul D. Hovland;Boyana Norris

  • Affiliations:
  • Institute for Scientific Computing, Aachen University of Technology, Aachen, Germany 52074;Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, USA 60439-4844;Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, USA 60439-4844

  • Venue:
  • Higher-Order and Symbolic Computation
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Automatic differentiation is a semantic transformation that applies the rules of differential calculus to source code. It thus transforms a computer program that computes a mathematical function into a program that computes the function and its derivatives. Derivatives play an important role in a wide variety of scientific computing applications, including numerical optimization, solution of nonlinear equations, sensitivity analysis, and nonlinear inverse problems. We describe the forward and reverse modes of automatic differentiation and provide a survey of implementation strategies. We describe some of the challenges in the implementation of automatic differentiation tools, with a focus on tools based on source transformation. We conclude with an overview of current research and future opportunities.