The data-flow equations of checkpointing in reverse automatic differentiation

Authors:
Benjamin Dauvergne;Laurent Hascoët
Affiliations:
TROPICS team, INRIA Sophia-Antipolis, Sophia-Antipolis, France;TROPICS team, INRIA Sophia-Antipolis, Sophia-Antipolis, France
Venue:
ICCS'06 Proceedings of the 6th international conference on Computational Science - Volume Part IV
Year:
2006

Citing 5
Cited 0

Compilers: principles, techniques, and tools

Compilers: principles, techniques, and tools
The Maple handbook: Maple V Release 4

The Maple handbook: Maple V Release 4
Evaluating derivatives: principles and techniques of algorithmic differentiation

Evaluating derivatives: principles and techniques of algorithmic differentiation
Minimizing the tape size

Automatic differentiation of algorithms
"To be recorded" analysis in reverse-mode automatic differentiation

Future Generation Computer Systems

Quantified Score

Hi-index	0.01

Visualization

Abstract

Checkpointing is a technique to reduce the memory consumption of adjoint programs produced by reverse Automatic Differentiation. However, checkpointing also uses a non-negligible memory space for the so-called “snapshots”. We analyze the data-flow of checkpointing, yielding a precise characterization of all possible memory-optimal options for snapshots. This characterization is formally derived from the structure of checkpoints and from classical data-flow equations. In particular, we select two very different options and study their behavior on a number of real codes. Although no option is uniformly better, the so-called “lazy-snapshot” option appears preferable in general.