Reducing reverse-mode memory requirements by using profile-driven checkpointing

  • Authors:

  • Mike Fagan;Alan Carle

  • Affiliations:

  • Rice University, Department of Computational and Applied Mathematics, MS134, 6100 Main Street, Houston, TX 77005-1892, USA;Rice University, Department of Computational and Applied Mathematics, MS134, 6100 Main Street, Houston, TX 77005-1892, USA

  • Venue:
  • Future Generation Computer Systems
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Reverse-mode derivative calculations have favorable time cost for many problems. Unfortunately ''real world'' reverse-mode computations frequently experience prohibitive space costs. To mitigate this space cost, users resort to checkpointing techniques to recompute, rather than save, the necessary values. Injudicious checkpointing, however, can destroy the favorable time performance that made reverse mode attractive in the first place. Consequently, reverse-mode users must spend significant amounts of development time analyzing and developing checkpointing schemes that complement their reverse-mode computation code. In this paper, we describe a particular instance of this checkpointing problem: we were using reverse-mode code generated by Adifor 3.0 to compute derivatives of a large computational fluid dynamics code. Our effort labored under the additional constraint that development time was minimal (as always, it was needed yesterday). Our solution was to use profiling to narrowly focus our checkpoint analysis. This profiling approach worked well for our problem. Furthermore, the profiling idea is sufficiently general that it should work well for other problems. This paper details both our results on our specific problem and guidelines for applying the profiling technique to other checkpoint-based reverse-mode development problems.