Enabling event tracing at leadership-class scale through I/O forwarding middleware

  • Authors:

  • Thomas Ilsche;Joseph Schuchart;Jason Cope;Dries Kimpe;Terry Jones;Andreas Knüpfer;Kamil Iskra;Robert Ross;Wolfgang E. Nagel;Stephen Poole

  • Affiliations:

  • Technische Universität Dresden & Oak Ridge National Laboratory, Dresden, Germany;Technische Universität Dresden, Dresden, Germany;Argonne National Laboratory, Argonne, IL, USA;Argonne National Laboratory, Argonne, IL, USA;Oak Ridge National Laboratory, Oak Ridge, TN, USA;Technische Universität Dresden, Dresden, USA;Argonne National Laboratory, Argonne, IL, USA;Argonne National Laboratory, Argonne, IL, USA;Technische Universität Dresden, Dresden, Germany;Oak Ridge National Laboratory, Oak Ridge, TN, USA

  • Venue:
  • Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Event tracing is an important tool for understanding the performance of parallel applications. As concurrency increases in leadership-class computing systems, the quantity of performance log data can overload the parallel file system, perturbing the application being observed. In this work we present a solution for event tracing at leadership scales. We enhance the I/O forwarding system software to aggregate and reorganize log data prior to writing to the storage system, significantly reducing the burden on the underlying file system for this type of traffic. Furthermore, we augment the I/O forwarding system with a write buffering capability to limit the impact of artificial perturbations from log data accesses on traced applications. To validate the approach, we modify the Vampir tracing toolset to take advantage of this new capability and show that the approach increases the maximum traced application size by a factor of 5x to more than 200,000 processes.