A science-gateway workload archive to study pilot jobs, user activity, bag of tasks, task sub-steps, and workflow executions

  • Authors:
  • Rafael Ferreira da Silva;Tristan Glatard

  • Affiliations:
  • CNRS, INSERM, CREATIS, University of Lyon, Villeurbanne, France;CNRS, INSERM, CREATIS, University of Lyon, Villeurbanne, France

  • Venue:
  • Euro-Par'12 Proceedings of the 18th international conference on Parallel processing workshops
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Archives of distributed workloads acquired at the infrastructure level reputably lack information about users and application-level middleware. Science gateways provide consistent access points to the infrastructure, and therefore are an interesting information source to cope with this issue. In this paper, we describe a workload archive acquired at the science-gateway level, and we show its added value on several case studies related to user accounting, pilot jobs, fine-grained task analysis, bag of tasks, and workflows. Results show that science-gateway workload archives can detect workload wrapped in pilot jobs, improve user identification, give information on distributions of data transfer times, make bag-of-task detection accurate, and retrieve characteristics of workflow executions. Some limits are also identified.