Swift: A language for distributed parallel scripting

  • Authors:

  • Michael Wilde;Mihael Hategan;Justin M. Wozniak;Ben Clifford;Daniel S. Katz;Ian Foster

  • Affiliations:

  • Computation Institute, University of Chicago and Argonne National Laboratory, United States and Mathematics and Computer Science Division, Argonne National Laboratory, United States;Computation Institute, University of Chicago and Argonne National Laboratory, United States;Mathematics and Computer Science Division, Argonne National Laboratory, United States;Department of Astronomy and Astrophysics, University of Chicago, United States;Computation Institute, University of Chicago and Argonne National Laboratory, United States;Computation Institute, University of Chicago and Argonne National Laboratory, United States and Mathematics and Computer Science Division, Argonne National Laboratory, United States and Department ...

  • Venue:
  • Parallel Computing
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Scientists, engineers, and statisticians must execute domain-specific application programs many times on large collections of file-based data. This activity requires complex orchestration and data management as data is passed to, from, and among application invocations. Distributed and parallel computing resources can accelerate such processing, but their use further increases programming complexity. The Swift parallel scripting language reduces these complexities by making file system structures accessible via language constructs and by allowing ordinary application programs to be composed into powerful parallel scripts that can efficiently utilize parallel and distributed resources. We present Swift's implicitly parallel and deterministic programming model, which applies external applications to file collections using a functional style that abstracts and simplifies distributed parallel execution.