DryadInc: reusing work in large-scale computations

  • Authors:
  • Lucian Popa;Mihai Budiu;Yuan Yu;Michael Isard

  • Affiliations:
  • UC Berkeley;Microsoft Research, Silicon Valley;Microsoft Research, Silicon Valley;Microsoft Research, Silicon Valley

  • Venue:
  • HotCloud'09 Proceedings of the 2009 conference on Hot topics in cloud computing
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Many large-scale (cloud) computations operate on append-only, partitioned datasets. We present two incremental computation frameworks to reuse prior work in these circumstances: (1) reusing identical computations already performed on data partitions, and (2) computing just on the newly appended data and merging the new and previous results.