Filter-embedding semiring fusion for programming with MapReduce

  • Authors:
  • Kento Emoto;Sebastian Fischer;Zhenjiang Hu

  • Affiliations:
  • The University of Tokyo, Hongo 7-3-1, Bunkyo, Tokyo, Japan;National Institute of Informatics, Tokyo, Japan;National Institute of Informatics, Tokyo, Japan

  • Venue:
  • Formal Aspects of Computing - Celebrating the 60th Birthday of Carroll Morgan
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

We show that MapReduce, the de facto standard for large scale data-intensive parallel programming, can be equipped with a programming theory in calculational form. By integrating the generate-and-test programming paradigm and semirings for aggregation of results, we propose a novel parallel programming framework for MapReduce. The framework consists of two important calculation theorems: the shortcut fusion theorem of semiring homomorphisms bridges the gap between specifications and efficient implementations, and the filter-embedding theorem helps to develop parallel programs in a systematic and incremental way.