Efficient transactions for parallel data movement

  • Authors:
  • Jay Lofstead;Jai Dayal;Ivo Jimenez;Carlos Maltzahn

  • Affiliations:
  • Sandia National Laboratories;Georgia Institute of Technology;University of California, Santa Cruz;University of California, Santa Cruz

  • Venue:
  • PDSW '13 Proceedings of the 8th Parallel Data Storage Workshop
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

The rise of Integrated Application Workflows (IAWs) for processing data prior to storage on persistent media prompts the need to incorporate features that reproduce many of the semantics of persistent storage devices. One such feature is the ability to manage data sets as chunks with natural barriers between different data sets. Towards that end, we need a mechanism to ensure that data moved to an intermediate storage area is both complete and correct before allowing access by other processing components. The Doubly Distributed Transactions (D2T) protocol offers such a mechanism. The initial development [9] suffered from scalability limitations and undue requirements on server processes. The current version has addressed these limitations and has demonstrated scalability with low overhead.