Eliminating Replica Selection - Using Multiple Replicas to Accelerate Data Transfer on Grids

  • Authors:
  • Jun Feng;Marty Humphrey

  • Affiliations:
  • University of Virginia;University of Virginia

  • Venue:
  • ICPADS '04 Proceedings of the Parallel and Distributed Systems, Tenth International Conference
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

Data-intensive, high-performance computing applicationsoften require the efficient transfer of terabytesor even petabytes of data in wide-area, distributed computingenvironments. To increase the efficiency of widearea data movement, researchers have devised varioustechniques such as TCP tuning, multiple streams andasynchronous I/O. This paper adopts a new approach to increaseperformance further by exploiting replica-levelparallelism in Grids. rFTP, a new grid data transferringtool, improves the data transfer rate and reliabilityon Grids by utilizing multiple replica sources concurrently.Experiments on the NPACI Grid show as much as a2.02x speedup over a single data source by adaptively retrievingpartial data segments from 4 replicas using the data provided by NWS.