File and Object Replication in Data Grids

  • Authors:

  • Heinz Stockinger;Asad Samar;Koen Holtman;Bill Allcock;Ian Foster;Brian Tierney

  • Affiliations:

  • CERN, European Organization for Nuclear Research, CH-1211 Geneva 23, Switzerland/ Institut for Computer Science and Business Informatics, University of Vienna, A-1010 Vienna, Austria;California Institute of Technology, Pasadena, CA 91125, USA;California Institute of Technology, Pasadena, CA 91125, USA;Mathematics and Computer Science Division, Argonne National Laboratory, IL 60439, USA;Mathematics and Computer Science Division, Argonne National Laboratory, IL 60439, USA/ Department of Computer Science &/ The Computation Institute, The University of Chicago, IL 60637, USA;CERN, European Organization for Nuclear Research, CH-1211 Geneva 23, Switzerland/ Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA

  • Venue:
  • Cluster Computing
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

Data replication is a key issue in a Data Grid and can be managed in different ways and at different levels of granularity: for example, at the file level or object level. In the High Energy Physics community, Data Grids are being developed to support the distributed analysis of experimental data. We have produced a prototype data replication tool, the Grid Data Mirroring Package (GDMP) that is in production use in one physics experiment, with middleware provided by the Globus Toolkit used for authentication, data movement, and other purposes. We present here a new, enhanced GDMP architecture and prototype implementation that uses Globus Data Grid tools for efficient file replication. We also explain how this architecture can address object replication issues in an object-oriented database management system. File transfer over wide-area networks requires specific performance tuning in order to gain optimal data transfer rates. We present performance results obtained with GridFTP, an enhanced version of FTP, and discuss tuning parameters.