Redundant parallel data transfer schemes for the grid environment

Authors:
R. S. Bhuvaneswaran;Yoshiaki Katayama;Naohisa Takahashi
Affiliations:
Department of Computer Science and Engineering, Graduate School of Engineering, Nagoya Institute of Technology, Japan;Department of Computer Science and Engineering, Graduate School of Engineering, Nagoya Institute of Technology, Japan;Department of Computer Science and Engineering, Graduate School of Engineering, Nagoya Institute of Technology, Japan
Venue:
ACSW Frontiers '06 Proceedings of the 2006 Australasian workshops on Grid computing and e-research - Volume 54
Year:
2006

Citing 5
Cited 1

Data management and transfer in high-performance computational grid environments

Parallel Computing - Parallel data-intensive algorithms and applications
Giggle: a framework for constructing scalable replica location services

Proceedings of the 2002 ACM/IEEE conference on Supercomputing
Replica Selection in the Globus Data Grid

CCGRID '01 Proceedings of the 1st International Symposium on Cluster Computing and the Grid
Resource Co-Allocation in Computational Grids

HPDC '99 Proceedings of the 8th IEEE International Symposium on High Performance Distributed Computing
Enabling the Co-Allocation of Grid Data Transfers

GRID '03 Proceedings of the 4th International Workshop on Grid Computing

Accessing data from many servers simultaneously and adaptively in data grids

Future Generation Computer Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we proposed dynamic co-allocation schemes for parallel data transfer in a grid environment, which copes up with highly inconsistent network performances of the servers. We have developed an algorithm using circular queue, with which, the data transfer tasks are allocated onto the servers in duplication. It is further improved to maintain the ordered delivery of data blocks, to suit such applications, like streaming. The schemes nicely tackle even when the link to servers under consideration is broken or become idle. We used Globus toolkit for our framework and utilized the partial copy feature of GridFTP. We compared our schemes with the existing schemes and the preliminary results show notable improvement in overall completion time of data transfer.