Using overlays for efficient data transfer over shared wide-area networks
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Journal of Network and Computer Applications
Redundant parallel file transfer with anticipative recursively-adjusting scheme in data grids
ICA3PP'07 Proceedings of the 7th international conference on Algorithms and architectures for parallel processing
Implementation of a medical image file accessing system in co-allocation data grids
Future Generation Computer Systems
Piggybacking for more efficient parallel out-of-core isosurfacing
EG PGV'06 Proceedings of the 6th Eurographics conference on Parallel Graphics and Visualization
| Hi-index | 0.00 |
It is increasingly common to use multiple distributed storage systems as a single data store within which large datasets may be replicated. Thus, we face the problem of how to access replicated data efficiently. Multiple-source parallel transfers can reduce access times by transferring data from several replicas in parallel. However, we then face the problem of deciding which data to fetch from which replicas. We propose a Tuned Conservative scheduling technique that uses predicted means and variances for network performance to make data selection decisions. This stochastic scheduling technique adjusts the amount of data fetched on a link according to not only the link performance but the expected variance in that performance. We incorporate our technique into the striped GridFTP server from the Globus Toolkit, and demonstrate that the technique can produce data transfer times that are significantly faster and less variable than those of other techniques.