Intelligent data staging with overlapped execution of grid applications
Future Generation Computer Systems
Multi-Replication with Intelligent Staging in Data-Intensive Grid Applications
GRID '06 Proceedings of the 7th IEEE/ACM International Conference on Grid Computing
| Hi-index | 0.00 |
Existing replica services on the Grid we know to date assumes point-to-point communication and file transfer protocol. As such, when hundreds to thousands of hosts on the Grid access a single dataset simultaneously, bottlenecks in networks and/or the data servers will hinder performance significantly. Instead, our replication framework couplesefficient, multicast techniques with a replica catalog that automatically detects simultaneous access to the replica by multiple nodes. As a prototype, we have designed and built a portable, XML-based replica location service accounting for such parallel transfer requests, and coupled it with a O(1) bulk file transfer system Dolly+[6]. The benchmarks show that the system is scalable and effective in reducing replication costs significantly in cluster-based replication scenarios.