Andrew: a distributed personal computing environment
Communications of the ACM - The MIT Press scientific computation series
LogP: towards a realistic model of parallel computation
PPOPP '93 Proceedings of the fourth ACM SIGPLAN symposium on Principles and practice of parallel programming
Modeling communication pipeline latency
SIGMETRICS '98/PERFORMANCE '98 Proceedings of the 1998 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems
The grid: blueprint for a new computing infrastructure
The grid: blueprint for a new computing infrastructure
GASS: a data movement and access service for wide area computing systems
Proceedings of the sixth workshop on I/O in parallel and distributed systems
The SDSC storage resource broker
CASCON '98 Proceedings of the 1998 conference of the Centre for Advanced Studies on Collaborative research
Legion: The Next Logical Step Toward a Nationwide Virtual Computer
Legion: The Next Logical Step Toward a Nationwide Virtual Computer
High Performance Messaging on Workstations: Illinois Fast Messages (FM) for Myrinet
Supercomputing '95 Proceedings of the 1995 ACM/IEEE conference on Supercomputing
A Proposal of Pipelined Image Processing in a Grid Environment
SAINT-W '04 Proceedings of the 2004 Symposium on Applications and the Internet-Workshops (SAINT 2004 Workshops)
| Hi-index | 0.00 |
Data intensive applications constitute a large and increasing share of Grid computing. However there are relatively few results on how to improve the efficiency of the basic data transfer mechanisms used to move large data set in and out of Grid nodes. In this paper we describe a simple and general technique to improve the throughput of data transfer protocols and we demonstrate it on the SDSC Storage Resource Broker (SRB), a remote storage access middleware for supercomputer applications. We achieve a maximum performance improvement of 43%/52% for remote reads/writes larger than 1MB with a few changes to the original SRB protocol. The protocol was restructured by introducing a notion of pipelining that enables the overlapping of the various stages of the data processing, such as network transfer and disk access. We present a detailed analysis of the pipelined SRB implementation and of the pipeline cost model we used to drive our design.