Grid-Oriented Storage: A Single-Image, Cross-Domain, High-Bandwidth Architecture
IEEE Transactions on Computers
Data transfers in the grid: workload analysis of globus GridFTP
DADC '08 Proceedings of the 2008 international workshop on Data-aware distributed computing
A Self-Managing Wide-Area Data Streaming Service using Model-based Online Control
GRID '06 Proceedings of the 7th IEEE/ACM International Conference on Grid Computing
| Hi-index | 0.00 |
In this paper, we describe the integration of dynamic right-sizing 驴 an automatic and scalable buffer management technique for enhancing TCP performance 驴 into GridFTP, a subsystem of the Globus Toolkit for managing bulk data transfers across computational grids. Such grids are often characterized by networks with large bandwidth-delay products. Unfortunately, many of today's grid applications use only a small fraction of available bandwidth because the default buffer sizes in TCP are tuned for yesterday's WAN speeds. Buffer sizes can be manually tuned to allow TCP flow control to adapt to high-speed WAN environments, but this is a tedious process. Although recent work has shown how to automatically tune system buffers during connection set-up, these values may not be appropriate for the connection's lifetime due to varying network delay and throughput.We show how using the technique of dynamic right-sizing (DRS) in GridFTP helps us optimize memory usage while maintaining high throughput over the lifetime of the connection. We also show how DRS enhances important GridFTP features such as striped and third-party data transfers in a scalable way. The technique is implemented entirely in user space so that end users do not have to modify the kernel.