A study of integrated prefetching and caching strategies
Proceedings of the 1995 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems
ACM Transactions on Computer Systems (TOCS)
Implementing cooperative prefetching and caching in a globally-managed memory system
SIGMETRICS '98/PERFORMANCE '98 Proceedings of the 1998 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems
The End-to-End Performance Effects of Parallel TCP Sockets on a Lossy Wide-Area Network
IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
Stork: Making Data Placement a First Class Citizen in the Grid
ICDCS '04 Proceedings of the 24th International Conference on Distributed Computing Systems (ICDCS'04)
Modeling and Taming Parallel TCP on the Wide Area Network
IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers - Volume 01
Software prefetching and caching for translation lookaside buffers
OSDI '94 Proceedings of the 1st USENIX conference on Operating Systems Design and Implementation
Adaptive data block scheduling for parallel TCP streams
HPDC '05 Proceedings of the High Performance Distributed Computing, 2005. HPDC-14. Proceedings. 14th IEEE International Symposium
Using overlays for efficient data transfer over shared wide-area networks
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
A new paradigm: Data-aware scheduling in grid computing
Future Generation Computer Systems
Balancing TCP buffer vs parallel streams in application level throughput optimization
Proceedings of the second international workshop on Data-aware distributed computing
Which network measurement tool is right for you? a multidimensional comparison study
GRID '08 Proceedings of the 2008 9th IEEE/ACM International Conference on Grid Computing
A data transfer framework for large-scale science experiments
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Data center networking with multipath TCP
Hotnets-IX Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks
A Data Throughput Prediction and Optimization Service for Widely Distributed Many-Task Computing
IEEE Transactions on Parallel and Distributed Systems
Budget-constrained bulk data transfer via internet and shipping networks
Proceedings of the 8th ACM international conference on Autonomic computing
Inter-datacenter bulk transfers with netstitcher
Proceedings of the ACM SIGCOMM 2011 conference
Prediction of Optimal Parallelism Level in Wide Area Data Transfers
IEEE Transactions on Parallel and Distributed Systems
Software as a service for data scientists
Communications of the ACM
Network-aware end-to-end data throughput optimization
Proceedings of the first international workshop on Network-aware data management
A Highly-Accurate and Low-Overhead Prediction Model for Transfer Throughput Optimization
SCC '12 Proceedings of the 2012 SC Companion: High Performance Computing, Networking Storage and Analysis
Network-aware data caching and prefetching for cloud-hosted metadata retrieval
NDM '13 Proceedings of the Third International Workshop on Network-Aware Data Management
| Hi-index | 0.00 |
Wide-area transfer of large data sets is still a big challenge despite the deployment of high-bandwidth networks with speeds reaching 100 Gbps. Most users fail to obtain even a fraction of theoretical speeds promised by these networks. Effective usage of the available network capacity has become increasingly important for wide-area data movement. We have developed a "data transfer scheduling and optimization system as a Cloud-hosted service", StorkCloud, which will mitigate the large-scale end-to-end data movement bottleneck by efficiently utilizing underlying networks and effectively scheduling and optimizing data transfers. In this paper, we present the initial design and prototype implementation of StorkCloud, and show its effectiveness in large dataset transfers across geographically distant storage sites, data centers, and collaborating institutions.