The Quickest Transshipment Problem
Mathematics of Operations Research
Queue - Storage
A measurement study of available bandwidth estimation tools
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Bridging the digital divide: storage media + postal network = generic high-bandwidth communication
ACM Transactions on Storage (TOS)
The Globus Striped GridFTP Framework and Server
SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
S3: a scalable sensing service for monitoring large networked systems
Proceedings of the 2006 SIGCOMM workshop on Internet network management
Operations Research: An Introduction (8th Edition)
Operations Research: An Introduction (8th Edition)
Scale and performance in the CoBlitz large-file distribution service
NSDI'06 Proceedings of the 3rd conference on Networked Systems Design & Implementation - Volume 3
An architecture for internet data transfer
NSDI'06 Proceedings of the 3rd conference on Networked Systems Design & Implementation - Volume 3
SIAM Journal on Computing
e-Science in the Cloud with CARMEN
PDCAT '07 Proceedings of the Eighth International Conference on Parallel and Distributed Computing, Applications and Technologies
Using overlays for efficient data transfer over shared wide-area networks
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Defining future platform requirements for e-Science clouds
Proceedings of the 1st ACM symposium on Cloud computing
New Algorithms for Planning Bulk Transfer via Internet and Shipping Networks
ICDCS '10 Proceedings of the 2010 IEEE 30th International Conference on Distributed Computing Systems
Software as a service for data scientists
Communications of the ACM
StorkCloud: data transfer scheduling and optimization as a service
Proceedings of the 4th ACM workshop on Scientific cloud computing
Modeling throughput sampling size for a cloud-hosted data scheduling and optimization service
Future Generation Computer Systems
| Hi-index | 0.02 |
Cloud collaborators wish to combine large amounts of data, in the order of TBs, from multiple distributed locations to a single datacenter. Such groups are faced with the challenge of reducing the latency of the transfer, without incurring excessive dollar costs. Our Pandora system is an autonomic system that creates data transfer plans that can satisfy latency and cost needs, by considering transferring the data through both Internet and disk shipments. Solving the planning problem is a critical step towards a truly autonomic bulk data transfer service. In this paper, we develop techniques to create an optimal transfer plan that minimizes transfer latency subject to a budget constraint. To systematically explore the solution space, we develop efficient binary search methods that find the optimal shipment transfer plan. Our experimental evaluation, driven by Internet bandwidth traces and actual shipment costs queried from FedEx web services, shows that these techniques work well on diverse, realistic networks.