The maximum concurrent flow problem
Journal of the ACM (JACM)
Fast approximation algorithms for multicommodity flow problems
STOC '91 Proceedings of the twenty-third annual ACM symposium on Theory of computing
SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
Introduction to Algorithms
Mobility increases the capacity of ad hoc wireless networks
IEEE/ACM Transactions on Networking (TON)
Routing in a delay tolerant network
Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications
IEEE/ACM Transactions on Networking (TON)
Practical guide to controlled experiments on the web: listen to your customers not to the hippo
Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Energy-aware server provisioning and load dispatching for connection-intensive internet services
NSDI'08 Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation
Adaptive file transfers for diverse environments
ATC'08 USENIX 2008 Annual Technical Conference on Annual Technical Conference
The cost of a cloud: research problems in data center networks
ACM SIGCOMM Computer Communication Review
Delay tolerant bulk data transfers on the internet
Proceedings of the eleventh international joint conference on Measurement and modeling of computer systems
VL2: a scalable and flexible data center network
Proceedings of the ACM SIGCOMM 2009 conference on Data communication
Finding a needle in Haystack: facebook's photo storage
OSDI'10 Proceedings of the 9th USENIX conference on Operating systems design and implementation
Exploiting similarity for multi-source downloads using file handprints
NSDI'07 Proceedings of the 4th USENIX conference on Networked systems design & implementation
Bandwidth on demand for inter-data center communication
Proceedings of the 10th ACM Workshop on Hot Topics in Networks
TailGate: handling long-tail content with a little help from friends
Proceedings of the 21st international conference on World Wide Web
Lowering Inter-datacenter Bandwidth Costs via Bulk Data Scheduling
CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
Design and implementation of an intelligent end-to-end network QoS system
SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Jetway: minimizing costs on inter-datacenter video traffic
Proceedings of the 20th ACM international conference on Multimedia
Sharing the cost of backbone networks: cui bono?
Proceedings of the 2012 ACM conference on Internet measurement conference
StorkCloud: data transfer scheduling and optimization as a service
Proceedings of the 4th ACM workshop on Scientific cloud computing
Achieving high utilization with software-driven WAN
Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM
Modeling throughput sampling size for a cloud-hosted data scheduling and optimization service
Future Generation Computer Systems
When the network crumbles: an empirical study of cloud network failures and their impact on services
Proceedings of the 4th annual Symposium on Cloud Computing
Virtualizing the access network via open APIs
Proceedings of the ninth ACM conference on Emerging networking experiments and technologies
3GOL: power-boosting ADSL using 3G onloading
Proceedings of the ninth ACM conference on Emerging networking experiments and technologies
Parallel routing on multi-core routers for big data transfers
Proceedings of the 2013 workshop on Student workhop
A survey of smart data pricing: Past proposals, current plans, and future trends
ACM Computing Surveys (CSUR)
Delay-tolerant bulk data transfers on the internet
IEEE/ACM Transactions on Networking (TON)
Hi-index | 0.00 |
Large datacenter operators with sites at multiple locations dimension their key resources according to the peak demand of the geographic area that each site covers. The demand of specific areas follows strong diurnal patterns with high peak to valley ratios that result in poor average utilization across a day. In this paper, we show how to rescue unutilized bandwidth across multiple datacenters and backbone networks and use it for non-real-time applications, such as backups, propagation of bulky updates, and migration of data. Achieving the above is non-trivial since leftover bandwidth appears at different times, for different durations, and at different places in the world. For this purpose, we have designed, implemented, and validated NetStitcher, a system that employs a network of storage nodes to stitch together unutilized bandwidth, whenever and wherever it exists. It gathers information about leftover resources, uses a store-and-forward algorithm to schedule data transfers, and adapts to resource fluctuations. We have compared NetStitcher with other bulk transfer mechanisms using both a testbed and a live deployment on a real CDN. Our testbed evaluation shows that NetStitcher outperforms all other mechanisms and can rescue up to five times additional datacenter bandwidth thus making it a valuable tool for datacenter providers. Our live CDN deployment demonstrates that our solution can perform large data transfers at a much lower cost than naive end-to-end or store-and-forward schemes.