Fine-grained mobility in the Emerald system
ACM Transactions on Computer Systems (TOCS)
The Coign automatic distributed partitioning system
OSDI '99 Proceedings of the third symposium on Operating systems design and implementation
Experience with distributed processing on a host/satellite graphics system
SIGGRAPH '76 Proceedings of the 3rd annual conference on Computer graphics and interactive techniques
Configurable applications for graphics employing satellites (CAGES)
SIGGRAPH '75 Proceedings of the 2nd annual conference on Computer graphics and interactive techniques
Globe: A Wide-Area Distributed System
IEEE Concurrency
A Bytecode Translator for Distributed Execution of ``Legacy'' Java Software
ECOOP '01 Proceedings of the 15th European Conference on Object-Oriented Programming
J-Orchestra: Automatic Java Application Partitioning
ECOOP '02 Proceedings of the 16th European Conference on Object-Oriented Programming
Dynamic Application Partitioning in VisualAge Generator Version 3.0
ECOOP '98 Workshop ion on Object-Oriented Technology
Pinpoint: Problem Determination in Large, Dynamic Internet Services
DSN '02 Proceedings of the 2002 International Conference on Dependable Systems and Networks
Improved Combinatorial Algorithms for the Facility Location and k-Median Problems
FOCS '99 Proceedings of the 40th Annual Symposium on Foundations of Computer Science
HPDC '96 Proceedings of the 5th IEEE International Symposium on High Performance Distributed Computing
Balancing Performance, Energy, and Quality in Pervasive Computing
ICDCS '02 Proceedings of the 22 nd International Conference on Distributed Computing Systems (ICDCS'02)
Expander flows, geometric embeddings and graph partitioning
STOC '04 Proceedings of the thirty-sixth annual ACM symposium on Theory of computing
Vivaldi: a decentralized network coordinate system
Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications
Profile-Driven Component Placement for Cluster-Based Online Services
IEEE Distributed Systems Online
Improved approximation algorithms for capacitated facility location problems
Mathematical Programming: Series A and B
Slingshot: deploying stateful services in wireless hotspots
Proceedings of the 3rd international conference on Mobile systems, applications, and services
Design and implementation of a single system image operating system for ad hoc networks
Proceedings of the 3rd international conference on Mobile systems, applications, and services
Hilda: A High-Level Language for Data-DrivenWeb Applications
ICDE '06 Proceedings of the 22nd International Conference on Data Engineering
Reliable and efficient programming abstractions for wireless sensor networks
Proceedings of the 2007 ACM SIGPLAN conference on Programming language design and implementation
MapReduce: simplified data processing on large clusters
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Dynamic function placement for data-intensive cluster computing
ATEC '00 Proceedings of the annual conference on USENIX Annual Technical Conference
Planetary-scale views on a large instant-messaging network
Proceedings of the 17th international conference on World Wide Web
Doloto: code splitting for network-bound web 2.0 applications
Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering
SCOPE: easy and efficient parallel processing of massive data sets
Proceedings of the VLDB Endowment
BotGraph: large scale spamming botnet detection
NSDI'09 Proceedings of the 6th USENIX symposium on Networked systems design and implementation
Wishbone: profile-based partitioning for sensornet applications
NSDI'09 Proceedings of the 6th USENIX symposium on Networked systems design and implementation
Matchmaking for online games and other latency-sensitive P2P systems
Proceedings of the ACM SIGCOMM 2009 conference on Data communication
Centrifuge: integrated lease management and partitioning for cloud services
NSDI'10 Proceedings of the 7th USENIX conference on Networked systems design and implementation
X-trace: a pervasive network tracing framework
NSDI'07 Proceedings of the 4th USENIX conference on Networked systems design & implementation
How to tell an airport from a home: techniques and applications
Hotnets-IX Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks
Location, location, location!: modeling data proximity in the cloud
Hotnets-IX Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks
Resource allocation across multiple cloud data centres
Proceedings of the 8th International Workshop on Middleware for Grids, Clouds and e-Science
Towards elastic transactional cloud storage with range query support
Proceedings of the VLDB Endowment
Exploiting locality of interest in online social networks
Proceedings of the 6th International COnference
Providing scalable database services on the cloud
WISE'10 Proceedings of the 11th international conference on Web information systems engineering
Adaptive, secure, and scalable distributed data outsourcing: a vision paper
Proceedings of the 2011 workshop on Dynamic distributed data-intensive applications, programming abstractions, and systems
Wide area placement of data replicas for fast and highly available data access
Proceedings of the fourth international workshop on Data-intensive distributed computing
Integrated data placement and task assignment for scientific workflows in clouds
Proceedings of the fourth international workshop on Data-intensive distributed computing
In-situ MapReduce for log processing
USENIXATC'11 Proceedings of the 2011 USENIX conference on USENIX annual technical conference
Online migration for geo-distributed storage systems
USENIXATC'11 Proceedings of the 2011 USENIX conference on USENIX annual technical conference
Declarative automated cloud resource orchestration
Proceedings of the 2nd ACM Symposium on Cloud Computing
Bandwidth on demand for inter-data center communication
Proceedings of the 10th ACM Workshop on Hot Topics in Networks
Latency-aware data partitioning for geo-replicated online social networks
Proceedings of the Workshop on Posters and Demos Track
In-situ MapReduce for log processing
HotCloud'11 Proceedings of the 3rd USENIX conference on Hot topics in cloud computing
TailGate: handling long-tail content with a little help from friends
Proceedings of the 21st international conference on World Wide Web
Building access oblivious storage cloud for enterprise
Hot-ICE'12 Proceedings of the 2nd USENIX conference on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services
Content and geographical locality in user-generated content sharing systems
Proceedings of the 22nd international workshop on Network and Operating System Support for Digital Audio and Video
Journal of Computer and System Sciences
Surviving failures in bandwidth-constrained datacenters
Proceedings of the ACM SIGCOMM 2012 conference on Applications, technologies, architectures, and protocols for computer communication
Surviving failures in bandwidth-constrained datacenters
ACM SIGCOMM Computer Communication Review - Special october issue SIGCOMM '12
Internet-based Virtual Computing Environment: Beyond the data center as a computer
Future Generation Computer Systems
Proceedings of the sixth ACM international conference on Web search and data mining
Proceedings of the 2013 international workshop on Hot topics in cloud services
GCplace: geo-cloud based correlation aware data replica placement
Proceedings of the 28th Annual ACM Symposium on Applied Computing
Pushing CDN-ISP collaboration to the limit
ACM SIGCOMM Computer Communication Review
Data-Intensive Cloud Computing: Requirements, Expectations, Challenges, and Solutions
Journal of Grid Computing
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
ACM SIGOPS 24th Symposium on Operating Systems Principles
SPANStore: cost-effective geo-replicated storage spanning multiple cloud services
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
Consistency-based service level agreements for cloud storage
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
When the network crumbles: an empirical study of cloud network failures and their impact on services
Proceedings of the 4th annual Symposium on Cloud Computing
Joint Server Selection and Routing for Geo-replicated Services
UCC '13 Proceedings of the 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing
Hi-index | 0.00 |
As cloud services grow to span more and more globally distributed datacenters, there is an increasingly urgent need for automated mechanisms to place application data across these datacenters. This placement must deal with business constraints such as WAN bandwidth costs and datacenter capacity limits, while also minimizing user-perceived latency. The task of placement is further complicated by the issues of shared data, data inter-dependencies, application changes and user mobility. We document these challenges by analyzing month-long traces from Microsoft's Live Messenger and Live Mesh, two large-scale commercial cloud services. We present Volley, a system that addresses these challenges. Cloud services make use of Volley by submitting logs of datacenter requests. Volley analyzes the logs using an iterative optimization algorithm based on data access patterns and client locations, and outputs migration recommendations back to the cloud service. To scale to the data volumes of cloud service logs, Volley is designed to work in SCOPE [5], a scalable MapReduce-style platform; this allows Volley to perform over 400 machine-hours worth of computation in less than a day. We evaluate Volley on the month-long Live Mesh trace, and we find that, compared to a state-of-the-art heuristic that places data closest to the primary IP address that accesses it, Volley simultaneously reduces datacenter capacity skew by over 2×, reduces inter-datacenter traffic by over 1.8× and reduces 75th percentile user-latency by over 30%.