SIGMOD '88 Proceedings of the 1988 ACM SIGMOD international conference on Management of data
On algorithms for efficient data migration
SODA '01 Proceedings of the twelfth annual ACM-SIAM symposium on Discrete algorithms
An Experimental Study of Data Migration Algorithms
WAE '01 Proceedings of the 5th International Workshop on Algorithm Engineering
Simple efficient load balancing algorithms for peer-to-peer systems
Proceedings of the sixteenth annual ACM symposium on Parallelism in algorithms and architectures
Dynamic Provisioning of Multi-tier Internet Applications
ICAC '05 Proceedings of the Second International Conference on Automatic Computing
Bigtable: a distributed storage system for structured data
OSDI '06 Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation - Volume 7
Dynamo: amazon's highly available key-value store
Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles
Online balancing of range-partitioned data with applications to peer-to-peer systems
VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
PNUTS: Yahoo!'s hosted data serving platform
Proceedings of the VLDB Endowment
Cassandra: structured storage system on a P2P network
Proceedings of the 28th ACM symposium on Principles of distributed computing
A load balancing framework for clustered storage systems
HiPC'08 Proceedings of the 15th international conference on High performance computing
Benchmarking cloud serving systems with YCSB
Proceedings of the 1st ACM symposium on Cloud computing
Characterizing, modeling, and generating workload spikes for stateful services
Proceedings of the 1st ACM symposium on Cloud computing
The SCADS director: scaling a distributed storage system under stringent performance requirements
FAST'11 Proceedings of the 9th USENIX conference on File and stroage technologies
Hippodrome: running circles around storage administration
FAST'02 Proceedings of the 1st USENIX conference on File and storage technologies
Aqueduct: online data migration with performance guarantees
FAST'02 Proceedings of the 1st USENIX conference on File and storage technologies
| Hi-index | 0.00 |
Sherpa is a large-scale distributed and globally replicated multi-tenant cloud data storage system. Sherpa scales by horizontally partitioning data into tablets and distributing these tablets across multiple servers. While Sherpa scales for increasing workload sizes by adding servers, it is vulnerable to load imbalance among tablets that cause hotspots to develop on just a few servers. In this paper we describe Yak, the Sherpa load balancer. Yak detects hotspots and then automatically balances load by migrating tablets from the overloaded servers, and also by splitting data into new tablets. We describe Yak's design principles, algorithms and architecture. We then evaluate Yak on workloads based on Sherpa production scenarios.