An empirical comparative study of decentralized load balancing algorithms in clustered storage environment

Authors:
Yun Wang;Xiangyu Luo;Feifei Yuan;Cong Li
Affiliations:
School of Computer Science & Engineering, Key Lab of CNII, MOE, Southeast University, Nanjing, China;School of Computer Science & Engineering, Key Lab of CNII, MOE, Southeast University, Nanjing, China;School of Computer Science & Engineering, Key Lab of CNII, MOE, Southeast University, Nanjing, China;School of Computer Science & Engineering, Key Lab of CNII, MOE, Southeast University, Nanjing, China
Venue:
ICPCA/SWS'12 Proceedings of the 2012 international conference on Pervasive Computing and the Networked World
Year:
2012

Citing 11
Cited 0

File Assignment in Parallel I/O Systems with Minimal Variance of Service Time

IEEE Transactions on Computers
Comparative Models of the File Assignment Problem

ACM Computing Surveys (CSUR)
On algorithms for efficient data migration

SODA '01 Proceedings of the twelfth annual ACM-SIAM symposium on Discrete algorithms
Programming a Hypercube Multicomputer

IEEE Software
(R) A Load - Balancing Algorithm for N-Cubes

ICPP '96 Proceedings of the Proceedings of the 1996 International Conference on Parallel Processing - Volume 3
The Google file system

SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
D-SPTF: decentralized request distribution in brick-based storage systems

ASPLOS XI Proceedings of the 11th international conference on Architectural support for programming languages and operating systems
A Decentralized Storage Cluster with High Reliability and Flexibility

PDP '06 Proceedings of the 14th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing
Kinesis: A new approach to replica placement in distributed storage systems

ACM Transactions on Storage (TOS)
CDRM: A Cost-Effective Dynamic Replication Management Scheme for Cloud Storage Cluster

CLUSTER '10 Proceedings of the 2010 IEEE International Conference on Cluster Computing
Data Migration in Heterogeneous Storage Systems

ICDCS '11 Proceedings of the 2011 31st International Conference on Distributed Computing Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Load balance is critical for large-scale storage systems to produce high I/O performance. Decentralized solutions are especially preferred for no single point of bottleneck. We implement four typical hypercube-based decentralized load balancing algorithms in a prototype storage system, and conduct extensive experiments with the system running on a testbed comprising 32 nodes. We compare the efficiency and scalability of the four algorithms through the experiments. The comparison results lead to the following new observations contrary to the conclusions obtained in previous simulation studies. Firstly, algorithms with no redundant load migration do not actually achieve savings of migration costs. Secondly, algorithms tolerating a certain degree of redundancy in load migration may achieve improvements in scalability. The two observations provide new insights into the design of load balancing algorithms in distributed storage systems.