Design and implementation of a random data-placement system with high scalability, reliability and performance

Authors:
Kun Liu;Wei Xue;Di Wang;Jiwu Shu
Affiliations:
Dept. of Computer Science and Technology, Tsinghua University, Beijing;Dept. of Computer Science and Technology, Tsinghua University, Beijing;Dept. of Computer Science and Technology, Tsinghua University, Beijing;Dept. of Computer Science and Technology, Tsinghua University, Beijing
Venue:
ICCS'06 Proceedings of the 6th international conference on Computational Science - Volume Part III
Year:
2006

Citing 11
Cited 0

The HP AutoRAID hierarchical storage system

ACM Transactions on Computer Systems (TOCS) - Special issue on operating system principles
LH*—a scalable, distributed data structure

ACM Transactions on Database Systems (TODS)
Continuous display using heterogeneous disk-subsystems

MULTIMEDIA '97 Proceedings of the fifth ACM international conference on Multimedia
Comparing random data allocation and data striping in multimedia servers

Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Compact, adaptive placement schemes for non-uniform requirements

Proceedings of the fourteenth annual ACM symposium on Parallel algorithms and architectures
High-Availability LH* Schemes with Mirroring

COOPIS '96 Proceedings of the First IFCIS International Conference on Cooperative Information Systems
LH*s: a high-availability and high-security scalable distributed data structure

RIDE '97 Proceedings of the 7th International Workshop on Research Issues in Data Engineering (RIDE '97) High Performance Database Management for Large-Scale Applications
A Fast Algorithm for Online Placement and Reorganization of Replicated Data

IPDPS '03 Proceedings of the 17th International Symposium on Parallel and Distributed Processing
A Continuous Media Placement Using B-ZBSR on Heterogeneous MZR Disk Arra

ICPP '99 Proceedings of the 1999 International Workshops on Parallel Processing
The Google file system

SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
Adaptive Overlapped Declustering: A Highly Available Data-Placement Method Balancing Access Load and Space Utilization

ICDE '05 Proceedings of the 21st International Conference on Data Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

As storage system scales to thousands of disks, data distribution, load balance and the support for heterogeneous disks become increasingly important. In this paper, we present a new data-placement method named Weighted Interval Algorithm (WIA) for heterogeneous disks. Through it is not optimal in some circumstances, the difference between WIA and the optimal algorithm is trivial. Combined with replication, WIA can nearly balance access load and space utilization and improve reliability simultaneously. For the first time, we implement a data-placement system with high scalability, reliability and performance. The experimental results show that WIA reduces the average response time by 14.8% and decreases coefficient of relative load from 78.09% to 47.46% while the difference of the ratio of space utilization between disks is not more than 0.79%.