Clustering metadata servers for extremely scalable and reliable services

Authors:
Keun-Tae Park;Myung-Hoon Cha;Young-soo Min;Young-Kyun Kim;Han Namgoong
Affiliations:
S/W Content Research Laboratory, Internet Platform Research Department, Electronics and Telecommunications Research Institute;S/W Content Research Laboratory, Internet Platform Research Department, Electronics and Telecommunications Research Institute;S/W Content Research Laboratory, Internet Platform Research Department, Electronics and Telecommunications Research Institute;S/W Content Research Laboratory, Internet Platform Research Department, Electronics and Telecommunications Research Institute;S/W Content Research Laboratory, Internet Platform Research Department, Electronics and Telecommunications Research Institute
Venue:
ICACT'09 Proceedings of the 11th international conference on Advanced Communication Technology - Volume 3
Year:
2009

Citing 4
Cited 0

The Vesta parallel file system

ACM Transactions on Computer Systems (TOCS)
Efficient Metadata Management in Large Distributed Storage Systems

MSS '03 Proceedings of the 20 th IEEE/11 th NASA Goddard Conference on Mass Storage Systems and Technologies (MSS'03)
Dynamic Metadata Management for Petabyte-Scale File Systems

Proceedings of the 2004 ACM/IEEE conference on Supercomputing
Lake: Towards Highly Manageable Cluster Storage for Extremely Scalable Services

ICCSA '08 Proceedings of the 2008 International Conference on Computational Sciences and Its Applications

Quantified Score

Hi-index	0.02

Visualization

Abstract

LakeFS[1] is a cluster file system designed for high scalable and reliable storage service. It provides excellent scalability and availability on data I/O operations. However, it lacks the ability to scale up the metadata operations, which makes the metadata server (MDS) as a single point of bottleneck especially when full scan of metadata is needed. In this paper, we present a way to cluster MDSs to solve the bottleneck problem. We also change the file system utilities to exploit the cluster fully. Experimental results show that the throughput of metadata operation is enhanced linearly as the number of MDS participating in the cluster increases.