Andrew: a distributed personal computing environment
Communications of the ACM - The MIT Press scientific computation series
Coda: A Highly Available File System for a Distributed Workstation Environment
IEEE Transactions on Computers
Efficient Metadata Management in Large Distributed Storage Systems
MSS '03 Proceedings of the 20 th IEEE/11 th NASA Goddard Conference on Mass Storage Systems and Technologies (MSS'03)
Dynamic Metadata Management for Petabyte-Scale File Systems
Proceedings of the 2004 ACM/IEEE conference on Supercomputing
Exporting Storage Systems in a Scalable Manner with pNFS
MSST '05 Proceedings of the 22nd IEEE / 13th NASA Goddard Conference on Mass Storage Systems and Technologies
CRUSH: controlled, scalable, decentralized placement of replicated data
Proceedings of the 2006 ACM/IEEE conference on Supercomputing
A comparison of file system workloads
ATEC '00 Proceedings of the annual conference on USENIX Annual Technical Conference
DOIDFH: an Effective Distributed Metadata Management Scheme
ICCSA '07 Proceedings of the The 2007 International Conference Computational Science and its Applications
A Formal Model of Crash Recovery in a Distributed System
IEEE Transactions on Software Engineering
Scalable performance of the Panasas parallel file system
FAST'08 Proceedings of the 6th USENIX Conference on File and Storage Technologies
Ceph: reliable, scalable, and high-performance distributed storage
Ceph: reliable, scalable, and high-performance distributed storage
MHS: A distributed metadata management strategy
Journal of Systems and Software
A transparently-scalable metadata service for the Ursa Minor storage system
USENIXATC'10 Proceedings of the 2010 USENIX conference on USENIX annual technical conference
The Hadoop Distributed File System
MSST '10 Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST)
Scale and concurrency of GIGA+: file system directories with millions of files
FAST'11 Proceedings of the 9th USENIX conference on File and stroage technologies
IEEE Communications Magazine
Kerberos: an authentication service for computer networks
IEEE Communications Magazine
| Hi-index | 0.00 |
We present the design and implementation of both an enhanced new type of OSD device, the OSD+ device, and a metadata cluster based on it. The new OSD+ devices support data objects and directory objects. Unlike "data" objects, present in a traditional OSD, directory objects store file names and attributes, and support metadata-related operations. By using OSD+ devices, we show how the metadata cluster of the Fusion Parallel File System (FPFS) can effectively be managed by all the servers in a system, improving the performance, scalability and availability of the metadata service. We also describe how a directory with millions of files, and accessed by thousands of clients at the same time, is efficiently distributed across several servers to provide high IOPS rates. The performance of our metadata cluster based on OSD+s has been evaluated and compared with that achieved by Lustre. The results show that our proposal obtains a better throughput than Lustre when both use a single metadata server, easily getting improvements of more than 60---80%, and that the performance scales with the number of OSD+s. They also show that FPFS is able to provide a sustained throughput of more than 70,000 creates per second, and more than 120,000 stats per second, for huge directories on a cluster with just 8 OSD+s.