Scale and performance in a distributed file system
ACM Transactions on Computer Systems (TOCS)
Parity declustering for continuous operation in redundant disk arrays
ASPLOS V Proceedings of the fifth international conference on Architectural support for programming languages and operating systems
The Zebra striped network file system
SOSP '93 Proceedings of the fourteenth ACM symposium on Operating systems principles
File server scaling with network-attached secure disks
SIGMETRICS '97 Proceedings of the 1997 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
ACM Transactions on Computer Systems (TOCS)
A cost-effective, high-bandwidth storage architecture
Proceedings of the eighth international conference on Architectural support for programming languages and operating systems
GPFS: A Shared-Disk File System for Large Computing Clusters
FAST '02 Proceedings of the Conference on File and Storage Technologies
SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
The Panasas ActiveScale Storage Cluster: Delivering Scalable High Bandwidth Storage
Proceedings of the 2004 ACM/IEEE conference on Supercomputing
Exporting Storage Systems in a Scalable Manner with pNFS
MSST '05 Proceedings of the 22nd IEEE / 13th NASA Goddard Conference on Mass Storage Systems and Technologies
High performance support of parallel virtual file system (PVFS2) over Quadrics
Proceedings of the 19th annual international conference on Supercomputing
Ursa minor: versatile cluster-based storage
FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
File system design for an NFS file server appliance
WTEC'94 Proceedings of the USENIX Winter 1994 Technical Conference on USENIX Winter 1994 Technical Conference
Ceph: a scalable, high-performance distributed file system
OSDI '06 Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation - Volume 7
RAIF: Redundant Array of Independent Filesystems
MSST '07 Proceedings of the 24th IEEE Conference on Mass Storage Systems and Technologies
Integrating parallel file systems with object-based storage devices
Proceedings of the 2007 ACM/IEEE conference on Supercomputing
Integrated system models for reliable petascale storage systems
PDSW '07 Proceedings of the 2nd international workshop on Petascale data storage: held in conjunction with Supercomputing '07
Integrated system models for reliable petascale storage systems
PDSW '07 Proceedings of the 2nd international workshop on Petascale data storage: held in conjunction with Supercomputing '07
A comparative experimental study of parallel file systems for large-scale data processing
LASCO'08 First USENIX Workshop on Large-Scale Computing
Proceedings of the 4th ACM international workshop on Storage security and survivability
HYDRAstor: a Scalable Secondary Storage
FAST '09 Proccedings of the 7th conference on File and storage technologies
WorkOut: I/O workload outsourcing for boosting RAID reconstruction performance
FAST '09 Proccedings of the 7th conference on File and storage technologies
A performance evaluation and examination of open-source erasure coding libraries for storage
FAST '09 Proccedings of the 7th conference on File and storage technologies
Abstract storage: moving file format-specific abstractions intopetabyte-scale storage systems
Proceedings of the second international workshop on Data-aware distributed computing
International Journal of High Performance Computing Applications
Safe and effective fine-grained TCP retransmissions for datacenter communication
Proceedings of the ACM SIGCOMM 2009 conference on Data communication
MHS: A distributed metadata management strategy
Journal of Systems and Software
PLFS: a checkpoint filesystem for parallel applications
Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis
I/O performance challenges at leadership scale
Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis
Lock management in blue whale file system
Proceedings of the 2nd International Conference on Interaction Sciences: Information Technology, Culture and Human
DiskReduce: RAID for data-intensive scalable computing
Proceedings of the 4th Annual Workshop on Petascale Data Storage
pNFS, POSIX, and MPI-IO: a tale of three semantics
Proceedings of the 4th Annual Workshop on Petascale Data Storage
Panache: a parallel file system cache for global file access
FAST'10 Proceedings of the 8th USENIX conference on File and storage technologies
In search of an API for scalable file systems: under the table or above it?
HotCloud'09 Proceedings of the 2009 conference on Hot topics in cloud computing
A transparently-scalable metadata service for the Ursa Minor storage system
USENIXATC'10 Proceedings of the 2010 USENIX conference on USENIX annual technical conference
RSEDP: an effective hybrid data placement algorithm for large-scale storage systems
The Journal of Supercomputing
Scale and concurrency of GIGA+: file system directories with millions of files
FAST'11 Proceedings of the 9th USENIX conference on File and stroage technologies
Why specialized disks for composite operations may be unnecessary
ACM SIGARCH Computer Architecture News
The impact of applications' I/O strategies on the performance of the Lustre parallel file system
International Journal of High Performance Systems Architecture
A cost-intelligent application-specific data layout scheme for parallel file systems
Proceedings of the 20th international symposium on High performance distributed computing
RAID level selection for heterogeneous disk arrays
Cluster Computing
TidyFS: a simple and small distributed file system
USENIXATC'11 Proceedings of the 2011 USENIX conference on USENIX annual technical conference
FTI: high performance fault tolerance interface for hybrid systems
Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis
On the duality of data-intensive file system design: reconciling HDFS and PVFS
Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis
Using active NVRAM for I/O staging
Proceedings of the 2nd international workshop on Petascal data analytics: challenges and opportunities
Towards scalable I/O architecture for exascale systems
Proceedings of the 2011 ACM international workshop on Many task computing on grids and supercomputers
Horus: fine-grained encryption-based security for high performance petascale storage
Proceedings of the sixth workshop on Parallel Data Storage
Easing the burdens of HPC file management
Proceedings of the sixth workshop on Parallel Data Storage
The TCP outcast problem: exposing unfairness in data center networks
NSDI'12 Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation
Rebuild processing in RAID5 with emphasis on the supplementary parity augmentation method[37]
ACM SIGARCH Computer Architecture News
Consistency and fault tolerance for erasure-coded distributed storage systems
Proceedings of the fifth international workshop on Data-Intensive Distributed Computing Date
File system virtual appliances: Portable file system implementations
ACM Transactions on Storage (TOS)
HFAA: a generic socket API for Hadoop file systems
Proceedings of the 2nd Workshop on Architectures and Systems for Big Data
OSDI'12 Proceedings of the 10th USENIX conference on Operating Systems Design and Implementation
Metadata Traces and Workload Models for Evaluating Big Storage Systems
UCC '12 Proceedings of the 2012 IEEE/ACM Fifth International Conference on Utility and Cloud Computing
lisa'12 Proceedings of the 26th international conference on Large Installation System Administration: strategies, tools, and techniques
ACM Transactions on Storage (TOS)
Memory-conscious collective I/O for extreme scale HPC systems
Proceedings of the 3rd International Workshop on Runtime and Operating Systems for Supercomputers
Zone-based data striping for cloud storage
IBM Journal of Research and Development
Rangoli: space management in deduplication environments
Proceedings of the 6th International Systems and Storage Conference
Insights for exascale IO APIs from building a petascale IO API
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
TABLEFS: enhancing metadata efficiency in the local file system
USENIX ATC'13 Proceedings of the 2013 USENIX conference on Annual Technical Conference
Trevi: watering down storage hotspots with cool fountain codes
Proceedings of the Twelfth ACM Workshop on Hot Topics in Networks
Structuring PLFS for extensibility
PDSW '13 Proceedings of the 8th Parallel Data Storage Workshop
Asynchronous object storage with QoS for scientific and commercial big data
PDSW '13 Proceedings of the 8th Parallel Data Storage Workshop
Sector-Disk (SD) Erasure Codes for Mixed Failure Modes in RAID Systems
ACM Transactions on Storage (TOS)
Scalable Metadata Management Through OSD+ Devices
International Journal of Parallel Programming
SD codes: erasure codes designed for how storage systems really fail
FAST'13 Proceedings of the 11th USENIX conference on File and Storage Technologies
Horus: fine-grained encryption-based security for large-scale storage
FAST'13 Proceedings of the 11th USENIX conference on File and Storage Technologies
FAST'14 Proceedings of the 12th USENIX conference on File and Storage Technologies
A novel approach to data deduplication over the engineering-oriented cloud systems
Integrated Computer-Aided Engineering
Hi-index | 0.00 |
The Panasas file system uses parallel and redundant access to object storage devices (OSDs), per-file RAID, distributed metadata management, consistent client caching, file locking services, and internal cluster management to provide a scalable, fault tolerant, high performance distributed file system. The clustered design of the storage system and the use of client-driven RAID provide scalable performance to many concurrent file system clients through parallel access to file data that is striped across OSD storage nodes. RAID recovery is performed in parallel by the cluster of metadata managers, and declustered data placement yields scalable RAID rebuild rates as the storage system grows larger. This paper presents performance measures of I/O, metadata, and recovery operations for storage clusters that range in size from 10 to 120 storage nodes, 1 to 12 metadata nodes, and with file system client counts ranging from 1 to 100 compute nodes. Production installations are as large as 500 storage nodes, 50 metadata managers, and 5000 clients.