A case for redundant arrays of inexpensive disks (RAID)
SIGMOD '88 Proceedings of the 1988 ACM SIGMOD international conference on Management of data
SOSP '91 Proceedings of the thirteenth ACM symposium on Operating systems principles
Petal: distributed virtual disks
Proceedings of the seventh international conference on Architectural support for programming languages and operating systems
File server scaling with network-attached secure disks
SIGMETRICS '97 Proceedings of the 1997 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
A security architecture for computational grids
CCS '98 Proceedings of the 5th ACM conference on Computer and communications security
Separating key management from file system security
Proceedings of the seventeenth ACM symposium on Operating systems principles
Scalable parallel application launch on Cplant™
Proceedings of the 2001 ACM/IEEE conference on Supercomputing
DYNAMO - DirectorY, Net Archiver and MOver
GRID '02 Proceedings of the Third International Workshop on Grid Computing
Active Storage for Large-Scale Data Mining and Multimedia
VLDB '98 Proceedings of the 24rd International Conference on Very Large Data Bases
A Toolkit for User-Level File Systems
Proceedings of the General Track: 2002 USENIX Annual Technical Conference
The SDSC storage resource broker
CASCON '98 Proceedings of the 1998 conference of the Centre for Advanced Studies on Collaborative research
The Globus Striped GridFTP Framework and Server
SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
Separating Abstractions from Resources in a Tactical Storage System
SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
FreeLoader: Scavenging Desktop Storage Resources for Scientific Data
SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
PVFS: a parallel file system for linux clusters
ALS'00 Proceedings of the 4th annual Linux Showcase & Conference - Volume 4
Cluster delegation: high-performance, fault-tolerant data sharing in NFS
HPDC '05 Proceedings of the High Performance Distributed Computing, 2005. HPDC-14. Proceedings. 14th IEEE International Symposium
Lerna: an active storage framework for flexible data access and management
HPDC '05 Proceedings of the High Performance Distributed Computing, 2005. HPDC-14. Proceedings. 14th IEEE International Symposium
Operating System Support for Space Allocation in Grid Storage Systems
GRID '06 Proceedings of the 7th IEEE/ACM International Conference on Grid Computing
IEEE Communications Magazine
CCGRID '09 Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid
Case studies in storage access by loosely coupled petascale applications
Proceedings of the 4th Annual Workshop on Petascale Data Storage
What Is the price of simplicity? a cross-platform evaluation of the SAGA API
EuroPar'10 Proceedings of the 16th international Euro-Par conference on Parallel processing: Part I
PetaShare: A reliable, efficient and transparent distributed storage management system
Scientific Programming
Experiences with 100Gbps network applications
Proceedings of the fifth international workshop on Data-Intensive Distributed Computing Date
International Journal of Business Information Systems
Future Generation Computer Systems
| Hi-index | 0.00 |
Many potential users of grid computing systems have a need to manage large numbers of small files. However, computing and storage grids are generally optimized for the management of large files. As a result, users with small files achieve performance several orders of magnitude worse than possible. Archival tools and custom storage structures can be used to improve small-file performance, but this requires the end user to change the behavior of the application, which is not always practical. To address this problem, we augment the protocol of the Chirp filesystem for grid computing to improve small file performance. We describe in detail how this protocol compares to FTP and NFS, which are widely used in similar situations. In addition, we observe that changes to the system call interface are necessary to invoke the protocol properly. We demonstrate an order-of-magnitude performance improvement over existing protocols for copying files and manipulating large directory trees.