SIGMOD '90 Proceedings of the 1990 ACM SIGMOD international conference on Management of data
Data prefetching in multiprocessor vector cache memories
ISCA '91 Proceedings of the 18th annual international symposium on Computer architecture
The design and implementation of a log-structured file system
ACM Transactions on Computer Systems (TOCS)
Informed prefetching and caching
SOSP '95 Proceedings of the fifteenth ACM symposium on Operating systems principles
Server-directed collective I/O in Panda
Supercomputing '95 Proceedings of the 1995 ACM/IEEE conference on Supercomputing
Disk-directed I/O for MIMD multiprocessors
ACM Transactions on Computer Systems (TOCS)
An extended two-phase method for accessing sections of out-of-core arrays
Scientific Programming
An efficient profile-analysis framework for data-layout optimizations
POPL '02 Proceedings of the 29th ACM SIGPLAN-SIGACT symposium on Principles of programming languages
Faster Collective Output through Active Buffering
IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
Profile-guided I/O partitioning
ICS '03 Proceedings of the 17th annual international conference on Supercomputing
An Efficient Lock Protocol for Home-Based Lazy Release Consistency
CCGRID '01 Proceedings of the 1st International Symposium on Cluster Computing and the Grid
Noncontiguous I/O Accesses Through MPI-IO
CCGRID '03 Proceedings of the 3st International Symposium on Cluster Computing and the Grid
Clusterfile: A Flexible Physical Layout Parallel File System
CLUSTER '01 Proceedings of the 3rd IEEE International Conference on Cluster Computing
Data Sieving and Collective I/O in ROMIO
FRONTIERS '99 Proceedings of the The 7th Symposium on the Frontiers of Massively Parallel Computation
HPDC '96 Proceedings of the 5th IEEE International Symposium on High Performance Distributed Computing
Collective Buffering: Improving Parallel I/O Performance
HPDC '97 Proceedings of the 6th IEEE International Symposium on High Performance Distributed Computing
Automatic ARIMA Time Series Modeling for Adaptive I/O Prefetching
IEEE Transactions on Parallel and Distributed Systems
Integrating collective I/O and cooperative caching into the "clusterfile" parallel file system
Proceedings of the 18th annual international conference on Supercomputing
GPFS: A Shared-Disk File System for Large Computing Clusters
FAST '02 Proceedings of the 1st USENIX Conference on File and Storage Technologies
Proceedings of the twentieth ACM symposium on Operating systems principles
The automatic improvement of locality in storage systems
ACM Transactions on Computer Systems (TOCS)
Towards higher disk head utilization: extracting free bandwidth from busy disk drives
OSDI'00 Proceedings of the 4th conference on Symposium on Operating System Design & Implementation - Volume 4
Fixed and Adaptive Sequential Prefetching in Shared Memory Multiprocessors
ICPP '93 Proceedings of the 1993 International Conference on Parallel Processing - Volume 01
PVFS: a parallel file system for linux clusters
ALS'00 Proceedings of the 4th annual Linux Showcase & Conference - Volume 4
An analytical approach to file prefetching
ATEC '97 Proceedings of the annual conference on USENIX Annual Technical Conference
Collective caching: application-aware client-side file caching
HPDC '05 Proceedings of the High Performance Distributed Computing, 2005. HPDC-14. Proceedings. 14th IEEE International Symposium
Scalable performance of the Panasas parallel file system
FAST'08 Proceedings of the 6th USENIX Conference on File and Storage Technologies
Measurement and analysis of TCP throughput collapse in cluster-based storage systems
FAST'08 Proceedings of the 6th USENIX Conference on File and Storage Technologies
Hiding I/O latency with pre-execution prefetching for parallel applications
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Parallel I/O prefetching using MPI file caching and I/O signatures
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
BORG: block-reORGanization for self-optimizing storage systems
FAST '09 Proccedings of the 7th conference on File and storage technologies
IPDPS '09 Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing
Safe and effective fine-grained TCP retransmissions for datacenter communication
Proceedings of the ACM SIGCOMM 2009 conference on Data communication
Data layout optimization for petascale file systems
Proceedings of the 4th Annual Workshop on Petascale Data Storage
Scalable I/O tracing and analysis
Proceedings of the 4th Annual Workshop on Petascale Data Storage
InterferenceRemoval: removing interference of disk access for MPI programs through data replication
Proceedings of the 24th ACM International Conference on Supercomputing
Boosting Application-Specific Parallel I/O Optimization Using IOSIG
CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
Hi-index | 0.00 |
I/O data access is a recognized performance bottleneck of high-end computing. Several commercial and research parallel file systems have been developed in recent years to ease the performance bottleneck. These advanced file systems perform well on some applications but may not perform well on others. They have not reached their full potential in mitigating the I/O-wall problem. Data access is application dependent. Based on the application-specific optimization principle, in this study we propose a cost-intelligent data access strategy to improve the performance of parallel file systems. We first present a novel model to estimate data access cost of different data layout policies. Next, we extend the cost model to calculate the overall I/O cost of any given application and choose an appropriate layout policy for the application. A complex application may consist of different data access patterns. Averaging the data access patterns may not be the best solution for those complex applications that do not have a dominant pattern. We then further propose a hybrid data replication strategy for those applications, so that a file can have replications with different layout policies for the best performance. Theoretical analysis and experimental testing have been conducted to verify the newly proposed cost-intelligent layout approach. Analytical and experimental results show that the proposed cost model is effective and the application-specific data layout approach achieved up to 74% performance improvement for data-intensive applications.