Extensible file system (ELFS): an object-oriented approach to high performance file I/O
OOPSLA '94 Proceedings of the ninth annual conference on Object-oriented programming systems, language, and applications
A novel application development environment for large-scale scientific computations
Proceedings of the 14th international conference on Supercomputing
Using MPI-2: Advanced Features of the Message Passing Interface
Using MPI-2: Advanced Features of the Message Passing Interface
Application-controlled file caching policies
USTC'94 Proceedings of the USENIX Summer 1994 Technical Conference on USENIX Summer 1994 Technical Conference - Volume 1
A high-performance distributed parallel file system for data-intensive computations
Journal of Parallel and Distributed Computing
| Hi-index | 0.00 |
Many scientific applications manipulate large amount of data and, therefore, are parallelized on high-performance computing systems to take advantage of their computational power and memory space. The size of data processed by these large-scale applications can easily overwhelm the disk capacity of most systems. Thus, tertiary storage devices are used to store the data. The parallelization of this type of applications requires understanding of not only the data partition pattern among multiple processors but also the underlying storage architectures and the data storage pattern. In this paper, we present a meta-data management system which uses a database to record the information of datasets and manage these meta data to provide suitable I/O interface. As a result, users specify dataset names instead of data physical location to access data using optimal I/O calls without knowing the underlying storage structure. We use an astrophysics application to demonstrate that the management system can provide convenient programming environment with negligible database access overhead.