A parallel out-of-core computing system using PVFS for Linux clusters

Authors:
Jianqi Tang;Binxing Fang;Mingzeng Hu;Hongli Zhang
Affiliations:
Harbin Institute of Technology, Harbin, China;Harbin Institute of Technology, Harbin, China;Harbin Institute of Technology, Harbin, China;Harbin Institute of Technology, Harbin, China
Venue:
SNAPI '04 Proceedings of the international workshop on Storage network architecture and parallel I/Os
Year:
2004

Citing 4
Cited 0

Compilation and communication strategies for out-of-core programs on distributed memory machines

Journal of Parallel and Distributed Computing - Special issue on compilation techniques for distributed memory systems
PVFS: parallel virtual file system

Beowulf cluster computing with Linux
Unifier: unifying cache management and communication buffer management for PVFS over InfiniBand

CCGRID '04 Proceedings of the 2004 IEEE International Symposium on Cluster Computing and the Grid
PVFS: a parallel file system for linux clusters

ALS'00 Proceedings of the 4th annual Linux Showcase & Conference - Volume 4

Quantified Score

Hi-index	0.00

Visualization

Abstract

Cluster systems become a new and popular approach to parallel computing. More and more scientists and engineers use clusters to solve problems with large sets of data for its high processing power, low price and good scalability. Since the traditional out-of-core programs are difficult to write and the virtual memory system does not perform well, we develop a parallel out-of-core computing system using PVFS named POCCS. POCCS provides convenient interface to write out-of-core codes and the global view of the out-of-core data. The software architecture, data storage model and system implementation are described in this paper. The experimental results show that POCCS extends the problem sizes that can be solved and the performance of POCCS is better than the virtual memory system while the data set is large.