A specialized data management system for parallel execution of particle physics codes
SIGMOD '88 Proceedings of the 1988 ACM SIGMOD international conference on Management of data
PODS '93 Proceedings of the twelfth ACM SIGACT-SIGMOD-SIGART symposium on Principles of database systems
Design and Evaluation of primitives for Parallel I/O
Proceedings of the 1993 ACM/IEEE conference on Supercomputing
Applications-driven parallel I/O
Proceedings of the 1993 ACM/IEEE conference on Supercomputing
The SEQUOIA 2000 storage benchmark
SIGMOD '93 Proceedings of the 1993 ACM SIGMOD international conference on Management of data
Shoring up persistent applications
SIGMOD '94 Proceedings of the 1994 ACM SIGMOD international conference on Management of data
Dynamic file-access characteristics of a production parallel scientific workload
Proceedings of the 1994 ACM/IEEE conference on Supercomputing
An efficient abstract interface for multidimensional array I/O
Proceedings of the 1994 ACM/IEEE conference on Supercomputing
Efficient Organization of Large Multidimensional Arrays
Proceedings of the Tenth International Conference on Data Engineering
Language, compiler and parallel database support for I/O intensive applications
HPCN Europe '95 Proceedings of the International Conference and Exhibition on High-Performance Computing and Networking
VLDB '94 Proceedings of the 20th International Conference on Very Large Data Bases
A data management approach for handling large compressed arrays in high performance computing
FRONTIERS '95 Proceedings of the Fifth Symposium on the Frontiers of Massively Parallel Computation (Frontiers'95)
Compiler support for out-of-core arrays on parallel machines
FRONTIERS '95 Proceedings of the Fifth Symposium on the Frontiers of Massively Parallel Computation (Frontiers'95)
High-Level Fault Tolerance in Distributed Programs
High-Level Fault Tolerance in Distributed Programs
Throughput of Existing Multiprocessor File Systems (An Informal Study)
Throughput of Existing Multiprocessor File Systems (An Informal Study)
Disk-directed I/O for an Out-of-Core Computation
Disk-directed I/O for an Out-of-Core Computation
Expanding the Potential for Disk-Directed I/O
Expanding the Potential for Disk-Directed I/O
Characterizing Parallel File-Access Patterns on a Large-Scale Multiprocessor
Characterizing Parallel File-Access Patterns on a Large-Scale Multiprocessor
Disk-directed I/O for MIMD multiprocessors
OSDI '94 Proceedings of the 1st USENIX conference on Operating Systems Design and Implementation
Parallel Input/Output with Heterogeneous Disks
SSDBM '97 Proceedings of the Ninth International Conference on Scientific and Statistical Database Management
| Hi-index | 0.00 |
Large multidimensional arrays are a common data type in high-performance scientific applications. Without special techniques for handling access to these arrays, I/O can easily become a large fraction of execution time for applications using these arrays, especially on parallel platforms. We show how to reduce the parallel I/O bottleneck for array data in closely-synchronized SPMD applications on distributed-memory platforms, through the use of server-directed I/O. This method allows array data requests on parallel platforms to be translated into long sequential disk reads and writes, while also minimizing the cost of rearranging data as they move between on-disk and in-memory schemas. We present experimental results from the implementation of server-directed I/O in Panda, showing that for I/O of large arrays, Panda utilizes nearly the maximum throughput of the underlying AIX file system on an IBM SP2. We also discuss Panda's user interface, an essential factor in Panda's high performance.