IEEE Transactions on Parallel and Distributed Systems
| Hi-index | 0.00 |
The use of parallel machines to solve large scalecomputational problems in science and engineeringhas increased considerably in recent times. Many ofthese problems have computational requirements whichstretch the capabilities of even the fastest machineavailable today. In addition to requiring a great deal ofcomputational power, these problems usually deal withlarge quantities of data up to a few terabytes. The mainmemory sizes of current parallel machines do not evencome close to matching these requirements; hence dataneeds to be stored on disks and fetched during the execution of the program. Unfortunately, current optimizing compilers for parallel machines provide supportonly for in-core computations in which the data setscan fit into memory. This limitation severely affectsthe performance of programs which depend on disk resident data. Our previous research demonstrated thatfile layout optimizations are extremely important foroptimizing such programs. In this paper we investigate solutions to the global I/O optimization problemfor out-of-core computations. Since the general problem is NP-complete, we present fast heuristics that canresult in near-optimal solutions for the programs encountered in practice. Preliminary results provide encouraging evidence that our algorithms can be successfulin optimizing out-of-core programs.