Data Locality Aware Strategy for Two-Phase Collective I/O

Authors:
Rosa Filgueira;David E. Singh;Juan C. Pichel;Florin Isaila;Jesús Carretero
Affiliations:
Department of Computer Science, Universidad Carlos III de Madrid,;Department of Computer Science, Universidad Carlos III de Madrid,;Department of Computer Science, Universidad Carlos III de Madrid,;Department of Computer Science, Universidad Carlos III de Madrid,;Department of Computer Science, Universidad Carlos III de Madrid,
Venue:
High Performance Computing for Computational Science - VECPAR 2008
Year:
2008

Citing 12
Cited 1

A shortest augmenting path algorithm for dense and sparse linear assignment problems

Computing
Server-directed collective I/O in Panda

Supercomputing '95 Proceedings of the 1995 ACM/IEEE conference on Supercomputing
File-Access Characteristics of Parallel Scientific Workloads

IEEE Transactions on Parallel and Distributed Systems
Implementation of collective I/O in the Intel Paragon parallel file system: initial experiences

ICS '97 Proceedings of the 11th international conference on Supercomputing
Optimizing noncontiguous accesses in MPI – IO

Parallel Computing
GPFS: A Shared-Disk File System for Large Computing Clusters

FAST '02 Proceedings of the Conference on File and Storage Technologies
Data Sieving and Collective I/O in ROMIO

FRONTIERS '99 Proceedings of the The 7th Symposium on the Frontiers of Massively Parallel Computation
Integrating collective I/O and cooperative caching into the "clusterfile" parallel file system

Proceedings of the 18th annual international conference on Supercomputing
Exploiting Lustre File Joining for Effective Collective IO

CCGRID '07 Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid
Disk-directed I/O for MIMD multiprocessors

OSDI '94 Proceedings of the 1st USENIX conference on Operating Systems Design and Implementation
Collective caching: application-aware client-side file caching

HPDC '05 Proceedings of the High Performance Distributed Computing, 2005. HPDC-14. Proceedings. 14th IEEE International Symposium
Cooperative write-behind data buffering for MPI i/o

PVM/MPI'05 Proceedings of the 12th European PVM/MPI users' group conference on Recent Advances in Parallel Virtual Machine and Message Passing Interface

Locality and topology aware intra-node communication among multicore CPUs

EuroMPI'10 Proceedings of the 17th European MPI users' group meeting conference on Recent advances in the message passing interface

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents Locality-Aware Two-Phase (LATP) I/O, an optimization of the Two-Phase collective I/O technique from ROMIO, the most popular MPI-IO implementation. In order to increase the locality of the file accesses, LATP employs the Linear Assignment Problem (LAP) for finding an optimal distribution of data to processes, an aspect that is not considered in the original technique. This assignment is based on the local data that each process stores and has as main purpose the reduction of the number of communication involved in the I/O collective operation and, therefore, the improvement of the global execution time. Compared with Two-Phase I/O, LATP I/O obtains important improvements in most of the considered scenarios.