Data transfer in advance on cluster

Authors:
Nilton Cézar de Paula;Gisele da Silva Craveiro;Liria Matsumoto Sato
Affiliations:
Polytechnic School, University of São Paulo, Brazil;School of Arts, Science and Humanities, University of São Paulo, Brazil;Polytechnic School, University of São Paulo, Brazil
Venue:
PaCT'07 Proceedings of the 9th international conference on Parallel Computing Technologies
Year:
2007

Citing 9
Cited 0

Throughput Analysis of Some Idealized Input, Output, and Compute Overlap Configurations

ACM Computing Surveys (CSUR)
Data management and transfer in high-performance computational grid environments

Parallel Computing - Parallel data-intensive algorithms and applications
Matchmaking: Distributed Resource Management for High Throughput Computing

HPDC '98 Proceedings of the 7th IEEE International Symposium on High Performance Distributed Computing
Resource Management through Multilateral Matchmaking

HPDC '00 Proceedings of the 9th IEEE International Symposium on High Performance Distributed Computing
The Kangaroo Approach to Data Movement on the Grid

HPDC '01 Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing
Network and cpu co-allocation in high throughput computing environments

Network and cpu co-allocation in high throughput computing environments
Stork: Making Data Placement a First Class Citizen in the Grid

ICDCS '04 Proceedings of the 24th International Conference on Distributed Computing Systems (ICDCS'04)
Distributed computing in practice: the Condor experience: Research Articles

Concurrency and Computation: Practice & Experience - Grid Performance
Multi-Replication with Intelligent Staging in Data-Intensive Grid Applications

GRID '06 Proceedings of the 7th IEEE/ACM International Conference on Grid Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Scientific applications are increasingly challenging computational platforms and software tools. In this scenario of improving performance demand, computer cluster users require for mechanisms that could reduce data transfer delay. To contribute to this question, we proposed a data transfer in advance mechanism that improved overall system performance by diminishing data-intensive job wait. We also designed and implemented the Integrated Scheduling System (ISS) to analyze and evaluate our proposal. This system automates the preparation, submission and tracking of job executions in a cluster. The mechanism is also combined with an I/O file operation and computation overlapping idea that results in significant improvement of performance rates, confirmed by some experiments.