Improving job scheduling performance with parallel access to replicas in Data Grid environment

Authors:
Junwei Zhang;Bu-Sung Lee;Xueyan Tang;Chai-Kiat Yeo
Affiliations:
School of Computer Engineering, Nanyang Technological University, Singapore, Singapore;School of Computer Engineering, Nanyang Technological University, Singapore, Singapore;School of Computer Engineering, Nanyang Technological University, Singapore, Singapore;School of Computer Engineering, Nanyang Technological University, Singapore, Singapore
Venue:
The Journal of Supercomputing
Year:
2011

Citing 22
Cited 0

Local area networks

Local area networks
The grid: blueprint for a new computing infrastructure

The grid: blueprint for a new computing infrastructure
Dynamic mapping of a class of independent tasks onto heterogeneous computing systems

Journal of Parallel and Distributed Computing - Special issue on software support for distributed computing
When the Herd Is Smart: Aggregate Behavior in the Selection of Job Request

IEEE Transactions on Parallel and Distributed Systems
Identifying Dynamic Replication Strategies for a High-Performance Data Grid

GRID '01 Proceedings of the Second International Workshop on Grid Computing
Metrics and Benchmarking for Parallel Job Scheduling

IPPS/SPDP '98 Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing
Benchmarks and Standards for the Evaluation of Parallel Job Schedulers

IPPS/SPDP '99/JSSPP '99 Proceedings of the Job Scheduling Strategies for Parallel Processing
Decoupling Computation and Data Scheduling in Distributed Data-Intensive Applications

HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
Performance Analysis of Scheduling and Replication Algorithms on Grid Datafarm Architecture for High-Energy Physics Applications

HPDC '03 Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing
Enabling the Co-Allocation of Grid Data Transfers

GRID '03 Proceedings of the 4th International Workshop on Grid Computing
Efficient Multi-Source Data Transfer in Data Grids

CCGRID '06 Proceedings of the Sixth IEEE International Symposium on Cluster Computing and the Grid
Implementation of a dynamic adjustment mechanism with efficient replica selection in data grid environments

Proceedings of the 2006 ACM symposium on Applied computing
Complete and fragmented replica selection and retrieval in Data Grids

Future Generation Computer Systems
Improvements on dynamic adjustment mechanism in co-allocation data grid environments

The Journal of Supercomputing
Efficient reuse of replicated parallel data segments in computational grids

Future Generation Computer Systems
A multiple parallel download scheme with server throughput and client bandwidth considerations for data grids

Future Generation Computer Systems
Implementation of a Cyber Transformer for Parallel Download in Co-Allocation Data Grid Environments

GCC '08 Proceedings of the 2008 Seventh International Conference on Grid and Cooperative Computing
Enhancement of anticipative recursively adjusting mechanism for redundant parallel file transfer in data grids

Journal of Network and Computer Applications
Accessing data from many servers simultaneously and adaptively in data grids

Future Generation Computer Systems
Dynamic replication algorithms for the multi-tier Data Grid

Future Generation Computer Systems - Special issue: Parallel computing technologies
The impact of data replication on job scheduling performance in the Data Grid

Future Generation Computer Systems
Implementation of a dynamic adjustment strategy for parallel file transfer in co-allocation data grids

The Journal of Supercomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Data Grid has evolved to be the solution for data-intensive applications, such as High Energy Physics (HEP), astrophysics, and computational genomics. These applications usually have large input of data to be analyzed and these input data are widely replicated across Data Grid to improve the performance. The job scheduling performance on traditional computing jobs can be studied using queuing theory. However, with the addition of data transfer, the job scheduling performance is too complex to be modeled. In this research, we study the impact of data transfer on the performance of job scheduling in the Data Grid environment. We have proposed a parallel downloading system that supports replicating data fragments and parallel downloading of replicated data fragments, to improve the job scheduling performance. The performance of the parallel downloading system is compared with non-parallel downloading system, using three scheduling heuristics: Shortest Turnaround Time (STT), Least Relative Load (LRL) and Data Present (DP). Our simulation results show that the proposed parallel download approach greatly improves the Data Grid performance for all three scheduling algorithms, in terms of the geometric mean of job turnaround time. The advantage of parallel downloading system is most evident when the Data Grid has relatively low network bandwidth and relatively high computing power.