A scheduling framework for large-scale, parallel, and topology-aware applications

Authors:
Valentin Kravtsov;Pavel Bar;David Carmeli;Assaf Schuster;Martin Swain
Affiliations:
Computer Science Department, Technion-Israel Institute of Technology, Technion City, Haifa, Israel;Computer Science Department, Technion-Israel Institute of Technology, Technion City, Haifa, Israel;Computer Science Department, Technion-Israel Institute of Technology, Technion City, Haifa, Israel;Computer Science Department, Technion-Israel Institute of Technology, Technion City, Haifa, Israel;Systems Biology Research Group, University of Ulster, Cromore Road, Coleraine, BT52 1SA, Northern Ireland, UK
Venue:
Journal of Parallel and Distributed Computing
Year:
2010

Citing 11
Cited 3

Convergence of an annealing algorithm

Mathematical Programming: Series A and B
A Graduated Assignment Algorithm for Graph Matching

IEEE Transactions on Pattern Analysis and Machine Intelligence
Developing a simulated annealing algorithm for the cutting stock problem

Computers and Industrial Engineering
Handbook of Scheduling: Algorithms, Models, and Performance Analysis

Handbook of Scheduling: Algorithms, Models, and Performance Analysis
Grid-enabling data mining applications with DataMiningGrid: An architectural perspective

Future Generation Computer Systems
A Fast and Efficient Algorithm for Topology-Aware Coallocation

ICCS '08 Proceedings of the 8th international conference on Computational Science, Part I
A Network Bandwidth-Aware Job Scheduling with Dynamic Information Model for Grid Resource Brokers

APSCC '08 Proceedings of the 2008 IEEE Asia-Pacific Services Computing Conference
Effects of Topology-Aware Allocation Policies on Scheduling Performance

Job Scheduling Strategies for Parallel Processing
Running Parallel Applications with Topology-Aware Grid Middleware

E-SCIENCE '09 Proceedings of the 2009 Fifth IEEE International Conference on e-Science
A meta-scheduling service for co-allocating arbitrary types of resources

PPAM'05 Proceedings of the 6th international conference on Parallel Processing and Applied Mathematics
An integration of global and enterprise grid computing: gridbus broker and xgrid perspective

GCC'05 Proceedings of the 4th international conference on Grid and Cooperative Computing

Optimizing end-to-end performance of data-intensive computing pipelines in heterogeneous network environments

Journal of Parallel and Distributed Computing
QCG-OMPI: MPI applications on grids

Future Generation Computer Systems
New capabilities in qoscosgrid middleware for advanced job management, advance reservation and co-allocation of computing resources --- quantum chemistry application use case

Building a National Distributed e-Infrastructure - PL-Grid

Quantified Score

Hi-index	0.01

Visualization

Abstract

Scheduling of large-scale, distributed topology-aware applications requires that not only the properties of the requested machines be considered, but also the properties of the machines' interconnections. This requirement severely complicates the scheduling process, as even a matching between a single multi-processor task and available machines in a single time slot becomes an NP-complete problem with no polynomial approximation. In this paper we propose a complete scheduling framework for multi-cluster, heterogeneous environments that provides, in practice, an efficient solution for the scheduling of topology-aware applications. The proposed framework is very flexible as it is composed of pluggable components and can be easily configured to support a variety of scheduling policies. We also describe three novel scheduling and coallocation algorithms that were developed and plugged into the framework. The proposed scheduling framework was integrated into the QosCosGrid system, where it is used as the main decision-making module.