An Opportunity Cost Approach for Job Assignment in a Scalable Computing Cluster

Authors:
Yair Amir;Baruch Awerbuch;Amnon Barak;R. Sean Borgstrom;Arie Keren
Affiliations:
John Hopkins Univ., Baltimore, MD;John Hopkins Univ., Baltimore, MD;Hebrew Univ. of Jerusalem, Israel;John Hopkins Univ., Baltimore, MD;Hebrew Univ. of Jerusalem, Israel
Venue:
IEEE Transactions on Parallel and Distributed Systems
Year:
2000

Citing 6
Cited 38

New algorithms for an ancient scheduling problem

STOC '92 Proceedings of the twenty-fourth annual ACM symposium on Theory of computing
New algorithms for an ancient scheduling problem

Journal of Computer and System Sciences - Special issue on selected papers presented at the 24th annual ACM symposium on the theory of computing (STOC '92)
Exploiting process lifetime distributions for dynamic load balancing

Proceedings of the 1996 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
On-line routing of virtual circuits with applications to load balancing and machine scheduling

Journal of the ACM (JACM)
The MOSIX multicomputer operating system for high performance cluster computing

Future Generation Computer Systems - Special issue on HPCN '97
Competitive routing of virtual circuits with unknown duration

SODA '94 Proceedings of the fifth annual ACM-SIAM symposium on Discrete algorithms

Dynamic Cluster Resource Allocations for Jobs with Known and Unknown Memory Demands

IEEE Transactions on Parallel and Distributed Systems
Opportunity Cost Algorithms for Reduction of I/O and Interprocess Communication Overhead in a Computing Cluster

IEEE Transactions on Parallel and Distributed Systems
A Case for Economy Grid Architecture for Service-Oriented Grid Computing

IPDPS '01 Proceedings of the 15th International Parallel & Distributed Processing Symposium
A computational economy for grid computing and its implementation in the Nimrod-G resource broker

Future Generation Computer Systems - Grid computing: Towards a new computing infrastructure
Adaptive Memory Allocations in Clusters to Handle Unexpectedly Large Data-Intensive Jobs

IEEE Transactions on Parallel and Distributed Systems
The use of economic agents under price driven mechanism in grid resource management

Journal of Systems Architecture: the EUROMICRO Journal
Memory Conscious Task Partition and Scheduling in Grid Environments

GRID '04 Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing
A cost-benefit flow control for reliable multicast and unicast in overlay networks

IEEE/ACM Transactions on Networking (TON)
A pricing strategy for job allocation in mobile grids using a non-cooperative bargaining theory framework

Journal of Parallel and Distributed Computing - Special issue: Design and performance of networks for super-, cluster-, and grid-computing: Part II
A taxonomy of market-based resource management systems for utility-driven cluster computing

Software—Practice & Experience
Design and Performance Evaluation of Queue-and-Rate-Adjustment Dynamic Load Balancing Policies for Distributed Networks

IEEE Transactions on Computers
Grid harvest service: a performance system of grid computing

Journal of Parallel and Distributed Computing
Design of a flexible platform for execution of medical decision support agents in the intensive care unit

Computers in Biology and Medicine
Data partitioning for multiprocessors with memory heterogeneity and memory constraints

Scientific Programming - International Symposium of Parallel and Distributed Computing & International Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogenous Networks
Regular Paper: Load-Balanced Drift-Diffusion Model Simulation: Cluster Software Performance Evaluation

International Journal of High Performance Computing Applications
On the performance-driven load distribution for heterogeneous computational grids

Journal of Computer and System Sciences
A macroeconomic model for resource allocation in large-scale distributed systems

Journal of Parallel and Distributed Computing
Comparative analysis of the prototype and the simulator of a new load balancing algorithm for heterogeneous computing environments

International Journal of High Performance Computing and Networking
Self-adaptive task allocation and scheduling of meta-tasks in non-dedicated heterogeneous computing

International Journal of High Performance Computing and Networking
Group decision making with multiple leaders: local rules, weighted networks and consensus

International Journal of Systems, Control and Communications
Reverse combinatorial auction-based protocols for resource selection in grids

International Journal of Grid and Utility Computing
Prediction of dynamical, nonlinear, and unstable process behavior

The Journal of Supercomputing
Real-time scheduling of divisible loads in cluster computing environments

Journal of Parallel and Distributed Computing
Model for simulation of heterogeneous high-performance computing environments

VECPAR'06 Proceedings of the 7th international conference on High performance computing for computational science
Enhanced real-time divisible load scheduling with different processor available times

HiPC'07 Proceedings of the 14th international conference on High performance computing
On fully distributed adaptive load balancing

DSOM'07 Proceedings of the Distributed systems: operations and management 18th IFIP/IEEE international conference on Managing virtualization of networks and services
Multi-round real-time divisible load scheduling for clusters

HiPC'08 Proceedings of the 15th international conference on High performance computing
On cost-aware monitoring for self-adaptive load sharing

IEEE Journal on Selected Areas in Communications
Distributed algorithm for a color assignment on asynchronous rings

IPDPS'06 Proceedings of the 20th international conference on Parallel and distributed processing
A brokering framework for large-scale heterogeneous systems

IPDPS'06 Proceedings of the 20th international conference on Parallel and distributed processing
A network evaluation for LAN, MAN and WAN grid environments

EUC'05 Proceedings of the 2005 international conference on Embedded and Ubiquitous Computing
Towards decentralized load balancing in a computational grid environment

GPC'06 Proceedings of the First international conference on Advances in Grid and Pervasive Computing
Design and analysis of a non-preemptive decentralized load balancing algorithm for multi-class jobs in distributed networks

Computer Communications
A new memory slowdown model for the characterization of computing systems

PaCT'07 Proceedings of the 9th international conference on Parallel Computing Technologies
Cost aware adaptive load sharing

IWSOS'07 Proceedings of the Second international conference on Self-Organizing Systems
Automatic resource-centric process migration for MPI

EuroMPI'12 Proceedings of the 19th European conference on Recent Advances in the Message Passing Interface
On-line fair allocations based on bottlenecks and global priorities

Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering
Review: A survey on resource allocation in high performance distributed computing systems

Parallel Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

A new method is presented for job assignment to and reassignment between machines in a computing cluster. Our method is based on a theoretical framework that has been experimentally tested and shown to be useful in practice. This 驴opportunity cost驴 method converts the usage of several heterogeneous resources in a machine to a single homogeneous 驴cost.驴 Assignment and reassignment are then performed based on that cost. This is in contrast to traditional, ad hoc methods for job assignment and reassignment. These treated each resource as an independent entity with its own constraints, as there was no clean way to balance one resource against another. Our method has been tested by simulations, as well as real executions, and was found to perform well.