Spawn: A Distributed Computational Economy
IEEE Transactions on Software Engineering
G-commerce: Market Formulations Controlling Resource Allocation on the Computational Grid
IPDPS '01 Proceedings of the 15th International Parallel & Distributed Processing Symposium
The ANL/IBM SP Scheduling System
IPPS '95 Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing
Core Algorithms of the Maui Scheduler
JSSPP '01 Revised Papers from the 7th International Workshop on Job Scheduling Strategies for Parallel Processing
User-Centric Performance Analysis of Market-Based Cluster Batch Schedulers
CCGRID '02 Proceedings of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid
Lottery and Stride Scheduling: Flexible Proportional-share Resource Management
Lottery and Stride Scheduling: Flexible Proportional-share Resource Management
Market-based Proportional Resource Sharing for Clusters
Market-based Proportional Resource Sharing for Clusters
SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
An Economy-based Accounting Infrastructure for the DataGrid
GRID '03 Proceedings of the 4th International Workshop on Grid Computing
Applying economic scheduling methods to Grid environments
Grid resource management
A price-anticipating resource allocation mechanism for distributed shared clusters
Proceedings of the 6th ACM conference on Electronic commerce
Software—Practice & Experience
Tycoon: An implementation of a distributed, market-based resource allocation system
Multiagent and Grid Systems
MapReduce: simplified data processing on large clusters
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Bigtable: a distributed storage system for structured data
OSDI '06 Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation - Volume 7
Evaluating MapReduce for Multi-core and Multiprocessor Systems
HPCA '07 Proceedings of the 2007 IEEE 13th International Symposium on High Performance Computer Architecture
Admission Control in a Computational Market
CCGRID '08 Proceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid
Mars: a MapReduce framework on graphics processors
Proceedings of the 17th international conference on Parallel architectures and compilation techniques
Scheduling: Theory, Algorithms, and Systems
Scheduling: Theory, Algorithms, and Systems
MapReduce optimization using regulated dynamic prioritization
Proceedings of the eleventh international joint conference on Measurement and modeling of computer systems
CellMR: A framework for supporting mapreduce on asymmetric cell-based clusters
IPDPS '09 Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing
Hadoop: The Definitive Guide
New challenges of parallel job scheduling
JSSPP'07 Proceedings of the 13th international conference on Job scheduling strategies for parallel processing
Improving MapReduce performance in heterogeneous environments
OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation
Parallel job scheduling — a status report
JSSPP'04 Proceedings of the 10th international conference on Job Scheduling Strategies for Parallel Processing
ARIA: automatic resource inference and allocation for mapreduce environments
Proceedings of the 8th ACM international conference on Autonomic computing
An adaptive scheduling algorithm for dynamic heterogeneous Hadoop systems
Proceedings of the 2011 Conference of the Center for Advanced Studies on Collaborative Research
Resource provisioning framework for mapreduce jobs with performance goals
Middleware'11 Proceedings of the 12th ACM/IFIP/USENIX international conference on Middleware
An economic approach for application qos management in clouds
Euro-Par'11 Proceedings of the 2011 international conference on Parallel Processing - Volume 2
Delay tails in MapReduce scheduling
Proceedings of the 12th ACM SIGMETRICS/PERFORMANCE joint international conference on Measurement and Modeling of Computer Systems
Cloud-based image processing system with priority-based data distribution mechanism
Computer Communications
Resource provisioning framework for MapReduce jobs with performance goals
Proceedings of the 12th International Middleware Conference
A Hybrid Scheduling Algorithm for Data Intensive Workloads in a MapReduce Environment
UCC '12 Proceedings of the 2012 IEEE/ACM Fifth International Conference on Utility and Cloud Computing
Multiple objective scheduling of HPC workloads through dynamic prioritization
Proceedings of the High Performance Computing Symposium
HAT: history-based auto-tuning MapReduce in heterogeneous environments
The Journal of Supercomputing
Proceedings of the 4th annual Symposium on Cloud Computing
Utility-Driven share scheduling algorithm in hadoop
ISNN'13 Proceedings of the 10th international conference on Advances in Neural Networks - Volume Part II
On the use of a proportional-share market for application SLO support in clouds
Euro-Par'13 Proceedings of the 19th international conference on Parallel Processing
Game-based scheduling algorithm to achieve optimize profit in mapreduce environment
ICIC'13 Proceedings of the 9th international conference on Intelligent Computing Theories
A MapReduce task scheduling algorithm for deadline constraints
Cluster Computing
| Hi-index | 0.00 |
We present the Dynamic Priority (DP) parallel task scheduler for Hadoop. It allows users to control their allocated capacity by adjusting their spending over time. This simple mechanism allows the scheduler to make more efficient decisions about which jobs and users to prioritize and gives users the tool to optimize and customize their allocations to fit the importance and requirements of their jobs. Additionally, it gives users the incentive to scale back their jobs when demand is high, since the cost of running on a slot is then also more expensive. We envision our scheduler to be used by deadline or budget optimizing agents on behalf of users. We describe the design and implementation of the DP scheduler and experimental results. We show that our scheduler enforces service levels more accurately and also scales to more users with distinct service levels than existing schedulers.