Dynamic load balancing efficiently in a large-scale cluster
International Journal of High Performance Computing and Networking
Proceedings of the 24th ACM International Conference on Supercomputing
A mean field model of work stealing in large-scale systems
Proceedings of the ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Load balancing via random local search in closed and open systems
Proceedings of the ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Lifeline-based global load balancing
Proceedings of the 16th ACM symposium on Principles and practice of parallel programming
Scheduling efficiently for irregular load distributions in a large-scale cluster
ISPA'05 Proceedings of the Third international conference on Parallel and Distributed Processing and Applications
An efficient dynamic load-balancing algorithm in a large-scale cluster
ICA3PP'05 Proceedings of the 6th international conference on Algorithms and Architectures for Parallel Processing
Dynamic diffusion load balancing
ICALP'05 Proceedings of the 32nd international conference on Automata, Languages and Programming
Compiler and runtime support for enabling reduction computations on heterogeneous systems
Concurrency and Computation: Practice & Experience
Load balancing via random local search in closed and open systems
Queueing Systems: Theory and Applications
Scheduling parallel programs by work stealing with private deques
Proceedings of the 18th ACM SIGPLAN symposium on Principles and practice of parallel programming
Friendly barriers: efficient work-stealing with return barriers
Proceedings of the 10th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments
Hi-index | 0.00 |
In this paper we analyze a very simple dynamic work-stealing algorithm. In the work-generation model, there are n (work) generators. A generator-allocation function is simply a function from the n generators to the n processors. We consider a fixed, but arbitrary, distribution $\cal D$ over generator-allocation functions. During each time step of our process, a generator-allocation function h is chosen from $\cal D$, and the generators are allocated to the processors according to h. Each generator may then generate a unit-time task, which it inserts into the queue of its host processor. It generates such a task independently with probability $\lambda$. After the new tasks are generated, each processor removes one task from its queue and services it. For many choices of $\cal D$, the work-generation model allows the load to become arbitrarily imbalanced, even when $\lambdah which allocates all of the generators to just one processor. For this choice of $\cal D$, the chosen processor receives around $\lambda n$ units of work at each step and services one. The natural work-stealing algorithm that we analyze is widely used in practical applications and works as follows. During each time step, each empty processor (with no work to do) sends a request to a randomly selected other processor. Any nonempty processor having received at least one such request in turn decides (again randomly) in favor of one of the requests. The number of tasks which are transferred from the nonempty processor to the empty one is determined by the so-called work-stealing function f. In particular, if a processor that accepts a request has $\ell$ tasks stored in its queue, then $f(\ell)$ tasks are transferred to the currently empty one. A popular work-stealing function is $f(\ell)=\lfloor \ell/2\rfloor$, which transfers (roughly) half of the tasks. We analyze the long-term behavior of the system as a function of $\lambda$ and f. We show that the system is stable for any constant generation rate $\lambdaf. Most intuitively sensible functions are included in this class (for example, every monotonically nondecreasing function f which satisfies $0 \leq f(\ell)\leq \ell/2$ and $f(\ell)=\omega(1)$ as a function of $\ell$ is included). Furthermore, we give upper bounds on the average system load (as a function of f and n). Our proof techniques combine Lyapunov function arguments with domination arguments, which are needed to cope with dependency.