Adaptive Reputation-Based Scheduling on Unreliable Distributed Infrastructures

  • Authors:
  • Jason Sonnek;Abhishek Chandra;Jon Weissman

  • Affiliations:
  • -;-;-

  • Venue:
  • IEEE Transactions on Parallel and Distributed Systems
  • Year:
  • 2007

Quantified Score

Hi-index 0.02

Visualization

Abstract

This paper addresses the inherent unreliability and instability of worker nodes in large-scale donationbased distributed infrastructures such as P2P and Grid systems. We present adaptive scheduling techniques that can mitigate this uncertainty and significantly outperform current approaches. In this work, we consider nodes that execute tasks via donated computational resources and may behave erratically or maliciously. We present a model in which reliability is not a binary property but a statistical one based on a node’s prior performance and behavior. We use this model to construct several reputationbased scheduling algorithms that employ estimated reliability ratings of worker nodes for efficient task allocation. Our scheduling algorithms are designed to adapt to changing system conditions as well as non-stationary node reliability. Through simulation we demonstrate that our algorithms can significantly improve throughput, while maintaining a very high success rate of task completion. Our results suggest that reputation-based scheduling can handle wide variety of worker populations, including non-stationary behavior, with overhead that scales well with system size. We also show that our adaptation mechanism allows the application designer fine-grain control over desired performance metrics.