Learning based opportunistic admission control algorithm for MapReduce as a service

  • Authors:

  • Jaideep Dhok;Nitesh Maheshwari;Vasudeva Varma

  • Affiliations:

  • Internation Institute of Information Technology, Hyderabad, Hyderabad, India;Internation Institute of Information Technology, Hyderabad, Hyderabad, India;Internation Institute of Information Technology, Hyderabad, Hyderabad, India

  • Venue:
  • Proceedings of the 3rd India software engineering conference
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Admission Control has been proven essential to avoid overloading of resources and for meeting user service demands in utility driven grid computing. Recent emergence of Cloud based services and the popularity of MapReduce paradigm in Cloud Computing environments make the problem of admission control intriguing. We propose a model that allows one to offer MapReduce jobs in the form of on-demand services. We present a learning based opportunistic algorithm that admits MapReduce jobs only if they are unlikely to cross the overload threshold set by the service provider. The algorithm meets deadlines negotiated by users in more than 80% of cases. We employ an automatically supervised Naive Bayes Classifier to label incoming jobs as admissible and non-admissible. From the list of jobs classified as admissible, we then pick a job that is expected to maximize service provider utility. An external supervision rule automatically evaluates decisions made by the algorithm in retrospect, and trains the classifier. We evaluate our algorithm by simulating a MapReduce cluster hosted in the Cloud that offers a set of MapReduce jobs as services to its users. Our results show that admission control is useful in minimizing losses due to overloading of resources, and by choosing jobs that maximize revenue of the service provider.