Adaptive operator selection with dynamic multi-armed bandits

  • Authors:
  • Luis DaCosta;Alvaro Fialho;Marc Schoenauer;Michèle Sebag

  • Affiliations:
  • INRIA Saclay, Orsay, France;INRIA-Microsoft Joint Lab, Orsay, France;INRIA Saclay, Orsay, France and INRIA-Microsoft Joint Lab, Orsay, France;INRIA Saclay, Orsay, France and INRIA-Microsoft Joint Lab, Orsay, France

  • Venue:
  • Proceedings of the 10th annual conference on Genetic and evolutionary computation
  • Year:
  • 2008

Quantified Score

Hi-index 0.02

Visualization

Abstract

An important step toward self-tuning Evolutionary Algorithms is to design efficient Adaptive Operator Selection procedures. Such a procedure is made of two main components: a credit assignment mechanism, that computes a reward for each operator at hand based on some characteristics of the past offspring; and an adaptation rule, that modifies the selection mechanism based on the rewards of the different operators. This paper is concerned with the latter, and proposes a new approach for it based on the well-known Multi-Armed Bandit paradigm. However, because the basic Multi-Armed Bandit methods have been developed for static frameworks, a specific Dynamic Multi-Armed Bandit algorithm is proposed, that hybridizes an optimal Multi-Armed Bandit algorithm with the statistical Page-Hinkley test, which enforces the efficient detection of changes in time series. This original Operator Selection procedure is then compared to the state-of-the-art rules known as Probability Matching and Adaptive Pursuit on several artificial scenarios, after a careful sensitivity analysis of all methods. The Dynamic Multi-Armed Bandit method is found to outperform the other methods on a scenario from the literature, while on another scenario, the basic Multi-Armed Bandit performs best.