Shifting Inductive Bias with Success-Story Algorithm, AdaptiveLevin Search, and Incremental Self-Improvement

  • Authors:
  • Jürgen Schmidhuber;Jieyu Zhao;Marco Wiering

  • Affiliations:
  • IDSIA, Corso Elvezia 36, CH-6900-Lugano, Switzerland E-mail: juergen@idsia.ch, jieyu@idsia.ch, marco@idsia.ch;IDSIA, Corso Elvezia 36, CH-6900-Lugano, Switzerland E-mail: juergen@idsia.ch, jieyu@idsia.ch, marco@idsia.ch;IDSIA, Corso Elvezia 36, CH-6900-Lugano, Switzerland E-mail: juergen@idsia.ch, jieyu@idsia.ch, marco@idsia.ch

  • Venue:
  • Machine Learning - Special issue on inductive transfer
  • Year:
  • 1997

Quantified Score

Hi-index 0.00

Visualization

Abstract

We study task sequences that allow for speeding up the learner‘saverage reward intake through appropriate shifts of inductive bias(changes of the learner‘s policy). To evaluate long-term effects ofbias shifts setting the stage for later bias shifts we use the“success-story algorithm” (SSA). SSA is occasionally called attimes that may depend on the policy itself. It uses backtracking toundo those bias shifts that have not been empirically observed totrigger long-term reward accelerations (measured up until the currentSSA call). Bias shifts that survive SSA represent a lifelong successhistory. Until the next SSA call, they are considered useful andbuild the basis for additional bias shifts. SSA allows for pluggingin a wide variety of learning algorithms. We plug in (1) a novel,adaptive extension of Levin search and (2) a method for embedding thelearner‘s policy modification strategy within the policy itself(incremental self-improvement). Our inductive transfer case studiesinvolve complex, partially observable environments where traditionalreinforcement learning fails.