Multivariate decision tree function approximation for reinforcement learning

  • Authors:
  • Hossein Bashashati Saghezchi;Masoud Asadpour

  • Affiliations:
  • Department of Electrical and Computer Engineering, University of Tehran, Iran;Department of Electrical and Computer Engineering, University of Tehran, Iran

  • Venue:
  • ICONIP'10 Proceedings of the 17th international conference on Neural information processing: theory and algorithms - Volume Part I
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

In reinforcement learning, when dimensionality of the state space increases, making use of state abstraction seems inevitable. Among the methods proposed to solve this problem, decision tree based methods could be useful as they provide automatic state abstraction. But existing methods use univariate, therefore axis-aligned, splits in decision nodes, imposing hyper-rectangular partitioning of the state space. In some applications, multivariate splits can generate smaller and more accurate trees. In this paper, we use oblique decision trees as an instance of multivariate trees to implement state abstraction for reinforcement learning agents. Simulation results on mountain car and puddle world tasks show significant improvement in the average received rewards, average number of steps to finish the task, and size of the trees both in learning and test phases.