Dimension reduction and its application to model-based exploration in continuous spaces

Authors:
Ali Nouri;Michael L. Littman
Affiliations:
Department of Computer Science, Rutgers University, Piscataway, USA 08854;Department of Computer Science, Rutgers University, Piscataway, USA 08854
Venue:
Machine Learning
Year:
2010

Citing 0
Cited 4

Instance-based reinforcement learning technique with a meta-learning mechanism for robust multi-robot systems

TAROS'11 Proceedings of the 12th Annual conference on Towards autonomous robotic systems
Sample aware embedded feature selection for reinforcement learning

Proceedings of the 14th annual conference on Genetic and evolutionary computation
Exploration in relational domains for model-based reinforcement learning

The Journal of Machine Learning Research
Phased searching with NEAT in a Time-Scaled Framework: Experiments on a computer-aided detection system for lung nodules

Artificial Intelligence in Medicine

Quantified Score

Hi-index	0.00

Visualization

Abstract

The sample complexity of a reinforcement-learning algorithm is highly coupled to how proficiently it explores, which in turn depends critically on the effective size of its state space. This paper proposes a new exploration mechanism for model-based algorithms in continuous state spaces that automatically discovers the relevant dimensions of the environment. We show that this information can be used to dramatically decrease the sample complexity of the algorithm over conventional exploration techniques. This improvement is achieved by maintaining a low-dimensional representation of the transition function. Empirical evaluations in several environments, including simulation benchmarks and a real robotics domain, suggest that the new method outperforms state-of-the-art algorithms and that the behavior is robust and stable.